The "plug" is (oddly) both the metal tines that go into the wall, and informally, the thing you plug it into. For example, if I'm holding my phone charger in my hand and looking around the room, I may say "Where's a plug I can use?"
Rarely do I say "socket" for that, but I will often refer to that as the wall "outlet." (I'm more likely ask if there's a convenient outlet than I am a convenient plug.)
You plug something in, and you unplug it. You don't (even though you would think it logical) "plug it out."