How AP damage works is detailed briefly on page 48 of Savage Worlds Deluxe Edition, in the introduction of the Gear chapter:
The weapon or round ignores this many points of Armor. […] Excess AP is simply lost.
The AP2 means that in this instance the effective Toughness is 8. If it was AP4 it would ignore up to 4 points of armor, but in this case there are only 2 points of armor, so AP4 in this instance would also mean that the effective Toughness is 8.
The other way you thought it might work wouldn't model armor piercing, but something more like a "hollowpoint" round that does an especially high amount of damage if it gets through armor, but isn't any better at getting through armor than a normal bullet.