Hard to explain or reproduce, but when form error messages are generated in checkout panes, they are stored in form_state. So if you submit the form, you get the messages over the desired pane. If, however, you edit some field that triggers an ajax request on the same pane, the messages should go away but they don't.
In my setup, pane has options for "new address", "existing address", "same address". Selecting one triggers ajax that renders a form with certain fields. If you select "new address", an addressfield form is rendered, with some required fields. I click submit, wich fails because required fields are empty. The messages show up in checkout pane properly. Then if I click "same address", the addressfield fields are gone after the ajax, but the error messages for the required fields get rendered again.
I found the bug is caused by Commerce and this simple patch prevents it.