It depends entirely on the application and scope of the directive, I go either two ways:
If it's between different buttons (if-elseif-else etc), the directive is very simple and explanatory by just looking at it, I use ng-if or ng-switch.
If it's a complex directive that lives on its own, meaning it has its own scope and functions, there's no reason for it to be cluttering the DOM. For instance, if you search through the DOM a lot, you're going to be adding more data to search through, then instead if you destroy and rebuild it only when its used. It also clutters the DOM if you have tons of custom directives that you show/hide.
If your application has several heavy directives that contain complexity and you only use them after you click on a button, it makes more sense to create/destroy, than to hide/show the directive (helps if the directive contains data that it is getting from a service).
Also it just feels more right to invoke sub applications when you need them, rather than hiding/showing them, though this is a personal preference and I might be wrong.
The majority of custom directives you use from 3rd parties, they create/destroy their directive rather than show/hide.
Not entirely sure how the $watchers work with custom directive when hidden, but you removed that possibility when you create/destroy directives which helps with performance. For instance, if you render your HTML often, then $watchers pile up if you have a lot of ng-if/ng-switch, on the contrary, if you create/destroy, then when the html DOM is destroyed, Angularjs won't check the $watcher because it doesn't exist so you gain performance.