I tend to trust sources who use their intelligence to point out flaws in their own positions. But is this more than an aesthetic preference on my part?
In my view, this is much more than an aesthetic preference. Suppose that you are an expert arguing for a particular hypothesis or policy. Your approach to competing or contrary positions can be:
(a) ignore them
(b) put them down rhetorically, but not substantively
(c) hold them to much higher standards of reason and evidence than you do for your own position
(d) hold both your position and competing positions to high standards of reason and evidence
It is relatively rare to observe (d). My sense is that even well-trained economists often choose (c), which is a subtle form of confirmation bias.*
If I can spot a problem in your argument that you have not addressed, then that by itself weakens your argument. In addition, it is a signal that you are not trustworthy, because you apparently lack either the competence or the honesty to address my issue. It is this signaling aspect that leads Tyler to worry that he could be fooled by an expert who is able to show an understanding of weaknesses with his (the expert's) position. My answer to that would be that you would have to look at the overall scientific discourse to determine whether the expert is defending himself by addressing straw men or by addressing important objections.
*For example, Gregory Clark, whose Farewell to Alms is an exemplary book in many respects, is sometimes closer to (c) than to (d) when he discusses the relative merits of institutions and labor force quality as explanations of underdevelopment. In criticizing the institutional explanation, he can cite counterexamples, in which countries with "bad" institutions nonetheless developed. However, he does not face up to the implications of the fact that immigrants to developed countries achieve much higher productivity rates. Clark tends to want to present institutional explanations as a failure and labor force explanations as a success. I think that an objective reading of the evidence would be more ambivalent.