If any scientist declares that he/she the results of a test depending on human memory is absolute fact. Almost every engineer and scientist out there would laugh at him or her and would probably think very little of them. Yet, people on this site keep insisting that using short term memory or long term memory is still a valid scientific approach. I really have nothing else to add except “whatever”.
No tests absolutely prove facts, and certainly relying on memory has limitations. But I don't see how you can claim that any tests using memory do not use a valid scientific approach. (This is not to mention experiments about memory itself, but of course that's not what you're talking about.) There are limitations with many different approaches, yet with the right design and controls, you can still get some valuable data. When you work with memory, you probably want to use short-term memory and control the environment to ensure there's no potato-chip eating distractions nearby.
What other kind of comparative listening experiment do you propose? Having a somebody listen to A in one ear and B in the other ear and express a preference for one--then swapping, and controlling for order effects, and so on? Make somebody listen to A and B several times each without saying which is which, and score each sample based on different parameters?
Edited by mikeaj - 3/14/12 at 9:44am