Originally Posted by tc4all
Many will love this type of test but I find it does less good then harm. First, my room and yours are not the same. Second, you have not or maybe have not tested dozens of brands I would or should be interested in. Third, your or even the group's hearing is probably not the same as mine. You have no idea where I may or may not have a drop out in my hearing spectrum. If I bought the same speakers but used different amps and cables my sound would be different. and on and on ...
All you will really establish with this is that for this group of speakers, with these cables and electronics, in this type of room with this type of construction and this furniture and this number of people of this size in the room, etc. XYZ speakers sound best. Seriously, this is a royal time killer that proves nothing.
There are, for argument sake, let us say 100 different speakers with a model over $30,000. (This would be the same for budget speakers but is more understandable like this.) That is a lot. No matter how we test, we can not decide which is best. Each was probably designed on a no holds barred idea to get what is, according to that manufacturer and the ears and instruments used to test it, the best possible sound ... period. If there was a "best one" everyone in that price market would buy it and the others would go under. That doesn't happen. 50 people will probably pick at least 20 different models they think are best. WE all hear differently and like different sounds. Some find bright wonderful and believe in it as more detail. Others like bass. Others want the sound to thrill them while others think it best when it lulls them into a sense of total relaxation. (By the way, Pink Floyd is so over processed that to use it to judge is a mistake, no matter how much you like the music.)
OK, so what good is a test like this. It is fun and fun reading, but for goodness sake, never, ever, read comparison test results like these and make even a cursory decision about what to buy, based on this. Go listen, borrow, take time and judge. Read about what is good, but not shoot-outs. Remember that listening to music for 5 or 6 hours is very different then listening for 1/2 hour. I say this specifically relating to the mention of Magies, but is equally relevent to others.
Buying speakers is a very personal process and to short change yourself by letting someone else do the, even cursory, cuts of the group for you is not a good idea. IN another but equally bad way, that is no better then reading advertisement hype and believing it.
One note about scientific tests. A man fills a glass with 1/2 Vodka and 1/2 water. He drinks it and gets drunk. Later he fills a glass with 1/2 Gin and 1/2 water, drinks it and gets drunk. The next day he fills a glass with 1/2 bourbon and 1/2 water, drinks it and gets drunk. His scientific conclusion is that the water is making him drunk. There is much more then meets the eye. Don't look for quick answers.
We fully expect knee jerk responses like this. That is why we're still a year from finishing our tests. One conclusion we have made is that cables DON'T make a difference if you select a good cable. We have had leanings towards one cable brand for some weird reason but other than one brand, the rest seem identical (ok ARE identical) as long as we're using 12 AWG with solid insulation we're ok.
As for your hearing being different than someone in our test group: Sure! You may have bad hearing. We have tested the hearing of our group. They all tested above average or better for their age & sex.
As for Pink Floyd being a bad selection: we went with something we knew was extremely well recorded. You may disagree, but that is why we have other tracks.
As for Amps, CD Players, etc. having better synergy we already considered that. So we started testing with different combinations after we eliminated about 40% of the speakers. It could be we eliminated a speaker that may have sounded better, but we're pretty sure we didn't do that. If you want I'll add the Bose 301 back into the test groups for instance.
So far the Magnepan MMG has been tested with 44 different amps, processors, and CD players. The results range in a 4% swing in one direction and a 7% swing in the other (being more preferred). Planar speakers seem to do better with Tubes for instance.
As for Rooms, our rooms are all identical. We do however have different room sizes. So yes, speakers are tested in different sizes (small, medium, large). Some do better than others.
As for us not considering some brand you'd consider: All major brands were present for the testing. The vast majority of whose who in hi-fi made it into our testing. Some esoteric brands we took some short cuts with. As mentioned in other replies we went with an older SoundLab unit for instance the A-1 (still tests well), and some other units were donated to us.
As for these tests not proving anything, that is exactly what we expect to be screamed from rooftops of just about every major audio manufacturer. They don't want people to know that their $30,000 amp was bested by an $800 amp in the majority of tests. They don't want people to know that their $7500 pair of speakers were bested by a $550 pair of speakers. The cable companies definitely don't want us to release our data. We've proven that there is a difference between cables, but at the same time we've proven there is no reason to ever spend more than 3 figures on cables. All expensive cables were indistinguishable from good less expensive cable.
By running tests on speakers using multiple rooms, in a controlled blind environment with multiple amps, processors, and cd players we can show human preference towards certain speakers. You may not like that we've done it, but the facts will be that scientifically we've covered just about every base.
Obviously these tests prove other things. Some people like the sound of cones over planars. Obviously anyone who reads our tests could fall in the same category. So obviously people will still need to go out and hear for themselves.
Still I'm positive that if anyone goes and demos any of our top 10 rated items in the pricing category they are looking at they should find a good unit without being fooled by salesmen.
Without a doubt, price, salesmen and other psychological factors seem to play a bigger factor than anything else.