Hydrogenaudio vs. Stereophile
Ross M. Miller
Miller Risk Advisors
November 9, 2009
Over the years I have frequently written about music and audio
equipment. This month I will move out from specifics to the big picture as
represented by two seemingly opposing views--those found in Stereophile,
the last mass-market audio magazine that appears in print form, and Hydrogenaudio.org,
a website mainly about the compression of digital audio into MP3, FLAC,
ACC, Ogg Vorbis, and various esoteric formats. Viewed simply, Stereophile
is about the subjective side of audio and Hydrogenaudio is about the
objective side. Things are far from that simple.
If Sterophile only had better photographs, it would qualify as a
bizarre form of fetish pornography. Instead of picturing women
unattainable to all but the select few, it specializes in picturing audio
equipment unattainable to all but the select (and possibly deranged) few.
Stereophile does contain useful information for the less privileged and
more sane. It reviews regularly reviews "budget" audio
components that range in price from $25 or less on eBay to a few thousand
dollars from the handful of retail audio salons that have not gone broke
I am a long-time Stereophile subscriber and, unlike many writers in
their letters to the editor section, I have yet to find a reason to cancel
my subscription in a verbal hissy fit. While some of Stereophile's writers
do come off as absolute lunatics, Michael Fremer springs to mind, even
analog-obsessed Sam Tellig seems like a reasonable, if obsessed, guy.
Glossy audio magazines used to be crowded field. Sterophile, which came
late to the party and at first appeared only sporadically, lived for
decades in the shadows of Audio, Stereo Review, and High Fidelity. But
those three publications are now gone while Sterophile still limps along.
Stereophile was founded by the recently departed J. Gordon Holt, an
over-the-top guy if ever there was one. He started Stereophile with the
idea of its being the first audio magazine to incorporate a subjective
view of audio equipment, treating it more like wine than sterile
sound-reproduction machinery. While the present-day, post-Holt Stereophile
is biased toward tubes and vinyl, Holt actively supported both solid-state
components and digital audio.
Holt's core audio belief was that the goal of audio equipment was to
reproduce live musical performances with the greatest accuracy possible.
Of course, most popular musical is not created live, but rather is
manufactured in the studio. Holt viewed pop recordings (and the
baby-boomers who listened to it) as essentially evil. Holt died believing
that baby boomers had led to the ruin of serious audio.
Stereophile's audio equipment reviews follow a set pattern. The main
narrative of each review tells cute stories about the equipment, the
set-up process, and then wallows in its sound. An extended sidebar by
editor John Atkinson goes into the numbers, measurements of the accuracy
of the equipment's sound reproduction. Editor Atkinson usually notes the
discrepancies between his objective measurements and the subjective
opinions of the reviewer.
A good example of a Stereophile review is the aforementioned Michael
Fremer's take on the dCS Scarlatti SACD/CD playback system. A mere
$80,000 gets you what Stereophile claims is the best CD player that money
can buy. Here is John Atkinson's picture of what a sine wave looks like
when played through this $80,000 player:
Of course, neither the blue nor red line looks anything like a sine wave;
a sine wave is nice and curvy. This thing looks like it was drawn during
an earthquake. Actually, this is quite good for a CD player. Moreover,
ever the cheapest piece of analog equipment outputs nice, pretty sine
waves when fed one, like these two:
These charts provide incontrovertible objective evidence that CD
players by their very digital nature butcher music, which is largely
combinations of sine waves a la Fourier. Compressing the WAV files on CDs
into the ubiquitous MP3s file only butchers them more. (Technically, the
WAV files are merely containers for raw pulse code modulation (PCM)
MP3s are where Hydrogenaudio comes is. Hydrogenaudio is not any kind of
magazine; it is the host of Internet forums, wikis, etc. Many of the
threads on its forums and wikis concern the best way to use software such
as EAC (Exact Audio Copy), LAME, and dBpoweramp to rip compressed files
from the WAV files on CDs. They are few waveforms presented on
Hydrogenaudio because it is clear that they more you compress music, the
worse it measures.
It does not follow, however, that measuring bad means that it sounds
bad. How music "sounds" is inherently subjective. Hydrogenaudio,
however, makes the subjective "scientific" by subjecting it to
double-blind testing, the gold standard of science.
The most popular form of double-blind testing on Hydrogenaudio is the
ABX test, and the site provides links to software that allows one to run
one's own ABX tests. The idea behind ABX testing is straightforward.
Suppose that A is the original CD track and B is an MP3 file that was
ripped from it. In an ABX test of the two, the listener would listen to
the CD track (A), then the MP3 file (B), and then an unknown track (X).
The compute known whether X is A or B and the listener can listened to A,
B, and X as much as he desires until he is ready to choose whether X is A
or B. Because the odds of choosing correctly for a single sample is 50%,
the process is repeated 10 or more times to establish statistical
Over the summer I ripped my CD collection to WAV and MP3 files using
dBpoweramp and LAME. Before I embarked upon this lengthy process, I
conducted by own ABX test. Using Roxy Music's "Avalon" as my
test sample, I compared a WAV file taking straight from the CD with two of
LAME's preset compression formats known as V0 and V4. The V0 format
compresses file to roughly 250kbps and the V4 format squeezes them
tighter, down to around 160 kbps. Using ABX testing I determined that I
could, with some effort, reliably tell the WAV file from the MP3 file with
V0 compression. I could not, however, tell the V0 and V4 MP3s apart.
I ripped my CDs to all three formats, but based on these results, I try
to listen to the WAV files whenever possible. I don't feel bad when I
listen to MP3s, however, because I found the difference between them and
the original WAV files to be rather subtle. This is not, however, an
opinion that I could post on Hydrogenaudio. That is because the discourse
on Hydrogenaudio is governed by a set of "Terms of Service." The
most important TOS is the eighth one:
8. All members that put forth a statement concerning subjective sound
quality, must -- to the best of their ability -- provide objective support
for their claims. Acceptable means of support are double blind listening
tests (ABX or ABC/HR) demonstrating that the member can discern a
difference perceptually, together with a test sample to allow others to
reproduce their findings. Graphs, non-blind listening tests, waveform
difference comparisons, and so on, are not acceptable means of providing
Violators of TOS 8 are first warned and then booted. As you might
imagine, the folks at Stereophile are reviled by Hydrogenaudio. While I
find that Hydrogenaudio is a great source of info about file compression
technology, I consider the ABX tests that they report to be worthless.
Stereophile editor John Atkinson even suggested on Hydrogenaudio itself
that all members post audiologist test results along with their ABX
results and he was broadly ridiculed by the membership.
I don't care what other people can hear; I care what I can hear and
clearly the Stereophile crowd cares what they can hear. The Hydrogenaudio
folk, however, are convinced that whatever the Stereophile's reviewers and
readers hear is imaginary. While it is easy to perform ABX tests of
digital music compressed in various ways, it ranges from difficult to
impossible to do any kind of double-blind testing on most audio equipment.
The subjective approach taken by Stereophile does not even attempt to make
double-blind comparisons; indeed, comparisons of components are frequently
done based on the reviewer's memory of sound. While the fascistic tone of
Hydrogenaudio more fanatical posters (and its Terms of Service) is
offputting, these posters are almost certainly correct that placebo effect
is rampant at Stereophile, where the more money that is spent, the more
powerful the placebo is.
While I may have once had certified "golden ears," that's
what the audiologist at GE told me, that does not mean that I'm about to
pay $80,000 for a CD player, even if it plays SACDs to boot. As a child of
the 1960s, everything now, from modest iPod on up, sounds a lot better
than it did back then. Vinyl does not sound so wonderful when it's played
on a $75 Garrard turntable and music on the typical AM radio of the time
was beyond dreadful. I just turned on my Roku SoundBridge R1000 to Radio
Paradise at 192 kbps and through the cobbled-together sound system in my
office and it sure blows anything that I could afford during the sixties
away, not to mention that there was no Radiohead back then. Of course,
such opinions are unwelcome on Hydrogenaudio, but I do not have to do an
impossible double-blind test to tell you that Internet radio coming
through my office sound system sounds different from (and better than) the
Bakelite AM radios of my youth.
That said, I have no great love for digital audio, I just don't have
the time to mess around with vinyl anymore and I am sure that I cannot
listen to it in my car. But I do listen to vintage speakers, right now
Sarah McLachlan is singing to me through my Advent /1 center-channel
speaker and I have a pair of original "big" Advent speakers in
my family room. Further discussion of the joy of vintage speakers will
have to wait until next year.
Copyright 2009 by Miller Risk Advisors. Permission granted to
forward by electronic means and to excerpt or broadcast 250 words or less
provided a citation is made to www.millerrisk.com.