As a lot of people know, I tend to frequent a lot of various music-related forums throughout the day.  Every now and then (approximately every 18 minutes) I end up running into a thread discussing which DAW sounds better.  Or as some people like to say, which DAW has the best sounding “Summing Engine”.  Now, I’ve looked into this in the past and posted my results about it, but there’s new people getting into music production every day, so it’s time to revisit the topic I think.

Originally I had planned to do a huge, comprehensive test among all of the latest DAWs I could get my hands on.  But, I’m super busy with the mastering business lately, and realistically I don’t have the time to learn the intricacies of each DAW to make sure that I’m doing the test as fairly as possible.  And besides, the test is easy enough for anyone to run on their own.  So I’m only going to focus on the two DAWs I know and use the most (which also happen have to most heated debates on inherent sound quality), Ableton Live and Apple Logic Pro.

So for this comparison I’m going to be using Apple Logic Pro v9.1.5 in 32bit mode, and Ableton Live v8.2.6.  The basic premise of the test is pretty simple, I’ll use the same set of audio stems in each application, and then compare the rendered results.  For those of you who’d like to use these same stems in your own DAW of choice, you can download all the 24bit stems here:

http://tarekith.com/assets/SoundQuality/LogicLiveStems.zip

Because I don’t have time at the moment to write new material for a test like this, I just used some stems from one of my recent songs.  I kept the stems at 17 bars to keep the file sizes smaller, with a short click at the very beginning to assist in lining up the files for comparison later on.  I did run the test on song-length stems as well, and got the same results as with these shorter files, for those that are curious.

Step one was to import all the stems into Logic, and lower each track fader to be exactly -3dB. (Command+Click on each image below to view it larger in a new Tab)

Next I made sure to change Logic’s pan law to “-3dB (Compensated)” in the project settings.  This way Logic is using the same pan law that Ableton Live uses (Live does not allow you to change the pan law).

After that I bounced all of the stems into a single stereo 24bit wav file.

Now to do the same in Ableton Live.  First step is to drag all of the stems into Live, MAKE SURE WARPING IS OFF, and then lower all of Live’s volume faders to -3dB.  It’s important in Live to actually type the exact value you want for the value faders.  Many people don’t realize this, but Live’s faders only show a resolution of one decimal place, but can actually be slightly different if you drag them with the mouse or use a MIDI controller.  For instance, if you drag them with the mouse they might really be set to -3.045dB, even though they show -3dB.  For day to day use this is no issue at all, but I want to make sure the volumes are identical to what I set in Logic.

Then I Exported the stems from Live into a single stereo 24bit wav file just like before.

Now the fun part.  The first thing I wanted to do is just listen to these two files and see if I could hear an obvious difference.  I dragged them both into Live (again making sure warping was off) and assigned a MIDI controller to mute one track while soloing the other.  This way I could instantly toggle between them with one button press.

Normally I’d get my wife to help me by toggling these while I wasn’t looking, so that the observations are done blind.  However she was watching Amazing Race on TV, so I had to just turn off the computer screen and do it manually a bunch of times without keeping count of how often I pressed the button. Not the most scientific, but regardless I could hear no difference between the two files anyway.  This was done at multiple volume levels with my monitors as well.

You can listen to the files yourself here:

http://tarekith.com/assets/SoundQuality/LiveTestFile.wav

http://tarekith.com/assets/SoundQuality/LogicTestFile.wav

Right click and choose “Save As” to download these to your computer if you want, and remember the little click you hear at the start of both files was intentional.

After that, I opened both files up Audiofile Engineering’s Wave Editor, and ran an audio analysis on them both.  You can see those results here:

As you can see, the results are identical.  Note that even though the “Selection Only” option is checked in the Analysis Window, the entire audio file was selected in both cases when I ran the analysis.

The final test was the infamous phase-cancellation test I’m sure many of you have seen mentioned before.  To perform this, I dragged both rendered files into Logic, used a Logic Gain plug-in on one of them to invert it’s phase, and then compared the combined output when both were played at the same time.  I used Logic metering, as well as Sonalksis’s Free-G plug in meter for greater resolution (and it’s free so other’s can use it for their own testing).  As you can see, when the phase of one of the files was inverted, the files COMPLETELY cancelled each other out.  I also repeated this test in Live using a Utility plug-in to invert the phase of one track, and got the same results.

This means they are bit for bit identical.

So, the results of this test show that Ableton Live and Apple Logic Pro produce exactly the same thing when you export a mixdown.

Everything else being equal.

That is the key point that people need to take away from this test, everything else being equal.  This test only shows that at their core, these two applications combine multiple tracks into a stereo wav file in exactly the same way, nothing else.  There are dozens of other aspects of each program that can affect the final audio quality of your mixdowns, and since there is no way to do a fair comparison of those, I’m not even going to bother trying.

For instance, both programs offer different time-stretching algorithms (Warping versus Flex-Time).  They both come with plug-ins (and presets) that designed and sound vastly different from each other.  They both handle things like automation data differently.  Just a few examples of places other than the supposed “summing engine” where what you do, and how you use each program, can impact the final sound of your productions.

I’m sure what I’ve done here won’t end the debate over which DAW sounds the best, but I do hope that in some small way it shifts the discussion to the aspects that truly make a measureable difference.  As I’ve shown here, if there is a difference in sound quality, it’s not in the way they combine multiple tracks into the final end result.

——————-

If anyone finds a flaw in my testing, or just wants to continue the debate, please feel free to discuss this in the comments section below.  Please don’t ask me to test other DAWs , I’ve provided the exact files I’ve used if you’re really that curious about it.  By all means feel free to post the results of any testing you do though, as I’m sure other people are curious as well.

——————-

UPDATE 12-06-2011

Well, it took awhile but the flood gates have opened about my Live versus Logic sound quality test that I just posted.  Some people have raised some good points about ways I could have modified the test to include other parameters, so I’ve gone back and done a few things differently as sort of a round two.   I also wanted to clarify a few questions that seem to keep popping up on different forums again and again.

First and foremost though, I wish people would have not just skimmed the article and actually read my conclusions.  I am NOT saying that Live and Logic ALWAYS sound the same.  The point of this test was to isolate one specific area for comparison, and show how at the core, the way these two programs combine multiple audio tracks into a stereo wav file is the same.  That’s it.

Like I stated in the original post, there’s a LOT of other areas where there will likely be difference in sound quality.  Instead of getting mad at me for not doing all the work for you, it would be great if people instead tested some of this on their own and said “hey look, here’s one area where I can reliable show a difference in signals”.

 

Anyway, here’s some other things I looked at over the last day, and some clarifications on the test itself:

- A few people mentioned that they hear the differences most notably with recorded instruments.  The guitar in this test was recorded live, it’s a Parker Dragonfly using a combination of the piezo and mag pickups, through a Pod HD500 and then into an RME Fireface400.

- Some people have questioned whether the soundcard I use (see above) could have any impact on the signals in the test.  Short answer is no, the soundcard doesn’t factor in at all until after the DAW has done it’s thing and that signal is trying to get out of your computer.  Or if your song sucks, maybe the signal is ashamed and is trying to stay in your computer, I don’t know.

- Other people wanted to know if perhaps the test would turn out differently if I use more than 9 audio tracks.  So I duplicated the tracks in each DAW many times, and added some other random loops from my collection as well (to rule out it just being these audio files that this was happening with).  In total, I used 80 stereo tracks, exported, and still got total cancellation when comparing the two.

- This last test was one of the most interesting.  Someone had suggested using the same 3rd party plug ins in both DAWs, and seeing if that had any impact on whether or not they cancel (or sound).  So I used a combination of Fabfilter Pro-L, DMG eQuality, Voxengo MSED and Polysquasher, and PSP Xenon, placed randomly across the different tracks (yet the same tracks in both DAWs).  Some were placed one after the other in series on the same track, others were solo by themselves on a single track.

Interestingly, when I compared these two results, they did NOT cancel, barely at all in fact.  As I dug into this some more, it seems that the Voxengo and PSP plugs were the primary cause, as once I removed these the signals almost cancelled.  Summed they were inaudible, but I was still seeing some very small signal around -96dB on the Free-G meter.  This gave me an AHA! moment though, when I realized this looked a lot like a dither signal.  Sure enough, I had forgotten that I had dithering enabled by default in Pro-L.  Once this was turned off, the two signals cancelled completely.

So, I’m really not sure what kind of conclusions one can draw at this point about this, other than some of the differences in this part of the test seemed to be down to the plug ins themselves.  Perhaps they report their latency differently, or have some sort of random processing happening as part of the way they work internally, I really don’t know.

 

Anyway, the long and short of all this is that all this testing was never meant to be a definitive statement about which DAW ultimately sounds better, or which people should prefer.  I’ve gotten a lot of surprisingly hateful emails from people calling me an “Ableton Fan Boy” (is that an insult?) among many other not so nice things.  At the end of the day yes, I do like Ableton Live for many things, but it’s only one of many tools at my disposal.

For instance, when clients send me mixdowns to work on, I don’t use Live unless they ask me to, I always reach for Logic first.  It’s faster for this type of work, has better automation functions, and quite frankly I like the way it’s plug ins sound better and how quickly I can add an EQ to a channel if I need to. (far from perfect though, Logic has been buggy as shit since OSX Lion came out).

As always I’m sure people will draw their own conclusions no matter what I say, but I do ask that instead of sending me nasty emails or message, maybe try instead to offer something more constructive to the conversation than “You must have tomatos in your ears!”.

——————-

UPDATE 12-13-2011

Well, it turns out a flaw has been found that invalidates my test.  In attempt to use measurement tools that others could also obtain for free, it’s been pointed out to me that the low-level resolution of the Free-G metering plug in was not sufficient to capture all of the audio signal.  An Ableton Forum user has brought to my attention that the last 3 bits of the null-test signal (the signal below -126dBFS) are in fact not bit for bit identical.

How much affect this has on the audible difference between the two signals is debatable (and I’m sure people will), but I have to withdraw my conclusion that Live and Logic produce bit for bit identical audio files given the conditions above.  My apologies for not being more thorough in my testing, you can now go back to arguing about which DAW sounds better