Checking for corrupt FLAC files

Recently I had some strange conditions during playback of certain FLAC files in my digital tango collection. First I thought there must be a bug in my player but checking the player’s logs showed that the problem was within the FLAC stream. Somehow, some of the files must have gotten corrupted. This could have different causes, like power failures and unclean shutdowns, maybe an error on the media or an uncompleted copy task. Once a byte get’s changed in the FLAC file in the worst case it might not be playable anymore. In my case, I had two different conditions: First the file would start playing and then after a certain point the music output stopped. In another situation the corrupted FLAC file would trigger an exception in the decoder and it crashed the whole player application.

As a matter of fact this is a feature of the FLAC audio format: “Suitable for archiving: FLAC is an open format, and there is no generation loss if you need to convert your data to another format in the future. In addition to the frame CRCs and MD5 signature, FLAC has a verify option that decodes the encoded stream in parallel with the encoding process and compares the result to the original, aborting with an error if there is a mismatch.”

So the good news are that the FLAC format has a MD5 checksum mechanism integrated which provides for an easy integrity check. There is also a command line programm to check a FLAC file:

flac -wst flacfile.flac

Here are the command options explained:
-w, –warnings-as-errors Treat all warnings as errors (which cause flac to terminate with a non-zero exit code).
-s, –silent Silent: do not show encoding/decoding statistics.
-t, –test Test (same as -d except no decoded file is written). The exit codes are the same as in decode mode.

Now this is useful to check an individual FLAC file but when you have to scan several thousand files it might be more useful to put it into a shell script and run it against the whole music folder. I found this script which I saved as flactest.sh in my home folder:

#!/bin/bash
cd ~/Musique
if [[ -f flac-errors.txt ]]; then
rm flac-errors.txt;
fi
touch flac-errors.txt
shopt -s globstar
for file in ./**/*.flac; do
flac -wst "$file" 2>/dev/null || printf '%3d %s\n' "$?" "$file" >> flac-errors.txt;
done

The script changes into the Musique folder in my home directory and then creates a text file called flac-errors.txt, if it’s running consecutively, it tests if this text files exists and when it exists it deletes and recreates it as an empty file prior to proceeding.

shopt -s globstar means that the Bash script will perform recursive globbing on ** – therefore matching all directories and files from the current position in the filesystem, rather that only the current level.

In the for loop it will loop through all FLAC files in the Musique folder and its subfolders performing the integrity test. If the FLAC file is OK, the output is send to /dev/null which means the output is deleted and if the test is not OK, meaning that there is corruption, it will be written into the flac-errors.txt file with a little formatting. So you will have the title and the path of the corrupt FLAC file written each on one line in the text file for a later analysis and eventual restoration of the dammaged files.

The script will take quite some time to loop through all files. What I do is opening the flac-errors.txt for continuous reading to see the progress in another console, like this:

tail -f /home/jens/Musique/flac-errors.txt

So every once in a while such a test might be a good idea to check if all the music files in the collection are still OK. This rules out bad surprises during playback!

By the way, the foobar2000 player has such an integrity test in the interface, Mixxx will write FLAC stream errors into its log file.

The test script can also be useful to be run on newly added folders in the music collection to check that all FLAC files are in OK condition and that the encoding worked out well, like a last test before playback or archival. The verify feature of the FLAC audio format is actually a big advantage compared to other formats which don’t have such a mechanism!

Fast money with Bandoneón arrabalero

Quote

pettorossi[París 1925] Al año de hallarse Pettorossi actuando en la orquesta de Bianco-Bachicha, realizó con el segundo de ellos un negocio no muy recomendable para los autores. El joven guitarrista era un inspirado autor de tangos y una madrugada cuando ya se habían retirado los habitués y el “Palermo” comenzaba a levantar las mesas y a cerrar sus puertas, esperó a que terminaran sus compañeros de orquesta de enfundar sus instrumentos hasta emprender la retirada, le propuso al bueno de Bachicha que le escuchara un tango recientemente compuesto por él. Bachicha extenuado por las horas de tarea complaciéndole el deseo se dispuso a escucharlo.

El autor comenzó a hacerle escuchar en su guitarra un tema de inspirada línea melódica que despertó sincera admiración en su oyente. Al terminar de ejecutarlo Pettorossi le preguntó:

-Qué te pareció…
-Muy inspirado. Te felicito… —respondió Bachicha dándole a entender con un enorme bostezo que era hora de irse a dormir.

Pettorossi le respondió a boca de jarro:

-1000 francos…
-¿Qué cosa… ? -preguntó sorprendido Bachicha.
-Te lo vendo… Esta noche o nunca… —dijo sonriendo el guitarrista.

Bachicha creyendo sacar de un imperioso apuro al bohemio sacó los mil francos y sin darle mayor importancia al préstamo se lo ofreció gentilmente, pero el guitarrista sabiendo que no era de los que tenían buena memoria para saldar deudas, desistió del ofrecimiento y repitió inflexible:

-Nada de deudas… Vengan los 1000…

Mediante esa suma la obra “Bandoneón arrabalero” —tal era su título— cambió de dueño. Operación cumplida.

Hallándose Pascual Contursi, el inspirado vate, de paso por París, Bachicha le hizo adaptar los versos y el tango fue mandado a imprimir.

Agreguemos que Pettorossi, para producir dinero inmediato, acostumbró muchas veces a realizar estas clases de compra-venta con muchas de sus obras que a la postre resultaban infalibles éxitos.

Enrique Cadícamo in “La Historia del Tango En París”, Corregidor Buenos Aires 1975

News from Don Esteban

On the 3rd of July 1936 the Juan D’Arienzo orchestra recorded the tango Don Esteban. Most reeditions, especially from the RCA Víctor company, for which the tango has been recorded in 1936, contain a different version as the one which can be found on the japanese CD-collection CTA (Club Tango Argentino), Juan D’Arienzo Vol. 2 CTA-302. This has been an ongoing discussion in tango newsgroups and found to be quite mysterious.

As Johan stressed it out in his article The Strange Case of Don and Mr. Esteban by comparing first the edition history of this tango through out the years and then analysing the tango itself, there might have been two takes during the recording session back on the 3rd of July 1936. This is his scheme with the differences between take 1 and 2 bar by bar:

structure

I have found now by coincidence an original 78 rpm pressing of the 1936 recording (Victor record number 37955) and as a matter of fact there have been clearly 2 takes of this tango. The Victor Encyclopedic Discography by the University of California contains a glossary which explains the term take:

Take numbers can usually be found on Victor 78-rpm discs on the record surface area just outside the record label, in the nine o’clock position. First takes are usually not indicated; there is no number in that area when a record derives from take 1.

The 78 rpm record in my collection has a take number 2 in the nine o’clock position as you can see on the following pictures:

78 rpm of the D'Arienzo Don Esteban recording take 2

don-esteban-take-2Now the question is which one is the first and which one the second take? A question which remains so fare without answers. In a recent discussion a friend said: “The CTA-version for me is jazzier because of the first piano solo which is played sycopated while the other is straight marcato. It sounds to me more like Biagi’s idea, and I can imagine that D’Arienzo preferred the straighter one and gave Biagi two more solos instead.”

On the B-side of the record number 37955 is the vals No llores madre and to disambigue it has a little “1”, as for take one in the nine o’clock position right beside the label.

folder.versofolderTo keep it short, the version by CTA is the second take, and my 78 rpm too. Now we can speculate what happened that day on the 3rd of July 1936. Maybe D’Arienzo was late and the orchestra recorded without him, possibly led by Biagi on the piano. The wax was already galvanised, when all of a sudden D’Arienzo rushed into the studio and ordered for a second take, lowering the part of Biagi as the two piano solos are now missing. By the way, in this scenario the jazzman would have been D’Arienzo. But most probably they were all present that day and just arrived at two different versions, maybe one more in the sense of Biagi and one more in D’Arienzo’s style. Then RCA Víctor said that they would send out both on the market and let the music lover decide which version they prefer. As D’Arienzo records sold like hot cakes, this seems to be the most plausible scenario 🙂 And it could be that take 1 has been intentionally released exclusively in Uruguay and that take 2 was intented for the domestic market. That would be an even more plausible explaination for the existence of the two Don Esteban versions (In this case the two piano solos could maybe be seen as a concession to what D’Arienzo thought would better meet the Uruguayan taste).

Now, on the other side, it would be interesting to find an Argentinian 78 rpm containing the first take to contradict this theory. (From what I was able to check with other collectors so fare only take 2 exists from RCA Víctor Argentina). And also, as you might know the first take has been handed down by RCA Víctor exclusively with a very strong echo (reverb effect). All CD versions, like from El Bandoneon and the Magenta version seem to be taken from that 1980 RCA Víctor LP. There is still no carefully restored version of this 1st take! I’m also wondering why CTA did not mention the “2” for second take on their CD, though it’s really evident when you have the 78 rpm record in your hands.

Check out the two versions, first the only reedition by RCA Víctor on the LP sampler Serie Tango De Ayer from 1980 (based on take 1) and then my restored version (based on take 2). Which one do you find jazzier?

El cencerro

Since quite recently I observe a general shift in tango djing. A lot of DJs are today putting a lot of effort into recovering the most authentic sound in the milongas. This translates into investing in high end sound devices, prefering lossless sound formats over lossy codecs, analysing sound files and playing correctly tuned tracks. Trying to find tango tracks in unfiltered versions and without certain artefacts which have been added to the original recordings in later copy cycles. Unveiling somehow unplugged, purer versions of the presented repertoire with a close to perfect sound. They rather like to have some surface noise than a cleaned to death recording!

To visualise this effort, one has to be aware that the time interval which is of most concern to modern dancers, 1925-1955, the Epoca de Oro with pre and afterparty, is parallel to the technical invention of the electrical recording systems in the second part of the 1920s and ends with the introduction of the modern vinyl record and the first stereo systems around the mid-1950s when the full frequency-range recording had been finally made possible.

Westrex electrical cutting

Westrex electrical cutting

If we look at the records’s frequency spectrum we can see that in the beginning certain frequencies especially in the mid and high ranges weren’t yet completely present and towards the 1950s these frequencies were successively added as the recording systems improved. Let’s take the year 1926, most of the record companies licenced the new Westrex system and were able to reproduce a recording bandwidth from 50 Hertz to about 6,000 Hertz, beyond which the high frequency sensitivity declined. This had been a sensational improvement compared to the former acoustic recording systems (200 to 2,400 Hertz and sometimes less) which were in place until around 1927 mostly in an effort to empty their stock of acoustic records. None of the older acoustic recordings (nearly everything before 1927) are played in any milongas nowadays. They don’t suit our needs because they are close to inaudible and one gets easily tired from them. But listen to a very early electrical Argentinian recording, A media luz interpreted by the Orquestra Francisco Canaro from around November 1926. Though from the early electrical recording days the sound quality is astonishingly good and it still fits perfectly into a typical today’s milonga program.

The missing frequencies let the tangos, we like so much, sound to our contemporary ear like if they were recorded on a different planet. And that’s maybe one of the reasons why sound engineers who transferred the early 78 rpm records to vinyl from the 1960s to the 1980s had added artefacts so that they sounded more familiar to their comtemporary hearing habits. By the late 1960s it must have been almost 15 years ago since the last 78 rpm records were sold, by the beginning of the 1980s, these old records had already disappeared from the market 30 years ago and playing 78 rpm records had been by then impossible for the large public. Therefore a new media had to be used for public reeditions to be able to still listen to the Epoca de Oro tango repertoire.

This had been in the beginning exclusively the new vinyl record and later the tape and compact cassette. A lot of people I meet are unaware of this circumstance and very much focused on the CD editions as the ultimate reference. Often they try to show me some hidden logic in how the tangos are distributed over a particular CD. Whereas the CDs often represent just a lot of 78 rpm records put together in a chronological order sometimes covering all 30 years of the larger Golden Age but in an uncomplete manner as a tango CD typically holds only around 20 to 30 tracks. This batch is often put together on an availability basis and rarely in an editorial manner. There are some exceptions though like the English Harlequin CD collection. In reality the reference is neither the CD nor the vinyl or tape. The reference is the original 78 rpm record release for mostly every published tango before 1955. A 78 rpm record looks iLP Carlos Di Sarlin size like a vinyl LP record but contains only one track per side. After 1955 started the vinyl era and some of the still present tango bands recorded then vinyl albums. In the case of D’Arienzo there were often a series of EP records and at a later stage a LP regrouping the EP releases. Di Sarli’s last recordings with Philips were later published on a single LP. I have a 1980’s reedition of this stereo record.

Grampian 636 Reverb Unit 1966

Grampian 636 Reverb Unit 1966

A typical artefact which had been introduced back then to the original recordings, is called a reverberation effect. While listening through the same titles of my collection I can distinguish versions with and without an echo effect. Famous examples are some of the early D’Arienzo recordings with Rodolfo Biagi on the piano. The initial recordings date from 1935 to 1939. If you listen to the echo containing versions, you have the impression that the orchestra must have been quite huge, recorded in a big hall with a lot of musicians. Whereas the standard version without the echo effect, sounds like very close and intimate. If I compare it to classical music, the echo version is the symphonic orchestra and the version without echo the chamber music.

The first known delay effects where available since around the 1950’s and were called Tape echo. To be more precise the echo effect is called a delay effect. With reverberation there are multiple delays and feedback so that individual echoes are blurred together, recreating the sound of an acoustic space. And I think this was precisely the intention of the sound engineers when they later added the echo to the old recordings: To add more depth to the flat sounding recording of the mono era. With the upcoming stereo culture of the 1960’s and 1970’s they wanted to give these old records a face lift and make them enter into the stereo age. As a side effect the echo also camouflaged imperfections of the original sound recording. And indeed when you listen to the echoed versions there is nearly no surface noise left. They make a clean impression.

Let’s have a closer look to the edition history of some of these tangos.

El cencerro performed by Juan D’Arienzo in 1937 is in it’s original 78 rpm record version of course without any echo effect. Later the same label, RCA Víctor, reissued El cencerro on a LP sampler called Evocando el Ayer, Vol.4, Serie Apoyando el Buen Tango CAL-3234. d'arienzo-vol4The 6 D’Arienzo instrumental LPs of this collection are very accurate reproductions of the original 78 rpm releases. They just contain some dating errors and some have no date indications at all but the sound quality is outstanding with no to very few reverberation effects and close to perfect tuning. This series had been issued somewhere in the mid to late 1970’s. In 1980 El cencerro had been again republished by RCA Víctor, this time on a 10 LP set called Serie Tango De Ayer, Vol. 3 D’Arienzo Biagi, containing a strong echo effect. It’s interesting to see how this tango is transfered later to CD: The El Bandoneón CD edition El Rey Del Compás EBCD43 (1999) as well as the Serie Tango Argentino CD Juan D’Arienzo Sus Primeros Exitós Vol. 2 (around 2004), seem to be based on the 1980 echo remix by RCA Víctor.  Also Milonga del corazón is the same version on the newer RCA Víctor BMG CD El Rey Del Compás 70 años El folderfolder.versoEsquinazo 1937-1938. It looks very much as if this LP had been the initial matrix for all follow-up D’Arienzo echo containing transfers. Therefore most of the dancers in America and also in Europe know only (or at least only had access to) the echo version as you can see on nearly all Youtube dancing videos like this memorable performance by Puppy Castello and Graciela Gonzalez from 1991 (I guess the music is played from a turntable or cassette deck and just from that aforementioned 10 LP set Serie Tango De Ayer, Vol. 3 D’Arienzo Biagi as neither the El Bandoneón nor the Tango Argentino CD existed at the time of this video!):

[youtube]http://www.youtube.com/watch?v=6f_BYsvVUz4[/youtube]

 

78RPM master record

78 rpm master record

Please see also in this context the mystery about the Don Esteban recording as described on Johan’s website and which appears to be the only ever reedition of this tango by RCA Víctor. According to Johan it looks like the masters might have been lost. This is particularily sad as this reedition and all others have this strong echo effect and the El Bandoneón and RCA BMG CD copies inherrited it! Also this version of Don Esteban appears to be slightly different to the japanese CTA version which would suggest that there might have been two seperate recording sessions back in the 1930’s

See here the sound samples:

In a way the echoed version of El cencerro made it into the salón and became some kind of accepted sound icon.

cencerros

cencerros

Maybe as a tribute to Juan D’Arienzo as he only made one recording of this particular tango back in 1937 when the recording techniques were still evolving. The echo version could be seen as an attempt to port the thin 1937 recording to a more space filling version as represented by the late D’Arienzo repertoire.

On the other hand nothing prevented the King of the Beat to publish a monumental version of El cencerro during his lifetime recording sessions which covered still another 20 years during the full frequency range stereo era from 1955 to 1975 — but as a matter of fact he didn’t, maybe he just forgot to do so … 🙂

If you want a version without the delay remix, you need to get Audio Park’s Epoca De Oro Vol. 1 APCD 6501 or CTA’s Juan D’Arienzo Vol.3 CTA-303. The japanese editions are based on their own 78 rpm transfers.

Now with this particular case presented in detail, I don’t know in how far people are aware that the echoed version is a pure invention? From the strict point of view of authenticity the echo version is untrue, some would say a lie. Back in the end of the 1970’s it had another meaning as we have seen and just until a couple of years ago it had been the only available version of this tango on second generation media, LPs, CCs and CDs! A lot of music in that time used extensively delay effects. Like on the 1977 David Bowie album Low which is interestingly from the same record label as the Juan D’Arienzo recordings, RCA Victor. This suggests that there must have been an interventionalist sound engineer at this company who added all sorts of artefacts to the historic recordings: Echo, tempo acceleration, low pass filters, etc. As to update these tunes to the taste of the day. And indeed other lables have been much more fidel when they restored and reedited their historic tango repertoire. In comparison EMI who inherited the ODEON catalogue had issued mostly very correct and authentic sounding LPs …

[youtube]Ihttp://www.youtube.com/watch?v=mkNmilE9ibk[/youtube]

 

Another added artefact to the original recordings is incorrect tempo as I just mentioned. This is actually by far a wider spread problem than echo effects, too strong noice reduction, wrong preamplification schemes or other record problems. The incorrect pitch which arises from incorrect record transcript sessions is the kind of problem which remains often unnoticed, but seems to haunt the listener unconsciously. Let’s see an example: Age Akkerman has described recently on his blog a case of an out of tune recording, Di Sarli’s interpretation of El ciruja. He  explains that he didn’t like this tango before and how retuning the tango to the correct pitch changed his mind and appreciation of this tango. Different tonal tunings can create different feelings or emotions in the listener. This can lead in the worst case to refuse and unlike a track. Most of these tonal shifts are due to a too fast running turntable during the transfer session. If you speed up or slow down a turntable you will hear how the keys of the music move around and change. Nobody really knows how these sometimes quite huge differences made it into the transfer recordings, it could have been via damaged transfer turntables or on “an early reel-to-reel tape recorder by changing the diameter of the capstan drive shaft, or using a different motor” [Wikipedia].  Nor do we really know if these changes had been done intentionally or by accident. But it seems that a lot of tangos are affected. Here is a quick test you can do in your own music collection to find such pitch problems: select the same tango, same recording date but from different sources/CD editions and check the duration and BPM fields. Are they different?

A couple of days ago I have recognised that there is something wrong with the pitch of Carlos Di Sarli’s Cosas olvidadas which sounded just too fast. Through an analysis of the partitionPartition Cosas olvidadas, determinating the original key most proberly used by the band in their arrangement and carefully listening to the voice in the recording, while slowing down the track, I have pitched down the track by a total factor of around 2.5%. Now it sounds natural. For a comprehensive guide to retuning tracks, please see Age Akkerman’s paper Lost in Keys and check the examples on his website. This kind of pitch shift can get really dangerous when women voices are involved. They are already very high and if they are then pitched up further they arrive in regions where they can produce a pain in the ear (See my article on Too fast El Bandoneón recordings).

See here the different version of Cosas olvidadas as selected from my music collection for a comparison:

Cosas olvidadas Carlos Di Sarli : Roberto Rufino TA Canta Roberto Rufino Sus Primeros Exitos Vol. 1 Duration 2:19
Cosas olvidadas Carlos Di Sarli : Roberto Rufino RR Various Artists 1941-1956 Vol. 3 Duration 2:15
Cosas olvidadas Carlos Di Sarli : Roberto Rufino BATC-DIEGON Milonga Del Sentimiento Duration 2:21
Cosas olvidadas Carlos Di Sarli : Roberto Rufino El Bandoneón El Señor Del Tango Duration 2:17
Cosas olvidadas Carlos Di Sarli : Roberto Rufino Retuned version from Sus Primeros Exitos Vol. 1 Duration 2:23

Now one could say that these different speeds don’t have any effect on the audience but I have talked to some dancers and it seems that often these pitch shifts are noticed, in most of the cases unconsciously, they say then, the music is not so nice or restless. With some DJ collegues I also recognised that they won’t play repertoire of which they have a lot of too fast transfers. they say then: “Ah, Donato, I don’t like” or “Late D’Arienzo is too hysterical”. Other DJs like these faster versions if they need to mobilise energy in tired dancers, then again I think it would be an advantage to first correct the tunings and to operate in a second step a further up pitching while locking the keys, this can be done live with modern dj programs like Mixxx or Traktor. That way tangos can be speed up without affecting the tonal keys.