RSS
Over at Audio Cookbook, John Keston has been running a few experiments with using image filters in Photoshop to process sound. Running the audio data through a Gaussian blur or Spherize filter, he was able to create some incredibly diverse effects from a simple electric piano input.
To test this concept I created a simple pattern with an electric piano patch and opened it in Photosounder. Without changing any settings I immediately saved the sound as a bitmap image. Next I opened the image in Photoshop and started experimenting with filters. Once I had some filtered images I loaded them back into Photosounder to see how they sounded. Gaussian blur and Liquefy created some unique effects, but my favorite of the bunch was Glowing Edges. This filter seems to transform the electric piano into a haunting choral passage.
John has a number of MP3 samples on his site. To get the audio in and out of Photoshop, he's using a tool called Photosounder which translates a waveform into bitmap data and vice-versa. I've never used the app, but from looking at the output images, time is represented on the x dimension, y represents the frequency, and the brightness of the pixel is determined by the amplitude at that frequency/time coordinate.
Anyone interested in writing a sound to image converter in Perl?
Processing Sound Using Photoshop
Audio Cookbook
Photosounder
Wonder how that program does the sound image conversion. How do they translate the pixels into sounds? I presume first off from what can be seen is colour -> B&W. Then do they average across a vertical stripe?
Reply to this comment
to first post (next time create a name or something) reread the paragraph after the text block, it explains it right there.
Reply to this comment
It's just a simple FFT using either Welch's method or Bartlett's method.
Your favourite mathematical software ( matlab , octave , python/numpy/scipy , scilab etc... ) can easily do it.
Reply to this comment
"time is represented on the x dimension, y represents the frequency, and the brightness of the pixel is determined by the amplitude at that frequency/time coordinate."
Yep - a spectrogram. Very cool idea.
Reply to this comment
robert : It's actually envelope detection on a filter bank. The difference with the techniques you mentioned is that it allows for a fixed pitch resolution, and therefore a time resolution that varies according to the pitch, which offers a better sounding compromise, but higher resolution algorithms are in the works anyways. By the way if you think that a basic implementation of the processing algorithms is all you need, you'll find the free and open source ARSS http://arss.sourceforge.net/ just as interesting ;-).
Reply to this comment
This program reminds me of RGS(Realtime Graphical(Granular?) Synthesizer) for the Amiga. I remember my friends thinking I was lightyears ahead of them in terms of music creation thanks to the ability to load sounds, save the image, edit the image and then reload it to see what it sounded like. I kept trying to get the source code for that so I could revamp it for 16 bit or greater sound but the creator said he thought he lost it, plus it was in some dialect of Forth that I don't recall seeing/using.
Reply to this comment
Aphex Twin used a spectrogram of his face as a sound source for the second track of the Windowlicker single. So... ooh!
Reply to this comment
http://hem.passagen.se/rasmuse/Coagula.htm
Reply to this comment
If you want to make graphical sound processing in real time, you may use cycling74's Max MSP Jitter environment. I wrote "A Tutorial on Spectral Sound Processing Using Max/MSP and Jitter", that might get you started (Computer Music Journal, Fall 2008). Max MSP embeds good tools to use FFT, and Jitter matrices and matrix manipulation tools make it easy to edit spectral data. The article is available for free: http://www.mitpressjournals.org/doi/abs/10.1162/comj.2008.32.3.87
In addition, I posted a number of patches about this kind of phase vocoder on http://www.cycling74.com/twiki/bin/view/Share/Jean-francoisCharles
Hope that helps.
Reply to this comment
i thought i was the only one hehe! Instead of using a sound to picture converter you can always just use cooledit pro (or adobe audition as it's now called) to open the file as raw data, this way you can just save it as a wave file when done with it.
-fluorescent grey
Reply to this comment
This workflow is now directly supported by Adobe Audition 3.0. You simply open an audio file, Save As choosing the "Spectral Bitmap Image (*.bmp)" then open the file in Photoshop and edit. After editing, import the .bmp back into Audition.
When saving an audio file as a bitmap, the phase map is automatically saved as well which should improve your results and/or provide more tweaking options with the phase bitmap file.
Reply to this comment