analysis, recording + other things
We tied up a few 'loose ends' with the basic audio functioning of
Tone.js and delved into various approaches to recording the audio output
of web-audio apps. To be sure, more 'loose ends' exist, but I think
we've managed to cover a lot.
Links
We looked briefly at the base Web Audio HTML5 API. This is the mozilla
link, and it's not a bad intro to how the web audio code underlying
Tone.js works:
- Web Audio API
-- Yotam had mentioned the "AudioWorklet" as a new feature that might be
very useful. The "WebAssembly" reference is also interesting.
- Web Audio tutorial
-- actually pretty decent. You'll see a lot of similarities between
how Web Audio works and Yotam's Tone.js.
Useful links for approaches to recording the output of web apps:
- JS Web Audio Recorder
-- this seems about the best JavaScript recording system out there. You
can integrate it into your own code without too much trouble, I hope.
- JS Web Audio Recorder Demo
- JS Web Audio Recorder Demo
-- two demos of the above (JS Web Audio Recorder), pretty much the same
- MediaRecorder
-- this is the system Yotam had mentioned. I think it's a little trickier
to set up and use than the JS Web Audio Recorder above.
- Soundflower
-- the OSX system for routing audio output from your browser to a
recording app on your local machine. Very useful.
- JACK audio system
- "Virtual Audio Cable"
-- these are two systems that can work on Windows, I think. I'm not 100%
sure if JACK can run as a standalone system. I usually see it integrated
into specific Windows (and other OSes) apps.
Class Downloads
- week8-html-files.zip
-- all the html/javascript files we created in class (covered below).
This archive also contains the digital audio file ("loocher441.wav")
we used in the class as well as the
startlocalserver command enabling us to load soundfiles into
the 'localhost' server for testing. See
week6 for instructions about
how to do this. We only needed this for the "grainplayer"
code.
In fact, the very first thing we did was to explore the "GrainPlayer"
object in Tone.js (note: because we are reading a soundfile resource,
you will need to put this on a web-hosted machine or use the
startlocalserver approach):
- grain1.html
-- just check to see if soundfile is loading and playing
- grain2.html
-- set up and use the Tone.GrainPlayer object
use your "view source" browser capability to see the code of these
examples
Next we tried some simple experiments in 'cross-synthesis', using
the level of a microphone-input signal to control the amplitude
and the filter characterstics of a noise signal:
- levelctrl1.html
-- control the amplitude of noise (using a Tone.Volume object)
- levelctrl2.html
-- control the amplitude of noise and the center frequency of
a bandpass filter applied to the noise
- levelctrl2a.html
-- use a try{ } catch() { } construction to catch the "-infinity"
annoying (but non-fatal) error when first starting the code
After having so much fun applying a simple amplitude control,
we then built a very basic FFT-based resynthesis instrument, using the
Tone.FFT analyser object:
- manyoscs.html
-- test code to see how many oscillators we could run
- resynth1.html
-- take the FFT bin/amplitude data and apply it to a bank of oscillators
- resynth2.html
-- randomize the oscillator frequencies within each bin to try to
eliminate the harmonicity of the output
- resynth3.html
-- use narrow-band noise for each bin instead of an oscillator
The problem is that Tone.FFT, relying on the underlying Web Audio
FFT/Analyser, only reports the amplitude values for each FFT
bin. Better FFT algorithms report both the amp AND the frequency
for the bins. We would need this for each oscillator in a bin
to play the correct frequency (obviously). The Tone.Convolve
object probably does this.
I pointed out the Tone.Convolve object but we didn't do anything with it.
Feel free to experiment on your own! Also, we didn't follow-up much
with Yotam's discussion of the offline-rendering capability of
Tone.js. Something for the Future.
We took a quick look at the Web Audio API, to see what it covered and
with an eye towards finding a way to record the output of our
web audio apps. The links above in the Links section
point to this information.
Because the Web Audio API doesn't really support output/buffer
recording or saving, we examined several other methods for doing
this. Again, they are linked above in the Links section.