Sound: Advanced Topics I
Ha ha!
Did I fake you out? The above text was generated by ChatGPT, given the prompt (from me): "tell students that this will be an exceptionally interesting class, focusing on VR, music and AI." Jeez, here I am about to retire, and my job just became trivially easy. I'll have ChatGPT write my syllabus! I'll get it to write the code for DSP algorithms (already been done, by the way...)! I'll have it write all my lectures in advance, and then generate a Brad-avatar to read them and I won't even have to be there! Heck, how do you know that ChatGPT isn't writing this very text that you're reading right now?!?. Bwah hah hah!!!!
Seriously(!) though, we will be talking about AI and LLMs towards the end of the class, but the bulk of the class will be focussed on two things -- algorithmic compositional approaches (sometimes called 'generative' or 'procedural' composition), and the deployment of them in VR (remember that? Before AI took over the techno-space?). Ultimately these can probably be coupled with AI models to produce truly new and exciting work, but the actual work to accomplish this still remains to be done. I think these tools will be important to understand and know how to deploy in The Future.
We'll take a look at aspects of algorithmic composition for the first half of the class. This will draw on some "tried-and-true" techniques, many of them have been used for decades.
Algorithmic composition will give us a base for approaching the last few weeks of the class devoted to current (and ever-changing) AI approaches to working with music. This field is presently exploding, and we'll do our best to cover some of the more salient issues. It is very much a moving target, however. I'll have a couple of guest-stars coming in to talk about their work.
The bulk of the second half of the class will be devoted to developing AR/VR/XR applications, working to embed the sound- and music-generating activities we had explored in the class previously in virtual constructions. We will be using the Unity development environment, and our obvious focus will be upon the integration of sound into the worlds we build.
Whew! To be sure, this is probably an impossible task to do in one semester. But you all are COLUMBIA students! Yay! You do IMPOSSIBLE things! Go Lions!
We will also be doing a fair amount of coding, BUT don't be worried if you have never done anything like this before. Everything we do will be with a musical goal in mind, and I think you'll pick up what we're doing without too much trouble. And for those with significant coding experience, the musical aspects should provide some new information that will allow you to expand your knowledge and keep the class from being too pedestrian. If you are feeling totally befuddled or bored, or even just a little befuddled or bored, please get in touch with Yanqi or me. We're here to help!
The computer language we will primarily use for most of the music coding we do will be RTcmix, a text-based sound synthesis and signal-processing language we developed here at Columbia. It's fairly straightforward, although it does support advanced programming structures. We will endeavor to keep things relatively simple in class. RTcmix can be run 'standalone', but we will be using its 'embedded' form, as an extension to existing development environments. Nearly all the class examples and exercises will be done using the [rtcmix~] object inside the Max/MSP or pd (Pure Data) music/media development environments. [NOTE: the [rtcmix~] object does not run in Max8.6 -- you will need to use Max8.5 (or lower) for this class.] This will set us up for deploying it in our Unity applications at the end of the class.
You should have at least some basic knowledge of digitial audio (how
to record sounds onto your computer, how to edit them, etc.) and
acoustics (frequency/amplitude/etc.). We won't be covering these
aspects of 'doing' computer music in class (but again, if you have
questions or are confused, please let us know!). We also don't assume
any great knowledge of music theory or composition, but we hope
you'll have some ideas of things you'd like to try by the end of the
term.
Collected here are links to the software we will be using in class:
Do let us know if you are having problems getting the
software to work for you. To be sure, this promises to be an intense term
because of the range of material we will be discussing.
In general, if you are having difficulty
understanding the programming paradigms
we are using or the applications and information we are covering,
be sure to talk to us. We'll be happy to sit down and work through
any issues with you.
Each week we do will become a link to information relating to that
class.
We will link all of the code,
patches and projects to each class in the syllabus for you to download
and use.
We'll try to keep up with linking in class patches, examples
and information, but we may fall behind. Yell at us when we do.
I say this every year, and generally people believe it (I think): by this point in your career the last thing you should be worrying about is a grade. The main thing is to find something that you'd really like to do and then do it. Please don't try to impress us with your consummate knowledge and skill, we are more impressed by people who do things. Honest!
Hope you enjoy the class!
