The construction of this system is an attempt to realize a particular dynamic in the act of composing: uncertainty. This system is an attempt to allow me to compose a piece that is controlled and defined while maintaining a level of uncertainty each time the piece is performed. Ideally the output of this system, while being probabilistically predictable, will never be 100% certain under any condition. It will likely feel a bit clumsy and temperamental while allowing the human performers to learn to shape it and train it even as it reshapes and reinterprets the composition it contains.
Spectral Analysis: Supercollider provides FFT tools for spectral analysis and resynthesis of audio in realtime.
Learning Space: an implementation of a Self-Organizing Map. A short-term window of memorized and contextualized sound data taken from the performer connected to the system input.
Knowledge Base: A database of prior training material in a compatibly analyzed format. The contents of the KB will be reorganized by a Naive Bayesian Method using Orange to optimize search returns. This is the true guts of the system. The output is dependent on the contents of this component. And any audio could be put in here. In practice it will likely contain short compositional elements. Phrases and patterns created to represent the intention of the composition. This is the data to be mined by the system's collaborators.
Spectral Synthesis: Resynthesis is possible via Supercollider's FFT ans ATS objects. However, depending on the latency and quality of returned results some re-keying or harmonizing may be necessary to control the experience in improvisational collaboration.
Ideally this vague and precise cross-referencing will result in a probabilistic spectral suggestion to the spectral synthesizer. Something like neither the input nor the stored musical phrases, but suggestive of both. In hope: something new. Will this seem no more than a complex audio filter? Will it be generative or derivative? Keeping the possibilities in mind I intend to shape the system to be capable of recontextualizing the knowledge it has to accompany the performance it hears. If the knowledge stored in the system is the intended composition in whole or in pieces, the output should be a reinvention of the system using the composition as a guideline.
Prelude (my winter break):
To start I must formalize a solid algorithm and investigate the implementation of each piece. I am giving myself until the beginning of winter quarter to be sure this is possible in the time alloted.
WEEK 1: Spectral Analysis:
I must implement a useful realtime spectral analysis algorithm to generate computational datum. SuperCollider has two potential methods for this: FFT and ATS. I will investigate the usefulness and properties of these components. A training routine must be developed to allow Supercollider to work through a collection of training files and populate a database for Orange.
WEEK 2: Knowledge Base:
Using the db created in week one and AILabs software Orange I will reorganize the spectral data using a Naive Bayesian Algorithm. I will also define essential search criteria to link to the Learning Space.
Experiment 1: Analyze a body of music and evaluate it with a variety of modules in Orange to determine the best fit AI module.
WEEK 3 Learning Space:
Implement a scalable SOM with nodes containing the appropriate matching data for referencing the K.B. Attempt to define the best scale and nodal/temporal window for capturing musical gestures.
Real time scalable nodes?
Based on key change?
or discreet pitches?
WEEK 4:
Put these first three components together and get verifiable data output in the form of debugging data spilled into the terminal.
Experiment 2: Get the system to respond to input and spew forth glorious debug data.
WEEK 5: Spectral Synthesis:
FFT and ATS both provide the functionality to resynthesize spectral data. Special care should be taken to ensure that what is being synthesized is musically scaled to what is currently being played by the performers and not a delayed reaction to what they had played. Even if it is delayed, by shifting the fundamental frequency and re-scaling the spectral data, the output can be relevant to the performers.
WEEK 6: I/O Fidelity and Attenuation:
If I make it this far and everything is sounding good, then I will know what to do next. If I meet with disaster up to this point then I will take what I can from my experiments and revise my goals. The outcome of this process is at best uncertain. Ideally, even in failure I can find some tractable results to create something meaningful out of my experiments and derive a useful implementation to move forward.
Experiment 3: Connect the spectral synthesis component to the output of the system and evaluate the effectiveness of the output. Attenuate each component as needed.
WEEK 7: Experiments with Performers:
Dependent of the previous results. If I am successful I will be working with the system and performers to find a synergy between them. Trying to strike a suitable balance. Otherwise I will be redirected in my intentions.
Experiment 4: Use the system with a performer and attenuate the output to be interesting and collaborative.
WEEK 8:
WEEK 9:
WEEK 10: