Experienced conductors of music ensembles are not metronomes: their hand movements and the speed at which the players perform exhibit a complex time and context dependent means of communication. The aim of this project is to analyze this relationship between the conductor's movements and the actual tempo as performed by the players, and apply the results from this analysis to construct a computer based system that will mimic the salient behavior of a real ensemble.

Models for different conductors are obtained by first having a conductor lead a live ensemble. Data from the conductor is obtained using 3-D position sensors; the performance is also recorded digitally in a sound file. Velocity, acceleration, direction, and position data of the movements are used to extract features and determine the location of beats. By synchronizing this information with the audio data and by using information about music performance and the score, a model is constructed that produces, at each point in time, the implied tempo for the ensemble. Thus this model ultimately deduces implied tempo from hand movements.

Once a model has been formed, the system can be put into "perform" mode in which the user can "conduct" in real time by controlling the playback speed of a MIDI sequencer. The conductor uses 3-D trackers to conduct; data from hand movements is fed into the model, processed in real time, and then used to control the tempo of the virtual ensemble. There is also a visualization of the user's movements. Once the user has conducted a piece it is also possible to replay the piece orally and visually. This computer system enables conducting students to experience the complex coupling between movements and actual tempo of the ensemble and to conduct a "virtual" ensemble or a mixed virtual/real ensemble.