App Anatomy: Virtuoso, a Music Sequencer
February 3, 2014
Virtuoso is a music sequencer for Windows 8.1. The app provides a fun and easy to use interface that allows you to compose original music using drums, bass, strings, vibes and guitar. Virtuoso will play 16 beats of music in a continuous loop. You can interact with the sequencer in real time changing notes, tempo and instrument volumes. You can save your songs to disk for playback at a later time.
Musicians can use Virtuoso as a scratch pad for their musical ideas and parents can use Virtuoso to teach their children the basics of music; rhythm, harmony and melody.
- 5 Instruments, 65 Voices
- 16 Beat Sequencer
- Change tempo and mix instruments in real time
- Play and Pause control
- Save your songs to disk
- Built in Tutorial
This article will cover the following topics:
- Class Structure
- Playing Audio using the SharpDX.XAudio2 API
- Background Threads
- Updating the UI from a background thread using the Dispatcher
- App State File IO using XML Serialization
- Deploying files at runtime using Zip File Decompression
Virtuoso at its core is a state machine. The root class of the app’s state machine is called CAppState. This class provides access to the CSong, CSoundLibrary, CStorage and CSequencer classes. I will review each of these classes in some detail.
A song consists of a grid of 512 notes for each of 5 instruments for a total of 2560 notes. The notes are logically arranged in 16 columns of 13 notes each. Each column represents an octave of notes from a low C to a high C. There are 16 columns representing the 16 beats of the sequencer pattern. A Song also tracks the volume settings for each instrument, the song name, description and tempo.
Each Note holds a set of simple properties, the sound bank association, the state of the note, on or off, and the name of the UI element that the note is associated with. The UI Element name is required so that as the note is being sounded, the rectangle on the screen can be animated from a background thread. More on that later.
Virtuoso provides 5 instruments, drums, bass, strings, vibes and guitar, that can be used to arrange a 16 beat song. Each instrument is represented by the CSoundBank class. The CSoundLibrary class is a collection of CSoundBank’s.
Each CSoundBank class contains a Sound class and a list of SoundFiles. There are 13 sound files per sound bank, a sound file for each note of the octave. The Sound class manages the DirectX XAudio2 classes used to play sounds and the SoundFile class manages an in-memory instance of a WAV file. Virtuoso uses the SharpDX library to access the DirectX audio API to play each active note within the song arrangement.
Playing Audio with the SharpDX.XAudio2 API
SharpDX is an open-source project delivering the full DirectX API under the .Net platform, allowing the development of high performance game, 2D and 3D graphics rendering as well as real-time sound application.
You can add SharpDX to your Visual Studio Solution through the NuGet package manager. Go to the Tools > Library Package Manager > Manage NuGet Packages for Solution
This will bring up the Mange NuGet Packages dialog box. Type SharpDX into Search and you will see a list of possible SharpDX libraries you can add to your solution.
For Virtuoso, I added SharpDX and SharpDX.XAudio2.
There are 2 classes that work with the XAudio2 API, SoundFile and Sound. The SoundFile class manages the in-memory WAV files that are being used by the app. Virtuoso uses 65 individual WAV files, 13 sounds for each of the 5 instruments. The Sound class encapsulates the code used to play a WAV file. There are 5 instances of the Sound class, one for reach instrument.
Here is the code that initializes the SoundFile class using the SharpDX XAudio2 API. This code loads the WAV file into memory and streams the contents into an AudioBuffer.
The Sound class uses the XAudio2, MasteringVoice and SourceVoice to play a WAV file that has been loaded into memory into an AudioBuffer.
This is the code that initializes the XAudio2 and MasteringVoice classes on instantiation.
Here is the code that plays a WAV file using a SoundFile. This code will use the SourceVoice class to play the sound.
Up to this point we have been examining the classes that mange our data; the song, notes and sounds. The CSequencer class uses this data to play the song.
In order for the sequencer to function smoothly and allow the user to continue to interact with the UI, turning notes on and off, setting volumes and tempo, the sequencer must run in a background thread.
Running code in a background thread is fairly straight forward on Windows. Using the ThreadPool API you can kick off a background thread that will execute a routine. If you design your routine to be a loop it will run until you tell it to stop.
I used the following MSDN sample to learn how to add a background thread to my app
To initialize the CSequencer class, the app state is passed in on instantiation. Note the local member _threadPoolWorkItem which is of type IAsyncAction. This is the private member of the class used to store the reference to the background thread.
The background thread is kicked off in the CSequencer.Run() method. I create a background thread by calling the Windows.System.Threading.ThreadPool.RunAsync() with the execution code defined inline and store a reference to this background thread in class property of type IAsyncAction.
To cancel the execution of the background thread I call the IAsyncAction.Cancel() method of the work item.
In Virtuoso, the user is in full control of the sequencer as the Play button is associated with the CSequencer.Run() method and the Pause button is associated with CSequencer.Stop() method.
Updating the UI from the Background Thread using the Dispatcher
One of the interesting features of Virtuoso is that as a note is played, the associated tile on the screen is animated. This gives the app UI some flare while also providing the user feedback on where the sequencer is in the loop.
Animating UI elements has to be done on the UI thread of your app. In the case of Virtuoso, the knowledge of when to animate is only known in the background thread which is managed by the sequencer class. So in order to accomplish the animation I had to find out how to update the UI from a background thread.
To follow the logic of this we will start at the end of the process and work our way back to the background thread. The first thing we will look at is how to animate the note tile. I used this sample from Iris Classon to create a routine that would animate the size of the rectangle:
Sample Code from Iris Classon – Creating a ScaleTransform animation in C# for WinRT Apps
I added a static class with this functionality to my solution called CAnimationHelper.
This routine takes a UIElement and animates the ScaleX and ScaleY properties. I then added a call to the ClickableItem() method in a method called Animate() in the MainWindow class:
In order to identify the UI element that is to animated, I store the name of that item in each Note class so that when I want to animate that UIElement from the background thread I can pass the name to this method and then look up the actual UIElement by using the FindDescendantByName() routine (see below). This will return the object in the visual tree that corresponds to the name provided. This recursive routine is encapsulated in a static class called FrameworkElementExtensions:
I store a reference to the MainWindow in the CAppState class which is passed to the CSequencer class on instantiation. Using this reference I can access the animate code through the CSequencer.UpdateUI() method:
I also use the MainWindow reference in the background thread to access the UI Thread Dispatcher. The Dispatcher provides services for managing the queue of work items for a thread. Through the UI Thread Dispatcher I can invoke code that runs on the UI thread from the background thread. When a note is played by the sequencer on the background thread, a call is made to animate the rectangle on the UI thread via the Dispatcher:
You can easily imagine this technique being used for other purposes such as updating the UI as progress is made on a task being run on a background thread, i.e. progress bars, countdowns, etc.
Saving and Loading Songs
In this release of Virtuoso, I provide a feature to save and load Song data to disk. This is not audio data but the song state data; name, description, tempo, notes, etc. I use XML serialization to save and load the song data to/form disk. The class that is serialized is the CSong class. The code that saves and restores the file from disk is found in the CStorage Class
The process of saving a file is:
- get a reference to the system folder (local, roaming)
- create/open an app folder in that location
- create/open a file in the app folder
Sample code from Isis Classon – Xml Serialization
Here is the code that implements this process in the Save() method of the CStorage class:
The resulting XML file looks like this:
And here is the code that loads the Song file from disk:
Deploying Sample Files at Runtime
Virtuoso comes with 3 sample song files. In order to have the sample song files available to the user I leveraged a feature of the Window platform which is the ability to unzip a Zip file at runtime.
I also needed to make sure that the Zip file was in the users roaming folder so I also leveraged the ability to copy a file from the app installation folder to the users roaming folder at startup. I created the
This capability is extremely useful for apps that need to provide data files, sample files to any other kind of app data locally on disk through the installation and startup procedures.
Sample Code from Iris Classon – Zip File Decompression
Here is the code I used to unzip the Zip file which is in the apps installation folder and save the resulting files into the song folder:
I had a blast designing and implementing Virtuoso. I used an iterative approach to development, first releasing a simple 1 instrument 16 beat sequencer called Comp. Then I evolved that application adding 4 more instruments and additional features. I will be adding more features to Virtuoso over time following the Continuous Innovation cycle as defined by Eric Reis in his book The Lean Startup.
I hope you found it useful to review the anatomy of my app. I look forward to down loading your app from the store real soon. –bob