App Anatomy: Virtuoso, a Music Sequencer – The Undocumented API
Loading...
X

App Anatomy: Virtuoso, a Music Sequencer

Overview

Virtuoso is a music sequencer for Windows 8.1. The app provides a fun and easy to use interface that allows you to compose original music using drums, bass, strings, vibes and guitar. Virtuoso will play 16 beats of music in a continuous loop. You can interact with the sequencer in real time changing notes, tempo and instrument volumes. You can save your songs to disk for playback at a later time.

Screen2

Musicians can use Virtuoso as a scratch pad for their musical ideas and parents can use Virtuoso to teach their children the basics of music; rhythm, harmony and melody.

Features

  • 5 Instruments, 65 Voices
  • 16 Beat Sequencer
  • Change tempo and mix instruments in real time
  • Play and Pause control
  • Save your songs to disk
  • Built in Tutorial

This article will cover the following topics:

  • Class Structure
  • Playing Audio using the SharpDX.XAudio2 API
  • Background Threads
  • Updating the UI from a background thread using the Dispatcher
  • App State File IO using XML Serialization
  • Deploying files at runtime using Zip File Decompression

App Anatomy

State Management

Virtuoso at its core is a state machine. The root class of the app’s state machine is called CAppState. This class provides access to the CSong, CSoundLibrary, CStorage and CSequencer classes. I will review each of these classes in some detail.

CappState

Songs

A song consists of a grid of 512 notes for each of 5 instruments for a total of 2560 notes. The notes are logically arranged in 16 columns of 13 notes each. Each column represents an octave of notes from a low C to a high C. There are 16 columns representing the 16 beats of the sequencer pattern. A Song also tracks the volume settings for each instrument, the song name, description and tempo.

CSong

Each Note holds a set of simple properties, the sound bank association, the state of the note, on or off, and the name of the UI element that the note is associated with. The UI Element name is required so that as the note is being sounded, the rectangle on the screen can be animated from a background thread. More on that later.

Sound

Virtuoso provides 5 instruments, drums, bass, strings, vibes and guitar, that can be used to arrange a 16 beat song. Each instrument is represented by the CSoundBank class. The CSoundLibrary class is a collection of CSoundBank’s.

Each CSoundBank class contains a Sound class and a list of SoundFiles. There are 13 sound files per sound bank, a sound file for each note of the octave. The Sound class manages the DirectX XAudio2 classes used to play sounds and the SoundFile class manages an in-memory instance of a WAV file. Virtuoso uses the SharpDX library to access the DirectX audio API to play each active note within the song arrangement.

CSoundLibrary

Playing Audio with the SharpDX.XAudio2 API

SharpDX is an open-source project delivering the full DirectX API under the .Net platform, allowing the development of high performance game, 2D and 3D graphics rendering as well as real-time sound application.

You can add SharpDX to your Visual Studio Solution through the NuGet package manager. Go to the Tools > Library Package Manager > Manage NuGet Packages for Solution

image

This will bring up the Mange NuGet Packages dialog box. Type SharpDX into Search and you will see a list of possible SharpDX libraries you can add to your solution.

image

For Virtuoso, I added SharpDX and SharpDX.XAudio2.

image

There are 2 classes that work with the XAudio2 API, SoundFile and Sound. The SoundFile class manages the in-memory WAV files that are being used by the app. Virtuoso uses 65 individual WAV files, 13 sounds for each of the 5 instruments. The Sound class encapsulates the code used to play a WAV file. There are 5 instances of the Sound class, one for reach instrument.

Support article on Playing Audio with SharpDX Xaudio2

Here is the code that initializes the SoundFile class using the SharpDX XAudio2 API. This code loads the WAV file into memory and streams the contents into an AudioBuffer.

image

 

 

 

 

 

 

 

 

The Sound class uses the XAudio2, MasteringVoice and SourceVoice to play a WAV file that has been loaded into memory into an AudioBuffer.

This is the code that initializes the XAudio2 and MasteringVoice classes on instantiation.

image

Here is the code that plays a WAV file using a SoundFile. This code will use the SourceVoice  class to play the sound.

image

Sequencing

Up to this point we have been examining the classes that mange our data; the song, notes and sounds. The CSequencer class uses this data to play the song.

CSequencer

In order for the sequencer to function smoothly and allow the user to continue to interact with the UI, turning notes on and off, setting volumes and tempo, the sequencer must run in a background thread.

Running code in a background thread is fairly straight forward on Windows. Using the ThreadPool API you can kick off a background thread that will execute a routine. If you design your routine to be a loop it will run until you tell it to stop.

I used the following MSDN sample to learn how to add a background thread to my app

Thread pool sample 

To initialize the CSequencer class, the app state is passed in on instantiation. Note the local member _threadPoolWorkItem which is of type IAsyncAction. This is the private member of the class used to store the reference to the background thread.

image

The background thread is kicked off in the CSequencer.Run() method. I create a background thread by calling the Windows.System.Threading.ThreadPool.RunAsync() with the execution code defined inline and store a reference to this background thread in class property of type IAsyncAction.

image

To cancel the execution of the background thread I call the IAsyncAction.Cancel()  method of the work item.

image

In Virtuoso, the user is in full control of the sequencer as the Play button is associated with the CSequencer.Run() method and the Pause button is associated with CSequencer.Stop() method.

Screen2

Updating the UI from the Background Thread using the Dispatcher

One of the interesting features of Virtuoso is that as a note is played, the associated tile on the screen is animated. This gives the app UI some flare while also providing the user feedback on where the sequencer is in the loop.

Animating UI elements has to be done on the UI thread of your app. In the case of Virtuoso, the knowledge of when to animate is only known in the background thread which is managed by the sequencer class. So in order to accomplish the animation I had to find out how to update the UI from a background thread.

To follow the logic of this we will start at the end of the process and work our way back to the background thread. The first thing we will look at is how to animate the note tile. I used this sample from Iris Classon to create a routine that would animate the size of the rectangle:

Sample Code from Iris Classon – Creating a ScaleTransform animation in C# for WinRT Apps

I added a static class with this functionality to my solution called CAnimationHelper.

image

This routine takes a UIElement and animates the ScaleX and ScaleY properties. I then added a call to the ClickableItem() method in a method called Animate() in the MainWindow class:

image

In order to identify the UI element that is to animated, I store the name of that item in each Note class so that when I want to animate that UIElement from the background thread I can pass the name to this method and then look up the actual UIElement by using the FindDescendantByName() routine (see below). This will return the object in the visual tree that corresponds to the name provided. This recursive routine is encapsulated in a static class called FrameworkElementExtensions:

image

I store a reference to the MainWindow in the CAppState class which is passed to the CSequencer class on instantiation. Using this reference I can access the animate code through the  CSequencer.UpdateUI() method:

image

I also use the MainWindow reference in the background thread to access the UI Thread Dispatcher. The Dispatcher provides services for managing the queue of work items for a thread. Through the UI Thread Dispatcher I can invoke code that runs on the UI thread from the background thread. When a note is played by the sequencer on the background thread, a call is made to animate the rectangle on the UI thread via the Dispatcher:

image

You can easily imagine this technique being used for other purposes such as updating the UI as progress is made on a task being run on a background thread, i.e. progress bars, countdowns, etc.

Saving and Loading Songs

In this release of Virtuoso, I provide a feature to save and load Song data to disk. This is not audio data but the song state data; name, description, tempo, notes, etc. I use XML serialization to save and load the song data to/form disk. The class that is serialized is the CSong class. The code that saves and restores the file from disk is found in the CStorage Class

image

The process of saving a file is:

  1. get a reference to the system folder (local, roaming)
  2. create/open an app folder in that location
  3. create/open a file in the app folder
  4. serialize/deserialize

Sample code from Isis Classon – Xml Serialization

Here is the code that implements this process in the Save() method of the CStorage class:

image

The resulting XML file looks like this:

image

And here is the code that loads the Song file from disk:

image

Deploying Sample Files at Runtime

Virtuoso comes with 3 sample song files. In order to have the sample song files available to the user I leveraged a feature of the Window platform which is the ability to unzip a Zip file at runtime.

I also needed to make sure that the Zip file was in the users roaming folder so I also leveraged the ability to copy a file from the app installation folder to the users roaming folder at startup. I created the

This capability is extremely useful for apps that need to  provide data files, sample files to any other kind of app data locally on disk through the installation and startup procedures.

Sample Code from Iris  Classon –  Zip File Decompression

Here is the code I used to unzip the Zip file which is in the apps installation folder and save the resulting files into the song folder:

image

Conclusion

I had a blast designing and implementing Virtuoso. I used an iterative approach to development, first releasing a simple 1 instrument 16 beat sequencer called Comp. Then I evolved that application adding 4 more instruments and additional features. I will be adding more features to Virtuoso over time following the Continuous Innovation cycle as defined by Eric Reis in his book The Lean Startup.

I hope you found it useful to review the anatomy of my app. I look forward to down loading your app from the store real soon. –bob

Leave Your Observation

Your email address will not be published. Required fields are marked *

Read previous post:
Nokia Windows Phone Developer Day

What: Windows Phone Developer Day When: Feb 1st, 12pm to 6pm Where: Microsoft Offices, 1 Cambridge Center, Cambridge, MA 02142...

Close