Creating Baroque Music with a Recurrent Neural Network

Bach Music Generator

Training a recurrent neural network to generate baroque music

Overview

The aim of this project is to use a time series neural network to generate music trained on 15 Chorales by Bach. A lot of this involves parsing information from midi files into the notes for the melody. There is not a lot of info for processing midi files…at all. So to start off, I’m only predicting the notes from the melody (as opposed to bass, rhythm, or whatever the different layers are called) and without time i.e. every note is assumed to last for the same time.

To do this, I will construct a basic recurrent neural network using subsequent notes as labels for the previous one. Future efforts can be made to incorporate multiple notes per entry and including note length as predictors.

This notebook is going to walk through the process of handling audio data and the building an RNN to generate music. In the end, I should be able to input a song and get a slightly (maybe) different song in a baroque style.

Processing Midi Files

Some functions I wrote for parsing the notes from the melody in the track into something the network can process and then back into a midi message

In [1]:
import numpy as np
import mido

def decodeMidi(midifile,num_layers=2):
    song = []
    for i, track in enumerate(midifile.tracks):
        song.append([])
        for msg in track:
            message = str(msg).split()
            if '<meta' not in message and 'control_change' not in message and 'program_change' not in message:
                channel = int(str(msg).split()[1].split("=")[-1])
                note = int(str(msg).split()[2].split("=")[-1])
                velocity = int(str(msg).split()[3].split("=")[-1])
                time = int(str(msg).split()[4].split('=')[-1])
                song[i].append([channel, note, velocity, time])

    song = [x for x in song if x]
    song = [song[:num_layers]]
    return np.array(song)

def encodeMidi(song):
    file = mido.MidiFile()
    for i in range(len(song[0])):
        track = mido.MidiTrack()
        track.append(mido.Message('control_change', channel=0, control=0, value=80, time=0))
        track.append(mido.Message('control_change', channel=0, control=32, value=0, time=0))
        track.append(mido.Message('program_change', channel=0, program=50, time=0))
        for j in range(len(song[0][i])):
            note = mido.Message('note_on',channel=int(song[0][i][j][0]), note=int(song[0][i][j][1]), velocity=int(song[0][i][j][2]), time=int(song[0][i][j][3]))
            track.append(note)
        file.tracks.append(track)
    return file


def midiToNote(midifile,num_layers=2):
    song = []
    for i, track in enumerate(midifile.tracks):
        song.append([])
        for msg in track:
            message = str(msg).split()
            if '<meta' not in message and 'control_change' not in message and 'program_change' not in message:
                note = int(str(msg).split()[2].split("=")[-1])
                song[i].append([note])

    song = [x for x in song if x]
    song = [song[:num_layers]]
    return np.array(song)

def predictionsToNotes(preds):
    song = preds[0][0]
    song = song.tolist()
    _ = 0
    noteIndex = []
    for i in song:
        best = max(song[_])
        key = song[_].index(best)
        noteIndex.append(key)
        # print(best,key)
        _ += 1
    return (noteIndex)

def notesToMidi(notes, velocity = 95, time = 116):
    file = mido.MidiFile()
    track = mido.MidiTrack()
    file.tracks.append(track)
    track.append(mido.Message('program_change', program=12, time=time))
    for i in range(len(notes)):
        track.append(mido.Message('note_on', note=notes[i], velocity=velocity, time=time))
    return(file)

Here I process the files, padding each track to 1000 messages (notes), labels are the subsequent notes.

In [2]:
from processingData import decodeMidi
import mido
import numpy as np
import os

files = os.listdir('train')
features = np.zeros(shape=(1,1000,88))
labels = np.zeros(shape=(1,1000,88))
for file in files:
    print(file)
    mid = mido.MidiFile("train/"+file)
    mid = decodeMidi(mid)
    mid = mid[0][0]
    featuresPart = []
    labelsPart = []
    for i in range(1001):
        if i < len(mid):
            onehot = [0] * 88
            onehot[mid[i][1]] = 1
            featuresPart.append(onehot)
            onehot = [0] * 88
            try:
                onehot[mid[i+1][1]] = 1
                labelsPart.append(onehot)
            except Exception:
                pass
        else:
            pad = [0] * 88
            featuresPart.append(pad)
            labelsPart.append(pad)
    featuresPart = np.array([featuresPart[:1000]])
    labelsPart = np.array([labelsPart[:1000]])
    features = np.concatenate((features,featuresPart))
    labels = np.concatenate((labels,labelsPart))


features = features[1:]
labels = labels[1:]
print(features.shape)
print(labels.shape)
01AusmeinesHerz.mid
02Ichdankdir.mid
03AchGott.mid
04EsistdasHeiluns.mid
05AnWasserflussen.mid
06Christus.mid
07Nunlob.mid
08Freueteuch.mid
09Ermuntredich.mid
10AustieferNot.mid
11Jesu.mid
12PuerNatusinBet.mid
13Alleinzudir.mid
14OHerreGott.mid
15ChristlaginTode.mid
(15, 1000, 88)
(15, 1000, 88)

Building and Training the Model

Use Keras’ simple RNN to train the model.

In [3]:
%%
from keras.models import Sequential
from keras.layers import TimeDistributed, SimpleRNN, Dense

from keras.callbacks import ModelCheckpoint

model = Sequential()
model.add(SimpleRNN(input_dim  =  88, output_dim = 88, return_sequences = True))
model.add(TimeDistributed(Dense(output_dim = 88, activation  =  "softmax")))
model.compile(loss = "mse", optimizer = "rmsprop", metrics=['accuracy'])
model.fit(features, labels,
          epochs=1000,
          batch_size=256,
          callbacks=[ModelCheckpoint("Simple_RNN_3", monitor='val_acc',save_best_only=True)])
model.save("Simple_RNN_3.h5")

Generating Music from Beethoven’s Moonlight Sonata

In [4]:
from keras.models import load_model
from processingData import decodeMidi, encodeMidi
import mido
import numpy as np
import pickle

song = mido.MidiFile("moonlightSonata.mid")
song = decodeMidi(song)
song = song[0][0]
test = []

for i in range(len(song)):
    onehot = [0] * 88
    onehot[song[i][1]] = 1
    test.append(onehot)
    onehot = [0] * 88

test = np.array([test[:1000]])
print(test.shape)

model = load_model('Simple_RNN_3.h5')
prediction = [model.predict(test)]

savePredictions = open("moonlightPrediction.pickle","wb")
pickle.dump(prediction, savePredictions)
savePredictions.close()
(1, 236, 88)

Converting Results to Midi

In [5]:
import pickle
from processingData import predictionsToNotes, notesToMidi
import mido
import numpy as np

pred = pickle.load(open('moonlightPrediction.pickle','rb'))
pred = predictionsToNotes(pred)
file = notesToMidi(pred,time=60)
print(file)

file.ticks_per_beat = 120
port = mido.open_output()

for msg in file.play():
    port.send(msg)
    

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s