In [48]:
%matplotlib inline
import seaborn
import numpy, scipy, sklearn, matplotlib.pyplot as plt, IPython.display as ipd, librosa, urllib

← Back to Index

Classification of Separated Signals

After you have performed signal decomposition (e.g. using librosa.decompose.decompose), try to classify and combine the separated signals.

Download an audio file containing bass drum, snare drum, and hi-hat:

In [11]:
filename = 'Classic Rock Beat 06.wav'
urllib.urlretrieve(
    'http://audio.musicinformationretrieval.com/Jam Pack 1/' + filename,
    filename=filename
)
Out[11]:
('Classic Rock Beat 06.wav', <httplib.HTTPMessage instance at 0x10d52cfc8>)

Load the audio file into an array:

In [12]:
x, fs = librosa.load(filename)

Listen to the audio:

In [18]:
ipd.Audio(x, rate=fs)
Out[18]:

Signal Decomposition

Compute the spectrogram:

In [13]:
X = librosa.stft(x)

Save the magnitude and phase of the spectrogram for later use:

In [148]:
Xabs = numpy.absolute(X)
Xphase = numpy.angle(X)

Perform nonnegative matrix factorization on the signal:

In [405]:
n_components = 30
W, H = librosa.decompose.decompose(Xabs, n_components=n_components, sort=True)
print W.shape
print H.shape

(1025, 30)
(30, 296)

Signal Component Reconstruction

Let's create a function that reconstructs a time-domain signal from the NMF outputs:

In [406]:
def reconstruct_signal(w, h, Xphase, length=None):
    Y = scipy.outer(w, h)*numpy.exp(1j*Xphase)
    y = librosa.istft(Y)
    if length:
        y = librosa.util.fix_length(y, length)
    return y

Reconstruct each signal component:

In [407]:
y_signals = numpy.array([reconstruct_signal(W[:,n], H[n], Xphase, length=len(x)) for n in range(n_components)])

Listen to one of the signal components:

In [408]:
ipd.Audio(y_signals[29], rate=fs)
Out[408]:

Feature Extraction

Compute features for each signal component.

In [373]:
def normalize(h):
    return numpy.array(h)/scipy.linalg.norm(h)
In [409]:
window_sz = 20
hop_sz = 10
features = numpy.array([
        [H[n, i:i+window_sz].mean() for i in range(0, H.shape[1]-window_sz, hop_sz)]
        for n in range(n_components)        
])
In [410]:
features.shape
Out[410]:
(30, 28)
In [411]:
plt.plot(H[0])
Out[411]:
[<matplotlib.lines.Line2D at 0x114d4d810>]
In [412]:
plt.plot(features[0])
Out[412]:
[<matplotlib.lines.Line2D at 0x1149b0290>]

Clustering

In [413]:
model = sklearn.cluster.KMeans(n_clusters=3)
In [414]:
labels = model.fit_predict(features)
In [415]:
print labels

[2 2 0 0 1 1 1 0 2 0 0 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 0 0]
In [416]:
bass_drum = y_signals[labels==0, :].sum(axis=0)
In [417]:
ipd.Audio(bass_drum, rate=fs)
Out[417]:
In [418]:
snare_drum = y_signals[labels==2, :].sum(axis=0)
In [419]:
ipd.Audio(snare_drum, rate=fs)
Out[419]:
In [420]:
hi_hat = y_signals[labels==1, :].sum(axis=0)
In [421]:
ipd.Audio(hi_hat, rate=fs)
Out[421]:
In [3]:
import numpy as np

labels = np.empty(20, np.int32)
labels[0:9] = 1 # First 10 are the first sample type, e.g. snare
labels[10:20] = 2 # Second 10 are the second sample type, e.g kick

model_snare = KNeighborsClassifier(n_neighbors = 1)
model.fit(scaledTrainingFeatures, labels.take(train_index, 0))
model_output = model_snare.predict(scaledTestingFeatures)

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-3-914682621b17> in <module>()
      5 labels[10:20] = 2 # Second 10 are the second sample type, e.g kick
      6 
----> 7 model_snare = KNeighborsClassifier(n_neighbors = 1)
      8 model.fit(scaledTrainingFeatures, labels.take(train_index, 0))
      9 model_output = model_snare.predict(scaledTestingFeatures)

NameError: name 'KNeighborsClassifier' is not defined

  1. Extract features from the drum signals that you separated in Lab 4 Section 1.

  2. Classify them using the K-NN model that you built.

    Does K-NN accurately classify the separated signals?

  3. Repeat for different numbers of separated signals (i.e., the parameter K in NMF).

  4. Overseparate the signal using K = 20 or more. For those separated components that are classified as snare, add them together using `sum}. The listen to the sum signal. Is it coherent, i.e., does it sound like a single separated drum?

...and more!

  • If you have another idea that you would like to try out, please ask me!
  • Feel free to collaborate with a partner. Together, brainstorm your own problems, if you want!

Good luck!

← Back to Index