In [ ]:

    import numpy, scipy, matplotlib.pyplot as plt, sklearn, librosa, mir_eval, IPython.display, urllib

← Back to Index

# Unsupervised Instrument Classification Using K-Means¶

This lab is loosely based on Lab 3 (2010).

## Read Audio¶

Retrieve an audio file, load it into an array, and listen to it.

In [ ]:

    urllib.urlretrieve?

In [ ]:

    librosa.load?

In [ ]:

    IPython.display.Audio?

## Detect Onsets¶

Detect onsets in the audio signal:

In [ ]:

    librosa.onset.onset_detect?

Convert the onsets from units of frames to seconds (and samples):

In [ ]:

    librosa.frames_to_time?

In [ ]:

    librosa.frames_to_samples?

Listen to detected onsets:

In [ ]:

    mir_eval.sonify.clicks?

In [ ]:

    IPython.display.Audio?

## Extract Features¶

Extract a set of features from the audio at each onset. Use any of the features we have learned so far: zero crossing rate, spectral moments, MFCCs, chroma, etc. For more, see the librosa API reference.

First, define which features to extract:

In [ ]:

    def extract_features(x, fs):
        feature_1 = librosa.zero_crossings(x).sum() # placeholder
        feature_2 = 0 # placeholder
        return [feature_1, feature_2]

For each onset, extract a feature vector from the signal:

In [ ]:

    # Assumptions:
    # x: input audio signal
    # fs: sampling frequency
    # onset_samples: onsets in units of samples
    frame_sz = fs*0.100
    features = numpy.array([extract_features(x[i:i+frame_sz], fs) for i in onset_samples])

## Scale Features¶

Use `sklearn.preprocessing.MinMaxScaler` to scale your features to be within `[-1, 1]`.

In [ ]:

    sklearn.preprocessing.MinMaxScaler?

In [ ]:

    sklearn.preprocessing.MinMaxScaler.fit_transform?

## Plot Features¶

Use `scatter` to plot features on a 2-D plane. (Choose two features at a time.)

In [ ]:

    plt.scatter?

## Cluster Using K-Means¶

Use `KMeans` to cluster your features and compute labels.

In [ ]:

    sklearn.cluster.KMeans?

In [ ]:

    sklearn.cluster.KMeans.fit_predict?

## Plot Features by Class Label¶

Use `scatter`, but this time choose a different marker color (or type) for each class.

In [ ]:

    plt.scatter?

## Listen to Click Track¶

Create a beep for each onset within a class:

In [ ]:

    beeps = mir_eval.sonify.clicks(onset_times[labels==0], fs, length=len(x))

In [ ]:

    IPython.display.Audio?

## Listen to Clustered Frames¶

Use the `concatenated_segments` function from the feature sonification exercise to concatenate frames from the same cluster into one signal. Then listen to the signal.

In [ ]:

    def concatenate_segments(segments, fs=44100, pad_time=0.300):
        padded_segments = [numpy.concatenate([segment, numpy.zeros(int(pad_time*fs))]) for segment in segments]
        return numpy.concatenate(padded_segments)
    concatenated_signal = concatenate_segments(segments, fs)

Compare across separate classes. What do you hear?

## For Further Exploration¶

Use a different number of clusters in `KMeans`.

Use a different initialization method in `KMeans`.

Use different features. Compare tonal features against timbral features.

In [ ]:

    librosa.feature?

Use different audio files.

In [ ]:

    #filename = '1_bar_funk_groove.mp3'
    #filename = '58bpm.wav'
    #filename = '125_bounce.wav'
    #filename = 'prelude_cmaj_10s.wav'

← Back to Index