Skip to content

M
musicinformationretrieval-com
Commit a96f4a15 authored by Steve Tjoa's avatar Steve Tjoa
Browse files
Options

initial genre recognition notebooks

parent 5f697ce0
Show whitespace changes
Inline Side-by-side
Showing with 28064 additions and 0 deletions
exercise_genre_recognition.html 0 → 100644
View file @ a96f4a15
This source diff could not be displayed because it is too large. You can view the blob instead.
exercise_genre_recognition.ipynb 0 → 100644
View file @ a96f4a15
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import seaborn\n",
"import numpy, scipy, matplotlib.pyplot as plt, sklearn, pandas, librosa, urllib, IPython.display, os.path\n",
"plt.rcParams['figure.figsize'] = (14,5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Homework Part 2: Genre Classification"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Goals"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. Extract features from an audio signal.\n",
"2. Train a genre classifier.\n",
"3. Use the classifier to classify genre in a song."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 1: Retrieve Audio"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Download an audio file onto your local machine."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"filename1 = 'brahms_hungarian_dance_5.mp3'\n",
"url = \"http://audio.musicinformationretrieval.com/\" + filename1\n",
"if not os.path.exists(filename1):\n",
" urllib.urlretrieve(url, filename=filename1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Load 120 seconds of an audio file:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"librosa.load?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"x1, fs1 = librosa.load(filename1, duration=120)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Plot the time-domain waveform of the audio signal:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"plt.plot?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Your code here:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Play the audio file:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"IPython.display.Audio?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Your code here:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 2: Extract Features"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For each segment, compute the MFCCs. Experiment with `n_mfcc` to select a different number of coefficients, e.g. 12."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"librosa.feature.mfcc?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"mfcc1 = librosa.feature.mfcc(x1, sr=fs1, n_mfcc=12).T"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mfcc1.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Scale the features to have zero mean and unit variance:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"scaler = sklearn.preprocessing.StandardScaler()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mfcc1_scaled = scaler.fit_transform(mfcc1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Verify that the scaling worked:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mfcc1_scaled.mean(axis=0)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mfcc1_scaled.std(axis=0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Repeat steps 1 and 2 for another audio file:**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"filename2 = 'busta_rhymes_hits_for_days.mp3'\n",
"url = \"http://audio.musicinformationretrieval.com/\" + filename2"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"urllib.urlretrieve?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Your code here. Download the second audio file in the same manner as the first audio file above."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Load 120 seconds of an audio file:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"librosa.load?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Your code here. Load the second audio file in the same manner as the first audio file."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Listen to the second audio file."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"IPython.display.Audio?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Plot the time-domain waveform and spectrogram of the second audio file. In what ways does the time-domain waveform look different than the first audio file? What differences in musical attributes might this reflect? What additional insights are gained from plotting the spectrogram? **Explain.**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plt.plot?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# See http://musicinformationretrieval.com/stft.html for more details on displaying spectrograms.\n",
"librosa.feature.melspectrogram?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"librosa.logamplitude?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"librosa.display.specshow?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**[Please share your answer in this editable text cell.]**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Extract MFCCs from the second audio file. Be sure to transpose the resulting matrix such that each row is one observation, i.e. one set of MFCCs. Also be sure that the shape and size of the resulting MFCC matrix is equivalent to that for the first audio file."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"librosa.feature.mfcc?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mfcc2.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Scale the resulting MFCC features to have approximately zero mean and unit variance. Re-use the scaler from above."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"scaler.transform?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Verify that the mean of the MFCCs for the second audio file is approximately equal to zero and the variance is approximately equal to one."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mfcc2_scaled.mean?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mfcc2_scaled.std?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 3: Train a Classifier"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Concatenate all of the scaled feature vectors into one feature table."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"features = numpy.vstack((mfcc1_scaled, mfcc2_scaled))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"features.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Construct a vector of ground-truth labels, where 0 refers to the first audio file, and 1 refers to the second audio file."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"labels = numpy.concatenate((numpy.zeros(len(mfcc1_scaled)), numpy.ones(len(mfcc2_scaled))))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a classifer model object:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Support Vector Machine\n",
"model = sklearn.svm.SVC()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Train the classifier:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"model.fit?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Your code here"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 4: Run the Classifier"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To test the classifier, we will extract an unused 10-second segment from the earlier audio fields as test excerpts:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"x1_test, fs1 = librosa.load(filename1, duration=10, offset=120)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"x2_test, fs2 = librosa.load(filename2, duration=10, offset=120)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Listen to both of the test audio excerpts:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"IPython.display.Audio?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"IPython.display.Audio?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Compute MFCCs from both of the test audio excerpts:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"librosa.feature.mfcc?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"librosa.feature.mfcc?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Scale the MFCCs using the previous scaler:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"scaler.transform?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"scaler.transform?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Concatenate all test features together:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"numpy.vstack?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Concatenate all test labels together:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"numpy.concatenate?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Compute the predicted labels:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"model.predict?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, compute the accuracy score of the classifier on the test data:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"score = model.score(test_features, test_labels)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"score"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Currently, the classifier returns one prediction for every MFCC vector in the test audio signal. Can you modify the procedure above such that the classifier returns a single prediction for a 10-second excerpt?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Your code here."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**[Explain your approach in this editable text cell.]**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 5: Analysis in Pandas"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Read the MFCC features from the first test audio excerpt into a data frame:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"df1 = pandas.DataFrame(mfcc1_test_scaled)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df1.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df1.head()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"df2 = pandas.DataFrame(mfcc2_test_scaled)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Compute the pairwise correlation of every pair of 12 MFCCs against one another for both test audio excerpts. For each audio excerpt, which pair of MFCCs are the most correlated? least correlated?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df1.corr()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df2.corr()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**[Explain your answer in this editable text cell.]**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Display a scatter plot of any two of the MFCC dimensions (i.e. columns of the data frame) against one another. Try for multiple pairs of MFCC dimensions."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df1.plot.scatter?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Display a scatter plot of any two of the MFCC dimensions (i.e. columns of the data frame) against one another. Try for multiple pairs of MFCC dimensions."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df2.plot.scatter?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Plot a histogram of all values across a single MFCC, i.e. MFCC coefficient number. Repeat for a few different MFCC numbers:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df1[0].plot.hist()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df1[11].plot.hist()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Extra Credit"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a new genre classifier by repeating the steps above, but this time use training data and test data from your own audio collection representing two or more different genres. For what genres and audio data styles does the classifier work well, and for which (pairs of) genres does the classifier fail?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a new genre classifier by repeating the steps above, but this time use a different machine learning classifier, e.g. random forest, Gaussian mixture model, Naive Bayes, k-nearest neighbor, etc. Adjust the parameters. How well do they perform?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a new genre classifier by repeating the steps above, but this time use different features. Consult the [librosa documentation on feature extraction](http://librosa.github.io/librosa/feature.html) for different choices of features. Which features work well? not well?"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.10"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
genre_recognition.html 0 → 100644
View file @ a96f4a15
This source diff could not be displayed because it is too large. You can view the blob instead.
genre_recognition.ipynb 0 → 100644
View file @ a96f4a15
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import seaborn\n",
"import numpy, scipy, matplotlib.pyplot as plt, sklearn, pandas, librosa, urllib, IPython.display, os.path\n",
"plt.rcParams['figure.figsize'] = (14,5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Homework Part 2: Genre Classification"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Goals"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. Extract features from an audio signal.\n",
"2. Train a genre classifier.\n",
"3. Use the classifier to classify genre in a song."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 1: Retrieve Audio"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Download an audio file onto your local machine."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"filename1 = 'brahms_hungarian_dance_5.mp3'\n",
"url = \"http://audio.musicinformationretrieval.com/\" + filename1\n",
"if not os.path.exists(filename1):\n",
" urllib.urlretrieve(url, filename=filename1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Load 120 seconds of an audio file:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"librosa.load?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"x1, fs1 = librosa.load(filename1, duration=120)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Plot the time-domain waveform of the audio signal:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"plt.plot?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Your code here:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Play the audio file:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"IPython.display.Audio?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Your code here:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 2: Extract Features"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For each segment, compute the MFCCs. Experiment with `n_mfcc` to select a different number of coefficients, e.g. 12."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"librosa.feature.mfcc?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"mfcc1 = librosa.feature.mfcc(x1, sr=fs1, n_mfcc=12).T"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mfcc1.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Scale the features to have zero mean and unit variance:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"scaler = sklearn.preprocessing.StandardScaler()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mfcc1_scaled = scaler.fit_transform(mfcc1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Verify that the scaling worked:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mfcc1_scaled.mean(axis=0)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mfcc1_scaled.std(axis=0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Repeat steps 1 and 2 for another audio file:**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"filename2 = 'busta_rhymes_hits_for_days.mp3'\n",
"url = \"http://audio.musicinformationretrieval.com/\" + filename2"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"urllib.urlretrieve?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Your code here. Download the second audio file in the same manner as the first audio file above."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Load 120 seconds of an audio file:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"librosa.load?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Your code here. Load the second audio file in the same manner as the first audio file."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Listen to the second audio file."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"IPython.display.Audio?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Plot the time-domain waveform and spectrogram of the second audio file. In what ways does the time-domain waveform look different than the first audio file? What differences in musical attributes might this reflect? What additional insights are gained from plotting the spectrogram? **Explain.**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plt.plot?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# See http://musicinformationretrieval.com/stft.html for more details on displaying spectrograms.\n",
"librosa.feature.melspectrogram?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"librosa.logamplitude?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"librosa.display.specshow?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**[Please share your answer in this editable text cell.]**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Extract MFCCs from the second audio file. Be sure to transpose the resulting matrix such that each row is one observation, i.e. one set of MFCCs. Also be sure that the shape and size of the resulting MFCC matrix is equivalent to that for the first audio file."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"librosa.feature.mfcc?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mfcc2.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Scale the resulting MFCC features to have approximately zero mean and unit variance. Re-use the scaler from above."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"scaler.transform?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Verify that the mean of the MFCCs for the second audio file is approximately equal to zero and the variance is approximately equal to one."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mfcc2_scaled.mean?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"mfcc2_scaled.std?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 3: Train a Classifier"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Concatenate all of the scaled feature vectors into one feature table."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"features = numpy.vstack((mfcc1_scaled, mfcc2_scaled))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"features.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Construct a vector of ground-truth labels, where 0 refers to the first audio file, and 1 refers to the second audio file."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"labels = numpy.concatenate((numpy.zeros(len(mfcc1_scaled)), numpy.ones(len(mfcc2_scaled))))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a classifer model object:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Support Vector Machine\n",
"model = sklearn.svm.SVC()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Train the classifier:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"model.fit?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Your code here"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 4: Run the Classifier"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To test the classifier, we will extract an unused 10-second segment from the earlier audio fields as test excerpts:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"x1_test, fs1 = librosa.load(filename1, duration=10, offset=120)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"x2_test, fs2 = librosa.load(filename2, duration=10, offset=120)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Listen to both of the test audio excerpts:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"IPython.display.Audio?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"IPython.display.Audio?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Compute MFCCs from both of the test audio excerpts:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"librosa.feature.mfcc?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"librosa.feature.mfcc?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Scale the MFCCs using the previous scaler:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"scaler.transform?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"scaler.transform?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Concatenate all test features together:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"numpy.vstack?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Concatenate all test labels together:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"numpy.concatenate?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Compute the predicted labels:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"model.predict?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, compute the accuracy score of the classifier on the test data:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"score = model.score(test_features, test_labels)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"score"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Currently, the classifier returns one prediction for every MFCC vector in the test audio signal. Can you modify the procedure above such that the classifier returns a single prediction for a 10-second excerpt?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Your code here."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**[Explain your approach in this editable text cell.]**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 5: Analysis in Pandas"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Read the MFCC features from the first test audio excerpt into a data frame:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"df1 = pandas.DataFrame(mfcc1_test_scaled)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df1.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df1.head()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"df2 = pandas.DataFrame(mfcc2_test_scaled)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Compute the pairwise correlation of every pair of 12 MFCCs against one another for both test audio excerpts. For each audio excerpt, which pair of MFCCs are the most correlated? least correlated?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df1.corr()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df2.corr()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**[Explain your answer in this editable text cell.]**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Display a scatter plot of any two of the MFCC dimensions (i.e. columns of the data frame) against one another. Try for multiple pairs of MFCC dimensions."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df1.plot.scatter?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Display a scatter plot of any two of the MFCC dimensions (i.e. columns of the data frame) against one another. Try for multiple pairs of MFCC dimensions."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df2.plot.scatter?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Plot a histogram of all values across a single MFCC, i.e. MFCC coefficient number. Repeat for a few different MFCC numbers:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df1[0].plot.hist()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"df1[11].plot.hist()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Extra Credit"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a new genre classifier by repeating the steps above, but this time use training data and test data from your own audio collection representing two or more different genres. For what genres and audio data styles does the classifier work well, and for which (pairs of) genres does the classifier fail?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a new genre classifier by repeating the steps above, but this time use a different machine learning classifier, e.g. random forest, Gaussian mixture model, Naive Bayes, k-nearest neighbor, etc. Adjust the parameters. How well do they perform?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a new genre classifier by repeating the steps above, but this time use different features. Consult the [librosa documentation on feature extraction](http://librosa.github.io/librosa/feature.html) for different choices of features. Which features work well? not well?"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.10"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment