After a week of project making, presenting, and paper writing, as well as four days of being sick, I think I'm ready to proceed with the next activity. There are three activities left, so this would be the third to the last activity.
In this activity, we will be playing notes based on a sheet music image. I tried to think of a good sheet music to use that wasn't too complicated, and I remembered the song we used to play for Music class in high school with our recorders, "Ode to Joy" by Ludwig van Beethoven. Or at least the main melody of it. The song can be summed up by this sheet music from Music-for-Music-Teachers.com:
 |
| Fig. 1. Sheet music to be used in this activity, |
In the sheet music, there are quarter notes and half notes. The morphological operation I will need should result in blobs that distinguish between the two. But first, I will chop up the entire sheet music image into four parts. Take the first part, here:
 |
| Fig. 2. First part of the sheet music. |
I plan on placing in a single array the entire melody for this first part. Thus, I initialize the array as an empty array S. I then use the reference given in our manual (http://www.phy.mtu.edu/~suits/notefreqs.html) to get the corresponding frequencies for the notes covered by the entire song.
C = 261.63*2;
D = 293.66*2;
E = 329.63*2;
F = 349.23*2;
G = 392.00*2;
A = 440.00*2;
B = 493.88*2;
And with the code snippet also from the manual, we use the function that generates sound waves based on the note and the duration.
function n = note(f, t)
n = sin (2*%pi*f*t);
endfunction;
For the values of t, the sheet music does not specify the tempo or beats per minute of the song. Thus, I will arbitrarily use 240 beats per minute, which corresponds to 0.25 seconds for the quarter note and 0.50 seconds for the half note.
t=soundsec(0.25);
t2=soundsec(0.50);
We now use the image in Fig. 2 to read the sheet music via image processing. After reading using imread() and converting to Grayscale from RGB using rgbtogray(), the image is inverted by subtracting each grayscale pixel value from 255. The image is then binarized by setting all grayscale values greater than zero as True and those that are zero as False. The resulting image is Fig. 3 below:
 |
| Fig. 3. Result from inversion and binarization of the image in Fig. 2. |
A rectangular structuring element with 1 pixel width and 2 pixels height, which looks like a short vertical line, is then used to erode the image. This causes the horizontal lines to disappear.
 |
| Fig. 4. Result from erosion using the vertical line structuring element. |
Another rectangular structuring element, this time with 2 pixels width and 1 pixel height or a short horizontal line, is used to erode the image in Fig. 4. This causes the vertical lines to disappear.
 |
| Fig. 5. Result from erosion using the horizontal line structuring element. |
Notice that the half note blob looks like it's separated into two curved lines instead of one hollow oval. Also, it would be best to make the stray parts of the G-clef and F-clef that look like a note be attached to the G-clef and F-clef blobs. Thus, I used a rectangular structuring element with 4 pixels width and 1 pixel height, which is a slightly longer horizontal line than the previous structuring element, to join the half note blob parts into one hollow oval, as well as the stray blobs of the G-clef and F-clef to the main G-clef and F-clef blobs.
 |
| Fig. 6. Result from dilation using the slightly longer horizontal line structuring element. |
Now we have a workable image. We use SearchBlobs() to assign a number to each of the blobs in the image, and the area of each blob is checked one by one using size(). I found that pixel areas greater than 40 and less than 45 correspond to the half note blobs, and that pixel areas greater than 45 and less than 60 correspond to the quarter note blobs. Thus, we can filter those blobs out and put them in their own images:
 |
| Fig. 7. Quarter note blobs from the first part of the music sheet. |
 |
| Fig. 8. Half note blobs from the first part of the music sheet. |
I then use SearchBlobs() twice: on the image containing quarter notes, and on the image containing half notes. Using AnalyzeBlobs() to get the centroids of each blob, I then obtained the y-axis location (corresponding to which note they are played at). I then used MS Paint to manually check the y-axis pixel ranges in Fig. 2 that correspond to which notes are played. I found these to be:
for i = 1:max(BW1)
if S5(i).Centroid(2) >= 50 & S5(i).Centroid(2) < 53
L2 = note(C,t);
elseif S5(i).Centroid(2) >= 47 & S5(i).Centroid(2) < 50
L2 = note(D,t);
elseif S5(i).Centroid(2) >= 44 & S5(i).Centroid(2) < 47
L2 = note(E,t);
elseif S5(i).Centroid(2) >= 41 & S5(i).Centroid(2) < 44
L2 = note(F,t);
elseif S5(i).Centroid(2) >= 38 & S5(i).Centroid(2) < 41
L2 = note(G,t);
elseif S5(i).Centroid(2) >= 35 & S5(i).Centroid(2) < 38
L2 = note(A,t);
elseif S5(i).Centroid(2) >= 32 & S5(i).Centroid(2) < 35
L2 = note(B,t);
else
L2 = note(E,t); //this would sound incorrect
end
S = cat(2,S,L2);
end
Looping across each numbered blob and comparing the y-axis location with which range it falls on, I then place it in an array L2 containing that note. The S array then compiles the arrays via concatenation. The x-axis location of the blob should also be usable to obtain the order in which the notes are played, but since the half note is always at the end in this particular song, I simply performed the blob scanning on the half notes after the quarter notes, attaching the half note at the end of the array S through concatenation.
I then used the sound() function to listen to the resulting S array, and the writewav() function to save it into a .wav file.
The other parts were made sure to have the same y-axis coordinate positions for the horizontal lines of the sheet music. Using the exact same code but changing the filename to the other parts, these were the results:
That ends Activity 9. This was difficult to do since I am still a bit sick. However, I'm glad I was able to do it.
Self-Evaluation: 9/10
No comments:
Post a Comment