Typing keyboard + python = Musical instrument !

In this post I will show you how to do this :

I am not the first to do that, but most people who play their computer on Youtube use either very expensive programs, or programs that won’t run on your computer, or not-so-efficient programs with not-that-much possibilities of extension, or cheap programs with a big lag between pressing the key and actually hearing the note.

So here is a very small Python script which will run fine even on a basic netbook. If you are not familiar with Python, you should take online courses , it is really worth it :) !
If you are faminiliar with Python, then you are welcome to improve the code on its Github page.

Transforming your keyboard into a mixtable

My original idea was to transform my computer keyboard into a mixtable, to make something like in this very awesome video:

So my first move was to make a program that would take a configuration file my_configuration.cf containing this:


q, dog.wav
w, cat.wav
e, tarzan.wav
r, scream.mp3

And then if you hit q you would hear a dog from the dog.wav soundfile, if you hit w you’d hear a cat, etc…
This is pretty easy to do with Python’s pygame package. Here is my code (inspired by similar stuff from the package Mingus):


import pygame as pg
import csv
import time


SAMPLE_WIDTH = 16
FPS = 44100
N_CHANNELS = 2
BUFFER = 2**9
    
def minimix(config_file,mode = 'instrument'):
    """
    Opens an interface that lets you press the keys of your keyboard
    to plays the related soundfiles as sepcified in the provided
    configuration file.
    
    Args:
       config_file (str):  A file associating keyboard keys with
                           file names. Each line must be of the form
                           key , filename.
                       
       mode (str) :
            instrument -- causes autorepeat of samples as long
                            as the key is pressed.
            sustain -- causes autorepeat of the samples until its
                       key is pressed again.
            player -- striking the key causes the sound to play once. 
    
    Returns:
       a list of the (time,events).
    """
    
    repeat = 0 if (mode is 'player') else (-1)
    
    pg.mixer.pre_init(FPS,-SAMPLE_WIDTH,N_CHANNELS,BUFFER)
    pg.init()
    screen = pg.display.set_mode((640,480))
    
    
    
    
    ##### READ CONF FILE
    
    key2sound = {}
    key2file = {}
    config = csv.reader(open(config_file, 'rb'), delimiter=',')
    
    for key, soundfile in config:
        
        key,soundfile = key.strip(' '),soundfile.strip(' ')
        
        if key is not '#':
            
            key2file[key] = soundfile
            key2sound[key] = pg.mixer.Sound(soundfile)
    
    events_list = []
    currently_playing = {k : False for k in key2sound.iterkeys()}
    
    
    ##### MAIN LOOP
    
    while True:
        
        event =  pg.event.wait()
      
        if event.type in (pg.KEYDOWN,pg.KEYUP):
            key = pg.key.name(event.key)
              
            if key in key2sound:
                
                if event.type == pg.KEYDOWN:
                    
                    if (mode == 'sustain') and currently_playing[key]:
                        
                        key2sound[key].fadeout(20)
                        currently_playing[key] = False
                        
                    else:
                        
                        key2sound[key].play(repeat) 
                        currently_playing[key] = True
                    
                    events_list.append((time.time(),key2file[key]))
                    
                elif event.type == pg.KEYUP and (mode == 'instrument'):
                    
                    key2sound[key].stop()
                    currently_playing[key] = False
                    
                    events_list.append((time.time(),key2file[key]))
            
            elif event.key == pg.K_ESCAPE:
                
                break
    
    pg.quit()
    
    return events_list

Transforming your keyboard into a musical instrument

If instead of using various noises like cat and dog you use different notes from the same instrument, then you turned your computer into some kind of piano. The problem is that a set of soundfiles with all the notes of an instrument is difficult to find on the internet, so I wrote a script that makes as many notes as you want from just one sound by shifting its pitch up or down. It uses the audio processing program Soundstretch that you will need to install first :

import os

def shift_wav(wavfile,output,shifts,verbose=False):
    """
    Makes new sounds by shifting the pitch of a sound.
    Requires soundstrech installed.
    
    Args:
        wavfile : Name of the file containing the original sound
        output: name to use as a prefix for the output files and for
                the output folder name
        shifts (list of int): specifies of how many half-tones the pitch
                shifts should be. For instance [-2,-1,1,2] will produce
                4 files containing the sound 2 half-tones lower, one
                halftone lower, one halftone higher and two halftones
                higher.
    """
    
    folder = os.path.dirname(output)
        
    if not os.path.exists(folder):
        
        os.makedirs(folder)
        
    for i,s in enumerate(shifts):
        
        outputfile = '%s%02d.wav'%(output,i)
        
        command = 'soundstretch %s %s -pitch=%d'%(wavfile,outputfile,s)
        if verbose:
            print command
        os.system(command)

Going further

There is so much one could do to improve the program.

On the musical side, for instance, finding configurations of the keyboard that are particularly ergonomic. In the video above I used this configuration:

azerty

I called it typewriter because it enables you to play very fast things while moving your hands a minimum (the video is a bad example :P) . But maybe there is better to find !

Also, one could start listing every cool piece of music that can be played on a typing keyboard. I use 46 keys ( almost 4 octaves !), that makes a lot of possibilities !

On the programming side, there is a lot of little things I can think of, like automatizing scale changes, introducing nuances, designing nice interfaces (why not a guitar-hero-like game where you would actually be playing music on a playback ?), writing a script that would take some sheet music (in a nice format, like ABC, MIDI, lylipond) and return the list of the keys you should strike to play it.

I actually wrote a lot more code, for instance to make it easier to write configuration files, for sound processing, etc., but since it is not strictly necessary I am not reporting it here (I’ll certainly put a working version on GitHub, or such, some day).

Philosophy of the musical keyboard

Your typing keyboard is a real instrument. Of course it is not its primary use, but our voice’s primary purpose was not to sing, either. Now do the math : how many people out there have a piano at home ? And how many have a computer ? That gives you an idea of how many people would like to play the piano, cannot, but could play their computers instead.

So promoting computer-keyboardism is ultimately about bringing music to the masses. It is about providing everyone with an instrument that you can practice at home, in the train, at work, and that will be familiar to anyone everywhere in the world.

There is more : how many of you, pianist readers, have started the piano for seduction purposes ? (yeah, sure, me neither…) But public places with a piano on which you could show off your mad skills are getting pretty rare, aren’t they ? Especially since most bars have traded their good old piano for a TV. But think about all these places with a computer at hand ! Yep, time for you to develop a talent that will be useful in real life !

So practice, get good, be one of the first composers for tomorrow’s instrument, impress your friends and spread the good news ! If you are still reading me after so much gibberish , then do not hesitate : you just proved how little you value your free time, you are the right person for the task !

About these ads

15 comments on “Typing keyboard + python = Musical instrument !

  1. connoor says:

    I’m totally in it for the seduction

  2. Wow, that’s really cool! I’m impressed by your playing as well :)

  3. Ben says:

    So awesome. I’ve visited this site with a couple of different people and watched the video several times.

    Could you post the two piano .wav files that you created? I’ll happily do the pitch shifting!!

  4. Insanityandme says:

    Would love a more detailed guide to installing and getting everything fixed :)! I love the idea of playing piano on my computer.

    I’m having trouble with soundstretch at the moment, I can’t install soundtouch because a exec format error when launching ./bootstrap to create a configure file etc.

  5. Insanityandme says:

    Traceback (most recent call last):
    File “example_piano-kb.py”, line 4, in
    from minimix.quick import *
    File “/Library/Python/2.7/site-packages/minimix/quick.py”, line 7, in
    from sound import *
    File “/Library/Python/2.7/site-packages/minimix/sound.py”, line 6, in
    import scipy.io.wavfile as wf
    ImportError: No module named scipy.io.wavfile

    Do you have any idea what this means? I’m a python/terminal newbie in general and would love some help, google isn’t really helping me at this moment…

    • Valentin says:

      Thanks for the feedback, I definitely need to write a proper installer for this one.

      This means that you must install the python package named Scipy. You can either do that OR you can download the latest version on the Github page, I just made the dependency to Scipy optional (meaning the program will not stop and will function normally if you don’t have scipy.

      • Insanityandme says:

        Out of curiousity, if it is optional and still functions without it why is it “nescessary”?

        Thanks a lot btw, I’m thinking of learning python and adding onto your repo, I’ve got knowledge in programming in general but not python! I do like it :)

      • Valentin says:

        Welcome in the world of Python, then :)
        Scipy (and more specifically its audio processing module) is necessary for a very particuliar function, “split_wav”, that will split a single wav file containing many samples into individual little wav files containing one sample each. This way you just take a micro and record in one take “A”,”B”,”C”, “pom”, “tss”, “dring”, etc. , you run the function on the file you obtained, and that gets you a new soundfont :). That is what I used before I found a way to shift the pitch of the sounds.

  6. Insanityandme says:

    So you’re saying you found out soundstretch existed but you still made it optional so you can use scipy instead? :D Sorry if I seem light headed or narrow minded… :D

  7. Insanityandme says:

    I have one issue left at this point, it seems that soundtouch (the library which soundstrech is included) is not able to be compiled and installed into my system. It complains with this sort of error:

    ./bootstrap: /usr/local/bin/autoreconf: /opt/local/bin/perl: bad interpreter: No such file or directory

    Any idea why?

    Thanks for any feedback!

  8. pKrime says:

    Hi, thank you for the music, man! :) I surfed the internet the whole day looking for a good solution to play notes in python and finally came to this.
    Just two things:
    – the scripts use unix paths, I changed all the path concatenations to os.join and it worked like a charm on windows. Neverthless you’re using os.path in some lines, is there a reason why you sticked to string concatenation?
    – I replaced a dict comprehension to make it python 2.6 compatible.

    Of course I’m willing to give my diff back if you’re interested, it’s little thing anyway.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s