How pitch shifting works

You record one good note and map it across the keyboard. Near the recording it sounds right. Further away it does not: low notes come out slow and heavy, high notes thin and short. This page covers why that happens, and how many notes you have to record to avoid it. The panels play audio.

Faster means higher

A sampler gets a higher note out of a recording by playing it faster, the same mechanism as a record played at the wrong speed. Play it at twice the rate and every cycle of the waveform arrives twice as often, so the pitch doubles.

Playing faster moves everything, not only the pitch. The length comes down by the same proportion: an octave up is half as long, an octave down is twice as long and twice as dull, with the detail that made the attack sound like a struck bar now spread over twice the time. Panel 1 plays a glockenspiel strike at whatever interval you set.

1. Transposing one recording. One strike of a real glockenspiel, played at other pitches the way a sampler does it: faster for higher notes, slower for lower ones. The dotted line marks where the recording originally ended, so a note transposed up stops short of it and one transposed down runs past. Move it a long way up and the bar appears to shrink; a long way down and it turns into something the size of a church bell.

Keeping the length

Moving the pitch without moving the length takes more work. The usual approach is to find the point where the waveform repeats, cut the recording into those repeating cycles, and then lay the cycles back down at a different spacing. Closer together and the waveform repeats more often, which is a higher note. Further apart and it is a lower one. You choose how many cycles to lay down, so the length stays where you put it.

The cost is texture. A rebuilt sound is not a replayed one, and on anything with a sharp attack or a noisy character the seams are audible as a flutter and a smearing of the attack. That cost is why samplers transpose by rate instead of pitch shifting every note, and why the pitch shifted setting in panel 2 sounds rough. Those artefacts come from the method rather than from an implementation a better plugin would improve on.

What pitch shifting gives you is the original length. It does not make a transposed sample sound like the real note, for the reason in the next section, and most pitch shifters work the same way.

2. Two ways to move pitch. The same note moved by the same interval, two different ways. The dotted line marks where the original recording ended. Resampling plays it faster, so it stops short of the line. Pitch shifting rebuilds it cycle by cycle and reaches the line exactly. Switch between them while it plays: the second holds its length and loses some texture doing it.

Why a transposed note sounds like a different instrument

Every sound has resonances that stay at the same frequency whatever note is played. In a voice they come from the size of the throat and mouth; in a guitar from the body, in a drum from the shell. They are called formants, and they are a large part of how you recognise what you are listening to. A singer moving from a low note to a high one keeps their formants where they are, because their head is still the same size.

Change the playback rate and the formants move with the pitch, which tells your ear the thing making the sound has changed size. Up an octave and the singer appears to have shrunk. That is the chipmunk effect, and it is why a piano sample transposed two octaves sounds like a toy piano rather than a piano playing high.

Pitch shifting does not fix this. Holding formants still while the pitch moves needs the spectral envelope measured and put back, which is a larger piece of machinery than either method here.

Panel 3 uses a sung vowel, since a voice shows formants more plainly than most instruments. It compares one recording transposed up a fifth against the same singer singing that note. Transposing moves the first formant from 770 Hz to 1018 Hz. Pitch shifting moves it to 1017 Hz. Singing the note leaves it at 770 Hz, which is the version the other two are imitating.

3. Formants. A sung vowel, and its spectrum. The humps are formants, resonances of the singer's throat and mouth, and they sit at fixed frequencies whatever note is sung. Compare the same note two ways: transposed up from the recording, and sung at that pitch in the first place. The transposed one has dragged its formants up with it, which is why it sounds like a smaller singer rather than the same one.

So how many recordings?

If transposing a long way sounds wrong, the fix is to transpose a shorter distance, which means recording more notes. A multisample instrument is exactly that: several recordings, each covering the few notes nearest to it.

How many depends on how far you can transpose before anyone notices, and that depends on the instrument. Smooth sustained sounds take more stretching than percussive ones. Anything with strong formants, voices above all, takes less. Three or four semitones is a safe maximum for most things, which works out at three or four recordings per octave.

4. How many recordings. A real glockenspiel, sampled at seven notes across two octaves, playing a scale from however many of those recordings you allow it. Each key is coloured by how far it had to be transposed to get there: pale is close to a recording, deep is a long way from one. With one sample the bars appear to change size as the scale climbs.

In practice

  • Record every third or fourth note for a first pass. If a zone sounds wrong at its edges, record a note in the middle of it rather than re-recording everything.
  • Put the root note where the zone is centred, so a zone stretches as far up as it does down. A zone that only ever transposes upward wastes half of its tolerance.
  • Give voices and anything with a strong body resonance more recordings. The formant problem gets audible within a couple of semitones on a voice and takes most of an octave on a soft pad.
  • Watch the memory. On hardware with a fixed sample budget, four recordings an octave across five octaves is twenty files before velocity layers. A few well-placed recordings do more than many badly placed ones.

Doing this in SampleStack

Drop a folder of recorded notes into a multisample collection and SampleStack detects each one's pitch, assigns it a root note, and builds the zones across the keyboard. You can see and change where every zone starts and ends, and play the result before exporting it to SFZ, Decent Sampler, Bitwig, Ableton, the Deluge and the rest. The guide to looping the sustains you record is here.

Questions

Why does my sample sound wrong when I play it high up the keyboard?
Because a sampler reaches a higher note by playing the recording faster, which moves everything in it, not only the pitch. The note gets shorter in the same proportion, and the resonances that make an instrument sound the size it is move up too. A couple of semitones is unnoticeable. An octave is usually obvious.
What is the difference between transposing and pitch shifting?
Transposing by changing playback rate moves pitch and length together, the way a record played at the wrong speed does. Pitch shifting keeps the length by cutting the recording into its repeating cycles and laying them down at a new spacing. Pitch shifting costs you some texture, since it is rebuilding the sound rather than replaying it, but it leaves the timing alone.
What are formants and why do they matter?
Formants are resonant peaks that sit at fixed frequencies regardless of the note being played. They come from the size and shape of the thing making the sound, a singer's throat or an instrument's body. When you change playback rate the formants move with the pitch, and that is what makes a sped-up voice sound like a smaller person rather than the same person singing higher.
How many notes do I need to record for a multisample instrument?
One every three or four semitones covers most instruments without the transposing being audible, which is three or four recordings per octave. Every octave is noticeably stretched at the edges of each zone. Chromatic is overkill for most things and costs you memory on hardware. Percussive and heavily resonant sounds need more recordings than smooth ones.

Written