Pocket Synth: Building a Playable Synthesizer With the Web Audio API

Play the Pocket Synth

Press Play on that page and a sixteen-step pattern starts. Now drag the Cutoff knob upward while it runs. The riff goes from muffled to bright, the screen swaps its oscilloscope for a drawing of the filter you are moving, and the pads light up in time with the notes.

There is no audio file behind any of it and no synth library. The browser generates the sound sample by sample from a graph of audio nodes, and a 25 millisecond timer keeps it in time. This post walks through how, including the two ways it managed to be silent on a phone.

The Signal Chain

Pocket Synth is a subtractive synth, the design most classic analog synthesizers share: start with a waveform that has more harmonics than you want, take some away with a filter, and shape the loudness over time.

per note:  osc ─┐
           osc ─┴─> low-pass filter ─> gain (envelope) ─┬─> bus
              ▲         ▲ filter envelope + LFO          └─> expression send ─┐
              pitch bend (a finger moving on the pad)                         │
                                                                              ▼
bus ─┬──────────────────────────────────────> master ─> limiter ─> soft clip ─> speakers
     ├─> delay ⟲ ─> echo gain ────────────────────┤
     └─> reverb send ─> convolver ────────────────┘

In Web Audio, each box is a node and each arrow is a connect() call. The top row is built fresh for every note. Everything below it is built once.

Two Oscillators per Note

An OscillatorNode produces one of four basic waves. They differ in which harmonics they contain, and that is the whole reason they sound different:

Wave	Harmonics	Sounds like
Sine	The fundamental only	A whistle, a tuning fork
Triangle	Odd ones, fading fast	A soft flute
Saw	All of them	Brass, strings, buzz
Square	Odd ones, fading slowly	Hollow, like a clarinet

A sine has nothing for the filter to remove, so the cutoff knob barely does anything to it. Saw and square are where a subtractive synth earns its name.

Each note gets two oscillators tuned a few cents apart, one flat and one sharp. Two nearly identical waves drift in and out of phase, and that slow shimmer is most of what people mean by a "fat" synth sound. The Detune knob sets how far apart they are; at zero the synth uses a single oscillator.

const midiToHz = (midi: number) => 440 * Math.pow(2, (midi - 69) / 12)

const tunings = patch.detune > 0 ? [-patch.detune, patch.detune] : [0]
const oscs = tunings.map((cents) => {
  const osc = ctx.createOscillator()
  osc.type = patch.wave
  osc.frequency.value = midiToHz(midi)
  osc.detune.value = cents
  return osc
})

Note 69 is the A above middle C at 440 Hz, and every semitone multiplies the frequency by the twelfth root of two. Oscillators are single-use: once stopped, one can never be started again, so every key press builds new ones.

The Envelope Is What Makes It an Instrument

Connect an oscillator straight to the speakers and you get a click, a drone and another click. What turns a drone into a pluck or a pad is the envelope: a level that is scheduled over time. Attack, decay, sustain and release are four numbers describing that curve.

const shape = (param: AudioParam, peak: number) => {
  param.setValueAtTime(0, time)
  // Attack: a straight ramp up to the peak
  param.linearRampToValueAtTime(peak, time + attack)
  // Decay: ease down to the sustain level and stay there while the key is held
  param.setTargetAtTime(peak * sustain, time + attack, decay / 3)
}

shape(amp.gain, VOICE_PEAK) // loudness
shape(filter.detune, envCents) // brightness

None of this runs on a JavaScript timer. The calls hand the audio thread a schedule and it follows that schedule with sample accuracy, even if the page is busy rendering.

The same shape is used twice. On the gain it makes the note louder and then quieter. On the filter it makes the note brighter and then darker, which is the "pew" at the front of a synth bass. The Env knob sets how far the filter is thrown open, up to four octaves above the cutoff.

Release starts when the key comes up, from wherever the envelope has got to, which may be halfway up the attack:

param.cancelAndHoldAtTime(time) // freeze at the level it has then
param.setTargetAtTime(0, time, release / 5)

setTargetAtTime approaches its target exponentially and never quite arrives. After five time constants it is within 1% of it, so the oscillators are stopped a little after that, when the cut is too quiet to click.

The four presets are mostly envelope settings. Pluck is a 3 ms attack with sustain at 25%. Pad is the same saw wave with a 500 ms attack and a 1.8 second release.

A Filter per Note

The low-pass filter is a BiquadFilterNode. Cutoff sets where it starts removing harmonics; resonance boosts the band right at the cutoff, which is the nasal, vowel-like edge on the Bass preset.

My first version had one filter that every note fed into. That is cheaper, and with a filter that never moves it sounds identical, because a filter is linear: filtering the sum of the notes gives the same signal as filtering each note and adding them up. It stopped being an option the moment the filter got its own envelope. A note struck now should be bright while a note held for a second has already gone dark, and one filter cannot be in both places.

So each note has its own filter, and three things move it at once:

The cutoff knob sets filter.frequency.
The filter envelope is scheduled on filter.detune, in cents above the cutoff.
The LFO, a single slow oscillator shared by every note, is connected into filter.detune as well.

An AudioParam adds together its scheduled value and anything connected to it, so the envelope and the LFO stack without any code to combine them. Working in cents rather than hertz means a sweep is the same musical distance wherever the cutoff sits.

The cutoff knob is not linear either. Pitch perception is logarithmic, so a knob that moved in equal steps of hertz would cram everything interesting into its first tenth:

const cutoffHz = (cutoff: number) => 60 * Math.pow(12000 / 60, cutoff)

Echo and Reverb

The echo is a DelayNode whose output goes through a gain of 0.42 and back into its own input, so each repeat is quieter than the last. Web Audio allows a cycle in the graph as long as there is a delay in it.

bus.connect(delay)
delay.connect(feedback)
feedback.connect(delay) // the loop
delay.connect(echo)

The delay time is three sequencer steps, a dotted eighth note, so the repeats land between the notes of a pattern instead of on top of them. Turn the Tempo knob and the delay time glides to its new length. A delay line that changes length bends the pitch of whatever is inside it, which is why the echoes wobble like tape for a moment.

Reverb is a ConvolverNode, which stamps the sound of a room onto a signal. The usual way to get a room is to load a recording of one. This one is generated in a loop: 2.2 seconds of random noise that fades out, different in the left and right channels so it sounds wide.

data[i] = (Math.random() * 2 - 1) * Math.pow(1 - i / length, 2.4)

The Reverb knob is a send: it sets how much of everything goes into the room. That matters in the next section, where a single note gets a way into the same room that the knob does not control.

A Pad Is Not a Switch

On a hardware synth, what your finger does after a key goes down is half the instrument. Press harder or rock the finger and the note swells or wobbles. Keyboards call it aftertouch; the controllers that track each finger separately call it MPE.

The pads here do a version of it. A pad remembers where the finger landed, and reads its movement from that point:

Sideways bends the pitch, up to a semitone either way, so rocking a finger is vibrato. A quarter of a pad's width is 76 cents.
Any direction opens that note's own send into the reverb. The further the finger strays from where it landed, the more of that one note goes into the room, whatever the Reverb knob says.

express(midi: number, bend: number, amount: number) {
  const voice = this.voices.get(midi)
  voice.oscs.forEach((osc, i) =>
    osc.detune.setTargetAtTime(this.tuning(voice, i) + bend, now, 0.012)
  )
  const level = amount * EXPRESSION_SEND
  // fast up, slow down: the bloom outlasts the wiggle that made it
  voice.send.gain.setTargetAtTime(level, now, level > voice.sendLevel ? 0.03 : 0.4)
}

The bend goes on osc.detune, the same parameter that keeps a note's two oscillators apart, so the pair moves together. The send is one more GainNode per held note, wired from the note's envelope straight into the convolver and closed until the finger moves. Measured, a note played with the finger pushed away from where it landed leaves about twenty times as much reverb behind as the same note played still.

Two numbers keep it playable. The first 8% of a pad's width of movement counts as nothing, because a finger settles as it lands and a tap should stay in tune. And a finger has to travel 30% of a pad's width past the edge of its pad before the next pad plays. Without that, a wiggle that began near an edge would turn into a trill between two notes. With it, sliding along the row still plays every pad in turn.

The screen draws the bend as you make it, and the pad's own light leans the way the pitch is going and glows wider as the reverb opens. Notes played from the computer keyboard have no position to read, so they are not affected.

Loud Enough, and Never Too Loud

A synth has an awkward range. One quiet note and eight loud ones come out of the same instrument. Set the level so a chord is safe and a single note is too quiet to hear on a phone. Set it so a single note is healthy and a chord runs off the top of the scale and clips.

Three stages deal with it:

Each note peaks at 0.32 of full scale, and the volume knob is squared before it is used as a gain. Loudness is heard on a log scale, so a squared knob sounds about half as loud at half way, where a linear one would sound barely quieter.
A limiter. This is a DynamicsCompressorNode with a threshold of -9 dB and a ratio of 10. One note arrives just under the threshold and passes almost untouched. A chord arrives well over it and is held down. The node also adds make-up gain of its own, about 5 dB at these settings, which is what lifts a single note to a healthy level.
A soft clipper. A limiter takes a few milliseconds to react, and a resonant filter can throw out a peak faster than that. The last node is a WaveShaperNode whose curve is a straight line up to 0.8 and then bends over, creeping towards full scale without reaching it.

const bent = size <= KNEE ? size : KNEE + (1 - KNEE) * Math.tanh((size - KNEE) / (1 - KNEE))

Measured at the last node before the speakers: one note peaks at between 0.4 and 0.65 of full scale depending on the preset, and a six-note chord at about 0.8. With every knob at its maximum and an eight-note chord held over a running pattern, it reaches 0.9. It does not reach 1.0.

Keeping Time Without a Timer You Can Trust

This is the part that looks easy. A sequencer plays a note every sixteenth, so setInterval at the step length should do it.

It does not. JavaScript timers fire when the main thread gets round to them. A 25 ms timer can arrive 10 ms late or 50 ms late, and a rhythm that is 30 ms off sounds drunk.

The fix is to never let the timer play anything. It wakes up every 25 ms, looks 120 ms ahead on the audio clock, and schedules every step that starts inside that window at its exact time:

private tick = () => {
  while (this.nextTime < ctx.currentTime + LOOKAHEAD) {
    this.scheduleStep(ctx, this.nextTime)
    this.nextTime += this.stepSeconds
  }
}

The timer can be as sloppy as it likes. As long as it wakes up at least once inside each window, every note has already been handed to the audio thread with a precise start time. A sequenced note is scheduled whole, release included, because its length is known up front.

Swing falls out of the same code: every second step is scheduled a little late. The arpeggiator is the same clock playing the notes you are holding instead of the pattern.

One consequence: a note is scheduled up to 120 ms before it sounds. If the pads lit up when their note was scheduled, the lights would run ahead of the music. So the engine keeps a list of when each note starts and ends, and on every animation frame the interface asks what is sounding on the audio clock right now and switches the lights to match.

The Knobs and the Screen

Each knob is a div with role="slider": drag up and down to turn it, hold Shift for fine steps, press it twice quickly to put it back, or focus it and use the arrow keys. A screen reader hears "Filter cutoff, 1.0 kHz", not a percentage.

Choosing a preset moves every knob at once. The dials are styled with a half-second CSS transition, so they sweep to their new positions together, and that transition is switched off while a knob is being dragged so it follows the hand exactly.

A button that switches something on has a small light above its legend. My first idea was to light the legend itself in lime. Measured against the body colours, lime lettering came out under the 4.5:1 contrast that small text needs, so the lettering stays white and the light does the signalling, which is also how real hardware does it.

The screen is one canvas with three jobs:

At rest it is an oscilloscope over a faint spectrum, both read from an AnalyserNode every frame. Drawn as the samples come, the wave would start at a random point in its cycle each frame and shiver sideways. Real scopes solve this with a trigger, and so does this one: each frame starts where the signal crosses zero on the way up.
While a knob turns it shows that control: the envelope's shape with the segment you are changing lit, or the filter's frequency response. The filter curve is computed from the same formula the browser's filter uses, so it is the real response for those settings. The dashed line is where the envelope opens it to, and the curve sways at the LFO's actual rate and depth.
Along the bottom, where the screen is tall enough, is the pattern, with the playing step lit.

When nothing has sounded or changed for 0.7 seconds the screen stops redrawing. A canvas repainted sixty times a second for no reason is a phone battery being emptied.

One Screen, Any Screen

The instrument is always exactly as big as the window allows, on a phone or a desktop, and the page never needs to scroll to reach part of it. No JavaScript is involved in that.

Every size inside the instrument is a multiple of one custom property, --u, which stands for one pixel of the design. The space the instrument sits in is a size container, and --u is set from that space with container query units:

.device {
  --u: min(calc(100cqw / 1120), calc(100cqh / 664), 1.3px);
  width: calc(1120 * var(--u));
  height: calc(664 * var(--u));
}

.dial {
  width: calc(56 * var(--u));
  height: calc(56 * var(--u));
}

The design is 1120 by 664. 100cqw / 1120 is the scale the width allows, 100cqh / 664 is the scale the height allows, and --u is the smaller of the two. Everything is drawn in multiples of it, so the instrument grows and shrinks as one object.

One shape does not suit every window, so there are four arrangements, picked by container queries on the shape of the space:

Wide, for desktops and tablets on their side: everything on show.
Tall, for tablets upright: everything on show, stacked.
Phone: the screen, one octave of pads, Play, and four tabs that swap which controls sit underneath.
Flat, for a phone on its side: the same four pages beside the screen, over two octaves of pads.

The markup is the same for all four. The phone's single octave is the stylesheet declining to draw the second one, and each tab's page is placed on the body's grid by name. That matters for a server-rendered page: the HTML that arrives is right for whatever device it lands on, with nothing to rearrange once the script loads.

Knobs and pads set touch-action: none, which is what stops a drag on a knob from scrolling the page. The body sets touch-action: manipulation, which stops two quick taps on a button from zooming in.

The Parts the Browser Makes Interesting

Sound needs a gesture, and a touch only counts when the finger lifts. Browsers do not let a page make noise until the visitor has clicked, tapped or pressed a key, so the AudioContext is created on the first one. On a touch screen that gesture is the finger coming up, not going down. The note a first tap starts is built while audio is still locked, and released the normal way it would be over before the audio clock had moved. That one note is given a fixed length of 180 ms instead, so it sounds as the finger lifts. Every note after it sounds as the finger lands.

The silent switch. On an iPhone, sound made with Web Audio counts as "ambient", like a game's sound effects, and the switch on the side of the phone mutes it. Everything still runs. The sequencer plays, the pads light, the scope draws waves, and nothing comes out. Declaring the page's audio as playback, the category music and video use, makes it sound whatever the switch says:

navigator.audioSession.type = 'playback'

Older versions of iOS have no audioSession. There, an <audio> element that is playing puts the whole page in the playback category, so the page loops one second of silence. The silence is a file rather than a blob made in the browser, because this site's Content-Security-Policy does not allow media from blob URLs.

A phone speaker has no bass. The other way to be silent on a phone is to be quiet and low. A phone speaker reproduces very little below a few hundred hertz, and a pattern that starts on the C below middle C, through a filter set low, lives almost entirely down there. The presets' cutoffs sit higher than they would on a studio synth, and on a phone the keyboard and the starting pattern begin an octave higher. Measured, about three quarters of the pattern's energy is then above 300 Hz. An octave lower it was a third.

cancelAndHoldAtTime is not everywhere. Where it is missing, the engine works out where the envelope has got to from when the note started, cancels the schedule and sets that value.

Stuck notes. If you hold a key and switch windows, the keyup goes to the other window and the note rings forever. The page lets go of every key when the window loses focus. When the tab is hidden it also stops the sequencer and suspends the audio clock: a background tab's timers are throttled to once a second, no lookahead window survives that, and nobody is listening.

Keys are for playing. Click a button in Chrome and it takes keyboard focus. On most pages nobody notices. Here the next thing you do is press a key to play a note, and two things go wrong: a focus ring appears around the button you clicked a minute ago, and Space, which should start and stop the pattern, presses that button again. So a click or a drag with the pointer leaves focus where it was, the way buttons on a Mac never take focus from a click. Tab still reaches every control, and the arrow keys turn whichever knob has focus.

Physical keys, not letters. The keyboard mapping uses event.code rather than event.key, so the home row is the lower pads and the row above is the upper pads on any layout, including AZERTY and Dvorak.

Sliding across the pads. The pad area captures the pointer on press. A drag plays each pad it moves onto, and a release that happens off the pads still ends the note. Every finger is tracked separately, so chords work on a touch screen, and each finger bends its own note.

A second finger never clicks. A browser turns a touch into a click only when it is the one finger on the screen. Hold a pad with one finger and tap Play or a tab with another, and the button gets a pointerdown and a pointerup and no click, so nothing happens. Holding a note while reaching for a control is how an instrument is played. The page keeps count of the fingers that are down, and when one that shared the screen lifts off the button it landed on, the page presses that button itself.

Things to Try

Press Play, then sweep Cutoff and Reso while the pattern runs.
Choose Pad, hold one pad and rock your finger from side to side. Then hold another and slide the finger up: no bend this time, only the room opening.
Turn on Arp and Hold, press three pads and let go. Now you have both hands free for the knobs. Try the Pad preset with the arpeggiator set to Rand.
Press Clear, then Rec, and play notes one at a time: each lands on the next step. Or leave the pattern playing with Rec on and play along; notes land on the nearest step.
Turn Tempo while notes are echoing and listen to the repeats bend.
Push Swing to about 30% for a shuffle.
Turn LFO depth up and watch the filter curve on the screen sway in time with what you hear.

What Is Not in It

There is no MIDI input, no way to save a sound or a pattern, and the sequencer holds one pattern of one note per step. Each of those is more interface than audio code; the graph would not change.