Example 04 · GRU

Wake Word Detection

An always-on listener that wakes a device when it hears its name, without streaming the microphone anywhere. Our 5,121-parameter network listens for one word in every second of audio, and we trained and tested it on real recordings from Google’s Speech Commands dataset.

Read the write-up on GitHub

A smart speaker shown cut away: a MEMS microphone at 16 kHz feeds an always-on microcontroller running log-mel features and a GRU, which wakes a sleeping main processor only for the wake word. Beside it, what the device hears: 'Marvin', other words such as 'house', 'go' and 'stop', and background noise like a running tap. — The wake-word detector sits between the microphone and everything that costs power, and only wakes the main processor when it hears its word.

parameters in the whole network: 5,121
median accuracy on 441 real recordings it never trained on: 85%
of weights, before quantizing to 8 bits: ~20 KB
its score for “marvin”, against 0.008 for “house” from the same speaker: 0.989

Where it's used

Listening all the time, on almost no power

A doorbell, a smart speaker or a hearing aid that responds to its name needs something listening all the time. Streaming the microphone to a server costs bandwidth, power and privacy, and keeping a full speech recognizer awake drains the battery.

A wake-word detector gets around both. It’s a small model on a low-power processor that checks the last second of audio for one word, and wakes the rest of the system only when it hears it.

A doorbell that answers to its name
A smart speaker that wakes only when it’s called
A hearing aid that responds when it hears its name
No audio has to leave the device to listen for the word

The data

Real voices, real recordings

We used Google’s Speech Commands dataset: one-second clips of single spoken words from thousands of people, recorded on their own devices. The wake word is “marvin”, with 2,100 clips of it, set against 2,302 clips of other words and background noise, so the network learns to stay quiet through both.

The wake word is just a setting. Training for a different word is a rerun, not a rewrite.

Training · 80%, 3,521 clips: what the network learns from
Validation · 10%, 440 clips: checked after each round of training, so we keep the best model
Test · 10%, 441 clips: kept aside until the very end. The scores below come from these.

Waveforms and log-mel feature images for three one-second clips: the word 'marvin' with two voiced syllables, the word 'house' from the same speaker with one burst and a hissing ending, and background noise from a running tap that fills all bands evenly. — Three clips as the network hears them: the waveform on top and the sound features below. “marvin” and “house” are the same speaker.

How we built it

Sound in, one decision out

Two pieces work together. The first turns each second of audio into 49 short frames of 20 sound-energy bands, the same first step full speech recognizers use, and it runs identically on the device against the live microphone. The second is a 32-unit GRU network that reads those frames one at a time and decides whether the clip was the wake word.

01
Frame
32 ms windows, every 20 ms: 49 per second
02
Transform
a 512-point FFT of each window
03
Bands
20 mel bands from 0 to 8 kHz, then the log of each
04
Normalize
every clip scaled the same way, whatever its volume

Pipeline: microphone at 16 kHz, frames of 512 samples every 320, Hann window, FFT and 20 log-mel bands, per-clip normalization; then each 20-value frame into a 32-unit GRU with a recurrent hidden state across the clip's 49 frames, a sigmoid output for wake-word probability, and a decision after the last frame: wake word if at least 0.5. — The whole pipeline. The network reads one frame per step and carries its memory across the clip, then makes its call after the last frame.

Loud or quiet, it hears the same thing. Real recordings vary enormously in volume and microphone quality. Each clip is normalized before the network sees it, which takes those differences out of the picture.

Results

85% on recordings it had never heard

We trained the network seven times and tested every run on the same 441 held-out clips, 217 of them “marvin”. The median run got 85.3% of them right, and the best got 89.1%.

Tuned to your product

Every listener balances two kinds of mistake: missing the wake word, and waking up when it shouldn’t. Across our runs, one almost never woke by mistake, on just 3.6% of other clips, while another caught all but 6% of the wake words.

Each product wants that balance somewhere different. A doorbell can afford the odd false wake-up, and a device that starts recording when it wakes can’t. In a product, we set the decision point deliberately on validation data, so it lands exactly where you need it.

Dot plot of seven runs: accuracy between 78 and 89 percent, median 85.3; false-reject rate between 6 and 26 percent, median 15.7; false-accept rate between 4 and 23 percent, median 14.3. — Seven training runs, each tested on the same 441 clips. Each dot is one run, and the bar is the median.

On the device

An always-on listener on a microcontroller

On the device, the same feature code runs against the microphone buffer and our library runs the network. Each second of audio takes 49 small FFTs and 49 steps of the 32-unit network, and a live listener slides that one-second window along the stream.

The 5,121 weights come to about 20 KB, and our tools convert a trained model to 8-bit integers and to a format that runs straight out of flash.

microphone audio, read a second at a time: 16 kHz
small FFTs and network steps per second of audio: 49 + 49
of weights, and smaller again as 8-bit integers: ~20 KB
to train a new model on a desktop: Seconds

Get the code and the full write-up on GitHub

Ready to build your next intelligent device?

Let's discuss your project and how Cole Design & Development can bring your idea to life.

Talk to Us Today Explore Our Services

Founded in 2008 · Hudson, Ohio · info@coledd.com