What is voice activity detection?
Introduction
Project introduction
What is voice activity detection?
Voice activity detection (VAD) answers one question about a recording, again and again, which is whether someone is speaking right now. It runs before almost every speech system. A video call uses it to stop sending audio while you are silent, a voice assistant uses it to know when you have finished a sentence, and a transcription tool uses it to skip the silent parts before a large speech model listens.
Sound reaches a computer as a long list of numbers called samples, the position of the microphone thousands of times per second. A detector cuts that list into short frames of about 25 milliseconds and decides speech or not speech for each one. Telephone standards such as G.729 Annex B have done this since 1996. WebRTC VAD, used in Chrome, works on frames of 10, 20 or 30 ms, and the small neural network Silero VAD takes less than a millisecond for a 30 ms chunk on one CPU thread.
Why is it hard?
In a quiet room speech is simply the loud part, so a loudness threshold works well. Press Listen on the running tap and look at the bars. The words are still easy to hear, yet every moment of the recording is now louder than the threshold, so a loudness detector calls all of it speech.
What still gives speech away is its shape across frequencies. A voice puts its energy into a pitch, its harmonics and the resonances of the mouth, mostly in the low and middle frequencies, and changes that pattern every few tens of milliseconds, while a tap or a fan hisses evenly. In this project you will measure the failure of loudness yourself and then build features and a small network that use the shape instead.
Where is someone speaking?
4 s, two spoken wordsWhat you will build
01
Loudness
Cut a recording into frames, measure each in decibels, and write a threshold detector with a hangover.
02
Frequencies
Build a mel filterbank and log-mel features, the picture of sound that speech models learn from.
03
Segments
Score detectors frame by frame and turn their decisions into speech segments with start and end times.
Start with frames
Cut a waveform into overlapping frames and measure each one in decibels.