Guiding EMG-to-Speech Synthesis via Silent Speech Recognition
GETS: Guiding EMG-to-Speech Synthesis via Silent Speech Recognition
Abstract
Electromyography-to-speech (ETS) offers a promising solution for voicing silent speech, yet conventional methods often struggle to achieve high semantic intelligibility.
We introduce GETS, a novel framework that establishes a new state-of-the-art (SOTA) with a Word Error Rate (WER) of 11.89% on the silent EMG test set.
GETS comprises a diffusion-based generative network conditioned on EMG signals, integrated with a semantic guidance mechanism derived from silent speech recognition (SSR).
Our framework not only demonstrates superior semantic intelligibility by effectively disambiguating underspecified EMG inputs but also faithfully preserves the prosodic characteristics—such as temporal alignment and energy contours—inherent in the original EMG signals.
This synergistic fusion of acoustic EMG cues and SSR-based semantic guidance establishes a new benchmark for high-fidelity, naturalistic silent speech communication.
Audio Samples
The audio samples below are synthesized speech from silent EMG signals.
No.
Ground Truth
Baseline (Gaddy & Klein)
Ours
1
WER: 33.33%
WER: 0%
his idea was that meteorites might be falling in a heavy shower upon the planet or that a huge volcanic explosion was in progress
this idea was that
pure ice might be falling in a heavy shower
above the planet or that a huge
bulk of the explosion was in
barcrest
his idea was that meteorites might be falling in a heavy shower upon the planet or that a huge volcanic explosion was in progress
2
WER: 50%
WER: 0%
henderson he called you saw that shooting star last night
enders in tikon do you follow that shooting star last night
henderson he called you saw that shooting star last night
3
WER: 33.33%
WER: 0%
there was a mouth under the eyes the lipless brim of which quivered and panted and dropped saliva
there was some path under the eyes the
limpless brim of which quivered and
panded and
trapped solaifer
there was a mouth under the eyes the lipless brim of which quivered and panted and dropped saliva
4
WER: 5.88%
WER: 0%
those who have never seen a living martian can scarcely imagine the strange horror of its appearance
those who have never seen a living martian can scarcely imagine the strange horror of its
parents
those who have never seen a living martian can scarcely imagine the strange horror of its appearance
5
WER: 55.56%
WER: 0%
besides that there was quite a heap of bicycles
but since then there was quite a
game of
mysegals
besides that there was quite a heap of bicycles
6
WER: 20%
WER: 0%
it would seem that a number of men or animals had rushed across the lawn
it would seem the number of men or
hennaples had rushed across the lawn
it would seem that a number of men or animals had rushed across the lawn
7
WER: 25%
WER: 4.17%
the case appeared to be enormously thick and it was possible that the faint sounds we heard represented a noisy tumult in the interior
the case appeared to
me enormously thick and it was possible that the faint sounds we heard
were presented towards the tumult
at the interior
the case appeared to
me enormously thick and it was possible that the faint sounds we heard represented a noisy tumult in the interior
8
WER: 17.39%
WER: 4.35%
i did not know it but that was the last civilised dinner i was to eat for very many strange and terrible days
i did not know it but that was the last
civilized sitter i was
heath for very many strange and terrible days
i did not know it but that was the last
civilized dinner i was to eat for very many strange and terrible days
9
WER: 30%
WER: 5%
by byfleet station we emerged from the pine trees and found the country calm and peaceful under the morning sunlight
by
my fleet of station we emerged from the pine trees and found the country calm and
made full under the
burning sunlight
by byfleet station we emerged from the pine trees and found the country calm and peaceful
on the morning sunlight
10
WER: 22.22%
WER: 7.41%
he saw this one pursue a man catch him up in one of its steely tentacles and knock his head against the trunk of a pine tree
he saw this one
first one man catch him up in one of
his steely
tickles and
to dock his head against the trunk of a pine tree
he saw this one pursue a man
catching him up in one of its steely tentacles and
knocked his head against the trunk of a pine tree
11
WER: 40%
WER: 10%
the horse took the bit between his teeth and bolted
the horse
looked the
mid between
the steve and bolted
the horse took the bit between his teeth and bolted
it
12
WER: 75%
WER: 12.5%
such things i told myself could not be
some of the things i told myself
couldnt be
me
such things i
call myself could not be
13
WER: 38.46%
WER: 15.38%
some of the refugees were exchanging news with the people on the omnibuses
some of the refugees
or extinjing news
were the people on the
enemy musters
some of the refugees were exchanging news with the people on the
inner benches
14
WER: 57.14%
WER: 21.43%
the chances against anything manlike on mars are a million to one he said
the
gents against
the indulging maylight of mars are a
millionth of one
if he said
it changes against anything manlike on mars
or a million to one he said
