Singing Speech Synthesizer With Theremin-Like Controls

The embedded video is a collection of soundbytes that give a good general background of the Glove-TalkII system from Sidney S. Fels and Geoffrey E. Hinton.at The University of British Columbia. But the brief samples of the system in operation that it includes are, frankly, not a great advertisement for its capabilities or its potential. The best videos don’t seem to be available as embeds, yet, and to really appreciate how Glove-TalkII works, I recommend following this link to download a short AVI clip from the project homepage, showing operator Sageev Oore “singing” the alphabet song using hand gestures.

As with a Theremin, pitch is controlled by hand position in space. Closed-finger gestures, with one hand, create voiced consonant sounds, while open-finger gestures with the same hand give sustained vowel sounds. Hard stops, like “puh” and “buh” and “tuh,” are controlled with the fingers of the opposite hand. Looks like lots and lots of fun. [Thanks, Laura!]

2 Responses to Singing Speech Synthesizer With Theremin-Like Controls

Austin.Y on February 21st, 2012 at 9:29 am said:

Lol @ 4:10

“potentially you could actually watch somebody’s movements (wiggles hands), and understand what they’re saying…without any sound”

Too bad no one has ever come up with a language that accomplishes that…

Reply ↓
Andrew Sayler on February 21st, 2012 at 10:22 am said:

This concept was introduced by H. W. Dudley, R. R. Riesz, and S. A. Watkins in an article called “Synthetic Speaker” in the Journal of the Franklin Institute, vol 227, June 1939.
Dudley worked extensively in the Bell Labs, where he laid the groundwork for various vocal compression techniques. The apparatus in the video bears much resemblance to what Dudley named the “Voder” His Voder “contained 10 contiguous band-pass filters which covered the speech band. These filters were activated by volume controls operated by finger keys. Three additional finger keys provided transient excitation of the selected filters for stop-consonant sounds. The filter was supplied with either a noise or buzz oscillator. A pedal controlled the pitch of the buzz oscillator.” (“ITT Reference Data for Radio Engineers”, Fifth Edition, 1968)
It’s interesting to see someone applying modern technology to concepts that were not practical within the technology of their day. Bravo!

Reply ↓