What is the sound of one person talking?

When researchers interviewed 40 Columbus residents about their opinions on life in the city, the scientists ignored what the people had to say. All the scientists really cared about was how they said it.

In a lab at Ohio State University , researchers recorded 30- to 60-minute casual interviews with these residents, talking about how Columbus has changed over the years, how families should get along, as well as issues in sports, traffic and politics.

The participants were told that the purpose of the study was to learn how people express "everyday" opinions in conversation. But when the interviews were over, the scientists didn't even examine the views that were expressed.

Instead, they painstakingly listened to every spoken word – indeed every syllable - along with coughs, laughs and pauses in the conversation, and then labeled what actually was spoken.

The result is a 306,652-word repository of how people in central Ohio speak. Now scientists from around the world, and in a variety of disciplines, can use the collection – called the Buckeye Speech Corpus – to advance their research. (Researchers refer to a collected body of recordings as a corpus.)

"A critical part of communication is how your ears translate the sounds coming out of my mouth into recognizable words," said Mark Pitt, one of the leaders of the project, and professor of psychology at Ohio State.

"You don't have to go to school to learn how to do it. It is fast and efficient. The scientific question is 'how is this done?' The corpus will help researchers find answers to that question and many others."

Other members of this large research team include Eric Fosler-Lussier, assistant professor of computer science and engineering, and Elizabeth Hume, professor of linguistics, both at Ohio State.

The corpus, available to researchers on DVD, includes the actual recordings of the interviews, a written transcript, and several levels of labeling. The first half of the corpus was officially released in March. A typical display of a stretch of speech from the DVD shows two visual representations of speech from one of the talkers, indicating volume and frequency. Below the displays is a line with a transcription of what the speaker is saying. Then comes a line of "phonemic labeling," which shows how each word spoken was pronounced – similar to the pronunciation guides found in a dictionary.

This phonemic labeling is a key part of the corpus, because in casual conversations, people don't always use the proper English pronunciations they were taught in school, said Laura Dilley, a post-doctoral researcher in psychology who works on the corpus.

"It's interesting because of the many kinds of modifications that happen to the acoustics of speech in conversations," Dilley said. "For example, we may pronounce 'don't you' as 'don'tcha' when we are talking.

"By looking at the phonemic transcriptions we can tell exactly how a person pronounced a word. And from that scientists may be able to make some inferences about why people produce the sound one way as opposed to another."

The interviews are also labeled to show when people made non-word sounds during a conversation, such as a cough.

"Even laughs are noted, because some researchers may be interested in how that is part of a conversation," Pitt said. "It can convey meaning as well."

The 40 residents were interviewed in 1999 and 2000. But it has taken until this year to label the speech of half of the speakers, Pitt said. The data from the remaining 20 participants should be available by the end of the year.

"Collecting the interviews took a long time. But what really takes a long time is labeling them. Researchers had to listen to each conversation and figure out what sounds of the language the speakers said or didn't say. It's very difficult and very time consuming," he said.

When you actually sit down to label each word, "they can blend together and be very difficult to identify," Pitt said. "Sometimes you scratch your head trying to figure out how to label the speech. But when you're just listening to it as part of the conversation, it is all perfectly intelligible. That's part of the mystery of communication that scientists are trying to figure out."

A combination of factors makes the Buckeye Speech Corpus a unique resource for researchers, according to Pitt and Dilley. For one, it is one of the largest corpora of high-fidelity, conversational speech available. Other corpora involve speakers reading words directly from a text, but that is very different than conversation. There are also corpora of conversational speech recorded over the telephone, but that does not have the high fidelity of this corpus, and the dynamic of face-to-face conversation.

Another key difference is that the Buckeye Corpus is available free to researchers, both in academia and industry.

Although the Buckeye Corpus has been available for only a few weeks, researchers from around the world are already ordering copies of it. A researcher from Italy ordered the corpus to study "slips of the tongue," such as when people accidentally substitute one word for another that sounds similar (saying "rabbit" instead of "habit").

Pitt and Dilley expect a lot more interest in the corpus.

"Across a variety of fields -- in communication, speech sciences, linguistics, speech and hearing, computer science, psychology – the corpus will provide different uses," Pitt said. "It was created for all these scientific communities."

Computer scientists could use the corpus to help improve speech recognition software. Communication researchers will find it useful to study conversational dynamics. Psychologists will be interested in what listeners are faced with in terms of the physical and acoustic properties of speech.

Eventually, the corpus will have a search function that will make it even more useful. "If a researcher wanted to look up all the uses or pronunciations of a certain word in the interviews, he or she could do that," Dilley said.

The work of transcribing and labeling the interviews allowed the Ohio State researchers to uncover some interesting facts about how people speak in conversation.

For instance, the speakers spoke a total of 306,652 words, but only 9,600 different words. Almost 80 percent of the total words spoken were one-syllable words. One and two-syllable words made up more than 90 percent of the total words spoken.

Slightly more than half of the words spoken (57 percent) were function words – words like prepositions, pronouns and conjunctions that have mostly grammatical uses within a sentence. The remaining 43 percent were content words, which includes nouns, verbs and adjectives.

The Buckeye Speech Corpus project was originally funded with a seed grant from Ohio State's Office of Research. It has since received funding from the National Institute on Deafness and Other Communication Disorders, which is part of the National Institutes of Health.

Ohio State may be emerging as a center for corpora of conversational speech, Pitt said, as the Department of Speech and Hearing Sciences was just awarded a large, multi-year grant from NIH to collect interviews with speakers native to Ohio, Wisconsin, and western North Carolina.

Source: Ohio State University

Related stories:

Fast-learning computer translates from four languages
Modern approaches to machine translation between languages require the use of a large ‘corpus’ of literature in each language. Now a European project has demonstrated a cheaper solution which compares favourably with the market leaders in translating from Dutch, German, Greek or Spanish into English.
Ability to listen to 2 things at once is largely inherited, says twin study
Your ability to listen to a phone message in one ear while a friend is talking into your other ear—and comprehend what both are saying—is an important communication skill that’s heavily influenced by your genes, say researchers of the National Institute on Deafness and Other Communication Disorders (NIDCD), one of the National Institutes of Health. The finding, published in the August 2007 issue of Human Genetics, may help researchers better understand a broad and complex group of disorders—called auditory processing disorders (APDs)—in which individuals with otherwise normal hearing ability have trouble making sense of the sounds around them.
A computer that can 'read' your mind
For centuries, the concept of mind readers was strictly the domain of folklore and science fiction. But according to new research published today in the journal Science, scientists are closer to knowing how specific thoughts activate our brains. The findings demonstrate the power of computational modeling to improve our understanding of how the brain processes information and thoughts.
Computer model reveals how brain represents meaning
Scientists at Carnegie Mellon University have taken an important step toward understanding how the human brain codes the meanings of words by creating the first computational model that can predict the unique brain activation patterns associated with names for things that you can see, hear, feel, taste or smell.
An evil storm is forecast
Wicked, Evil, Foul, Bad -- all words meaning essentially the same thing, yet we don't talk about "evil weather," "foul witches" or the "forces of bad."
Estrogen and progesterone receptor isoforms expression in the stomach of Mongolian gerbils
Gerbils were treated with estradiol and E2+ progesterone. Stomach proteins were immunoblotted for Estrogen-alpha, ER-beta, progesterone receptor-A, PR-B. ER-alpha, PR-A and PR-B were detected. PR isoforms were not regulated by hormones. E2 down-regulated ER-alpha in corpus. This suggests that E2 and P4 actions in stomach are mediated by their nuclear receptors.
Major universities see promise in Google Book Search settlement
(PhysOrg.com) -- Stanford University, the University of California and University of Michigan announce today their joint support for the outstanding public benefits made possible through the proposed settlement agreement submitted to the United States District Court, Southern District of New York by Google Inc. and plaintiffs the Authors Guild, Inc. et al.
Bays on US Gulf Coast vulnerable to flooding
The most comprehensive geological review ever undertaken of the upper U.S. Gulf Coast suggests that a combination of rising seas and dammed rivers could flood large swaths of wetlands this century in one or more bays from Alabama to Texas.

News discussion:

Technology news