DeafTeacher - The tool for teaching of deaf people (1996)
Learning and practising basic speech abilities is an ex-tremely hard task for a deaf person. His or her essential problem consists in the fatal lack of the feedback infor-mation about the produced sounds. For the speech thera-pist involved in the training, the main problem is how to translate the missing acoustic information, whose nature is quite complex, back into a form that would be accept-able and understandable for the trained subject. In order to help both the sides, the deaf and the therapist, we have developed a program that is capable of visual-ising speech signals.. The program gets data directly from the microphone and displays them on the computer screen. The signal is presented as a standard waveform and in the form of a spectrogram. This gives a skilled user a more complete view on the characteristics of the produced speech. The most important feature of the program is its ability to display multiple speech data. It means that the users can see not only the currently spoken utterance but also several samples of the same utterance recorded by refer-ence speakers. The multiple plots allow for visual com-parison of different pronunciation styles and give the therapist a chance to demonstrate and explain the major differences between the speech of the trained subject and that of the other speakers. Usually, the users data are displayed in the topmost window while the other windows serve for the refer-ences. However, it is also possible to place a series of repeated attempts into several different windows. In this way, the history of the training can be kept, evaluated and utilised for motivating the trained person towards further improvement. We have also tried to utilize the existing speech recog-nition system in training the hearing-impaired people. Though the recogniser performance is often negatively influenced by very poor pronunciation, the classification results give at least an approximate, global, measure of the speech quality. That is why the recogniser has been also built into the program. Its output is displayed in the upper part of the screen. However, when using the automatic speech recognition we must be aware of potential danger that is hidden in the classification method. The classifier always searches for the model that has the highest likelihood. Sometimes, if the pronunciation is very bad, and also in case the utterance is wrongly detected, the recogniser output may be inadequate or even very strange. If it happens fre-quently it can cause serious loss of motivation of the trained person. To cope with this problem we have tested an alternative strategy. For people with very bad speak-ing abilities we first create models based on their own (no matter that bad) pronunciation. With these models the people usually achieve quite fair recognition results which establishes their positive motivation. Then, these user specific models can be replaced by (or alternatively melted with) the standard speech models. This should be done gradually, in accord with the progress in training. Up to now, the visual feedback system has been tested - with promising results - by many deaf people.