Sound Recognition and Information Transmission System for Hearing Impaired People and Elderly People

Currently, Japan is becoming a very aged society, and elderly people suffering from deafness are also increasing. The number of Japanese hearing impaired people is increasing. Also, the number of hearing impaired people in Japan is increasing. In this paper, we describe ways to recognize and transmit various sounds in life for the purpose of supporting hearing impaired people and elderly people. About sound recognition, we focused on the sound made by things, not by the human voice. In addition, visual information and tactile information can be mentioned as a transmission method. This time we examined transmission by vibration.


Introduction
According to the report of the Cabinet Office, as of October 2017, the population of Japanese elderly (Over 65 years of age) accounted for 27.7% of Japan's total population.In this way, elderly people account for a quarter of the population and lack of caregivers is a problem (1) .According to the population statistics of the number of hearing-impaired people announced by Japan Trak 2015, it gradually increased from around 45 and the number of hearingimpaired people over 75 was 6732128.
In addition, people who are over 75 years old recognize that hearing-impaired is 41.6%, nearly half of whom have suspicion of hearing-impaired.Many people feel a decline in hearing due to the influence of aging (2) .
In this paper, we describe a system that catches the sound emitted from an object and communicates to hearing impaired with vibration.In Japan where the number of hearing impaired people increases, various devices are being orchestrated as welfare for the hearing impaired (3) .There are various types of welfare equipment (4) for hearing impaired people and elderly people, but many devices that convert audio information to other information perform event processing for a predetermined action and notify them.As an example, this is equivalent to a product that transmits the sound of an interphone or FAX by flashing a lamp.Although these products are useful, they are low value added freedom cost and performance as one product.Therefore, the burden on the caregiver never decreases greatly.Also, important things and information that you want to know will be different in each person.In that case, the equipment must be collected separately, which adds cost.Therefore, we propose a system that learns sounds and generates events for every sound.

Constitution
A schematic diagram of the system is shown in Fig. 1.The software language of this system uses Python3.Components external USB microphone, Arduino and a motor are used as hardware

Software part
The software part of the system consists of two kinds of serial communication program and sound recognition.In the sound recognition program, external sound is read with a USB microphone.Thereafter, predetermined feature parameters are extracted.Further, this system read multiple sound analyzed all the sounds, extract the features.When picking up the sound for matching with a microphone, it performs the same analysis and features.If they have the same characteristics as the feature parameter was previously extracted as referring, Arduino will perform serial communication.

Hardware part
The hardware part of the system consists of Arduino and the motor.There are several ways of communicating information to the hearing impaired.Examples include transmission by visual information using a smartphone and transmission by vibration information (5) .Based on the data sent from python by serial communication, conditional branch is made by Arduino.Then, the vibration is transmitted by the motor.Prepare several vibration patterns.Then, the kind of sound is judged by the given vibration.

Experiment
"Sound of the voice" can be divide into "voice" and "sound".Both are waveforms composed of sine waves.Therefore, it can be identified by analyzing the waveforms.However, since it requires advanced processing to distinguish between "voice" and "sound", we focused on "sound".

Waveform Feature Extraction
First, we extract the characteristics of a sound mixed with a certain frequency and noise.Frequencies in the experiment are 4000 [Hz], 6300 [Hz], 8000 [Hz].Fig. 3 shows the power spectrum by Fourier transformation.As shown in Fig. 3, it can be seen that the spectrum of each frequency becomes large.

Sound Characteristics Extraction
Then we apply experiments before and experiment with "the sound of things".As the source of the subject of the experiment, we used sounds of "door", "chime" and "end of washing machine".We read these sounds into the system and put out the power spectrum.Experimental results show Fig. 4. As shown in Fig. 4, each sound had a frequency difference.The sound of the "door" had the maximum power spectrum at about 100 [Hz].The sound of the "chime" had the maximum power spectrum at about 900 [Hz].The sound of the "end of washing machine" had the maximum power spectrum at about 2500 [Hz].

Frequency Recognition
A specific frequency can be seen in the sound as shown in 3.1 and 3.2.In this experiment, the machine learns a sinusoidal sound of a specific frequency.Then, sine wave sound again to the USB microphone.We conducted the experiment to inform the transmission system if the frequency of the picked up sound and the frequency of the learned sound are the same.The frequencies are 4000 [Hz], 6300 [Hz] and 8000 [Hz].The results are shown in Table .1.Table .1 shows recognition of frequency can be discriminated with high accuracy.

Sound Recognition
The previous experiment focused on specific frequencies and we analyzed and recognized.The sounds that you usually hear are made by synthesizing various frequencies, but there are frequencies.We can recognize by reading them.
In this experiment, we used three sounds such as "door closing sound", "chime" and "end sound of washing machine".We use these three sounds to recognize sound.In this recognition, the sound collected by the microphone is saved as a numerical value in the file.After that, we used a method to compare with the characteristics of the sound learned beforehand.The results are shown in Table .2.

Transmission Experiment
The result done in the experiment 3.4 to Arduino by serial communication, and the motor of the circuit is rotated to be transmitted people by vibration.Assign the type of vibration to each sound and check whether it is actually sent.The results are shown in Table .3.They were accurately communicated.

Fig.4 Power Spectrum of Sound
Table .1 Frequency results