Study of onomatopoeia expression caption on TV contents

Caption exists on many TV programs. Caption is very important for hearing-impaired person to watch TV. They don’t understand TV without captions. So, there are captions in many TV programs. However, there have not been sound's caption on TV.I think that sound's caption is important for hearing-impaired person. Because, hearing-impaired person can not understand in TV programs. In this paper, we analyzed sound's caption on TV. We used the onomatopoeia for sound's caption. We call this caption onomatopoeia caption. We used five kind of animations for experiment. About five animations, we made five patterns of the caption size on TV. Those sizes are 10%,15%,20%,25%,30% on the TV screen size. And, we made five patterns of the caption position on TV. Those positions are the right, the top, the left, the down, near the sound source. As a result, it is appropriate the caption size is as big as moving things in the animation. It is appropriate caption position not overlap moving things in TV.


Introduction
There are about over 360000 hearing-impaired people in Japan.83% of those people live with normal-hearing people.It is difficult for the hearing-impaired person to hear a sound from TV. Therefore when hearing-impaired people watch a television, they perhaps watch TV normal-hearing people.
Captions exist on many TV programs.Captions are very important for hearing-impaired person to watch TV.The way was researched which indicates the balloon on a television, and serif put the balloon on the television.[1] And the way was researched which hearing-impaired person can understand closed caption.[2] We thought hearing-impaired person cannot understand the onomatopoeia.Be-cause onomatopoeia consist of voice and sound.But, in previous study, hearing-impaired person can understand onomatopoeia.[3] Various sounds except serif in TV rarely caption.So, if we subtitled the sound in the TV, a sense of reality increases at the time of the watching TV.
In this report, when we synchronize sound and onomatopoeia captions and appear on TV, we evaluated the effect by the size of the caption and the position of the caption.

Animating onomatopoeia captions 2.1 Onomatopoeia
Onomatopoeia is called Giseigo in Japanese.Giseigo is Giongo and Gitaigo.In particular, a Giongo is the something which expresses "the sound that a thing emits" with words.(e.g., "Wan Wan" which is a dog barks.).We show an example of the onomatopoeia in Table 1.In the experiment from this dissertation, I used two animations they are much moving things and few moving things in animation.
When I gave onomatopoeia captions, I thought about the kind of impression pro-vided by the difference of the movement.In addition, I referred to "an onomatopoeia word dictionary" when I made the animation which gave onomatopoeia on choosing onomatopoeia to use.[4]  animations with few moving things in the animation.III, IV, and V are the animations with much moving things in the animation.

Method of animating the onomatopoeia captions
Using the animation which I explained in 2.2, I gave an appropriate to each animations.About five animations, we made five size patterns of the caption on the TV size.Those sizes are 10%, 15%, 20%, 25% and 30% on TV.And, we made the animation of five positions patterns of captions.The indication point of captions is the right, the top, the left, and source of the sound.At the same time, we conduct the survey.I make 50 animations in total.I used "VideoPad" of NCH Software to make the captions animation.

Evaluation experiment 3.1 Experiment method
We did the evaluation experiment about the subtitled animation.Five people rated the animations they are adults.They evaluated all 50 kinds of animations with the scale of "which caption is the most preferable".The evaluation grade is a scale from -2 to 2, we made the average.Evaluation grade 2 is excellent, 1 is good, 0 is average, -1 is fair and -2 is poor.

Evaluation about the size of suggestion captions
We show the evaluation about the size of the caption in Fig. 1.According to Fig. 1, the size of the 15% captions resulted more than 1 in all animations.On the other hand, the size of 30% captions resulted in the -2 in nearly all animations.When I analyze the survey, to discover the reason why the 30% evaluations were low, an-swers such as" the caption is too big" and, the size of the captions does not connect with the volume of sound" were given.In addition, we thought the size of the 10% captions was low, because of I and III had a bigger moving things of animations than other animations.From these, it is appropriate that the caption size is as big as the moving things of the animation.(All the animations of the moving things were 15%-20%).Fig. 6.Preference scores about size of onomatopoeia captions

Evaluation about the position of suggestion captions
We show the evaluation about the position of the captions in Fig. 2. When only IV showed captions to the left, it was highly evaluated.Source captions were highly evaluated in I, II, IV and V.When we analyze the survey, for reasons given why source of evaluations are high, answers included "I could presume the place where a sound is coming from."and ,"source captions gives us the feeling of being at the place."Raters gave such comment.A reason down captions are highly regarded, is that it is thought that moving things of the animation does not overlap with the captions of the under position.As a reason for low ratings of source captions in III, raters commented that" we see the cause which sound source moved to various places."From these, it is appropriate that caption position not overlap with moving things in animation.

Conclusions
In this dissertation, we evaluated captions output with the size of the caption at 10-30% of five-phases in five kinds of animations and we evaluated five kinds of onomatopoeia giving location of captions of the source, left, right, up and down.As a result, it is appropriate that the caption size is as big as moving things of the animation.It is appropriate that caption position not overlap moving things in animation.

Table 2 .
Details of the animation gave onomatopoeia captions.

Table 2 .
Details of the evaluation grades.