Overcome the long distance : A universal method for sea wave matching

A sea wave height measurement method based on stereo vision is proposed in this paper. This method uses two cameras taking photos of sea and calculates the sea wave height with pairs of images. In the step of obtaining 3-D sea wave information in long distance (over 3Km), we proposed a Block extraction method for sea wave extraction and a Multi-constraints sea wave matching method for corresponding points matching. Block extraction extracts sea waves from divided blocks of sea areas and in each block set thresholds according to the gray scale distribution in each parts. Multiconstraints sea wave matching method matches sea waves with multi constraints and corresponds points based on corresponding sea waves. Experimental results support the applicability of this sea wave extraction method in long distance. Matching results show the correct of the sea wave matching method.


Introduction
As a coastal country, Japan has been frequently affected by tsunami.After the huge damage caused by the tsunami at East Japan Earthquake in 2011, it has become urgent to establish a system that forecasts tsunami.To detect the formation and to estimate the arrival time of tsunami, we proposed a sea wave height measurement method based on stereo vision.The challenge is to obtain 3-D information required in calculating the height of sea waves from sea images.Due to the non-feature of sea and irregularity of waves, we divided the information obtain into two steps, sea wave extraction and sea wave matching.Previous research on sea wave extraction had been done at close distance(less than 100m).Sea waves can been seen very clearly in close distance images, extraction can be done by setting double thresholds to gray scale distribution of the image.However, when the distance extends to long distance (over 3Km), this method cannot be applied due to the complexity of images (1,2) .Fig. 1 shows the comparison between images in close get visually smaller that only the white top can been seen.Besides, as the range of sea zone extends in long distance image, sea zone presents in different position.To improve the extraction method, we have proposed Block extraction method.By dividing sea zone into several same-sized blocks and setting double thresholds in each block, we can eliminate the influence of brightness difference.
In the sea wave matching step, two main challenges needed to be pay attention to: one is that the sea wave has no particular features, and the other is to each single feature sea waves show similarity.Directing point match methods (3) do not apply to this system.Thus we try to match sea waves first instead of point matching directly.In close distance, the base line between two cameras is short the shapes of sea wave remain almost the same in left and right image, template matching method using gray scale can correspond sea waves correctly.With the distance extends, the base line between two cameras extends, the shape of sea waves change a lot in left and right image.Simply using gray scale template matching (4) can hardly obtain correct matches.We propose a multi-constraints sea wave matching to solve this problem.In this method, we set several constraints to limit the searching area.While searching corresponding sea wave from right image for left image, using position and size as constraints and rule out some interference and then use shape similarity to obtain the best match.This paper presents the theory of these two methods and shows the results of them.

Methods
The system we proposed for sea wave forecast is based on stereopsis technology which requires two cameras.Use two cameras to take photos of target sea zone as Fig. 2 shows, obtain the information of sea waves from the photos.Send photos to computer and the image processing flow is shown in Fig. 3, use two cameras which are exactly the same to take photos from sea, extract sea wave from pairs of images and match sea waves from the extracted sea wave images.

Sea wave extraction
When the distance extends to over 1 km, the shadow of sea waves cannot be seen from the images, however white top of sea waves still can be seen clearly from Fig. 4 (a).We choose the white top as extracting object.As the sea zone is normally single color and contains very few color information.Extraction process according to RGB channels shows no advantages but a waste of time.
Here   refers to the maximum gray scale of the image and ̅ is the average gray scale of the image.For situations contain not only sea, we rule out the interference first before extraction.

Sea wave matching
To calculate wave height, we need to find corresponding points correctly firstly.However, sea waves are irregular and usually present in different shape which makes it very hard to get particular obvious features.Existing feature-based matching methods like Scale-invariant feature transform (5) (SIFT) shows inadaptability to sea wave matching.We find it better matching the corresponding sea waves and then find the Noticing the same sea wave shows relatively the same size in the left and right image despite of the difference in shape as Fig. 5 shows.Also the relative position in left and right image does not change too much.As none of the features is obvious enough to be a matching standard.We try to mix all the single feature and use several features as constraints and find this proposal feasible.Thus the multi-constraints sea wave matching method was proposed.
In this method, we use three feature constraints: size, position and shape.And the parameters of these features in this method are as followed.
For the extracted image, label all the separate sea waves and calculate the size of each sea wave.Meanwhile obtain the gravity of each sea wave which is used to mark the position.Fig. 6 shows the definitions of these features.
For a certain sea wave SW i as Fig. 6 (a) shows, S i represents the size of SW i , G i (x 0 ,y 0 ) refers to the gravity position of SW i , here N is the total number of sea waves.Meanwhile, obtain the minimum and maximum value of this sea wave in x-axis and y-axis, here x 1 and x 2 refers to minimum and maximum value of this sea wave in x-axis, y 1 and y 2 refers to the minimum and maximum value of this sea wave in y-axis.Define such a template as Fig. 6 (b) shows, here (2.5) Consider sea waves as matching patterns, the process of sea wave matching is simplified to a template matching problem.For each sea wave in left image, matching is a process of searching area that match to a template sea wave best in right image.Different from the traditional template matching, we use the extracted binary images to do the comparison other than the gray scale image which simplifies the calculation process.
Before giving the summary of this method, first we define the similarity criteria for position, size.Note sea wave in left image as SW l and SWr in right image.The gravities of SW l and SW r are G l (x l0 ,y l0 ) and G r (x r0 ,y r0 ).(H l1 ,H l2 ,W l1 ,W l2 ) refers to the template size for SW l .For SW l in left image, the position similarity criteria is The size similarity criteria is Now we give the summary of this method.1) For a given pair of image, filter those small sea waves by setting a size threshold  ℎ .Reserve those sea waves whose sizes are bigger than the threshold.2) For each sea wave template in left image, search the possible matches in the right image according to position.As in stereo matching, the corresponding points lie in epi-polar line, with the camera data from calibration mentioned before, we can limit the searching area to a band region as Fig. 7 (a) shows.

Experiments and Results
In order to verify validity of Extraction and Matching method of sea wave from the long distance image, a series of experiments had been done.Two CCD cameras of high sensitivity are used; the resolution of each camera is 1920x1080 pixels.In order to take the wave clearly, two 500-mm long focus lenses are used.The distance between two cameras is 8 meters, and object distance is about from 3 km to 4 km.The setting height of two cameras is about 14 meters above the sea.See axis of two cameras is adjusted according to the above distance parameters.One set of ordinary PC is used for the image input and image processing, and the image input speed is 30 sheets per second.The photography scene can be classified into 3 main types: image with only sea, image with sea and sky, image with sea and island.

Results of sea wave extraction
Here gives the extraction results in different situations, Fig. 8 is the result of only sea which targeted at 3 KM, Fig. 9 is the result of sea and sky which targeted at 4 KM, Fig. 10 is the result of sea and island which targeted at 3 KM.Though all the extraction is done in sea zone, the process of removing interference object has some influence on the extraction results.In the situation with only sea, the sea waves can be extracted mostly and the shapes can be saved according to result in Fig. 8.The bigger the sea wave is, the more complete the shape can be saved.The brighter the sea wave is, the more complete the shape can be saved.While the image contains sky and sea the extraction around the boundary between sky and sea show inaccuracy, some extracted parts are not real sea waves but the brightness difference caused by sky.In the situation with island, the area close to the island may be extracted wrong parts.However, most sea wave can be extracted correctly and the shapes can be saved.

Results of sea wave matching
Though we had conducted several experiments, in this paper, we choose the most recent experiment data as examples.The parameters set this time are as followed.n 1 =50; n 2 =50; S thre =500(pixel); Here we choose 3 pairs of images which targeted at 4 KM from different time.Fig. 11 was taken at 11:30 A.M, Fig. 12 was taken at 14:30 and Fig. 13 was taken at 17:30.The matching results show that after filter those very small sea waves, the left ones can be matched correctly, especially for those with relative big size.The bigger the size is, the more similar the shapes are in left and right image, thus the match of those sea waves can obtain higher precise.
However, limited to the precise of extraction, there are some wrong matches.One problem of this method is that due to the search is one direction (in this paper, left to right), more than one sea wave in left image may match the same one in right image.
Finally, we give the sea level trend calculated with the proposed methods in this paper.Fig. 14 (a) is the sea level trend calculated by the proposed methods and (b) is the sea level trend data from the government database.The trend we calculated from 6:00a.m to 18:00p.m corresponds to the real sea level trend from the government data base which supports the accuracy of the proposed methods.

Conclusions
In this paper, to solve the problems occurred during the case of long-distance sea measurement such as sea wave extraction and sea wave matching; we proposed Block extraction methods for the former and Multi-constraints sea wave match for the latter.
As distance extends, a series problems occur and situations became complicated.Extraction target in close distance cannot be seen in long distance image and extended range contributes to obvious brightness difference.By changing the extraction target and dealing extraction in blocks we can extract sea waves despite of brightness difference.The consistence on gray scale distribution enables us to use double thresholds in each block.
The proposed method for sea wave match are actually a process of downsize the search area and reduce the influence from other sea waves.Matching method at close distance has a limit on the matching numbers of sea waves, Multi-constraints sea wave match fix this problem and can match quantity of sea waves.However, this method is affected directly by the extraction results precise and the quality of images may also influence the match precise.
The proposed methods can solve the problems occurred with extended distance and fix the inadaptability for the methods in close distance.Experiments done at long distance (4KM and further) support these two methods.The sea waves can be extracted from the images and using the gravity from the matching sea wave to calculate the sea wave height.Sea level trend graph at 4KM with information obtained from pairs of photos using these proposed methods shows correspondence to the real sea level trend.The proposed methods can overcome distance and extract information from long distance images.

Fig. 1
Fig.1 The comparison between close and long distance

Fig. 4 (
b) is the gray scale image.The contraction between sea wave wand the non-wave sea zone is very obvious which makes sea waves extraction easier.Meanwhile, this single channel process can save time thus the extraction is done through gray scale distribution.Noticing he image presents different brightness in long distance as Fig.4 (c) shows, we divide the sea zone into several same blocks to eliminate the influence of brightness difference.Investigations on each divided block from a quantity of images suggest the gray scale distribution on each part is Gauss distribution despite of the difference of gray scale range.Fig.4 (d) gives the distribution in each part, thus we draw an empirical extraction formula, as written in equation (2.1).

Fig. 4
Fig.3 Image processing flow Fig.5 Left and right image at long distance