Fast Image Alignment for Inspection of FCB Bump Coin System

FCB Bump Coin System is a machine that makes the bump height on the front side of FCB (Flip Chip Ball Grid Array) PCB uniform by pressing with high temperature and pressure vessels. In the process of pressing, defects such as scratches in SR (Solder Regist) area, breakages in MLCC (Multi Layer Ceramic Capacitor), etc. can occur on the backside of the product. FCB bump coin system inspects for defects on the backside of the product. It is essential for the inspection algorithm in the system to align inspection images with reference image. For fast and accurate inspection, the alignment algorithm is the most crucial element. And the image size the system provides is 4072x 3096 for which widely used methods take much time to process due to its big size. This paper introduces a method of fast image alignment for inspection of FCB bump coin system. It includes a global alignment method and a specific area selection method for fast image alignment. While the proposed method is simple, the experiment shows that it gives a fast and accurate image alignment result for FCB bump coin inspection.


Introduction
Image alignment is the process of finding the correspondence between the original reference image and the target image.Image alignment is crucial for various pattern recognition, image processing and machine vision tasks, including object tracking (1) , object recognition (2) , occlusion handling (3) , industrial inspection (4)(5) , etc.In PCB inspection, an inspection image is compared to an image from CAD to detect defects.To compare two images, image alignment is needed in PCB inspection.Many image alignment methods have been proposed for several decades and are mentioned in the survey (6) .FCB bump coin system is a machine that makes the bump height on the front side of FCB (Flip Chip Ball Grid Array) PCB uniform by pressing with high temperature and pressure vessels (7) .In the process of pressing, defects such as scratches in SR (Solder Regist) area, breakages in MLCC (Multi Layer Ceramic Capacitor), etc. can occur on the backside of the product.FCB bump coin inspection is used for inspecting the backside of the product with a fast and accurate algorithm.The backside images of FCB PCB are shown in Fig. 1.
In Fig. 1, the inspection areas are selected to look for MLCC breakages, lack of MLCC body, rotated MLCC and SR scratches.However the inspection areas exclude bump areas which are circle shaped.These products shown in Fig. 1 are not inspected by comparing CAD data, but inspected by comparing the golden reference image which is captured from the standard product.Thus, it cannot be known which specific area should be inspected for MLCC breakages or SR scratches.In order to recognize the area type, the user of the inspection system should assign which portion is to be inspected for MLCC and which portion is to be inspected for SR.The assigned result is shown in Fig. 2. The area with a red rectangle means it is an MLCC inspection area and the one with a blue rectangle means it is an SR inspection area.
After assigning the region to be inspected, it is necessary to align each inspection area in an inspection image with a reference image.If all of these areas are aligned with inspection images by the template matching method (8) , it takes much time to align the images.It cannot meet the required processing speed of the bump coin system inspection.If template matching is done on one area only, it cannot produce accurate alignment results.For example, it can be inaccurately aligned when the area with small MLCC chips, which requires more precise alignment, uses the alignment result of the area with big MLCC chips.It is not suitable that the big MLCC chip area is used to align with the small MLCC chip area because of the difference in resolution.
Moreover, MLCC chips can be shifted in the manufacturing process.For inspecting the shifted MLCC chips, each MLCC chip has to be aligned.To find the alignment position of each shifted MLCC chip, we propose a alignment method using chain data.From the reference image, an inner chain data and an outer chain data can be calculated.Using these data, each MLCC chip can be aligned proper.The required time for inspection in the system is within 1 sec, but the actual required time only for image alignment without inspection time should be within 500 msec.And since the target product is controlled in an accurate tray system, we only need to consider translation errors without having to consider rotation and scale errors.To avoid the long processing time and inaccurate results, this paper proposes the fast image alignment method for FCB bump coin system inspection.The proposed method performs global alignment between the reference image and the inspection image for fast alignment.For fast alignment, we project the whole inspection image in a horizontal and vertical direction.Using projected images, we can take the translation alignment values of x-axis and y-axis.It gives a fast and accurate alignment result because the projected images have the feature of image distribution and the dimension of projected images is hugely reduced.After reducing the searching size for aligning by global alignment, we propose a template matching method with selection of specific areas.Despite the reduction of searching areas, it takes too much time for aligning many inspection areas.Thus, the proposed method increases the alignment speed by selecting specific areas.The rest of this paper is organized as follows: We propose the fast image alignment method which is suitable for FCB bump coin system inspection.Also we show the experiment result by comparing our proposed method to other methods, and conclude our paper as well as describe future work.

FAST IMAGE ALIGNMENT FOR INSPECTION OF FCB BUMP COIN SYSTEM 2.1 Global Alignment between the Reference Image and the Inspection Image
Since the FCB PCBs are controlled in the accurate tray system, the images for inspection do not have rotation and scale errors.In this paper, we propose a method for finding a translation alignment value.For fast and accurate alignment of the inspection area, we pre-align between a whole reference image and the whole inspection image.Pre-Aligning by using the whole images can increase the speed and accuracy in the template matching as the searching area is reduced when the template matching is performed by using the inspection area.For fast alignment, we project the whole inspection image in a horizontal and vertical direction.Comparing between the projected reference image and inspection image, the distributions of image values are similar.Therefore, just by using the projected images, we can achieve an accurate alignment result of x-axis and y-axis.Also it gives a fast alignment result because the dimension of projected images is hugely reduced.
The given image size is 4072 x 3096.It takes too much time to align if the pre-alignment is performed by using such a big size image.To reduce the running time, down-sampling whole images are used.To obtain the vertical and horizontal alignment value, the down-sampled images are changed to the vertical and horizontal vector.The vertical and horizontal vectors are presented as follows.( where Gx is the mean value of the reference horizontal vector, Ix is the mean value of the inspection horizontal vector, Gx is the standard variation of the reference horizontal vector, and Ix is the standard variation of the inspection horizontal vector.And the vertical alignment value can be presented similarly to (2).
(3) Both the horizontal and vertical alignment value dx, dy are the alignment value of maximizing the cross-correlation.dx and dy are the alignment value of down-sampled images.We define that the down-sampling rate is s.Thus, the true alignment values can be approximated as (4)

Specific Area Selection for Fast Template Matching and MLCC alignment using chain data
The alignment result that is explained in the previous section can have an alignment error due to the down-sampled and projected images.To reduce such an alignment error, alignment for each inspection area is required.Based on the result in the previous section, template matching for each inspection area can be applied by setting a searching area.However, it takes too much time if template matching is performed in all inspection areas.And if the template matching is applied to only one inspection area, it can be an incorrectly alignment value.Therefore, we propose the method of selecting a specific area for fast template matching.In this paper, Normalized Cross Correlation (NCC) (8) is used for template matching.
Let us first define the conditions of specific areas for template matching.The conditions are as follows.
C1. when the size of the inspection area is big and the area is an SR area.
C2-1. when the size of the inspection area is not big and the inspection area is the first area to be aligned.C2-2.when the size of the inspection area is small and the area is an MLCC area.
The reason why C1 and C2 are separated is for a fast alignment.When NCC is applied to a big area, template matching takes too much time.Thus, down-sampled images are used for this template matching with a big area.And the reason why C2 is separated is for an accurate alignment.If C1 and C2-1 are used, the alignment can have a small error.The small error does not affect a big area, but small MLCC areas can be affected.Though the alignment only has 1-2 pixel errors, inspection result for the small MLCC area can be wrong.MLCC chips can be shifted in the manufacturing process.For inspecting the shifted MLCC chips, each MLCC chip has to be aligned.To find the alignment position of each shifted MLCC chip, we propose the alignment method using chain data.From the reference image, we can obtain the each MLCC image.We make two chain data from the MLCC image: inner chain and outer chain.Inner chain is obtained by erosion of Morphology operation and Outer chain is obtained by dilation.Inner chain is placed inside of MLCC chip, so the intensity of inner chain is high.Outer chain is placed outside of MLCC chip, so the intensity is low.Using this concept, we can formulate a cost function.
where s is the transition offset in the searching area, o is the pixel set of outer chain, i is the pixel set of inner chain, p and q are the pixel position on each pixel set, and I is the pixel intensity.It means the summation of the pixel intensity on the inner chain is maximized and the summation of the pixel intensity on the outer chain is minimized.Using (5), we can obtain the proper alignment

Experimental Result
The image size in the experiment is 4096x3072, 20 images of each of the three different models are used(total of 60).
The down-sampling coefficient in 2.1 method is set to 0.1.
When the global alignment is performed for sixty images by 0.1 down-sampling, the average run time is 11.637ms.
Comparing the inspection results with a different coefficient, it gives the same inspection results.The inspection result is presented in Fig. 3.It is evaluated by the defect type.
Comparing 2.2 method with 2.1 to 2.2 method without 2.1, the run time of 2.2 method without 2.1 is reduced as low as the time of global alignment, but the alignment performance is worse than the method with 2.1.The difference in alignment performance is shown in Fig. 4.
Table 1 shows the average run time results of the proposed method with inspection, template matching using one inspection area with inspection, and template matching without MLCC chip alignment.According to Table 1, the method using only one inspection area is the fastest and the method using all inspection areas is half as fast as the proposed method.Comparing the inspection results of the proposed method with the method without MLCC chip alignment, two methods regarding the 60 images give the same inspection results.Otherwise, the inspection method with one inspection area alignment gives 8 false alarms of 60 images.Because of this alignment error, the false alarm during inspection is occurred.Comparing the average run times in Table 1 and the inspection results, the proposed method gives faster inspection than template matching with all inspection areas and gives a better inspection result than template matching with only one inspection area.
Lastly, we compare the proposed method to other existing methods for pattern matching.The methods we compare are SIFT method which is proposed by Lowe [10] and the pattern matching method using Bayesian sequential hypothesis testing by Pele et al. [11].SIFT method takes 120.38 sec for each template.But alignment for each inspection area is required in order to reduce the alignment error.When the number of template is 9 like Fig. 2, it takes over than 1000 sec.The method using Bayesian sequential hypothesis testing by Pele et al. takes 110.5 sec.These methods take much time for extracting features from such a big size image.In the paper Pele et al. proposed, it detects with a 2197 pixel template image in a 640 x 480 pixel image.But our experiment needs to detect with a 219780 pixel template image in a 4096 x 3072 pixel image.And these methods can handle not only translation errors but also scale and rotation errors.

Conclusion
In this paper, we proposed the fast image alignment method for FCB bump coin system inspection.FCB bump coin system inspection needs a fast and precise alignment method.In order to run fast image alignment, we described the global alignment method between a reference image and a inspection image, the specific area selection method for fast template matching, and MLCC alignment using chain data.The proposed method gave a better inspection result and run time than the method using only global alignment, the method using only template matching, and the method with global alignment and template matching.The inspection machine with the proposed method has   The proposed method can useful for any applications that require translation errors only in very short time.These days, many inspection systems require a fast image alignment method.It is useful to search for a matched area in a big image with a small template image.
The proposed method can be applied to many inspection systems and it has already been used in the field of panel production process.The proposed method assumed that the inspection image had only translation.For future work, we need to research about image alignment methods for the case of small rotation error.The FCB product is mounted in the tray, so normally the rotation error does not occur.However, the rotation error can occur due to mis-grabbing tray of the motor, small size error between the tray and product, and etc.

Table 1 .
Comparison of the average run time results.