Development of General Wiimote 3D Localization Scheme and its Application on Robot Arm Attitude Detection

Indoor localization has been receiving great interest and 2D localization system based on infrared LED-based wiimote camera is also well-developed. Although 3D positioning techniques based on infrared LED detection have been proposed and validated, it is not yet matured for general indoor localization due to its insufficient sensing zone and specific layout attitude, which may not always have been satisfied. In this work, based on previous research, a layout is proposed to extend the sensing region to cover the entire indoor space by deploying many wiimote 3D pairs and make sure the object can be detected by at least two wiimote pairs during motion. However, this also implies that the original optimal deployed attitude cannot be maintained. As a result, to deal with this problem, a variable-angle dual wiimote 3D system is developed and corresponding localization performance and sensitivity analysis is carried out for clarifying the above concern. From sensitivity analysis result, better performance in X displacement detection is achieved with the orientation setting in 90◦, and the results also shows the system to be more sensitive to the error in orientation. Furthermore, the scheme is also applied to robot arm attitude determination, and shows the high possibility for this system developing into an effective external sensor for robot arm controlling. It is expected that this wiimote 3D localization system to have a great potential on inexpensive and precise general indoor localization system.


Introduction
Indoor localization becomes increasingly important due to the development of intelligent living and building automation technologies, where accurate position information is a fundamental task on indoor mobile robot navigation [1]. Navigation, control coordinated motion, and subsequent tasks of mobile robot could be achieved with the position information. Several well-known indoor localization schemes, for instance, inertial units [2], ultrasonic [3], and vision cameras [4], have already shown their performance. However, due to restrictions of position timing, position accuracy, and complex indoor environments, a flawless positioning technique has not yet been achieved.
IR indoor positioning, which determines the location by projecting and receiving a modulation IR line with optical sensor devices, such as wiimote CMOS cameras, is another approach. This approach can be relatively accurate in 2D indoor localization applications and it already demonstrated its performance in various applications [6]. Li et al developed 3D localization scheme based on dual wiimote layout. This scheme contains one pair of wiimote cameras, which is assembled based on the CMOS camera built in Wii Remote. The 2D position of infrared LED is captured by the cameras in the form of pixel coordinates, and transformed into visual angle corresponding to the CMOS camera.
Although the feasibility has been demonstrated, in comparison with the wiimote 2D localization, this wiimote 3D localization scheme [7] [8] shows inferior performance in both the available sensing zone and the achievable accuracy in detection.
Chan et al [9] further utilized dual wiimote 3D to improve the sensing resolution. The scheme, called as the fixed-angle dual wiimote 3D localization, consists of placing two wiimote 3D pairs in opposite direction, the sensing resolution can be improved. The function of the additional pair of wiimote cameras is basically to offer another set of position data to compensate the inaccurate data of the original pair of wiimote cameras. Based on this investigation, a scenario is drawn to localize mobile units in entire indoor space is drawn in Fig. 1(a) and is expected to further expanded to a general indoor localizing system as shown in Fig. 1(b). That is, by deploying many wiimote 3D modules to guarantee a moving object is seen at least by two wiimote 3D module simultaneously, the sensing zone can be expanded. However, this also implies that the specific optimized layout proposed by Chan cannot be met in this case. The major concern of the newly proposed scheme, called as variable-angle dual wiimote 3D localization, is therefore the sensing performance against the layout geometric parameters. Therefore, the performance variations with different  layout parameters should be investigated. By evaluating the sensitivity of the wiimote 3D scheme against deployment attitude variations, it is possible to optimize the sensor layout inside the workspace.
In this work, both simulation and experiments are carried out to address the above sensor robustness concerns for the variable-angle dual wiimote 3D localization scheme. Two geometric variables, distance and orientation between coordinates of wiimote cameras, are mainly the key parameters focused in this paper. Furthermore, experimental results are then compared with the simulation results for the purpose of validation. Finally, after validation, a robot arm with several IR LEDs are then served as a platform for demonstrating the effectiveness of the variable-angle dual wiimote 3D localization by detecting the attitude of the robot as the first step to be integrated as an external position sensor for robot control.
The rest of the paper presents the technical detail of the development of wiimote 3D localization. In section 2, the background on wiimote 3D localization is briefly reviewed for providing the background of this work. Section 3 describes the robustness analysis, and in section 4, robot arm attitude detection experiments, which are verification of the schemes mentioned in section 2. Consequently, discussion and conclusion are made in section 5.

Wiimote 3D Localization System
Wiimote 3D localization system developed from single wiimote 3D localization system, which contains only one pair of wiimote cameras [7] [8], fixed-angle dual wiimote camera system, with an additional pair of wiimote cameras for data compensation [9], and to variable-angle dual wiimote 3D localization system, which allows more flexiblew deployment. These systems will be described in the following sections.

Single Wiimote 3D Localization System
Based on the success of wiimote 2D localization, 3D localization based on wiimote was proposed and validated by Li and Chen [7] [8] using a pair of wiimote CMOS camera in a stereo vision fashion. Schematically shown in Fig. 2, with the known distance between the two CMOS cameras, the pixel coordinates detected by these cameras are eventually converted to the physical location of the detected IR LEDs based on geometry and kinematics. However, since the visible angle of a single CMOS camera is 32 • in vertical and 42 • in horizontal directions, which is already not big enough, the valid zone for wiimote 3D localization is limited on the intersections of detected zones from both cameras. As a result, the sensing zone of wiimote 3D is actually much smaller than that in 2D localization. The scheme has successfully demonstrated its applicability on determining the attitude of a four-bar linkage mechanism and a selfmade robot arm. However, since the pixel resolution is actually distance dependent, the performance of this scheme quickly became worse if the distance between the IR LED and the CMOS camera pair exceeds 1.5m.
To demonstrate the distance-dependent localization error, a simple experiment is carried out. As shown in Fig. 3, infrared LED is placed 80 centimeters away from wiimote cameras and is moved in an additional 80 centimeter away from its original position in Z direction and moves back and forth in X direction in the range of negative 10 centimeter and positive 10 centimeter. Comparison between the captured 3D position of the infrared LED and the real  infrared LED position is shown in Fig. 3. Difference between these sets of data increased while the infrared LED moves away from the wiimote cameras. To improve this disadvantage, Chan et al proposed a scheme to use two wiimote pairs and placed in opposite direction to reduce the spatial dependent error. Such fixed-angle dual wiimote 3D localization scheme has also demonstrated its applicability elsewhere [9] and a brief test is performed below.

Fixed-Angle Dual Wiimote 3D Localization System
As mentioned above, an additional pair of wiimote cameras, hereinafter referred to as "wiimote camera 2", is placed on the opposite side to the original pair of wiimote cameras, hereinafter referred to as "wiimote camera 1", shown as Fig. 4. While the Z direction distance between the infrared LED and wiimote camera 1 is detected to be further than the distance between infrared LED and wiimote camera 2, the system will take the 3D position data captured by the wiimote camera 2 as a compensation [9] and the data selection depends on certain criteria such as the RMS of data fluctuation. Experimental results, as shown in Fig. 5, show that this system is capable to improve the inaccuracy problem of single wiimote 3D localization scheme. By this we learned that the localization accuracy can be improved using multiple wiimote 3D pairs. The CMOS cameras are very cheap and it is no problem to deploy as much as possible in a living space such that a moving object can always be seen by at least two wiimote 3D pairs. However, in practical sense, it is virtually impossible to fix the arrangement of the dual wiimote 3D pair in a fixed angle sense in any times. As a result, a variable-angle dual wiimote 3D localization scheme is developed to enhance the flexibility in wiimote camera setup to increase the adaptively for using in various indoor environments.

Variable-Angle Dual Wiimote 3D Localization System
As addressed above, the ultimate goal of this research is to build up a general large area indoor localization system such that the moving object can be seen by at least two wiimote 3D pairs at any locations and any moments, including passing certain shielding location where the object may not be observed by some wiimotes. However, this implies that the configuration of the dual wiimote 3D can- not have an optimized fixed angle just like Chan's improvement [9]. As a result, the sensitivity or the robustness of the dual wiimote localization scheme against is important. If the localization performance is not too sensitive to the deploy angle, the whole indoor localization would be more consistent. On the other hand, if the performance is too sensitive to the deploy angle, special care should be prepared for maintaining a consisted performance. In order to address the concern shown right above, a test system with adjustable deploy angle and distance of wiimote 3D module is designed to perform the sensitivity analysis. The system is schematically shown in Fig. 6. The object, containing an IR LED, will be mounted on a linear motor with known position to serve as the standard. Wiimote camera 1 is deployed in the same position as previ-ous setup (at world coordinate (0, 0) and the orientation is in Z direction) but the wiimote position and orientation of wiimote camera 2 can be adjusted (its location is at world coordinate (a, b) with a coordinate rotation of θ) . Through the detected IR LED signal, both wiimote 3D pairs will also estimate the positions. With different deployment manner, the localization accuracy is then compared.
Coordinate transformation is a key calculation for such positioning system revolution. 3D position of the infrared LED is sent back to the computer in the coordinate which takes wiimote camera 2 as main reference. Coordinate transformation is later processed using LabVIEW. The transformation is based on the setting of wiimote camera 2, including position (a, b) and the coordinates rotation of θ. The position captured according to the coordinates of wiimote camera 2 is transformed into the position which takes the coordinate of wiimote camera 1 as the main reference. Since the purpose of the development of this system is to improve the capability of this localizing system to adjust to various conditions, multiple choices of settings of wiimote cameras might be available in the same time. As the result, the method to determine the preferred choice with best performance is required. Therefore, the robustness related to the deployment of wiimote camera 2 is discussed. Related robustness analysis is carried out and will be mentioned in detail in the next section.
Further analysis and experiments corresponding to variable-angle dual wiimote 3D localization is executed for the discussion and demonstration of the performance of this system. To determine the best setup of wiimote cameras, robustness analysis is carried out and discussed separately in angle and position perspective, which are mentioned previously as the main settings of wiimote camera 2. Besides, the application of this system on robot arm attitude detection is implemented as a practical utilization of this system in indoor positioning purpose. These researches related to variable-angle dual wiimote 3D localization will be presented in detail in the following sections.

Analysis and Experimental Results
Robustness analysis is important for the development of the general indoor localization. Here, robustness of variableangle dual wiimote 3D localization is discussed in two geometric variables, the orientation and the position of the deployment, which are the main setting parameters of this system. Simulation and experiments are carried out under the assumption that only one variable, either orientation or position, is mistakenly taken into position calculation. Simulation and graphs are performed utilizing LabVIEW. The detailed simulation and experiment procedure will be clarified in the following subsections.

Simulation Results
The overall simulation flow is shown in Fig. 7. The simulation is discussed related to two main deployment parameters, the angle setup and distance setup. All setup of both wiimote cameras are assumed to be in the expected condition, except for either the angle or the distance. Since the error in single variable setting is unknown in advance, the parameter used in the calcula-  tion of the 3D position of infrared LED is still the expected value. As the result, the final result of positioning appears to be inaccurate. Finally, the error of position in each error magnitude is accumulated and built into a sensitivity curve shown in Fig. 8. and Fig. 9. Different curves with different basic parameter are placed in the same figure for comparison. The "degree"s shown in legends of Fig. 8 indicate the setting of both the expected orientation and the position of wiimote camera 2. For example, the "60 degree" shown in Fig. 8 means wiimote camera 2 is placed in the position (16.08, 30), the position where takes 120 cm as radius and the mid-point as center rotates with the angle of 60 • .
The simulation results are shown in Fig. 8 and Fig. 9. As shown in Fig. 8(a), if we take ±5 • as typical angle setup uncertainty range and discuss the influence of the angle setup uncertainty, the 90 • angle setup case shows better robustness in X displacement detection performance than other angle setups, since the related average positioning error is generally smaller than other cases. In Fig. 8(b), the 0 • and 180 • cases show better robustness in Z displacement detection performance comparing to other cases. Besides, when discussing distance setup uncertainty, in Fig. 9(a), the 90 • case has worse X displacement detection robustness. In Fig. 9(b), the 0 • and 180 • cases are more sensitive to the distance setup uncertainty in Z displacement detection performance. In the other hand, recording the average positioning error of both angle and distance setup uncertainty range limit of each orientation setup, Fig. 10 is drawn with the orientation setup in the horizontal axis and the average positioning error as the vertical axis. From this figure, the similar discussion of simulation results could also be made.
From the simulation results, when discussing angle setup uncertainty, the 90 • angle case has better robustness in X  displacement detection, and the 0 • and 180 • is less sensitive to angle uncertainty in Z displacement detection. While discussing the influence of distance setup uncertainty, the system is more sensitive to distance uncertainty under 90 • in X displacement detection, and the system has worse robustness in Z displacement detection under 0 • and 180 • angle setup.

Experimental Results
Corresponding experiments are carried out for comparison of the simulation mentioned above. As shown in Fig. 11, IR LED is placed on a 2-axis Yokogawa linear servomotor and moves from (0,80) to (0,160). The wiimote cameras are deployed in both "90 degree" and "135 (−45)" cases in order to compare with the result concluded from the simulation. The experimental results, as shown in Fig. 12, show that, while discussing the influence of angle uncertainty, the X displacement results show close relationship to the simulation results. However, in Z displacement discussion, the results cannot be distinguished easily. Besides, while discussing the effect of distance setup uncertainty, as shown in Fig. 13, the X displacement experimental results are also close to the simulation results, but the Z displacement results do not show the same phenomenon.
From the comparison of the simulation and experimental results shown above, we can briefly conclude that the system has better X displacement detection robustness when   the angle setup is 90 • . What's more, despite the difference between the simulation and experimental Z displacement detection results, the system can still be defined to be generally more sensitive to angle setup uncertainty comparing to distance setup uncertainty.

Robot Arm Attitude Detection
The usage of robot arm increases extremely nowadays due to the promotion of Industry 4.0. As the results, the detection of the robot arm attitude is one of the required application in indoor localization. In this research, to meet the common request, a demonstration of robot arm attitude detection is conducted utilizing the systems of the wiimote 3D localization scheme.
In this application, the desired observing points of the robot arm are the ankles and the object held by the gripper. This setting shows the requirement of ordering the observed points while positioning in order to ensure the correct information of robot arm attitude. To meet this situation, IR LED switching technique [7] [8] is applied to this demonstration. Besides, to conquer cases that the robot arm may not be visible due to obstacles and various attitude, variable-angle dual wiimote 3D localization system is applied in this demon- stration, as shown in Fig. 14. In this application, three IR LEDs are located on the three ankles of the robot arm and one is located on the gripped object, as shown in Fig. 15(a). The LEDs are controlled to light in sequence in order to determine its actual representing location. Variable-angle dual wiimote 3D localization system is utilized as the main system for positioning. The angle setup of this system is -45 • and the distance setup is 180 cm. In this experiment, the robot arm is first turn to face wiimote camera 1, in this case, the robot arm is invisible to wiimote camera 2. Later, the robot arm is commanded to face wiimote camera 2, which makes the wiimote camera 1 unable to detect the manipulator's attitude. In Fig. 15(b), the results of detection are shown. The IR LEDs successfully show each representing location and the location data captured by the wiimote camera 2 can be adopted to compensate the missing data of wiimote camera 1.
Since the result of variable-angle dual wiimote 3D localization system has shown as a valid robot arm attitude detection, in the near future, general wiimote 3D localization will be applied to the detection of robot arm attitude in order to realize the concept of general wiimote 3D localization system. It is believed that with a higher resolution and accuracy, this system is capable to work as an external sensor detecting both the robot arm and the object held by the gripper for the feedback control of robot arm while the  current sensor for such control is unable to take the gripped objects into consideration, especially for cases when the objects are flexible and in various shapes.

Discussion and Conclusions
The development of intelligent living and building automation leads to spotlights on indoor localization. As the result, a series of research on wiimote 3D localization is executed in order to establish an inexpensive and precise indoor localization system. The Single wiimote 3D localization is initially developed as an application of 3D positioning utilizing Wii Remote, but is later replaced with fixed-angle wiimote 3D localization system due to the inaccuracy caused by the increase in distance. However, the limitation of the angle deployment is not suitable for the demand of flexibility of allocation corresponding to the various conditions of indoor environments. Hence, variable-angle dual wiimote 3D localization system is developed to meet the requirement. Furthermore, robustness analysis is carried out for the determination of the deployment preference. The results show that the system has better robustness in X displacement detection under 90 • angle setup when discussing the effect of the angle setup uncertainty, and it is generally more sensitive to angle setup uncertainty comparing to the effect of distance setup uncertainty. In addition, robot arm attitude detection is carried out as a practical application and the result shows this system as a valid positioning approach. In conclusion, with further research on wiimote 3D localization, we believe this simple and effective system has great potential on the establishment of indoor localization system and external sensor for mobile robots.