Perception of Approaching Objects in Bilateral Control Using Proximity Sensor

In this paper, a new method for the perception of approaching objects in bilateral control using a hybrid proximity/force sensor is proposed. Recently, a demand for teleoperation which can accomplish safe, compliant and complicated tasks in a remote area is increasing. One method for teleoperation is bilateral control which can control both position and force in the master and the slave side. In bilateral control, an operator usually gets the environmental information of the slave side with vision sensors. However, in typical conditions, for example, when the interrupting objects exist between the vision sensor and end-effector like the body of the manipulator, the operator cannot observe the objects near the end-effector. So in this paper, for getting the complimentary information about environments, a new method for the perception of approaching objects in bilateral control using a hybrid proximity/force sensor is proposed. The hybrid proximity/force sensor can measure both distance between end-effector and objects and contact force. The approaching objects can be detected before objects touch the end-effector with this sensor. The operator can feel the approaching objects by vibrating only the manipulator on the master side. Hence, the operator can realize safe and soft contact with the hybrid sensor even when information of the vision sensor cannot be used. The effectiveness of the proposed method is verified by the experiments.


Introduction
Recently, the demand for safe and compliant motion control for robot manipulators is increasing because of the recent trend for robots working with a human. For these robots, the perception of the surrounded environments by sensors and the sophisticated control strategy is needed to realize the required motion. An example of these robots is the robots with bilateral control (1) . Bilateral control is one of the methods of teleoperation and humans can operate the objects in a remote area. Hence, the human can operate the objects in dangerous area, such as space, nuclear plants, deep sea and so on. By using bilateral control, the human can accomplish complicated and various tasks in remote area.
In bilateral control, the 4-channel bilateral control system is widely used (2) . In this control strategy, the information of the position and force is communicated between the master and slave side. Especially, the acceleration based control system can realize the high performance teleoperation. And the robustness can also be obtained by utilizing the disturbance observer (DOB) (3) , which calculates and compensates the disturbance including contact force, gravitational force, friction and so on. Furthermore, in direct drive, the contact force can be calculated without force sensor, by the Reaction force observer (RFOB) (4) .
In general, in bilateral control, cameras are utilized to observe surrounded environments. And the operators can operate the objects in the slave side. However, in typical condition, the operators cannot observe the objects near the sensor, for example, when the interrupting objects exist between the camera and end-effector like the body of the manipulator. In this situation, the operators cannot detect the objects until they touch the slave robot.
Hence, in this paper, to get complementary information about environments especially before the robot contacts the objects, a new method for the perception of approaching objects in bilateral control using a hybrid proximity/force sensor is proposed. The proximity sensor is widely studied to observe the object before the object is touched by the manipulator. As for the proximity sensor, various types of proximity sensors have been proposed.
For example, the flexible proximity sensor (5) , Net-structure proximity sensor (6) , matrix type proximity sensor (7) are proposed to realize safe robot-human contact.
And the proximity sensors attached to the robot hand is also proposed (8) . The grasping of the various types of moving object with the robot hand is realized by using photoreflector and Time-of-Flight (TOF) Sensor (9,10) .
In the proposed method, the novel hybrid proximity/force sensor with photoreflector is utilized. The hybrid proximity/force sensor consists of the photoreflector and the polyurethane layer and is attached to the end-effector of the motor. With this sensor, the distance between the sensor and the object and the contact force is calculated by these materials. Hence, approaching objects can be observed even when interrupting objects exists between the camera and end-effector. And the contact force can be observed when the robot has a high reduction ratio mechanism. When the objects approach the objects on the slave side, the master side vibrates and the operator can perceive the approaching objects. The amplitude of the vibration becomes large as the distance between the sensor and objects becomes small. The vibration in the master side is eliminated when the force and the position information is sent to the slave side by the lowpass filter. Hence, only the master side vibrates with the approaching objects, and the operator of the master side can observe the approaching objects without any additional mechanism. Finally, the effectiveness of the proposed method is verified in the experiments about bilateral control. This paper is organized as follows. In section II, the hybrid proximity/force sensor is described. In section III, the controller design of the proposed method is described.
To verify the effectiveness of the proposed method, experimental results are shown in section IV. Finally, the conclusion of this paper is described in section V.

Hybrid Proximity/Force Sensor
In this section, an explanation of the hybrid position/force sensor is described. The appearance of the sensor is shown in Fig. 1. And the detailed structure of the sensor is shown in Fig. 2. The size of the sensor is 15mm * 15mm. The position/force sensor can be separated into three layers. At first, in the base layer, the photoreflector which measures the distance from the object to sensor by using infrared ray is attached. In the middle layer, the transparent acrylic resin is attached in order to attach the surface layer appropriately. Finally, in the surface layer, the polyurethane sheet is attached. The Thickness of the sheet is 5mm in this paper. The feature of the sensor can be easily changed by changing the hardness of the polyurethane. In this case, the relatively soft polyurethane is selected, so when objects contact the sensor, the position of the surface   of the sensor is changed. As mentioned above, the distance between the sensor and the object is calculated by using the photoreflector. In the phororeflector, the phototransistor measures the illuminance of the infrared ray emitted by the LED. And the illuminance can be calculated as the value of the output voltage of the phototransistor. Generally, the relationship between the distance and the output voltage !"# can be expressed as where , !$$ , and !$$ is the coefficient and offsets.
This equation is based on the inverse-square law of illuminance (11) . These parameters are different from the materials of the object. In non-contact motion, based on the calculated distance, the sensor can observe the distance between the sensor and the object. Hence, in this situation, the sensor works as the proximity sensor. And in contact motion, the object is getting into the surface layer. The image of the contact force and the displacement is shown in Fig. 3. In this situation, the reaction force by the polyurethane is burdened to the sensor and the object. As the distance between surface of the polyurethane and photoreflector is constant, displacement of the surface, described as ∆ in Fig. 3. can easily be calculated.
Hence, under assumption that the stiffness of the polyurethane is constant, the contact force of the object can be calculated as where denotes the stiffness of the polyurethane. Hence, in this situation, the sensor works as a force sensor. As mentioned above, this sensor can work as both a position/force sensor in each situation. Mainly in this paper, this sensor is utilized for the perception of the approaching objects, as the proximity sensor.

Hybrid Proximity/Force Sensor
In this section, the controller design of the proposed method is described.
The block diagram of the controller is shown in Fig. 4.
, ○ /, ○ / denote the reaction force, friction, position, current, command value, response value, reference value, compensated value, estimated value and nominal value. And ○ + , ○ -denote the master side and the slave side. The block diagram of the disturbance observer is also described in Fig. 5. In the bilateral control, the relationship between master and slave is described as And based on position PD controller and the force controller, the force reference can be expressed as

̈-
where . , , , . denote position gain, velocity gain, force gain of the controller. However, different from the standard bilateral control, the vibrated command is applied to the command in master side in the proposed controller. By adding this applied command, the approaching object can be detected for the operator before the object touches the end-effector. Hence, the operator can reduce the speed  of the master robot before touching the object, and can reduce the impact force when the motor touches the object. In this method, the applied vibrated command 0*1 is described as Where (+. , 0*1 , -denote the amplitude of the vibration, the period determination coefficient and the sampling time of the controller. The larger amplitude (+. is desired when the distance becomes smaller to transmit the approaching object appropriately to the operator. In this paper, the amplitude (+. is set to where #2 and denote the threshold and the coefficient. In (1), the amplitude (+. is zero when the object is relatively far from the sensor and when the object touches the sensor. And (+. is proportional to the distance, so the operator can feel the large vibration when the distance becomes smaller and can detect the approaching object. However, only applying this vibrated command, the slave side may also be vibrated because the position and force of the master and the slave side are synchronized as shown in (3) and (4). Hence, when the information of the master position is transmitted to the slave side, the vibration about position and force is eliminated by lowpass filter as shown in (9).
where 6,*$$ denotes the cut-off frequency of the LPF. And setting to the low cut-off frequency of the RFOB, the force response vibration is also eliminated when the contact force is transmitted to the slave side. In proposed control, the position relationship between the master and the slave is different from the (3), and is described as (10) is the position relationship of the master side, and (11) is the counterpart in slave side. By this control strategy, only the master side vibrates by the vibrated command, and the operator can detect the approaching object before the object touches the sensor.

Setup
In this section, the experiments about the proposed method are described. To calculate the distance between the hybrid sensor and object, sensor calibration is needed. Hence, at first, the preliminary experiments about calibration are described. After that, experiments about the proposed method are described. The experimental setup is shown in Fig. 6. The parameters of the experiment are described in Table 1. Two linear motors were utilized in the experiments, one was for the master robot, and the other was for the slave robot. And the hybrid proximity/force sensor was attached to the end-effector of the slave robot. As a photoreflector RPR-220 was utilized, and the A30 polyurethane was utilized as the surface layer. In this section, the iron block was utilized as the environment. In order to measure the contact force, a force sensor was attached next to the iron block. The value of the contact force by force sensor was utilized only to calibrate the contact force, not to control robots.

Calibration
Before experiments, the calibration about the photoreflector should be done in order to calculate the relationship about the photoreflector and distance or contact force. In the calculation step, the relationship of distance and illuminance was measured by moving the linear motor of the slave side with the constant speed, 2 mm/s. The measured relationship is shown in Fig. 7. Based on Fig. 7, the coefficient of (1) was calculated. And the relationship between calculated distance and actual distance is shown in Fig. 8. As shown in Fig. 8, the calculated distance was well fitted to the actual value, especially in the small distance zone. Same as position calibration, the contact force calibration was also executed. The relationship between the calculated force and the contact force measured by the force sensor is shown in Fig. (9). The same as position calibration,    the force calibration was also fitted to the actual value. In this paper, the important point is the calculation of the distance. Hence, the value of the calibrated distance is only considered in this paper.

Bilateral Control
In this section, based on the calibrated value, the verification of the bilateral control with hybrid sensor is described. The operator moved the master robot and tried to touch the object. And the position and force response with/without the proposed controller was compared. At first, experimental results without proposed controller are shown in Figs. 10 and 11. As this is a very standard 4ch bilateral controller, the position and the force of the slave side were well followed to the master side. And the experimental results with proposed method are shown in Figs. 12 and 13. As shown in These figures, the position and the force of the slave side were well followed to the master side with the proposed method the same as standard 4ch bilateral controller. And the calculated distance by the photoreflector is shown in Fig. 14. Based on this calculated distance, the amplitude of vibration was determined and the vibrated command is added to the master side. The enlarged views about Fig. 12 are shown in Figs. 15 and 16. As shown in Figs. 15 and 16, the vibration was added to the master side without transmitting the vibration to the slave side. And the amplitude of the vibration became larger when the calibrated distance was smaller. Hence, the operator was detected the approaching object before reaching to the object without any additional actuator.
And to verify the effectiveness of the proposed method, the additional experiment was performed. The setup of this experiment in the slave side is shown in Fig. 17. In this experiment, the small paper box was putted in front of the end-effector. And the operator pushed this paper box by using master robot in standard and proposed bilateral controller. The calibration was executed beforehand. And #2 is set to 0.02 m. The slave side was hidden by the iron plate for the operator.
The time-lapse of the experiment with/without proposed controller is shown in Fig. 18. In Fig. 18, the time interval is about 0.5 s. Without proposed method, the operator could not feel the approach of the paper box before the sensor touches to the object. Hence, the slave side    touches to the object with high speed and contact force. As a result, as shown in Fig. 19, the shape of the paper box is changed. In contrast, with the proposed controller, the operator could detect the object before contact. Hence, the operator could reduce the speed of the master robot and realize the soft touch to the paper box as the vibration was stopped when the distance was reaching to zero. As a result, the operator could touch to the paper box without changing its shape.

Conclusions
In this paper, a new method for the perception of approaching objects in bilateral control using a hybrid proximity/force sensor is proposed as one application for the hybrid sensor. The hybrid proximity/force sensor consists of the photoreflector and the polyurethane sheet. Both contact force and the distance between sensor and object can be measured by this sensor. In the proposed bilateral control, the hybrid position/force sensor is attached to the slave side to detect the approaching objects. When the sensor detects the approaching objects, the sensor calculates the distance between the sensor and the object. And the vibrated position command is added to the master side whose amplitude is becoming larger when the distance becomes smaller. Hence, by the vibration of the master side, the operator can detect the approaching object. The filtered position/force response of the master side is transmitted to the slave side not to transmit the vibration of the master side. The effectiveness of the proposed control is verified by the experiments. And the application for manipulators that cannot estimate the reaction force by RFOB is one feature work to be implemented. Furthermore, in the proposed method, the role of the proximity sensor is only utilized for control strategy. Hence, by utilizing both force/proximity roles, the more safe and sensuous control strategy may be realized.

Acknowledgment
This work was supported by JSPS KAKENHI Grant   Fig. 18. Time-lapse of the additional experiment. Without proposed method, the operator could not see the position of the object. Hence, the operator could not detect the object before touching to the object (A1~A3). As a result, the operator broke the paper box with the slave robot (A4). In contrast, with the proposed method, at first, the operator could not detect the object (B1). However, as the distance to the object becomes less than the threshold, the operator could detect the object with vibration (B2). And when the slave side touches to the object, the vibration is stopped. Hence, the operator can detect the contact of the object (B3). And as a result, the soft touch to the paper box is realized because of the vibration (B4).