Development of a projection-type simple stereoscopic display with a touch panel functionality

Currently, the penetration rate of mobile phones is high. As a result, various apps have been developed and services have been provided. So, we wondered if there was a way to make the game more entertaining. In addition, the number of people who use online shopping has increased. Since online shopping is purchased without looking at the actual product, trouble may occur. Therefore, we wondered if there was a way to convey information that was close to the real thing to customers. Our research aims to achieve these two goals. In order to achieve this research purpose, we will develop a projection-type simple stereoscopic display with a touch panel functionality. First, a game character or product is projected on the display, and the game character or product is touched and operated. As a result of the experiment, we were able to move the image projected on the display.


Introduction
Currently, the penetration rate of mobile phones is extremely high. Originally, mobile phones were developed so that you can be contacted by email or phone when you go out. As the penetration rate increased, various services were provided and applications were developed to see if there was anything else that could be done using mobile phones. This made it possible to take pictures, play games, and listen to music. As a result, they have come to carry mobile phones not only for communication but also to reduce travel time and waiting time. We wondered if we could enjoy it more by devising something on mobile phones. So, we focused on 3D games. Now, with the development of VR, it has become possible to enjoy 3D games. But you need VR goggles. And the VR goggles are relatively expensive. Therefore, by developing a cheaper device, we propose a method for enjoying 3D games on smartphones that many people have.
In addition, the spread of mobile phones and PCs has increased the penetration rate of the Internet. As a result, various companies have begun to sell products on the Internet. Online shopping is very convenient because you can shop without leaving your house. However, various troubles are occurring in this online shopping. One of the troubles caused by online shopping is that the appearance and size of the product are different from what you expected. Therefore, we propose a method to prevent troubles caused by misunderstanding of this product.

Proposal method
We thought that it could be realized by showing the image of the smartphone three-dimensionally and touching and moving the image. First, we will explain the game. By making the game characters appear three-dimensional, you can feel as if they were there. In addition, you can enjoy a three-dimensional game by freely operating the character. Next, as a cause of troubles in online shopping, it is difficult to convey the actual information because the information on the Internet is characters, voices, and images. Therefore, by projecting the product three-dimensionally like the real thing, it is possible to obtain information close to the real thing. We also thought that you could get more detailed information that you wanted to know by touching and moving the image of the product and looking at it from the angle you wanted to see.
We used an optical illusion trick such as Pepper's Ghost as a method of projecting three-dimensionally. Pepper's Ghost is an optical illusion trick that projects a display image onto a half mirror to make the image appear to float. The half mirror fixes the angle with the display at 45 degrees. First, prepare images from various directions of what you want to look three-dimensional on the smartphone side. The prepared image is projected on the half mirror. Create a halfmirror multi-sided display to project the prepared image. (Fig. 1) This makes it possible to change the appearance of the image depending on the viewing direction and make it look three-dimensional.
Next, a method of moving the projected image by adding a touch function to the display will be described. In this method, touch determination is performed by a mechanism in which a current flows by touching a display to which a voltage is applied. First, no current flows just by applying voltage to the display. This is because the display is not grounded and the circuit is not closed. When you touch the display, the circuit is closed and current flows because the person is grounded. Place a resistor between the voltage source and the display to determine that current has flowed. When a current flows, a voltage difference occurs across the resistor. (Fig. 2) Whether or not the display is touched is judged by the presence or absence of this voltage difference. The display needs to be conductive so that current can flow when the display is touched. Therefore, a conductive sheet is used for the display.

Prototype configuration
The configuration of the prototype is as shown in Fig.  3. First, the touch detection unit detects whether or not the display has been touched. It also detects which surface was touched. This detection result is transmitted to the tablet communication unit. Next, the information received from the touch detection unit in the tablet communication unit is transmitted to the screen display unit. Finally, the image is switched and projected on the display based on the information received on the screen display.

Touch detector
(a) Touch detection circuit The touch detection circuit is as shown in Fig. 4. Prepare as many touch detection circuits as shown in Fig. 4 for the number of display surfaces. When not touching the display, there is no voltage difference across the resistor R. There is a voltage difference when you touch the display. This voltage difference is measured by Arduino to determine if you have touched the display. Since the voltage difference between both ends of the resistor R when the display is touched is very small, it is amplified by the instrumentation amplifier (AD623AN).
(b) PWM circuit Uses PWM control by FPGA as high frequency voltage. The human body has capacitance. Its capacitance is 100pF. Therefore, a high frequency voltage is required to make it conductive. FPGA can handle higher frequencies than Arduino. Arduino can't handle frequencies up to ~ kHz. On the other hand, FPGA can handle frequencies up to ~ MHz. Therefore, FPGA is suitable for this study, which requires a high-frequency power supply.

Tablet communication part
It uses a wireless module called ESP32-WROOM-32D to send the information received from Arduino to the tablet. The ESP32-WROOM-32D is a wireless module that combines Wi-Fi and Bluetooth in one module. In this research, ESP32-WROOM-32D is used as a Bluetooth module. Send the information received from Arduino to the tablet via Bluetooth communication.

The part that displays the image
Create an app that switches images according to the information received from the Bluetooth module. The application is created with Android Studio, an application development software. Android Studio allows you to develop Android apps in the language Java or Kotlin.
First, it detects and lists Bluetooth devices. Then, select the Bluetooth device to connect from the detected Bluetooth devices. Switch images based on the information received from the Bluetooth module about which side you touched. The image is switched so that the image projected on the touched surface is projected on the front. Also, because this is a prototype, we made it possible to select the image to be projected on the display from the photo folder of each tablet.

Experimental result 4.1 Comparison of resistor R
The magnitude of the resistor R of the touch detection circuit was changed by 1kΩ, 5kΩ, 10kΩ, and 100kΩ, and the voltage difference across the resistor R was measured. The locations where the voltage was measured are V1, V2, and V3 in Fig. 5. For PWM, the frequency was set to 500kHz and the duty ratio was set to 50%. When the magnitude of the resistor R was 5kΩ, 10kΩ, or 100kΩ, the voltage of V3 increased when the display was touched. Therefore, it was found that touch detection can be performed under this condition. On the other hand, when the magnitude of the resistor R was 1kΩ, there was no change in the magnitude of V3. Therefore, it was found that touch detection cannot be performed under this condition. In addition, as a result of the experiment, it was found that the most ideal result is obtained at 10kΩ. After that, the experiment was conducted with the

Frequency comparison
The frequency of the voltage output from the FPGA was changed to 1kHz, 100kHz, 500kHz and 1MHz. Then, the voltage difference across the resistor R at that time was measured. The duty ratio was set to 50%. As a result of the measurement, it was found that the voltage of V3 increases as the frequency increases. This is considered to be due to the capacitance of the human body. When the frequency is low, almost no current flows. After 500kHz, there was almost no difference in change. Based on the above results, the experiment was conducted with a PWM frequency of 500kHz.

Duty ratio comparison
Changed the PWM duty ratio to 30%, 50% and 70%. The voltage difference across the resistor R at that time was measured. As a result of the measurement, the voltage of V3 increased under all conditions. Therefore, touch detection is possible at any duty ratio. As a result of the experiment, it was found that the difference between when the display was touched and when it was not touched was the largest when the duty ratio was 50%. Therefore, this time is the most suitable. Therefore, the duty ratio was set to 50%. Figure 6 shows the voltage waveforms observed by the above experiments.

Conclusions
In this research, we have developed a projection-type simple stereoscopic display with a touch panel functionality. We were able to make it look three-dimensional by projecting it using Pepper's Ghost. In addition, the voltage difference caused by the voltage difference across the resistor attached to the display when touched by the display was measured under various conditions. As a result, touch detection could be performed by adjusting the touch detection circuit and voltage source. By creating an app on the tablet that switches images according to the touch detection result, we were able to operate the projected image.