An Implementation of Application Design Framework for a Dressed-up Touchscreen with a Tactile Feedback

The products with the touchscreen are increasing while there are elderly or handicapped people who cannot operate touchscreen well.Therefore, the researches of the system supporting operations of a touchscreen by giving fingers a touch feeling become popular. We aim for the realization of the touchscreen that can provide the click feeling and the position of the button by a simple mechanism keeping the flexibility of the touchscreen (4-5). To be spread the welfare systems employing our proposal, providing a generic shared developing environment over applications is required. The developing environment must be a simple mechanism and less dependence to the device. We have proposed the hardware framework with a simple structure[4].In addition, we have described a concept of the general software framework on this hardware framework[5].However, since the software framework was a conceptual image, we need further examination to practice implementing the many features that are independent on the specified devices. For example, these features are the user interface and the switching of the application. In this study, to realize the device-independent software developing environment, we employ GTK(The GIMP toolkit) as the infrastructure of user interface. We also develop the touchscreen system on the Linux to hide the real devices. We develop the touchscreen library combining with the GTK library and the touchscreen library. The touchscreen library is a set of low level functions that cover the details of the real device. In this paper, using their libraries, the study case was implemented the software framework on the prototype. Further, this paper verified that the application actually can be developed using the software framework.


Introduction
It is difficult for persons who are unfamiliar with a touchscreen like elderly or handicapped people to operate the electronic products employing touchscreen as user interface [1].What cannot recognize the buttons on the touchscreen is one of the significant problems to make the operation more difficult.
Therefore, the systems supporting operations of a touchscreen by giving fingersatouch feeling have been developed in the past [2][3][4][5][6][7].Their systemsare not the same as the sense of the button switch with a click feeling that we are accustomed to.In addition,the users cannot recognize button positions on their systems, and they are notsuitablefor downsizing of device.We aim to realize a touchscreen that can provide the position of the button and the click feeling by a simple mechanism, keeping the flexibility.The flexibility means that our touchscreen is able to adapt to any user interface when changing the button sheet.Actually, we have proposed the necessary hardware framework to comprise the features described above [7].Also, we have developed a prototypedhardware according to the hardware framework.We had some testees operate the prototype.Then, it resulted that our touchscreen is easier to manipulate than the conventional one.Accordingly, the result indicates usability of our proposal.In order to make our touchscreen widely used,we need not only the general framework of the hardware shown in [7] but also a general framework of the software,design framework.According to the design framework, any programmers can develop uniformly the various applications on the touchscreen proposed.
In this paper, after we briefly explain the hardware framework of the proposed touchscreen, we describe the design framework of the application.Then, through the application examples on the prototype, we demonstratethat an application can be developed actually according to the design framework.

Hardware framework
Fig. 1 shows a hardware framework of the proposed touchscreen system.Mainly, the proposed touchscreen consists of a switch underneath the LCD panel and the plates with identification number at the top of the LCD panel.When a finger pushes the touchscreen, the switch underneath the LCD panel is also pressed.Thus, this touchscreen gives the user the click feeling that is the same as that of mechanical switch we are accustomed to.Since the touchscreen is only executed by pushing the switch, an incorrect input by just touching can be prevented easily.
For the plate put on the top of the LCD panel, the places where icons are displayed in the touchscreen are hollowed out.Therefore, the edges around the buttons on the hollowed part give the user's finger unevenness.That is, the position and boundaries of the icons can be recognized by the finger.The ID can be read by placing the plates on the reader, the system can determine the type of plate.Therefore, the proposed touchscreen can correspond to the various arrangement of the button.In addition, the plate can be used or not can be use depending on the application.Information such as the placement of the buttons on the screen hierarchy and each layer of the plate and default is stored in a storage device.The necessary information according to the plate switching and button pushing is selected from the storage device.

Design framework
Fig. 2 shows a design framework of the proposed touchscreen system.We explain the concept called an object in each screen on the touchscreen.The object has coordinates of the button, images of icons, background images, and programs invoked by pushing a button.By the state transitions among objects invoked by pushing the buttons, the screen switching or the execution content switching is realized.The object holds the number of the buttons(m) and the x0,y0: origin coordinates of the touch button h,w: width and height of the touch button number of background images(n).Each button has coordinates of the origin (x0, y0), and the height and the width of the icon (h, w).Theimg_p of each button is a pointer to the image displayed when the button is pushed.The Img_r is a pointer to the image displayed when the button is released.The act_pis a pointer to a program to be executed when the button is pushed.The next_s is a pointer to the next object to transition when the button is pressed.
According to the concept of Fig. 2, basic function groups(library) to develop applications are provided.For example,they are the function to display the image designated for the area of the LCD, the function to be displayed to read background image from the SD card and so on.
Incidentally, it is possible to switch among multiple applications by changing the plates shown in Fig. 1.A plate corresponds to one or more applications.Each application corresponds to one or more objects.

Adaptation example
We have developed an application using the proposed design framework.This case study develops a calculator on our prototype hardware.Incidentally,the application of the developed calculator can be four arithmetic operations,and is the same as the use of a common calculator.

System Overview
We explain the structure of the prototype that was used to validatewhether our design framework can realize an application or not.Fig.3 shows a block diagram of the prototype.
The LCD module consists of the LCD with the 240x320 of 3.2 inch resistive type touchscreen, the LCD controller of SolomonSystechSSD1289and the AD converter for the touchscreen of SHENZHEN XPTEK TECHNOLOGY xpt2046.
This prototype usesthe plateID as RFID.The RFID reader is ID Innovations ID-12.The switches underneath the LCD module are push-button switches of OMRON B3F5000.
These three parts are connected to the microcomputer board of Strawberry Linux STBee that equipped with ST Micro's STM32 microcontroller.
Software to run on STM32calculates the X-Y axis of the finger sent from the AD converter attached to LCD module .Depending on the pressing of the switch, the microcomputer performs the processing corresponding to the default state and the current plate.

Application design and operation check
Fig. 4shows the appearance of the calculator we designed and the operation example.
In this case study, the plate we madecorresponds to one application and the application has one object.The number of images of the touch button (m) is 16.They include 0-9 of numbers, the sign of the four arithmetic operations, the clear, and the equal sign.The number of the background image (n) is one.The images when pushed and released are same.When each numeric button is pushed,the numeral of the pushed button is displayed in the background image and memorized.Whena calculation button is pushed, the action performs four arithmetic operations corresponding to the kind of pushed button, by using memorized numerals.Finally, the action displays the operation result.

5.Conclusion
In this paper, we proposed a design framework for application in the dressed-up touchscreen with a tactile feedback.Further, this paper indicates that the application can be developed in practice by using the design framework through a case study of a calculator.
As future work, we will increase application examples, and perform an experiment to the more testees.