Development of learning analysis software using handmade clickers

In this paper, learning analysis software is proposed. In the traditional achievement test, students cannot get results and feedbacks of the test immediately, because many tests of the students must be marked by teacher’s own hand. Since the results and feedbacks are not returned until the mark is finished, the students take a long time to receive the results. In the previous study, the handmade clicker, which has been proposed by us to support active learning, can also realize the analysis of the student’s answers immediately unlike the traditional achievement test. The proposed analysis software can provide feedbacks for both the students and the teacher’s class by analyzing not only score but also response speed. The results provided by the proposed software are divided by the following 3 types: (1) all of the students, (2) each question and (3) each student. Owing to this analysis way, the proposed analysis software offers not only general analyzed results such as average score and correct answer rate, but also learning level and personality of the each student. Furthermore, advices for improving the learning method are also offered from these various viewpoints. The validity of the proposed software is confirmed by experiments using a demonstration data.


Introduction
In recent years, Ministry of Education, Culture, Sports, Science and Technology (MEXT) recommends the introduction of active learning into undergraduate education.To support the introduction of active learning, clickers are marketed.The clocker is a family of the audience response systems (ARS), and it is easy to realize an active learning by using the clicker.In past studies, the educational effects of the clicker have been reported (1)(2)(3) .The usability of a clicker was discussed by Yamada through the experiments (1) .In the Yamada's report (1) , the following advantages are discussed: (1) a real time interactive class is realized, (2) a teacher can deal with a class which is attended by many students, (3) a learning efficiency is improved and (4) many answers of the students can be corrected quickly and be saved.Furthermore, in the healthcare education, various practice reports.Nekoda reported the educational effects of the clicker in the case of the class of epidemiology (2) .Then, Kawabata et al. investigated the educational effects in a clinical observation training (3) .In the results of these studies (2,3) , good effectiveness of the clicker was discussed.From these studies, the good educational effects of the clicker has been verified.However, it is difficult to introduce the clicker into the class.That's why the cost of the clicker is not cheap.Concretely, the company A is selling the clicker system including 30 slave devices for about JPY300,000.On the other hand, the company B is selling the clicker for team based learning for about JPY900,000.In addition, a low-cost clicker is also marketed.The clicker is relatively cheap, but the cost of the clicker is more than JPY30,000 per one device.
To solve this problem, the various clickers have been developed as a previous study.The developed clickers are mainly divided into two classes, which are software-based clicker and hardware-based clicker.Ohmi et al. proposed a software-based clicker utilizing wireless fidelity (Wi-Fi) and open source software (5) .In the software-based clicker, students attend the class by connecting software-based clicker through the prepared router.By using software-based clicker, an interactive class can be realized easily if students have mobile device which can connect an internet.However, it is difficult to introduce the software-based clicker without technological knowledge for wireless network.That's why the setting of the software-based clicker requires the technological knowledge.On the other hand, a hardware-based clicker utilizing infrared communication was proposed by Yamada et al. (6) .The hardware-based clicker can be realized without network environment, because the slave devices of the hardware-based clicker communicate with the master device directly by using infrared communication.However, as author explained in the report, the communication distance of the hardware-based clicker is too short, and the communication angle is too small due to the infrared communication.Therefore, a novel handmade clicker and its data correcting software have been proposed by us to introduce active learning into a class easily (7,8) .The handmade clicker can be built at low cost owing to small circuit components.Furthermore, by using ZigBee wireless communication, interference of signals can be overcome.However, in the handmade clicker, the analysis function for the answer of the students is only showing a simple calculated results, showing rank of each students and making graph of the distribution of the score.Therefore, the teacher analyzes the answers of the students oneself to give feedbacks to the students.
In this paper, learning analysis software is proposed to give feedbacks and helpful information to both the teacher and the students.Unlike the traditional achievement test, the proposed software offers feedbacks and helpful information on the spot by analyzing the files saved by the data correcting software of the handmade clicker.Concretely, the analyzed results for each student consist of general analyzed results such as score, average score and deviation.In addition, the proposed software also offers as follows: (1) learning level of the subject (2) assumed personality of the student and (3) advice for improving learning method by analyzing not only score but also response speed.Then, the analyzed results for the teacher is as follows: (1) correct answer rate of each questions, (2) score distribution of each student, (3) average score, (4) correct answer rate of all students, (5) difference between teacher's definition of difficulty and calculated difficulty based on correct answer rate in the subject, (6) correct answer rate of each question and (7) difference between teacher's definition of difficulty and calculated difficulty based on correct answer rate in each question.Therefore, the proposed software can offer feedbacks and helpful information from various viewpoints.
The proposed software is experimented by using demonstration data to confirm the validity.

System architecture
Fig. 1 shows the system architecture of the handmade clicker.As Fig. 1 shows, the handmade clicker system mainly consists of slave devices, a master device and data correcting software on a computer.The operation of the handmade clicker is as follows: At first, the slave device transmits the device number and the answer number to the master device using ZigBee wireless communication.Then, the received signal of the master device is converted to the numerical data by terminal software Tera Term on the computer.Finally, the data correcting software analyzes the data on computer and displays the analyzed results.These operations are carried out by clicking a button one time on the data correcting software.In the handmade clicker, all the software can be available for free.Therefore, the required cost is only for hardware parts which are the master device and the slave devices.
The usability, the validity and the educational effectiveness of the handmade clicker have been verified by previous studies (7,8) .

Master device and slave device
Fig. 2 shows the outward appearance of the reception device and the slave device.The master device is consists of the reception device and a computer.The reception device is connected with a computer through a USB cable.The circuit configuration of the slave device is so simple.Concretely, the slave device requires only 9 components.The commercially available components are used to build the slave device.The circuit components is as follows: a microcontroller PIC12F1822, a XBee module, a button, 4 LEDs and 2 resister.The handmade clicker is realized at lower cost than the marketed clickers.Concretely, the cost of each device is as follows: (1) the reception device is JPY3,438 including a USB cable and (2) the slave device is JPY2,859 per device (7,8) .In addition, the communication problem of the hardware-based clicker is overcome, because the wireless communication is offered by XBee module instead of the infrared communication, where XBee module is based on a 2.4 GHz IEEE 802.15.4.From this reason, the 65,536 slave devices can be connected to the master device at most.The slave device mainly has 2 functions which are the selection of the answer number and the transmission.By pushing the button on the surface of the slave device in short period which is less than 1 second, the answer number is selected.The selected number is shown by LEDs on the surface of the slave device.When the button is pushed in long time, the answer is transmitted to the master device.In the communication, the delay of the communication is about 33ms regardless of the distance (7) .Furthermore, the maximum distance between the slave device and the master device is less than 40m, because the maximum transmissible distance of XBee module is less than 40m.From these reasons, the handmade clicker has good performances as an ARS.The transmitted data is received by the master device and analyzed by the handmade data correction software.

Data correcting software
The data correcting software of the handmade clicker was developed by using an Excel VBA.The data correction software offers the following functions: (1) showing questions, (2) correcting data of the students and (3) marking answers.The results provided by these functions are shown on Excel worksheets.When the teacher asks question to the students, the following items must be prepared on the software in advance: (1) questions, (2) options, (3) correct answer numbers and (4) a teacher's definition of difficulty.Fig. 3 shows the Graphical User Interface (GUI) of the data correcting software on an Excel.In the data correcting software, the simple results, such as correct answer rate, answer speed and so on, can be obtained on the spot.Furthermore, as Fig. 3 shows, the results can be shown graphically.The data correcting software also has the function to save the data of the students.By using the function, the teacher can review the results of students whenever the teacher needs.The results, which is offered by the data correcting software, are as follows: score, deviation value, average score, correct answer rate and answer speed.However, the teacher must analyze the answer of students oneself to utilize these data effectively.Since it takes long time to analyze a lot data, it is difficult for the teacher to finish the analysis of these data on the spot like a traditional achievement test.That is why the teacher must analyze students' answer by one's own hand to provide the detailed feedbacks.Therefore, in a class, the data correcting software can realize only an interactive class.deviation and correct answer rate, but also learning level of each student and the personality by analyzing not only score but also response speed.Owing to these analysis way, the proposed software can offer feedbacks and helpful information from various viewpoints.Furthermore, the analysis time of the proposed software is in an instant, so the analyzed results can be offered on the spot unlike the traditional achievement test.Therefore, the students can solve many questions and get the feedbacks during a class.The proposed software can be controlled easily.The functions can be carried out by clicking one time a button on the proposed software, and each analyzed result is shown on created worksheet of the Excel automatically.selecting the analyzed result name from the dropdown list, the selected analyzed result can be shown.Each function are explained in detail.In explaining the functions, the proposed analysis software is demonstrated by using a demonstration data, which consists of 20 questions and 50 students.Then, the score of each question is adjusted to 5 points, so the maximum score of the demonstration test is 100 points.

General analysis function
General analysis function is all students' analyzed results.Fig. 5 shows a Graphical User Interface (GUI) of General analysis function displayed on an Excel.As Fig. 5 show, General analysis function shows the following results: (1) a graph of correct answer rate, (2) a graph of distribution of the score, (3) average score, (4) correct answer rate and (5) calculated difficulty based on correct answer rate.In the results of General analysis function, students' learning level can be concluded from the two graphs, average score and correct answer rate.In the graph of correct answer rate, the learning level can be assumed from the form of the graph.For example, it is presumed that the learning level is low when all of the correct answer rate is low like Fig. 5.By setting questions systematically, students' learning level can be analyzed in detail.In the graph of distribution of the score, teacher can understand the difficulty of the subject for the students and distribution of students' learning level.In addition to this advantage, each student can understand ones learning level in the class.General analysis function also displays two difficulty, which are the teacher's definition and the calculated difficulty based on the correct answer rate, on the GUI of General analysis function.The teacher's definition is the difficulty which is defined by the teacher in advance, and the teacher's difficulty must be set Fig. 5. GUI of General analysis function on an Excel.to an integer value from 1 to 5. The calculated difficulty is defined by the following equation: In Eq. ( 1), Dcal is the calculated difficulty and CA is the correct answer rate of each student.By comparing the defined difficulty and the calculated difficulty, the gap between teacher's understanding and students' actual learning level can be shown.
In General analysis function, the learning level of whole classes and its distribution are mainly guessed.Therefore, the teacher can improve one's education technique by utilizing the analyzed results as a feedback.

Analysis function for each question
Analysis function for each question is shown by analyzing information of each question which is corrected by the data correcting software.Fig. 6 shows the GUI of Analysis function for each question on an Excel.The teacher can get learning levels for the each question easily by utilizing the function.As Fig. 6 show, Analysis function for each question displays the following analyzed results for each question: (1) the selection rate of each option, (2) correct answer rate and (3) calculated difficulty based on correct answer rate.In the results of Analysis function for each question, the analyzed results show not only the correct answer rate but also the incorrect answer rate.The teacher can find the inclination of the student's answer way Fig. 6.GUI of Analysis function for each question on an Excel.
by knowing the option which is selected the most.Furthermore, the teacher's definition of difficulty and the calculated difficulty based on correct answer rate are shown as in the general analysis result.The equation for the calculation of the difficulty is the same as eq.( 1).In Analysis function for each question, the actual difficulty of each question is realized.Furthermore, by comparing between the teacher's definition of difficulty and the calculated difficulty based on correct answer rate, the teacher can get ones understanding for students in detail, as well.Therefore, in Analysis function for each question, the teacher can improve one's educational techniques by getting the difference between one's recognition for learning level and the estimated learning level in more detail than the results of General analysis function.In Analysis function for each question, the analyzed results on question by question can be displayed specially.

Analysis function for each student
Analysis function for each student shows analyzed results of the personal achievements, and it is given for each student by analyzing each student's information.Fig. 7 shows the GUI of Analysis function for each question on an Excel in the case of that the slave device number is 37.The students can get one's weak points for the subject easily by utilizing the function.As Fig. 7 show, Analysis function for each student shows the following analyzed results for each student: (1) personal score, ( 2) personal correct answer rate, (3) personal deviation in all students, (4) personal respond rate, (5) bar graph showing learning level, (6) bar graph showing personality of the participant and (7) advice for the learning of the student.In the results of Analysis function for each student, the general analysis results such as the Fig. 7. GUI of Analysis function for each student on an Excel.
personal score, the personal correct answer rate and the personal deviation are performed.In addition, the proposed software shows the graph showing learning level and personality of the students, advice for the learning of the student by performing the analysis including the response speed.The learning level is calculated from the deviation and the response speed.As Fig. 7 shows, the X-axis of the bar graph showing learning level has the following four categories: (1) good and fast, (2) good and slow, (3) bad and fast and (4) bad and slow.In the categories, good and bad mean the evaluation of the personal deviation.Fast and slow mean the evaluation of the personal response speed.The class of each student is decided by a matching degree.
The matching degree for the categories is decided by 2 steps.At first, matching degrees of the precision and the response speed are calculated by the following equations: In Eq. (2-5), MDx (x is g, b, f or s) means the matching degree, where the meaning of each subscript are as follow: g is good evaluation, b is bad evaluation, f is high speed response and s is slow speed response.As with MDx, μx means the membership functions.In addition, x1 is the personal deviation of the student, x2 is the response speed of the student and n is the number of all students.In this operation, the matching degrees are decided in the range from 0 through 1.
Finally, the calculated matching degrees are converted into four matching degrees.The four matching degrees are shown on X-axis of the bar graph showing Learning level.The matching degrees are calculated by the following logical operations: In Eq. (6-9), MDi (i = 1, 2, 3, 4) means the matching degree, where the meaning of each matching degree is as follows: MD1 is good and fast, MD2 is good and slow, MD3 is bad and fast and MD4 is bad and slow.In Fig. 7, the results of Eq. (6-9) are shown on X-axis of bar graph for the learning level.By seeing the graph, the learning level can be concluded.For example, it is estimated that the student of Fig. 7 understands the subject well, because the matching degree of the evaluation is good.However, since the student takes long time to solve the questions, it is also estimated that the achievement degree is not enough.Therefore, the student should solve a lot of questions to improve one's learning method.In Analysis function for each student, the advice for improving one's learning level is also shown automatically like the above example.The advice is decided by using the maximum value in the matching degrees calculated by Eq. (6-9).Table 1 shows the advices.As Table 1 shows, there are 4 advices on the proposed software to improve the learning method.By planning how to study referring to the estimated advice, the student can improve one's learning method and study the subject efficiently.
Table 1.Advices estimated from the matching degree.

Maximum value Advice MD1
Your evaluation is excellent.You should try more difficult subjects.MD2 Your evaluation is great, but your response speed is slow.As for advice, you should solve a lot of questions to improve your response speed.MD3 Your evaluation is not great, but you solved the questions quickly.As for advice, please solve the questions at a timely pace.MD4 You may have misunderstood the subject.As for advice, please review and study the subject.
Analysis function for each student also shows the characteristic of student's answer way as a bar graph, the name of which is Personality of the student.In the function, the student's answer of each question counts and classifies.Concretely, the personality of the student can be assumed by counting the frequency of each category: A, B, C, D, E and F. Fig. 8 shows how to classify answers into A to F. As Fig. 8 shows, the conditions for classifying are as follows: category A is added 1 if the answer is correct and fast, category B is added 1 if the answer is correct and normal response speed, category C is added 1 if the answer is correct and slow, category D is added 1 if the answer is incorrect and fast, category E is added 1 if the answer is incorrect and normal response speed and category F is added 1 if the answer is incorrect and slow.At this time, the response speed is defined as follows: the response speed is fast if the response speed is top 30%, the response speed is slow if the response speed is bottom 30% and other response speed is defined as the normal response speed.In the function, the personality of the student is considered from student's answer way.For example, the personality of the student shown in Fig. 7 is assumed that there is a Is it a correct answer?
Is the respond speed fast?
Is the respond speed slow?
Add 1 to the count of A. because one's response speed is slow in most cases.On the possibility that the student is careful, other hand, there is a possibility that the student is hasty when one's response speed is fast in most cases.By analyzing the personality, the advices to improve the learning method are given to the students from another viewpoint.Therefore, in Analysis function for each student, the student can improve the way to learn by getting the general analysis results, the estimated learning level, the assumed personality and the advice.

Conclusions
Learning analysis software using handmade clickers has been proposed in this paper.The validity of the proposed software was confirmed by the experiments with the demonstration data which consists of 20 questions and 50 students.The analysis functions of the proposed software are as follows: (1) General analysis function, (2) Analysis function for each question and (3) Analysis function for each student.From these analyzed results, the advices for improving a learning method can be offered from various viewpoints.Furthermore, the analysis functions can be carried out on the spot.Therefore, since the proposed software can offer the feedbacks and helpful information for not only the teacher but also the students, the proposed software can contribute to the improvement of the learning.
The detailed experiments using actual data are left to a future study.

Fig. 3 .
Fig. 3. GUI of the data correcting software on an Excel.

A
learning analysis software is proposed for both the teacher and the students to support the improvements of education and learning.As with the data correcting software, the proposed software is developed by using Excel VBA, and the analyzed results are shown on Excel worksheets.The proposed software offers the feedbacks and the helpful information by analyzing the data saved by the data correcting software of the handmade clicker.The analyzed results provided by the proposed software are divided by the following 3 types: (1) all of the students, (2) each question and (3) each student.Each analyzed results are shown by the following functions: (1) General analysis function, (2) Analysis function for each student and (3) Analysis function for each question.Furthermore, in the analysis function for each question, the results includes not only general analyzed results such as average score, Fig. 4. Dropdown list for showing the analyzed result.

Fig. 8 .
Fig. 8. Flow chart of how to classify answers into A to F.