Subjective Evaluation Experiment of Grayscale Color to Examine VAS Measurement Method

We have been researching subjective evaluation experiments using Visual Analog Scale (VAS). VAS, which marks the subject's subjective evaluation degree on a straight line, has the advantage of expressing subjectivity more intuitively. In this paper, we report an attempt of the subjective evaluation related to human lightness sense as fundamental research on what kind of color sense human has and how it affects other sensitivity. In this study, we used VAS to obtain answers on where each gray image with different lightness corresponds to on the grayscale. Numeric values obtained by this evaluation experiment were analyzed to confirm the characteristics of the sensitivity of the individual. Moreover, at the same time, it was examined whether the VAS accurately reflected the subjective view of the person by the answer of what percentage of the grayscale was by the numeric value input. In order to conduct experiments smoothly, we developed an app that can perform VAS measurement and numeric value input simultaneously. We presented ten grayscale images to 13 subjects and obtained subjective measurements using this app. It was confirmed that when the lightness of the grayscale image is high, it can be recognized relatively accurately, but when the lightness is low, it tends to feel darker.


Introduction
The research which measures the sensitivity information from various biological signals such as brain wave, heart rate, and, the pulse wave is advanced.There are studies such as whether vision and smell are related to each other (1) , and whether measuring pulse waves when listening to a piece of certain music reduces stress (2) .In general, much information can be obtained from the visual sense, and it is said that the visual sense had the most crucial role in the five senses.It is considered that the human sense differs from person to person, and even if the color is the same, it has different sensitivity such as feeling bright and feeling dark.There are individual differences in visual accuracy, and it is anticipated that the accuracy lowers in the place where the variance increases in visual sense measurement experiment.However, it is expected that there will be a certain tendency as a whole.In this study, we pay attention to the lightness sense of a person and try a subjective evaluation of an individual, such as whether he or she looks bright when he or she sees the same image.Our goal is to evaluate the performance of the VAS method.We considered that the VAS is the best for conducting such subjective evaluation experiments.

Visual Analog Scale
There are four scale levels: nominal scale, ordinal scale, interval scale, and ratio scale.The ordinal scale represents the difference between the objects.The interval scale assumes that the differences are equally spaced.The ratio scale can also be treated as a numeric value.Generally, Likert Scale (LS) is assumed to be ordinal scale, but VAS is also considered to be able to be treated as an interval scale, and we also use VAS in this viewpoint.
VAS can be treated as an interval measure and used for more statistical analysis.Also, it is easier to obtain values that reflect subjective views intuitively than LS that is evaluated in 5 or 7 steps.Moreover, there is a case that the bias from the halo effect, generosity effect, and centralization tendency appears in the evaluation in LS.For example, in the case of a 5-point evaluation, various biases may occur such as a centralization tendency that avoids 1 and 5, which are the extremes of the evaluation, and the evaluation result gathers in 3, which is the center.We adopted VAS in order to avoid these biases (3,4) .
VAS is an evaluation method in which a user's subjective evaluation is indicated between 0% and 100% on a horizontal straight line.It becomes possible to evaluate the sensitivity of the examinee by the sensory scale instead of the order scale, and various statistical analyses can be carried out.VAS is a commonly used method for pain assessment (5) and subjective evaluation.We have been used the VAS for studies of a painting appreciation (6) , conversation with communication robot (7,8) , impression of sounds (9) , and youth mental health (10) .The details of VAS as a subjective evaluation method are described in our previous report (6) .

Experimental method
In the experiment, ten arbitrary grayscale images are selected and presented to the subject to obtain a VAS response.The grayscale value is assumed to be in the interval [0, 1].Ten grayscale values used in this experiment are 0.125, 0.200, 0.250, 0.375, 0.400, 0.600, 0.675, 0.750, 0.800 and 0.875, corresponding to Q1 to Q 10.These values are presented in random order during the experiment.The experiment was conducted on both the VAS app for Android Tablet and the questionnaire.We also added the function to input a numeric value respond to VAS measurement into the VAS app.

Experimental result 4.1 Experimental results obtained using the app
First, we will introduce the experimental results using the VAS application.The number of subjects is 13, and all are male students aged 18-20.The number of grayscale questions presented is ten.Table 1 shows the degree of gray scale [0-1] used in the experiment.The order of the questions presented to the subject is actually random.
The subjective evaluation of VAS is shown in Table 2, and the subjective evaluation by numeric value input is shown in Table 3, respectively.Since the space is limited, it is represented by the second decimal place.Hereinafter, to simplify the notation, the VAS measurement method is described as "VAS", and the method of directly entering a numeric value is described as "NUM".Based on the data obtained by the VAS and the data obtained by the numeric value input, respective representative values (maximum, minimum, mean, median and standard deviation) are shown in Table 4 and Table 5.
First, we focused on the representative values, mean and median.Table 6 and Table 7 show the percentages of these two representative values that correspond to the grayscale values presented to the subjects in the experiments shown in Table 1.A value above 100% indicates that the user felt whiter than the actual data, and a value below 100% indicates that the user felt blacker than the actual data.It can be confirmed that the match rate of grayscale decreases rapidly as the color close to black such as Q3, Q2, and Q1.Both show that the subject feels darker than the actual color.
Tab. 4  Also, no significant difference was found between VAS measurement and NUM input.Both confirmed the same trend.
Next, we try to draw a graph and understand it visually, since it is difficult to grasp the characteristics of the experimental results with only these numeric values.Two types of graphs are shown to examine the trends.
One is the most popular graph combining of mean and standard deviation is shown in Figure 4.The other is the box-and-whisker plot shown in Figure 5, 6, and 7. Box-and-whisker plots are robust summary statistics called five-number summary, and outliers can also be identified.By combining beeswarm plots, all data are also displayed, so it is possible to grasp the overall trend (Figure 5, 6).The crosses in these Figures are the grayscale values presented to the subject shown in Table 1.Here, in order to facilitate visual understanding, the data of the VAS measurement is drawn in blue, and the data of the numeric value input is drawn in red.Finally, Figure 7 shows the result of the superposition of box plots of VAS and NUM, and Figure 8 shows the result of the superposition of both histograms.Both show that there is no significant difference between VAS and NUM.

Additional experimental results obtained using the question paper
In order to verify the conclusions obtained from the experimental results using the app, we performed a similar experiment using a questionnaire.The number of subjects is 66, and all are male students aged 18-20.The representative values (maximum, minimum, mean, median, and standard deviation) of the measured values by question paper are shown in Table 8.Table 9   Figure 9 shows the combining mean and standard deviation.Also, the superposition of the box plot and the beeswarm plot is shown in Figure 10. Figure 11 shows the histogram of this data.From these analyses, the following could be confirmed in the experiment using the questionnaire.It can also be confirmed that the match rate of grayscale decreases rapidly as the color close to blacks such as Q3, Q2, and Q1.

Conclusions
We conducted a grayscale evaluation experiment using VAS measurement and numeric value input using a VAS app for 13 subjects.There was no significant difference between the VAS input and the numeric value input.From this result, it has been confirmed that the VAS measurement method of intuitively marking on a straight line has sufficiently effective performance as a subjective evaluation method.Also, it was confirmed that the grayscale color close to black among the ten questions tends to be evaluated as blacker than it actually is through all the experiments.Besides, it was also confirmed that the grayscale color close to white such as Q8 and Q9 has a high matching rate, and the grayscale color such as Q6 has a large dispersion.It was possible to conduct experiments and analysis on subjective evaluation with only basic statistics by using VAS.It is interesting to be able to confirm the non-linearity of human sense in this experiment.We also confirmed that combining box-and-whisker plots and beeswarm plots makes it easier to visually understand the summary statistics and the overall distribution of data and their trends.
The results of this experiment are likely to depend on factors such as tablet brightness, indoor lighting, printing paper and printer.However, we believe that this method is effective as a human subjective evaluation method, not a detailed study of human physiological characteristics.We will continue to work on applying this subjective evaluation method to various fields in the future.

Fig. 11 .
Fig. 11.Histogram by VAS (paper) shows the percentages of this representative values that correspond to the grayscale values presented to the subjects in the experiments shown in Table1.