The Compact Fuzzy Association Rules For Data Classification

Data classification mining is a method to find data generalization in a form of rules then used these rules to predict some unknown value in the future data. But in actual applications, the rules may be of low accuracy and the number of rules may be so overwhelmed that users could not efficiently apply them. Therefore, this research proposes the development of data classification algorithm with compact fuzzy association rules to optimize accuracy and interpretability of the model. To evaluate the performance of the proposed method, this research will compare accuracy of the classification model and the number of rules against 9 different data classification algorithms. The results showed that our CCFAR algorithm is comparable in terms of accuracy. When considering both accuracy and size of model, our algorithm is the best one.


Introduction
Data classification technique is one of a data mining widely used to predict the future data by inducing the model from sophisticated data.For instance, in a medical science the model is used to predict a patient who is risky of having a breast cancer (1) .In a commercial bank, the model is used to screen credit requests and evaluate the credit rating of consumer.For its potential benefits, many researches have long been concentrating on improving the efficiency of data classification by proposing algorithms like C4.5 (2) and OneR (3) .But these algorithms have a low predictive accuracy.Therefore, Liu and Ma (4) proposed a new method call the "associative classification" which is a combination between the association rule mining technique and the data classification technique.Their proposed algorithm was called the Classification based on Association Rules (CBA).The CBA has been tested its predictive performance by comparing with the C4.5, and it turns out that has high accuracy than C4.5 algorithm.This CBA has drawn attention from many researches to develop algorithm based on associative classification technique, such as GARC (5) algorithm, OAC (6) algorithm, and many more.However, the inherent problem of associative classification is that the association process can handle only symbolic and binary values.It cannot process continuous values.To solve this problem, we propose to use a fuzzy set to transform the continuous value to the degree of membership (7)(8)(9) .
In this paper, we present the idea and the development of an algorithm called data classification with compact fuzzy association rules (CCFAR).Our main focus is to optimize both an accuracy and interpretability of the model.In addition, we applied the concept of OneR (3) algorithm to select the best rules and reduce the number of rules in the final result.

Related Works
Classification task is the mainstream of many researches are concentrating on developing algorithms for increasing the accuracy and reducing the number of rules.We can summarize the researches that are related to our works into 3 groups, that are group of data classification, group of associative classification, and group of fuzzy associative classification.The details of 3 groups are as follows: Firstly, the group of data classification is the initiative concept of data mining to classify target variable with some related features.Examples of this group are as follows: -Quinlan (2) proposed C4.5 algorithm which is well known in the classification because the algorithm is able to process quickly and the model is easily understandable.The main concept of this algorithm is the use of heuristic search to construct a decision tree and prune tree.
-Cohen (9) proposed an algorithm called RIPPER (Repeated Incremental Pruning to Produce Error Reduction) developed from the IREP* algorithm by the principle of growing and pruning techniques to select the low error rate rules.
-Holte (3) proposed an algorithm named OneR or One-Attribute Rule, which is an algorithm that easy-to-understand algorithm and the model is a compact set of rules.OneR is simple because it select a single attribute with the fewest error to build the model.
Secondly, the group of associative classification uses the association mining to build the association rules, then using many techniques of data classification to make association rules appropriate for classification.Examples of this group are as follows: -Liu and Ma (4) proposed CBA algorithm, which produced is a high accuracy for the data classification.The CBA composed of 2 steps: to build the model with CBA-RG (to create the association rules), and CBA-CB (to make association rules for classification by selecting the low error rate rules).
-Chen and Zhang (5) proposed an algorithm called GARC (Gain Based Association Rule Classification) with a efficacy in classifying data used the model is a compact.Because of the GARC was using the information gain threshold to create frequent item-sets and then it used redundant and conflictive techniques to build the class association rules.
-Hu and Li (6) proposed an algorithm named OAC (Optimal Association Classifier) with a good efficacy in classifying data.The principle of OAC was to generate association rules by using the constraint of apriori (10) algorithm, then it selected the class association rules by the method call OCARM (Optimal Class Association Rule Mining (11) ).
Finally, the group of fuzzy associative classification uses the fuzzy set to controll the continuous value in the association rule mining processing.Examples of this group are as follows: -Pach et al. (7) proposed an algorithm called CFARC (Compact Fuzzy Association Rule-Based Classifier) to resolve the problem of how to define the minimum support and minimum confidence in association rule mining by applying fuzzy correlation threshold.In their experimental results, they showed a good accuracy and a compact set of model.
-Chen (8) proposed an algorithm named CFAR (Classification with Fuzzy Association Rules).It generated rules by using the apriori algorithm, then it selected the rules that have highest confidence value and deleted remaining rules with lowest confidence value.
-Hühn and Hüllermeier (12) proposed an algorithm called FURIA (An Algorithm For Unordered Fuzzy Rule) based on the Ripper algorithm.But the FURIA used unordered rule set to reduce bias of the target class and it used stretching technique to obtain the generalized rules.

Preliminaries
In this section, we introduce the basic definitions of fuzzy sets, fuzzy association rules and fuzzy associative classification rules.

Fuzzy Sets
Fuzzy sets are sets that cannot explain something clearly.For example, an explanation of the people who are very tall, someone say that the people who are 180 cm.high are very tall, but others may say that the people who are 185 cm.high are very tall.So, we can see from this example that it is impossible to tell exactly regarding who is very tall.Therefore, to solve such problem Zadeh (13) proposed the concepts of fuzzy sets to explain something that is not clear with the degree of membership function (rang value of the degree is [0, 1]).For instance, the people who are 180 cm.high; they are medium tall at the degree of 0.7 and very tall at the degree of 0.3.The people who are 185 cm high, they are medium tall at the degree of 0.1 and very tall at the degree of 0.9.From this example, we can see the people who are 180 cm.high and 185 cm.high would be moderately tall and very tall, respectively with the different degrees.
Therefore, this research applied the concept of fuzzy sets to explain the continuous value of numeric data.We used fuzzy partitions with fuzzy c-means (FCM (14) ) algorithm.

Fuzzy partitions
In this section, we present the important step of our CCFAR algorithming for transform the continuous value (Fig. 1 a) to partitions with the FCM algorithm.The FCM is used to indicate the degree of membership in a set with the value ranging from 0 to 1 (Fig. 1 c), which is different from the k-means algorithm that indicates the degree of membership in a set with the discrete value (Fig. 1 b).
The processes of FCM are composed four main steps.
1) Initialize C and μ ij 2) Calculate the center vectors by Eq.( 1) 3) Compute and update membership of data by Eq.( 2) Where x i is data vector, m is fuzziness that can be any real number equals or greater than 1, μ ij is the degree of membership of x i to be in the cluster j, N is data for clustering, C is number of clusters and c j is center of cluster j.
Fig. 1.The membership functions of k-means and FCM.

Fuzzy Association Rule
Association rule is originally an analysis of customer purchases (also called Market Basket Analysis) by storing the items in the basket of the customer into transaction.Then, it analyzed these transactions to discover the association rules.The association rules are expressions of the type X → Y, where X and Y are sets of items.This means that if customers buy item X, then customers buy item Y together.But the processing of the association rule mining is unable to handle the continuous value (Table 1).So, we used fuzzy set to convert the continuous value into a degree of belonging to a set.For example, the Table 2 shows the degree of attribute age associated with linguistic values low, medium and high.
In recent years, many researches have proposed methods to mining fuzzy association rules from continuous value (15,16) .The processing of fuzzy association rules are composed of 2 steps.First step, create frequent Item-sets by counting item-sets that have fuzzy support (Eq.3) higher than the minimum fuzzy support.Second step, create fuzzy association rules from frequent item-sets that have fuzzy confidence (Eq.4) higher than the minimum fuzzy confidence.Let <Z: A> be a fuzzy item-set denoted as <Z: A> = [<z i1 : A i,j > ∪ <z i2 : A i,j > ∪ …∪ <z iq : A i ,j >], where q ≤ n + 1, z i is an attribute, such as Age, Income and Balance, A i,j is a fuzzy interval, such as Low, Medium and High, t k is a transaction, and N is the number of transactions.Table 1.The example of data.

Fuzzy Associative Classification Rule
Fuzzy associative classification rule and Fuzzy association rule are similar techniques in terms of processing steps, but they differ at consequent part of rule.The fuzzy associative classification rule must contain only one class label at consequent part of rule ( = 1 , , k ).Therefore, the fuzzy confidence of fuzzy associative classification rule can be defined as follows:

Our Proposed Methodology: CCFAR
In this section, we described our algorithm that has been named data classification algorithm with compact fuzzy association rules (CCFAR) to obtain the fuzzy associative classification rules with optimum combination of accuracy and interpretability of the model.Our methodology is composed of five steps (Fig 2 ).
1) Data Screening: The data characteristic of CCFAR must be numeric data because of the transformation of fuzzy set that will convert numeric data only.At current stage, our algorithm assumes that there is no missing value in the data.
2) Data Partitioning: This step is transforming the numeric data to fuzzy intervals or fuzzy sets.In the CCFAR algorithm, we used FCM to make the transformation because of it easy-to-understand property and high performance.As an example, we used the data from Table 1 and we defined fuzzy intervals to be 3 partitions; the results were represented in table 2.

Fig. 2. The development of data classification algorithm
with compact fuzzy association rules.
3) Frequent Fuzzy Item-Sets Searching: The fuzzy frequent item-sets are fuzzy item-sets that have fuzzy support more than minimum fuzzy support (γ).At this step, the process are the same as CFARC (7) algorithm (Fig. 3).
4) Fuzzy classification association rule (FCARs) generation: This step is similar to the CFARC (7) algorithm (Fig. 4), but it differed in the computation of score values of all rules.In this research we computed score values of all rules by using our new metric as shown in Eq.9 fuzzy correlation (Eq.7), fuzzy confidence (Eq.6) and firing strength( ) (Eq.8).The intuitive idea is that we need a high correlated, high confidence, as well as high support rules.FCARs Fig. 3.The frequent fuzzy item set searching (7) .Fig. 4. The fuzzy classification association rule generation (7) 5) Fuzzy Classification Association Rule Selection: This step is selecting the FCARs to be used in the data prediction.It's composing of 4 parts as follows (Fig. 5): -Frist part: The rules that have the same class label and the same size will be grouped together.After that, this algorithm will select rule with the highest SCORE from each group.
-Second part: The rules with the highest SCORE in each group will be counted the frequency of attributes in antecedent of rule.For example: From Table 3, the frequency of attribute Age is 5 (No. 1, 2, 3, 4 and 5), attribute Inc is 3 (No.3, 5 and 6) and attribute Bal is 1 (No. 2).Thus, the highest frequent attribute is the attribute Age.*If frequencies of attributes are equal, attributes will be randomly selected.

Experimentation and Results
In this section, we evaluated our CCFAR algorithm using three measurements accuracy: Acc (Eq.10), number of the models (Compact Value: CV) that we normalized it to be the Normalization of Compact Value: NCV (Eq.11), and the combination of Acc and NCV in which we call Suitable Rule: SR (Eq.12).Our algorithm was tested on three different data sets and compared with nine different algorithms.These algorithms are grouped as: data classification, associative classification and fuzzy associative classification.The nine algorithms are namely C4.5, RIPPER, OneR, CBA, GARC, OAC, FURIA, CFAR, and CFARC.The data are taken from the UC Irvine Machine Learning Repository, namely Iris, Heart, and Pima (details shown in Table 4).The classification performances of all algorithms were measured by ten-fold cross validation.

Frequent 1 . 1 . 1 .
fuzzy item set searching (an Apriori fuzzy implementation) Input: DF fuzzy data Output: the set of frequent fuzzy item set Method: Determine the supports of the classes by the distribution of classes; 2. Set the minimal fuzzy support (γ) to the half of the minimum frequency of classes; 3. Generate the 1-candidate fuzzy items; 4. Calculate FS values then select the frequent fuzzy items from the 1-candidate which has FS > γ, and n = 2; 5.While there exist some n-1 size frequent item sets: Generate the n-size candidate sets from n-1 size frequents (and 1-size frequents); Fuzzy classification association rule generation Input: a set of frequent fuzzy item sets Output: positive correlated FCARs separated by size Method: Generate association rules with class label consequent from all the frequent item sets to consider the size of item sets 2. Calculate the Score values of all the rules; 3. Select rules with positive SCORE value for all size; Fuzzy classification association rule selection Input: a set of FCARs Output: the Compact FCARs Method: Select rules with the highest SCORE for each class in each size of the rules; 2. Find maximum frequent attribute from step 1 then select OneR; 3. Select highest SCORE and shortest rule from a set of FCARs with OneR.(If a set of FCARs does not have a set of OneR then create new rule); 4. Remove redundancy rules by select the shortest rule; the number of true positive examples, FP is the number of false positive examples, TN is the number of true negative examples and FN is the number of false negative examples.N V AL = Avg(CV) ax −Avg(CV) AL Avg(CV) ax −Avg(CV) n Av ( V) is an average of compact value, AL is an Algorithm.

Table 2 .
The degrees of attribute age with FCM algorithm.

Table 3 .
The FCARs with SCORE.

Table 5 .
The Accuracy (Acc) of classification of CCFAR algorithm and other algorithms.

Table 6 .
The number of classification rules (Compact Value) of CCFAR algorithm and other algorithms.