Copyright: Copyright RuleQuest Research 1997


Overview of See5
Overview

See5 analyzes data to produce decision trees and/or rulesets that relate a case’s class to the values of its attributes.

An application is a collection of text files.  These files define classes and attributes, describe the cases to be analyzed, provide new cases to test the classifiers produced by See5, and specify misclassification costs or penalties.

This help contains concise instructions for setting up See5 applications files and running See5.  A more detailed tutorial is available in browser format.

  Click to view the tutorial.


 File Names and Types

Every file associated with an application has a name of the form filestem.extension where filestem identifies the application and extension defines the file’s type or function.  Uppercase and lowercase letters in the filestem and the extension are not interchangeable; in particular, the extension should use only lowercase letters exactly as indicated below.

A See5 application consists of two mandatory files and two optional files.

Names file filestem.names (required).  This defines the attributes and classes associated with the data for this application.

 	Data file filestem.data (required). This provides the actual cases that will be analyzed by See5 in order to produce a classifier.

 	Test file filestem.test (optional). This file contains cases that are not used to produce a classifier, but instead allow the predictive accuracy of the classifier to be estimated.

 	Costs file filestem.costs (optional). In some applications there is a much higher penalty for making classification errors of certain kinds.  This file defines these penalties in the form of costs.

See5 writes files of two main types:

 	Results file filestem.out.  This file contains the output for the most recent classifier constructed for this application.

 	Classifier files filestem.tree, filestem.rules and filestem.set (for use by See5 only).


Names File

The names file filestem.names contains a series of entries defining classes, attributes, and their values.  The file is free-format with the exception that the vertical bar ‘|’ causes the rest of that line to be skipped.  Each entry is terminated with a period.

Each name is an arbitrary string of characters with some restrictions:

•	A name may contain tabs and spaces, but any sequence of tabs and spaces is treated as a single space.

•	Any special characters ‘,’ (comma), ‘:’ (colon), ‘.’ (period) or ‘|’ (vertical bar) that are part of a name must be prefixed by the escape character ‘\’ (backslash).


Class and attribute names
Classes and Attributes

  Click to see a sample names file.

The first entry in the names file specifies the classes, either by nominating a discrete-valued attribute that contains the class values, or by defining the classes directly as a list of class names separated by commas.

The rest of the names file consists of one entry for each attribute.  Attributes are of two kinds: explicitly-defined attributes (specified by type, values etc.) and implicitly-defined attributes (specified by formulas).


Explicitly-Defined Attributes

The entry for an explicitly-defined attribute begins with the attribute name followed by a colon (‘:’), and then one of the following:

‘ignore’, indicating that this attribute should not be used in classifiers.

‘label’, indicating that the value of this attribute is used only to identify particular cases. A label attribute can be useful for allowing particular cases to be identified easily, especially in cross-referencing (see below).  If more than one label attribute appears, only the last is used.

‘continuous’, for attributes with numeric values.

‘date’, for attributes whose values take the form YYYY-MM-DD or YYYY/MM/DD.

‘discrete’ followed by an integer N, instructing See5 to assemble a list of up to N values that appear in the training cases.

a list of the allowable discrete values of the attribute, separated by commas.  (This form is preferred to the previous one since it allows the input to be checked.  Also, an attribute nominated as the class attribute must be of this form rather than the ‘discrete N’ form above.)

The list of values can be prefaced by ‘[ordered]’ to indicate that the values are given in a meaningful order – See5 can exploit this information to produce more sensible groupings of values.

The entry for each attribute is terminated by a period.


Implicitly-Defined Attributes

An implicitly-defined attribute is one whose value is calculated from the values of previously-defined attributes.  An entry for an implicitly-defined attribute has the form

attribute name := formula .

The formula is written in the usual way, using parentheses where needed, and may refer to any attribute defined before this one.  Constants in the formula can be numbers, discrete attribute values enclosed in string quotes (e.g., “small”), and dates.  The operators and functions that can be used in formulas are:

and,  or

+,  -,  *,  /,  % (modulus), ^ (power)

>,  >=,  <,  <=,  =, !=,  <> (the last two both meaning not equal)

log, exp, sin, cos , tan, int (integer part of)

Attributes defined in this way have values that are either numbers or logical values true and false, depending on the formula.  For example, x := a + b. defines a number, but x := a > b. defines a logical value.

Dates YYYY/MM/DD or YYYY-MM-DD are stored internally as the number of days from a base date, and so can appear in some formulas.  For example, subtracting one date from another gives the number of days between them.  Also, if D is a date attribute, then

day of week := (D + 1) % 7 + 1.

defines a new attribute ‘day of week’ whose value is the day of the week on which D falls, ranging from 1 (Monday) to 7 (Sunday).


Data File

The data file filestem.data contains the cases that will be analyzed in order to produce the classifier.  Each case is represented by a separate entry in the file; the order of cases in the data file is not important.

A case entry consists of the values of the explicitly-defined attributes (in the same order that they appear in the names file).  If the names file defines the class directly rather than by nominating a class attribute, these entries are followed by the class value.  All values are separated by commas and the entry is terminated with a period. Unknown values for one or more of the attributes are represented by a question mark (‘?’) and non-applicable values by ‘N/A’.

Note that there are no entries for implicitly-defined attributes since these are computed from the values of other attributes using the given formula. If one or more of the previously-defined attributes has an unknown value so that the formula cannot be evaluated, the value of the implicitly-defined attribute is also unknown.

As with the names file, the rest of the line after a vertical bar is ignored, and multiple whitespace characters are treated as a single space.

  Click for an example.


Test File

The optional test file filestem.test contains cases in the same format as the data file.  After the cases in the data file have been analyzed, the predictive accuracy of the classifier generated from them can be estimated by evaluating it on new cases.  This is carried out automatically by See5 whenever a test file is present.


Costs File

In many applications, a classifier is evaluated in terms of its error rate on new cases.  However, some applications do not adhere to this simple pattern because misclassification errors of one kind are more serious than those of another kind.  Consider a credit screening application in which applicants for credit cards are classified as good or bad credit risks.  The consequence of misclassifying a good credit risk as bad is a lost credit card sale; the cost of misclassifying a bad credit risk as good can be much higher.

See5 allows for an optional costs file filestem.costs that specifies different costs for some or all combinations of actual and predicted class.  Each line in this file has the form

predicted class, actual class : cost

where the cost is a non-negative real number.  It is not necessary to cover all combinations of classes since an unspecified combination has the default cost of 1.0.

If the costs file is present, See5 will attempt to construct a classifier that minimizes misclassification cost rather than error rate.

  Click for an example.


Locate data
Step 1: Locate the Application’s Data

The first step in constructing a classifier for an application is to locate that application’s data files on your computer.  All these files must be together in one folder, although the same folder may contain several applications.

The left-most button on the toolbar invokes a standard specify-or-browse dialog to find a file whose name is filestem.data for some filestem.  Identifying this file provides the application filestem, and See5 then looks in the folder for other relevant files for this application.

After the application’s data has been located, the edit menu allows the .names file and .costs file to be edited using Wordpad.  If no .costs file exists, a stub file is created before Wordpad is invoked.


Select options
Step 2: Select Options

Once the application has been identified, See5 can then be used to construct a classifier.  Several options that can be used to influence this process are specified in the following dialog box:

See5 can construct classifiers as decision trees or rulesets.  The default (decision trees) is used unless the Rulesets box is checked.

Rules can be listed by class or by their importance to classification accuracy.  If the latter utility ordering is selected, the rules are grouped into a number of bands (default 5).  Errors and costs are reported individually for the first band, the first two bands, and so on.

By default See5 generates a single classifier.  The Boosting option causes a number of classifiers to be constructed; when a case is classified, all these classifiers are consulted before a decision is made.  Boosting often gives higher predictive accuracy at the expense of increased classifier construction time.

By default, See5 deals separately with each value of an unordered discrete attribute.  If the Subset option is chosen, these values are grouped into subsets.

If the data are very numerous, the Sampling option may be useful.  This causes only the specified percentage of the cases in the filestem.data file to be used for constructing the classifier.  This option automatically causes the classifier to be tested on a disjoint sample of cases in the filestem.data file.  If the training sample is less than 50%, this test sample is the same size; otherwise, it contains all the cases that do not appear in the training sample.  (So, if the sample option is set to 30%, the test sample will also contain 30% of the cases; if the sample option is set to 75%, the test sample will consist of the remaining 25% of the cases.)  By default, the samples are redrawn every time a classifier is constructed, so the classifiers built with the sampling option will usually change too.  This resampling can be prevented using the Lock sample option; the sample is unchanged until a different application is loaded, the sample percentage is altered, the option is unselected, or See5 is restarted.

The Cross-validate option can be used to estimate the accuracy of the classifier constructed by See5 even when there are no separate test cases.  The data are split into a number of blocks equal to the chosen number of folds (default 10).  Each block contains approximately the same number of cases and the same distribution of classes.  For each block in turn, See5 constructs a classifier using the cases in all the other blocks and then tests its accuracy on the cases in the hold-out block.  In this way, each case in the data is used just once as a test case.  The error rate of the classifier produced from all the cases is estimated as the ratio of the total number of errors on the hold-out cases to the total number of cases.  (The results for a cross-validation depend on the way that the cases are divided into blocks.  This division changes each time a cross-validation is carried out, so don’t be surprised if the results are not identical to the results from previous cross-validations with the same data.) Since the classifiers constructed during a cross-validation use only part of the training data, no classifier is saved when this option is selected.

In applications with differential misclassification costs, it is sometimes desirable to see what effect the costs file is having on the construction of the classifier.  If the Ignore costs file box is checked, See5 will construct a classifier as if all misclassification costs are the same.

The remaining options are intended for advanced users who are familiar with the way See5 works.  If the following descriptions do not make much sense to you, it is probably a good idea to use their default values.

When a continuous attribute is tested in a decision tree, there are branches corresponding to the conditions

attribute value <= threshold and attribute value > threshold

for some threshold chosen by See5.  As a result, small movements in the attribute value near the threshold can change the branch taken from the test.  The Fuzzy thresholds option softens this knife-edge behavior for decision trees by constructing an interval close to the threshold.  Within this interval, both branches of the tree are explored and the results combined to give a predicted class.  Note: fuzzy thresholds do not affect the behavior of rulesets.

The decision trees and rulesets constructed by See5 are simplified before they are presented to the user.  The Pruning CF parameter governs the extent of this simplification.  A higher value produces more elaborate decision trees and rulesets, while a lower value causes more extensive simplification.

The final parameter also influences the complexity of the classifier.  Setting a higher minimum number of cases restricts the tests that can be used near the leaves of the decision tree and usually leads to smaller decision trees and rulesets.


Examine, edit, or print the results
Step 3: Examine, Edit, or Print the Results

When the classifier has been constructed, See5 displays a new window that presents the classifier (or classifiers) and associated results.  The window is equipped with a scroll tab that allows different parts of the output to be viewed.  The output is also saved in a file named filestem.out, overwriting the output produced when any previous classifiers for this application were constructed.  The File tab provides an option to print the results, while the Edit tab provides a facility to select the results and copy them to the clipboard so that they can be pasted into other applications.

The systems menu (obtained by clicking the small icon in the top left-hand corner of the results window) contains an additional action Switch to Wordpad.  This causes the results viewer to be replaced with the Wordpad editor.  Wordpad allows the results to be edited, searched, printed, and saved to a different file if required.


Classifying New Cases Interactively

The classifier generated in Step 2 can be used to predict the class of a new case.  If more than one classifier has been produced, See5 always uses the most recent.

See5 will ask you for the values of the attributes needed to assign the case to a class.  The attributes requested, and the order in which they are requested, depend on the classifier itself.  For instance, a decision tree classifier will first request the value of the attribute tested at the root of the tree (or the attributes needed to determine its value, if the root attribute has been defined by a formula); the next attribute requested will depend on the branch taken from the root, and so on.

When See5 requests the value of a continuous attribute, you should key in a number , ‘?’  (meaning ‘not known’), or ‘N/A’ (meaning ‘not applicable’) followed by enter.  Similarly, a date should be entered as ‘?’ or YYYY-MM-DD or YYYY/MM/DD.  For a discrete attribute, See5 will display the possible values of that attribute together with ‘?’; you should select one of them by double-clicking on it.

After all the necessary attribute values have been entered, See5 shows the class to which the case is predicted to belong.  If there are two or more plausible classes, See5 will show the most likely two or three and indicate their plausibility with a small histogram.  You can then choose to classify a new case, retry the same case changing some of the attribute values, or return to the See5 main window.


Cross-Referencing Classifiers and Data

Complex classifiers, especially those generated with the boosting option, can be difficult to understand. See5 incorporates a unique facility that links data and the relevant sections of (possibly boosted) classifiers. The Cross-Reference button brings up a window showing the most recent classifier for the current application and how it relates to the cases in the data, test or cases file.  (If more than one of these is present, a menu will prompt you to select the file.)

The window is divided into two panes, with the classifier on the left and a list of cases on the right. Each case has a [?] tag (that is red if the case is misclassified), its label or number, and its actual class. Clicking on the label or case number shows the part(s) of the classifier(s) relevant to that case.  This might be a single rule or a single leaf, but under certain conditions it might be several leaves or rules.  These conditions are: the selected case has missing values for one or more attributes, so that it cannot be mapped to a single leaf of a decision tree; the case is covered by two or more rules in a ruleset; or boosted classifiers are being used, so that each constituent classifier will usually have one or more leaves or rules relevant to the case.

Clicking on a leaf or a rule shows all cases to which that leaf or rule is relevant.

The attribute values of a particular case can be displayed by clicking on the [?] text to the left of the case label or number. Values of attributes marked ‘label’ or ‘ignore’ appear lighter as a reminder that they play no part in classification.

The save button causes the information displayed in the cross-reference  window to be written as an ascii file.  When this button is pressed, a dialog box prompts for the name of the file to be used.

 Finally, clicking on the reset button re-initializes the window to show the complete classifier and all cases.