ART DataConfig Example

Overview

In their derivations, ART modules have some special requirements when it comes to their input features. FuzzyART in particular, and subsequently its derivatives, has a requirement that the inputs be bounded and complement coded. This is due to some consequences such as weight decay that occur when using real-valued patterns rather than binary ones (and hence operations like fuzzy membership).

Preprocessing of the features occurs as follows:

1. The features are linearly normalized from 0 to 1 with respect to each feature with linear_normalization. This is done according to some known bounds that each feature has.
2. The features are then complement coded, meaning that the feature vector is appended to its 1-complement (i.e., $x \rightarrow \left[x, 1-x\right]$) with complement_code.

This preprocessing has the ultimate consequence that the input features must be bounded. This many not be a problem in some offline applications with a fixed dataset, but in others where the bounds are not known, techniques such as sigmoidal limiting are often used to place an artificial limit.

DataConfig

Regardless, this process requires some a-priori knowledge about the minimums and maximums that each feature can have, which is stored as a preprocessing configuration. This preprocessing configuration is saved in every ART module as a DataConfig object called config, which we can see is uninitialized at first:

# Load the library
using AdaptiveResonance

# Create a new ART module and inspect its uninitialized data config `config`
art = FuzzyART()
art.config

DataConfig(false, Float64[], Float64[], 0, 0)

We see that the type of art.config is DataConfig. We can see what the internal elements of this struct are with fieldnames:

fieldnames(AdaptiveResonance.DataConfig)

(:setup, :mins, :maxs, :dim, :dim_comp)

We see that the dataconfig has a boolean setup flag, minimum and maximum feature vectors, dimensionality of the data, and the complement coded dimensionality (twice the size of the original dimension).

Automatic Configuration

In batch training mode, the minimums and maximums are detected automatically; the minimum and maximum values for every feature are saved and used for the preprocessing step at every subsequent iteration.

# Load data
using MLDatasets        # Iris dataset
using DataFrames        # DataFrames, necessary for MLDatasets.Iris()
using MLDataUtils       # Shuffling and splitting

We will download the Iris dataset for its small size and benchmark use for clustering algorithms.

# Get the iris dataset
iris = Iris(as_df=false)
# Manipulate the features and labels into a matrix of features and a vector of labels
features, labels = iris.features, iris.targets

([5.1 4.9 4.7 4.6 5.0 5.4 4.6 5.0 4.4 4.9 5.4 4.8 4.8 4.3 5.8 5.7 5.4 5.1 5.7 5.1 5.4 5.1 4.6 5.1 4.8 5.0 5.0 5.2 5.2 4.7 4.8 5.4 5.2 5.5 4.9 5.0 5.5 4.9 4.4 5.1 5.0 4.5 4.4 5.0 5.1 4.8 5.1 4.6 5.3 5.0 7.0 6.4 6.9 5.5 6.5 5.7 6.3 4.9 6.6 5.2 5.0 5.9 6.0 6.1 5.6 6.7 5.6 5.8 6.2 5.6 5.9 6.1 6.3 6.1 6.4 6.6 6.8 6.7 6.0 5.7 5.5 5.5 5.8 6.0 5.4 6.0 6.7 6.3 5.6 5.5 5.5 6.1 5.8 5.0 5.6 5.7 5.7 6.2 5.1 5.7 6.3 5.8 7.1 6.3 6.5 7.6 4.9 7.3 6.7 7.2 6.5 6.4 6.8 5.7 5.8 6.4 6.5 7.7 7.7 6.0 6.9 5.6 7.7 6.3 6.7 7.2 6.2 6.1 6.4 7.2 7.4 7.9 6.4 6.3 6.1 7.7 6.3 6.4 6.0 6.9 6.7 6.9 5.8 6.8 6.7 6.7 6.3 6.5 6.2 5.9; 3.5 3.0 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 3.7 3.4 3.0 3.0 4.0 4.4 3.9 3.5 3.8 3.8 3.4 3.7 3.6 3.3 3.4 3.0 3.4 3.5 3.4 3.2 3.1 3.4 4.1 4.2 3.1 3.2 3.5 3.1 3.0 3.4 3.5 2.3 3.2 3.5 3.8 3.0 3.8 3.2 3.7 3.3 3.2 3.2 3.1 2.3 2.8 2.8 3.3 2.4 2.9 2.7 2.0 3.0 2.2 2.9 2.9 3.1 3.0 2.7 2.2 2.5 3.2 2.8 2.5 2.8 2.9 3.0 2.8 3.0 2.9 2.6 2.4 2.4 2.7 2.7 3.0 3.4 3.1 2.3 3.0 2.5 2.6 3.0 2.6 2.3 2.7 3.0 2.9 2.9 2.5 2.8 3.3 2.7 3.0 2.9 3.0 3.0 2.5 2.9 2.5 3.6 3.2 2.7 3.0 2.5 2.8 3.2 3.0 3.8 2.6 2.2 3.2 2.8 2.8 2.7 3.3 3.2 2.8 3.0 2.8 3.0 2.8 3.8 2.8 2.8 2.6 3.0 3.4 3.1 3.0 3.1 3.1 3.1 2.7 3.2 3.3 3.0 2.5 3.0 3.4 3.0; 1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 1.5 1.6 1.4 1.1 1.2 1.5 1.3 1.4 1.7 1.5 1.7 1.5 1.0 1.7 1.9 1.6 1.6 1.5 1.4 1.6 1.6 1.5 1.5 1.4 1.5 1.2 1.3 1.5 1.3 1.5 1.3 1.3 1.3 1.6 1.9 1.4 1.6 1.4 1.5 1.4 4.7 4.5 4.9 4.0 4.6 4.5 4.7 3.3 4.6 3.9 3.5 4.2 4.0 4.7 3.6 4.4 4.5 4.1 4.5 3.9 4.8 4.0 4.9 4.7 4.3 4.4 4.8 5.0 4.5 3.5 3.8 3.7 3.9 5.1 4.5 4.5 4.7 4.4 4.1 4.0 4.4 4.6 4.0 3.3 4.2 4.2 4.2 4.3 3.0 4.1 6.0 5.1 5.9 5.6 5.8 6.6 4.5 6.3 5.8 6.1 5.1 5.3 5.5 5.0 5.1 5.3 5.5 6.7 6.9 5.0 5.7 4.9 6.7 4.9 5.7 6.0 4.8 4.9 5.6 5.8 6.1 6.4 5.6 5.1 5.6 6.1 5.6 5.5 4.8 5.4 5.6 5.1 5.1 5.9 5.7 5.2 5.0 5.2 5.4 5.1; 0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 0.2 0.2 0.1 0.1 0.2 0.4 0.4 0.3 0.3 0.3 0.2 0.4 0.2 0.5 0.2 0.2 0.4 0.2 0.2 0.2 0.2 0.4 0.1 0.2 0.1 0.2 0.2 0.1 0.2 0.2 0.3 0.3 0.2 0.6 0.4 0.3 0.2 0.2 0.2 0.2 1.4 1.5 1.5 1.3 1.5 1.3 1.6 1.0 1.3 1.4 1.0 1.5 1.0 1.4 1.3 1.4 1.5 1.0 1.5 1.1 1.8 1.3 1.5 1.2 1.3 1.4 1.4 1.7 1.5 1.0 1.1 1.0 1.2 1.6 1.5 1.6 1.5 1.3 1.3 1.3 1.2 1.4 1.2 1.0 1.3 1.2 1.3 1.3 1.1 1.3 2.5 1.9 2.1 1.8 2.2 2.1 1.7 1.8 1.8 2.5 2.0 1.9 2.1 2.0 2.4 2.3 1.8 2.2 2.3 1.5 2.3 2.0 2.0 1.8 2.1 1.8 1.8 1.8 2.1 1.6 1.9 2.0 2.2 1.5 1.4 2.3 2.4 1.8 1.8 2.1 2.4 2.3 1.9 2.3 2.5 2.3 1.9 2.0 2.3 1.8], InlineStrings.String15["Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-setosa" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-versicolor" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica" "Iris-virginica"])

Because the MLDatasets package gives us Iris labels as strings, we will use the MLDataUtils.convertlabel method with the MLLabelUtils.LabelEnc.Indices type to get a list of integers representing each class:

labels = convertlabel(LabelEnc.Indices{Int}, vec(labels))
unique(labels)

3-element Vector{Int64}:
 1
 2
 3

Manual Configuration

As mentioned before, we may not always have the luxury of having a representative dataset in advance. Alternatively, we may know the bounds of the features but wish to run incrementally rather than in batch. In these cases, we can setup the config the various DataConfig constructors.

For example, if the features are all bounded from -1 to 1, we have to also specify the original dimension of the data in DataConfig(min, max, dim):

# Reinitialize the FuzzyART module
art = FuzzyART()
# Tell the module that we have 20 features all ranging from -1 to 1
art.config = DataConfig(-1, 1, 20)

DataConfig(true, [-1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], 20, 40)

If the features differ in their ranges, we can specify with DataConfig(mins, maxs):

# Assume some minimum and maximum values for each feature
mins = [-1,-2,-1.5]
maxs = [3, 2, 1]
art.config = DataConfig(mins, maxs)

DataConfig(true, [-1.0, -2.0, -1.5], [3.0, 2.0, 1.0], 3, 6)

Here, we don't need to specify the feature dimensionality because it is inferred from the length of the range values.

This page was generated using DemoCards.jl and Literate.jl.