SONFIS

Self-Organizing Neuro-Fuzzy Inference System (SONFIS). A structural adaptation algorithm that combines a parameter learning algorithm with three operators — GrowNet, SplitSubNet, and VanishNet — to iteratively update both the parameters and the rule structure of a rule-reduced ANFIS model.

At each iteration, GrowNet is attempted first: if no new subnets are grown, SplitSubNet is attempted. VanishNet is always applied regardless of the outcomes of the previous operators. The algorithm stops when no structural updates occur or the maximum number of iterations is reached.

Warning

This algorithm is only applicable to instances of neuro_fuzzy_toolbox.models.rule_reduced_ANFIS.

Note

For more details on the SONFIS algorithm, refer to the original paper: SONFIS: Structure Identification and Modeling with a Self-Organizing Neuro-Fuzzy Inference System.
For more details on its implementation in the toolbox, see SONFIS.

Instantiation

The following parameters are available when instantiating the SONFIS algorithm:

Ngrow (int): Minimum number of poorly modeled samples required to grow a new subnet.
dGrow (float): Threshold for identifying poorly modeled samples. A sample is considered poorly modeled if its maximum firing level across all subnets is less than or equal to this value raised to the power of the input dimensionality.
Nsplit (int): Minimum number of samples associated with a subnet for it to be considered for splitting.
eSplit (float): Minimum loss value of the samples associated with a subnet for it to be considered for splitting.
Nvanish (int): Maximum number of samples associated with a subnet below which its age counter is incremented.
lVanish (int): Maximum age of a subnet before it is removed.
max_iterations (int): Maximum number of structural adaptation iterations.
ANFIStrainer (base_model_trainer): Instantiated training algorithm that defines how the model parameters are updated at each iteration. Any training algorithm available in Neuro-Fuzzy Toolbox can be used.
early_stopping (nft.EarlyStopping): Early stopping mechanism applied at the SONFIS iteration level (Default: None).
lse_for_new_consequents (bool): If True, the consequent parameters of rules generated by GrowNet or SplitSubNet are initialized using least-squares estimation instead of random initialization (Default: False).
lse_for_new_consequents_lambda (float): Lambda value for Ridge regularization in the least-squares initialization of new consequent parameters. If 0., no regularization is applied (Default: 0.).
last_training_iteration (bool): If True, performs a final parameter update over all subnets after the SONFIS algorithm finishes (Default: False).

Example

First, instantiate the parameter training algorithm to be used inside SONFIS:

import neuro_fuzzy_toolbox as nft
import torch
import torch.nn as nn

ANFIStrainer = nft.Hybrid_learning_algorithm(
    epochs=500,
    loss_function=nn.MSELoss(),
    optimizer=torch.optim.AdamW,
    optimizer_params={'lr': 1e-4, 'weight_decay': 1e-3},
    early_stopping=nft.EarlyStopping(patience=40, delta=1e-4)
)

Then, instantiate the SONFIS algorithm:

sonfis = nft.SONFIS(
    Ngrow=30,
    dGrow=0.8,
    Nsplit=25,
    eSplit=1.2,
    Nvanish=5,
    lVanish=3,
    max_iterations=100,
    ANFIStrainer=ANFIStrainer,
    early_stopping=nft.EarlyStopping(patience=7, delta=0.01),
    last_training_iteration=True
)

The following arguments are available when calling the algorithm via __call__:

ANFISmodel: Rule-reduced ANFIS model to train.
train_loader: DataLoader with the training data.
val_loader: DataLoader with the validation data (Default: None).
verbose: Whether to print progress and structural update messages at each iteration (Default: True).

Assuming model is an instantiated rule_reduced_ANFIS model and train_loader and val_loader are PyTorch DataLoaders, training is invoked as follows:

sonfis(model, train_loader, val_loader, verbose=True)

Important

The early stopping mechanism passed to ANFIStrainer operates at the parameter update level within each SONFIS iteration, monitoring the validation loss computed on the val_loader passed to __call__. The early stopping mechanism passed directly to SONFIS operates at the iteration level and monitors the overall validation loss across iterations. Both are independent and can be configured separately.

Important

The training batch size is determined by the DataLoader, so this should be taken into account when defining it (see PyTorch DataLoaders).