.. _sonfis-usage:

SONFIS
======

Self-Organizing Neuro-Fuzzy Inference System (SONFIS). A structural
adaptation algorithm that combines a parameter learning algorithm with three
operators — GrowNet, SplitSubNet, and VanishNet — to iteratively update both
the parameters and the rule structure of a rule-reduced ANFIS model.

At each iteration, GrowNet is attempted first: if no new subnets are grown,
SplitSubNet is attempted. VanishNet is always applied regardless of the
outcomes of the previous operators. The algorithm stops when no structural
updates occur or the maximum number of iterations is reached.

.. figure:: ../../_static/SONFIS_pseudocode.png
    :width: 100%
    :align: center

.. warning::
    This algorithm is only applicable to instances of
    :class:`neuro_fuzzy_toolbox.models.rule_reduced_ANFIS`.

.. note::
    - For more details on the SONFIS algorithm, refer to the original paper:
      `SONFIS: Structure Identification and Modeling with a Self-Organizing
      Neuro-Fuzzy Inference System <https://doi.org/10.1080/18756891.2016.1175809>`_.
    - For more details on its implementation in the toolbox, see
      :ref:`SONFIS`.

Instantiation
-------------
The following parameters are available when instantiating the SONFIS
algorithm:

- **Ngrow** (``int``): Minimum number of poorly modeled samples required to
  grow a new subnet.
- **dGrow** (``float``): Threshold for identifying poorly modeled samples. A
  sample is considered poorly modeled if its maximum firing level across all
  subnets is less than or equal to this value raised to the power of the input
  dimensionality.
- **Nsplit** (``int``): Minimum number of samples associated with a subnet for
  it to be considered for splitting.
- **eSplit** (``float``): Minimum loss value of the samples associated with a
  subnet for it to be considered for splitting.
- **Nvanish** (``int``): Maximum number of samples associated with a subnet
  below which its age counter is incremented.
- **lVanish** (``int``): Maximum age of a subnet before it is removed.
- **max_iterations** (``int``): Maximum number of structural adaptation
  iterations.
- **ANFIStrainer** (``base_model_trainer``): Instantiated training algorithm
  that defines how the model parameters are updated at each iteration. Any
  training algorithm available in Neuro-Fuzzy Toolbox can be used.
- **early_stopping** (``nft.EarlyStopping``): Early stopping mechanism applied
  at the SONFIS iteration level (Default: ``None``).
- **lse_for_new_consequents** (``bool``): If ``True``, the consequent
  parameters of rules generated by GrowNet or SplitSubNet are initialized
  using least-squares estimation instead of random initialization
  (Default: ``False``).
- **lse_for_new_consequents_lambda** (``float``): Lambda value for Ridge
  regularization in the least-squares initialization of new consequent
  parameters. If ``0.``, no regularization is applied (Default: ``0.``).
- **last_training_iteration** (``bool``): If ``True``, performs a final
  parameter update over all subnets after the SONFIS algorithm finishes
  (Default: ``False``).

Example
#######
First, instantiate the parameter training algorithm to be used inside SONFIS:

.. code-block:: python

    import neuro_fuzzy_toolbox as nft
    import torch
    import torch.nn as nn

    ANFIStrainer = nft.Hybrid_learning_algorithm(
        epochs=500,
        loss_function=nn.MSELoss(),
        optimizer=torch.optim.AdamW,
        optimizer_params={'lr': 1e-4, 'weight_decay': 1e-3},
        early_stopping=nft.EarlyStopping(patience=40, delta=1e-4)
    )

Then, instantiate the SONFIS algorithm:

.. code-block:: python

    sonfis = nft.SONFIS(
        Ngrow=30,
        dGrow=0.8,
        Nsplit=25,
        eSplit=1.2,
        Nvanish=5,
        lVanish=3,
        max_iterations=100,
        ANFIStrainer=ANFIStrainer,
        early_stopping=nft.EarlyStopping(patience=7, delta=0.01),
        last_training_iteration=True
    )

The following arguments are available when calling the algorithm via
``__call__``:

- **ANFISmodel**: Rule-reduced ANFIS model to train.
- **train_loader**: DataLoader with the training data.
- **val_loader**: DataLoader with the validation data (Default: ``None``).
- **verbose**: Whether to print progress and structural update messages at
  each iteration (Default: ``True``).

Assuming ``model`` is an instantiated ``rule_reduced_ANFIS`` model and
``train_loader`` and ``val_loader`` are PyTorch DataLoaders, training is
invoked as follows:

.. code-block:: python

    sonfis(model, train_loader, val_loader, verbose=True)

.. important::
    The early stopping mechanism passed to ``ANFIStrainer`` operates at the
    **parameter update** level within each SONFIS iteration, monitoring the
    validation loss computed on the ``val_loader`` passed to ``__call__``. The
    early stopping mechanism passed directly to SONFIS operates at the
    **iteration** level and monitors the overall validation loss across
    iterations. Both are independent and can be configured separately.

.. important::
    The training batch size is determined by the DataLoader, so this should
    be taken into account when defining it (see
    :ref:`PyTorch DataLoaders <DataLoaders_usage>`).