Training Algorithms

This module contains the documentation for the training algorithms available in Neuro-Fuzzy Toolbox.

Hybrid Learning Algorithm

class neuro_fuzzy_toolbox.training.training_algorithms.Hybrid_learning_algorithm(epochs, loss_function, driver=None, ridge_lambda=0.0, early_stopping=None, optimizer=torch.optim.Adam, optimizer_params={})[source]

Bases: base_model_trainer

Hybrid learning algorithm for training an Adaptive Neuro-Fuzzy Inference System (ANFIS) model.

Note

This algorithm is based on the hybrid training algorithm proposed by Jang in 1993. For more information, see: ANFIS: adaptive-network-based fuzzy inference system.

Pseudocode:

Pseudocode of the hybrid learning algorithm.
__call__(model, train_loader, val_loader=None, verbose=True)

Trains a model using the training algorithm.

Parameters:

  • model (torch.nn.Module) – Model to train.

  • train_loader (DataLoader) – DataLoader containing the training data.

  • val_loader (DataLoader) – DataLoader containing the validation data. Defaults to None.

  • verbose (bool) – If True, prints progress messages during training. Defaults to True.

__init__(epochs, loss_function, driver=None, ridge_lambda=0.0, early_stopping=None, optimizer=torch.optim.Adam, optimizer_params={})[source]

Initializes a new Hybrid_learning_algorithm instance.

Parameters:

  • epochs (int) – Number of training epochs.

  • loss_function (torch.nn.Module) – Instantiated loss function to use during training.

  • driver (str) – Backend function to use for the least-squares estimation of the consequent parameters. Valid values are 'gels', 'gelsy', 'gelsd', and 'gelss'. If None, defaults to 'gels'.

  • ridge_lambda (float) – Lambda value for Ridge regularization in the least-squares estimation. If 0., no regularization is applied. Defaults to 0..

  • early_stopping (EarlyStopping) – Early stopping mechanism to use during training. Defaults to None.

  • optimizer (torch.optim.Optimizer) – Optimizer class to use during training. Defaults to torch.optim.Adam.

  • optimizer_params (dict) – Parameters to pass to the optimizer. Defaults to {}.

Basic Optimizer Based Training Algorithm

class neuro_fuzzy_toolbox.training.training_algorithms.Basic_optimizer_training_algorithm(epochs, loss_function, early_stopping=None, optimizer=torch.optim.Adam, optimizer_params={})[source]

Bases: base_model_trainer

Optimizer-based training algorithm for a machine learning model. Trains all model parameters using a single optimizer.

Pseudocode:

Pseudocode of the basic optimizer-based training algorithm.
__call__(model, train_loader, val_loader=None, verbose=True)

Trains a model using the training algorithm.

Parameters:

  • model (torch.nn.Module) – Model to train.

  • train_loader (DataLoader) – DataLoader containing the training data.

  • val_loader (DataLoader) – DataLoader containing the validation data. Defaults to None.

  • verbose (bool) – If True, prints progress messages during training. Defaults to True.

__init__(epochs, loss_function, early_stopping=None, optimizer=torch.optim.Adam, optimizer_params={})[source]

Initializes a new Basic_optimizer_training_algorithm instance.

Parameters:

  • epochs (int) – Number of training epochs.

  • loss_function (torch.nn.Module) – Instantiated loss function to use during training.

  • early_stopping (EarlyStopping) – Early stopping mechanism to use during training. Defaults to None.

  • optimizer (torch.optim.Optimizer) – Optimizer class to use during training. Defaults to torch.optim.Adam.

  • optimizer_params (dict) – Parameters to pass to the optimizer. Defaults to {}.

Double Optimizer Based Training Algorithm

class neuro_fuzzy_toolbox.training.training_algorithms.Double_optimizer_training_algorithm(epochs, loss_function, early_stopping=None, mode=0, prems_optim=torch.optim.Adam, prems_optim_params={}, cons_optim=torch.optim.Adam, cons_optim_params={})[source]

Bases: base_model_trainer

Dual-optimizer training algorithm for an Adaptive Neuro-Fuzzy Inference System (ANFIS) model. Uses separate optimizers to update the premise and consequent parameters independently.

The update strategy is controlled by the mode parameter:

  • Mode 0: In each epoch, a single forward pass is performed and both the premise and consequent parameters are updated simultaneously using their respective optimizers.

  • Mode 1: Each epoch consists of two sequential passes through the data. In the first pass, only the consequent parameters are updated; in the second pass, only the premise parameters are updated.

__call__(model, train_loader, val_loader=None, verbose=True)

Trains a model using the training algorithm.

Parameters:

  • model (torch.nn.Module) – Model to train.

  • train_loader (DataLoader) – DataLoader containing the training data.

  • val_loader (DataLoader) – DataLoader containing the validation data. Defaults to None.

  • verbose (bool) – If True, prints progress messages during training. Defaults to True.

__init__(epochs, loss_function, early_stopping=None, mode=0, prems_optim=torch.optim.Adam, prems_optim_params={}, cons_optim=torch.optim.Adam, cons_optim_params={})[source]

Initializes a new Double_optimizer_training_algorithm instance.

Parameters:

  • epochs (int) – Number of training epochs.

  • loss_function (torch.nn.Module) – Instantiated loss function to use during training.

  • early_stopping (EarlyStopping) – Early stopping mechanism to use during training. Defaults to None.

  • mode (int) – Update strategy to use. 0 updates premises and consequents simultaneously in a single pass; 1 updates them sequentially in two separate passes per epoch. Defaults to 0.

  • prems_optim (torch.optim.Optimizer) – Optimizer class to use for the premise parameters. Defaults to torch.optim.Adam.

  • prems_optim_params (dict) – Parameters to pass to the premise optimizer. Defaults to {}.

  • cons_optim (torch.optim.Optimizer) – Optimizer class to use for the consequent parameters. Defaults to torch.optim.Adam.

  • cons_optim_params (dict) – Parameters to pass to the consequent optimizer. Defaults to {}.

Base Model Trainer

class neuro_fuzzy_toolbox.training.training_algorithms.base_model_trainer(epochs, loss_function, early_stopping=None, optimizer=torch.optim.Adam, optimizer_params={})[source]

Base class for representing a training algorithm for a machine learning model.

Warning

This class should not be instantiated directly. It is intended to be subclassed by other classes.

Note

Subclasses must implement the following methods:

  • _update_parameters(model, loader): executes the parameter update during a single training epoch.

  • init_optimizer(self, model, optimizer, optim_params): instantiates the optimizer to be used during training.

To be compatible with the SONFIS algorithm, subclasses must additionally implement:

  • _sonfis_update_parameters(model, train_loader, val_loader, freezed_subnets): applies the training algorithm within the SONFIS procedure.

  • _sonfis_init_optimizer(model, freezed_subnets): initializes the optimizer for use within the SONFIS procedure.

__call__(model, train_loader, val_loader=None, verbose=True)[source]

Trains a model using the training algorithm.

Parameters:

  • model (torch.nn.Module) – Model to train.

  • train_loader (DataLoader) – DataLoader containing the training data.

  • val_loader (DataLoader) – DataLoader containing the validation data. Defaults to None.

  • verbose (bool) – If True, prints progress messages during training. Defaults to True.