Metrics

Metrics introduction

nngeometry.metrics.FIM(model, loader, representation, n_output, variant='classif_logits', device='cpu', function=None, layer_collection=None)

Helper that creates a matrix computing the Fisher Information Matrix using closed form expressions for the expectation y|x as described in (Pascanu and Bengio, 2013)

Parameters
  • model (torch.nn.Module) – The model that contains all parameters of the function

  • loader (torch.utils.data.DataLoader) – DataLoader for computing expectation over the input space

  • representation (class) – The parameter matrix representation that will be used to store the matrix

  • n_output (int) – Number of outputs of the model

  • variants (string 'classif_logits' or 'regression', optional) –

    (default=’classif_logits’) Variant to use depending on how you interpret your function. Possible choices are:

    • ’classif_logits’ when using logits for classification

    • ’regression’ when using a gaussian regression model

  • device (string, optional (default='cpu')) – Target device for the returned matrix

  • function (function, optional (default=None)) – An optional function if different from model(input). If it is different from None, it will override the device parameter.

  • layer_collection (layercollection.LayerCollection, optional) – (default=None) An optional layer collection

nngeometry.metrics.FIM_MonteCarlo(model, loader, representation, variant='classif_logits', trials=1, device='cpu', function=None, layer_collection=None)

Helper that creates a matrix computing the Fisher Information Matrix using a Monte-Carlo estimate of y|x with trials samples per example

Parameters
  • model (torch.nn.Module) – The model that contains all parameters of the function

  • loader (torch.utils.data.DataLoader) – DataLoader for computing expectation over the input space

  • representation (class) – The parameter matrix representation that will be used to store the matrix

  • variants (string 'classif_logits' or 'regression', optional) –

    (default=’classif_logits’) Variant to use depending on how you interpret your function. Possible choices are:

    • ’classif_logits’ when using logits for classification

    • ’classif_logsoftmax’ when using log_softmax values for classification

    • ’segmentation_logits’ when using logits in a segmentation task

  • trials (int, optional (default=1)) – Number of trials for Monte Carlo sampling

  • device (string, optional (default='cpu')) – Target device for the returned matrix

  • function (function, optional (default=None)) – An optional function if different from model(input). If it is different from None, it will override the device parameter.

  • layer_collection (layercollection.LayerCollection, optional) – (default=None) An optional layer collection