Vector representations¶
In NNGeometry, vectors are not just a bunch of scalars, but they have a semantic meaning.
nngeometry.object.vector.PVector
objects are vectors living in the parameter space of a neural network model. An example of such vector is \(\delta \mathbf w\) in the EWC penalty \(\delta \mathbf w^\top F \delta \mathbf w\).
nngeometry.object.vector.FVector
objects are vectors living in the function space of a neural network model. An example of such vector is \(\mathbf{f}=\left(f\left(x_{1}\right),\ldots,f\left(x_{n}\right)\right)^{\top}\) where \(f\) is a neural network and \(x_1,\ldots,x_n\) are examples from a training dataset.
- class nngeometry.object.vector.FVector(vector_repr=None)¶
Bases:
object
A vector in function space
- get_flat_representation()¶
- class nngeometry.object.vector.PVector(layer_collection, vector_repr=None, dict_repr=None)¶
Bases:
object
A vector in parameter space
- Param
- add_to_model(model)¶
Updates model parameter values by adding the current PVector
Note. This is an inplace operation
- clone()¶
Returns a clone of the current object
- copy_to_model(model)¶
Updates model parameter values with the current PVector
Note. This is an inplace operation
- detach()¶
Detachs the current PVector from the computation graph
- dot(other)¶
Computes the dot product between self and other
- Parameters
other – The other PVector
- static from_model(model)¶
Creates a PVector using the current values of the given model
- static from_model_grad(model)¶
Creates a PVector using the current values of the .grad fields of parameters of the given model
- get_dict_representation()¶
- get_flat_representation()¶
Returns a Pytorch 1d tensor of the flatten vector.
Warning
The ordering in which the parameters are flattened can seem to be arbitrary. It is in fact the same ordering as specified by the
layercollection.LayerCollection
object.- Returns
a Pytorch Tensor
- norm(p=2)¶
Computes the Lp norm of the PVector
- size()¶
The size of the PVector, or equivalently the number of parameters of the layer collection
- nngeometry.object.vector.random_fvector(n_samples, n_output=1, device=None)¶
- nngeometry.object.vector.random_pvector(layer_collection, device=None)¶
Returns a random
nngeometry.object.PVector
object using the structure defined by the layer_collection parameter, with each components drawn from a normal distribution with mean 0 and standard deviation 1.The returned PVector will internally use a flat representation.
- Parameters
layer_collection – The
nngeometry.layercollection.LayerCollection
describing the structure of the random pvector
- nngeometry.object.vector.random_pvector_dict(layer_collection, device=None)¶
Returns a random
nngeometry.object.PVector
object using the structure defined by the layer_collection parameter, with each components drawn from a normal distribution with mean 0 and standard deviation 1.The returned PVector will internally use a dict representation.
- Parameters
layer_collection – The
nngeometry.layercollection.LayerCollection
describing the structure of the random pvector