nerva_torch.weight_initializers

Weight and bias initialization helpers for linear layers.

Functions

set_bias_normal(b[, mean, std])	Normal (Gaussian) initialization with given mean and std.
set_bias_uniform(b_[, a, b])	Uniform initialization within [a, b).
set_bias_zero(b)	Set all bias values to zero.
set_layer_weights(layer, text)	Initialize a layer's parameters according to a named scheme.
set_weights_he_normal(W)	He / Kaiming normal initialization (for ReLU).
set_weights_he_uniform(W)	He / Kaiming uniform initialization (for ReLU, less common).
set_weights_normal(W[, mean, std])	Normal (Gaussian) initialization with given mean and std.
set_weights_uniform(W[, a, b])	Uniform initialization within [a, b).
set_weights_xavier_normal(W)	Xavier / Glorot normal initialization (for tanh/sigmoid).
set_weights_xavier_uniform(W)	Xavier / Glorot uniform initialization (for tanh/sigmoid).
set_weights_zero(W)	Initialize weights to zero.

nerva_torch.weight_initializers.set_bias_zero(b: torch.Tensor)

Set all bias values to zero.

nerva_torch.weight_initializers.set_bias_uniform(b_: torch.Tensor, a: float = 0.0, b: float = 1.0)

Uniform initialization within [a, b).

nerva_torch.weight_initializers.set_bias_normal(b: torch.Tensor, mean: float = 0.0, std: float = 1.0)

Normal (Gaussian) initialization with given mean and std.

nerva_torch.weight_initializers.set_weights_uniform(W: torch.Tensor, a: float = 0.0, b: float = 1.0)

Uniform initialization within [a, b).

nerva_torch.weight_initializers.set_weights_normal(W: torch.Tensor, mean: float = 0.0, std: float = 1.0)

Normal (Gaussian) initialization with given mean and std.

nerva_torch.weight_initializers.set_weights_xavier_uniform(W: torch.Tensor)

Xavier / Glorot uniform initialization (for tanh/sigmoid).

K = fan-out (output size) D = fan-in (input size)

nerva_torch.weight_initializers.set_weights_xavier_normal(W: torch.Tensor)

Xavier / Glorot normal initialization (for tanh/sigmoid).

K = fan-out (output size) D = fan-in (input size)

nerva_torch.weight_initializers.set_weights_he_normal(W: torch.Tensor)

He / Kaiming normal initialization (for ReLU).

K = fan-out (output size) D = fan-in (input size)

nerva_torch.weight_initializers.set_weights_he_uniform(W: torch.Tensor)

He / Kaiming uniform initialization (for ReLU, less common).

K = fan-out (output size) D = fan-in (input size)

nerva_torch.weight_initializers.set_weights_zero(W: torch.Tensor)

Initialize weights to zero.

Note: Initializing all weights to zero is generally not recommended because it causes all neurons to learn the same features during training, leading to symmetry that prevents effective learning and updates (the “symmetry breaking” problem). This initializer can be useful for biases or special cases but should be avoided for weights.

nerva_torch.weight_initializers.set_layer_weights(layer, text: str)

Initialize a layer’s parameters according to a named scheme.