BatchNorm2d — PyTorch 1.10.1 documentation
https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm2d.htmlBecause the Batch Normalization is done over the C dimension, computing statistics on (N, H, W) slices, it’s common terminology to call this Spatial Batch Normalization. Parameters num_features – C C from an expected input of size (N, C, H, W) (N,C,H,W) eps – a value added to the denominator for numerical stability. Default: 1e-5
LayerNorm — PyTorch 1.10.1 documentation
pytorch.org › docs › stableThe mean and standard-deviation are calculated over the last D dimensions, where D is the dimension of normalized_shape.For example, if normalized_shape is (3, 5) (a 2-dimensional shape), the mean and standard-deviation are computed over the last 2 dimensions of the input (i.e. input.mean((-2,-1))).
torch.norm — PyTorch 1.10.1 documentation
pytorch.org › docs › stabletorch.norm. torch.norm(input, p='fro', dim=None, keepdim=False, out=None, dtype=None) [source] Returns the matrix norm or vector norm of a given tensor. Warning. torch.norm is deprecated and may be removed in a future PyTorch release. Its documentation and behavior may be incorrect, and it is no longer actively maintained.
BatchNorm2d — PyTorch 1.10.1 documentation
pytorch.org › docs › stableBatchNorm2d. Applies Batch Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension) as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift . \beta β are learnable parameter vectors of size C (where C is the input size). By default, the elements of.
torch.nn.functional.normalize — PyTorch 1.10.1 documentation
https://pytorch.org/docs/stable/generated/torch.nn.functional.normalize.htmltorch.nn.functional.normalize — PyTorch 1.10.1 documentation torch.nn.functional.normalize torch.nn.functional.normalize(input, p=2.0, dim=1, eps=1e-12, out=None) [source] Performs L_p Lp normalization of inputs over specified dimension. For a tensor input of sizes (n_0, ..., n_ {dim}, ..., n_k) (n0 ,...,ndim ,...,nk ), each n_ {dim} ndim