University of Michigan - Ann Arbor

To examine how network layers transform intermediate representations on CIFAR-10, capture the output of each $$3 \times 3$$ convolution immediately after batch normalization and before subsequent ReLU activation or element-wise addition. For each layer, summarize the strength of these pre-activation responses using their standard deviation, enabling comparisons across layers and architectures. In a residual network, the measurement describes the learned residual branch $$\mathcal{F}(\mathbf{x})$$ rather than the combined block output $$\mathcal{F}(\mathbf{x}) + \mathbf{x}$$.

Learn Before

Related