Completeness contract
This article covers 18 / 18 pages of 5_Ch5-Image classification.pdf. The canonical source is 13,823,037 B, with SHA-256 45102cfc6e7add2d671924ca2f3bc2e373270a915236db364f00f9871cd9ba52. All 18 pages were cross-checked against 200 dpi renders and extracted text. A second Drive copy with the same filename has a different byte-level SHA but is pixel-identical on all 18 / 18 pages, so it is recorded only as re-save provenance.

Keep the three layers distinct.
Separate ① reconstruction of the 2025 source PDF, ② editorial and mathematical verification, and ③ research updates as of 2026-08-18. Preserve the source’s historical wording; place present-day limitations and corrections in a separate layer.

#Ledger overview

• PDF: 18 / 18 pages
• PDF SHA-256: 45102cfc6e7add2d671924ca2f3bc2e373270a915236db364f00f9871cd9ba52
• Source formula records: 52 — 19 display, 33 inline
• PDF-source content records: 135
• Source visual groups: 34
• Readable or partially readable handwritten annotation clusters: 15
• Editorial audits: 5
• Research updates as of 2026-08-18: 13
• Part V display-lesson state: 19 / 19 unreviewed — no generic visual is attached during source reconstruction; exact-ID lessons are reviewed separately in the follow-up to #38.

#Reconstruction of the source PDF

#PDF p.1 — 5.0 Reviewing 2D convolution

#The lecture cover reads ECE5992: Modern Artificial Intelligence, Il Yong Chun, 2025-03-19. Chapter 5 is titled ‘Modern AI for image classification.’

#Section 5.0 begins with a review of 2D convolution and max pooling.

The source lists constructor forms for the PyTorch convolution modules Conv1d, Conv2d, and Conv3d.

torch.nn.Conv1d(in_channels, out_channels, kernel_size, stride=1, padding=0,
                dilation=1, groups=1, bias=True, padding_mode='zeros', ...)
torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0,
                dilation=1, groups=1, bias=True, padding_mode='zeros', ...)
torch.nn.Conv3d(in_channels, out_channels, kernel_size, stride=1, padding=0,
                dilation=1, groups=1, bias=True, padding_mode='zeros', ...)

The example is a 2D convolution with 64 input channels, 64 output channels, kernel size 3, padding 1, and stride 1.

Source figure. The Step 1/Step 2 diagrams show how kernel responses across input channels combine to produce an output channel.

#Formula and numerical ledger for this page

• MAI-P5-001 · PyTorch convolution example configuration: $\mathrm{in}=64,\ \mathrm{out}=64,\ k=3,\ p=1,\ s=1$

#Diagram, table, and chart mapping for this page

#Instructor’s handwritten notes

P5-ANN-01 · PDF p.1 · Partially legible. Channel and stride notes around the convolution diagram. Some handwriting is uncertain; no definitive numbers or meanings are assigned.

#PDF p.2 — Reviewing max pooling

The source lists the PyTorch max-pooling modules MaxPool1d, MaxPool2d, and MaxPool3d.

torch.nn.MaxPool1d(kernel_size, stride=None, padding=0, dilation=1,
                   return_indices=False, ceil_mode=False)
torch.nn.MaxPool2d(kernel_size, stride=None, padding=0, dilation=1,
                   return_indices=False, ceil_mode=False)
torch.nn.MaxPool3d(kernel_size, stride=None, padding=0, dilation=1,
                   return_indices=False, ceil_mode=False)

The max-pooling example uses stride 2 and padding 1.

Source figure. Step 1 shows a padded grid and a 3x3 pooling window; Step 2 shows the window after it moves to the next position.

The slide contrasts the spatial movement of convolution and pooling using actual grid values.

#Formula and numerical ledger for this page

• MAI-P5-002 · PyTorch max-pooling example configuration: $s=2,\ p=1$

#Diagram, table, and chart mapping for this page

#PDF p.3 — 5.1 ImageNet and ILSVRC

#5.1 Trends of DL-based image classifications.

ImageNet is introduced as an image database organized according to the WordNet noun hierarchy.

The source specifies 1,000 object classes, approximately 1.2M training images, 50K validation images, and 100K test images.

ILSVRC is described as a major driver of deep learning research over its eight-year history, 2010–2017.

Source figure. The ILSVRC trend chart shows classification error falling sharply from the high levels of the early 2010s through AlexNet, ZFNet, GoogLeNet/VGGNet, ResNet, Inception-v3/WRN, and SENet.

#Formula and numerical ledger for this page

• MAI-P5-003 · ImageNet object-class count: $N_{\mathrm{class}}=1000$

• MAI-P5-004 · Dataset split sizes: $N_{\mathrm{train}}\approx1.2\mathrm{M},\ N_{\mathrm{val}}=50\mathrm{K},\ N_{\mathrm{test}}=100\mathrm{K}$

#Diagram, table, and chart mapping for this page

#Instructor’s handwritten notes

P5-ANN-02 · PDF p.3 · Partially legible. A short handwritten note to the right of the ILSVRC trend chart. Only markings concerning the human/error trend are legible; reconstructing a full sentence is deferred.

#PDF p.4 — 5.2 AlexNet: Architecture and input

#5.2 AlexNet — Overview / Details.

AlexNet is introduced as the ILSVRC 2012 winner and a large CNN for its time, with approximately 60M parameters and 650K neurons.

It classifies an input image into one of 1,000 classes. Each component of the 1,000-dimensional output is interpreted as a class probability, and the components sum to 1.

Input images are prepared as RGB 256x256 images: resize the shorter side to 256, then crop. For grayscale images, replicate the single channel to obtain three RGB channels.

During training, take a random 227x227 crop within the 256x256 image and feed it to the first layer.

AlexNet consists of five convolutional layers and three fully connected layers.

ReLU follows convolutional and fully connected layers. The first two convolutions are followed by local normalization and then pooling. The slide adds that normalization later proved to offer limited benefit.

A central point is that ReLU accelerated deep CNN training compared with saturating activations such as tanh and sigmoid.

#Formula and numerical ledger for this page

• MAI-P5-005 · AlexNet scale stated in source: $N_{\theta}\approx60\mathrm{M},\ N_{\mathrm{neuron}}\approx650{,}000$

• MAI-P5-006 · 1000-dimensional classifier output: $\mathbf{p}\in\mathbb{R}^{1000}$

• MAI-P5-008 · RGB input size: $256\times256\times3$

• MAI-P5-009 · Random crop fed to the first layer: $227\times227$

• MAI-P5-010 · AlexNet layer summary: $5\ \mathrm{conv}+3\ \mathrm{FC}$

#Diagram, table, and chart mapping for this page

#Instructor’s handwritten notes

P5-ANN-03 · PDF p.4 · Partially legible. Input/weight notes above the AlexNet architecture and notes beside the crop. They are not promoted to formal equations.

#PDF p.5 — AlexNet: ReLU·augmentation·dropout

Source figure. A CIFAR-10 plot states that ReLU (solid) reached 25% training error about six times faster than tanh (dotted).

For large positive or negative inputs, tanh saturates and its slope approaches zero, which can slow gradient descent.

The intuition is that ReLU speeds optimization because its slope in the positive region stays away from zero. The source also notes that its slope in the negative region is zero.

Mirroring and random crops mitigate overfitting. The source describes mirroring as doubling the dataset and random cropping as expanding it by a factor of 2048.

Dropout is another form of regularization.

Source figure. The bottom of the page shows examples of intermediate AlexNet feature maps.

#Formula and numerical ledger for this page

• MAI-P5-011 · Training-error level used in ReLU-vs-tanh comparison: $e_{\mathrm{train}}=25\%$

• MAI-P5-012 · Source says ReLU reached 25% error six times faster: $T_{\tanh}\approx6T_{\mathrm{ReLU}}$

• MAI-P5-013 · Mirroring doubles the dataset: $|\mathcal{D}_{\mathrm{mirror}}|=2|\mathcal{D}|$

• MAI-P5-014 · Source-reported random-cropping augmentation factor: $|\mathcal{D}_{\mathrm{crop}}|\approx2048|\mathcal{D}|$

#Diagram, table, and chart mapping for this page

#Instructor’s handwritten notes

P5-ANN-04 · PDF p.5 · Partially legible. Gradient/slope notes around the ReLU/tanh plot and notes on feature maps.

#PDF p.6 — 5.3 VGGNet: Stacking small kernels deeply

#5.3 VGGNet — Overview / Motivation / Details.

VGG16 is introduced as a 16-layer CNN and a leading ILSVRC 2014 model; the source gives an ImageNet test accuracy of 92.7%.

Instead of AlexNet’s large early kernels (11x11 and 5x5), VGG uses sequences of small 3x3 filters.

The first convolution uses a 3x3 receptive field and stride 1, compared with AlexNet’s 11x11 kernel and stride 4.

Stacking smaller convolutions provides a large receptive field while inserting more nonlinear activations.

The slide suggests that small convolutions reduce overfitting and describes 3x3 as the smallest size that captures left–right and up–down spatial information.

VGG16 has approximately 138M parameters and is described as 13 convolutional layers plus three fully connected layers.

#Formula and numerical ledger for this page

• MAI-P5-015 · VGG16 depth: $L_{\mathrm{VGG16}}=16$

• MAI-P5-016 · Source-reported VGG16 test accuracy: $\mathrm{Acc}_{\mathrm{ImageNet}}=92.7\%$

• MAI-P5-017 · VGG versus early AlexNet kernel sizes: $3\times3\quad\text{vs.}\quad 11\times11,\ 5\times5$

• MAI-P5-018 · VGG16 parameter count: $N_{\theta,\mathrm{VGG16}}\approx138\mathrm{M}$

• MAI-P5-019 · VGG16 layer decomposition: $13\ \mathrm{conv}+3\ \mathrm{FC}=16\ \mathrm{layers}$

#Instructor’s handwritten notes

P5-ANN-05 · PDF p.6 · Partially legible. Notes summarizing the small-kernel/more-layer idea in the VGG section.

#PDF p.7 — VGG16 details and the transition to GoogLeNet

VGG takes RGB 224x224 inputs; the source describes center cropping to standardize ImageNet input size.

Convolutions mainly use 3x3 kernels, with some 1x1 convolutions. A 1x1 convolution combined with ReLU increases nonlinearity without changing the receptive field.

Hidden layers use ReLU. AlexNet-style Local Response Normalization is described as increasing time and memory costs with little accuracy improvement.

Pooling reduces feature-map dimensionality and parameter burden, while channel counts grow from 64 to 128 to 256 to 512.

The fully connected layers have 4096, 4096, and 1000 channels.

Two 3x3 convolutions correspond to a 5x5 receptive field; three correspond to a 7x7 receptive field.

Three 3x3 layers provide three ReLU operations and use 27C^2 parameters, fewer than the 49C^2 parameters of one 7x7 layer.

#Section 5.4 introduces GoogLeNet as the ILSVRC 2014 winner, noting its improvement over ZFNet/AlexNet and lower error than VGGNet.

#Formula and numerical ledger for this page

• MAI-P5-020 · VGG image input: $224\times224\times3$

• MAI-P5-021 · Channel growth described in source: $64\rightarrow128\rightarrow256\rightarrow512$

• MAI-P5-022 · Three FC widths: $4096\rightarrow4096\rightarrow1000$

#Diagram, table, and chart mapping for this page

#Instructor’s handwritten notes

P5-ANN-06 · PDF p.7 · Partially legible. Notes such as “input channel size” around the VGG receptive-field and parameter comparisons.

#PDF p.8 — 5.4 GoogLeNet: 1×1 bottlenecks and Inception

GoogLeNet uses 1x1 convolutions internally and replaces the final fully connected layer with global average pooling. The source credits both ideas to Network In Network.

An Inception module applies different convolution and pooling branches to the same input in parallel, then stacks or concatenates their outputs.

NIN’s 1x1 convolution is introduced as a way to increase representational nonlinearity when combined with ReLU.

GoogLeNet uses 1x1 convolutions for dimensionality reduction, reducing computational bottlenecks and enabling greater depth and width.

Applying a 48-channel 5x5 convolution directly to a 14x14x480 input gives a stated operation count of 112.9M.

First reducing the input to 16 channels with a 1x1 convolution and then producing 48 channels with a 5x5 convolution reduces the stated count to 5.3M.

Source figure. The original Inception module places 1x1, 3x3, and 5x5 convolutions alongside 3x3 max pooling in parallel.

#Formula and numerical ledger for this page

#Diagram, table, and chart mapping for this page

#Instructor’s handwritten notes

P5-ANN-07 · PDF p.8 · Partially legible. Big-O and bottleneck notes around the Inception/1x1 calculations.

#PDF p.9 — Inception and global average pooling

Each Inception branch extracts different features. Their outputs are concatenated and passed to the next module.

Because the original Inception module is computationally expensive, 1x1 dimensionality reduction is inserted before costly branches.

#Global average pooling.

An AlexNet-style final fully connected layer is illustrated as requiring 7x7x1024x1024=51.3M connections.

GoogLeNet’s global average pooling averages each 7x7 feature map down to 1x1, creating a channel descriptor without separate fully connected connections.

#Formula and numerical ledger for this page

• MAI-P5-031 · Global average pooling geometry: $7\times7\times1024\rightarrow1\times1\times1024$

#Diagram, table, and chart mapping for this page

#Instructor’s handwritten notes

P5-ANN-08 · PDF p.9 · Partially legible. Notes on 7x7/1x1 dimensions and pooling around the fully connected/GAP diagrams.

#PDF p.10 — The complete GoogLeNet architecture, auxiliary classifiers, and the transition to ResNet

The GAP example is stated to require zero additional weights.

The source reports an approximately 0.6% improvement in top-1 accuracy from replacing fully connected layers with GAP, connecting this to NIN’s idea for reducing overfitting.

#Overall architecture.

The complete GoogLeNet architecture has 22 layers and achieves its depth by linking multiple Inception modules.

It is described as much deeper than AlexNet, ZFNet, and VGGNet, yet shallower than the later ResNet.

#Auxiliary classifiers for training.

The intermediate softmax branches are auxiliary classifiers used only during training. Their losses enter the total loss with weight 0.3.

The authors’ claim is presented that these branches mitigate vanishing gradients and also provide regularization.

#5.5 ResNet — Overview.

ResNet is introduced as the ILSVRC 2015 winner, with residual learning enabling much deeper networks at similar parameter counts.

The source introduces the degradation problem: accuracy saturates and then worsens as depth increases. This is not attributed solely to overfitting, because the deeper plain model’s training error itself rises.

#Formula and numerical ledger for this page

• MAI-P5-032 · Source example: GAP adds no FC connections: $N_{\mathrm{weight,GAP}}=0$

• MAI-P5-033 · Source-reported top-1 gain after replacing FC with GAP: $\Delta\mathrm{Acc}_{\mathrm{top1}}\approx+0.6\%$

• MAI-P5-034 · Source says 22 layers in total: $L_{\mathrm{GoogLeNet}}=22$

#Diagram, table, and chart mapping for this page

#Instructor’s handwritten notes

P5-ANN-09 · PDF p.10 · Partially legible. Reduction/parameter notes around GAP and the GoogLeNet architecture, and skip-connection notes at the start of ResNet.

#PDF p.11 — 5.5 ResNet: Degradation and residual blocks

A passage from the ResNet paper on degradation emphasizes that increasing the depth of an already sufficiently deep model can raise training error.

The proposed solution is residual learning, centered on residual or skip connections.

Source figure. A residual block combines the nonlinear transform F(x) with the identity x through element-wise addition.

Source figure. A ResNet-18 architecture example is presented as blocks A–E.

Source figure. ImageNet Figure 1 compares training and validation errors for plain 18/34-layer networks and 18/34-layer ResNets.

The plain 34-layer model has higher validation error than the plain 18-layer model, as well as higher training error throughout, demonstrating degradation.

Source footnote. Preserve the note that vanishing or exploding gradients can impede convergence from the outset, and that normalized initialization, intermediate normalization, and ReLU can mitigate them.

#Formula and numerical ledger for this page

#Diagram, table, and chart mapping for this page

#Instructor’s handwritten notes

P5-ANN-10 · PDF p.11 · Partially legible. Skip-connection, identity, and gradient-direction notes around the residual block.

#PDF p.12 — ResNet details, BN, and 5.6 WRN

With ResNet, the 34-layer model outperforms the 18-layer model and achieves lower training error, illustrating the resolution of plain-network degradation.

#Further details.

ResNet is described as using batch normalization after every convolutional layer.

BN normalizes hidden activations using the current batch’s mean and variance and is generally applied immediately before the nonlinear function.

The source says the explanation for BN’s effectiveness remains unsettled. It presents the original internal-covariate-shift account, the hypothesis of reduced interdependence between layers, and the optimization-landscape-smoothing account alongside one another.

The source states that ResNet does not use dropout.

#5.6 Wide residual network (WRN).

Widening ResNet can produce shallower networks with equal or better accuracy, reducing the layer count and training time.

From circuit complexity, the source notes that shallow circuits may require exponentially more components than deeper circuits.

The diminishing-feature-reuse problem is described: in thin, deep ResNets, gradients may bypass residual weights, leaving only some blocks to learn useful representations.

WRN widens networks by increasing feature planes. It uses deepening factor l and widening factor k: k=1 gives the ResNet width, while k>1 makes it k times wider.

Source footnote. Preserve the note that BN is sometimes applied after the nonlinear function.

#Formula and numerical ledger for this page

• MAI-P5-037 · WRN widening-factor definition: $k=1\Rightarrow\text{ResNet width},\qquad k>1\Rightarrow k\times\text{wider}$

#Instructor’s handwritten notes

P5-ANN-11 · PDF p.12 · Partially legible. Short notes on researchers/optimization around the BN discussion, and notes on widening WRN.

#PDF p.13 — Comparing WRNs and 5.7 DenseNet

Source figure. The source compares basic, bottleneck, basic-wide, and wide-dropout residual-block variants.

Source table. The ILSVRC 2012 single-crop validation table compares top-1/top-5 error, parameter counts, and batch time for ResNet-50/101/152, WRN-50-2-bottleneck, and pre-ResNet-200.

WRN-50-2-bottleneck is described as faster than ResNet-152 with one-third as many layers, while achieving results close to pre-ResNet-200.

Pre-Activation ResNet uses BN–ReLU–Conv, whereas the original ResNet uses Conv–BN–ReLU. The source says pre-activation is generally better, but offers no clear improvement on ImageNet at around 100 layers.

#5.7 DenseNet — Overview.

DenseNet received the CVPR 2017 Best Paper Award. Each layer takes all preceding feature maps as input and passes its own feature maps to all subsequent layers.

A standard network has L connections, whereas an L-layer DenseNet has L(L+1)/2 connections and uses concatenation instead of summation.

The l-th layer is described as receiving l inputs comprising all preceding feature maps.

#Formula and numerical ledger for this page

• MAI-P5-038 · Standard network connection count: $N_{\mathrm{conn,standard}}=L$

• MAI-P5-040 · The l-th dense layer receives l inputs: $N_{\mathrm{input},l}=l$

#Diagram, table, and chart mapping for this page

#Instructor’s handwritten notes

P5-ANN-12 · PDF p.13 · Partially legible. Pre-activation notes in the WRN table and recycling/connection notes on the DenseNet diagram.

#PDF p.14 — Dense block: sequential/addition/concatenation

In the dense-block discussion, a standard feed-forward network connects the l-th output to the (l+1)-th input.

The standard transition is defined as x_l=H_l(x_{l-1}), with H_l representing a composite function such as convolution, ReLU, pooling, or BN.

ResNet uses identity mapping and element-wise addition to obtain x_l=H_l(x_{l-1})+x_{l-1}.

DenseNet concatenates all preceding features and supplies them as x_l=H_l([x_0,...,x_{l-1}]).

DenseNet defines H_l as the sequence BN → ReLU → 3x3 convolution.

Transition layers between dense blocks use a 1x1 convolution followed by 2x2 average pooling.

Feature maps within a dense block have the same spatial size, allowing concatenation. The final dense block is followed by GAP and a softmax classifier.

#Formula and numerical ledger for this page

• MAI-P5-044 · DenseNet transition-layer structure: $1\times1\ \mathrm{conv}\rightarrow2\times2\ \mathrm{avgpool}$

#Diagram, table, and chart mapping for this page

#Instructor’s handwritten notes

P5-ANN-13 · PDF p.14 · Partially legible. Input/output/concatenation notes around the standard-network, ResNet, and DenseNet equations.

#PDF p.15 — DenseNet growth rate and advantages

Reusing preceding feature maps allows the network to be thinner and more compact.

The growth rate k is defined as the number of channels added by each layer.

Source figure. The forward-propagation concatenation diagram shows how each layer receives accumulated features from earlier layers.

#Advantages of DenseNet.

Strong gradient flow: error signals from the final classification layer reach earlier layers directly, producing an implicit deep-supervision effect.

Parameter/computational efficiency: the source contrasts ResNet layer parameters proportional to CxC with DenseNet parameters proportional to lkk, where k<<C.

Source figure. The diagram compares parameter scaling under ResNet and DenseNet connectivity.

Source figure. A plot compares top-1 error against parameter counts for DenseNet and ResNet families.

#Formula and numerical ledger for this page

• MAI-P5-045 · Growth-rate definition: $k=\text{additional channels contributed by each dense layer}$

• MAI-P5-046 · Source connectivity parameter scaling: $N_{\theta,\mathrm{ResNet},l}\propto C^2$

#Diagram, table, and chart mapping for this page

#Instructor’s handwritten notes

P5-ANN-14 · PDF p.15 · Partially legible. Emphasis marks on growth rate k, gradient flow, and the k<<C parameter comparison.

#PDF p.16 — DenseNet feature reuse and 5.8 SE Networks

Each DenseNet layer receives all preceding features, enabling more diverse and richer patterns.

A standard convolutional classifier mainly uses the most complex final features, whereas a DenseNet classifier can use features at multiple levels of complexity.

Following the authors’ argument, the source connects this to potentially smoother decision boundaries and better performance when data are scarce.

#5.8 Squeeze-and-Excitation Networks.

The SE architecture is introduced as the ILSVRC 2017 winner. It adds content-aware, adaptive channel weighting in place of the equal channel weighting of a conventional CNN.

The SE block is introduced as a lightweight gating mechanism that strengthens channel interdependencies with little additional cost.

Squeeze reduces each feature map to a single number, producing a C-dimensional vector. A two-layer neural network outputs another vector of the same size, whose C values rescale the original feature maps channel by channel.

#Formula and numerical ledger for this page

• MAI-P5-048 · Squeezed channel descriptor size: $\mathbf{z}\in\mathbb{R}^{C}$

#Diagram, table, and chart mapping for this page

#PDF p.17 — SE squeeze·excitation·scale

SE weights rescale the original feature maps according to each channel’s importance.

Source figure. The SE diagram follows transform X→U, squeeze F_sq, excitation F_ex, and scale F_scale, alongside the HxWxC tensor geometry.

The squeeze equation computes z_c as a spatial average.

The excitation equation generates channel gates s through a two-layer W1/W2 bottleneck, activation delta, and sigmoid.

The scale equation recalibrates each channel by multiplying s_c and u_c.

SE blocks are described as easy to integrate into popular convolutional modules such as ResNet and Inception.

Source figure. The SE-ResNet module inserts global pooling → FC(C/r) → ReLU → FC(C) → sigmoid → scale into the residual branch; r is the reduction ratio.

Source figure. Training and validation top-1 error curves are compared for SE-ResNet-50 and ResNet-50.

Top-1 scoring checks whether the class with the highest predicted probability matches the target label.

The statement that SE blocks add depth in a computationally efficient way continues onto the first line of p.18.

#Formula and numerical ledger for this page

• MAI-P5-052 · SE bottleneck widths with reduction ratio r: $1\times1\times C\rightarrow1\times1\times\frac{C}{r}\rightarrow1\times1\times C$

#Diagram, table, and chart mapping for this page

#Instructor’s handwritten notes

P5-ANN-15 · PDF p.17 · Partially legible. Channel-aggregation notes to the left of the squeeze/excitation diagram and a “weight matrix” note on SE-ResNet.

#PDF p.18 — References

#References.

• [1] Simonyan & Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition, ICLR 2015.

• [2] Szegedy et al., Going Deeper with Convolutions, CVPR 2015.

• [3] LeCun et al., Gradient-Based Learning Applied to Document Recognition, Proc. IEEE 1998.

• [4] Lin, Chen & Yan, Network in Network, ICLR 2014.

• [5] He et al., Deep Residual Learning for Image Recognition, CVPR 2016.

• [6] Ioffe & Szegedy, Batch Normalization, ICML 2015.

• [7] Santurkar et al., How Does Batch Normalization Help Optimization?, NeurIPS 2018.

• [8] Huang et al., Densely Connected Convolutional Networks, CVPR 2017.

• [9] Hu, Shen & Sun, Squeeze-and-Excitation Networks, CVPR 2018.

#Editorial and mathematical verification

#P5-E001

Read the VGG slide’s statement that “3x3 is the optimal/smallest possible size” as lecture intuition about the smallest odd kernel that captures spatial context in both directions. A 1x1 convolution also exists; the statement does not establish that 3x3 is universally optimal for every purpose.

#P5-E002

The AlexNet slide’s intuition that “most neurons usually end up positive” is not a general theorem. Keep that sentence separate from the historical explanation that ReLU’s nonsaturating positive branch helped optimization.

#P5-E003

As in the source, retain multiple hypotheses about BN’s effectiveness. Do not identify internal covariate shift as the sole established cause; note that later work discusses optimization landscapes and reparameterization.

#P5-E004

DenseNet’s claim of “higher computational and memory efficiency” concerns parameter reuse and compactness. Dense concatenation can increase activation-memory traffic, so this claim should not be equated with hardware efficiency in modern implementations.

#P5-E005

Preserve the source wording that the SE architecture won ILSVRC 2017, while reading the end of the challenge and SENet’s competition results in their historical context.

#Research updates as of 2026-08-18

This section does not count toward PDF page coverage. Rather than replacing the lecture’s AlexNet-to-SE history, it connects primary sources to additional axes relevant to image classification as of 2026-08-18: architecture, training, robustness, and transfer.

#P5-R001

ConvNeXt progressively modernized ResNet using a Transformer-era recipe, showing that a pure ConvNet can remain competitive with ViT/Swin-family backbones.

• Evidence: peer-reviewed / official preprint

#P5-R002

ConvNeXt V2 jointly designed architecture and self-supervised pretraining by combining a fully convolutional masked autoencoder with Global Response Normalization.

• Evidence: peer-reviewed / official preprint

#P5-R003

Vision Transformer established a pure Transformer route that treats image patches as a token sequence and transfers large-scale pretraining to image recognition.

• Evidence: peer-reviewed / official preprint

#P5-R004

Swin Transformer used shifted local windows and hierarchical feature maps to serve as a general-purpose backbone for classification and dense prediction. Swin V2 strengthened stability at larger scales and resolutions.

• Evidence: peer-reviewed / official preprint

#P5-R005

ResNet-RS research separated architectural differences from training and scaling recipes, showing how a strong recipe can substantially improve the classic ResNet baseline itself.

• Evidence: peer-reviewed / official preprint

#P5-R006

ImageNet-1K top-1 accuracy alone cannot adequately characterize generalization. ImageNet-C/P separately measure robustness to common corruptions and perturbations.

• Evidence: peer-reviewed / official preprint

#P5-R007

ImageNet-A used adversarial filtration to assemble naturally occurring hard images on which models commonly fail, revealing the gap between in-distribution accuracy and robustness.

• Evidence: peer-reviewed / official preprint

#P5-R008

Modern neural classifiers can have poor confidence calibration despite high accuracy. Temperature scaling remains a simple post-hoc baseline.

• Evidence: peer-reviewed / official preprint

#P5-R009

CLIP used natural-language supervision to construct zero-shot classifiers, making distribution shifts beyond ImageNet and transfer evaluation central questions for image classification.

• Evidence: peer-reviewed / official preprint

#P5-R010

In the era of foundation vision encoders, frozen or transferable representations and downstream linear-probe or fine-tuning performance have become more important than a fixed 1,000-way classifier.

• Evidence: official preprint (DINOv2)

#P5-R011

DINOv3 used large-scale self-supervised training and Gram anchoring in 2025 to counter dense-feature degradation and pursue general-purpose encoder performance across vision tasks.

• Evidence: technical report / official preprint

#P5-R012

SigLIP 2 combined captioning, self-distillation, masked prediction, and online curation to improve multilingual zero-shot classification, retrieval, transfer, localization, and dense features together.

• Evidence: technical report / official preprint

#P5-R013

The historical AlexNet → VGG → Inception → ResNet → DenseNet → SE progression remains useful for understanding architecture. Practical comparisons in 2026 must also consider training recipes, pretraining data, calibration, distribution shift, and transfer costs.

• Evidence: synthesis grounded in cited primary sources

#A compact map, reread from a 2026 perspective

AlexNet made large-scale CNN training practical through ReLU, data augmentation, and dropout. VGG emphasized the simplicity of deep stacks of small 3×3 kernels; GoogLeNet, 1×1 bottlenecks, multi-branch Inception, and GAP; ResNet/WRN, identity paths and the depth/width trade-off; DenseNet, feature reuse through concatenation; and SE, adaptive channel-wise recalibration.

In 2026, evaluation also includes ViT/Swin attention backbones, modernized ConvNets such as ConvNeXt, training/scaling recipes, self-supervised or multimodal foundation encoders, and calibration, distribution shift, robustness, and transfer. Asking only which architecture has the highest ImageNet top-1 accuracy is therefore no longer enough.