Convolutional Neural Networks

If you flatten a 256x256 image and feed it into a fully connected neural network, the first hidden layer alone would require millions of parameters. Convolutional Neural Networks solve this by exploiting the spatial structure of images — the fact that nearby pixels are related, and the same feature (an edge, a curve, a corner) can appear anywhere in an image.

The Convolution Operation

A convolution layer slides a small filter (also called a kernel) across the input image. At each position, it computes the dot product between the filter weights and the corresponding patch of pixels, producing a single output value. Doing this across the entire image produces a feature map.

import numpy as np

def convolve2d(image, kernel):
    kh, kw = kernel.shape
    ih, iw = image.shape
    out_h = ih - kh + 1
    out_w = iw - kw + 1
    output = np.zeros((out_h, out_w))
    for i in range(out_h):
        for j in range(out_w):
            output[i, j] = np.sum(image[i:i+kh, j:j+kw] * kernel)
    return output

A 3x3 edge-detection filter applied to a 28x28 image produces a 26x26 feature map. The filter has only 9 parameters, but it is applied at every spatial location — this weight sharing is what makes CNNs so parameter-efficient compared to fully connected layers.

Multiple Filters, Multiple Feature Maps

In practice, a convolutional layer uses many filters in parallel. Each filter learns to detect a different pattern. The first layer might learn to detect horizontal edges, vertical edges, and diagonal gradients. Deeper layers combine these into more complex features: curves become shapes, shapes become object parts.

A layer with 32 filters of size 3x3 applied to a single-channel input produces 32 feature maps — a 3D output volume of shape (height, width, 32). Each filter produces its own map capturing a different aspect of the input.

Padding and Stride

Without padding, each convolution reduces the spatial dimensions. Padding adds a border of zeros around the input so the output has the same spatial size as the input ("same" padding). "Valid" padding means no padding, letting dimensions shrink naturally.

Stride controls how many pixels the filter moves at each step. A stride of 1 moves one pixel at a time. A stride of 2 skips every other position, halving the output size and acting like a form of downsampling.

Pooling Layers

Pooling layers reduce spatial dimensions by summarizing regions of the feature map. Max pooling takes the maximum value in each region. Average pooling takes the mean. A 2x2 max pool with stride 2 halves both height and width, discarding fine-grained spatial detail while retaining the presence of features.

def max_pool2d(feature_map, pool_size=2, stride=2):
    h, w = feature_map.shape
    out_h = (h - pool_size) // stride + 1
    out_w = (w - pool_size) // stride + 1
    output = np.zeros((out_h, out_w))
    for i in range(out_h):
        for j in range(out_w):
            region = feature_map[i*stride:i*stride+pool_size,
                                  j*stride:j*stride+pool_size]
            output[i, j] = np.max(region)
    return output

Pooling introduces a degree of translation invariance — a feature detected slightly to the left or right still activates the same pooled output. This is desirable for classification where you want to recognize an object regardless of its exact position.

A Typical CNN Architecture

A standard CNN stacks convolutional and pooling layers to progressively extract higher-level features at lower spatial resolution, then flattens the result and passes it through fully connected layers to produce class predictions:

Input (32x32x3)
  -> Conv(32 filters, 3x3) + ReLU  -> (30x30x32)
  -> MaxPool(2x2)                   -> (15x15x32)
  -> Conv(64 filters, 3x3) + ReLU  -> (13x13x64)
  -> MaxPool(2x2)                   -> (6x6x64)
  -> Flatten                        -> (2304,)
  -> Dense(128) + ReLU
  -> Dense(10) + Softmax            -> class probabilities

Why CNNs Work So Well for Images

Three structural properties make CNNs well-suited to visual data. Local connectivity means each neuron only looks at a small spatial region, matching the local structure of natural images. Weight sharing means the same filter is applied everywhere, drastically reducing parameters and encoding the prior that a feature detector should work the same anywhere in the image. Hierarchical composition means shallow features combine into deep features, mirroring how biological visual systems process scenes from edges to objects.

Architectures like ResNet, VGG, and EfficientNet are all CNNs at their core, extended with techniques like residual connections, batch normalization, and depthwise separable convolutions to push performance further. But the fundamental mechanics — convolution, activation, pooling — remain the same as described here.