What is a
Neural Network?

You've heard the phrase "neural network" thrown around constantly. It powers self-driving cars, generates images from text, translates languages in real-time, and beats world champions at chess and Go. But what actually is it?

Let's cut through the hype and build up an honest, intuitive understanding from scratch.

The Basic Idea: Learning from Examples

A neural network is a function approximator. Its job is to learn a mapping from inputs to outputs by seeing many examples.

Imagine you want to teach someone to tell cats from dogs. You don't write a rulebook. You show them thousands of pictures and say "this is a cat," "this is a dog" — and they figure out the pattern on their own. That's exactly what a neural network does.

A Network of Neurons

The name "neural network" is inspired by the brain, though modern neural networks are a highly simplified abstraction. A neuron in a neural network does something very simple:

1. Take several numbers as input
2. Multiply each by a weight (how important is this input?)
3. Add a bias (a baseline shift)
4. Pass the result through an activation function
5. Output a single number

Mathematically: output = activation(w₁x₁ + w₂x₂ + ... + b)

These neurons are organized into layers: an input layer, one or more hidden layers, and an output layer.

The "Deep" in Deep Learning

"Deep" refers to having many hidden layers. Each layer learns to represent the data at a different level of abstraction.

For image recognition, a deep network might learn:

• Layer 1: Detect edges and color gradients
• Layer 2: Combine edges into shapes and textures
• Layer 3: Combine textures into object parts (ears, wheels, eyes)
• Layer 4: Combine parts into complete objects

This hierarchical representation is what gives deep networks their extraordinary power. They build up complexity from simplicity, layer by layer.

How Does it Learn? (A Peek at Training)

The magic is in the weights. A network starts with random weights — it makes terrible predictions. But through training, it adjusts those weights to make better and better predictions.

Here's the loop:

1. Forward pass: Feed in input, get a prediction
2. Compute loss: Measure how wrong the prediction was
3. Backpropagation: Figure out which weights caused the error
4. Update weights: Nudge each weight to reduce the error
5. Repeat for thousands/millions of examples

We'll cover this in much more detail in the next few lessons. For now, the key insight: training is just iterative error correction.

Your First Neural Network: 10 Lines of PyTorch

Here's what a simple neural network looks like in code:

import torch
import torch.nn as nn

# Define a 3-layer neural network
model = nn.Sequential(
    nn.Linear(784, 256),   # Input layer: 784 → 256 neurons
    nn.ReLU(),               # Activation function
    nn.Linear(256, 128),   # Hidden layer: 256 → 128 neurons
    nn.ReLU(),               # Activation function
    nn.Linear(128, 10)    # Output layer: 128 → 10 classes
)

# Count parameters
total_params = sum(p.numel() for p in model.parameters())
print(f"Model has {total_params:,} learnable parameters")
# Output: Model has 235,146 learnable parameters

That's it. In 7 lines, you've defined a network with 235,000 learnable parameters that can classify handwritten digits. We'll train it from scratch in Lesson 12.

What's Next?

In the next lesson, we'll zoom in on the single neuron — the perceptron — and build one from absolute scratch in Python, no libraries. Understanding the neuron deeply will make everything else click.