What is a neural network? Strip away the brain metaphor and it is a stack of layers doing arithmetic, where each layer passes numbers to the next, and every connection has a weight that gets adjusted during training. That is genuinely the whole idea. The name is a loose analogy to neurons, not a description of how brains work.
The Basic Unit
A single artificial neuron does two things:
- Takes its inputs, multiplies each by a weight, adds them up, plus a bias
- Passes the result through an activation function — a nonlinear squashing operation
That second step is what makes the whole thing useful. Without nonlinearity, stacking layers would be pointless: a chain of linear operations collapses into a single linear operation, no matter how many layers you add. The activation function is what lets a network represent genuinely complicated relationships.
How It Learns
Training is simpler conceptually than most explanations make it:
- Feed in an example, get an output
- Compare that output to the correct answer, measure the error with a loss function
- Work out which weights contributed to the error, using backpropagation
- Nudge every weight slightly in the direction that reduces error — gradient descent
- Repeat, millions of times
Nothing is programmed with rules. The network has a fixed structure and a huge number of adjustable numbers, and training finds values for those numbers that happen to work on the examples. “Learning” is a search through parameter space, not comprehension.
The Main Architectures
| Type | Good at | Used for |
|---|---|---|
| Feedforward / MLP | Simple mapping, tabular data | Classification, basic prediction |
| Convolutional (CNN) | Spatial patterns, images | Image recognition, medical scans |
| Recurrent (RNN / LSTM) | Sequences, older generation | Largely superseded by transformers |
| Transformer | Context across long sequences | Modern language models, translation |
Transformers are why language AI changed so abruptly around 2018-2022. Their attention mechanism lets every position in a sequence look at every other position directly, which handles long-range context far better than reading word by word in order.
What It Is Not Doing
That distinction is the part most explanations of what is a neural network leave out.
This is where honest framing matters, because the vocabulary invites confusion.
- It does not “understand.” A network learns statistical relationships between inputs and outputs. Strong performance on a task is not evidence of comprehension, and the difference shows up when you move slightly outside the training distribution.
- It does not reason step by step by default. Language models can produce reasoning-like text because reasoning-like text appears in training data. Whether that constitutes reasoning is genuinely contested.
- It does not know when it is wrong. Networks produce confident, fluent output whether or not the content is correct. Hallucination is a structural feature, not a bug to be patched once.
- It is not energy-cheap. Training large models consumes enormous amounts of electricity.
Why Size Changed Everything
The core techniques date to the 1980s and earlier. What changed was scale: more data, more parameters, and vastly more compute — particularly GPUs, which happen to be extremely good at the matrix multiplication these networks are built from.
Researchers observed that performance improves predictably as you scale up, following fairly smooth power laws. That predictability is what justified the huge investment in ever-larger models, and it is why the field’s progress was driven as much by hardware and data availability as by new mathematics.
Where It Genuinely Excels
- Pattern recognition in images — medical imaging, defect detection, satellite analysis
- Translation and speech transcription
- Protein structure prediction — this one has been transformative for biology
- Compression, denoising, and signal processing
- Any task with lots of examples and fuzzy, hard-to-write-down rules
Keep Exploring
- How does machine learning work? — the wider field this sits inside
- What is an algorithm? — the underlying concept
- How does memory work? — what the analogy is reaching for
1 thought on “What Is a Neural Network? The Idea Behind Modern AI”