Software 2.0
Andrej Karpathy · 2017
"Neural networks are a new software paradigm where code is learned from data, not written by hand."
In traditional software ('Software 1.0'), a human engineer writes explicit step-by-step instructions telling the computer exactly what to do in every situation. Karpathy's argument: a large and growing share of valuable software is now better built a completely different way — you define a goal (a loss function measuring how wrong the model currently is), provide labeled examples, and let an optimization algorithm (like gradient descent) search through billions of possible internal settings until it finds ones that achieve the goal. The 'program' that results isn't human-readable code — it's a giant set of numbers (weights) that a human didn't write line by line.
Read more about the topic
The Bitter Lesson
"General methods that leverage computation ultimately beat human-knowledge-engineered approaches in AI."