In a recent speech, Andrej Karpathy has classified software into three broad categories. I have been thinking a lot about them. Here are my reflections on this topic.
Definition
| Software 1.0 | Software 2.0 | Software 3.0 |
|---|---|---|
| Needs a step-by-step algorithm, where it is told exactly what to do | Can be trained to do tasks which don’t have an exact definition | Can learn on the fly and perform tasks based on prompts |
Examples of the three categories
| Software 1.0 | Software 2.0 | Software 3.0 |
|---|---|---|
| Can draw geometric shapes | Can draw a cat | Can draw an imaginary animal based on a description |
As you can see, software 1.0 has been around since the mid-20th century, software 2.0 was impossible for a very long time. Tremendous advances in hardware, and multiple theoretical breakthroughs in machine learning made neural networks commonplace - which is what software 2.0 is. Software 3.0 is the most recent category. It is a way to provide a ‘prompt’ to a certain kind of software 2.0 which can ’learn’, ’think’, and behave in a possibly new way.
How to build these
| Software 1.0 | Software 2.0 | Software 3.0 |
|---|---|---|
| Express the algorithm in a programming language | Create a dataset, and choose a training method | Write what you want, in plain English |
The third column shows how insanely powerful software 3.0 is. It has opened up the field of software development to everyone who knows English. Which means, very soon every human being would be a software 3.0 programmer - because almost all spoken languages can be translated to English.
When to choose a particular category
As of now software 3.0 has caught fire 🔥. It is called ‘prompt engineering’, and also ‘vibe coding’ in the case of software creation. Perhaps it will be the first approach to use in software creation. We’ll go over to software 2.0 or software 1.0 only when software 3.0 is not helping us. Or, maybe not. Let’s look at some pros and cons.
Pros
| Software 1.0 | Software 2.0 | Software 3.0 |
|---|---|---|
| Easier to reason about, very reliable. Uses fewer hardware resources, hence power-efficient | Can do things which are impossible with software 1.0. Closer to the human brain | Much faster than software 2.0, and still retains the advantage over software 1.0. Also, more creative. |
Cons
| Software 1.0 | Software 2.0 | Software 3.0 |
|---|---|---|
| Cannot learn | Takes a lot of hardware resources to learn, also takes time | Less predictable |
It is clear that software 1.0 is the way to go if the task can be defined in steps, and also if we want to save money. While software 2.0 and software 3.0 keep pushing the boundary of what is possible. For example, creating new kinds of software is being done using software 3.0. However, software 3.0 need not be used to create a boilerplate project - something that has a defined structure. This is an easy trap to fall into, if you are mostly interacting with a software 3.0 chat window. Just the other day I caught myself using AI to create a basic expressjs project.
The future
Will there be a software 4.0? Maybe it will be something that can create new software 2.0. On the other hand, we might discover predictable algorithms for some software 2.0 tasks and then they’ll come down to the level of software 1.0.
Another way to think about the future is that hardware was the OG software 1.0. When software 1.0 arrived it looked vastly superior to hardware. Now we are blown away by software 3.0 a.k.a. prompt engineering. Maybe we are gradually climbing a ladder of increasing ‘intelligence’, starting from transistors and ending at prompts today.
One thing is certain - all the three categories of software are going to be important and useful. And every software x.0 uses software (x-1).0 under the hood. So it is not possible to do anything without using software 1.0 under the hood. Just like it is not possible to run anything without hardware.