There Will Be An Open-Source GPT-4 Model And It Will Revolutionize Small Business

Hot take: By the end of the year, we will see multiple open-source models that are as good as GPT-4 or better.

Recently, Databricks coined Mosaic’s law (great naming btw.). It postulates that soft- and hardware improvements will reduce the cost of running ML models by 75% yearly.

A model that costs $100M to train in one year, will cost around $1.5M after three years. In other words, something that was only possible for nation-states and the world’s largest corporations will quickly come within reach of a cliche startup founder with a hoodie and a laptop.

This will create an unimaginable amount of opportunities to build niche AI companies.

It will usher in the age of the AI SMB. Tiny teams of a handful of people who will dominate a small local niche.

Let’s linger on that for a second and explore the interaction between stronger models and dropping prices.

When building chat systems three years ago, people would use what was called a retriever-reader architecture. You can think of that as an early form of RAG. The retriever component would retrieve text relevant to the user’s question. The reader would then make sense of it and extract the final answer from a tiny context window of around 1000 tokens.

Compared to today, these systems were bad! Really bad.

At best, the systems would produce the correct answer two out of three times. To improve performance people would simplify the task by breaking it up into several parts and solving them one by one.

Let’s look at an example of a customer service bot for realtors.

Any user query would first pass through an intent classifier. This is a separate model that groups the queries into classes. Examples of these classes could be complaints, questions, and appointment scheduling.

Each class would then be handled by separate models. This would get very complicated. Fast.

If a given problem requires eight different classes, and each class has its own retriever and reader models. In total, this quickly amounted to over a dozen models. Each model required the curation or labeling of specific datasets. Different training algorithms and evaluation pipelines had to be implemented. Then there were multiple deployment pipelines, monitoring, the list goes on.

This meant that something as simple as a specialized customer service chatbot would require a year to develop and a team of five engineers.

Today, this can be done in weeks not months.

The long context windows and amazing few-shot abilities of modern LLMs make it possible to replace most of this work with prompt engineering and a few API calls.

In some cases, this means a cost reduction of 1000x while simultaneously reducing the time required by 80%.

We are at a point in time where this will allow teams with little to no funding to build amazing products. This opens up opportunities that historically could only be tackled by venture-funded startups. Now, the deflationary properties of technology allow small teams of bootstrappers to enter the game.

Today, GPT-3.5 are not yet good enough. However, GPT-4 is and LLaMA 3 is very close to catching up.

If Mosaic’s Law holds for another 12 months, we will close out the year with multiple GPT-4-level models that are open-source. By that time the window of opportunity will open to solve high-value use cases that were previously uneconomical to solve with AI.

And 2025 is going to be pretty amazing!

What an amazing time to be alive.

Lots of love and see you next time!

P.S. If you found this useful, please, share it with a friend or subscribe here ⭕️ if you haven’t already.