7 Interesting Predictions For AI In 2023 From The State Of AI Report

A new year is ahead and it will be packed with exciting progress in machine learning. I compiled a list with the most interesting predictions for 2023 from the State Of AI Report.

Let's jump right in!

1) A SOTA Language Model is Trained on 10x More Data Than Chinchilla

​
Language models like Lambda and GPT3 are significantly undertrained. DeepMind proposed Chinchilla, a model that has similar performance to GPT3 with less than half the size (70B vs. 175B). Hence in 2023, significant performance gains will likely come from cleaner/larger datasets.

​

2) Generative Audio Tools Emerge and Will Attract 100K Developers

​
Audio generation has approached human levels. If enough data of your voice is available, the generated speech can even sound amazingly authentic (this is also true for Drake lyrics). Leaving the uncanny valley of awkward robot voices will make adoption surge.

3) NVIDIA Announces Strategic Partnership With AGI-Focused Company Organisation

Usage statistics in AI research show that NVIDIA’s adoption is 20x-100x larger than that of competitors. If NVIDIA could pick or even help create a winning organization, this would cement their position.

4) Investment of >100M Into a Dedicated AI Alignment Organisation

Artists were not happy as model-generated artwork won an art competition in Colorado. Advances such as this will cause sentiments about AI safety to aggravate.

5) Proposal to Regulate AGI Labs Like Biosafety Labs Gets Backing By EU, GB, or US politicians

OpenAI scrambles to prevent ChatGPT from showing people how to build bombs. Responding to an outcry from artists, Stability AI has announced they will allow artists to opt out such that their work is not used as training data. As negative impacts accumulate, regulation gains momentum.

6) GAFAM invests another >$1B Into an AGI or Open-Source AI company Like OpenAI or StabilityAI

The increase in the cost of model training has led to more and more innovation happening in the industry. Regardless of the economy’s choppy waters, big tech knows that staying ahead of the curve on machine learning will guarantee smooth sailing.

7) DeepMind Will Train 10B Parameter RL Model an Order of Magnitude Larger Than GATO

Currently, most machine learning models are very specialized. They can do one thing and one thing only. In 2022 DeepMind released GATO. This multi-modal model can, among other things, generate text, control a robot arm, and play video games. However, this line of research does not simply make models more versatile. The possibility of using sequence data for training increases the diversity and availability of training data.

If you made it all the way here, thank you! As always, it was a privilege making this for you and I sincerely hope you found it useful!

If you are not subscribed yet, at The Decoding ⭕ I send out a thoughtful newsletter about ML research and the data economy once a week. No Spam. No Nonsense. Click here to sign up!