Make History And Win 1 Million Dollars On This Fascinating AI Treasure Hunt

This week’s story sounds like it was taken straight from a science fiction novel.

The leaders of the Church are shaking in fear because of what AI could bring to light.

Thousands of years ago, a massive volcanic eruption wiped out a monumental city in a matter of hours. Among the thousands of destroyed houses was one very special estate. It belonged to a close relative of the most powerful Kaiser that ever lived.

On his estate was a vast library filled with thousands of papyrus scrolls of unspeakable value.

The scrolls contain texts from long-lost secrets about philosophy, science, and possibly even about the origins of modern religions. When the house was destroyed along with the library, the conditions under the scorching hot lava miraculously preserved the scrolls. Under the stone, the scrolls survived for thousand of years.

The scrolls were discovered but have become so fragile that they cannot be opened anymore without destroying them. So, scientists are using modern particle accelerators and AI to unlock the secrets hidden in them. A price of $1M will go to whoever manages to read the scrolls first.

Pretty good no?

The best part about this story is that it is not made up. Okay, I might have been adding some drama in my depiction of church leaders shaking under their cassocks. I am pretty sure they neither know what is going on nor are they reading this newsletter.

In this week’s edition, we will look at a spine-tingling story behind the Vesuvius Challenge and see how computer vision can help to unlock the secrets of the past.

Let’s jump in!

What Actually Happened

In 79 AD the Vesuvius volcano erupted and buried the city of Pompeii. What very few people know is that multiple cities were also destroyed in the incident. One of these cities was Herculaneum.

We can think of Herculaneum as the Beverly Hills of Pompeii.

The city was full of marvelous villas and estates. One of the more impressive ones belonged to Caesar’s father-in-law. It goes without saying, the guy was very powerful, well-connected, and super-rich.

Inside his estate was a giant library full of scrolls from the Greek and Roman times.

When the villa was destroyed, the heat of the lava carbonized (turning to charcoal without burning) the scrolls. This has preserved them for almost 2000 years. Since the 18th century, different groups tried to dig up the scrolls.

To date, more than 1800 scrolls have been excavated and most likely there are many more underground.

Some people speculate that his library might even contain scrolls from the library of Alexandria that burned down a few years before. From these scrolls, we might discover completely new philosophical schools, scientific secrets of the Greeks and heck! maybe drafts of the bible with GPT watermarks on them.

However, there is a catch!

Quite frankly, the scrolls have more resemblance with a cigarette bud that a roll of papyrus.

Looking at the image above, it is needless to say that simply unrolling them is not really an option.

In the 17 hundreds, an Italian monk painstakingly tried to unroll some of the scrolls over several decades. The result was mostly papyrus confetti. He managed to uncover a few intact fragments that had philosophical texts written in Greek on them.

This is obviously not scalable and would destroy most of the texts. However, if we could read the scrolls this would more than double the amount of text that was handed down to us from the Greek and Roman times. The value of that is obviously hard to overstate!

But, if we cannot unroll the scrolls, how are we supposed to find what’s written on them?

How To Read The Scrolls Without Opening Them

The Herculaneum scrolls are not the first carbonized scrolls to be found.

In 1970, a number of 2000-year-old scrolls were discovered in the En-Gedi Oasis close to the Dead sea. With no Italian monks at hand and the foresight that opening the scrolls would destroy them Dr. Seals from the University of Kentucky pioneered a method called virtual unwrapping.

It allows us to read the scrolls without opening them.

First, a high-resolution CT scan is created of each scroll. The scan creates digital slices from the scroll. The slices are created lengthwise, similar to how a cucumber is cut. Now, to perform the virtual unwrapping a sheet of the scroll is traced along the cross-sections.

In the image above, you can see an animation of how this is done cross-section by cross-section until a connected piece of the scroll is extracted. These connected pieces are then virtually flattened in order to read the text (click here to see a short video of this).

So far so good. Why can we not just do the same with the Scrolls from Herculaneum?

There are a few challenges with applying this technique to the Herculaneum scrolls. On the one hand, the scrolls are very tightly wrapped and generally in pretty bad shape. One the other hand, the ink in the Herculaneum scrolls is radiolucent. This means that X-rays pass through the ink the same way they pass through the papyrus.

As a result, the ink, in the CT scans, is not visible to the human eye.

But there is good news. It has been shown that neural networks can pick up on subtle patterns in the scans that are created by the ink [2]. Next, we will look at how neural networks are being trained on the scans and how to win the price.

Read on!

The Challenge of Training On The Fragments

As mentioned above, a few of the scrolls were unrolled by an exceptionally patient Italian monk.

Some of the resulting fragments have legible ink on them.

This opens a window of opportunity to train a model on the fragments and use it to read the full scrolls. In order to train a model, 4µm 3D X-ray scans were created for the fragments. Then, additional infrared images were taken of the scroll fragments to make the ink more visible.

Next, the areas with ink were hand-labeled. Finally, the labeled images were aligned with the scans to create input and label pairs.

The data paper, in which they trained a model on the fragments, reports a pretty low recall (in the 40% range).

However, their approach appears to be quite basic. They formulated the problem as a patch-wise binary classification. So, for each patch, their model predicted ink vs. no ink. Furthermore, the final accuracy might not need to be very high to make the text readable.

Most likely, translating the model to the full scrolls will be the tough nut to crack.

Alongside the fragment datasets, we are provided with 8µm 3D X-ray scans of two full scrolls. As a matter of fact we are only given half of the scan data each of the two scrolls. The other half is held out as a validation set. Each half-scroll scan consists of 14,000 .tif files with 120MB each. Since each slice is 8µm tall, the scroll half is 11.2cm tall.

The two scrolls need to be virtually unwrapped first.

The software to do the unwrapping is provided. Some manual work is required to get it going, but all the pieces are there. I dearly hope that the challenge attracts many brilliant minds from all over the world!

If you have some time on your hand, or you simply want to make some money to buy a few A100 GPUs go and check out the challenge!

The best ink detection model gets $100K and whoever is the first to read four separate passages on one of the full scrolls wins $700K. Additional $200K of prices will be announced in the coming months.

Money aside, the thought that some guy or girly with a cup of coffee and a laptop could create a model which unlocks this trove of wisdom makes me excited about the present and the future alike.

What an exciting time for science and humanity!

As always, I really enjoyed making this for you and I sincerely hope you found it useful!

If you did find it useful and are not subscribed to the newsletter yet, you can do that here! I send out a thoughtful 5-minute email every week to keep you in the loop about machine learning research and the data economy.

Thank your for reading and I see you next week ⭕!

​References:

[1] https://en.wikipedia.org/wiki/Herculaneum_papyri​

[2] https://raw.githubusercontent.com/educelab/EduceLab-Scrolls/main/paper/EduceLab-Scrolls.pdf