The world is witnessing a tectonic shift in terms of its priorities. Elon Musk recently posted that the upcoming versions of Grok will be utilized to rewrite the entire corpus of human knowledge, adding missing information and deleting errors.
I find this to be super interesting, and it reminds me of Isaac Asimov’s Foundation series, where Hari Seldon chose Terminus, a remote planet situated at the edge of the galaxy, to host a project to preserve human knowledge. The reason why Hari Seldon chose to do so was because he was able to foresee the collapse of the Galactic Empire, and to mitigate the miseries associated with such a gigantic civilizational collapse, Hari created a repository of knowledge to preserve humanity’s collective wisdom and accelerate recovery. This compilation and repository came to be known as the Encyclopedia Galactica.
So, do we need an Encyclopedia Galactica today? The answer seems to be an emphatic yes because it is a great way to preserve our collective wisdom. But the real questions are how we create it, who creates it, and what goes inside it. With the advent of AI and LLMs (Large Language Models), creating such an enormous repository that is interactive in nature is indeed feasible now.
In fact, if we look closely at the way the LLMs function, they are already living encyclopedias of sorts. However, their accuracy, or the ability to extract truthful information from them, remains under duress, but with each iteration, these LLMs are becoming incrementally better.
In his post, Elon also highlighted that there is “far too much garbage in any foundation model trained on uncorrected data.” So, basically, LLMs scrape through the entire data available on the internet and in that process ingest things that are totally trash. Of course, we have algorithms, compute, and architecture to help LLMs sift the real grain from the chaff, but in this era of noise and misinformation, it is easy to get lost. Today, anything that becomes more vocal or visible presents itself like it is the truth. The whole conception of truth has become convoluted. We are living in an era where people constantly argue that my truth is better than your truth. And it is not just the people who do that. Be it corporations or even countries, they too are thriving on spreading propaganda and fake news.
We had Wikipedia, which definitely had the potential to become an Encyclopedia Galactica. It still continues to be a great source for everything that is not related to politics or where politics has not been forced. But sadly, we all know that it has lost its ability to be objective and has fallen into the rabbit hole of misinformation, falsities, and rumormongering.
Back when the internet was not there, Encyclopedia Britannica was considered to be the all and end all for all types of general information. However, the problem with Encyclopedia Britannica and Wikipedia is that they are heavily edit dependent. The content on these platforms is at the mercy of their editors. In the name of being politically correct, authenticity and truth are lost. Though Encyclopedia Britannica is integrating AI, but its potential to be Hari Seldon’s Encyclopedia Galactica
Coming back to AI and LLMs, today we have so many of them. All the LLMs are trying to become better as a product and integrate more features that would increase their utility. Any one of them could be a good contender for being Encyclopedia Galactica. I mostly use Grok, so I have a bias for that. But otherwise, all the present models, Gemini, ChatGPT, Claude, Grok, etc., are running neck and neck with each other.
It is an interesting time to live in. From quantum computing to drone wars to AI revolution, everything about this era is so chaotic, and let us hope that this disruption and pandemonium leads us to the creation of an Encyclopedia Galactica.