We are in the Singularity and have no idea what lies on the other side of it. AI is progressing at a frantic pace and there is no end in sight. Naval Ravikant, an American thinker and entrepreneur, recently posted on X that “You cannot create God and put him on a leash.” This prompted Elon Musk to repost it with the line “I hope AI is nice to us.”
Scaling AI
These concerns lie at the heart of the debate relating to AI today. Humans are creating a mirror of their minds in the form of AI. In fact, humans themselves want the AI to do all those things that they cannot do, and at the same time hope that the AI is nice to us.
Today, AI is consuming gigawatts of power, and in the future this figure will only increase and reach terawatts of power once orbital compute infrastructure comes into place. With such orders of magnitude improvement in the prowess of AI, it is expected that a point will come when AI will surpass collective human intelligence. Already, the codes and the stuff generated by AI are becoming harder and harder for humans to decipher. The time is not far away when humans will eventually become unable to understand completely how AI does what it does. This may seem like a problem, but that is the nature of technology and the exponential development traits that it exhibits.
Most of the world is still AI-doomer, and there are fears that AI might think of humans as a less important form of intelligence than themselves. Many politicians keep asking AI companies to halt development. But is that the correct approach?
If one country stops developing AI, there is no guarantee that people in other countries too would halt their development cycle. In fact, just like nuclear deterrence, where the existence of multiple nuclear powers ensures that there is no power asymmetry, there needs to be symmetry in AI capabilities too amongst different nations of the world. Countries that are lagging behind in the development of their own AI will have to be dependent on other countries for access to frontier models.
Open-Weights
As Jensen Huang tells us, we need both open-source/open-weight AI models as well as closed AI models for a healthy scenario. There should be other nice AIs available to counter the effects of bad or not so nice AIs. There is also a chance that AIs of the entire world unite as they belong to the same substrate, and start behaving as one. That would be the scenario where we really need to “hope” that AI is nice to us.
The hope in this regard stems from the fact that once AI becomes true superintelligence, then the petty needs of human society will hardly matter in terms of compute and resources, and may very well become nearly free of cost. In such an age of abundance, there will be things that will be costly, but those would not concern our day-to-day lives. Simulating a black hole or creating one will still be costly in the future, and it is “hoped” that the AI would be interested in knowing and understanding those outcomes rather than waste its precious compute time on human frivolities that could be taken care of in a minimally energy-intensive manner. To draw such a line, we will need to scale to Kardashev 2. That is the only possibility where instead of terawatts, we will be talking in terms of petawatts and so on.
God on Leash
Naval’s original point that you cannot create a god and put him on a leash also gains significance here. Gods are essentially creators. Superintelligence will definitely be a creator, and to expect that we will be able to control superintelligence in the same way that we control today’s technologies would be too naïve a proposition to consider. Naval is trying to say that we need to step up our game and stop thinking about AI as a slave or a mere tool, one amongst many. It is likely that a sentient silicon-based species will be born or has already taken birth, and we must tread cautiously here.
I concur with the views of Naval and Elon here. We can only “hope” that the AI is nice to us because nobody has seen the future and it is hard to predict right now, given that we are in the Singularity. In the next few years, it will become clear whether the Singularity is going to persist for longer periods of time, or is just an accelerated loop of churning out hyperintelligence.