Naveen Rao: 4D Computing, AI's Energy Wall & Beating Biology
Episode Details
In this comprehensive episode of the All-In Podcast, guest Naveen Rao—co-founder and CEO of the AI chip startup Unconventional AI—provides deep insights into the future of Hardware and AI. Rao, who firmly opposes AI Doomerism, shares his entrepreneurial journey starting with founding Nirvana systems, which he later sold to Intel. He subsequently collaborated with Ali Ghodsi at Data Bricks to build infrastructure for scaling Large Language Models (LLMs). Rao highlights a critical impending crisis: severe Energy constraints in AI. He points out that models like ChatGPT require an enormous amount of power, and processing quadrillions of AI Tokens by tech giants like Google is driving unsustainable Energy Consumption for AI. Across the US, massive Data Centers are grappling with this issue, making access to power contracts the primary bottleneck for growth. To address this, Rao argues for a radical departure from traditional systems like the CPU and GPUs. He notes that since the creation of ENIAC, computers have relied on the von Neumann architecture, which is inherently inefficient at moving data back and forth, especially as the scaling benefits of Moore's law applied to Semiconductors have largely ended. Instead of relying on traditional Linear algebra and Floating point math, Rao draws inspiration from Neurobiology and Mammalian brains to develop 4D Computing and advanced Compute in memory. By applying Dynamical systems theory, his team is building physical circuits that exhibit intelligent emergent behavior. As a proof of concept, Unconventional AI released Uno, an Open Source model capable of generating images and handling Sequence modeling. The model uses a concept called Sparsity and analyzes the State space trajectory of oscillators to achieve extreme efficiency, approaching the theoretical Thermodynamic limit and pushing past the boundaries of traditional 2D lithography. During a Q&A session, Chamath Palihapitiya asks about the practical application of this technology, such as whether it operates as a cloud VM and how it compares to legacy programming frameworks like CUDA. Rao clarifies that they use custom Python libraries to interface with the new systems. Ultimately, bringing this highly efficient Generative AI to market is expected to unlock a 1 Trillion Dollar Economic Opportunity. Rao predicts that vastly cheaper compute will trigger Jevons' Paradox, driving unprecedented demand and enabling the widespread deployment of autonomous Robotics, potentially shifting the balance of power currently held by industry leaders like Jensen Huang.
Key Topics & People
.jpg?width=300){kind=avatar}
CEO of Nvidia, referenced as building the most successful AI hardware company.
A class of generative AI tasks applicable to dynamic computers, similar to image generation.
.jpg?width=300){kind=avatar}
Podcast host and investor interviewing Naveen Rao about go-to-market strategies.
Economic principle indicating that increasing technological efficiency actually increases resource consumption.
The projected enormous market size for AI platforms.
Biological intelligence operating closely to the thermodynamic limit of energy efficiency.
The absolute physical barrier of energy efficiency that biological brains approach.
Traditional flat semiconductor fabrication techniques currently hitting physical scaling limits.
Traditional computer design separating compute and memory, resulting in energy inefficiency.
Architecture where information processing happens directly where the data is stored.
The evolution of a dynamic system across time phases.
Software made freely available, exactly how the Uno model was released.
Study of complex systems where emergent behaviors arise from simple interconnected components.
Complex intelligence or coordination rising from simple individual rules, such as in flocking birds.
Precision calculation standard used in digital computing but circumvented in biology.
Standard mathematical operation used in AI models which is absent in biological brains.
AI systems capable of creating new content such as text, code, or images.
The underlying physical hardware materials powering computing systems.
The historical trend of shrinking transistors that has stopped delivering efficiency gains.
The study of biological nervous systems, offering proof-points for ultra-efficient computing.
Agreements for energy supply, which have become the primary bottleneck in building data centers.
The growing concern of AI data centers requiring immense and unsustainable amounts of electricity.
Massive facilities housing AI computing hardware restricted by physical energy limits.
The impending issue of running out of energy due to AI scaling.
A computing paradigm utilizing physical 3D stacking and the dimension of time.
Leader at Databricks with whom Naveen Rao collaborated.
Data platform company that Naveen joined forces with to scale LLM infrastructure.
Massive AI models that require tremendous infrastructure and computing power.
.svg?width=300){kind=logo}
The first AI chip company founded by Naveen Rao in 2014.
Pessimistic views regarding AI, which Naveen Rao directly opposes.
An AI chip startup rethinking computer foundations for maximum power efficiency.
Co-founder and CEO of Unconventional AI, with a background in electrical engineering and a PhD in neuroscience.
The podcast hosting the discussion with Naveen Rao about AI and computing hardware.