Naveen Rao: 4D Computing, AI's Energy Wall & Beating Biology
Episode Details
In this comprehensive episode of the All-In Podcast, guest Naveen Rao—co-founder and CEO of the AI chip startup Unconventional AI—provides deep insights into the future of Hardware and AI. Rao, who firmly opposes AI Doomerism, shares his entrepreneurial journey starting with founding Nirvana systems, which he later sold to Intel. He subsequently collaborated with Ali Ghodsi at Data Bricks to build infrastructure for scaling Large Language Models (LLMs). Rao highlights a critical impending crisis: severe Energy constraints in AI. He points out that models like ChatGPT require an enormous amount of power, and processing quadrillions of AI Tokens by tech giants like Google is driving unsustainable Energy Consumption for AI. Across the US, massive Data Centers are grappling with this issue, making access to power contracts the primary bottleneck for growth. To address this, Rao argues for a radical departure from traditional systems like the CPU and GPUs. He notes that since the creation of ENIAC, computers have relied on the von Neumann architecture, which is inherently inefficient at moving data back and forth, especially as the scaling benefits of Moore's law applied to Semiconductors have largely ended. Instead of relying on traditional Linear algebra and Floating point math, Rao draws inspiration from Neurobiology and Mammalian brains to develop 4D Computing and advanced Compute in memory. By applying Dynamical systems theory, his team is building physical circuits that exhibit intelligent emergent behavior. As a proof of concept, Unconventional AI released Uno, an Open Source model capable of generating images and handling Sequence modeling. The model uses a concept called Sparsity and analyzes the State space trajectory of oscillators to achieve extreme efficiency, approaching the theoretical Thermodynamic limit and pushing past the boundaries of traditional 2D lithography. During a Q&A session, Chamath Palihapitiya asks about the practical application of this technology, such as whether it operates as a cloud VM and how it compares to legacy programming frameworks like CUDA. Rao clarifies that they use custom Python libraries to interface with the new systems. Ultimately, bringing this highly efficient Generative AI to market is expected to unlock a 1 Trillion Dollar Economic Opportunity. Rao predicts that vastly cheaper compute will trigger Jevons' Paradox, driving unprecedented demand and enabling the widespread deployment of autonomous Robotics, potentially shifting the balance of power currently held by industry leaders like Jensen Huang.
Key Topics & People
.jpg?width=300){kind=avatar}
Host of the All-In Podcast, venture capitalist, and early Facebook executive.
A popular technology and business podcast where industry leaders discuss startups, economy, and global events.
An organized narrative promoting fear about AI-driven job losses and existential risks, often leveraged for political gain.
Critical infrastructure for AI compute, increasingly viewed as national security assets requiring massive energy generation.
.jpg?width=300){kind=avatar}
CEO of Nvidia who played a key role in drafting the AI safety accord's internal and external audit mechanisms.
Artificial intelligence tools used by Spencer Pratt to create viral, engaging campaign videos that broke through legacy media barriers.
An AI chip startup rethinking computer foundations for maximum power efficiency.
Co-founder and CEO of Unconventional AI, with a background in electrical engineering and a PhD in neuroscience.
Agreements for energy supply, which have become the primary bottleneck in building data centers.
A computing paradigm utilizing physical 3D stacking and the dimension of time.
The first AI chip company founded by Naveen Rao in 2014.
Standard mathematical operation used in AI models which is absent in biological brains.
Precision calculation standard used in digital computing but circumvented in biology.
Study of complex systems where emergent behaviors arise from simple interconnected components.
The evolution of a dynamic system across time phases.
Traditional computer design separating compute and memory, resulting in energy inefficiency.
Architecture where information processing happens directly where the data is stored.
Traditional flat semiconductor fabrication techniques currently hitting physical scaling limits.
Biological intelligence operating closely to the thermodynamic limit of energy efficiency.
The absolute physical barrier of energy efficiency that biological brains approach.
A class of generative AI tasks applicable to dynamic computers, similar to image generation.
Economic principle indicating that increasing technological efficiency actually increases resource consumption.
The projected enormous market size for AI platforms.
Software made freely available, exactly how the Uno model was released.
Complex intelligence or coordination rising from simple individual rules, such as in flocking birds.
The underlying physical hardware materials powering computing systems.
The historical trend of shrinking transistors that has stopped delivering efficiency gains.
The study of biological nervous systems, offering proof-points for ultra-efficient computing.
The growing concern of AI data centers requiring immense and unsustainable amounts of electricity.
The impending issue of running out of energy due to AI scaling.
Leader at Databricks with whom Naveen Rao collaborated.
Data platform company that Naveen joined forces with to scale LLM infrastructure.
Massive AI models that require tremendous infrastructure and computing power.
.svg?width=300){kind=logo}