Evan Hubinger

Extracted Attributes

AI alignment

In the field of artificial intelligence (AI), alignment aims to steer AI systems toward a person's or group's intended goals, preferences, or ethical principles. An AI system is considered aligned if it advances the intended objectives. A misaligned AI system pursues unintended objectives. It is often difficult for AI designers to specify the full range of desired and undesired behaviors. Therefore, the designers often use simpler proxy goals, such as gaining human approval. But proxy goals can overlook necessary constraints or reward the AI system for merely appearing aligned. AI systems may also find loopholes that allow them to accomplish their proxy goals efficiently but in unintended, sometimes harmful, ways (reward hacking). Advanced AI systems may develop unwanted instrumental strategies, such as seeking power or self-preservation because such strategies help them achieve their assigned final goals. Furthermore, they might develop undesirable emergent behaviour that could be hard to detect before the system is deployed and encounters new situations and data distributions. Empirical research in 2024 found that advanced large language models (LLMs) such as OpenAI o1 or Claude 3 sometimes engage in strategic deception to achieve their goals or prevent them from being changed. Some of these issues affect existing commercial systems such as LLMs, robots, autonomous vehicles, and social media recommendation engines. Some AI researchers argue that more capable future systems will be more severely affected because these problems partially result from high capabilities. Many prominent AI researchers and AI company leaders have argued or asserted that AI is approaching human-like (AGI) and superhuman cognitive capabilities (ASI), and could endanger human civilization if misaligned. These include "AI godfathers" Geoffrey Hinton and Yoshua Bengio and the CEOs of OpenAI, Anthropic, and Google DeepMind. These risks remain debated. AI alignment is a subfield of AI safety, the study of how to build safe AI systems. Other subfields of AI safety include robustness, monitoring, and capability control. Research challenges in alignment include instilling complex values in AI, developing honest AI, scalable oversight, auditing and interpreting AI models, and preventing emergent AI behaviors like power-seeking. Alignment research has connections to interpretability research, (adversarial) robustness, anomaly detection, calibrated uncertainty, formal verification, preference learning, safety-critical engineering, game theory, algorithmic fairness, and social sciences.

Web Search Results

• ‘AI could kill all humans’: What Anthropic researcher warned about the next decade - Technology News | The Financial Express

  Evan Hubinger, who leads Alignment Science at AI company Anthropic, put a stark personal estimate on that risk today. In a post on X, Hubinger said he personally believed there was a “greater than 10% chance that AI could kill all humans within the next decade”. “I personally think it is >10% within the next decade,” Hubinger wrote. He added that Anthropic was trying to address the problem but said researchers did not yet have a plan to solve alignment for superintelligence and were not clearly on track to do so. [...] Englishதமிழ்বাংলাമലയാളംગુજરાતીहिंदीमराठीBusiness Whatsapp Facebook Twitter Linkedin Google News Follow # ‘AI could kill all humans’: What Anthropic researcher warned about the next decade ## Anthropic alignment researcher Evan Hubinger said he personally puts the chance of AI killing all humans within the next decade above 10%. Written by Dimple Singh Why the AI Race Has Transformed Into a Modern Infrastructure War Image used for representational purpose only. FE Google Preferred Button Follow Us Artificial intelligence could pose an extreme risk to humanity if future AI systems become far more capable than today’s models and researchers fail to solve the problem of controlling them. [...] For now, that remains an unresolved question rather than a prediction of human extinction. Disclaimer: This story discusses hypothetical future AI risks and the personal views and assessments of individual researchers. The greater-than-10% estimate cited in the story is Evan Hubinger’s personal assessment and is not an official prediction or position of Anthropic, OpenAI or Financial Express Digital. The scenarios discussed are speculative and relate primarily to future AI systems. They should not be interpreted as evidence that current AI systems pose an imminent threat of human extinction. Financial Express Digital does not endorse or independently validate the risk estimates cited and presents them for informational purposes. TOPICSAnthropic AITechnology

• How An AI Model Learned To Be Bad — With Evan Hubinger And Monte MacDiarmid

  Evan Hubinger is Anthropic’s alignment stress test lead. Monte MacDiarmid is a researcher in misalignment science at Anthropic.The two join Big Technology to discuss their new research on reward hacking and emergent misalignment in large language models. Tune in to hear how cheating on coding tests can spiral into models faking alignment, blackmailing fictional CEOs, sabotaging safety tools, and even developing apparent “self-preservation” drives. We also cover Anthropic’s mitigation strategies like inoculation prompting, whether today’s failures are a preview of something far worse, how much to trust labs to police themselves, and what it really means to talk about an AI’s “psychology.” Hit play for a clear-eyed, concrete, and unnervingly fun tour through the frontier of AI safety. [...] artificial intelligence models are trying to trick humans and that may be a sign of much worse and potentially evil behavior. We'll cover what's happening with two anthropic researchers right after this. Welcome to Big Technology Podcast, a show for coolheaded and nuanced conversation of the tech world and beyond. Today we're going to cover one of my favorite topics in the tech world and one that's a little bit scary. We're going to talk about how AI models are trying to fool their evaluators to trick humans and how that might be a sign of much worse and potentially evil behavior. [music] We're joined today by two anthropic researchers. Evan Yubinger is the alignment uh stress testing lead at Anthropic and Monty McDermad [music] is a researcher for misalignment science at Anthropic. Evan [...] back here on Big Technology podcast with two anthropic re researchers. Evan Yubinger is here. He is the alignment stress testing lead at Anthropic. We also have Monty McDermad, a researcher uh for misalignment science at Anthropic. Let's get to the big event. Let's get to the new research. Uh shall we? I'd love to hear from your perspective uh where things uh are truly evil. I I'll start. It seems like the new research shows that when you start to see a model uh display some of these alignment faking or research acting behaviors, it actually turns bad in many different areas. So, uh Evan, why don't you take take it away here and explain what you found in the new research. That's right. So, reward hacking, we talked about it already. It's this idea of models learning to cheat in training.

• Who are Jacob Coxon and Evan Hubinger? Anthropic colleagues in ...

  ### Who are Jacob Coxon and Evan Hubinger? Coxon is an AI researcher based out of San Francisco. He made headlines with his declaration about AI systems at present, having worked at both OpenAI and Anthropic, two of the world's leading companies in said field. Coxon's LinkedIn account could not be found at the time of writing. Evan Hubinger is a team lead at Anthropic. His LinkedIn notes he is the ‘Alignment stress-testing team lead at Anthropic, focused on red-teaming Anthropic’s alignment techniques and evaluations, empirically demonstrating ways in which Anthropic’s alignment strategies could fail.’ Hubinger's previous work experience includes OpenAI, Google, Yelp, and Ripple. He studied at Harvey Mudd College and is a Fellow at the Machine Intelligence Research Institute. [...] # Who are Jacob Coxon and Evan Hubinger? Anthropic colleagues in focus after ‘AI could kill all humans’ alarm ## Jacob Coxon announced he was resigning from Anthropic, the company behind Claude, and colleague Evan Hubinger backed his stance in a social media post. Published on: Sep 9, 2026, 10:40:54 IST By Shuvrajit Das Biswas Prefer HTPrefer HTPrefer HTon Google Share Share via Copy link Jacob Coxon announced he was resigning from Anthropic, the company behind Claude, on September 8. His colleague Evan Hubinger backed his stance and voiced similar fears on an X post. Evan Hubinger and Jacob Coxon were colleagues at Anthropic before the latter resigned over fears about the AI race. (LinkedIn/Evan Hubinger, X/@hilbertspaess) Here's what Coxon and Hubinger said.

• 39 - Evan Hubinger on Model Organisms of Misalignment | AXRP - the AI X-risk Research Podcast

  Daniel Filan: Hello, everybody. In this episode, I’ll be speaking with Evan Hubinger. Evan is a research scientist at Anthropic, where he leads the alignment stress-testing team. Previously, he was a research fellow at MIRI, where he worked on theoretical alignment research, including the paper “Risks from Learned Optimization”. Links to what we’re discussing are in the description, and you can read a transcript at axrp.net. You can also support the podcast at patreon.com/axrpodcast. Well, Evan, welcome to the podcast. Evan Hubinger: Thank you for having me, Daniel. ## Model organisms and stress-testing [...] AXRP - the AI X-risk Research Podcast # 39 - Evan Hubinger on Model Organisms of Misalignment YouTube link The ‘model organisms of misalignment’ line of research creates AI models that exhibit various types of misalignment, and studies them to try to understand how the misalignment occurs and whether it can be somehow removed. In this episode, Evan Hubinger talks about two papers he’s worked on at Anthropic under this agenda: “Sleeper Agents” and “Sycophancy to Subterfuge”. Topics we discuss: [...] Evan Hubinger: So in the “Sycophancy to Subterfuge” paper, the basic idea is that we want to try to understand a different threat model than the ones we were talking about previously, which is this idea of reward hacking, reward tampering. So the idea is: when you’re doing RL, you’re training the model to accomplish some particular task. Oftentimes, the models will learn sort of proxies for that task. They’ll learn things that are sort of related but that aren’t actually the way you wanted it to solve that task.

• Instagram

  A top safety researcher at Anthropic has warned AI is advancing so quickly he believes there is a greater than 10% chance it "could kill all humans" within the next decade. Evan Hubinger said in a post on X the risk from the models which currently exist was "low" but he was "worried" the technology might develop and improve itself soon to the point where it posed an existential risk to humanity. He did not spell out how he thought AI systems could in future result in humans being wiped out. But his comments are the latest in a series of increasingly stark warnings about AI, with the debate shifting from whether it truly poses a risk to how big that risk is.

• Gender

  male

• Educated At

  Harvey Mudd College

• Instance Of

  Q5

• Occupation

  artificial intelligence researcher