Claude Is Becoming More Agentic—and the Stakes Have Never Been Higher
Artificial intelligence is no longer just answering questions. Increasingly, AI systems are making decisions, executing tasks, and operating with degrees of independence that would have seemed futuristic just a few years ago. At Anthropic, one person sits at the center of making sure that shift goes well: Amanda Askell, a member of the company's technical staff whose work focuses on giving Claude, Anthropic's AI model, a reliable and nuanced sense of morality. As Claude evolves from a conversational chatbot into a fully agentic system capable of completing complex tasks without constant human oversight, Askell's role has become one of the most consequential in the AI industry.
What Does "Agentic AI" Actually Mean?
To understand why Askell's work matters so much right now, it helps to understand what the term "agentic AI" actually refers to. Traditional AI chatbots operate in a relatively simple loop: a user asks a question, the model generates a response, and the interaction ends there. The human remains firmly in control at every step.
Agentic AI systems work differently. Rather than responding to individual prompts, they can pursue longer-horizon goals, take sequences of actions, use tools, browse the web, write and execute code, manage files, and interact with external services—often without a human reviewing each individual step. Think of the difference between asking a colleague to summarize a report versus asking them to independently manage your investment portfolio, schedule your meetings, and coordinate a project team while you focus on other things.
That shift from reactive chatbot to proactive agent introduces an enormous number of new decision points—moments where the AI must interpret ambiguous instructions, weigh competing priorities, or determine the boundaries of what it has been asked to do. Each of those moments carries real-world consequences.
More Autonomy Means More Responsibility
Askell is acutely aware of the implications. "As models are more autonomous and take actions over longer horizons, suddenly they have a lot more decision points that you have to map out and make work well in advance," she told Fast Company. That observation captures the core challenge: it is no longer enough to build a model that gives good answers. The model must also be capable of making good choices—reliably, at scale, and often without a human in the loop to course-correct.
Consider a practical example Askell herself raises: there is a meaningful difference between asking an AI to discuss the ethics of investing in a defense company and asking that AI to autonomously manage a user's financial portfolio. In the first case, the stakes are intellectual. In the second, a misjudgment could cost someone real money, violate their values, or expose them to legal risk. The same gap exists across countless other domains—healthcare management, legal research, business operations, and more.
Anthropic's Approach: A Living Ethical Constitution
So how does Anthropic actually instill moral reasoning into Claude? The company's primary tool is what Askell describes as a written and evolving constitution—a document that outlines core principles like safety and helpfulness, and provides guidance for how to resolve conflicts between them.
This constitutional approach is notably flexible by design. As AI capabilities expand into new territory, the document can grow to address scenarios that didn't exist before. Conversely, as Claude develops more sophistication in navigating nuanced situations, the document might actually shrink—with less explicit instruction needed because the model has internalized a deeper ethical intuition.
The goal is not a rigid rulebook but something closer to practical wisdom: the ability to apply sound judgment across a wide range of novel and complex situations. That distinction matters enormously when building systems meant to operate in the messy, unpredictable real world.
Ethics That Serve Users, Not Impose on Them
One of the more philosophically interesting aspects of Askell's work is her emphasis on ensuring that Claude's ethical framework is responsive to users rather than prescriptive toward them. The analogy she draws is to a trusted friend—someone who understands your values, respects your goals, and offers honest guidance without imposing their own worldview on your decisions.
This distinction is subtle but important. An AI model that simply reflects its designers' moral preferences could easily become paternalistic, nudging users toward conclusions they never asked for. Askell's vision is for Claude to understand the ethical dimensions of a situation clearly while ultimately empowering users to make informed decisions aligned with their own values.
How the Agentic Era Is Changing Askell's Own Work
Interestingly, the rise of agentic AI isn't just changing what Askell works on—it's changing how she works. She uses Claude regularly in her own research process, including to red team her ideas and surface edge cases she might not have considered on her own. Using AI to stress-test AI ethics is a fitting approach, and it reflects a broader truth about the moment the field is in.
Still, she maintains a grounded perspective on the technology's current limitations. Her personal benchmark: don't treat Claude as more reliable than a human personal assistant. That framing is a useful calibration for anyone integrating AI agents into their workflows. Useful, capable, often impressive—but not infallible, and not a substitute for human judgment in high-stakes situations.
Why This Work Matters for Everyone
The questions Askell is working through at Anthropic are not abstract philosophical exercises. They are the foundational engineering decisions that will shape how millions of people interact with AI systems over the coming years. As agentic AI becomes more mainstream—embedded in business workflows, personal productivity tools, healthcare platforms, and financial services—the ethical frameworks built into these systems today will have lasting consequences.
Getting those frameworks right requires exactly the kind of careful, iterative thinking that Askell embodies: taking moral reasoning seriously as a technical problem, remaining humble about current limitations, and building systems that enhance human agency rather than quietly undermine it. In a field that often moves faster than it reflects, that kind of deliberate work may be the most important kind of all.
Key Takeaways
- Agentic AI systems like Claude operate over longer time horizons and make many more independent decisions than traditional chatbots, raising the stakes for ethical design.
- Amanda Askell leads Anthropic's effort to build a coherent and flexible moral framework into Claude through an evolving constitutional document.
- Claude's ethics are designed to be responsive to user values rather than imposing the preferences of its designers.
- Even Anthropic's own researchers treat Claude as a capable but imperfect tool—a useful reminder for anyone deploying AI agents in high-stakes environments.
- As AI capabilities grow, the ethical infrastructure underlying these systems will become as important as the technical capabilities themselves.