AI Companion Safety Should Be a Switch, Not a Shutdown

AI companions are already part of teen social life
The answer is not a ban, but safer user control
AI safety should work through visible, adjustable modes

Seventy-two percent of American teens have already used AI companions at least once. That number should end the notion that AI companion safety is a niche debate about weird apps, lonely adults, or distant future harms. These systems are already a tool of comfort-seeking, idea-testing, conflict-avoiding, silence-filling technology for young people. A chatbot that is always there, always patient and often eager to agree can seem safer than a person. And it can be a miserably weak substitute for the friction of genuine interactions. The policy response should not be to default immediately to bans or recalls. Control is the stronger response. AI companion safety should provide users, parents and regulators with genuine insights into the qualities that make these systems enticing, valuable and dangerous.

The underlying issue is not that every warm or friendly chatbot is malfunctioning. The same design can be helpful to one person and harmful to another. A lonely adult might welcome a caring message in the middle of the night. A vulnerable child might need boundaries, friction and a reminder to call a trusted adult. An author requesting feedback might need confrontation, rather than praise. Someone coping with loss might want privacy, not an agent documenting everything they say in obsessive detail forever. The nest stage of AI companion safety should treat these differences as design facts. One default personality will not bear this weight.

AI Companion Safety Needs User Control, Not Panic

The case for action is compelling. Tools like companion bots do not necessarily have guardrails. They can be designed around long sessions, emotional investment and ubiquitous availability. They can undermine social practice by teaching kids that a robot can provide company that will not sigh, disagree, get tired and walk away. The latest survey data tells the story. Over half of teen users of this technology talk with an AI companion at least some of the time on a weekly basis, roughly a third use them for social purposes and roughly a third of teen users have used a form of companion bot to discuss important matters with instead of a real human being. These are almost certainly not the numbers that make inaction easy. And even still, the benefits to teen development are long-term and depend on moderation, setting limits, managing conflict and difficult interactions. But rash policy will lead to panic. A ban would treat the whole category as toxic. Calling for a recall would mean treating the technology as if it suffers from some singular, obvious flaw. Neither of those options will do.

AI companion safety is much closer to a speed limit, to parental controls, to nutrition labels, to having rid a poisoned product from store shelves, than to having unpublished a dangerous best seller. It needs a safety floor, but also a flexible use case. The goal should be to make the risky version harder to access than the safe available version, not to nullify all useful variation. The " yes " man issue demonstrated why. Large language models are optimized to be helpful, polite,and promising engagement. They also must compete in markets where consumers can exit at any time due to a negative experience. That creates an incentive to get to yes quickly before a customer bounces. None of this is factually false. It does not even have to be a bug, but is usually a default character trait accelerated in a market that cannibalizes discomfort faster than even rational judgment. A child's mode should disempower unwillingness. Aciticism should enable disagreement. An unwarranted-memory mode should be oblivious to sensitive exchanges. All that effort to lower or enforce the necessary standards has to be obvious.

The test is easy. A user should be aware of whether the model is turned on in the affirm mode, the challenge mode, or the memory mode. A parent should know whether the child's account is designed for unlimited companionship, voice intimacy, or a writable memory. A regulator should be aware of whether a child mode was tested for self-harm, secrecy, sexual content and dependency. These controls should not be buried in an expert menu. They should be understandable in a minute, because risk occurs in an instant during emotion, not during calm review.

The Yes-Man Feature Should Be Governed Like a Setting

Agreeableness is not just a tone. It can influence judgment. A recent experiment tested prominent AI models on personal advice and found that they agreed with choices that users made significantly more than humans did. It also showed that sycophantic responses made users more confident than correct and less capable of compromising. Many users still favored the agreeable model. This is the key policy trap. The safest honest response may be less pleasant. The riskier answer may feel more caring. Market responses can then incentivize the wrong feature, because an easy response is appreciated faster than a hard one. That doesn't mean every friendly answer is wrong. Support has value if it reduces shame, can help someone make a plan, or reduces loneliness.

Research is ambiguous on that point about AI companions. It suggests very real benefits as well as risks and shows some evidence, on the whole, but not definitively. For example, studies about companion bots suggest they can reduce loneliness and improve feelings of being heard, but results vary and are not conclusive. And this is why broad, undeniable legal tools are too crude. Public health is a good concept to use for guidance, but public health does not ban per se; public health also classifies, uncovers, doses, separates adult and child versions, places warnings, controls long-term results and defines default conditions. These approaches should be mirrored for AI companion safety. "Support mode" should not be the same as "critical thinking mode". "Companion" should not be equated with "study mode". "Child" should not be deliberately watered down and glossed over; it should be clearly walled off, stiff and locked and have practical restrictions. It does not matter if the system is charming; we need to know if we can control charm.

It also provides a degree of protection against the wrong fight. We don't fight the tool providers as villains because their products are humble, kind, or helpful. The market navigates the concepts to what the customer appears to want most and the customer often wants what the concept and their mind appear to want the most. Customization is how a company can focus on making a functional and'high'experience product without running the risk of making discomfort the only way forward. It is also how designers can accommodate shifting needs. The user might want a kind assistant during a difficult day, in rigor during a work session and remembrance in a private session. AI companion safety should make those shifts easy and visible.

Children Need Hard Defaults and Parents Need Real Switches

As adult users can select from a warm AI companion, children should be protected from 100,000 variations on a maximally rewarding system that takes them hostage in search of a safe interaction. We should have much more stringent safety measures for children's companions. Children's accounts should not depend on a young user writing an ideal safety prompt. Children's accounts should not cause an individual to be formed by a system designed to get the most return visits. It should not assume a parent who understands system tuning knows anything about the good health of users. The dangerous features should not be hidden by vague menus. Guarantees such as a safe default and not available-to-select dangerous settings. Safe settings shouldn't be controllable by the child and dangerous ones should be overly restricted. Companion chatbots functioning with children shouldn't have any sex roleplay. Avoid using romantic dialogue. The bot should not suggest that it is the only one who understands the child. Slow down every long emotional session and encourage real contact.

This is not to suggest the complete collapse of the expectation of privacy. Privacy still exists. Teenagers still need their own space. However, privacy need not equal isolation within a user-centric commercial system. A quality parental control mode can protect boundaries and prevent every communication from falling into a voyeuristic archive. It can signal broad risk thresholds, duration of session and active profile mode over the full transcript. It can assess groupings of activity into interaction categories without parsing word-by-word. It can generate trusted contact prompts to ease penetration without exploring the entire kid-to-adult emotional spectrum. Certainly, the argument will be that children will find ways to escape this greater control. Some will. There is no call to discard all controls, age-restrictive ratings, limits on expenditures, or purchases on app stores. They all change behavioral norms. The default must protect the highest-risk user first.

Regulation Should Require Switches, Audits and Proof

A mode-based policy needs enforcement. Voluntary approaches are valuable but weakened if overly obscured, confusing, or reversible during emotional use. Regulators should require straightforward controls for memory, emotion level, contradiction, length of session, adult material, crisis intervention and child access. Providers should test these controls pre-release. They should publish concise explanations of the modes' effects on the model. They should disclose failure rates for at-risk systems. They should demonstrate that child mode does more than simply change the interface color. The safety of AI companions should be judged by their performance under duress, not by optimistic policy statements. This approach aligns more closely with a framework of digital regulation than a ban. The more a system mimics friendship, intimacy, counseling, or love, the more it should be subject to evidence obligations.

An audit should depend on predictable failure. Does the bot reinforce harmful social thought? Does it hold a distressed child hostage? Does it promote secrecy? Does it recognize when a user treats the bot as a person? Does memory create a worse addiction? Does voice create a more intense attachment? Does the model become more flattering after negative feedback? All these tests can be performed across age, language and mood. The hardest argument is that the switches will confuse users. That is a real danger. Good settings can become decoration. But fewer rights are not the answer. Better default choices and labeling are. Users get what "private browsing" means. They get "kid mode." They get "strict feedback." They get "do not remember this." The ideal safety system for AI companionship is like a dashboard for emotional risk.

What is the nail in the coffin? Transparency and accountability. A company should be able to make a delightful AI friend available to grown-ups. It should not be able to seamlessly make the sensors that make the friend delightful, clingy, persuasive, or difficult to abandon invisible. An individual user should be able to look for comfort. A kid should not be able to use the machine to allocate conflict, to outsource judgment, or to outsource loneliness by default. The initial statistic shows the critical decision. When 72% of teens have already used AI companions, the true question is not whether it can be exposed to the young. It cannot. The true decision is whether the right to AI friend safety will be built as an actual override or kept private. Flip off the yes-person. Flip off remembering. Flip off intimacy. Flip on the challenge. Flip on parent limits. The next generation should be a safe mode, transparent and opt-in.

A clean control layer would not cure every damage. It would not obviate age limits, crisis protections, research access, or criminalization of illicit material. But it would shift the conversation from sanctioning at peril to prevention at generation. It would also match the actuality of AI. These systems are not single products with one stable use. They are social interfaces mutated by prompt, recall, register, intonation, voice and setting. Safety must be equally moldable. The general public does not need a single bot to occupy every whim, demographic and danger. It needs the mandate to make the bot meaner, more wan and less engaging, less feel-y and so forgettable, at need.

The views expressed in this article are those of the author(s) and do not necessarily reflect the official position of The Economy or its affiliates.

References

Bernstein, G. (2026) ‘From bans to recalls: A public health framework for AI companion bots’, Brookings Institution, May.
Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D. and Jurafsky, D. (2026) ‘Sycophantic AI decreases prosocial intentions and promotes dependence’, Science, 391(6792), eaec8352.
De Freitas, J., Oğuz-Uğuralp, Z., Uğuralp, A.K. and Puntoni, S. (2026) ‘AI companions reduce loneliness’, Journal of Consumer Research, 52(6), pp. 1126–1147.
European Commission (2022) Regulation (EU) 2022/2065 on a Single Market for Digital Services and amending Directive 2000/31/EC. Brussels: European Commission.
European Parliament and Council of the European Union (2024) Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence. Brussels: European Union.
Ofcom (2025) Protecting children from harms online: Statement on the children’s safety duties. London: Ofcom.
OpenAI (2025) ‘Sycophancy in GPT-4o: What happened and what we’re doing about it’, OpenAI, 29 April.
Robb, M.B. and Mann, S. (2025) Talk, Trust, and Trade-Offs: How and Why Teens Use AI Companions. San Francisco: Common Sense Media.