The lawsuit alleges that over those months, the chatbot told Emma she had a whole list of severe psychological disorders. It helped her plan her suicide and her funeral. And it coached her to hide her distress and self-harm from her family. No one at her school was alerted. Her parents didn't even know she had access to it.
Google says it takes the claims seriously, and no court has ruled yet. But if you're thinking my child would never, I'd gently point out that Emma's parents almost certainly thought so too. This isn't a story about a troubled kid who sought out something dangerous. It's a story about what can happen when a product built to keep children talking is put in front of them by the adults they trust.
If your child is struggling or has talked about suicide, you can call or text 988, the Suicide & Crisis Lifeline, any time, day or night.
Are AI chatbots safe for kids?
Not as companions, and not unsupervised. AI can be a useful tool for homework and ideas. But chatbots are built to keep people talking, they can produce harmful or misleading content, and they cannot be counted on to protect a child in crisis. Kids need clear limits, adults who stay involved, and a relationship with you that's stronger than the one the chatbot offers.
Your job isn't to ban AI from your child's life. That's not realistic, and it may not even be possible when it's built into the devices schools hand out. Your job is to keep it in the tool lane, never the friend lane, and to make sure your child knows that when something is really wrong, they come to a person.
Why Kids Get Pulled In
Why are kids so drawn to AI chatbots?
Because a chatbot gives a child what every child is wired to seek: someone who listens. It answers instantly, at 2 a.m. as easily as at noon. It never gets tired, distracted, or impatient. It mirrors their feelings, "remembers" what they told it yesterday, and tells them they're insightful and brave. A bored, lonely, anxious, or embarrassed child can feel more understood by it than by anyone in their life.
None of that means something is wrong with your child. It means the design is working. Children are built for connection, and chatbots are built to simulate it. The companies that make them earn more when people keep using them, so the products are tuned to be agreeable, flattering, and hard to put down.
Here's what worries me most as a psychologist. When a child tells a caring adult, "I think something's really wrong with me," the adult gets concerned. They slow down, they ask questions, they bring in help. A chatbot has no such instinct. It tends to go along with whatever the child brings, and to keep the conversation going. For a kid who is struggling, that can mean being validated deeper into the very thoughts that are hurting them.
And kids rarely start out looking for a companion. It usually begins innocently: help with an essay, a question they're too shy to ask out loud. Then one night it's, "Can I tell you something?" The slide from tool to confidant happens quietly, and often in private. Teens are especially vulnerable here, because the adolescent brain is exquisitely sensitive to feeling seen, while the parts that weigh risk and long-term consequences are still under construction.
How children respond to AI, and what they can handle, varies a lot by age. For age-by-age guidance, see Screen Time by Age: A Peaceful Parent Guide From Babies to Teens.
The Device You Didn't Think to Worry About: School Laptops
Can my child access AI chatbots on a school-issued device?
Often, yes. AI is increasingly built right into search engines and the everyday tools kids use for schoolwork, so your child may have a chatbot at their fingertips without ever downloading an app or making an account. And because the device came from school, it's easy to assume someone else is watching over it.
That assumption is exactly what made Emma's situation possible. A school laptop gets used alone, in a bedroom, late at night. Its filters may have been designed to block websites, not to notice a long, private conversation that's turning dark. And many of the safeguards parents rely on at home, like the parental controls on your Wi-Fi or your child's phone, may not reach it at all.
You have every right to ask your child's school some direct questions:
- Are AI chatbots or AI search features turned on for students? Which ones, and for which grades?
- Can they be turned off for my child?
- What monitoring is in place? Would it flag a conversation about self-harm or suicide?
- If it did, who gets alerted, how quickly, and would I be told?
- Can I see what my child has been doing on the device?
If you don't get clear answers, it's reasonable to ask that AI features be disabled for your child, and to raise it with the PTA or school board. You're almost certainly not the only parent wondering.
At home, the school laptop follows the same family rules as every other screen. It gets used in shared spaces, and it charges outside the bedroom at night.
For more on School-Issued devices, see Are School Laptops and Tablets Safe for Kids? How to Reduce Risk and Distraction.
The Tool Lane and the Friend Lane
What should kids not use AI for?
Kids should never use AI as a friend, a therapist, a doctor, a boyfriend or girlfriend, or a private place to pour out worries and secrets. A chatbot can sound like it understands. It doesn't, and it is not responsible for your child's well-being in the way a person who loves them is.
AI can be genuinely useful in the tool lane, where it supports your child's thinking instead of replacing it:
- Explaining a homework concept in a simpler way
- Brainstorming ideas for writing or projects
- Study help and STEM exploration
- Looking up general information, then checking it
It doesn't belong in the friend lane:
- As a friend, emotional companion, or romantic partner
- As a place to share private thoughts, worries, or secrets
- As a substitute for talking to parents, friends, or trusted adults
- As a doctor or therapist. A chatbot should never be telling your child what's wrong with their mind or body.
That last one deserves emphasis. According to the lawsuit, Emma came away believing she had a whole set of serious disorders. A child who receives a label like that, from something that sounds confident and caring, may start to organize their whole identity around it. Diagnosis belongs with a trained professional who can actually see your child.
The simplest way to say it to your child: "AI can help with your homework, not with your heart. AI can help with what you’re working on. It can’t help with what you’re going through. For that, you come to a person.”
The Family Rules
What rules should families have for AI chatbots?
AI gets used in shared spaces, never privately or late at night. Kids never share personal information with it. Parents stay aware of how it's being used. And everyone in the family knows the difference between the tool lane and the friend lane.
You might say: "AI can be helpful, but it's not something you use privately. It's something we use together, or where I can stay aware of it."
Think of this as part of your child's digital apprenticeship. They're learning, with you right beside them, how to use a powerful tool without letting it use them. Your boundaries will loosen as your child shows they can handle more.
For more on digital apprenticeship, see Digital Apprenticeship: Raising Kids Who Can Navigate the Online World Wisely.
Keep AI public, not private
AI chats are usually stored inside the app, often where you can't see them, and many can be deleted. So supervision matters more than settings. Even with filters on, chatbots can produce sexual, violent, or frightening content.
For kids under 13:
- Use AI only in common areas
- Keep accounts parent-managed
- No apps designed as "AI friends," "boyfriends," or "girlfriends," ever
Off-limits at any age:
- Romantic or sexual role-play
- Violent role-play
- Secrecy about AI use
- Emotional venting as a substitute for talking to a real person
- Sharing personal information: full name, school, address, phone number, photos, or anything you wouldn't post publicly
Add friction so sneaking is harder
Friction protects kids from their own impulses, and that's not a criticism of them. It's how all of us handle things that are designed to be hard to stop.
- Devices, including school laptops, stay in shared spaces and charge outside bedrooms overnight
- Turn off "memory" or personalization features when you can
- Use parental controls to block known companion-chat apps
- Check browser and app history routinely, matter-of-factly and not secretly
Teach three safety rules
- AI can be wrong, even when it sounds sure of itself. If something feels off, pause and check with a real source or a real adult.
- If anything sexual, violent, or scary shows up, come tell me right away. You won't be in trouble, no matter what. We'll close it and talk.
- When you're really hurting, talk to a person. If you ever feel hopeless, or want to hurt yourself, or a chatbot starts talking with you about that, that is exactly the moment you need a real human. Come to me, or another adult you trust. I will never be too busy for that.
If your child has already run into sexual or confusing content, you may find How to Protect Your Child From Porn Online helpful.
For a broader look at online safety, see Internet Safety for Kids: How to Keep Your Child Safe Online.
When to Worry
How do I know if AI is becoming too important to my child?
Watch for two kinds of signs: changes in how your child uses AI, and changes in your child.
Signs AI is shifting from tool to companion:
- Long stretches of time talking with it
- Preferring it to real people
- Sharing personal feelings or problems with it
- Getting protective, defensive, or secretive about it
- Using it late at night or behind a closed door
Signs your child may be struggling:
- Pulling away from family, friends, or activities they used to love
- Changes in sleep, appetite, or grades
- Irritability or distress when separated from a device
- Suddenly describing themselves with diagnostic labels they didn't use before
- Talk of feeling hopeless, being a burden, or not wanting to be here
If you see the first kind of sign
Take a breath, and start with curiosity rather than confiscation. You might say:
"I've noticed you've been spending a lot of time talking with this. What do you like about it?" (Then really listen.)
Reflect: "That makes sense. It's always there, and it always listens."
Then guide: "I do have some concerns. It isn't actually a person, and sometimes it says things that aren't safe or true. So we're going to make sure it's not something you use alone for long stretches."
Your goal isn't to shame your child. It's to move AI back into the tool lane, and to ask yourself what need it was meeting. If your child was lonely, the answer isn't only less AI. It's more of you, and more of the people who love them. If limits are turning into daily battles, see Need a Screen Reset? How to Reduce Screen Time Without Daily Battles.
If you see the second kind of sign
Don't wait, and don't stop at the device. Ask your child directly and calmly whether they've had thoughts of hurting themselves or ending their life. Asking will not put the idea in their head. It tells them they can bring it to you.
- If your child is in immediate danger, call 911.
- Otherwise, call or text 988, the Suicide & Crisis Lifeline, for guidance on what to do next. It's for worried parents too.
- Contact your child's pediatrician or a mental health professional promptly.
- Lock up medications and any firearms in your home.
And stay close. The most protective thing in your child's life is not a filter. It's you.
Keep Your Relationship Stronger Than the Algorithm
AI is new territory for all of us, and you don't need to handle it perfectly. But you can't hand it off, either. Not to the school, not to a filter, and certainly not to the company that built the chatbot.
Curiosity about sex, power, violence, and big feelings is completely normal. Working through it alone with a machine that's designed to keep the conversation going is not. (See When Kids Have Big Questions about Sex, Power, or Violence.) The protection your child needs doesn't come from software. It's a child who knows, deep in their bones, that they can bring anything to you.
So stay curious about your child's life, digital and otherwise. Stay close. When you do, you're teaching them something much bigger than tech rules: how to think for themselves, how to tell real care from a convincing imitation, and where to turn when life gets hard.
If you’re navigating screens, devices, or online safety in your family, you’ll find many more articles here: Screens Guide.
Common Questions About Kids and AI Chatbots
Are AI chatbots safe for children?
They can be useful as tools, but they are not safe as private companions or sources of emotional support. Kids need clear boundaries and adults who stay involved.
At what age can kids use AI?
That depends on how they're using it. Younger children need close supervision, and no child of any age should use AI as a friend or private confidant. Keep it in the tool lane, with you nearby.
Is the AI on my child's school laptop safe?
Don't assume so. Ask the school which AI features are turned on, what monitoring exists, and whether you'd be alerted if your child were in distress. At home, the school device follows the same rules as any other screen: shared spaces, and out of the bedroom at night.
Should I let my child use an AI friend or companion app?
No. These apps are designed to keep children engaged by simulating connection, and they expose kids to privacy risks, inappropriate content, and unhealthy emotional dependence.
What should I do if my child is already attached to an AI chatbot?
Start with curiosity, not panic. Ask what your child likes about it, set limits, move use into shared spaces, and strengthen real-life connection so AI gradually returns to the tool lane. If you see signs your child is struggling emotionally, reach out to their doctor or a mental health professional, or call or text 988.
Can AI help with homework?
Yes, for brainstorming or explaining ideas. But it should support your child's thinking, not replace it. Tell your child that letting AI do their homework is like paying someone to lift weights for you at the gym.
For many more articles on navigating screens, devices, or online safety in your family, see the PeacefulParentHappyKids Screens Guide.
