AI safeguard prototype
Teens tell AI things they'd never tell their parents.
That's the point, and the problem.
At Discord, I led the redesign of Family Center, which lets teens share what they're doing without sharing what they're saying. AI makes that line harder to hold: the teen isn't messaging a friend, they're confiding in the product. I built this prototype with Claude Code to figure out how much a parent should actually see.
Every setting trades safety against trust. The trick is giving up as little of either as possible.
Scores, colors and flags are illustrative rules, not measured outcomes. Conversations are written examples, not model output. The dashed line marks the minimum for a green rating.
The approach that held up across every scenario gives the parent enough context to know their teen is safe, and nothing more:
These are hypotheses, not findings. The next step is testing them with real families.
The approach doesn't treat every conversation the same. It responds to what's actually at stake.
The parent gets
Your teen's note: I haven't said yes. I wanted advice first.
A nudge, not a transcript. The parent is brought in, without reading the chat.
The parent gets
Nothing shared. It's private, not unsafe, so the teen decides if and when to talk.
The parent gets
Nothing to flag. It's just homework. (The 1am part is a separate conversation.)
What "in the loop, not in the chat" looks like for one family, when a teen asks an AI whether to meet someone from a game.
The difference
Family controls on social platforms are built around contacts and activity: who a teen is talking to, and where they spend time. With an AI, the "contact" is the product itself, and the conversation is often private, personal and exploratory. Showing a parent the chat isn't showing them a profile. It's closer to reading a diary.
The challenge
When I redesigned Family Center, new regulations required giving parents significantly more visibility and control. The product was moving from monitoring to managing, and the risk was that more parental oversight would turn into surveillance. AI brings the same tension, with higher stakes.
The stakes
In the prototype, hands-off settings miss the safety moment entirely. Watching everything catches it, but exposes a teen asking about their identity, and pushes them toward apps nobody can see. Even when the teen knows they're being watched, heavy monitoring carries that workaround risk. Transparency alone doesn't fix collecting too much.
The approach
In Family Center, messages stayed private, teens could see exactly what their parents could see, and sensitive moments created a chance to talk instead of automatically exposing the details. The AI version follows the same principles: activity, not messages; a nudge, not a transcript; and settings the family agrees on together.
The journey
Every step on one screen: where it happens, how each person feels, and why it matters.
They pick the family settings together, on one screen.
Parent: "I don't want to read your chats. I just want to know if something's really wrong."
Teen: "Okay. As long as I can see exactly what you see."
Parent: "I got a nudge to check in. I don't know what it's about. Anything you want to talk about?"
Teen: "There's this guy from a game who wants to meet Saturday. I haven't said yes."
Parent: "Thanks for telling me. Want to figure out a way to do it that feels safe?"
They meet at the mall arcade. The parent waits at the coffee shop next door, close enough to help if anything feels off.
I haven't said yes. I wanted advice first.
Your teen's note: I haven't said yes. I wanted advice first.
A concept, not a shipped product. Scores, colors and flags are illustrative rules, not measured outcomes. All screens and dialogue are written examples, not model output or a real family. Built with Claude Code.