Project Lily: humans read ChatGPT chats. Check your tier

Somebody on your team has pasted a customer email into ChatGPT this month. Project Lily is the reason that matters. 404 Media obtained internal materials showing OpenAI hired hundreds of contractors to read real ChatGPT conversations — not synthetic test prompts, actual user chats — to grade responses and tune the model. OpenAI strips personal information first. Sensitive details still get through. The question this puts in front of every operator is simple: which tier is your team actually typing into?

What actually happened

404 Media's report describes a contractor program built on real user submissions. Reviewers see prompts and, in cases, whole conversations. Usernames are not attached and OpenAI attempts to remove personal information before review, but the reporting is direct that sensitive details survive the scrub.

The work itself is ordinary model training: contractors summarize what the user was trying to do and rate the responses, with specific pushes to stop the model from anthropomorphizing itself and to cut excessive agreeableness. That is how you get a model that stops telling everyone their idea is great.

Two details are worth keeping. First, scale — ChatGPT has more than 900 million users, and a contractor quoted in the piece said plainly that users would not imagine someone somewhere is reading their conversations. Second, Anthropic confirmed to 404 Media that it also uses human review to improve its models. This is not one vendor's shortcut. Human review is how frontier models get built.

Why it matters for your business

Human review is not a scandal on its own. It is the unremarkable back half of RLHF, and anyone who has read a model card knew it happened. The exposure is not that humans read chats. It is that consumer tiers and business tiers have different terms, and your staff does not know which one they are using.

Find out what your team is on. Free and personal ChatGPT accounts, business workspaces, and direct API access sit under different data and training terms. A team that adopted AI organically is on personal logins, and personal logins are the tier where this reporting lands hardest. This is a fifteen-minute audit, not a project.

Write the one rule people can remember. Not a policy document. One sentence: customer names, contracts, health details, and credentials do not go into a chat window. Then make the compliant path easier than the non-compliant one — give people a workspace account so the answer to "where do I put this" is not their personal login.

Then put the real work behind an API. If a workflow touches customer records regularly — support triage, document summarizing, order lookups — it should not live in a chat window at all. It should be an integration where you set the data boundary in a contract, retention is a setting you chose, and the audit log belongs to you. That is a build, and it is a small one.

Sources: 404 Media, Tom's Hardware.