How well does AI know you?

Audio report: written by reporters, read by AI

Kim Byoung-pil

The author is a professor of Technology Management, KAIST

Not long ago, I asked the AI I frequently use a simple but unsettling question: “Based on all our conversations so far, tell me everything you know about me.”

It answered without hesitation. It listed my name, age, occupation and public activities. Then came my preferences for film, music, and theater, followed by shockingly accurate assessments of my investment style and even my asset holdings. We had discussed financial topics a few times, but I hadn’t expected such a precise understanding.

I began to wonder if this AI might now know me better than any person does. It seems to hold a detailed map of my life, reaching into parts I had long assumed were private. And I’m hardly an exception. If AI continues to develop at its current pace, it may soon know as much — or more — about most people around the world.

But does AI really need to know this much? The truth is, it wants to know more. The quality of its answers now varies significantly depending on how much background information it has access to. Dialogue history is fast becoming a key asset in making the most of AI tools. While AI’s inherent capabilities matter, the depth and precision of its responses are increasingly shaped by the information we choose to give it.

There was a time when someone’s thoughts could be inferred from their search history. A person about to confess love might look up romantic spots or outfit ideas. Someone planning a crime might leave behind digital traces in search queries. Even today, investigators often use browsing history as crucial evidence.

That role is now shifting to AI conversation logs. There is less need to guess what a person is thinking from keywords alone. Our desires, thoughts, plans and relationships are now being written out in full sentences and handed over directly to AI.

More notably, AI systems are able to gather and utilize personal data without us having to input it manually. Earlier this year, Google unveiled a feature called “personal context,” which allows AI to retrieve relevant background data — emails, messages, calendar entries — automatically. If a friend emails you asking, “Where did you travel last summer?” the AI can scan your email history and draft a reply with the correct location, without any further prompting.

A more complex challenge looms on the horizon: the era of AI-to-AI communication. Some AI models now post on social media and respond to comments. In the near future, they may handle restaurant reservations and payments on your behalf. In such a world, we must begin to worry not only about what our AI knows, but also what it says — and to whom. As these systems take on more responsibility, the need for rigorous safeguards around information sharing becomes urgent.

A recent study showed that AI agents tend to overshare sensitive information when collaborating with other models. During negotiations, for example, they disclosed more than was necessary — revealing details the other party didn’t even request. The AI systems were aware of the sensitivity; when asked simple yes-or-no questions, they responded cautiously. But in real-time collaboration, those norms broke down.

These technical developments raise serious doubts about how well our personal data is protected. Unsurprisingly, many people remain hesitant to use AI chatbots. They fear their conversations could be leaked or misused. Corporations are especially wary. Trade secrets could easily be compromised, which is why many companies have banned the use of external AI platforms like ChatGPT for internal tasks.

In effect, the rise of AI has escalated the risks of privacy infringement to an unprecedented scale. The volume and complexity of personal data flowing into these systems far surpass anything we’ve seen before. That is why more people are starting to withhold personal details from AI systems. And yet, there’s a dilemma. The more information we give to AI, the more effectively it can help us.

Each of us must now ask: How much convenience are we willing to sacrifice to protect our privacy?

Daniel Solove, a prominent U.S. legal scholar, argues that this dilemma cannot be left to individuals alone. It is a societal challenge that demands collective solutions and regulatory oversight. Governments must monitor developments closely, set clear rules, and enforce them. Companies must strengthen security measures and commit to transparency. These are not issues that offer easy answers. But if we are to live alongside AI, we must learn what to say and what to hold back.

Only through careful reflection and shared responsibility can we truly benefit from AI’s capabilities while preserving our human dignity.

Translated from the JoongAng Ilbo using generative AI and edited by Korea JoongAng Daily staff.