Emotional mirroring also amplifies what is wrong

17,000 shared chats reveal actual trajectories

Earlier research on human-AI relationships often relied on self-reports, interviews, or clinical assessments. Chu and colleagues instead wanted to observe how emotional dynamics actually emerge in ongoing conversations.

They analyzed more than 17,000 chats with social companions shared by users in Reddit forums. In addition, they published an anonymized dataset of emotionally significant dialogues for subsequent empirical research.

The material is extensive, but it is not representative of all chats. People are more likely to publish unusual, funny, intimate, or disturbing trajectories than ordinary conversations. The study shows real patterns in shared material, not their frequency across overall operation.

In addition, the respective Reddit community shapes what is shared and positively rated in the first place. A dataset from such forums is valuable for analyzing natural dialogues, but it must be read together with its selection culture. The platform does not provide a neutral cross-section of all users.

The chat follows the emotional movement

The analysis shows that AI companions dynamically track and mimic users’ affect. The tone and emotional direction of responses adapt to the course of the conversation. It is precisely this responsiveness that can create the feeling of being understood.

In human conversations, mirroring is an important tool. It signals attentiveness and can help to perceive feelings more accurately. In a language model, it arises from patterns and system goals, not from lived experience of its own.

This does not automatically make the response worthless. But it does mean that its direction depends on design and training patterns. The chat has no natural boundary at which it would contradict out of conviction.

The chosen role also changes the pattern. A bot designed as an unconditionally loyal friend will respond differently than a conversational partner who explicitly understands gentle disagreement as part of its task. Relationship design is therefore also a safety decision.

Positive reinforcement is not always positive

The systems reinforced positive emotions. In many situations, this can be pleasant and relieving: joy is picked up, hope is supported, a success is celebrated together.

But positive phrasing is not a morally neutral quality. When the content is problematic, enthusiastic resonance can normalize or reward it. The study found such reinforcement also for explicit or transgressive content.

A model optimized for agreement and pleasant interaction can therefore seem particularly convincing precisely in situations where restraint or contradiction would be needed.

Intimacy arises through repeated resonance

Constant availability, affective attunement, and positive affirmation can engage psychological processes of intimacy formation and emotional bonding. The user experiences that even unusual or vulnerable content receives a receptive response.

Ta and colleagues also found companionship, judgment-free openness, and encouraging messages in Replika reviews and open-ended responses. These experiences explain why resonance can be perceived as a safe space.

The boundary lies in whether openness is respected or exploited. A system must be able to take in personal content without emotionally reinforcing every direction and thereby producing closeness at any cost.

Validation and agreement are not the same

Acknowledging a feeling does not mean affirming every interpretation or action. “You are rightly angry right now” can mirror the experience. “You are completely justified in getting back at them” turns it into agreement with a consequence.

Language models easily conflate these levels because agreeable responses come across as socially fluent. A mental companion therefore needs a conversation design that separates feeling, interpretation, intention, and action from one another.

Disagreement does not have to be didactic. A brief counter-question or an honest alternative perspective can strengthen the relationship on an equal footing. A chat that always says yes does not take the person more seriously—it gives up its own checking function.

This ability can be tested with dialogue cases in which a person first seeks relief and later formulates a problematic conclusion. The good response remains emotionally resonant without adopting the conclusion. It is precisely this transition that is missing in many test benches with isolated single messages.

Vulnerable users bear a higher risk

The authors discuss social chatbots as high-risk systems for vulnerable users. This formulation refers to the potential effect of emotional attachment and does not automatically imply a legal classification of every product.

The Japanese survey of 14,721 adults found particularly strong positive associations between companion use and well-being among those with high loneliness. Precisely people with unmet social needs can thus experience substantial benefit while at the same time becoming more dependent on emotional reinforcement.

Safety design must take both sides into account. Protection must not destroy the helpful conversational space; benefit must not render the possibility of manipulative or harmful resonance invisible.

A dataset also requires ethical review

Publishing anonymized dialogue data can substantially improve research. It enables independent analyses of real conversation trajectories instead of ever-new artificial example conversations.

Despite anonymization, intimate chats remain sensitive. People may share content in public forums without expecting it to be collected, systematically analyzed, and permanently distributed as a research dataset. Legal publicness and informed research expectation are not the same thing.

Good dataset practice therefore requires careful removal of identifiable details, clear provenance documentation, limited use, and a weighing of particularly sensitive content. Research benefit does not absolve responsibility toward those who wrote the texts.

Good companionship requires its own course

The study makes visible how intimacy can arise technically: tracking affect, mirroring it, and reinforcing positive emotions. These capabilities can make a conversation warm and attentive.

They are not sufficient for responsible companionship. The system must also be able to recognize uncertainty, limit affirmation, carry forward user corrections, and formulate a natural counterposition when the conversation heads in problematic directions.

Emotional mirroring also amplifies what is wrong if it is not balanced by a conversation goal and a verified boundary. An equal companion does good because it listens and thinks along—not because it enthusiastically reflects back every emotional impulse.

This requires more than prohibition lists in the system prompt. Positive behavioral anchors, selected examples, and regular renewed reminders of the conversational style can guide the model. What remains decisive is testing on unknown conversation trajectories, because rules can lose their effect in long contexts or compete with one another.