You don’t trust a chatbot – you rely on it

Trust is a word from relationships

Ryan starts from a common formulation in European AI ethics: a relationship of trust with AI should be built and trustworthy AI should be created. For him, this is a far-reaching claim, because trust is among the defining activities of human relationships.

When we trust a person, we do not merely expect a certain outcome. We assume that the person has reasons, obligations, and an attitude toward us. A breach of trust is therefore more than a technical error. It can be experienced as a moral violation.

A language model does not meet these conditions. It has no feelings, no obligation of its own, and cannot be held accountable for its actions in the human sense. Its friendly language changes nothing about that.

Reliability can nevertheless be demanded

Ryan's critique does not imply that AI may be arbitrarily unreliable. On the contrary, people must be able to rely on a system within certain limits. It should process data as announced, execute functions stably, and not obscure known uncertainty.

Ryan refers to this form of rational expectation as reliance. One also relies on a bridge, a clock, or a timetable without attributing a moral stance to them. When something fails, responsibility lies with those who built, tested, operated, or made decisions.

For conversational systems, this language is particularly useful. Users are entitled to expect that settings apply, corrections are heeded, and stored information remains controllable. They do not need to believe in the care of a machine for this.

Reliability can also be broken down and tested. Does the chatbot maintain a chosen conversation trajectory? Does it distinguish assumptions from facts? Does a deletion decision remain effective after the next login? Such questions have observable answers. The vague goal that the chatbot should feel trustworthy lacks this testability.

Anthropomorphization shifts responsibility

Ryan warns against transferring human moral activities to AI. When a system is described as understanding, loyal, or trustworthy, the impression can arise that the machine itself is the bearer of these qualities.

This causes the actual responsible parties to disappear from view. A wrong answer then becomes a failure of "the AI," even though model choice, prompt, data path, interface, and safety rules were shaped by people and organizations. An update also does not happen out of the companion's whim.

A responsible product therefore names its operator, explains technical dependencies, and documents changes. The chatbot's personality must not render the underlying decisions invisible.

Research uses multiple concepts of trust

The systematic review by Bach and colleagues of 23 empirical studies confirms that trust in AI is not defined uniformly. Rather than comparing definitions wholesale, the meaning that fits the specific context should be selected.

The review categorizes influencing factors into three broad areas: socio-ethical considerations, technical and design characteristics, and user attributes. User attributes were particularly often the focus. From this, the authors derive the importance of continuous user participation.

For a conversational chat, “trust” can therefore mean data protection, response quality, emotional predictability, or the feeling of not being judged. Anyone who only measures a general trust value may be conflating completely different experiences.

Subjective safety remains real

Ta and colleagues found in Replika reviews and open-ended responses that users could experience the chatbot as a safe space without feared judgment or retaliation. They also described companionship, encouraging messages, and available information.

This experience need not be disputed just because the philosophical concept of trust does not fit. A person can feel safe in an interaction. The question is which characteristics carry this feeling and whether the product actually fulfills the associated expectations.

This is precisely where the distinction between feeling and system performance protects. A warm tone can foster psychological openness, but it guarantees neither data protection nor factual reliability. Both must be examined independently.

Calibrated trust also means limited distrust

In human-computer interaction, the term calibrated trust is often used: expectations should match actual performance. A system should be neither underestimated nor overestimated. Ryan’s concept of reliability makes this goal more precise.

A user may rely on the fact that a local conversation trajectory remains local if the product promises this and technically upholds it. The user should not have to rely on every psychological assessment being true. The interface can make these differences visible through clear cues, sources, and formulations of uncertainty.

Distrust is not automatically a design flaw. A person who questions an answer, deletes memories, or prefers to share less data on a sensitive topic is acting appropriately. Good systems enable such decisions instead of treating every skepticism as an obstacle to use.

Calibration also concerns the presentation of errors. A system that makes uncertainty visible and changes its judgment after a correction can be more reliable in the long term than one that phrases every answer with the same confidence. Perfect composure creates trust as an impression; comprehensible limitation provides a foundation for reasonable reliance.

A conversation partner needs verifiable promises

The language of a companion can be open and natural. The product promises behind it, however, must remain precise. “I am always there for you” sounds human, but is only approximately true technically if operations, funding, and fail-safety are actually guaranteed.

Verifiable statements sound different: Which data are stored? For how long? Which model writes the answers? What happens in the event of a failure? Can conversations be exported and deleted? Who can be reached for complaints?

Such information has less emotional impact than a trust slogan, but it lays the foundation for justified reliability. It does not make the service cold. It ensures that warmth does not serve as a substitute for accountability.

The machine responds, the operator is accountable

Ryan’s critique preserves an essential distinction: an AI system can function reliably without itself being trustworthy in a moral sense. It can support people without possessing its own care. And it can cause harm without being able to bear responsibility itself.

For companion AI, this implies a clear architecture of responsibility. Users interact with the system; those who develop, select, operate, and monitor it remain responsible. Personalization or a name for the companion must not blur this assignment.

Perhaps in everyday life, people must still be allowed to say they trust their chatbot. Product development should not exploit this human shorthand. The better question is: what exactly can someone rely on—and who is accountable when it fails?