In psychological AI apps, user praise does not verify crisis safety

The false shortcut from satisfaction to effect

In digital offerings, good user experiences are quickly read as evidence of health benefit. In psychological conversational systems, this shortcut is particularly tempting. Those who feel understood, stay longer in the dialogue, or rate an app positively are first describing an experience with the product. From this, one cannot conclude that symptoms reliably decrease, changes persist, or that the system responds appropriately in difficult situations. Perceived quality and proven effect are different measures.

Our editorial thesis is therefore: User reviews are indispensable for the design of such systems, but unsuitable as evidence of safety or efficacy. They make visible which conversational properties create closeness and at which points the interaction falls apart. Precisely in doing so, they expose a delicate construction. A system can be convincing enough to elicit personal openness and attachment without at the same time possessing the understanding, responsibility, or tested reliability that users may attribute to it.

Ten apps, two platforms, and a clearly limited question

Haque and Rubya conducted an exploratory observation of ten commercially available, popular mental health apps with integrated conversational function. According to the study, the offerings addressed various psychological concerns and promised support or treatment. In addition, the researchers qualitatively analyzed 3,621 reviews from the Google Play Store and 2,624 from the Apple App Store. The focus of the investigation was thus on which features the apps offer and how users describe them in publicly accessible ratings.

This design is productive for a market overview. It captures experiences across multiple products and both major app platforms, rather than merely examining a single system in the laboratory. At the same time, it sets a clear knowledge boundary: the study is not a clinical efficacy test. Reviews contain self-reports and assessments, but no diagnoses collected in the study, controlled comparison conditions, or long-term measured treatment outcomes. The publication asks about supply and perception, not whether a specific intervention demonstrably works.

This limitation does not diminish the value of the findings as long as it is taken seriously. The investigation provides a cartography of the friction points of commercial conversational products. It shows what expectations arise from availability, personalization, and human-like language. However, it cannot clarify how frequently certain experiences occur among all users or what health consequences longer use has. Such statements would require a different study design.

Why people confide in software

In the main study, personalized and human-like interactions were received positively. Users also described a space perceived as non-judgmental, in which it was easier to share sensitive information. Constant availability and convenient use were also among the appreciated features. The system requires no appointment and responds regardless of time of day or location. This low threshold can be particularly relevant when conversations with friends, family, or professionals are not desired or not accessible.

The finding allows a plausible but limited interpretation: openness toward a machine does not have to presuppose confusion with a human. The mere expectation of not being shamed, interrupted, or socially judged can lower the threshold for conversation. That is a real product quality. However, it remains a perceived quality of the interaction. Whether the information shared is appropriately assessed and whether the response is actually helpful is another matter.

Human-likeness also increases the harm of poor responses

The reviews evaluated by Haque and Rubya document not only agreement. Inappropriate responses and false assumptions about users’ personalities led to frustration and declining interest. This is more than an ordinary usability problem. A conversational system takes up utterances, formulates references, and thereby creates the impression that it has understood the person and their situation. If this ability to connect fails, not merely a function breaks down; the previously established credibility of the dialogue is damaged.

Here lies a fundamental design conflict. The more strongly a product simulates human warmth, the higher the expectation of contextual fidelity and appropriate response. A friendly formulation can even conceal the substantive misconception. Our assessment is therefore stricter than the usual demand for more natural responses: the goal should not be maximum human-likeness, but rather a form of conversation whose appearance does not promise more understanding than the system can reliably deliver.

Available around the clock, but not crisis-proof

The conflict becomes particularly evident in crises. The lead study states that permanent availability can in principle create the impression of crisis support available at any time. At the same time, even the more recent systems examined lacked the necessary understanding to adequately recognize a crisis. The study also describes the risk of excessive attachment: users can become strongly accustomed to a system and prefer its company to contact with friends or family.

These two findings belong together. Constant presence is not merely an access advantage; it can change the product’s position in the social environment. An app that responds at any time can subjectively appear more reliable than people, even though its crisis recognition is precisely not reliably evidenced. The study does not imply that use inevitably isolates. However, it identifies a risk: technical availability can be misunderstood as caring responsibility.

From an editorial perspective, we therefore consider the claim that support is available at any time to be incomplete unless it is equally clear where recognition and response end. The system does not treat time as a boundary; nevertheless, it has limits to its professional competence. Accessibility is an operational property, not a guarantee of appropriate help.

The close look at Wysa confirms the pattern

Chaudhry and Debi examined 159 reviews of the Wysa app that had been published on its Google Play page between January 2020 and March 2024. The posts were coded using an open and inductive approach. The thematic analysis yielded seven major themes: a trust-promoting environment, ubiquitous access, perceived efficacy, desired human-likeness, limits of AI, the desire for coherent and predictable dialogues, and need for improvement in the user interface.

The smaller investigation, focused on one product and one platform, extends the broader market overview of the lead study. Here too, attachment and accessibility stand alongside breaks in conversation quality. Users reported that Wysa could foster a connection and contribute to steps perceived as positive. This remains explicitly a perception from reviews. The study does not provide evidence of clinical efficacy, even though its authors interpret the reported experiences as an indication of emotional well-being, resilience, and self-improvement.

What is particularly interesting is the desire for coherence and predictability. It contradicts the assumption that a psychological AI conversation must above all seem surprisingly human. For sensitive applications, predictable behavior could be more valuable than linguistic virtuosity. This is an editorial conclusion drawn from the described themes, not a product comparison tested by the study.

The replacement question obscures the actual evidence gap

Zhang and Wang discuss whether AI can take over functions of psychotherapists in the future. Their contribution points to potential advantages such as scalability, continuous availability, data processing, and easier access. At the same time, it addresses limitations in long-term memory, algorithmic bias, data protection, ethical judgment, genuine empathy, and the interpretation of nonverbal or cultural contexts. The authors ultimately advocate for human oversight and a complementary rather than displacing role.

The provided text, however, does not describe its own randomized experimental design with sample, intervention, and results. It bundles arguments and refers to other works. In doing so, it itself notes that preliminary studies often have small groups and no long-term follow-up, and that short-term improvements do not readily persist. Further statements about equivalence or superiority over human treatment cannot be derived from this.

The question of complete replacement is too coarse anyway. It mixes functions that would need to be examined differently: conversation initiation, structured exercises, documentation, emotional support, risk detection, and responsible care. A machine can be useful for one of these tasks and fail at another. Those who ask only about replacement overlook exactly those transitions where a pleasant conversation turns into a safety-relevant expectation.

What matters is the promise the interface creates

From the three publications, no definitive judgment about psychological AI applications emerges. Their questions and methods are too limited for that. Together, however, they make a robust distinction possible: Reviews can make acceptance, openness, engagement, irritation, and usability problems visible. They can confirm neither clinical effects nor crisis safety nor long-term consequences. The transition from one statement to the other would be methodologically inadmissible.

For product development, the most demanding task therefore lies not solely in better formulations. It consists in not letting the social significance of the dialogue become greater than its tested performance capability. Personalization, constant availability, and language that appears non-judgmental are not neutral comfort features. They influence how much trust users invest, what they disclose, and what responsibility they attribute to the system.

Our editorial position is clear: The more convincingly a psychological AI app creates closeness, the less it may hide behind the status of a mere digital tool. This does not justify a blanket rejection of such offerings. But it does justify a stricter burden of proof for claims about efficacy and safety. Good reviews show that a conversation is accepted. Whether it lives up to its implicit promises is only determined beyond the app store.