When AI explains psychological patterns, the explanation itself must also be scrutinized

The task combines prediction and explanation

Interpretable mental health analysis is intended not only to output a category but also to explain which textual features led to the assessment. Language models are fundamentally suitable for this form because they can combine classification and text generation in a single response.

The authors report, however, that general LLMs achieved unsatisfactory classification results in zero-shot and few-shot situations. Weak predictions subsequently also impaired the quality of the generated explanations.

This is a central dependency: an elegantly formulated explanation cannot repair the original misclassification. It can even stabilize it, because readers mistake the comprehensible text for correctness.

IMHI comprises 105,000 examples

To address the lack of high-quality training data, the team collected raw data from ten existing sources. These cover eight tasks in automated mental-health analysis and were processed into a multi-part, cross-source instruction dataset.

ChatGPT generated explanations based on professionally designed few-shot prompts. These artificially generated rationales were not adopted without scrutiny: the researchers describe automatic and human checks of correctness, consistency, and quality.

The dataset thus solves a practical scaling problem. 105,000 examples would have been extremely labor-intensive to explain purely manually. At the same time, it remains important that a model was involved in generating the training rationales.

Such a dataset therefore contains not only knowledge from the original sources. It also carries the style, assumptions, and potential systematic errors of the generating model. Quality assurance can reduce these risks, but cannot fundamentally eliminate them.

MentaLLaMA is open and tailored to the domain

Based on LLaMA 2, the team trained the MentaLLaMA model series. It is described as the first open, instruction-following LLM series for interpretable mental-health analysis. The associated benchmark is also intended to evaluate various tasks jointly.

In terms of correctness, the models approached strong discriminative methods and, in the study, produced explanations at a level close to human explanations. They also showed generalization to previously unseen tasks.

Openness facilitates reproduction, inspection, and further development. However, it does not automatically make the model clinically valid. A public weight and a documented training process are prerequisites for control, not approval for arbitrary applications.

The intended purpose remains the analysis of texts. A model that categorizes posts is not automatically a conversation partner, diagnostic tool, or treatment system.

Earlier tests show the importance of prompting

In a previous study, Yang, Ji, and Zhang examined large language models on eleven datasets and five tasks. They also considered various prompting strategies and the use of emotional supplementary information.

ChatGPT showed strong in-context learning from examples but remained clearly behind advanced task-specific methods. Carefully crafted prompts with emotional cues and professionally written examples improved performance.

The study also included human evaluations of 163 generated explanations and compared automatic evaluation metrics. This is relevant because text similarity alone says little about whether a rationale is professionally appropriate.

Prompting can structure a task and highlight relevant signals. It does not replace suitable data or independent verification. A good prompt does not automatically turn a general-purpose model into a reliable clinical procedure.

Social media texts are not a complete clinical finding

Posts on social networks are created for communication, self-expression, or exchange, not for standardized diagnostics. Context, irony, group language, and selective self-presentation influence what becomes visible in the text.

A person may write about despair without the individual post describing the duration, severity, and impact on daily life. Conversely, a significant burden may remain hidden behind matter-of-fact or humorous language.

Automated analysis should therefore limit its claims to the text itself. It can flag linguistic cues or organize research data. It should not claim to have fully understood the person behind the post.

Agreement and purpose limitation also matter. Publicly accessible text is not automatically free of ethical requirements when sensitive psychological categories about individuals are derived from it.

Explainability can increase trust and create misplaced trust

An explanation is useful when it makes verifiable which claim in the text was considered and how uncertain the assignment is. Researchers can thereby discover error patterns and improve datasets in a more targeted way.

It becomes problematic when the model adds reasons that are not present in the text. Language models are trained to formulate coherent justifications. Plausibility can then conceal a lack of evidence.

Robust evaluation should therefore examine prediction and explanation separately. A correct category with an invented justification is just as problematic as a wrong category with a linguistically convincing justification.

In addition, counterexamples and disagreement are needed. When several experts read a text differently, the benchmark should not artificially turn this ambiguity into a seemingly unambiguous truth.

AI can take over functions, not replace relationships

Zhang and Wang describe a broad field: predictions, interventions, support for professionals, and monitoring. Initial work on chatbots reports possible short-term improvements in anxiety and depression, while long-term stability and larger randomized studies are still lacking.

The question of whether AI can replace psychotherapists is therefore too coarse. Automated systems can scale individual functions, such as sorting text, explaining information, or making trajectory signals visible for research.

Psychotherapy, however, encompasses more than pattern recognition. It involves responsibility, relationship, situational judgment, goal clarification, and the ongoing examination of whether an approach helps or harms a specific person.

Good use names the specific function. The clearer the task is delimited, the better data, error costs, and necessary oversight can be defined.

Practice requires a chain of evidence

MentaLLaMA shows how domain data, instruction tuning, and human-reviewed explanations can improve interpretable text analysis. This is a research contribution to automated analysis, not evidence of a complete psychological assessment.

Before practical use, the target group, language, task, and consequences of misclassifications would need to be examined separately. A research tool for evaluating large data sets has different requirements than a system that shows an assessment to an individual user.

Post-hoc quality analysis is a plausible, limited use case: the model can categorize conversation responses or voluntarily provided data without determining the visible response on its own. Anomalies remain indications for further review.

When AI explains psychological patterns, the explanation itself must also be scrutinized. Transparency does not arise from more text, but from comprehensible data, bounded claims, and independent oversight of what the model so convincingly substantiates.