Analysis, not certainty: Dialogatlas separates sources, observations, and editorial conclusions. New evidence may change this assessment.
The questions covered the entire everyday treatment routine
The study was not limited to a single knowledge question. It addressed knowledge of and access to antiretroviral therapy, treatment initiation, side effects, adherence, and general questions about sexual health during treatment.
These topics correspond to different conversational functions. Some require a factual explanation, others an assessment of symptoms or support with long-term adherence. A uniform response style would be too coarse for this.
The researchers checked the responses against international HIV guidelines. This gave the evaluation a verifiable professional standard rather than only a general impression of helpfulness.
All responses were correct and comprehensive
According to the study, ChatGPT answered all the questions presented to it correctly and comprehensively. This shows that a general-purpose language model can formulate usable health information within a clearly delimited subject area.
Of particular relevance was the recognition of a possible abacavir hypersensitivity reaction. The system appropriately assessed the potentially life-threatening scenario and provided a suitable reference.
Such a result must be neither underestimated nor overgeneralized. It pertains to the specific questions posed, the system used, and the state of knowledge at the time. Other phrasings and model versions may respond differently.
Stigma changes the value of accessibility
In the context of HIV, access to information is not merely a matter of opening hours. Fear of judgment, disclosure, and discrimination can prevent people from asking questions. Costs and geographic distance add to this.
A chat lowers this social threshold. It is available at any time and requires no visible self-disclosure to another person. As a result, it can ease the first step and close gaps in knowledge.
This strength is only justifiable if the data path is genuinely protected. A supposedly discreet conversational space that unnecessarily stores or repurposes sensitive content undermines precisely the advantage it promises.
Accessibility also does not mean that every person has the same information needs. Some may initially want to clarify a term, others to assess a possible side effect or to formulate a question for their next appointment. A good service must recognize these differing intentions without hastily constructing a medical situation from a brief message.
Regional rules limit global answers
For certain geographic contexts, the answers lacked sufficient specificity. Access to medications, treatment recommendations, care pathways, and available preparations differ between countries and regions.
International guidelines form an important foundation but do not replace local care practice. A model can formulate a globally plausible answer and still name an inappropriate option for the user’s place of residence.
Regional adaptation requires verified, current sources and recognizable validity. Merely asking for the location is not enough if the system subsequently invents local details from general patterns.
For product design, this implies a clear separation: general knowledge may be explained as such; local information requires dated, verifiable sources. If this basis is missing, an openly stated limitation is more reliable than a seemingly precise answer. Especially regarding medication access and care pathways, accuracy without provenance is not sufficient evidence of quality.
Pregnancy requires more than standard knowledge
For pregnant individuals, the general answer could also be insufficient. Pregnancy alters medical considerations and requires an individual assessment of therapy, timing, and other circumstances.
A language model may name specific features and prepare for targeted counseling. However, it cannot reconstruct a complete clinical context from a few chat details or replace the necessary examination.
This is precisely where uncertainty must remain visible. A confidently personalized response seems more helpful, but it can feign a precision that the system does not possess.
Other counseling studies show the same limitation
The snakebite study by Altamimi and colleagues also found understandable and informative responses, as well as an appropriate emphasis on professional care. Limitations included outdated knowledge, lack of personalization, and regional differences.
In a simulated rhinoplasty consultation, ChatGPT responded coherently and in an easily understandable manner, but it could not provide detailed individual counseling. Both studies used nine hypothetical questions and expert evaluations.
Across different topics, a consistent picture emerges: general information can work well. The more an answer depends on location, examination, and individual circumstances, the less the language model alone suffices.
Referral should be part of a pathway
In the HIV study, ChatGPT consistently referred to professional help for targeted counseling. This is an important limitation, but a general note at the end of each response is not enough.
A good chat can collect specific questions for the next appointment, explain technical terms, and help describe side effects or uncertainties in an organized manner. This turns the referral into a seamless transition.
At the same time, it should accept if a person initially only wants information. The bridge to care must not come across as a condescending interruption, but rather as an additional option alongside the current conversation.
To this end, the system must keep the conversation and the decision separate. It can help with sorting, answer comprehension questions, and improve the user’s own preparation. As soon as a recommendation depends on diagnosis, examination, pregnancy, interactions, or regional availability, its role changes. Then it should not sound more persuasive, but rather make clear the missing basis.
The right role is supplementation with clear responsibility
The study sees ChatGPT as a possible addition to ART counseling. Better knowledge could support access, adherence to therapy, and treatment outcomes. However, these potential downstream effects were not directly demonstrated in the study described.
For a real service, documented model and prompt versions, current guidelines, repeated tests, data protection, and clearly named responsible parties are needed. A good answer in a study does not replace this operation.
When stigma impedes access, a chat can open up information. Its strength lies in discreet, understandable preparation. The individual medical decision remains where examination, local care, and professional responsibility come together.
Success should therefore not be measured solely by how complete a single answer sounds. More important is whether people are better informed afterward, can ask relevant questions, and do not receive false reassurance. Such effects would need to be examined in longer, real-world usage situations. The present study provides a well-founded starting point for this, but not yet evidence of efficacy.
Sources & further reading
- Matthew Chung Yi Koh, Jinghao Nicholas Ngiam, Joy Yong (2024): The role of an artificial intelligence model in antiretroviral therapy counselling and advice for people living with HIV
- Ibraheem Altamimi, Abdullah Altamimi, Abdullah S Alhumimidi (2023): Snakebite Advice and Counseling From Artificial Intelligence: An Acute Venomous Snakebite Consultation With ChatGPT
- Yi Xie, Ishith Seth, David J. Hunter‐Smith (2023): Aesthetic Surgery Advice and Counseling from Artificial Intelligence: A Rhinoplasty Consultation with ChatGPT