Blog

Evaluating a Nationally Localized AI Chatbot for Personalized Primary Care Guidance: Insights from the HomeDOCtor Deployment in Slovenia

In response to the growing demand for accessible and trustworthy healthcare in regions experiencing physician shortages, Slovenia has developed an innovative AI-powered digital health assistant named HomeDOCtor. This conversational chatbot is designed to provide citizens with around-the-clock primary care guidance, offering personalized medical support grounded in national clinical protocols. Developed by the Jožef Stefan Institute in collaboration with academic and medical partners, and funded through the European Union’s Horizon Europe ChatMED project, HomeDOCtor serves as a pioneering example of how artificial intelligence can be ethically and effectively integrated into public health infrastructure. With over 143,000 Slovenians lacking a personal physician and many more facing limited access to specialized care, the chatbot addresses a critical gap in healthcare delivery by offering safe, guideline-compliant triage and self-care advice directly to users via their devices.

The core of HomeDOCtor lies in its use of a retrieval-augmented generation (RAG) architecture, which enables it to retrieve relevant clinical information from a Redis-powered vector database of curated Slovenian medical guidelines. When a user inputs a question—be it through text, image, or document—the system semantically searches its local knowledge base, incorporates relevant medical snippets, and uses large language models (LLMs) such as GPT-4o, Gemini 2.5, or open-source alternatives to generate accurate and contextually grounded responses. It uses a stateless session design that ensures no personal data are stored beyond each interaction, preserving user anonymity while still offering medically useful responses in under three seconds on average.
In testing, HomeDOCtor has shown exceptional performance. In international benchmarking using 100 standardized clinical vignettes, it achieved ≥95% Top-1 diagnostic accuracy—on par with or better than leading AI models like ChatGPT-4o. More notably, when evaluated on a set of 150 internal medicine questions from the University of Ljubljana’s medical curriculum, HomeDOCtor scored 90.7%, significantly outperforming ChatGPT-4o’s 80.7% (p = 0.0135). This improvement demonstrates the value of tailoring AI systems to national healthcare contexts. The chatbot’s responses are not only accurate but also align with local medical norms and linguistic expectations, using empathetic, patient-centered communication in multiple languages. Its design includes specific phrasing strategies to support users with cognitive or psychological vulnerabilities, further enhancing accessibility.

Despite its success, the system remains a research prototype with some limitations. It does not yet support personalized or longitudinal health tracking, primarily due to legal restrictions around data storage. Usability feedback has so far been informal, though a formal study is planned for future releases. The system’s architecture, however, is modular and scalable, making it well-suited for deployment in other countries. Efforts are already underway to adapt it for additional European healthcare environments, with attention to cultural relevance, local clinical guidelines, and regulatory compliance.
The broader implications of HomeDOCtor are significant. It represents a replicable model for safely deploying AI in healthcare, especially in countries where language-specific resources, regulatory complexity, and infrastructure limitations hinder the use of global LLMs. By showing that a nationally adapted, GDPR-compliant AI chatbot can deliver high-quality care guidance and outperform general-purpose systems in real-world use, Slovenia’s initiative offers a compelling path forward for other nations seeking to responsibly integrate AI into their healthcare systems. As Europe advances toward inclusive, equitable digital health transformation, HomeDOCtor stands as both a technological achievement and a case study in how AI can meaningfully support public health without compromising safety, trust, or privacy.

To try HomeDOCtor: https://home-doctor.ijs.si

To top