Researchers from Karabuk University and Antalya Oral and Dental Health Hospital assess ChatGPT 3.5 and Google Gemini performance in addressing parent queries on pediatric dental trauma. They employ the DISCERN instrument and PEMAT-P tool to evaluate response quality, understandability, and actionability. Both chatbots deliver comparable guidance, with Gemini showing marginally higher reliability and ChatGPT demonstrating superior clarity, yet neither system substitutes professional dental consultation.

Key points

  • ChatGPT 3.5 and Google Gemini are evaluated using the DISCERN instrument, with Gemini achieving marginally higher mean reliability scores.
  • PEMAT-P analysis shows ChatGPT delivers superior understandability and both chatbots provide similar actionability for pediatric dental trauma guidance.
  • Study uses 17 IADT-based case scenarios with inter-rater Cohen’s kappa of 0.72–0.78 and parametric statistical tests to compare chatbot performance.

Why it matters: This study validates AI chatbots as accessible, consistent sources of pediatric dental trauma guidance, heralding scalable support alongside clinical expertise.

Q&A

  • What is the DISCERN instrument?
  • How does PEMAT-P measure actionability?
  • Why can’t AI chatbots replace dentists?
  • What factors influence chatbot reliability?
  • How were the case scenarios designed?
Copy link
Facebook X LinkedIn WhatsApp
Share post via...


Read full article

AI Chatbots in Healthcare

AI chatbots are computer programs that use artificial intelligence to simulate human-like conversations. They process natural language inputs to provide information, answer questions, and guide users through complex topics. In healthcare, chatbots can offer instant responses to patient queries, deliver educational content, and support decision-making when medical professionals are not immediately available.

Key Features of AI Chatbots
  • Natural Language Processing (NLP): Enables chatbots to understand and generate human language effectively.
  • Machine Learning Models: Underlying algorithms, such as transformer-based architectures, trained on large datasets.
  • 24/7 Accessibility: Provides users with round-the-clock access to basic medical information and guidance.
  • Scalability: Can handle multiple concurrent interactions without additional staffing.
Benefits in Patient Education
  • Clarity: Presents complex medical concepts in simple, conversational language.
  • Consistency: Delivers standardized information based on up-to-date guidelines.
  • Engagement: Encourages users to ask follow-up questions and explore topics interactively.
  • Resource Efficiency: Frees healthcare professionals to focus on complex cases by managing routine inquiries.

Evaluating Health Information Quality

To ensure patients receive reliable guidance, educational materials and chatbot responses must be evaluated using standardized tools. Two widely used instruments are the DISCERN questionnaire and the PEMAT-P assessment.

DISCERN Instrument

The DISCERN instrument is a 16-item questionnaire designed to judge the trustworthiness and quality of written health information, especially treatment choices. It is divided into three sections: reliability, details of treatment options, and overall quality. Each question is scored from 1 (low quality) to 5 (high quality), allowing a total score that classifies information as very poor to excellent.

PEMAT-P Tool

The Patient Education Materials Assessment Tool for Printed Materials (PEMAT-P) measures two aspects: understandability and actionability. Understandability assesses whether users can process and explain the information. Actionability evaluates if users know exactly what steps to take. Scoring involves rating each item as agree, disagree, or not applicable, resulting in separate percentage scores for both domains.

Artificial intelligence in pediatric dental trauma: do artificial intelligence chatbots address parental concerns effectively?