Clinics and Practice · Published 2026-07-31 · DOI 10.3390/clinpract16080142
Keisuke Seki, Minori Kashima, Taiki Akiyama, Atsushi Kobayashi, Ko Dezawa, Yoshimasa Takeuchi, Mika Furuchi, Atsushi Kamimoto
<b>Background/Objectives:</b> The mandibular cortical index (MCI) is a valuable screening tool for osteoporosis on dental panoramic radiographs, but its assessment is subject to inter-examiner variability. This study evaluated the reproducibility and inter-rater agreement of MCI classification by a closed-source generative AI tool (NotebookLM, Google) compared with eight dentists of varying clinical experience. <b>Methods:</b> One hundred panoramic radiographs were classified according to the three-category MCI in two sessions held at least two weeks apart. Intra-examiner reliability, inter-examiner agreement, and agreement with a reference radiologist were assessed using linearly weighted kappa coefficients. The study was designed as a descriptive reliability study rather than a formal equivalence trial. <b>Results:</b> The intra-examiner reliability of the AI was exceptionally high (κ = 0.987). However, agreement between the AI and the dentists remained at “slight agreement” or lower (κ < 0.2) for every pairing, with 95% confidence intervals that included zero; no formal global hypothesis test was performed, and these individual interval estimates should not be interpreted as proof of the absence of agreement beyond chance. A “two-level discrepancy,” in which the AI interchanged Class 1 (normal) and Class 3 (severe), occurred in 10–18% of cases. The dentists showed a possible learning effect, with inter-examiner agreement improving between sessions. <b>Conclusions:</b> Despite the high reproducibility of the NotebookLM configuration evaluated in this study, agreement with the dentists remained at “slight” or lower (κ < 0.2) in MCI classification. As classifications were not validated against bone mineral density or an adjudicated reference standard, these findings concern reproducibility and agreement rather than diagnostic or screening performance and characterize a single LLM-based platform rather than generative AI in general.
Abstract from DOAJ. Public domain (CC0 1.0).
Read the article at the publisher →
Seki, K., Kashima, M., Akiyama, T., et al. (2026). Reproducibility of Mandibular Cortical Index Classification Among Dental Examiners and a Single Generative AI Platform: An Observer Agreement Study. Clinics and Practice. https://doi.org/10.3390/clinpract16080142