Good evaluation to warrant better digital health

mibe000293 10.3205/mibe000293 urn:nbn:de:0183-mibe0002936 Editorial Good evaluation to warrant better digital health Gute Evaluation für eine bessere digitale Versorgung Hübner Hübner Ursula U Prof. Dr.

Hochschule Osnabrück University of Applied Sciences, Albrechtstr. 30, 49076 Osnabrück, GermanyResearch Center for Health and Social Informatics, Hochschule Osnabrück University of Applied Sciences, Osnabrück, Germany

u.huebner@hs-osnabrueck.de author German Medical Science GMS Publishing House

Düsseldorf

610 EFMI STC 2025 20251017 engl This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 License. Dieser Artikel ist ein Open-Access-Artikel und steht unter den Lizenzbedingungen der Creative Commons Attribution 4.0 License (Namensnennung). 1860-9171 21 GMS Medizinische Informatik, Biometrie und Epidemiologie GMS Med Inform Biom Epidemiol 21 EditorialThis special issue of MIBE comprises contributions submitted to the Special Topic Conference “Good Evaluation – Better Digital Health” of the European Federation of Medical Informatics (EFMI). EFMI Special Topic Conferences (STC) are scientific events focussing on a dedicated topic in biomedical and health informatics and are addressing particularly young and early career scientists. This year, the STC is hosted by the German Association of Medical Informatics, Biometry and Epidemiology (GMDS</PlainText></TextGroup>) and organised by Osnabrück University of Applied Sciences. Publishing contributions to a European conference illustrates MIBE’s general offer to colleagues from Europe and beyond to publish their work in this journal. As a first step, posters submitted as a three-page poster contribution to the STC were considered for publication and underwent the same rigorous peer review process as all contributions to the conference before they were proposed for publication in MIBE. Here they were peer reviewed again. </Pgraph><Pgraph>Nine out of 20 poster contributions submitted were accepted and published in this special issue covering topics of the theme “Good Evaluation – Better Digital Health”. Evaluation is a centre piece of the software and systems life cycle and starts – in contrast to usual anticipations – not at the end when the system has been rolled out but at the very beginning of the development process. It thus allows the integration of empirical patterns against which the testing can be performed such as user expectations and requirements. The articles “Co-creating a cancer screening dashboard with screening invitees and experts“ by Oldhoff-Nuijsink et al. <TextLink reference="1"></TextLink> and “Building a digital platform for collaborative second opinions in rare disease: Integrating AI and healthcare networks for improved care“ by Lima et al. <TextLink reference="2"></TextLink> illustrate the meaningfulness of early evaluations. Conversely and even a step earlier, evaluations of patients’ behaviour can stimulate the needs to foster long-term medication adherence through digital methods as was shown by Kim et al. in their study “Adherence to hormonal therapy in breast cancer patients: EHR-based retrospective data analysis” <TextLink reference="3"></TextLink>.</Pgraph><Pgraph>Evaluations and tests that are derived from frameworks, theories and concepts have a different background, however, need not contradict more empirically grounded evaluation targets. It is therefore useful to identify new frameworks as in the case of Hülsmann et al. in “Identifying assessment frameworks for digital public health interventions: First results of a scoping review“ <TextLink reference="4"></TextLink> or to augment existing ones as was proposed by Grashof et al. in “A new perspective on eHealth acceptance: Combining health-related factors with the Technology Acceptance Model“ <TextLink reference="5"></TextLink>.</Pgraph><Pgraph>The emergence of more and more AI models for use in healthcare increases the necessity to conduct proper evaluation studies before such models can be integrated into systems for clinical use. The study by Galland-Decker et al. “Performance evaluation of Meditron3-70B in medical coding: Current limitations and integration perspectives for clinical practice” <TextLink reference="6"></TextLink> investigated the performance of a medical large language model to generate SNOMED and ICD codes from vignettes and hinted at current flaws. </Pgraph><Pgraph>Evaluating a system often means evaluating the context in which the system is embedded in. This also applies to AI systems and their rootedness in models that have been developed on a specific set of data to be applied to fresh clinical data. The study by Slob et al. “FAIVOR – a push-button system for AI validation within the hospital“ <TextLink reference="7"></TextLink> describes a tool that serves as an AI model repository and allows the evaluation of a given model on local data. </Pgraph><Pgraph>Evaluating applications and systems is often accompanied by educational measures to really obtain better digital health which means nothing less than better use from better digital systems. Two articles address education and training: “Developing competencies in health informatics: Blended teaching method “by Mannevaara et al. <TextLink reference="8"></TextLink> and “AI and AR for inclusive health education“ by Focsa <TextLink reference="9"></TextLink>. </Pgraph><Pgraph>The studies presented in this special issue give rise to longitudinal measures to evaluate the output, outcome and impact of digital systems to finally appraise whether an application really contributed to better digital health.</Pgraph></TextBlock> <TextBlock name="Notes" linked="yes"> <MainHeadline>Notes</MainHeadline><SubHeadline>Competing interests</SubHeadline><Pgraph>The author declares that she has no competing interests.</Pgraph></TextBlock> <References linked="yes"> <Reference refNo="1"> <RefAuthor>Oldhoff-Nuijsink C</RefAuthor> <RefAuthor>Buitendijk F</RefAuthor> <RefAuthor>Rolink M</RefAuthor> <RefAuthor>Derksen ME</RefAuthor> <RefAuthor>Peute LWP</RefAuthor> <RefAuthor>Fransen MP</RefAuthor> <RefTitle>Co-creating a cancer screening dashboard with screening invitees and experts</RefTitle> <RefYear>2025</RefYear> <RefJournal>GMS Med Inform Biom Epidemiol</RefJournal> <RefPage>Doc20</RefPage> <RefTotal>Oldhoff-Nuijsink C, Buitendijk F, Rolink M, Derksen ME, Peute LWP, Fransen MP. Co-creating a cancer screening dashboard with screening invitees and experts. GMS Med Inform Biom Epidemiol. 2025;21:Doc20. DOI: 10.3205/mibe000292</RefTotal> <RefLink>https://doi.org/10.3205/mibe000292</RefLink> </Reference> <Reference refNo="2"> <RefAuthor>Lima V</RefAuthor> <RefAuthor>Bernardi F</RefAuthor> <RefAuthor>Ferraz V</RefAuthor> <RefAuthor>Alves D</RefAuthor> <RefTitle>Building a digital platform for collaborative second opinions in rare disease: Integrating AI and healthcare networks for improved care</RefTitle> <RefYear>2025</RefYear> <RefJournal>GMS Med Inform Biom Epidemiol</RefJournal> <RefPage>Doc19</RefPage> <RefTotal>Lima V, Bernardi F, Ferraz V, Alves D. Building a digital platform for collaborative second opinions in rare disease: Integrating AI and healthcare networks for improved care. GMS Med Inform Biom Epidemiol. 2025;21:Doc19. DOI: 10.3205/mibe000291</RefTotal> <RefLink>https://doi.org/10.3205/mibe000291</RefLink> </Reference> <Reference refNo="3"> <RefAuthor>Kim S</RefAuthor> <RefAuthor>Park YE</RefAuthor> <RefAuthor>Lee Y</RefAuthor> <RefAuthor>Lee JW</RefAuthor> <RefTitle>Adherence to hormonal therapy in breast cancer patients: EHR-based retrospective data analysis</RefTitle> <RefYear>2025</RefYear> <RefJournal>GMS Med Inform Biom Epidemiol</RefJournal> <RefPage>Doc18</RefPage> <RefTotal>Kim S, Park YE, Lee Y, Lee JW. Adherence to hormonal therapy in breast cancer patients: EHR-based retrospective data analysis. GMS Med Inform Biom Epidemiol. 2025;21:Doc18. DOI: 10.3205/mibe000290</RefTotal> <RefLink>https://doi.org/10.3205/mibe000290</RefLink> </Reference> <Reference refNo="4"> <RefAuthor>Hülsmann H</RefAuthor> <RefAuthor>Swoboda W</RefAuthor> <RefAuthor>Holl F</RefAuthor> <RefTitle>Identifying assessment frameworks for digital public health interventions: First results of a scoping review</RefTitle> <RefYear>2025</RefYear> <RefJournal>GMS Med Inform Biom Epidemiol</RefJournal> <RefPage>Doc17</RefPage> <RefTotal>Hülsmann H, Swoboda W, Holl F. Identifying assessment frameworks for digital public health interventions: First results of a scoping review. GMS Med Inform Biom Epidemiol. 2025;21:Doc17. DOI: 10.3205/mibe000289</RefTotal> <RefLink>https://doi.org/10.3205/mibe000289</RefLink> </Reference> <Reference refNo="5"> <RefAuthor>Grashof R</RefAuthor> <RefAuthor>Breil B</RefAuthor> <RefAuthor>Lipprandt M</RefAuthor> <RefTitle>A new perspective on eHealth acceptance: Combining health-related factors with the Technology Acceptance Model</RefTitle> <RefYear>2025</RefYear> <RefJournal>GMS Med Inform Biom Epidemiol</RefJournal> <RefPage>Doc16</RefPage> <RefTotal>Grashof R, Breil B, Lipprandt M. A new perspective on eHealth acceptance: Combining health-related factors with the Technology Acceptance Model. GMS Med Inform Biom Epidemiol. 2025;21:Doc16. DOI: 10.3205/mibe000288</RefTotal> <RefLink>https://doi.org/10.3205/mibe000288</RefLink> </Reference> <Reference refNo="6"> <RefAuthor>Galland-Decker C</RefAuthor> <RefAuthor>Usenbacher M</RefAuthor> <RefAuthor>Nunes C</RefAuthor> <RefAuthor>Bouche F</RefAuthor> <RefAuthor>Carra G</RefAuthor> <RefAuthor>Boillat-Blanco N</RefAuthor> <RefAuthor>Hartley MA</RefAuthor> <RefAuthor>Sallinen A</RefAuthor> <RefAuthor>Raisaro JL</RefAuthor> <RefAuthor>Bastardot F</RefAuthor> <RefTitle>Performance evaluation of Meditron3-70B in medical coding: Current limitations and integration perspectives for clinical practice</RefTitle> <RefYear>2025</RefYear> <RefJournal>GMS Med Inform Biom Epidemiol</RefJournal> <RefPage>Doc15</RefPage> <RefTotal>Galland-Decker C, Usenbacher M, Nunes C, Bouche F, Carra G, Boillat-Blanco N, Hartley MA, Sallinen A, Raisaro JL, Bastardot F. Performance evaluation of Meditron3-70B in medical coding: Current limitations and integration perspectives for clinical practice. GMS Med Inform Biom Epidemiol. 2025;21:Doc15. DOI: 10.3205/mibe000287</RefTotal> <RefLink>https://doi.org/10.3205/mibe000287</RefLink> </Reference> <Reference refNo="7"> <RefAuthor>Slob D</RefAuthor> <RefAuthor>Akhmad E</RefAuthor> <RefAuthor>Garcia González J</RefAuthor> <RefAuthor>Amiri S</RefAuthor> <RefAuthor>Kasalica V</RefAuthor> <RefAuthor>Georgievska S</RefAuthor> <RefAuthor>Choudhury A</RefAuthor> <RefAuthor>Lobo Gomes A</RefAuthor> <RefAuthor>Dekker A</RefAuthor> <RefAuthor>van Soest J</RefAuthor> <RefTitle>FAIVOR – a push-button system for AI validation within the hospital</RefTitle> <RefYear>2025</RefYear> <RefJournal>GMS Med Inform Biom Epidemiol</RefJournal> <RefPage>Doc14</RefPage> <RefTotal>Slob D, Akhmad E, Garcia González J, Amiri S, Kasalica V, Georgievska S, Choudhury A, Lobo Gomes A, Dekker A, van Soest J. FAIVOR – a push-button system for AI validation within the hospital. GMS Med Inform Biom Epidemiol. 2025;21:Doc14. DOI: 10.3205/mibe000286</RefTotal> <RefLink>https://doi.org/10.3205/mibe000286</RefLink> </Reference> <Reference refNo="8"> <RefAuthor>Mannevaara P</RefAuthor> <RefAuthor>Saranto K</RefAuthor> <RefAuthor>Kinnunen UM</RefAuthor> <RefTitle>Developing competencies in health informatics: Blended teaching method</RefTitle> <RefYear>2025</RefYear> <RefJournal>GMS Med Inform Biom Epidemiol</RefJournal> <RefPage>Doc13</RefPage> <RefTotal>Mannevaara P, Saranto K, Kinnunen UM. Developing competencies in health informatics: Blended teaching method. GMS Med Inform Biom Epidemiol. 2025;21:Doc13. DOI: 10.3205/mibe000285</RefTotal> <RefLink>https://doi.org/10.3205/mibe000285</RefLink> </Reference> <Reference refNo="9"> <RefAuthor>Focsa M</RefAuthor> <RefTitle>ARISE: AI and AR for inclusive health education</RefTitle> <RefYear>2025</RefYear> <RefJournal>GMS Med Inform Biom Epidemiol</RefJournal> <RefPage>Doc12</RefPage> <RefTotal>Focsa M. ARISE: AI and AR for inclusive health education. GMS Med Inform Biom Epidemiol. 2025;21:Doc12. DOI: 10.3205/mibe000284</RefTotal> <RefLink>https://doi.org/10.3205/mibe000284</RefLink> </Reference> </References> <Media> <Tables> <NoOfTables>0</NoOfTables> </Tables> <Figures> <NoOfPictures>0</NoOfPictures> </Figures> <InlineFigures> <NoOfPictures>0</NoOfPictures> </InlineFigures> <Attachments> <NoOfAttachments>0</NoOfAttachments> </Attachments> </Media> </OrigData> </GmsArticle>