Coordinator research program Communication in Healthcare; endowed professor 'Communication in healthcare, especially in primary care', Radboud University, the Netherlands
Automated medical reporting: from multimodal inputs to medical reports through knowledge graphs.
Maas, L., Kisjes, A., Hashemi, I., Heijmans, F., Dalpiaz, F., Dulmen, S. van, Brinkkemper, S. Automated medical reporting: from multimodal inputs to medical reports through knowledge graphs. In: Proceedings of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2021) - Volume 5: HEALTHINF, pages 509-514, ISBN: 978-989-758-490-9Download the PDF
Care providers generally experience a high workload mainly due to the large amount of time required for adequate documentation. This paper presents our visionary idea of real-time automated medical reporting through the integration of speech and action recognition technology with knowledge-based summarization of the interaction between care provider and patient. We introduce the Patient Medical Graph as a formal representation of the dialogue and actions during a medical consultation. This knowledge graph represents human anatomical entities, symptoms, medical observations, diagnoses and treatment plans. The formal representation enables automated preparation of a consultation report by means of sentence plans to generate natural language. The architecture and functionality of the Care2Report prototype illustrate our vision of automated reporting of human communication and activities using knowledge graphs and NLP tools.