High Throughput Neurological Phenotyping with MetaMap
Keywords:
MetaMap, phenotyping, high throughput, ontologies, natural language processing
Abstract
The phenotyping of neurological patients involves the conversion of signs and symptoms into machine readable codes selected from an appropriate ontology. The phenotyping of neurological patients is manual and laborious. MetaMap is used for high throughput mapping of the medical literature to concepts in the Unified Medical Language System Metathesaurus (UMLS). MetaMap was evaluated as a tool for the high throughput phenotyping of neurological patients. Based on 15 patient histories from electronic health records, 30 patient histories from neurology textbooks, and 20 clinical summaries from the Online Mendelian Inheritance in Man repository, MetaMap showed a recall of 61-89%, a precision of 84-93%, and an accuracy of 56-84% for the identification of phenotype concepts. The most common cause of false negatives (failure to recognize a phenotype concept) was an inability of MetaMap to find concepts that were represented as a description or a definition of the concept. The most common cause of false positives (incorrect identification of a concept in the text) was a failure to recognize that a concept was negated. MetaMap shows potential for high throughput phenotyping of neurological patients if the problems of false negatives and false positives can be solved.
Downloads
Download data is not yet available.
Metrics
Metrics Loading ...
PlumX Statistics
References
1. Agirre, E., & Edmonds, P. (2007). Word sense disambiguation: Algorithms and applications (Vol. 33). Springer Science & Business Media.
2. Amberger, J. S., & Hamosh, A. (2017). Searching online mendelian inheritance in man (OMIM): a knowledge base of human genes and genetic phenotypes. Current Protocols in Bioinformatics, 58 (1), 1–2.
3. Aronson, A. R. (2000). The MetaMap mapping algorithm. National Institutes of Health (NIH), Technical Report (https://lhncbc.nlm.nih.gov/ii/information/Papers/mm.mapping.pdf)
4. Aronson, A. R. (2001a). Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. Proc AMIA Symp. 2001:17-21.
5. Aronson, A. R. (2001b). MetaMap variant generation. 2014-08-15. nlm.nih. gov/papers/references/mm.variants.pdf .
6. Aronson, A. R., & Lang, F.-M. (2010, 05). An overview of MetaMap: historical perspective and recent advances. Journal of the American Medical Informatics Association, 17 (3), 229-236. Retrieved from https://doi.org/10.1136/jamia.2009.002733
7. Bashyam, V., Divita, G., Bennett, D. B., Taira, R. K., & Browne, A. C. (2007). A normalized lexical lookup approach to identifying UMLS concepts in free text. In Medinfo 2007: Proceedings of the 12th World Congress on Health (Medical) Informatics; Building Sustainable Health Systems (p. 545). IOS Press
8. Brown P. (2001). Principles of Post-coordination Concept Construction and Retrieval. Proceedings of the AMIA Symposium, 869.
9. Brennan, P. F., & Aronson, A. R. (2003). Towards linking patients and clinical information: detecting UMLS concepts in email. Journal of biomedical informatics, 36(4-5), 334-341.
10. Cohen, R., Gefen, A., Elhadad, M., & Birk, O. S. (2011). CSI-OMIM-Clinical synopsis search in OMIM. BMC Bioinformatics, 12 (1), 1-10.
11. Collins, F. S., & Varmus, H. (2015). A new initiative on precision medicine. New England Journal of Medicine, 372 (9), 793–795.
12. Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Dean, J. (2019). A guide to deep learning in healthcare. Nature medicine, 25 (1), 24–29.
13. Friedlin, J., & Overhage, M. (2011). An evaluation of the UMLS in representing corpus-derived clinical concepts. In AMIA annual symposium proceedings (Vol. 2011, p. 435).
14. Fu, S., Chen, D., He, H., Liu, S., Moon, S., Peterson, K. J., and others (2020). Clinical concept extraction: a methodology review. Journal of Biomedical Informatics, 103526.
15. Gondolo, T. (2005). Neurology study guide: Oral board examination review. Springer Nature.
16. Gooch, P. and Roudsari, A. (2011). A tool for enhancing MetaMap performance when annotating clinical guideline documents with UMLS concepts. Paper presented at the IDAMAP Workshop at 13th Conference on Artificial Intelligence in Medicine (AIME'11), 02-07-2011 - 06-07-2011, Bled, Slovenia.
17. Hamosh, A., Scott, A. F., Amberger, J., Valle, D., & McKusick, V. A. (2000). Online mendelian inheritance in man (OMIM). Human mutation, 15 (1), 57–61.
18. Hedeler, C., Parsia, B., & Brandt, S. (2014, May). Estimating and analyzing coordination in medical terminologies. IEEE. In 2014 IEEE 27th International Symposium on Computer-Based Medical Systems (pp. 357-362).
19. Hier, D. B., & Brint, S. U. (2020). A neuro-ontology for the neurological examination. BMC Medical Informatics and Decision Making, 20 (1), 1–9.
20. Hier, D. B., Kopel, J., Brint, S. U., Wunsch, D. C., Olbricht, G. R., Azizi, S., & Allen, B. (2020). Evaluation of standard and semantically-augmented distance metrics for neurology patients. BMC Medical Informatics and Decision Making, 20 (1), 1–15.
21. Kimia, A. A., Savova, G., Landschaft, A., & Harper, M. B. (2015). An introduction to natural language processing: how you can get more from those electronic notes you are generating. Pediatric emergency care, 31 (7), 536–541.
22. Kohler, S., Doelken, S. C., Mungall, C. J., Bauer, S., Firth, H. V., Bailleul-Forestier, I., and others (2014). The human phenotype ontology project: linking molecular biology and disease through phenotype data—Nucleic Acids Research, 42 (D1), D966–D974.
23. Kohler, S., Øien, N. C., Buske, O. J., Groza, T., Jacobsen, J. O., McNamara, C., and others (2019). Encoding clinical data with the human phenotype ontology for computational differential diagnostics. Current protocols in human genetics, 103 (1), e92.
24. Marrero, M., Urbano, J., S´anchez-Cuadrado, S., Morato, J., & G´omez-Berb´ıs, J. M. (2013). Named entity recognition: fallacies, challenges and opportunities. Computer Standards & Interfaces, 35 (5), 482–489.
25. Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2018). Deep learning for healthcare: review, opportunities, and challenges. Briefings in bioinformatics, 19 (6), 1236–1246.
26. Mork, J. G., & Aronson, A. R. (2006). Filtering the UMLS Metathesaurus for MetaMap. National Library of Medicine Technical Report. (https://lhncbc.nlm.nih.gov/ii/information/Papers/filtering09.pdf)
27. Navas, H., Lopez Osornio, A., Gambarte, L., Elías Leguizamón, G., Wasserman, S., Orrego, N., Luna, D. and de Quirós, F.G.B. (2010). Implementing rules to improve the quality of concept post-coordination with SNOMED CT. In MEDINFO 2010 (pp. 1045-1049). IOS Press.
28. NCBO BioPortal. (2021). Neurologic examination ontology. https://bioportal.bioontology.org/ontologies/NEO.
29. Okumura, T., & Tateisi, Y. (2012). A lightweight approach for extracting disease-symptom relation with MetaMap toward automated generation of disease knowledge base. International Conference on Health Information Science. pp. 164–172.
30. Osborne, J. D., Lin, S., Zhu, L. J., & Kibbe, W. A. (2007). Mining biomedical data using MetaMap transfer (MMtx) and the unified medical language system (UMLS). In Gene Function Analysis (pp. 153-169). Humana Press
31. Patterson, O., Igo, S., & Hurdle, J. F. (2010). Automatic acquisition of sublanguage semantic schema: towards the word sense disambiguation of clinical narratives. In AMIA Annual Symposium Proceedings (Vol. 2010, p. 612). American Medical Informatics Association.
32. Pratt, W., & Yetisgen-Yildiz, M. (2003). A study of biomedical concept identification: MetaMap vs. people. 2003, 529. (AMIA Annual Symposium Proceedings)
33. Robinson, P. N. (2012). Deep phenotyping for precision medicine. Human Mutation, 33 (5), 777–780.
34. Schuemie, M. J., Kors, J. A., & Mons, B. (2005). Word sense disambiguation in the biomedical domain: an overview. Journal of Computational Biology, 12 (5), 554–565.
35. Ubogu, E. E. (2005). Neurology oral boards review. Humana Press.
36. Xiao, C., Choi, E., & Sun, J. (2018). Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review. Journal of the American Medical Informatics Association, 25 (10), 1419–1428.
2. Amberger, J. S., & Hamosh, A. (2017). Searching online mendelian inheritance in man (OMIM): a knowledge base of human genes and genetic phenotypes. Current Protocols in Bioinformatics, 58 (1), 1–2.
3. Aronson, A. R. (2000). The MetaMap mapping algorithm. National Institutes of Health (NIH), Technical Report (https://lhncbc.nlm.nih.gov/ii/information/Papers/mm.mapping.pdf)
4. Aronson, A. R. (2001a). Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. Proc AMIA Symp. 2001:17-21.
5. Aronson, A. R. (2001b). MetaMap variant generation. 2014-08-15. nlm.nih. gov/papers/references/mm.variants.pdf .
6. Aronson, A. R., & Lang, F.-M. (2010, 05). An overview of MetaMap: historical perspective and recent advances. Journal of the American Medical Informatics Association, 17 (3), 229-236. Retrieved from https://doi.org/10.1136/jamia.2009.002733
7. Bashyam, V., Divita, G., Bennett, D. B., Taira, R. K., & Browne, A. C. (2007). A normalized lexical lookup approach to identifying UMLS concepts in free text. In Medinfo 2007: Proceedings of the 12th World Congress on Health (Medical) Informatics; Building Sustainable Health Systems (p. 545). IOS Press
8. Brown P. (2001). Principles of Post-coordination Concept Construction and Retrieval. Proceedings of the AMIA Symposium, 869.
9. Brennan, P. F., & Aronson, A. R. (2003). Towards linking patients and clinical information: detecting UMLS concepts in email. Journal of biomedical informatics, 36(4-5), 334-341.
10. Cohen, R., Gefen, A., Elhadad, M., & Birk, O. S. (2011). CSI-OMIM-Clinical synopsis search in OMIM. BMC Bioinformatics, 12 (1), 1-10.
11. Collins, F. S., & Varmus, H. (2015). A new initiative on precision medicine. New England Journal of Medicine, 372 (9), 793–795.
12. Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Dean, J. (2019). A guide to deep learning in healthcare. Nature medicine, 25 (1), 24–29.
13. Friedlin, J., & Overhage, M. (2011). An evaluation of the UMLS in representing corpus-derived clinical concepts. In AMIA annual symposium proceedings (Vol. 2011, p. 435).
14. Fu, S., Chen, D., He, H., Liu, S., Moon, S., Peterson, K. J., and others (2020). Clinical concept extraction: a methodology review. Journal of Biomedical Informatics, 103526.
15. Gondolo, T. (2005). Neurology study guide: Oral board examination review. Springer Nature.
16. Gooch, P. and Roudsari, A. (2011). A tool for enhancing MetaMap performance when annotating clinical guideline documents with UMLS concepts. Paper presented at the IDAMAP Workshop at 13th Conference on Artificial Intelligence in Medicine (AIME'11), 02-07-2011 - 06-07-2011, Bled, Slovenia.
17. Hamosh, A., Scott, A. F., Amberger, J., Valle, D., & McKusick, V. A. (2000). Online mendelian inheritance in man (OMIM). Human mutation, 15 (1), 57–61.
18. Hedeler, C., Parsia, B., & Brandt, S. (2014, May). Estimating and analyzing coordination in medical terminologies. IEEE. In 2014 IEEE 27th International Symposium on Computer-Based Medical Systems (pp. 357-362).
19. Hier, D. B., & Brint, S. U. (2020). A neuro-ontology for the neurological examination. BMC Medical Informatics and Decision Making, 20 (1), 1–9.
20. Hier, D. B., Kopel, J., Brint, S. U., Wunsch, D. C., Olbricht, G. R., Azizi, S., & Allen, B. (2020). Evaluation of standard and semantically-augmented distance metrics for neurology patients. BMC Medical Informatics and Decision Making, 20 (1), 1–15.
21. Kimia, A. A., Savova, G., Landschaft, A., & Harper, M. B. (2015). An introduction to natural language processing: how you can get more from those electronic notes you are generating. Pediatric emergency care, 31 (7), 536–541.
22. Kohler, S., Doelken, S. C., Mungall, C. J., Bauer, S., Firth, H. V., Bailleul-Forestier, I., and others (2014). The human phenotype ontology project: linking molecular biology and disease through phenotype data—Nucleic Acids Research, 42 (D1), D966–D974.
23. Kohler, S., Øien, N. C., Buske, O. J., Groza, T., Jacobsen, J. O., McNamara, C., and others (2019). Encoding clinical data with the human phenotype ontology for computational differential diagnostics. Current protocols in human genetics, 103 (1), e92.
24. Marrero, M., Urbano, J., S´anchez-Cuadrado, S., Morato, J., & G´omez-Berb´ıs, J. M. (2013). Named entity recognition: fallacies, challenges and opportunities. Computer Standards & Interfaces, 35 (5), 482–489.
25. Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2018). Deep learning for healthcare: review, opportunities, and challenges. Briefings in bioinformatics, 19 (6), 1236–1246.
26. Mork, J. G., & Aronson, A. R. (2006). Filtering the UMLS Metathesaurus for MetaMap. National Library of Medicine Technical Report. (https://lhncbc.nlm.nih.gov/ii/information/Papers/filtering09.pdf)
27. Navas, H., Lopez Osornio, A., Gambarte, L., Elías Leguizamón, G., Wasserman, S., Orrego, N., Luna, D. and de Quirós, F.G.B. (2010). Implementing rules to improve the quality of concept post-coordination with SNOMED CT. In MEDINFO 2010 (pp. 1045-1049). IOS Press.
28. NCBO BioPortal. (2021). Neurologic examination ontology. https://bioportal.bioontology.org/ontologies/NEO.
29. Okumura, T., & Tateisi, Y. (2012). A lightweight approach for extracting disease-symptom relation with MetaMap toward automated generation of disease knowledge base. International Conference on Health Information Science. pp. 164–172.
30. Osborne, J. D., Lin, S., Zhu, L. J., & Kibbe, W. A. (2007). Mining biomedical data using MetaMap transfer (MMtx) and the unified medical language system (UMLS). In Gene Function Analysis (pp. 153-169). Humana Press
31. Patterson, O., Igo, S., & Hurdle, J. F. (2010). Automatic acquisition of sublanguage semantic schema: towards the word sense disambiguation of clinical narratives. In AMIA Annual Symposium Proceedings (Vol. 2010, p. 612). American Medical Informatics Association.
32. Pratt, W., & Yetisgen-Yildiz, M. (2003). A study of biomedical concept identification: MetaMap vs. people. 2003, 529. (AMIA Annual Symposium Proceedings)
33. Robinson, P. N. (2012). Deep phenotyping for precision medicine. Human Mutation, 33 (5), 777–780.
34. Schuemie, M. J., Kors, J. A., & Mons, B. (2005). Word sense disambiguation in the biomedical domain: an overview. Journal of Computational Biology, 12 (5), 554–565.
35. Ubogu, E. E. (2005). Neurology oral boards review. Humana Press.
36. Xiao, C., Choi, E., & Sun, J. (2018). Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review. Journal of the American Medical Informatics Association, 25 (10), 1419–1428.
Published
2022-02-08
How to Cite
Hier, D. B., Yelugam, R., Carrithers, M. D., & Wunsch II, D. C. (2022). High Throughput Neurological Phenotyping with MetaMap. European Scientific Journal, ESJ, 18(4), 37. https://doi.org/10.19044/esj.2022.v18n4p37
Section
Special Editions
Copyright (c) 2022 Daniel B. Hier, Raghu Yelugam, Michael D. Carrithers, Donald C. Wunsch II
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
.jpg)
