Generic placeholder image

Recent Advances in Computer Science and Communications

Author(s): Narayan C. Debnath* and Archana Patel*

DOI: 10.2174/2666255815666220411081837

DownloadDownload PDF Flyer Cite As
Ontology Evaluation Tools: Current and Future Research

Article ID: e110422203361 Pages: 14

  • * (Excluding Mailing and Handling)

Explore Articles
About Journal
For Authors
For Editors
For Reviewers

Abstract

In recent years, the increasing interest in ontologies resulted in the developing and publishing of many ontologies in the same or different domains. When users try to reuse the existing ontologies in their applications, they may encounter problems determining the most appropriate ontology based on their needs. Ontology evaluation is a process that helps users to examine the quality of the ontology based on different attributes. Many accessible and usable tools for ontology evaluation have been studied in the literature. However, finding an efficient ontology evaluation tool, following ontology specifications and their requirements (advantages/disadvantages), is still missing, limiting the researchers from determining possible future research. This paper aims to help new researchers and practitioners identify appropriate ontology evaluation tools based on their requirements and provide guidelines for future research directions on the same topic. This paper provides a detailed description of the different types of ontologies and classifies the available ontology evaluation tools into two categories, namely domain dependant ontology evaluation tools and domain-independent ontology evaluation tools.

Keywords: Ontology, evaluation, ontology evaluation tools, evaluation metrics, evaluation approaches, computational guarantees.

Graphical Abstract

[1]
V. Ravi Sanka, A. Damodaram, and P. Radha Kris, "Ranking ontologies based on OWL language constructs", Inf. Technol. J., vol. 9, no. 3, pp. 553-560, 2010.
[http://dx.doi.org/10.3923/itj.2010.553.560]
[2]
Web Ontology Language, Available from: https://www.w3.org/OWL/
[3]
M. d’Aquin, and H. Lewen, "Cupboard-A place to expose your ontologies to applications and the community", In: L. Aroyo, Ed., The Semantic Web: Research and Applications. ESWC 2009.Lecture Notes in Computer Science, vol. 5554. Springer: Berlin, Heidelberg, 2009.
[http://dx.doi.org/10.1007/978-3-642-02121-3_81]
[4]
M. Ohta, K. Kozaki, and R. Mizoguchi, A quality assurance framework for ontology construction and refinement.Advances in Intelligent Web Mastering–3., Springer: Berlin, Heidelberg, 2011, pp. 207-216.
[http://dx.doi.org/10.1007/978-3-642-18029-3_21]
[5]
P. Buitelaar, and P. Cimiano, Ontology learning and population: Bridging the gap between text and knowledge., vol. 167. Ios Press: Amsterdam, Netherlands, 2008.
[6]
N.F. Noy, and M. Klein, "Ontology evolution: Not the same as schema evolution", Knowl. Inf. Syst., vol. 6, no. 4, pp. 428-440, 2004.
[http://dx.doi.org/10.1007/s10115-003-0137-2]
[7]
Science Direct, Available from: https://www.sciencedirect.com/browse/journals-and-books?subject=computer-science
[8]
ACM Library, Available from: https://dl.acm.org/
[9]
IEEE Explore, Available from: https://ieeexplore.ieee.org/Xplore/home.jsp
[10]
Google Scholar, Available from: https://scholar.google.com/
[11]
M. Amith, Z. He, J. Bian, J.A. Lossio-Ventura, and C. Tao, "Assessing the practice of biomedical ontology evaluation: Gaps and opportunities", J. Biomed. Inform., vol. 80, pp. 1-13, 2018.
[http://dx.doi.org/10.1016/j.jbi.2018.02.010] [PMID: 29462669]
[12]
J. Gelernter, and J. Jha, "Challenges in ontology evaluation", ACM J. Data Inf. Qual., vol. 7, no. 3, pp. 1-4, 2016.
[http://dx.doi.org/10.1145/2935751]
[13]
A. Degbelo, "A snapshot of ontology evaluation criteria and strategies", In Proceedings of Semantics, 11-14 September, 2017, Amsterdam, Netherlands, 2017, pp. 1-8
[http://dx.doi.org/10.1145/3132218.3132219]
[14]
R. Lourdusamy, and A. John, "A review on metrics for ontology evaluation", In 2018 2nd International Conference on Inventive Systems and Control (ICISC)., Coimbatore, India, IEEE, 2018, pp. 1415-1421
[http://dx.doi.org/10.1109/ICISC.2018.8399041]
[15]
T. Aruna, K. Saranya, and C. Bhandari, "A survey on ontology evaluation tools", In 2011 International Conference on Process Automation, Control and Computing, Coimbatore, India, IEEE, 2011, pp. 1-5
[16]
J. Brank, M. Grobelnik, and D. Mladenic, "A survey of ontology evaluation techniques", In Proceedings of the conference on data mining and data warehouses (SiKDD 2005), Ljubljana, Slovenia, 2005, pp. 166-170
[17]
L. Obrst, T. Hughes, and S. Ray, "Prospects and possibilities for ontology evaluation: The view from NCOR. MITRE CORP MCLEAN VA, May 22-26, 2006 Edinburgh, UK",
[18]
J. Raad, and C. Cruz, "A survey on ontology evaluation methods", In Proceedings of the International Conference on Knowledge Engineering and Ontology Development, part of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (KEOD), Lisbon, Portugal, 2015, pp. 179-186
[http://dx.doi.org/10.5220/0005591001790186]
[19]
A. Hicks, "Metrics and methods for comparative ontology evaluation", Cienc. Inf., vol. 46, no. 1, 2017.
[20]
A. Gómez-Pérez, "Some ideas and examples to evaluate ontologies", Proceedings the 11th Conference on Artificial Intelligence for Applications, 20-23 Feb. 1995, Los Angeles, CA, USA, 1995, pp. 299-305.
[http://dx.doi.org/10.1109/CAIA.1995.378808]
[21]
L. Obrst, B. Ashpole, W. Ceusters, I. Mani, and S. Ray, "The evaluation of ontologies: Toward improved semantic interoperability", In: Semantic Web., 2007, pp. 139-158.
[22]
Y.K. Hooi, M.F. Hassan, and A.M. Shariff, "Ontology evaluation—A criteria selection framework", In: Mathematical Sciences and Computing Research, 19-20 May 2015, Ipoh, Malaysiap, IEEE, 2015, pp. 298-303.
[http://dx.doi.org/10.1109/ISMSC.2015.7594069]
[23]
H. Hlomani, and D. Stacey, "Approaches, methods, metrics, measures, and subjectivity in ontology evaluation: A survey", Semantic Web J., vol. 1, no. 5, pp. 1-11, 2014.
[24]
S. Lovrencic, and M. Cubrilo, "Ontology evaluation-comprising verification and validation", Central European Conference on Information and Intelligent Systems, Faculty of Organization and Informatics, September 24-26 2008, Varaždin, Croatia, 2008.
[25]
J. García, F. Jose’García-Peñalvo, and R. Therón, "A survey on ontology metrics", In: World Summit on Knowledge Society., Springer: Berlin, Heidelberg, 2010, pp. 22-27.
[26]
A. Kayed, "Ontology evaluation: Which test to use", In: 2013 5th International Conference on Computer Science and Information Technology, 27-28 March 2013, Amman, Jordan., 2013, pp. 45-48.
[http://dx.doi.org/10.1109/CSIT.2013.6588756]
[27]
M. Compton, P. Barnaghi, L. Bermudez, R. Garcia-Castro, O. Corcho, S. Cox, J. Graybeal, M. Hauswirth, C. Henson, A. Herzog, V. Huang, K. Janowicz, W.D. Kelsey, D. Le Phuoc, L. Lefort, M. Leggieri, H. Neuhaus, A. Nikolov, K. Page, A. Passant, A. Sheth, and K. Taylor, "The SSN ontology of the W3C semantic sensor network incubator group", J. Web Semant., vol. 17, pp. 25-32, 2012.
[http://dx.doi.org/10.1016/j.websem.2012.05.003]
[28]
S. Kolozali, T. Elsaleh, and P.M. Barnaghi, A Validation Tool for the W3C SSN Ontology based Sensory Semantic Knowledge., 2014, pp. 83-88.
[29]
M. Amith, F. Manion, C. Liang, M. Harris, D. Wang, Y. He, and C. Tao, "OntoKeeper: Semiotic-driven ontology evaluation tool for biomedical ontologists", In International Conference on Bioinformatics and Biomedicine (BIBM), 3-6 December 2018, Madrid, Spain, 2018, pp. 1614-1617
[http://dx.doi.org/10.1109/BIBM.2018.8621458]
[30]
M. Amith, F. Manion, C. Liang, M. Harris, D. Wang, Y. He, and C. Tao, "Architecture and usability of OntoKeeper, an ontology evaluation tool", BMC Med. Inform. Decis. Mak., vol. 19, no. 4, suppl. 4, p. 152, 2019.
[http://dx.doi.org/10.1186/s12911-019-0859-z] [PMID: 31391056]
[31]
G.R. Roldan-Molina, J.R. Mendez, I. Yevseyeva, and V. Basto-Fernandes, "Ontology fixing by using software engineering technology", Appl. Sci. (Basel), vol. 10, no. 18, p. 6328, 2020.
[http://dx.doi.org/10.3390/app10186328]
[32]
A. Duque-Ramos, J.T. Fernández-Breis, R. Stevens, and N. Aussenac-Gilles, "OQuaRE: A SQuaRE-based approach for evaluating the quality of ontologies", J. Res. Pract. Inf. Technol., vol. 43, no. 2, pp. 159-176, 2011.
[33]
A. Duque-Ramos, J.T. Fernández-Breis, M. Iniesta, M. Dumontier, M.E. Aranguren, and S. Schulz, "Evaluation of the OQuaRE framework for ontology quality", Expert Syst. Appl., vol. 40, no. 7, pp. 2696-2703, 2013.
[http://dx.doi.org/10.1016/j.eswa.2012.11.004]
[34]
S. Tartir, I.B. Arpinar, M. Moore, A.P. Sheth, and B. Aleman-Meza, "OntoQA: Metric-based ontology quality analysis", In IEEE Workshop on Knowledge Acquisition from Distributed, Autonomous, Semantically Heterogeneous Data and Knowledge Sources, November 27, 2005, Houston, Texas. LSDIS, 2005.
[35]
S. Tartir, and I.B. Arpinar, "Ontology evaluation and ranking using OntoQA", In International conference on semantic computing (ICSC 2007), 17-19 Sept. 2007, Irvine, CA, USA, IEEE, 2007, pp. 185-192
[http://dx.doi.org/10.1109/ICSC.2007.19]
[36]
OWL Validator, Available from: http://visualdataweb.de/validator/
[37]
RDF/XML Validator, Available from: https://www.w3.org/RDF/Validator/
[38]
N. Guarino, and C.A. Welty, An overview of OntoClean. Handbook on ontologies., 2004, pp. 151-171.
[http://dx.doi.org/10.1007/978-3-540-24750-0_8]
[39]
R.Q. Dividino, M. Romanelli, and D. Sonntag, "Semiotic-based Ontology Evaluation Tool (S-OntoEval", In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), May 2008, Marrakech, Morocco, 2008.
[40]
C.V.S. Avila, G. Maia, W. Franco, T.V. Rolim, A.D.O. da Rocha Franco, and V.M.P. Vidal, "OntoVal: A tool for ontology evaluation by domain specialists", In ER 2019 – 38th International Conference on Conceptual Modeling, Salvador, Bahia, Brazil., 2019, pp. 143-147
[41]
A. Patel, C.N. Debnath, A. Mishra, and S. Jain, "Covid19-IBO: A Covid19-impact-on-Indian-banking ontology along with an efficient schema matching approach", New Gener. Comput., vol. 39, no. 3-4, pp. 647-676, 2021.
[42]
A.B. Bouiadjra, and S.M. Benslimane, "FOEval: Full ontology evaluation", In 2011 7th International Conference on Natural Language Processing and Knowledge Engineering, 27-29 Nov. 2011, Tokushima, Japan, IEEE., 2011, pp. 464-468
[http://dx.doi.org/10.1109/NLPKE.2011.6138244]