Classifying metrics for assessing object-oriented software maintainability: a family of metrics’ catalogs

Abstract

Currently, Object-Oriented Programming (OOP) is one of the most used paradigms. Complementarily, the software maintainability is considered a software attribute that plays an important role in quality level. In this context, the Object-Oriented Software Maintainability (OOSM) has been studied through years, and many researchers have proposed a large number of metrics to measure it. As a consequence of the number and diversity of metrics, beyond the no standardization in metrics definition and naming, the decision-making process about which metrics can be adopted in experiments on OOSM, or even their using in software companies is a difficult task. Therefore, a systematic mapping study was conducted in order to find which metrics are used as indicators in OOSM assessments. There was an initial selection of 5175 primary studies and 138 were selected, resulting in 568 metrics found. Analyzing the 568 metrics, inconsistencies in metrics’ naming were found because there were metrics with the same names but different meanings (8 cases involving 17 metrics) and also, there were metrics with different names, however with similar meanings (32 cases involving 214 metrics). Moreover, a metrics’ categorization has been proposed to facilitate decision-making process about which ones have to be adopted, and 7 categories and 17 subcategories were identified. These categories represent the evaluation scenarios where OOSM metrics should be used. Additionally, a metrics’ web portal was developed to provide information about the metrics collected in this research, and to generate metrics’ catalogs according to the context of their adoption. This portal can also be systematically fed by other researchers that work with OOSM metrics, making the results of this work the first steps towards metrics’ standardization, and the improvement of the metrics’ validation. Finally, a quasi-experiment was conducted to check the coverage index of the catalogs generated using our approach over the catalogs suggested by experts. 90% of coverage was obtained and this result was confirmed with 99% of confidential level using the Wilcoxon Test. Complementarily, a survey was conducted to check the experts’ opinion about the catalog generated by the portal when they were compared by the catalogs suggested by the experts. Thus, the coverage evaluation can be the first evidences of the usefulness of the proposed approach for metrics’ choice in OOSM evaluation.

Atualmente, Programao Orientada a Objetos (POO) um dos paradigmas mais utilizados. Complementarmente, a manutenibilidade de software considerada um atributo de software que desempenha um papel importante com relao ao nvel de qualidade. Neste contexto, a Manutenibilidade de Software Orientado a Objetos (MSOO) foi estudada atravs de anos e vrios pesquisadores propuseram um elevado nmero de mtricas para a medir. Como consequncia do nmero e da diversidade de mtricas existentes, alm da no padronizao nas definies e nomenclatura, a tomada de deciso sobre quais mtricas podem ser adotadas para realizar estudos em MSOO difcil. Desta forma, um mapeamento sistemtico foi realizado a fim de encontrar quais mtricas so usadas como indicadores de MSOO. Houve uma seleo inicial de 5175 estudos primrios e 138 artigos foram selecionados, resultando em 568 mtricas encontradas. Analisando as 568 mtricas, foram encontradas inconsistncias na nomenclatura destas mtricas, pois havia mtricas com nomes iguais mas significados diferentes (8 casos envolvendo 17 mtricas) e tambm mtricas com nomes diferentes e significados semelhantes (32 casos envolvendo 214 mtricas). Alm disto, uma categorizao destas mtricas foi proposta, sendo identificadas 7 categorias e 17 subcategorias. Estas categorias representam os cenrios de adoo de mtricas de MSOO. Adicionalmente, um portal web de mtricas foi desenvolvido para fornecer informaes sobre as mtricas para outros pesquisadores e tambm gerar catlogos de mtricas de acordo com o contexto da aplicao das mesmas. Este portal tambm pode ser alimentado sistematicamente por outros pesquisadores que lidam com mtricas de MSOO, fazendo com que os resultados deste trabalho possam representar os primeiros passos para padronizao e compreenso destas mtricas. Por ltimo, um quasi-experimento foi realizado para checar o grau de cobertura do catlogo proposto pela abordagem aqui apresentada quando o mesmo comparado com catlogos sugeridos por especialistas. 90% de cobertura foi obtido e este resultado foi confirmado com 99% de grau de confiana usando o Teste de Wilcoxon. De forma complementar, houve uma pesquisa de opinio para checar se os especialistas acharam catlogo gerado usando a nossa abordagem semelhante ou melhor do que o sugerido por eles. Sendo assim, os resultados da anlise da cobertura dos catlogos pode servir como indcios da utilidade da abordagem proposta para a escolha de mtricas na avaliao de MSOO.