This application allows to inspect the distributions (probability densities) of a large corpus of metrics. It was conceived to demonstrate the significant differencs in these densities when they become conditional on the context. For example, the distribution of the TLOC metric differs vastly among the contexts "Programming language" and "Middleware".

In order to operationalize a metric in a given context, it is transformed into a score using some ideal value. Some metrics have an implicit ideal value. For example, the McCabe Cyclomatic Complexity has a lowest-possible (and desirable) ideal value of 1. This application allows to transform any metric into a distance using a user-chosen explicit ideal value. After transforming a metric into a distance, its distribution reflects the observed distances from the chosen ideal value. When we then proceed to obtaining the complementary cumulative distribution function, the CCDF, those distances can be translated into scores. The application also allows checking metrics values from own applications in specific contexts.

Elements

• Controls:
  • Metric Selector: Used to select a single metric for operationalization across the entire corpus and all the various domains. Metrics are grouped into discrete (if discrete, the metric is of integral type) and continuous.
  • Distribution Type Selector: This application supports the four principal types PDF (probability density function), CDF (cumulative density function), CCDF [1-CDF] (complementary cumulative distribution function; sometimes also called survival function), and PPF (percent point or quantile function). Furthermore, the application offers different sources for each fitted random variable. These are:
    • Parametric: A total of approximately 100 continuous different distributions were attempted to fit to each metric, in each domain. The best fitting distribution was selected using a two-sample Kolmogorov—Smirnow test [3]. We even fit continuous distributions to discrete data, but not vice versa.
    • Parametric (discrete): For those metrics that are of discrete nature, that is, integral, we also attempt to fit more than approximately 20 discrete distributions and select the best fitting using the Epps—Singleton two-sample goodness-of-fit test [4].
    • Kernel Density Estimation: Using a Kernel Density Estimation allows us to estimate a probability density for continuous and discrete metrics. In cases of the (C)CDF derived from KDE fit, we sample a large number inversely from the estimated density, fit a Gaussian KDE, and then obtain an ECDF in order for it to become smooth. This ECDF can be reversed (to become an ECCDF) and inversed (to become an EPPF). The smoothed versions are useful for cases when the actual data is scarce and the ECDFs have large jumps.
    • Empirical: While the (C)CDF and EPPF are derived from the data's ECDF, the EPMF is a frequency table over the data. For obtaining exact scores, ECDF and EPMF should be used.
  • Status indicator: Shows "Ready." when the application is ready to be used. Some actions will require a significant amount of processing during which the status will show a throbber and a status text. It may also show errors, if any. For example, selecting a discrete distribution for a continuous metric is an error.
  • Transformation Type Selector: Used to select an explicit value to transform the selected metric into a distance from that value, which is context-specific. While the infimum and supremum were taken directly from the data, the expectation (E[X]), median, and mode for continuous metrics were taken from PDFs that were obtained by fitting a Gaussian Kernel on a large individual inverse sample from the original data. This was done to obtain more real-valued data as transformation values are otherwise very similar and repetitive. For discrete metrics, these transformation values were computed ordinarily.
  • Own Metric Value / Probability Input: A numeric input field that enables the user to check their own metric value or probability against the currently selected distribution- and transformation type. If no transformation is chosen or transformation using the context's ideal value is disabled, the own metric's value is shown as a vertical line on the plot. Note that when checking a probability against a PPF, the checkbox for applying the selected transform is disabled.
  • Apply transform using ideal value: If selected, the own metric's value will be transformed into a distance (has no effect if no transformation was chosen). The transformation is calculated as abs(ideal - [own metric value]). Note that when a transformation is chosen and this box is checked, the vertical line in the plot indicating the own metric's value will disappear, as the distance is dependent on the context and this would otherwise require drawing a line for each transformed value / distance.
  • Contain Plot Button: When this button is clicked, the plot will be panned and zoomed to exactly contain the shown distributions. Note that the plot itself also has a reset button.
  • Cutting off smoothed distributions: This checkbox is only enabled if the chosen distribution is not of type E(C)CDF. Since smoothed distributions were obtained using Kernel density estimation, the bandwidth of Gaussian Kernels will indicate non-zero probability beyond actual observed values. Checking this box will allow to obtain a value from a smoothed distribution while simultaneously ascertaining that it will not be out of range.
• Data Table: A table that shows context-specific values (according to each domain).
  • Domain: The domain/context.
  • Used Transformation Value: If a transformation other than <none> was chosen, this column shows the context-dependent value for the conditional distribution. This value is used to transform the own metric's value into a distance from the transformation value.
  • Metric Value / Metric Distance / Probability: If an own metric value or probability is checked against the corpus, the value is shown here. If a transformation other than <none> is chosen and the checkbox for applying transformations is selected, then the own metric's value is transformed into a distance using each domain's ideal value as abs(ideal - value). The title of this column will change accordingly.
  • Relative Likelihood / Cumulative Probability / Corresponding Score / Value of Random Variable: Shows the (transformed) own metric's value for each domain or the predicted metric's value for a given probability. For probability densities (PDF), the column name will be "Relative Likelihood". For ECDFs the title will be "Cumulative Probability", for ECCDFs / Scores the title will be "Corresponding Score", and for PPFs the title will be "Value of Random Variable".
  • Parametric Distribution: Shows the name of the best-fitting parametric distribution if the selected type of distribution is a parametric PDF, CDF, or CCDF. Shows <not parametric>, otherwise. Note that in most cases, even with a significance level of alpha=0.001, it was not possible to fit a parametric distribution. In those cases, the value of this column shows <not possible>.
  • Statistic: This is the statistical test's value. For continuous random variables, we compute the two-sample Kolmogorov—Smirnov goodness-of-fit test [3], and for discrete random variables we use the Epps—Singleton two-sample goodness-of-fit test [4].
• Plotting Area: The plot is the main interactive user element of the "Metrics As Scores" web application. It is best utilized using a mouse. Panning the plot can be achieved by using the left mouse button, zooming by using the mouse wheel. By default, x- and y-axis are zoomed simultaneously. The plot features additional controls located on top of the right edge for zooming along the x- or y-axis separately. Each domain in the legend on the right can be (de-)selected. By hovering the plot, a crosshair is shown. It allows to obtain values for relative likelihood, cumulative probability, and scores. The labels for both axes will change accordingly, based on the distribution type and whether a transformation is selected.