Software quality is the degree to which software possesses a desired combination of attributes. This desired combination of attributes must be clearly defined; otherwise, assessment of quality is left to intuition. An appropriate set of software metrics must be identified in order to measure the software quality attributes.
The purpose of software metrics is to make assessments throughout the software life cycle as to whether the software quality requirements are being met. The use of software metrics reduces subjectivity in the assessment of software quality by providing a quantitative basis for making decisions about software quality. Of course, the use of software metrics does not eliminate the need for human judgement in software evaluations.
OO-SQA provides more than twenty important metrics which reflect not only the basic aspects of the software quality, but also the special aspects of the Object-Oriented software qualities. Use of these metrics will make software quality visible and result in the great improvement of software quality as well as the acceleration of the software development.
More specifically, the use of OO-SQA will:
For different projects or different organizations, the quality requirements in terms of quality factors and sub-factors may differ from each other. So it is essential to allow the selection of the metrics related to the established factors and sub-factors. Meanwhile, the importance of the metrics may be different for established factors and sub-factors. It is also necessary to set the importance of each metric for a particular project.
On the other hand, a good quality assurance tool should enable the managerial and technical personnel to obtain feedback by evaluating the software products and processes at the elementary metrics level, and analyzing the metrics values to estimate and assess the quality factors.
With Panorama OO-SQA, you may specify which metrics you wish to use for a particular project, and specify what weight, or importance, each metric has for the project. You may also specify the acceptable bounds for the values of each metric. The values which fall beyond the acceptable range will be highlighted. The resulting measurements of your program quality are displayed in five forms: Kiviat diagrams, bar graphs, metrics reports, and multiple metrics diagrams.
A Kiviat diagram is a pie-like diagram that contains radially extending lines that each represents a metric and two concentric circles that represent the acceptable bounds for the metrics. The value for each metric is shown as a point on the corresponding line. If the point falls within the bounds, it is considered acceptable.
The weighted Bar Graph shows whether the value for each metrics is acceptable. The area displayed on the Bar Graph is the product of metric value and respective weight. The weight for a certain metric reflects the degree of importance of this metric in the evaluation of the program.
The histogram displays the values for each metrics respectively in a way easy to be understood. It shows the number of classes or functions in the vertical dimension and the values of the classes and functions for the metrics in the horizontal dimension.
The Metrics Report lists the values of metrics obtained from static analysis or dynamic analysis for each class or function.
A Multiple Metrics Diagram is a variation of the Kiviat diagram. The pie in a Multiple Metrics Graph is divided into unequal slices, where each slice represents a metric, and the size of the slice represents the importance of the metric. Points representing metrics values are connected to form a polygon. The size of the polygon can be compared with the size of the goal circle.
The metrics provided by Panorama OO-SQA include:
The metrics above can be selected for functions or classes. The class metric value is the sum of the metric values of all its member function. When "With Base Classes" is selected, the metric values of member functions inherited from base classes are also added to the sum.
The eight object-oriented quality assurance metrics above can be selected for classes only.
OO-SQA can work alone or with other tools of the Panorama Family.
When OO-Test is available, the various accumulated test coverage data can be shown on the metrics diagrams. Without OO-Test, the static analysis result can also be shown on the metrics diagrams.
This section covers:
To invoke OO-SQA from the Panorama C/C++ MAIN menu:
The OO-SQA menu bar will pop up.
In the menu bar, the FOLLOW SUBSET check button is set by default. You can unselect it by clicking on the button.
If you select FOLLOW SUBSET, OO-Diagrammer will show quality analysis data of only the files you have selected in the Panorama C/C++ MAIN Menu. (See "SUBSET pull-down menu".) If you unselect FOLLOW SUBSET, OO-SQA will show the quality analysis data of all the files in the project.
The FUNCTION-CLASS SELECTION dialog box pops up:
The corresponding METRICS window will pop up.
Choose BAR GRAPH from the OO-SQA menu bar, select FUNCTIONS or CLASSES on the CLASS-FUNCTION SELECTION dialog box, then click OK. The bar graph for functions or classes will pop up.
Alternatively, you may click BAR GRAPH on the TYPE pull-down menu, a bar graph diagram for function or class will be generated.
For each of the metrics applied, Bar Graph shows the weight (the importance of the metrics) in WEIGHT column and the percentage of code passed or failed in the PASSED and the FAILED columns respectively.
Bar Graph graphically shows the values above. The size of the area for each metrics depends on the percentage and the weight. Blue bars represent PASSED values, and red bars represent FAILED values.
The Bar Graph of functions is shown below:
The Bar Graphs of classes is shown below. Eight object-oriented metrics are provided for measuring the quality of classes.
To display the Kiviat Diagram:
Choose Kiviat Diagram from OO-SQA menu bar, select FUNCTIONS or CLASSES on the CLASS-FUNCTION SELECTION dialog box, then click OK. The Kiviat diagram for functions or classes will pop up.
Alternatively, you may click KIVIAT DIAGRAM on the TYPE pull-down menu, a Kiviat diagram for function or class will be generated.
The metrics applied are listed in a Kiviat diagram. The values in MAXIMUM STD and MINIMUM STD columns are specified by the user in a standards (.std) file or by system defaults.
The data obtained from the user’s program are listed respectively in the HIGH VALUE, AVERAGE VALUE, and LOW VALUE columns: the values in blue are in the acceptable range, while the values in red are out of the range.
In a Kiviat diagram, the inner circle represents the MINIMUM STD values, and the outer circle represents the MAXIMUM STD values. Each line extending from the center of the diagram represents a metric. The value for each metric is shown as a point on the corresponding line. If the point falls between the two circles, the value is acceptable.
A polygon can be formed by connecting a set of points that represent values for a particular function or for all the functions. Blue polygon represents AVERAGE values for the program; magenta polygon represents MAXIMUM values for the program; cyan polygon represents MINUMUM values for the program; black polygon represents the actual values for each function.
The Kiviat diagram for functions is shown below:
The Kiviat diagrams for classes is shown below. Eight object-oriented metrics are provided for measuring the quality of the classes.
Choose REPORT from the OO-SQA menu bar, select FUNCTIONS or CLASSES on the CLASS-FUNCTION SELECTION dialog box, then click OK. The reports for functions or classes will pop up.
Alternatively, you may click REPORT on the TYPE pull-down menu, a report for function or class will be generated.
This report lists for each function or class its values for the metrics selected and the name of the file that contains it. Values in red indicate that these metrics values are not in the acceptable range.
The report for functions is shown below:
The report for classes is shown below. Eight more metrics are provided for measuring the quality of the classes.
To display the Multiple Metrics Diagram:
Choose Multiple Metrics from OO-SQA menu bar, select FUNCTIONS or CLASSES on the CLASS-FUNCTION SELECTION dialog box, then click OK. The multiple metrics diagram for functions or classes will pop up.
Alternatively, you may click MULTIPLE METRICS on the TYPE pull-down menu, a multiple metrics diagram for function or class will be generated.
For each of the metrics applied, Multiple Metrics Graph lists the WEIGHT (the importance of the metric), the AVERAGE metric values for the functions, and the percentage of functions whose metrics value is within (PASSED) or outside (FAILED) the acceptable bound. In the GOALS column, a set of standards values (MAXIMUM or MINIMUM) are listed.
In the Multiple Metrics Diagram, the red circle represents the goal. The circle can be either the inner or the outer bound. As in the Kiviat Diagrams, the inner bound represents the MINIMUM standard values, while the outer bound represents the MAXIMUM standard values.
The polygon is formed by connecting the points that represent the average values for the metrics applied.
The area of the polygon can be compared with the area of the red circle (normalized to the value of 100).
In this example, the inner bound is used as the goal for evaluating test coverage values.
In the following example, the outer bound is used as the goal for evaluating complexity values.
You can select from Multiple Metrics Diagram for functions or for classes.
To display Histograms
Choose HISTOGRAM from OO-SQA menu bar, select FUNCTIONS or CLASSES on the CLASS-FUNCTION SELECTION dialog box, then click OK. The histogram for function or class will pop up.
Alternatively, you may click HISTOGRAMS on the TYPE pull-down menu, a submenu of metrics pops up. Click the metric required, a histogram of the metric selected will be generated.
In the histograms generated, the vertical axis represent the number of classes for each column, while meaning of the horizontal axis depends on the individual metric.
Basic quality assurance metrics
OO-SQA provides the following metrics which are considered as fundamental for software quality assurance:
= lines of code, comments, and white spacing in each function/class.
Functions/classes are easier to understand and maintain when their sizes are small.
= Percentage of lines of code out of total lines in each function/class.
Functions/classes that do not have enough comments and white spacing can be harder to read and understand.
= Percentage of lines of comments or white spacing out of total lines in each function/class.
Comments that explain what the function/class does make the code easy to understand. White spaces help make the code easy to read.
Each function has a base complexity of 1 and each decision or loop statement (e.g. if, for, or while) adds a complexity of 1 to the base complexity.
Each N-way switch adds a complexity of (N-1).
The more complex a program is, the more difficult it is to read, understand, and maintain.
Each function has a base complexity of 1 and each decision or loop statement (e.g. if, for, or while) adds a complexity of 1 to the base complexity.
Each switch statement adds a complexity of 2.
The more complex a program is, the more difficult it is to read, understand, and maintain.
= the minimum number of instrumentation points required for recording all block test coverage (sc0) data. It is the sum of visible segments.
Test complexity measures how much effort it takes to test the function/class using a particular test coverage metrics. The higher the test complexity is, the harder to fully test the code.
= the minimum number of instrumentation points required for recording basic segment test coverage (sc1) data. It is the sum of visible segments plus basic invisible segments (if, do-while, switch, and high-end loop boundary invisible segments).
J-complexity1 covers J-complexity0.
Test complexity measures how much effort it takes to test the function/class using a particular test coverage metrics. The higher the test complexity is, the harder to fully test the code.
= the minimum number of instrumentation points required for recording all segment test coverage (sc1+) data. It is the sum of visible segments and all (basic and low-end loop boundary) invisible segments.
J-complexity1+ covers J-complexity1.
Test complexity measures how much effort it takes to test the function/class using a particular test coverage metrics. The higher the test complexity is, the harder to fully test the code.
= the minimum number of instrumentation points required for recording all condition-segment test coverage (J-Coverage) data. It is the sum of all visible and invisible segments plus all outcomes of all conditions in all the decision statement.
J-complexity2 covers J-complexity1+.
Test complexity measures how much effort it takes to test the function/class using a particular test coverage metrics. The higher the test complexity value is, the harder to fully test the code.
= The percentage of visible segments that have been executed.
Untested code may contain hidden bugs, so well-tested code are less likely to be buggy. Thorough testing is especially important for class libraries that will be reused.
= The percentage of visible and basic invisible segments that have been executed. Basic invisible segments consist of if, do-while, switch, and high-end loop boundary invisible segments.
sc1 covers sc0.
Untested code may contain hidden bugs, so well-tested code are less likely to be buggy. Thorough testing is especially important for class libraries that will be reused.
= The percentage of visible and all (basic and low-end loop boundary) invisible segments that have been executed.
sc1+ covers sc1.
Untested code may contain hidden bugs. Well-tested code are less likely to be buggy. Thorough testing is especially important for class libraries that will be reused.
= The percentage of visible, invisible, special invisible, and condition outcomes that have been executed.
J-Coverage covers sc1+. It is the strongest test coverage metrics provided by Panorama C/C++.
Untested code may contain hidden bugs. Well-tested code are less likely to be buggy. Thorough testing is especially important for class libraries that will be reused.
= The percentage of true condition outcomes that have been executed.
Untested code may contain hidden bugs. Well-tested code are less likely to be buggy. Thorough testing is especially important for class libraries that will be reused.
= The percentage of false condition outcomes that have been executed.
Untested code may contain hidden bugs. Well-tested code are less likely to be buggy. Thorough testing is especially important for class libraries that will be reused.
= The percentage of true and false condition outcomes that have been executed.
Untested code may contain hidden bugs. Well-tested code are less likely to be buggy. Thorough testing is especially important for class libraries that will be reused.
= The percentage of branches that have been executed.
Untested code may contain hidden bugs. Well-tested code are less likely to be buggy. Thorough testing is especially important for class libraries that will be reused.
OO-SQA provides a number of object-oriented metrics, including:
= The level of inheritance for a class. (0 if it has no base class)
Inheritance is a key feature of object-oriented programming. The deeper the level of inheritance, the more potential for reusing inherited functions, and the more the class takes advantage of object-oriented technology. On the other hand, deeper inheritance level may also make it more difficult to understand the behavior of the class. It is important to strike the correct balance.
= The number of classes that directly inherit a class.
Inheritance is a key feature of object-oriented programming. The greater the number of children of a class, the more potential for reusing of its member functions, which also means a greater need for thorough testing of its member functions.
= The number of couples with other classes.
A couple occurs between two classes when the method of one class calls the method of another class, and vice verse.
The higher the coupling number, the more complex and less modular the design, and the more difficult it is to be reused. To promote encapsulation and modularity, the coupling number should be kept low.
= The total number of methods in a class and the methods called by these methods. Each method is counted exactly once.
The larger the number of methods that can be invoked in a class, the more complex it is and the harder to debug and test.
= The total number of methods in a class.
The larger the number of methods, the higher the level of encapsulation, and the more it takes advantage of object-oriented technology. On the other hand, the class may also be more complex. It is important to strike a balance.
= The total number of functions that use the methods of a class.
The larger the number of method users, the better the class takes advantage of object-oriented technology. It is also more important to thoroughly test the methods.
= The total number of lines in the base classes of a class.
The more lines of code reused, the less effort it takes to code, maintain, and test this class.
This reduces the cost of development.
= The percentage of lines of code in a class that are inherited from base classes.
The more lines of code reused, the less effort it takes to code, maintain, and test this class. This reduces the cost of development.
Selecting functions or classes
You can select the functions or classes for which you wish to generate software metrics.
To select functions:
A SELECTION dialog box pops up.
You may select the functions for which the software metrics form is to be generated, then click OK. The metrics form will be generated.
To make selection, you may click to select the individual function, or you may click at one function then drag the cursor to select multiple functions.
To select classes:
A SELECTION dialog box pops up.
You may select the classes for which the software metrics form is to be generated, then click OK. The metrics form will be generated.
To make selection, you may click to select the individual class, or you may click at one class then drag the cursor to select multiple classes.
To customize the view of a Kiviat Diagram or a Multiple Metrics Diagram with its text section in a different position, click on VIEW in the METRICS window. The items in the VIEW pull-down menu only apply to Kiviat Diagrams and Multiple Metrics Diagrams. These items will become inactive in Bar graph, histograms and Reports.
Choose TOP from the VIEW pull-down menu.
Choose BOTTOM from the VIEW pull-down menu.
Choose HIDE from the VIEW pull-down menu.
Setting standard metrics values
For different projects or group of programmers, the quality requirements of the program may be quite different. OO-SQA allows you to set the standard values for the metrics to measure the quality of your source code.
The METRICS: STANDARD dialog box will pop up:
The number of metrics shown in the STANDARDS dialog box depends upon whether FUNCTIONS or CLASSES is selected in the FUNCTIONS-CLASSES SELECTION dialog box. If FUNCTIONS is selected, 16 metrics will be shown in this dialog box. If CLASSES is selected, 24 metrics will be included in the STANDARDS dialog box.
You can also click one of the other buttons in the action area at the bottom of the dialog box:
OO-SQA allows you to specify the colors of the diagrams and further customize the diagrams.
The COLORS Property dialog box will pop up:
You may specify different colors for the following eight elements:
Click OK to accept new setting and close the dialog box.
Click LOAD to open a file selection dialog box to load a set of colors from a .col file.
Click SAVE to open a dialog in which you may enter the directory and name of a file for storing the current color settings. The file name must have .col extension.
Click DEFAULT to set the color values to the default values.
Click RESET to reset the color values to the original unchanged values.
Click CANCEL to close the dialog box and discard any changes.
Use SHOW to choose which items are displayed when showing various metrics diagrams.
To set the display mode:
Press check buttons to enable or disable the kinds of polygons to be showed in the kiviat diagram. They are AVERAGE, MAXIMUM, MINIMUM, and SELECTED. The default setting are all selected.
Press radio button to choose the mode for showing REPORT.
NORMAL will show a normal report in which the metrics values for all functions/classes selected are displayed.
DEVIATION will show a deviation report in which only those functions/classes with metrics values out of acceptable range are displayed.
Press radio button to choose the mode for showing MULTIPLE MERICS.
GROW will set inner bound as the goal. This can be used to analyze, for example, test coverage. The average values of test coverage for selected functions should be higher than the MINIMUM standard values.
SHRINK will set the outer bound as the goal. This can be used to analyze, for example, complexity. The average values of complexity for selected functions should be lower than the MAXIMUM standard values.
The class metric value is the sum of the metric values of all its member functions. When "With Base Classes" on "OPTIONS" pull-down menu is selected, the metric values of member functions inherited from base classes are also added to the sum. When "Without base classes" is selected, only the values of its own member functions are added.
If the software being analyzed is running when OO-SQA is activated, the test coverage results may change from time to time. Click "Refresh test data" on "OPTIONS" menu to include the newest coverage data if generated.
To print software metrics on a PostScript printer or to a file in PostScript format:
The PANORAMA METRICS: PRINT dialog box will pop up.
If you have selected TO PRINTER, enter the device name next to that button.
If you have selected TO FILE, enter the pathname of the file in the text input area next to that button.
To exit the metrics window:
To exit from OO-SQA:
Click QUIT on the OO-SQA menu bar.