Software Measurement and Metrics

Software measurement and metrics close out the quality management module. Earlier sessions built the context: the process-based quality approach adapted from manufacturing, the role of standards, reviews and program inspections, and quality management in agile development. This session covers product metrics, how measurement data is analyzed, and software analytics. Process metrics and process measurement follow in the next session, together with software process improvement.

The session moves in a natural arc: first, why measurement matters and why it is hard (measurement turns quality from an opinion into a number, but the link between what we can measure and what we want is only an assumption); second, the product metrics themselves — the static metric tool kit (size, cyclomatic complexity, identifiers, nesting, the fog index) and the object-oriented CK suite; third, the process of measuring a real system — the five-stage product measurement process; fourth, how to read the numbers you collected without fooling yourself — data is never self-explaining; and finally, software analytics, where big data techniques are pointed at software data. The exam guidance and industry applications are collected at the end.

12.1 Measurement in Software Engineering

To see where measurement fits, recall the module so far. The process-based approach to quality management was adapted from the manufacturing industry, and a quality culture is needed to maintain quality for standards that a process-based approach alone cannot deliver. Standards establish good practices and the wisdom gained from past experience; they allow continuity when different teams work on a project, let the organization share corporate knowledge, and provide a framework for process and product standards. The ISO 9001 standards framework was one of the reference points. Quality assurance draws on reviews and program inspections, which complement execution-based testing. In agile development, quality management leans on team culture and informal approaches: standards-based, document-oriented quality management suits large plan-driven projects, while smaller teams need a lighter, culture-based system. Even agile projects should periodically set aside one of the sprints to develop documentation for the project so that some documentation and standards are maintained. Within this picture, software measurement supplies the quantitative evidence.

12.1.1 Why Measurement Matters

Measurement is fundamental to any engineering discipline. Without measurement, you cannot compare your work against a benchmark or a standard — measurement is what turns quality from an opinion into a number. A metric — a way to assign a value to a property of a system or to a quality attribute — helps you identify components whose quality is substandard or anomalous. Some attributes of software can be measured directly; these are the internal attributes: cyclomatic complexity, lines of code, depth of inheritance, program size, and the number of error messages, some of which can be counted and some computed easily. Internal attributes can be used, indirectly, to relate to the external quality attributes — maintainability, reliability, usability, reusability, portability, safety, security, and so on. The module earlier listed about fifteen quality attributes in all.

A useful way to keep the vocabulary straight is to separate three related ideas, each one a step up in abstraction:

• A measure is one raw quantitative observation — for example, the number of errors found in one component during review.
• A metric relates several measures — for example, the average number of errors found per review across a project. The IEEE Standard Glossary of Software Engineering Terminology defines a metric as "a quantitative measure of the degree to which a system, component, or process possesses a given attribute."
• An indicator is a metric or a combination of metrics that gives insight into the process, the project, or the product — insight you can act on, such as "this component needs rework before release."

In this session the word metric is used for the assignment of a value to a property of the product; the terms metric, measure, and measurement are often used loosely in practice, and that is fine as long as the underlying idea — a number assigned by a defined rule — stays the same.

12.1.2 Internal Attributes and External Quality Attributes

Several points about the internal–external link are worth stating carefully. First, many external qualities are emergent properties: they can be observed only when the complete system is executed, not by reading the code. Second, it is impossible to draw a precise mathematical relationship between internal attributes and external attributes. Empirical models exist that formalize and validate an interrelation between the two, but even so, it is quite difficult to relate measurable internal attributes to subjective external qualities that are perceived by users and depend on user experience and on requirements. Third, the relationship varies with the development process, the technology, and the type of system being developed. So when a product metric is said to relate to an external attribute, that link is only an assumption.

This is also why the discussion of a measurement program comes with a warning label: the link between an internal metric and an external quality attribute depends on the development process, the technology, and the type of system being developed. A cyclomatic complexity of 20 in a safety-critical embedded controller means something different from a cyclomatic complexity of 20 in a one-off internal report generator.

12.1.3 Why Measurement Is Hard in Industry

Measurement is a challenging aspect of software engineering in industry, and most companies do not run an extensive metrics program, mainly because they are not sure of the return on investment. There are no standards for software metrics. Most of the available models — even the mathematical ones — are empirical observations made on plan-driven projects over a long period, so they may not apply to agile methods or to the more recent reuse-oriented development. Company processes are often not standardized, and code metrics from plan-driven environments may not transfer to modern development such as ERP systems or commercially available off-the-shelf products. On top of all this, an extensive metrics program adds real overhead to the process: data collection, documentation, and the rest.

12.1.4 Empirical Software Engineering and Benchmarks

Measurement and metrics are the basis of empirical, or experimental, software engineering. In this research area, experiments on software systems and the collection of data about real projects are used to form and validate hypotheses about software engineering methods and techniques. But research data cannot be applied to a project directly, as a benchmark. Research studies often use generic products available to a large number of people — think of a widely sold product like Microsoft Office — and data available from open source projects cannot be compared with data specific to an internal project. The circumstances differ: the team, the requirements, the business considerations, and the team's competence all vary from one project to the next, so it is difficult to relate generic research results to a project that differs from the research study.

Put concretely: if a research study reports that the average cyclomatic complexity of modules in a widely used product is 6, you cannot conclude that your modules should average 6 too. Your team size, your requirements, your deadlines, and your technology stack are all different. Generic research results give you a direction of thinking — which attributes tend to correlate with which problems — not a target number to match.

12.2 Product Metrics: Dynamic and Static

12.2.1 What Product Metrics Are

Product metrics, also called predictor metrics, quantify the internal attributes of a software system. Typical internal attributes include the depth of the inheritance tree in an object-oriented design, the cyclomatic complexity of a program or module, program size in lines of code or thousands of lines of code (LOC, KLOC, and related counts), the number of error messages produced during a review or a testing process, and the length of a new software manual — plus many other measurable attributes. These metrics can be related to external qualities only by assumption: if cyclomatic complexity is high, the product may be difficult to maintain because it is difficult to understand. There is no clear, consistent relationship between measurable internal attributes and external attributes such as maintainability, because the relationship depends on the development process, the technology, and the type of system being developed.

A quick note on program size conventions, since they appear throughout the session: LOC means lines of code, and one KLOC is a thousand lines of code (1000 LOC). Saying "the system is 250 KLOC" is shorthand for "about 250,000 lines of code." Size is the simplest internal attribute to count, which is why it appears so often in both product and process metrics.

12.2.2 Dynamic Metrics

Product metrics fall into two classes. Dynamic metrics are collected by measurements made of a program in execution. They can be collected during system testing, or after the system has gone into use — for example, during acceptance testing in the user's environment. Examples of dynamic metrics are the number of bug reports raised during a testing process and the time taken to complete a computation.

The key word is execution: a dynamic metric does not exist until the program runs. The same source code, measured statically, gives the same size and complexity numbers whether it runs well or badly; a dynamic metric is where behavior enters the picture. This is why dynamic metrics are the ones that speak about how the software actually performs for its users.

12.2.3 Static Metrics

Static metrics are collected by measurements made of representations of the system: the requirements specification, the high-level design, the detailed design, the code, or the documentation. Many different static metrics have been proposed, and many experiments have tried to derive and validate a relationship between them and quality attributes such as system complexity and maintainability — but none of those experiments has been conclusive. By and large, though, program size in lines of code and cyclomatic complexity appear to be reasonable predictors of understandability, system complexity, and maintainability: the more complex the system, the harder it is to understand, and the harder it is to change or maintain.

12.2.4 Which Quality Attributes Each Class Predicts

A clear relationship exists between dynamic metrics and software quality characteristics. Because it is fairly easy to measure the execution time required for a particular function and the time required to start up a system, these measurements relate directly to system efficiency. Likewise, the number of failures and the type of failures over a period of time can be logged and related directly to the reliability of the software — how reliability is measured over a long period of usage, including mean time between failure, comes later in the course. So dynamic metrics deserve a central place in quality measurement. Static metrics, by contrast, have an indirect relationship with quality attributes: they help assess the complexity, understandability, and maintainability of a system or its components by correlating internal measurements with externally observable properties. In short, dynamic metrics speak to performance, efficiency, and reliability while the system runs, and static metrics speak to complexity, understandability, and maintainability of the code and its representations.

When to pick which: use dynamic metrics when the question is about behavior in use (efficiency, reliability, failures); use static metrics when the question is about the code itself (how hard it is to understand, change, and maintain). Most serious quality programs use both — static metrics early to flag risky components, dynamic metrics later to confirm how the running system behaves.

12.3 Static Product Metrics in Detail

12.3.1 Fan-in and Fan-out

Two design-time metrics appear especially in design work. Fan-in is the number of functions or methods that call another function or method. Fan-out is the number of functions that are called by a function x. A high fan-out value suggests that the overall complexity of module x may be high, because of the complexity of the control logic needed to coordinate the called components and to share parameters among them.

The mirror case is worth noting too: a high fan-in means that many other components depend on module x, so x is tightly coupled to the rest of the design. A change to x then has wide knock-on effects — every caller has to be rechecked. Both numbers measure coupling, but from opposite sides: fan-out counts what x depends on, fan-in counts who depends on x. Keeping both low is the classic "high cohesion, low coupling" design goal, expressed as numbers.

Visual intuition: picture a call graph, a diagram of the program's components where each component is a box and each arrow points from a caller to the component it calls. Fan-in of a box is the number of arrows pointing into it; fan-out is the number of arrows pointing out. A box with many outgoing arrows (high fan-out) is a traffic controller — it must coordinate everything it points to. A box with many incoming arrows (high fan-in) is a shared utility — everyone relies on it, so it must be stable. The takeaway: either number unusually high is a design smell worth investigating.

12.3.2 Length of Code

Length of code — the number of lines of code — is the measure of the size of a program. Generally, the larger the code or component, the more complex and error prone it is likely to be, and length of code has been found to be related to predicting error proneness in components. The class was asked a practical question about big code bases:

This exchange carries the real intuition behind length of code: size predicts effort and risk not because big code is automatically bad, but because understanding is the bottleneck. A million-line codebase with no design documentation defeats you before you write a single line — you cannot see what is connected where. That is exactly the situation the length-of-code metric is pointing at: larger components demand more effort to understand, and error proneness grows with size.

Real-world: legacy code bases of millions of lines are exactly the situation where the length-of-code intuition shows its value — without design documentation, understanding what is connected where is the real bottleneck. In practice, teams apply this metric by flagging unusually large components for refactoring or for extra review, long before the component fails.

12.3.3 Cyclomatic Complexity

Cyclomatic complexity — also called control complexity — is the more interesting of the static metrics, and it is usually computed during white box testing. It is a measure of the control complexity of a program: the number of branches, that is, conditional statements in a module, plus the number of loops. The professor's own plain-language description: it is computed using a simple formula — the number of predicate nodes or control statements plus one. A predicate node is a branch: an if-then branch in the code. Writing the cyclomatic complexity as $V(G)$ (the standard graph-theory label) and the number of predicate nodes as $P$:

$$V(G)=P+1$$

where $V(G)$ is the cyclomatic complexity of the program's flow graph $G$, and $P$ is the number of predicate nodes — decision points such as if, else if, case, or loop conditions, each of which can send control down two or more paths. Every symbol is a count, so the result is a whole number: a program with no decision points at all has $V(G)=0+1=1$ — a single straight-line path.

Worked example — minimum white box test cases. Suppose a module contains 4 conditional statements. Then the number of predicate nodes is $P=4$, and the cyclomatic complexity is:

$$V(G)=P+1=4+1=5$$

Visual intuition: draw the module's flow graph — start node at the top, end node at the bottom, and each if statement as a diamond-shaped predicate node with two arrows leaving it (true path and false path). The loop back is a virtual arrow from the end back to the start that makes the graph one closed region. Count the enclosed regions of this picture: that count is $V(G)$. For the example above the graph has exactly 5 regions. Converting the program into this flow graph helps you assess which branches or which parts are to be tested, and lets you record the test cases or results accordingly. The takeaway: each region is one test path you must not skip.

Cyclomatic complexity is also related to program understandability: programs with high cyclomatic complexity — many nested conditional statements — are not easy to understand, and the component may also be large. A question came up about its use in practice:

12.3.4 Length of Identifiers

Length of identifiers is a simple but telling metric. Normally, variable names, classes, methods, and function names should be long enough to indicate which module they belong to and what they measure, instead of being called x or y. The longer the identifiers, the more likely they are to be meaningful and understandable — for the developers and for those who will maintain the code later. The class was polled on the practice:

The metric captures a habit every programmer meets: computeArea() tells you what the function does; c() tells you nothing. The cost of a longer name is a few keystrokes; the benefit is that every future reader — including your future self — does not have to decode the code to rediscover its meaning.

Real-world: the same habit applies to file names, class names, and even attachments — a name that carries the lecture number and date is easier to sort and recognize than a generic one. Version-controlled codebases make the same point: commit messages and branch names that say what they mean save a team hours of archaeology.

12.3.5 Depth of Conditional Nesting

Depth of conditional nesting is a measure of the depth of nesting of if statements in a program. It is independently calculated, though it is related to cyclomatic complexity. Deeply nested if statements are hard to understand and are potentially error prone: the higher the depth of conditional nesting, the more difficult the code is to understand. The class had met this in practice:

Where cyclomatic complexity counts how many decision points exist, depth of conditional nesting counts how many decisions deep the logic goes at its worst point — an if inside an if inside an if has nesting depth 3. Two programs can have the same cyclomatic complexity but very different nesting: ten sequential if statements (depth 1, easy to read) versus ten nested ifs (depth 10, a maze). Depth is the metric that catches the maze. The standard fix for high nesting depth is early exits and helper functions that flatten the decision tree.

Visual intuition: draw the code as an indentation tree, with each nested if one level deeper. Depth of conditional nesting is the number of levels in the tallest branch of that tree. The takeaway: a tall, skinny tree is much harder to read than a wide, shallow one.

12.3.6 Documentation and the Fog Index

Documentation can also be measured. Some documents are difficult to read simply because they are verbose: instead of a crisp set of bullet points, the text runs to pages. The fog index is a measure of the average length of words and sentences in documents; the higher a document's fog index, the foggier and the more difficult it is to understand. No formula was stated for it in the discussion — the idea is the measure itself.

For reference, the standard version is the Gunning fog index, which turns the same idea into a number. With $w$ the number of words in the sample, $s$ the number of sentences, and $l$ the number of long words (three or more syllables, excluding proper nouns and familiar compound words):

$$\text{Fog}=0.4\times \left(\frac{w}{s}+100\times \frac{l}{w}\right)$$

The first term punishes long sentences, the second term punishes long words. A fog index of 12 means the text demands roughly a high-school-graduate reading level; values above 18 are a warning that the document will not be read. The lecture's point stands even without the arithmetic: a high fog index means high reading difficulty.

Clear and concise writing has long been expected in reports and documents: understandability comes from the crispness of the content rather than its length. Long documents with long words and long sentences are hard to follow, and nobody has the time to read them these days, while a crisp set of bullet points is easier. This connects back to the quality management module, where a quality plan should be short and usable — otherwise nobody reads it and the purpose of the quality plan is defeated.

12.4 Object-Oriented Metrics: The CK Suite

12.4.1 The CK Suite

While the static metrics above apply to any program, object-oriented programs bring their own set of metrics. Two software engineers — Chidamber and Kemerer, whose 1994 paper proposed the metric suite commonly called the CK suite — proposed these object metrics almost 30 years ago, and even now these metrics are used to assess the quality of object-oriented programs. The suite measures weighted methods per class, depth of inheritance tree, number of children, coupling between object classes, response for a class, and lack of cohesion in methods. A few of these metrics are widely applicable; still, they are mostly for academic purposes, and in practice you rarely come across serious calculations of all of them.

The six metrics split naturally into two groups: three that describe the shape of a class hierarchy (weighted methods per class, depth of inheritance tree, number of children) and three that describe how classes interact (coupling between object classes, response for a class, lack of cohesion in methods). Some UML design tools collect these values automatically as diagrams are created, which is how the metrics reach industrial use without hand calculation.

12.4.2 Weighted Methods per Class

Weighted methods per class (WMC) is the number of methods in each class weighted by the complexity of that particular method: a simple method may have a complexity of 1, and a large, complex method a much higher value. In sum form, with $c_i$ the complexity of method $i$ and $n$ the number of methods in the class:

$$\text{WMC}=\sum _{i=1}^n{c}_i$$

where $\text{WMC}$ is the weighted methods per class, $n$ is the number of methods declared in the class, and $c_i$ is the complexity assigned to method $i$ — for example, 1 for a simple getter, a higher number such as 4 or 5 for a method with many branches. The metric counts each method exactly once, and the sum runs over all $n$ methods, $i=1,2,\dots ,n$.

A larger value of this metric means a more complex object class, and complex objects are more likely to be difficult to understand, may not be logically cohesive, and cannot be reused effectively as super classes in an inheritance tree. The exam-style question is direct: given a class diagram with methods and their complexity weights, sum the weights to get WMC.

12.4.3 Depth of Inheritance Tree

Depth of inheritance tree (DIT) represents the number of discrete levels in the inheritance tree where subclasses inherit attributes and operations, or methods, from their super classes. The deeper the inheritance, the more complex the design, and the more object classes may have to be understood in order to understand the object classes at the leaves of the tree. In deep class hierarchies, each subclass may modify some of the attributes and operations, so you cannot go by the attributes and operations defined in the parent class at the root of the tree. Deep inheritance makes the code difficult to understand and so difficult to maintain, because the subclasses have modified the inherited attributes and methods to suit their own functionality.

Visual intuition: draw the inheritance tree as a root class at the top and each subclass one level below its superclass. DIT is the number of levels between the root and the class you are looking at — the class at the root has DIT 0, its direct subclass DIT 1, and so on. Each level means another layer of inherited attributes and operations that may have been overridden. The takeaway: at the leaves of a deep tree you can no longer trust the root's definitions, so understanding a leaf means walking the whole path back to the root.

12.4.4 Number of Children

Number of children (NOC) is the number of immediate subclasses of a class, and it measures the breadth of a class hierarchy — just as depth of inheritance tree measures its depth. Picture a tree with too many branches at the root level. A high number of children may indicate greater reuse, and it means more effort should be made in validating base classes, because of the number of subclasses that depend on them.

The two hierarchy metrics are complements: DIT measures how deep the tree grows, NOC measures how wide it grows at any one node. A base class with NOC = 12 subclasses is heavily reused — good for reuse, but every defect in the base class propagates to twelve subclasses, which is why the base class deserves the most careful validation in the project. An exam-style computation from a class diagram is just counting: DIT is the path length from the root, NOC is the number of direct subclass arrows leaving the class.

12.4.5 Coupling, Response, and Cohesion

The suite also includes coupling between object classes, response for a class, and lack of cohesion in methods (LCOM). Like the rest of the suite, these cannot be used directly to measure external quality attributes such as maintainability, usability, and reusability.

• Coupling between object classes (CBO) counts how much coupling exists between classes — how often methods of one class use methods or instance variables of another class. High CBO means classes are highly dependent, so changing one class is likely to affect many others.
• Response for a class (RFC) counts the number of methods that could potentially be executed in response to a message received by an object of that class — the class's own methods plus the methods it calls. A high RFC means a class has a large behavioral surface, which goes with complexity and error proneness.
• Lack of cohesion in methods (LCOM) is computed by considering pairs of methods: it is the difference between the number of method pairs that share no attributes and the number of method pairs that share attributes. High LCOM means the class's methods work on disjoint data — a sign that the class may be doing several unrelated jobs and could be split. The metric exists in several variations, and it is not clear that it adds information beyond the other metrics.

12.4.6 Do OO Metrics Predict Quality?

Across the suite, there is still not enough evidence to relate these object-oriented metrics to external quality attributes, even though many empirical studies have been carried out. Things have not really changed over the last three decades: the metrics are used and studied, and researchers keep trying to relate them to the external attributes of the product, but we still do not know how to use the measurements of object-oriented programs to draw reliable conclusions about their quality. Quality itself is subjective, and these properties are even more subjective. Metrics can be taught as an academic exercise — you can go deep into metrics, calculate a lot of them with statistical tools, and even use automated tools to compute some of them — but the relationship between measurable internal attributes and observable external qualities is still not well defined.

Real-world: these metrics apply to everyday object-oriented development in Java, Python, C++, or any other language; if you have taken an object-oriented programming course, you may be interested in calculating some of these metrics on your own programs to understand what they mean for your own reasons. In industry, automated analysis tools compute the CK suite over entire codebases, and teams use the numbers to target refactoring and testing effort — while keeping the evidence gap in mind.

12.5 The Product Measurement Process

12.5.1 The Five Stages

Measurement is fundamental to any engineering discipline, and without measurement you cannot compare anything against a benchmark or a standard. The measurement process is part of a software quality management process. Product measurement proceeds through a sequence of activities: it can be drawn as a V-shaped diagram, but in practice it is a linear sequential model. Each system component can be analyzed separately using a range of metrics, and the metric values can be compared across different components, and perhaps against historical measurement data collected from previous projects. An anomalous measurement — one that deviates significantly from the norm — usually indicates a problem with the quality of that component. The typical stages are: choose the measurements to be made, select the components to be assessed, measure the component characteristics, identify anomalous components, and analyze the anomalous components.

Visual intuition: the process is often drawn as a V-shaped diagram — the five stages running down one side and up the other — but in practice it is a linear sequential model: you finish stage one before starting stage two. Think of a filter funnel: the first stages narrow thousands of components down to a few suspicious ones, and the final stage decides, one by one, whether each suspicion is real.

12.5.2 Choose the Measurements to Be Made

First, choose the measurements to be made. The questions that the measurement is intended to answer should be formulated first, and the measurements required to answer those questions should be defined; measurements that are not directly relevant to the questions need not be collected. This is the essence of the goal-question-metric approach, which is covered in more detail later. A medical analogy carries the point: a doctor prescribes a certain set of tests to assess the various aspects of the patient's medical condition, and many parameters that are not relevant to the questions at hand are excluded.

The trap this stage prevents is collecting data "because it is easy to collect." If the question is "which components are hardest to maintain?" then cyclomatic complexity, length of code, and depth of conditional nesting are relevant; the number of pages of documentation may not be. Starting from the question keeps the later stages cheap and the results interpretable.

12.5.3 Select the Components to Assess

Next, select the components to be assessed. You need not assess metric values for all the components of the software system — there will be thousands of them. You can select a representative selection of components for measurement, which allows an overall assessment of system quality. At other times, you may wish to focus on the core components of the system that are in almost constant use: the quality of these core components is more important than the quality of components that are infrequently executed.

Two selection strategies, two purposes: a representative sample gives a fair overall picture of the system (the equivalent of a survey), while a core-component focus concentrates effort where problems hurt most (the equivalent of guarding the bridge, not the walkway). A defect in a component executed once a month costs little; a defect in the login component costs everything.

12.5.4 Measure Component Characteristics

Then measure the component characteristics. The selected components are measured and the associated metric values are computed. This step normally involves processing the component representation — the design, the code, and so on — using an automated data collection tool. The tool may be written specifically for the purpose, or it may be a feature of the design tools already in use.

The word automated matters: computing cyclomatic complexity by hand for a few modules is fine as a classroom exercise, but a system of thousands of components needs tools. Modern IDEs and design tools ship with metric computation built in, so this stage is often just running the tool and exporting the numbers — which is also why the earlier stages matter: the tool produces values for every metric it knows, and stage one decided which of those values you will actually use.

12.5.5 Identify Anomalous Measurements

Next, identify anomalous measurements. After the component measurements are made, compare them with each other and with previous measurements recorded in the measurement database. Look for unusually high values or unusually low values for each metric: these suggest there could be problems with the components that exhibit them. A blood test report works the same way: for every parameter there is a range of minimum and maximum values, and in the report itself the anomalous measurements — those above or below the range — are highlighted in bold-faced text and sometimes underlined. Those anomalous values point the doctor at the areas of concern and help reach a diagnosis. The class connected this to a familiar metric:

The CGPA example shows the whole anomaly logic in one picture: a threshold (the cut-off), a comparison (above or below), and a flag (anomalous) — with the honest reminder that the flag is not the verdict. Anomaly identification is deliberately mechanical; the judgment is left to the next stage.

Real-world: CGPA cut-offs above 8 for campus recruitment are a live example of setting an anomaly threshold on a metric; the same thinking transfers to software metrics like cyclomatic complexity. Quality dashboards automate the same pattern: every metric has a range, and out-of-range values are highlighted in the dashboard, exactly like bold text on a blood test report.

12.5.6 Analyze the Anomalous Components

Finally, analyze the anomalous components. When components show anomalous values for the chosen metrics, examine them and decide whether the anomalous value means the quality of the component is compromised or substandard. An anomalous metric value for complexity does not necessarily mean a poor quality component. The logic may be a very good algorithm with deeply nested statements — the most efficient way of actually solving the problem. The algorithm may be efficient and still look complex; how to assess the complexity of algorithms is something you have studied in data structures and algorithms courses. In every sphere of life, wherever you observe anomalous values, you try to correlate them with the context and with other parameters, and see whether the situation is genuinely risky. In the diabetes example, HbA1c gives the range of control of blood sugar over the past three months: between 6 and 7 is good control, above 7 is poor control, and below 6 is good, non-diabetic. A single high value may correlate with other factors and turn out to be less risky than it appears. More than the actual value, the trend of the values over a period of time may be of more interest: a random blood sugar test taken on a particular day, at any point of time with a glucometer, can show a very high or a very low value, and you cannot reach a conclusion about the patient from one outlier — you look at the trend and analyze whether the value is an exception or a sign of an underlying cause.

12.5.7 Keep a Measurement Database

Wherever possible, maintain all the collected data as an organizational resource, and keep historical records of all projects, even when the data has not been used during a particular project. Once a sufficiently large measurement database has been established, you can make comparisons of software quality across projects and validate the relationship between the measurable internal attributes and the external quality characteristics.

The database is what makes stages four and five possible in the first place: "unusually high" means high compared with something, and the something is the historical distribution in the database. Companies that skip the database get the numbers but lose the norm — and without the norm, anomaly identification degrades into guesswork. This is the organizational investment that makes the whole measurement process compound in value over time.

12.6 Interpreting Measurement Data

12.6.1 Data Is Never Self-Explaining

When you collect quantitative data about software and the software process, you have to analyze the data to understand its meaning and interpret it correctly. It is very easy to misinterpret data and make incorrect inferences. You cannot look at data in isolation; you must also consider the context in which the data was collected and the question being asked. There are situations where the measurement itself is ambiguous, and you have to understand why a particular situation has arisen.

The discipline this demands is uncomfortable because it is slow: a number asks for a story before it earns a conclusion. Who collected the value? Under what conditions? What changed in the environment at the same time? Those questions are not decoration — they are the interpretation.

12.6.2 The Help-Desk Calls Example

A worked example shows the trap. Suppose the number of faults or defects in the program drops after debugging in the next release, but the number of help desk calls increases. Two very different interpretations are possible.

Measurements of processes and products are not insulated from their environment, and they must not be read in isolation. To analyze the help desk data you need more than the count: who made the call, how the software is used, why the call was made, and whether anything external changed at the same time — a new release, a price change, a rival product, a change in the support process itself.

12.6.3 Human Factors and Productivity Metrics

The scientific approach to this problem is to reduce the number of factors that might affect the measurements made. But the products and processes being measured are not insulated from their own environment, and the business environment is constantly changing; changes in work practice may make comparisons of data invalid, because comparing a new way of working with an earlier one compares two different contexts. Quantitative data about human activities cannot be taken at face value. A programmer may be efficient on a particular day, or more creative, or may have other issues bothering him, so his performance or the quality of his code suffers and more defects appear. Productivity expectations often say a person should write a thousand lines of code in a day. Some days you are not that productive; some days you put in more effort, come up with a clearer idea, and solve the problem quickly — in fewer lines of code. A creative solution may fix the bug at the root cause with a small change instead of writing a fresh program. A question came up about measuring productivity itself:

Real-world: productivity measured by lines of code per day is still a common management shorthand, and this is why it misleads — the same argument applies to story points and other effort metrics. Any single-number measure of human output rewards writing lots of code and punishes the efficient fix; teams that want truthful signals pair such numbers with code review, defect data, and delivery outcomes rather than reading them alone.

12.7 Software Analytics

12.7.1 From Big Data Analytics to Software Analytics

Technology can support measurement and analysis through software analytics. You are already familiar with big data analytics — complemented these days by artificial intelligence and machine learning. Big data analytics means analyzing a huge volume of data to gather insights about a given situation that would be otherwise impossible to glean with normal computation. The concept started with data warehousing many years ago, and cloud-based big data analytics has since emerged as a means of discovering insights by automatically mining and analyzing very large volumes of data collected automatically. It becomes possible to discover relationships between data items that typical manual data analysis and modeling could not find. Applying such techniques to data about software and software processes is exactly what software analytics is.

The chain is simple to state: big data analytics works on huge, automatically collected datasets; software produces exactly such datasets (usage logs, failure reports, repository histories); so the same machine-learning and data-mining tool kit can be pointed at software data. The novelty is not the tool kit — it is the data source.

12.7.2 What Makes Software Analytics Possible

Two factors have made software analytics possible. The first is the automated collection of user data by software product companies when their product is used — platforms like Teams, Google, or Office. If the software fails, information about the failure and the state of the system can be sent over the internet from the user's computer to servers run by the product company. The result is a large volume of data — transaction data as well as system usage data — about individual products, collected online from millions of users and made available for analysis. The second factor is open source: developers use a lot of code from open source software hubs like SourceForge or GitHub, and the source code of open source tools is available for automated analysis and can sometimes be linked with the data in the open source repository. The class was asked about their own practice:

Real-world: failure telemetry from products like Teams, Google services, and Office is the reason software analytics is feasible at all — millions of users generate the data automatically. When a crash dialog appears on a user's screen, the report flying back to the vendor is a data point: the failure, the system state, the version, the usage pattern. Multiply that by millions of users and you have the dataset for analytics without anyone writing a single survey.

12.7.3 The Goal of Software Analytics

Software analytics has a working definition: analytics on software data, process data, and product data, for managers and software engineers and quality managers — including project managers, quality managers, and program managers. The aim is to empower the development individuals and teams to gain and share insights from their data, so they can make better decisions. The point is not to derive general theories about software development; it is to identify specific issues that are of interest to software developers and managers. Analytics aims to provide information about these issues in real time, so that actions can be taken in response to the information. Information needs differ for different users — developers, testers, and so on — and for various activities such as refactoring software, releasing software, and understanding the needs of software customers. Data mining and data analysis tools support big data software analytics; in general, you have a large number of options and must choose the right tool for the right task, which means studying and experimenting with several tools to discover which are best suited and most effective.

12.7.4 Analytics Tools: Guidelines

There are a number of guidelines for choosing such tools. Tools should be easy to use, because managers are unlikely to have detailed experience with big data analytics. Tools should run quickly and produce concise, simple reports rather than large volumes of information — think of a dashboard with neat charts compared with a long Excel file filled with data. Tools should make many measurements using as many parameters as possible, because it is impossible to predict in advance what insights might emerge from different factors and from the correlations between them. Tools should be interactive, so that managers and developers can explore the data and the analysis reports — and they are interested in different things, so the tools should be designed accordingly. Finally, tools should not be predictive; they should support decision making based on the analysis of past and current data.

12.7.5 Maturity and Limits

Software analytics is still immature. We do not yet have enough data — especially data that is specific to your organization — and it is too early to predict what effect it may have on software engineering and its processes as we look towards the rest of the decade. Our knowledge depends on data collected from large companies and their essentially generic products, and it is not clear whether the tools and techniques appropriate for large, widely used products can also be used with custom software built for a specific company. Small companies are unlikely to invest in the data collection systems and big data systems required for automated analysis, so they cannot use software analytics as extensively as a product company like Google or Microsoft — they do not have the data. There is no measurement culture, projects are usually one of a kind, and each is unique, so comparing projects is like comparing apples to oranges: it is not easy to compare the data and draw insights unless the projects share many similar attributes. In short, big data analytics has not matured in software as much as it has in other industry sectors — manufacturing, healthcare, or service sectors like banking.

12.8 Wrapping Up the Quality Management Module

12.8.1 What Quality Management Covers

Software quality management is concerned with ensuring that the software has a minimum number of defects and that it reaches the required standards of maintainability, reliability, and usability. It includes defining standards for the processes and for the products, and establishing the processes to check that these standards have been followed. Software standards are important for quality assurance because they provide a benchmark to compare your actual data against, and they identify or represent the best practices in the process and product industry — a solid foundation for building good quality software. Reviews, walkthroughs, and inspections of the software process deliverables — documents, programs, code, design — are the quality checkpoints: the review process involves a team of people who check the quality standards of the products being reviewed, and reviews are the most widely used technique for assessing the quality of the various work products in the software development process. Program inspections, or peer review, go further: a small team typically checks the code line by line, looking for possible errors, omissions, and areas of improvement, and the problems detected are then discussed at a code review meeting.

12.8.2 Measurement and the IEEE Definition

Software measurement gathers quantitative data about software and the software processes, and the values of the metrics collected can be used to make inferences about product quality and process quality. In this session, the product quality metrics were the ones particularly useful for highlighting anomalous components that may have quality problems; those components can then be analyzed in more detail, as in the product measurement process. Measurement is part of what makes software engineering an engineering discipline at all: the IEEE definition of software engineering speaks of a defined and measurable product and a systematic, quantifiable approach to design, development, and maintenance — true of any engineering. This is one of the answers to the mid-semester question about the similarities and differences between software engineering and traditional engineering disciplines.

12.8.3 Where We Go Next

The next session moves from product measurement to software process improvement, including software process metrics and their analysis — the process side of the same measurement story. The product metrics of this session measured the artifact; process metrics measure the activities that produce it — effort, time, defect rates per phase — and they feed software process improvement, where the measured numbers become the evidence for changing how the team works. The measurement habits from this session — ask the question first, compare against history, distrust face-value numbers — carry straight into that material.

Exam Guidance Summary

• No numericals on the metrics are expected in the exam: not on cyclomatic complexity, and not on the object-oriented metrics. Numerical work is unlikely there.
• One plausible question type is computation from a given artifact: given a module code or a class diagram, you may be asked to calculate the weighted methods per class, the depth of inheritance, or the number of children — easy to do once you know the definitions: sum the method complexity weights, count the levels from the root of the inheritance tree, and count the immediate subclasses.
• Even where a numerical is possible, it is a direct application: cyclomatic complexity is $V(G)=P+1$ with $P$ the number of predicate nodes, so a module with four conditional statements needs five white box test cases.
• Broadly, what matters for software metrics is the purpose of metrics and the relationship between the internal metric and the external observable qualities. If you know that relationship — internal attributes are measurable, external qualities are not, and the link is an assumption — you are in good shape.
• The product measurement process and its five stages (choose measurements, select components, measure, identify anomalies, analyze anomalies) are conceptual knowledge: know the stages in order and why anomalous does not mean poor quality.
• The mid-semester question on the similarities and differences between software engineering and traditional engineering disciplines connects here: the IEEE definition — a defined and measurable product with a systematic and quantifiable approach — is one of the key answers.

Key Industry Applications

• Most companies run only a limited metrics program because the return on investment is uncertain; metrics carry overhead in data collection and documentation.
• Research data from generic products like Microsoft Office and from open source projects cannot be used as a direct benchmark for internal projects — team, requirements, business considerations, and competence all differ.
• Dynamic metrics (bug reports during testing, execution times) are collected during system testing and acceptance testing in the user's environment; they feed performance, efficiency, and reliability assessments, including mean time between failure.
• Cyclomatic complexity is a practical white-box testing tool for deciding the number of test cases per module; $V(G)=P+1$ gives the minimum basis set of independent paths to execute.
• Automated data collection tools — custom-written or built into existing design tools — compute metric values during the measurement process.
• Measurement databases as an organizational resource enable cross-project quality comparisons over time.
• The medical analogies (blood test ranges, HbA1c, glucometer readings) illustrate anomaly thresholds and trend analysis used in quality dashboards.
• CGPA cut-offs above 8 used by companies for campus recruitment show how anomaly thresholds are applied in practice.
• Software analytics depends on telemetry from products like Teams, Google, and Office, and on open source repositories like GitHub and SourceForge.
• Analytics dashboards are replacing long Excel files for managers; tools must be easy to use, fast, and interactive, and support decisions rather than predictions.
• Object-oriented metrics from the CK suite (weighted methods per class, depth of inheritance tree, number of children, coupling between object classes, response for a class, lack of cohesion in methods) still support quality assessment of Java, Python, and C++ codebases three decades after they were proposed.