Thoughts on Metrics

Just a few thoughts on metrics … specifically software metric. A software metric defines a standard way of measuring some attribute of the software development process or an attribute of a software component. A software metric allows us to compare and evaluate one process or component with another, and plan to improve quality of a component or improve efficiency of a process. For example, you might ask, “what design verification activities most effectively identify software defects for my software system?” What would you do or where would you go to answer that question?

Data Driven Process

I would suggest that you would want to have data to support your answer, whatever that answer might be. Ideally, your data would span the entire software development lifecycle for your software system. We need to know where and how the defect got into the software. For example, the defect could be a missing or poorly communicated “system requirement,” and unrelated to the software coding at all. In fact, IEC 62304 identifies three sources for software defects:

Missing or poorly communicated system requirement
Missing or poorly communicated software requirement or software design specification
Software coding error

We might expand #3 to include:

Error in custom authored software, open source software or other acquired software
Improper use or interface with open source software or other acquired software

Using this information for example, we can address the process by which system requirements are elicited and generated from the Design Input. It could be that the problem might be addressed with earlier review and evaluation of system requirements, or involving subject matter experts to breakdown the medical impact at a system level so that the variability from software can be critically reviewed earlier. Certainly defects in software requirements and/or design should cause us to analyze how software requirements are generated or design is actually done (if at all) and reviewed. Coding defects may lead to changes in coding standards.

Another useful metric is when and where was the defect found in the lifecycle? This can help us consider methods and processes to find and eliminate defects easier in the process. A defect found early in the lifecycle is a “gift” – it is a good thing. Celebrate it. That is the goal.

Obviously, taking the time to properly categorize defects can provide great benefits for quality improvement. However, often I hear, “we don’t have time to properly investigate root causes.” Wisdom suggests otherwise … I advise to listen to her.

Thoughts on Metrics

Data Driven Process

Brian Pate

SoftwareCPR Training Courses:

Subscription Services

Corporate Office