Just a few thoughts on metrics … specifically software metric.  A software metric defines a standard way of measuring some attribute of the software development process or an attribute of a software component. A software metric allows us to compare and evaluate one process or component with another, and plan to improve quality of a component or improve efficiency of a process.  For example, you might ask, “what design verification activities most effectively identify software defects for my software system?”  What would you do or where would you go to answer that question?

Data Driven Process

I would suggest that you would want to have data to support your answer, whatever that answer might be.  Ideally, your data would span the entire software development lifecycle for your software system.  We need to know where and how the defect got into the software.  For example, the defect could be a missing or poorly communicated “system requirement,” and unrelated to the software coding at all.  In fact, IEC 62304 identifies three sources for software defects:
  1. Missing or poorly communicated system requirement
  2. Missing or poorly communicated software requirement or software design specification
  3. Software coding error
We might expand #3 to include:
  • Error in custom authored software, open source software or other acquired software
  • Improper use or interface with open source software or other acquired software

Using this information for example, we can address the process by which system requirements are elicited and generated from the Design Input.  It could be that the problem might be addressed with earlier review and evaluation of system requirements, or involving subject matter experts to breakdown the medical impact at a system level so that the variability from software can be critically reviewed earlier.  Certainly defects in software requirements and/or design should cause us to analyze how software requirements are generated or design is actually done (if at all) and reviewed.  Coding defects may lead to changes in coding standards.

Another useful metric is when and where was the defect found in the lifecycle?  This can help us consider methods and processes to find and eliminate defects easier in the process.  A defect found early in the lifecycle is a “gift” – it is a good thing.  Celebrate it. That is the goal.

Obviously, taking the time to properly categorize defects can provide great benefits for quality improvement. However, often I hear, “we don’t have time to properly investigate root causes.” Wisdom suggests otherwise … I advise to listen to her.
About the author

Brian Pate helps medical device companies achieve efficient and FDA regulatory compliant product development to produce higher quality and clinically valued software. He began his career in clinical research in 1985 with the Department of Anesthesiology at UAB developing closed-loop control systems for the automated delivery of gases and control. In 1990, he made the switch from university research to the medical device industry designing control systems, communication interfaces, user interface, and other software for real-time embedded systems and clinical information systems, working for medical device companies including Johnson & Johnson, Baxter Healthcare, and GE Medical. Today, he is a Partner and the General Manager of Crisis Prevention and Recovery LLC (dba SoftwareCPR®), a general-purpose regulatory consulting firm that is recognized globally for their expertise with standards and national regulations pertaining to medical device, mobile medical app, and HealthIT software. He has taught the AAMI/FDA course on Software Regulation to FDA Reviewers at FDA and is currently the lead faculty for the public version of that course taught annually along with FDA staff. Brian served on the AAMI/FDA TIR working group that created AAMI TIR32 Guidance on the application of ISO 14971 to Software (later superseded by IEC 80002-1). He later served on the original AAMI/FDA working group that created the AAMI TIR45-2012 TIR Guidance on the use of Agile practices in the development of medical device software and is currently the co-chair leading the creation of the 2nd edition of TIR45. He has served as faculty for all offerings of the AAMI/FDA Compliant Use of Agile Methods public course. Brian also served as an instructor for the AAMI Design Controls course. He is also a member of the Underwriters’ Laboratories Standards Technical Panel 5500, Remote Software Updates. He now serves as a member of the AAMI Software Committee.

Upcoming SoftwareCPR Training Courses:

Public Course – Jan 9-11, 2023 – Risk Management (in-person)

Our newly updated ISO 14971:2019 Medical Device Risk Management, A Software Organization’s Perspective public training course is now open for registration!

Where:  Tampa, Florida

  • Coverage of ISO 14971:2019, IEC 62304; amd1, and IEC/TR 80002-1.
  • System level hazards analysis – mapping to software, cybersecurity, and usability
  • Why FMEA is incomplete for medical device risk management.
  • How to perform software hazards analysis.
  • And more!

3-days onsite with group exercises, quizzes, examples, Q&A.

Instructors: Dr. Peter Rech, Brian Pate

Discount Registration through October 31, 2022.  Reserve your spot!

Register here: https://events.eventzilla.net/e/2023-softwarecpr-public-training-course–iso-14971-medical-device-risk-management-a-software-organizations-perspective-2138576610

 

Public Course – Dec 12-15, 2022 – Being Agile & Yet Compliant (virtual)

COST: 4 half days for $1,920 per person

HOURS: 11 am until 3 pm EDT each day

TRAINING LOCATION: Virtual – live online

Register here:

https://events.eventzilla.net/e/december-2022-softwarecpr-agile-and-compliant-training-course-2138573767

 

Corporate Office

15148 Springview St.
Tampa, FL 33624
USA
+1-781-721-2921
Partners located in the US (CA, FL, MA, MN, TX) and Canada.