Understanding the Value of Unit Testing

Obviously, unit tests have their greatest value at the time of the development of the unit itself. Well-designed unit tests provide evidence that the unit performs its intended function, that the software design executes as intended, and allows the developer (or tester) to test the unit with inputs and states that may be difficult to produce at an integrated level.

However, once that unit is complete, running the unit test again may have very low value. Certainly it will obtain the same result when executed as a unit test. However, this seems to be at odds with the prevailing opinion that unit tests should be run daily or with every build. One might ask the question, “Why should I re-run a unit test for a software unit that has not changed?” Let’s explore this question just a bit.

Re-running a unit test provides some evidence that the software unit did not change, or at least it did not change in such a way that it affected the pass/fail criteria of the unit test. This is an important distinction as there may be some changes to units for which the unit test will not discriminate. Assuming the unit test has adequate and proper coverage, this can be an acceptable situation.

Secondly, when unit tests are automated, re-running is essentially free and it can help convince outsiders like FDA that there has been enough testing. For instance, statements like “100% unit test coverage was repeated with the final build” can be reassuring to the regulator and make it unnecessary or unlikely for them to determine the impact of some unit tests that were run on very early builds of the system. Keep in mind that the term “free” is used loosely here and does not apply to all systems.

Thirdly, you might be using a scheme where the inputs are different for each run or randomized in some fashion. Clearly regression runs of unit tests have value in this case.

Finally, you may use a log generated from a unit test for some other purpose and you need that logged data for the latest release. Rather than having to justify why the data is not from the production release or equivalent, you simply run it again.

SoftwareCPR can provide assistance ranging from validation documentation review and assessment, to hands-on risk-based V&V planning, testing, and development of unit tests automation for standalone or embedded medical device software including Mobile Medical Apps.

Alan Kusinitz provided input and review.

About the author

Brian Pate helps medical device companies achieve efficient and FDA regulatory compliant product development to produce higher quality and clinically valued software. He began his career in clinical research in 1985 with the Department of Anesthesiology at UAB developing closed-loop control systems for the automated delivery of gases and control. In 1990, he made the switch from university research to the medical device industry designing control systems, communication interfaces, user interface, and other software for real-time embedded systems and clinical information systems, working for medical device companies including Johnson & Johnson, Baxter Healthcare, and GE Medical. Today, he is a Partner and the General Manager of Crisis Prevention and Recovery LLC (dba SoftwareCPR®), a general-purpose regulatory consulting firm that is recognized globally for their expertise with standards and national regulations pertaining to medical device, mobile medical app, and HealthIT software. He has taught the AAMI/FDA course on Software Regulation to FDA Reviewers at FDA and is currently the lead faculty for the public version of that course taught annually along with FDA staff. Brian served on the AAMI/FDA TIR working group that created AAMI TIR32 Guidance on the application of ISO 14971 to Software (later superseded by IEC 80002-1). He later served on the original AAMI/FDA working group that created the AAMI TIR45-2012 TIR Guidance on the use of Agile practices in the development of medical device software and is currently the co-chair leading the creation of the 2nd edition of TIR45. He has served as faculty for all offerings of the AAMI/FDA Compliant Use of Agile Methods public course. Brian also served as an instructor for the AAMI Design Controls course. He is also a member of the Underwriters’ Laboratories Standards Technical Panel 5500, Remote Software Updates. He now serves as a member of the AAMI Software Committee.

Need Your Input!

We have been researching where and when to hold our next 3-day 62304/FDA training course and we want your feedback!
1) Does the location in the US for a public course matter to you … for example, west coast (e.g., San Francisco area) vs. east coast (e.g., Boston)?
2) What locations do you prefer?
3) Would you prefer just the 3-day FDA/62034 course or to have the option of a full week 3-day course followed by cybersecurity 1-day, and a usability 1-day?
4) What other countries would you want to see our course offered publicly?
5) What are best two months of the year for the course?  What are the least favored (not workable) months?
Please respond by email to

office@softwarecpr.com by Sep 25, 2019 if possible!

Corporate Office

15148 Springview St
Tampa, FL 33624
USA
+1-781-721-2921
Partners located in the US (CA, FL, MA, MN) and Italy.