Independent Verification and Validation (IV&V) is a regulatory imperative for software deployments within the life sciences industry. The FDA provides guidance on what is expected during the validation process and how to conduct due diligence for the validation process.
IEEE 1012 is the industry standard for IV&V. This industry standard provides the background, rationale, process, list of deliverables and standards for the conduct of validation exercises. A common question always comes up during my discussions with clients and prospects and that is the question of return on investment of the validation process. As life sciences firms implement validation processes for their enterprise cloud and on-premise systems, increasingly many companies are turning to outsourcing of the IV&V process to save both time and money.
Validation is traditionally thought of as a regulatory imperative and is generally accounted for in capitalized expenses or direct expenses to the business. However, as outsourcing becomes more prevalent and companies consider validation as more of a legal best practice, life sciences executives are rethinking IV&V strategies and seeking a cost-effective approach to ensuring compliance and software quality.
It is understood that releasing enterprise software with defects is risky business and costly. For software applications driving highly regulated processes deficient software can have a direct impact on product quality or other critical quality attributes. IV&V is designed to improve the quality and reliability of software systems and detect bugs/defects prior to production use.
IV&V activities should be conducted INDEPENDENTLY of the development team. Many organizations often blur the lines between the two teams but they are distinct. The figure below shows a typical development team and the correlation between IV&V activities. The IV&V team receives information from the development team
The typical V-model for independent validation and verification is highlighted in the figure below. Lean validation methods and agile processes are designed to streamline efforts, eliminate waste from the validation and improve efficiencies thereby improving ROI.
WHAT IS RETURN ON INVESTMENT (ROI) & WHY SHOULD YOU CARE?
ROI is a performance measure used to evaluate the efficiency of an investment or to compare the efficiency of a number of different investments. To calculate ROI, the benefit (return) of an investment is divided by the cost of the investment; the result is expressed as a percentage or a ratio.
A return on investment formula can be expressed as:
ROI = (Gain from Investment – Cost of Investment)/Cost of Investment
For most life sciences companies, validation is considered a regulatory imperative where the return on investment is not necessarily factored into the decision to validate. However, validation can be thought of as a legal best practice which can offer return on investment and key benefits. The key questions to consider are:
- What is the cost of the IV&V investment?
- What are the tangible and non-tangible benefits gained by life sciences companies?
IV&V along with the cost of quality associated with such projects can typically cost 5 to 10 percent of the total cost of an IT project, depending on the complexity of the project and the specific scope of QA/IV&V activities.
A defect that costs $1 to fix in the requirements or design phase costs $100 to fix after the software goes into production (live) use.
Several studies indicate that the ROI on IV&V and software quality assurance investments can be 2 to 10 times the investment in QA/IV&V activities depending on how its measured. There are several studies on the market that address the return on investment for validation as highlighted below.
- Using Software Process Simulation to Assess the Impact of IV&V Activities; David M. Raffo, Umanath Nayak, Siri-on Setamanit, Patrick Sullivan, Wayne Wakeland; Portland State University,Portland, Oregon, USA
- ESTIMATING DIRECT RETURN ON INVESTMENT OF INDEPENDENT VERIFICATION AND VALIDATION, James B. Dabney (Department of Systems Engineering, University of Houston), Gary Barber, Titan Systems Corp., NASA IV&V Facility
- ESTIMATING DIRECT RETURN ON INVESTMENT OF INDEPENDENT VERIFICATION AND VALIDATION USING COCOMO-II, James B. Dabney (Department of Systems Engineering, University of Houston), Gary Barber, Titan Systems Corp., NASA IV&V Facility
ROI analysis is an important tool to understand the cost-effectiveness of your validation efforts. This is why you should care. Automation of the validation process can play an important role in maximizing your return on investment. Join us for one of our weekly webinars and let us show you how.
WHAT SHOULD YOU PAY FOR IV&V?
IV&V delivers tremendous value for life sciences companies in early detection and elimination of software defects. It is reasonable to expect IV&V cost to at least equal internal defect discovery costs or from 5- 10% of a typical project. This depends on several factors including how customized the system is. Bespoke development projects are must costlier than COTS systems that require little or no customization.
The simple cost Break Even Point is reached when:
- IV&V cost = IV&V Savings
- (1)(0.3)($x/SM)(Zsize)= (100)( Y)($x/SM)
- (0.3) Zsize = 100 Y
- (0.3)Zsize/100 = Y *SM – size metric could be expressed as Lines of Code, Function Points, …
One way to achieve greater ROI is through automation. Automated validation testing can accelerate validation and improve ROI over time. Automated testing systems like ValidationMaster™ allow you to create a reusable test script library to facilitate regression testing and improve initial testing.
CLOSING THOUGHTS
It should be understood that there is tremendous value inherent in the validation process. There are several final points for your consideration:
- No commercial or bespoke software is defect free
- Early detection is designed to save both time and money
- Reducing costs by minimizing validation may actually result in INCREASED COST during the operational phase of the system.
- Validation testing should be automated to achieve maximum ROI
