Software Validation

chavak

Member
Join Date
Jul 2002
Posts
750
Hi Everyone,

I have been pushed into a black hole of conducting software validation for PLC programs on the machines we have and the new ones we going to built to satisfy FDA requirements(Medical devices manufacturer). Anyway the argument is a 'black box validation would be sufficient'. I believe a lot of you out there would have experienced it, but it is totally new to me. Where should I start, what are the documentations to be prepared, how to conduct the validation (for a black box type).

Another big question will be what is the difference between a black box validation and a white box validation. Where can I find infomation about those. Any help rendered is very much appreaciated.

Thank You.
 
Black box testing the term used when the test is performed by someone with no knowledge of the software. A better description is here.

White box testing is normally performed by the software designer and is normally an ongoing test during the design - get small parts of the program working rather than write the full code and then start to debug it!

The only documentation I can think of is a cause and effects chart like the one below. I hope this is enough to get you started.

Susan
 
Validation 101

Validaton starts with the Functional Spec. This document should state exactly what the system should do under any given cirmmstances, predominantly the normal, but including the abnormal.

A well written Functonal Spec makes the validation relatively simple (relative to understanding brain surgery, that is). A poorly written FRS ("Functional Requirements Specification") will be almost impossible to validate.

The ideal FRS is nothing more than a bulleted list of all the functoions. The ideal PLC program would have one rung for each of these functions, with little to no interaction between rungs.

I have yet to see an ideal FRS or PLC program (but I'm trying!)

In black box testing, you write a Test Plan based on the functional spec, without knowing anything about how the PLC and the rest of the system is going to accomplish the function (good code, spaghetti code, well-trained gerbels pulling on strings - it doesn't matter as long as "Pushing the START button causes the moter to run"). You treat the PLC as a mysterious "black box" full of black magic.

In white box testing, you look under the hood, and see exactly how the PLC and test each line of code indivually and in tandem. It's almost impossible to do true white box texting, because you have to write your test to "prove" that there are no scan race issues, no conficts within the program (and I'm not just talking about double coiling here, but every kind of bug that you have ever seen.) In sort, you are trying to write a test to PROVE that your code is bug-free.

At the end of validation (and something to keep in mind from the onset since it's easier to hit a goal if you know what to aim for) is the "Tracabliity Matirx". Again in an ideal world, this is the bulleted list of the system functions, with a reference to which section of the FRS discusses it, which test in the Test Plan tests it, which SOP (Standard Operating Procedure) uses the function, and, if you really want to go overboard, which line(s) in the PLC control that function.

With the Trace Matrix (so the theory goes), if the system needs to be changed to add an enhancement or modify a feature, you should be able to tell which sections of the FRS are affected, what tests will need to be repeated, what procedures will have to be adjusted, and where the code will be changed.

Again, this is all theory. Each pharaceutical company has their own interpretation of 21 CFR, the federal regulations that address how food and drugs should be made (in a general sense). They likely will have a validation department which may have some sample docs to give you, and guidelines for filling in those docs. Some only trace the FRS to the Test, and leave it at that. Some include the SOPs.

There's another doc, the "Detailed Design Specification" in which you take every function and describe how you are going to program that function, and how you will present it to the operator (assuming an HMI). Sometimes this doc is required, sometimes it's enough to have your program listing, and spreadsheets for data table layout serve as the DDS. Some companies trace to the DDS, some not.

There is also the URS, "User Requirements Specification" which is a precursor doc to the FRS, where the pharmaceutical company spells out what they WANT, then you write the FRS saying what the system WILL DO, and the DDS saying HOW it will do it, followd by the test plans (IQ/OQ/PQ), which PROVE that what you want, what you said, and (in the case of white box testing) how it works are all there.

That should be enough to get you started. If you need some more pointers, check out GAMP.org - they've set the standard (which you can pur¢ha$e).

Good luck!
 
Last edited:
Thank You Allen and Susan,

Is FDA regulations for software validation is based on Gamp4 or vice-versa. During our discussions earlier Gamp4 was also discussed as a guideline, and we have decided to pur¢ha$e one set of it.

Another doubt I have is, do I have to do the Software Validation on IQ/OQ/PQ, or does it have to be done once. If so when should I do it, during IQ or, on or before OQ and PQ? The PLC program is going to be the same in its functionality for all the 3 phases of machine validation.

Thank You
 
Validation 101b

Is FDA regulations for software validation is based on Gamp4 or vice-versa.

FDA regulations are spelled out in 21 CFR (Code of Federal Regulations). The part that affects us controls guys most are Part 210 - Current Good Manufacturing Practice in Manufacturing, Processing, Packaging, or Holing of Drugs; General and Part 211 - Current Good Manufacturing Practice for Finished Pharamceuticals. Since you're working with medical devices, there may be other sections that apply (Part 26, for example, although that applies to everybody too).

The Regulations are mostly legalese and definition of terms. When boiled down, they amount to three "simple" rules:

1) State up front what the system will do (and document it).
2) Make a system that does what you say it will (and document it).
3) Prove that the system does what its supposed to (and document it).

And then, once it's all working, don't change ANYTHING. Whatever you have done to make the drug come out the way it does the first time, that's what you have to do always, regardless of how silly or awkward it is to repeat it that way. And it has to be all down in writing, so that 1) someone new can be trained on the system, and 2) the FDA inspector can verify that this is what's going on.

Well, this is all well and good, but just how do you go about implementing such a structure? Well, way back when, the pharmaceutical companies wanted to know that too. So they asked the government (who wrote the rules) to explain what the rules mean. The answer (like the Supreme Court's definition of pornography) was "We won't tell you what's good, but we'll tell you if it's bad."

Well, since that was no help, the pharaceuticals kicked in some money and formed GAMP to set up a standard by which they could all live by. They figured (righly) that if they each came up with their own interpretations, the goverment inspectors would play one company off another ("We like how you did THIS, but that other company did THAT better. So we're going to fine you.") which would cost them money.

And the FDA inspectors were happy because it gave them some guidance to the "I'll know it when I see it" policy. So while GAMP (now up to revision 4) isn't the de jure, it is the de facto standard (See, Ken, you're not the only one who speaks latin around here)

[D]o I have to do the Software Validation on IQ/OQ/PQ, or does it have to be done once. If so when should I do it, during IQ or, on or before OQ and PQ?

Definitions:
IQ - Instrument (sometimes Installation) Qualification
OQ - Operational Qualification
PQ - Process Qualification

Each pharmaceutical does it a little differently and so again I say "Check with the client", but the most common method is this:

IQ - verify that each device installed a)matches the one specified (part/model number; tag nameplate, etc), b)that it functions as specified (power requirements, output ranges, sensing capablities, calibration certificates, etc), and c) that it is installed correctly (wire numbers, wires have continuity to terminal blocks to PLC). This will include the hardware aspects of the PLC and HMI.

OQ - verify that the PLC/HMI can do everything that the specifcation says they can, from testing each individual device/sensor to running a full blown recipe (usually using only water as raw materials).

PQ - verify that the drug that is to be made is the one that is actually made. That the CIP system actually does sufficiently clean the vessel. Verify that the SOPs are correct and sensible.

In short, the IQ tests the hardware (and the electrical contractor), the OQ tests the software (and the systems integrator), and the PQ tests the system as a whole (and the chemical engineer/process designer).



That's one way. I have also seen the OQ as a test only of the signals and scaling of the PLC/HMI (forcing individual outputs, verify screen displays), and the PQ will be for the coordinated system (phases, recipe running). Where the HOA logic gets tested can get fuzzy - that's why I like the first approach.

Any more questions?




In reviewing my post, I meantioned 21 CFR Parts 210 and 211, but forgot to talk about the infamous 21 CFR Part 11 - Electronic Records / Electronic Signatures. How are you addressing THOSE issues?(it's not covered in GAMP4)
 
Last edited:
There should be always be some sort of change control procedure implemented as part of a validation plan. That way modifications can be made after the validation process without too many headaches. That way if you find a process task is difficult to perform, you can modify the code with minimal effort. One example would be in the case a SLC or PLC5 program, the procedure should say a compare will be performed to prove no code has been modified except what was intended. As long as the change is documented and proven with more documented testing (IQ/OQ for hardware CHANGES + PQ), there is no reason you can't change the program after validation.
 
But flippancy is the norm here

Yes, there must be a change control procedure in place. That sort of thing is usually covered in the QAP ("Quality Action Plan") document, or some other document that covers, from a company-wide perspective, how the company handles changes.

I was a little flippant with my comment "once it's all working, don't change ANYTHING", but I wanted to stress to someone who isn't familiar with validation that they can't just be making on-the-fly bug fixes, improvements, enhansements, etc. that many are used to making.

I've got one pharmaceutical client who has PLC with a bug in it. I found the bug 6 months ago (although they'd been living with it for much longer). The bug is still there. The solution is simple - two rungs of ladder logic need to be swapped (the bug is a simple scan-race issue). But in order to make that 5 minute change, they need to write up a change order (stating the problem, the impact, the solution, the test to prove the solution, and any other affected documents/systems), get it signed off by at least 5 people (Quailty, Safety, Production, Engineering, and ???) along with the proposed change to the SOP (since they have a work-around for the bug, they'll have to skip that step) all before I can be called in to make the fix.

So in that respect, I stand by my statement: DON'T CHANGE ANYTHING.

Ain't Validation FUN?
 

Similar Topics

I am curious if there are any other people on the forum that work for FDA regulated companies? The reason I ask is that we are, and being such, we...
Replies
4
Views
3,703
Hi!! I'm looking for Temperature rise calculation software from Rockwell, I just download "Product selection toolbox 2022" but this software is...
Replies
1
Views
109
hi all i am new here i have a mitsubishi smart touch hmi i.e ms-60t-pe but i cannot find a software to edit and download a program in it any help...
Replies
3
Views
97
I HAVE SMART TOUCH MS-60T-Pe MITSUBISHI HMI BUT I CANT FIND A SOFTWARE TO EDITE AND DOWNLOAD A PROGRAM IN IT.......CAN ANYONE HELP PLEASE!
Replies
0
Views
46
Hi, Do you have any trick to make a software working when the 30 days demo version is expired? I've tried to uninstall/clean the registers etc...
Replies
6
Views
244
Back
Top Bottom