Erez Kaminski: Hi, everybody. My name is Erez. I'm the founder and CEO of Ketryx. I used to lead AI and machine learning at Amgen, and I now help teams all over the world develop and deploy highly regulated medical software faster. When I say faster, I mean we can get teams to deploy every week or every day if they want to, or every month or every quarter, depending on the applicability for their software.
This webinar is being recorded. It'll be sent to you along with the slides following the end. And a cool thing is we're also going to send you a PCCP template that you can use if you want. And we're also happy to talk about how to integrate that template into your existing quality system. Please put your questions in the Q&A area at any time.
I can answer some of them live. Many of them will be answered asynchronously by some folks who are sitting here and helping me. If you happen to leave at any point, please do the survey that's at the end of the session. And finally, we have a link in the webinar chat for our next webinar, which is "SBOM and Beyond: FDA Compliance and Cybersecurity Vulnerability Management." So please register if you want to learn more about how to comply with all the new regulations around the PATCH Act and SBOM and so on.
So thank you, everybody, for coming today. Very excited to be here, and I see a lot of familiar names, if not faces. So thank you all. This was a very well-requested webinar, and I'll just get started. Like I said, this webinar is going to be about how to implement a PCCP, a predetermined change control plan, within a CI/CD environment in your company, both for greenfield projects and projects that have been ongoing.
We've done this with folks more than once or twice, and we ourselves, I guess in some way, do PCCPs every time we do a small incremental release. My background is I used to be a scientist. I then went to work at a company called Wolfram Research, developing Mathematica, Wolfram|Alpha, and the Wolfram Cloud. I was hired by Amgen, the large pharmaceutical company, the largest independent biopharmaceutical company, to be the head of AI for their medical device and combination product division. I went back to school after that and then started this company.
I also help write standards related to cloud development, AI, and security, whether that's 115 or 34971, which are technical reports about cloud and AI. I spend my days trying to help life science manufacturers develop and deploy software and products faster and safer. We always like to say that Ketryx is a way to make safer software faster. Before we get started with what a PCCP is, I wanted to share why we're even here in this room talking about AI. Why is the FDA launching so much new guidance around this?
I'd say the core aspect is that AI is really helping people out there in the real world. I love this example. And I see people in the room that I know who have companies very related to this. This example specifically talks about how an AI company used a program called SubtleMR that can connect to existing original equipment, MRIs like these. I think this is a Siemens machine in this case. And with this aftermarket offering, it can reduce the noise of the image and, as a result, reduce scanning time by about 60%.
Now think what a wonderful product this is. One, it provides companies with a way to increase sales and have a value-added aftermarket offering for their existing customer base. Two, it increases the throughput of an imaging facility. Right? So the hospital gets more money, but also the staff gets to do more of what they love, right, which is working with patients, getting them in and out of the machine.
Patients benefit because they spend only about a third of the time in the machine compared to the current state, and everybody around the ecosystem benefits. So everybody gets access to better medicine faster, and I think that's one of the big parts of AI in medicine. It's not just about replacing what people are doing or changing how they think about things. It's about enhancing existing processes and accelerating them and making things more... I think that's one of the true goals of AI. AI is increasing efficiency in health care, and we all know we need a lot of that.
So as a result of so many hundreds of devices being approved with AI models and AI subsystems, the FDA has decided to modify existing guidances and expectations and allow manufacturers like the folks in this room to release regulated software faster, with continuous integration and continuous deployment, assuming you set it up well and you coordinate with the FDA ahead of time. That's called predetermined change control plans. So what is a PCCP, or AI/ML PCCP, all about? It's about the ability to legally change marketed devices with less regulatory burden, assuming you've preapproved the way you're going to change with the agency. It comes down to four core aspects.
Number one is what do you plan to change? Number two is how are you going to manage your data life cycle? Number three is how are you going to perform the change? And number four is how are you going to evaluate the performance of the model given the change? Right? So how are you going to do it?
What are you going to do with the data, which is part of this whole process? How are you going to do it? And then how are you going to measure success? It's very similar to change management procedures that have always existed. Just in this case, we preapprove it with the agency to give us a lot of freedom to work.
The idea is mostly that the agency wants to see that you have control of your processes and the way the product changes. And if you do that, they'll allow you to work much faster than before. Right? So you could maybe do a version a week or a version a day or a version a month. Again, you still need to generate a DHF and all the other evidence and put it in the right folder, so to speak.
But you would be able to do this very rapidly, at scale. And I'll say that, of course, this is still based on IEC 62304. We have some great webinars on IEC 62304 and how you integrate it with tools and how you leverage Jira and GitHub for that. But I'll say that 62304 is a great standard for software life cycle management. It could be commonly used in many different industries.
We use it for our GxP software because we develop 62304-compliant software in Ketryx, and it still works for AI/ML. It's just a different specification or software item. So 62304 has these different phases, for example, software development planning. Here you can see the different activities as part of this phase and then the different deliverables.
And, of course, this might look slightly different based on your quality system. After each phase, you can do an audit of your materials to make sure you have them. Right? In the next phase, you do software requirements analysis, get all your requirements, and produce an SRS.
Then you would design your architecture and produce an architecture chart. And, of course, while this seems like a waterfall, you don't necessarily have to do it sequentially through time, but you can produce it at the end. You can work through it. Right? You can make sure you're doing this work in this way because each thread here doesn't necessarily need to be all the use cases, all the requirements.
All the artifacts should be a thread of traceability. But artifacts can be produced ad hoc, and likely should be produced ad hoc if you're trying to do PCCP and rapid change. Then we have the detailed design following the architectural design, where we explain in fine detail what the system does. We then implement the units and verify them. We integrate and test, and then we perform system testing and then, of course, release and post-market surveillance following ISO 13485 and ISO 14971.
Really, PCCP mostly touches these components. Maybe some performance requirements, if you think of your metrics as performance requirements. But it basically says, hey, why aren't you able to change this area whenever you think it's appropriate, assuming that you've done thorough testing and you understand how to control that change, the implications of that change, how to measure it, and how to automatically update it? I think this is a very powerful way to think of things, where we don't really need a ton of new guidances and standards and regulations.
What we really need is to comply with the existing regulations in a slightly different way: to change the way we think of compliance, moving from a 20th-century compliance mindset, which is how we translated the rules back then, to a 21st-century compliance mindset, which is how we're going to translate the rules now given what we know about cloud and AI and continuous integration and continuous deployment, which is a tool used by almost every company in the world. And this is all part of the software manufacturing process. So I'd say that this is a very procedural way to look at this, and this is a very functional way to look at the same thing. Right?
Under ISO 13485, ISO 14971, Part 820, whichever one of these groups of regulations, guidances, or standards you want to select, what you need to do is really plan what the device should do, have an initial risk assessment, and design, implement, build, and test it rapidly, ideally all of the time. Right? Produce artifacts like the DHF and software bill of materials, and then go to verification and validation, also part of the DHF, final risk analysis, release, distribution, and post-market surveillance. Two things are interesting here. One, because we're now working on cloud and advanced software, instead of a very small embedded device that maybe we've built completely in house, we now have thousands of different dependencies we use.
Right? If you're using TensorFlow or PyTorch, any one of these AI packages, you're likely using thousands of different dependencies as part of your software supply chain. We need to have a way to control all that. Number two is, as we deploy it into the cloud, the agency and European regulators are quite interested to understand how exactly you're doing that. And we know from audits at partners that this is starting to come up more and more: show us the path of traceability.
So if this is how it looks for software, right, what changes for AI? The things that change are quite subtle. Number one is, along with traditional passive post-market surveillance, there's a more active continuous model validation, where from a safety standpoint we want to continue to monitor the model to make sure that the error is not growing out of control. The second thing is that in this design, implement, build, test cycle, there are just more steps. The steps are very similar, but there are just slightly more of them.
So you don't just need to design your software now. You need to design your data. And you don't just need to design your data. You need to design your software still, and then your model. Of course, this is the thing that all the machine learning architects love to do, playing around with architectures, learning more about new architectures.
You then need to implement the software and train the model, right, on your data, really implement it, build the software, and then test the model and integrate it back into the software. It's an AI subsystem integrated back into the software. And this is what it means to manufacture AI for highly regulated applications. So unlike software, we just have more steps here and a slightly more active approach, I guess, to model validation.
But the real change is the fact that you don't just need to design software. You need to design data and models and then, of course, train the model and test it, ideally test it totally separately, so it's ready for PCCP. Just as a note, I'm happy to get questions on this. Some of them I'll answer live. Some of them other folks will answer.
So please feel free to post some questions. At the end of the day, how does PCCP tie to this? One is we've got to agree on the metrics. What are we trying to optimize in this model? Is it a classification model, a prediction model?
What's the score we're trying to base it on? Accuracy, F1, or any other metric your machine learning scientists find appropriate for your intended use and what you're doing with this actual model. The next part is ensuring the metrics are met. I know that sounds basic, but I think it's important to think about it. Right?
We need to separate thinking about the metrics, agreeing these are the right metrics, and then ensuring they're met with every build of the system, assuming we want to deploy it as part of a PCCP. So the idea here is, of course, we have a software system. There's a data system, a model, the AI system, and then we independently test the AI system and integrate that into the software. And, of course, we need to do certain data cleaning and data management activities to make sure that's safe. I'm going to leave this here for a second before we start getting into how to validate AI systems.
I'll say that there are three things. You need to understand the critical quality attributes, like the metrics, you need to understand risks, and you need to understand the underlying datasets. I'll just share the poll we did at the beginning of this, so folks can see. I can see that most people, right...
We're still trying to figure a lot of things out. We're beginning to implement it. We have plans to integrate it into our QMS, but we haven't done it. So many people have not really thought about this. I'm glad you're here in this webinar.
I'm happy to help, and my team's happy to help and happy to share more information, as well as the template we're going to share at the end. And then there's this last question: do you have plans to integrate it? Like, how are you actually going to do this live? I think it's very challenging, and doing it manually is against the spirit of the regulation, because the whole point of this new guidance is basically that you could do this really, really fast, and you need to validate only certain metrics or verify certain metrics.
So why don't we just do that all automatically? At least as automated as our risk appetite allows, depending on your scale and your type of device. So I see two questions here. I'm just going to read them for a second and see if we should address them live. Yeah.
Can you give an example of the type of changes that fall under a PCCP instead of just regular software changes? I think it's just more extensive than regular software changes. One is you could approve a lot of different things with the agency. It still remains to be seen what they'll allow you to do and not do. But basically, you can really substantially change your software and the architecture of the model.
Right? Replace it with a totally new model and retest it, which makes sense because model architecture is not as deterministic as software architecture, in the sense that we don't really, really know how it all works. We're mostly feeding it data and testing a lot of different aspects. So maybe putting in a different model. I think that's one aspect of this rapid change, and also the fact that the change is preapproved, so we can start changing things in a more extensive way.
And then I see another question: how do you functionally distinguish between them within your QMS? I think we're going to show that in a moment here. So I'll answer that live in a little bit. That's going to be part of the demo. How do you functionally distinguish them in your QMS?
And then the last question here, thank you, is how do you handle the situation with a third-party model? Well, in the exact same way. We're going to see how we deal with the subsystem, and I think we'll see that it's very similar to just having a third party be the subsystem instead of your own subsystem. So again, you put them under your agreement. You bring them in as you do with other subsystems of your device.
Happy to talk about that further. So coming back to how to validate AI systems, there are these three aspects you want to know. You want to understand the CQA, the critical quality attribute, in this case likely metrics for the model. What's the method you're using? What are the metrics that are appropriate for that?
One of the biggest games in machine learning is that there are a lot of metrics you could use to test different aspects of the model. And that's where your subject matter expert for AI comes in and says this is the appropriate one, or these are the appropriate ones, one or more. Right? Then potential risks. There are a lot of risks that are unique to AI, whether that's data drift or unexpected decisions or all kinds of unique cyber vulnerabilities that could be exploited in the model or in the data.
So we need to make sure that we're verifying that those risks are controlled, and then making sure that our underlying dataset, as we update it, is still appropriate for our intended population. Right? If you are thinking of serving people in certain regions with certain ages and ethnic groups, it's quite important to make sure, when we update the model, that we're not in a situation where we thought we were serving all of the US, but in reality we're only serving patients at MGH in Boston, Massachusetts, a different population group. We need to make sure we're debiasing it and looking at that.
So what does a PCCP look like functionally? Let's just go through a few steps. First, we have an approved model, and then we're trying to get to a release of a new model. Right? So we have an approved model.
It's been out there in the wild. People are using it. We're getting new data from patients. And what we want to do, like any other great AI company, is leverage that data to create a better model that can serve patients better and give them more accurate results for their disease, knowing what we know from real-world data. So the first step is we want to analyze the data and make sure the data is of the right quality.
It's for the right population that we expect. And then number two is to segregate the data into training and test data and make sure that we shuffle it correctly so it's mixed in the right way. After we separate it into the training data, the thing I'm going to train on, and the test data, what I'm going to test that the model works on, we then want to send it to the model and retrain it. Maybe it's even a new model with a new architecture, because new architectures come out all the time, especially if you're using third-party foundation models. These things get updated all the time, every week.
Right? There's a new model that's slightly better than the other model. Right? Is it ChatGPT, or is it Claude, or is it a million other things? And then we get a new model, and now we need to make sure that the model matches our acceptance criteria, the impact assessment passes, and basically our PCCP can be done, and then we release it.
We need to both make sure the model infrastructure is still validated and the model is retrained, and then we need to make sure the system is really validated and accepted. Right? Because depending on your system, the model being accurate could depend on many other things. So going forward, I want to talk for a second about what CI/CD is, and then I'll bring both of these together. So what is a PCCP and what is CI/CD?
Many developers, especially cloud developers, view this as their, I'd say, their temple almost. This is a belief system. The belief is that we need to continuously develop software, integrate it into our main branch, and then deploy it to the server. In many companies, this happens every hour or so. There are release trains that go out every hour.
If your new software code is integrated into the system and all the automated tests have passed, it gets kicked off on a train into the cloud every 60 minutes or so in most large tech companies. What happens is we have this main branch where all the development happens. A developer then takes that branch, often called the feature branch. They do a lot of work. Right?
They do their own work on this, ideally trying to tweak something small. Right? We want changes to be as small as possible to increase quality. And then they have a change request, merge it, and have a code review, as done in any other industry as well as our industry, only here it's required by law under GPSV. And then they continuously integrate it into the main branch and continuously deploy it to the production server.
And just to be clear, the reason people do CI/CD is, one, you can deploy features faster to customers, but two, it's safer. You produce a higher-quality product by making smaller and smaller changes, constantly updating the external product environment, and then getting feedback on it. So this is a really, really important concept, CI/CD. I think it's extremely challenging to do validated CI/CD, so CI/CD inside a validated environment, and I've spent the better part of my adult life trying to figure out how I can help people all around the world do this, with an amazing team of folks.
But how do the two ideas relate, PCCP and CI/CD? I'm sure the folks here noticed from this drawing that CI/CD is very much a 21st-century compliance problem, and we need to transition from how we've thought about doing compliance to how compliance needs to be done today. This is what it looks like at scale. Right? So let's just pick a place here to start the cycle.
Instead of new product introduction, I want to pick after release, deployment, and monitoring, maybe here, post-market surveillance. So we get a lot of new data about the product. We then plan our next release. We develop. We design the data.
We make sure we grab the data, make sure it's debiased, design the model, make sure the model is validated given our PCCP, configure a software supply chain, validate our software, release it, deploy, and then monitor and go all the way back. So this is how both of these ideas intersect, where the PCCP part is really tied to this. You do the PCCP more or less here, and here, making sure the software is still validated and the model has met all your metrics. And then you keep releasing it, continuously or manually. Right?
I don't think that all applications need to be released every hour, every day, every week. I work with a lot of customers who have ultra-high-risk applications, and I don't think that would be appropriate for them. But I think a huge majority of applications, essentially the lower-risk ones, the Class I and sometimes Class II that folks work on, can be deployed like this regularly, and companies, patients, and hospitals would all benefit from that. And this is just going to get more and more important as we take medicine away from the hospital and into the home for folks to be treated there. I'll stop here for a second to see if there are any questions, because we're about to go into the demo in just a few slides.
We're going to go through a PCCP and do this together. Good. So let's keep going. So how do we actually do this? I think we had one question about how you functionally distinguish them.
Part of the strategy is building the architecture. But how do you design the world you work in to allow you to do this? Because I think in the very near term, five or ten years, we're going to see two types of regulated companies in life science: the companies who can take advantage of this, with leaders who set them up to do that, and the companies who did not. The companies who did have preapproved the PCCP with the FDA and have designed their systems for this. And by the time the companies who did not are on version two, they're on version 50.
And they have a much better product with much better functionality, and it's safer, and they have fewer recalls because they're doing smaller releases at any time. So there are three things we think about. Number one is architecture, exactly like this question from Mary: how do you distinguish functionally between them?
Two is how do you enforce procedures automatically and generate evidence automatically? And number three, how do you integrate risk analysis into your configuration management, to make sure that when you're working on something, you're working on the right thing? One of the challenges is that while PCCP requires you to be very organized, most software does not look like this. Most software looks like what they call a ball of mud, where you just try to figure out how it works, and you broke something and broke something else. That's not the type of software we need in order to take advantage of this.
In fact, in many cases it's not the type of software even very forward-thinking tech companies build anymore. They're actually trying to build things in a quite organized way in order to make them more maintainable by a smaller staff. What we need is this. We need a LEGO tower, and then we need to be able to grab one brick of this LEGO tower, the AI/ML subsystem, lift everything off, change that brick for another brick, and retest that it still has the same qualities or attributes. It's very hard to do.
It's very hard to teach people to do that. But the advantage, again: people who won't be doing this will be on version two. People who will be doing this will be on version 50. So we said one major thing is to design your system with PCCP in mind. Today we're going to talk about how you could do that in a system, and I'm going to demo that system, and then we're going to talk about the other strategies.
One example of a system like this is a system that interacts with patients. Maybe it's an irregular arrhythmia notification system, and it has two different subsystems. One is this hardware subsystem. We're not going to talk about that too much. The other is this AI/ML subsystem.
We're going to address that mostly. That AI/ML subsystem is going to be changing rapidly and often. And, of course, it has different libraries or SOUP items that it uses. Before we go into that, I just want to have one more poll question. So please fill out the poll.
We'll see what people say, and then we'll get into the demo. And, of course, during the demo we're happy to get questions and answer things. What we're going to do is bring this specific system to life, and then we're going to do a PCCP together today in the next 15 minutes. I hope that will show you how this is going to look for all companies in the future, because there is a way to do it every day automatically, or every week, or every month, but with more control and safety than we have today. So I'm about to close the poll.
I'll just give it another second here because I see folks are still filling in the poll. Thank you for that. And yeah, we know that there's a lot of regulatory froth and lack of understanding of how this looks and what to do. And we're trying to understand. It's very hard to do this.
As someone who worked for a large biopharmaceutical company, I'm well aware of these challenges. Okay. So let me share these poll results, and I think we're going to see that many of us are in a very similar boat. Right? We're trying to do this stuff together.
One of the things we want to do as a company is help bring that cross-industry knowledge into different companies, to help you know how to do this and reduce the time you spend on nondifferentiated parts of your product, like the infrastructure for PCCP. But let's get into a demo now. I think I shared the results already, so that's good. And right now I'm just going to get into a demo and show you this exact system. So again, remember, there's a top-level system, or an application or product.
There are two subsystems. One of them is the software AI subsystem, which we're going to do a PCCP on. The other is a hardware subsystem. Let's just get started. So I'm now jumping into Ketryx.
All of you are welcome to try it at app.ketryx.com. I'll ask someone from the office to please share a link to app.ketryx.com. You can connect this to Jira. This is something you can do in a few minutes and try out for yourself. And what you see here is the exact diagram we have here.
Right? We have this irregular arrhythmia notification system. Its two subsystems are the hardware and the software subsystem. We're going to talk about how we're going to link these different systems up, particularly how this software subsystem is linked to this product. And then we're going to make sure we're running through a process with it, doing it right, and going from there.
So let's dive into this AI/ML subsystem. This is the Ketryx project. We have all the different modules of Ketryx, whether that's traceability or risk management and so on, or SBOM management. And I'm on the All Items screen. This is a screen that allows me to navigate all the different items.
In this case, I see that I have some requirements that are in Jira, where I can see different things. There are really three things we do with Jira. Number one is create traceability. We can see here the traceability of this item to other items based on the Jira links and links across other systems. In this case, we see it has one parent requirement and many different specifications.
Number two, we provide a means of approval. We have this approval panel, where you can sign in a Part 11-compliant way depending on your role. And number three is we provide a complete auditable history of Jira or any other connected system, including all the changes that happened to it, based on ALCOA++ requirements, as well as all the different documents generated using this configuration item, which is stored in Jira. There are other types of configuration items. For example, you can have specifications that sit in Git.
So I just clicked on this. I went to the Ketryx record, and now I'm in Git looking at another configuration item. In this case, it's a software item, and it has that CS-1 as a requirement it fulfills. We see that mentioned here in the code comment. And we do the same thing: traceability, approval, history, and so on.
So you can store items either in Jira or in Git. And it doesn't just need to be a code comment in Git. It can also be different kinds of tests, like this type of test, a Cucumber test. It could also be a Markdown file. I'll try to find a good Markdown file here.
Just a moment. No, those are still code comments. So here's an example of a Markdown file. So you can store it however you want to, and developers love this.
They love to store that information this way. It also makes it easier to change and manage that change. The second thing you can do on this screen is look at different versions. What we want to do today is approve version 1.1. First, I want to see the difference between version 1.1 and version 1.
So let's see the difference between version 1.1 and the baseline, the last approved version, version 1. And let's see only changed items that are risk controls. Well, none. How about changed items that are missing tests? Well, none.
How about new items? We have none. So the difference between version 1 and version 1.1 is that we added risks, three risks in this case: model hallucination, interrupted data connection, and so on. And we can see that this model hallucination is, again, a Ketryx record. You can see it in Ketryx.
You can download it as a Word document or a PDF. You can see it in other ways. So what we do now, after we work through this change in Jira, is go and see how we're progressing. We can go to our traceability area, and I'm going to align this by design output because I think it's going to be slightly more organized. And we can see that we have several different specifications that are in Git.
They have different verification tests. There are several different design input requirements that have validation tests. It's all very neatly organized. We can see that everything is okay and passed, and there are no errors. And we can see up top here that the traceability is 100%.
So 100% of the design inputs are covered by design outputs, 100% of the design outputs are covered by tests, and so on. If something wasn't covered, we could use this to discover that. We can also use this to find gaps in the traceability and the traceability level. So now what I want to do is make sure we see that we can have automated tests in this case, and that automated testing might be part of your PCCP.
Right? Part of viewing that metric and proving that metric. And then maybe we should look at the risks for a moment to make sure all the risk controls are met. So we're in version 1.1. We see that we have three risks: model hallucination, interruption of data connection, and model drift.
All the risks are acceptable, as we can see. We can move here and see the different risk calculations, the different mitigations, and the different residual risk calculations. And if we need a benefit-risk analysis, we could require that to happen. So we know all the risk controls. And the last thing is we can see all the risk controls for a moment and make sure they're tested in this version.
Another cool thing in Ketryx is that you can automatically ensure that all risk controls are tested as part of a release. So as part of your PCCP, you can make sure that all the risk controls, which are ideally automated tests, have run and been tested, and you can see all of them. The next thing I want to show you is the architecture of this product for a moment. We see here this automatically generated architecture diagram. We're in this AI subsystem.
It has an AI module, and it has a sensor module for reading the sensor, giving some warnings, and then doing the model inference. So we see the subsystem. We can also see it in a tree view, if we want, to see how it's structured. We can also see just the first two levels of it and not more, to understand how it's structured. And this, of course, can be printed out in your documents, again as part of the PCCP.
And now let's go to the release screen to start going through a PCCP. So I go to the release screen. Let's go to the release for version 1.1. We can see all the different aspects of this project. Right?
I have 20 controlled items. Three of them have changed recently. All my design controls are ready. All my tests are executed. All my risks are done.
No changed vulnerabilities or change impact analysis. All my risk controls are verified. I can click here to see that all my documentation is ready. Specifically, it's a shallower system. I just have an SBOM, a risk management file, an SRS, an SDS, a test plan, a testing report, a predetermined change control plan report, and a conformance report, which we're going to show later on.
And then we can see all the different milestones that are part of this project. Milestones are a way to block or monitor information. So we can say that we want to collect all the real-world evidence and do certain things, like data cleaning, make sure that data cleaning is happening, configure different things, and then configure a bias report and get ready for a release. And then we can also decide which types of artifacts we want to generate as part of this. If you notice, this is very similar to the setup of 62304 workflows, where we can create different milestones for different phases.
So we do all this, and then we generate a document, right, for the PCCP conformance. I'll generate this document for a second. We'll just open it to see what we mean by this. Here's a report. Of course, it's a templated system, so it could be adjusted to your templates.
We can see here the different test cases and the test case titles, and whether each passed or didn't pass, and a link to it. And, of course, you can add much more information than this. This is just the very basic testing, and we can automatically produce this for every release. And this is what you need to do in order to do PCCPs fast enough, especially if you want to do it for adaptive algorithms automatically. And now, after we've looked at this and the different documents, I want to go back into the higher-level system.
We just spent time in this machine learning subsystem exploring it, and now let's go one level up, back into the irregular arrhythmia notification system. So I go back to my projects and click on this system. This is the application where things are stored. Again, it's very similar. Right?
I see that it's a lot of Jira projects. I can look at its traceability. As you see, the traceability table is different. It's configured to be a system and subsystems. I could look at all the different user needs to make sure they're met.
In this case, it's a very simple demo system. It has just one use case, or one user need, different system requirements, and different subsystem requirements. If you notice, some of them come from the system we were just in, the CS-1 system, different specifications for those subsystems, and then different testing for the subsystems, as well as the system if you wanted to. Again, we can see that everything has passed, or whatever has not passed is not included in our test plan in this case, because we already have a strong baseline. And we can, of course, move between different versions to see how this changed over time, if it has changed.
Let's put ourselves back on version 2. In this case, we want to see the architecture diagram for a moment. So we see that we are in this system, the application we're in right now. There are two aspects to it, monitoring and the device administration UI. And we see that the monitoring dashboard uses this AI PCCP subsystem, particularly the AI module in that subsystem.
And then this sensor calibration manager uses some specification in the sensor. Right? The hardware system. All of this, like I said, is automatically generated and easy to see. We see the gray parts are external systems that are validated separately.
The purple is an external protocol, and everything else in red is this system. It could also be color-coded based on the risk level. After we saw that we have these different subsystems, the AI/ML subsystem among them, I want to move from here and show you how to release this. Before I go through the whole release process, I just want to show how the DHF would look. The DHF looks something like this.
If I go to the history and design history, I see that there are a lot of different documents here. These are documents from the DHF for my primary system, the system I'm in. And then here at the bottom are the different milestones, including the product milestone and the subsystem milestone, and the reference project, the report from version 1, which we can see is not released yet, unlike version 1 of the optical heart sensor. Sorry, version 1.1 and version 1.
And this would be all the different information from that, generating another file for us. Right? So we just leveraged that subsystem to create the DHF for this system, and we could do that in multiple places. So we can actually spread that model across many different systems easily. And now let's go to the release.
We want to approve version 2. That's the version that has the PCCP. We see that it's done. Right? I can see this checklist.
All my design controls, item validation, dependencies, test execution and test plan, traceability, and documents are done. The only thing that's really missing is that I don't have any approvals for the milestones, and then the final release approval. So let's just take care of those two milestones. Right? So I click on this.
I go into the milestones. It's ready to be approved. The only thing that needs to happen is I need to approve the subversion. So let's go to that subversion. Let's approve it.
If you notice, I'm in version 1.1 of the AI PCCP subsystem, unlike where I just came from, oops, which is the product I was just in, the higher-level system. So we go back into that. I approve it. It's Part 11 compliant.
I need to give my biometric signature. I've approved it. Voilà. The model is approved. I've just done a PCCP.
I've tested it all, and let's now go back. So we're approving here. Let's approve this milestone. So we've approved the milestone. Now let's approve the V&V report.
Again, you see it just blocked me because the underlying subsystem wasn't approved. So, Mary, I hope this answers your question of how we segment things. Right? That subsystem we were just working on is totally segmented away. And then let's approve this milestone.
So I'm approving the last milestone as part of this PCCP. And then let's go to the release, and we see that the release is ready to go. The checklist is done. The only thing it's waiting for is me. All of this, by the way, can be edited to reflect your organizational procedures and bring that QMS to life.
I approve it. I'm using my security key. I scan it. I approve. Fireworks. We just did a PCCP together, including creating all the DHF documents.
And what I mean by that is we can go back to the release documents, and we can see the different documents for this system, whether that's the code review or the risk management file for the system or the SRS. We can see the architecture chart for the system. I'll just open up a few different things. Oh, I probably don't have the right app for that. So let's look at the risk management file for this system.
So we see the risk management file here, the approval of it, right, the generation that just happened, and then also all the different fields and metrics and calculations, and all the different information on items related to risk, the different risk controls, and their execution. You see that in this case, a lot of my risk controls for a risk came from the subsystem. And here's what it's a risk control for, and the different tests. And now if I keep going down the DHF, I can eventually get to my AI system, and here is the underlying risk management file for that system. And again, I can tie all the risk controls into automated testing to make sure that every time I do a CI/CD build, an engineer can't break what we're working on.
And then finally, I can see the PCCP conformance report we showed earlier. Again, I'm in the top-level project. This is now part of a different DHF, of that higher-level project. Wonderful. So I will give this a moment of pause, because I know that this is a big topic for folks, and I'll see if there are any questions.
One second. I just realized my screen is a bit frozen, so maybe I'll ask someone to let me know if there are any questions I should address live, or any questions that folks can answer asynchronously. I just can't see the questions right now.
Gabriel: Hey, guys. Jumping on here to provide a question.
Erez Kaminski: Thanks, Gabriel.
Gabriel: So we have a question that says: in the demo, I didn't see anything about data analysis and management. What data did I select for training versus test versus validation?
Erez Kaminski: What a great question. It's funny, because we were just talking ahead of time about how someone would ask that. I'd say that this is a very simplified demo. What I would really do for that is have another project just for the data, and then I would do a similar procedure for the data that gets fed into this AI subsystem. So we didn't really do that, but you could do that, where data is in some cases just a software item. Right? Because it's just a code entry.
Any other questions there, Gabriel?
Gabriel: Yeah. We have another great one from the crowd. What is the process of evaluating whether a change is within the scope of the authorized PCCP, and checking whether the authorized modification protocols are followed per the PCCP? How do you label PCCP changes and non-PCCP changes within the AI/ML subsystem in Ketryx?
Erez Kaminski: So let's go there. There are a bunch of different ways to do this. Right? This is one of the challenges of building something like this that works with large companies and small companies. It needs to be quite accommodating. You could say that every release here is a PCCP release. You can also create different types of releases, like a full release or a patch release or an incremental release that is based on certain items. I'd say that most PCCP releases are more incremental in nature.
And then you can write all that information here from a scoping and baseline perspective. And at the end of the day, to know that you meet the metrics, I would say there are just certain tests. And maybe you need to create a test plan or a test protocol that has those tests as part of the PCCP and that has to pass. In my opinion, working with a bunch of companies, thinking about this for many years, and designing systems to do this, I'd say: why don't you just say that every release of the AI/ML subsystem is compliant with the PCCP? It only has automated tests that test those exact things, and that's the only way it can be done. And you tie those tests into your pipeline, so you have to run them.
Any other questions, Gabriel? And again, I'm happy to take more questions on this.
Gabriel: Yeah. Absolutely. If there are any follow-up questions on that one, feel free to add them in. One more question on change assessment: how are you able to link the significant change assessment and regulatory assessment against every software change made in Ketryx?
Erez Kaminski: Yeah. So there are all kinds of ways to do that. One is to just create documents that describe those changes and the different assessments. You can also create a regulatory project and then link it, kind of like another system that links into this system, with all the different regulations, and then you can diff the changes if you want to do that. And again, you can create fields as part of the release area that are mandatory, that say, you know, I have to write this kind of assessment or not.
And frankly, I would say that for PCCP-like changes, I would suggest modifying the SDLC to not require any significant assessments of that nature. Just have it be done automatically, unless it's a change more significant than what we defined in the PCCP. Cool. So...
Gabriel: One more. Yeah. One last question, Erez, on working with partners. How do we manage the control of components and models or algorithms that are developed by partners?
Erez Kaminski: Yeah. So it depends what kind of partner. If it's totally on someone else's server, I would say that you should be testing it routinely. And I would want to see some evidence of that, in the same way you interact with vendors as part of ISO 13485. Again, it's probably going to be SOUP if it's an AI system, because they're not developing under the same type of ecosystem that you are, in the same type of life cycle.
And then I'd want to at least qualify that incoming item as I update and change. And if you're saying, you know, I leverage this other system that's constantly changing, I would just create a bunch of automated tests to test it frequently to make sure it meets my PCCP requirements. And we have partners who basically outsource one of the subsystems, and we link into a Jira project in their vendor's hands, which helps a lot with that. Cool. And I'll take some more questions at the end.
I just want to make sure we finish on time and get enough time for questions. So, going forward: we did the demo. I just want to recap the three major strategies, I would say, you need to cover to build the foundation for rapid change. Number one is design your architecture with this change in mind. Ideally, this AI subsystem and this hardware subsystem would be connected to many different systems.
Right? Not just one irregular arrhythmia notification system. Number two is you want your procedures to be enforced automatically and your evidence to be generated automatically. Like I showed you, I can enforce certain procedures around requirement traceability, risk control verification, document generation, approval, and segmentation automatically through the system. And it can also happen as part of a push, in a CI/CD kind of way.
And that helps a lot. Right? Especially if some subsystems are much less risky than the top-level application or another subsystem. Number three is you want to think about how risk analysis is integrated into the way you manage your configuration. One way to think about it is that in the grand scheme of things, we're trying to get ready for post-market surveillance and the modifications that happen.
So we want to have everything well segmented and delineated under 62304. The other part is I want to be able to say this specification deals with a very low-risk or a very high-risk thing. And as a result, it has certain procedures that it should or shouldn't go through, and you should enforce that automatically to make sure it happens every single time. And the last thing I wanted to share is really how we think of governing this. This is a format that I've shared with a lot of different businesses.
Some of them run on this now. It's, I'd say, one way for a large organization to do AI program governance. You typically would have an AI center of excellence, where you have PCCP experts and AI validation experts as well as AI experts. They work with different business units to do this work. Different business units have different applications that get served to them. Right?
They create the models, and business units apply them to their specific needs, maybe modify them, and then run different programs. As part of those programs, you attach a PCCP and an AI validation strategy or metrics, and those get approved by the governance committee. Now what helps here is that the more this governance committee does this work, the better they will get at doing it. The more the staff at the AI Center of Excellence will get better at PCCP and AI validation. The more they sit together and think about these things, the more you'll be able to deploy it in the business units, and the more programs you'll be able to run.
And that's it for our session today. I know we have five minutes left, so I'm going to wait and answer questions, and I'll be here over time to answer even more questions. And I'll say that if you want to move faster, use your preferred dev tools, automate your DHF, and enforce your SOPs in those different tools, we'd love to talk to you about it. We'd love for you to try the product. It's free to try at app.ketryx.com.
It also has certain free features that we'd welcome people to use, like our eQMS.