Erez Kaminski: Welcome, everybody, to this webinar. Today's topic is validating AI and large language models within GxP use cases. We're going to review a lot of use cases, and we're going to give some very practical advice about what to do. We have now worked with a bunch of teams facing this challenge and thinking: how can you develop and deploy this type of safety-critical system that is ever changing and statistical in nature?
To us, it all starts with this diagram you see right now on the screen, where you have all these different systems being made. I know it's a little hard to see; we'll highlight this a little later. That purple box is a specific AI use case being made in this factory for validation, this factory for GxP development and product making, and reviewed by different teams and different stakeholders. And we know that a lot of the sources of that information come from model providers like Anthropic, OpenAI, and others.
And that's the question: how can you build these types of validated use cases and ship them around the world? So, Gabriel, maybe... Before we start: we're going to record the session, and you'll get the slides at the end. Please ask questions.
We have folks waiting to respond to questions. We'll have a feedback survey at the end, or if you exit in between, please give us your feedback. We use it to develop and improve our webinars. We now have multiple webinars every week, every month, and all of them are available for free on demand on our website. So you're welcome to watch historical webinars on everything from Part 11 compliance to AI building.
And let's start, as all good things should, with a survey. The survey is: what GxP AI use cases are you currently considering? Let's get started with this.
My name is Erez, founder and CEO of Ketryx. I used to be a physicist a long time ago, and then I worked at a company that makes tools for physicists.
I had a stint at Amgen helping develop AI solutions across the company, mostly focused on combination products. Then I went to MIT, where I studied computer science. My focus was actually the limits of machine learning models during black swan events, or critical events. And then I started Ketryx as a student at MIT. I was lucky enough to find some great partners in the life science space and great folks to work with, like Gabriel.
What we do at Ketryx is accelerate the rate of safe innovation, mostly focused on complex systems, AI software, software-enabled hardware, and the use of methods like AI within the manufacturing and development of safety-critical products. Gabriel?
Gabriel Pascualy: Hi, everyone. I'm Gabriel Pascualy. I run our solutions engineering group here at Ketryx.
I started my career as an embedded systems security engineer, so deep in the cybersecurity space at the MITRE Corporation, focused on all things security. I went on to work at Amgen, following a master's and an MBA at MIT. During my time at Amgen, I was applying AI more on the manufacturing side and ran into a lot of the challenges of validating AI for these use cases. It's through this experience at MIT and Amgen that I met Erez and joined full time.
I've been helping folks evaluate and implement our platform for a few years now, so I've seen a lot of different use cases and a lot of value delivered to our partners. Looking forward to this conversation.
Erez Kaminski: Wonderful. Let me introduce Ketryx.
Ketryx is a way to develop and deploy regulated products, mostly in life science, through AI-driven automation. It's a combination of validated automation and AI tooling that connects to the systems you already have, like Jira, GitHub, and AWS, and allows you to automatically generate all evidence of compliance and accelerate your product delivery, connecting quality and compliance. It's used by three of the top five medical device manufacturers and many other leaders in life science, including many of the leading life science companies in the world. We're going to talk mostly about AI and how to build AI in regulated environments. That's something I've been doing for the better part of the last seven years of my life.
We have helped a lot of companies actually do that and ship these types of products. Then we'll have a demo. The demo will be in Ketryx, but it won't really be a demo of the product. It'll be a demo of how you validate a GxP system for AI. We'll talk about the common use cases.
Why is validating AI, and validating in general, so difficult? How do you validate GxP AI systems? Strategies, metrics, monitoring. We'll do a case study of a partner we worked with, and then a demonstration that Gabriel will lead. Please feel free to ask questions as much as you can.
We love questions. We love engagement. We love participation. We love answering questions. So let's look at a few use cases.
It was actually hard to build a slide with all the use cases we've seen and had in mind. We decided not to even include commercial marketing, some of which are actually regulated practices and use cases that are very common now, and to really focus on the ones that have to do with touching the product or the quality of the product.
So we could say everything from drug discovery, to using not just information about the drugs but precision oncology information like pathology slides. We have a customer we're going to talk about later that uses us for this type of solution and then actually resells that solution to many of the top pharma companies under GxP practices. Then there's the ability to optimize clinical trials, whether that's recruitment strategies, the data transformation, the processing of the notes, or anything of that nature. Automating labs and manufacturing sites.
We've seen a lot of posts recently from folks about how labs and manufacturing are getting automated more and more, in this dream of creating the automated chemist, the automated scientist, to do work while everybody's sleeping. A lot of use cases around change management, and how to streamline the documentation of validation and other things, assuring traceability. Post-market surveillance, of course; I saw that was a pretty big topic. In our use cases, this is one of the hottest areas of AI and has been for a long time. People have been building AI systems for post-market surveillance and complaint triage management for a long time.
And definitely, there's been a huge breakthrough in the past few years. This is the main research area for me, both as a student and as a professional, in a bunch of different scenarios. And then there's this idea of automating documentation, not just for change management, but for regulatory submission and in general, like the IND and NDA documentation. We've seen a lot of great use cases, and we know that's coming. And the last part of this use case question is that we know a lot of these do need to be validated, do need to be done in a GxP environment, in order to leverage them to the maximum ability. That's what we're going to talk about today.
I would also start by asking: what's the impact of validation? Why do you even want to talk about this as you set up this question of how this works, why you need to modernize, why it's so difficult, why you need to think about this before doing tons of AI projects? Well, it's because validation is a major cost structure even for traditional embedded or manufacturing automation software, even worse for the cloud, and much more challenging for AI and such systems. If you think that traditionally it would cost 30 to 40% of the project cost, it could be even higher for AI systems and cloud-based systems. That's based on my experience doing it as well.
Figuring out the validation strategy, and the validation strategy at scale, is the first step to understanding how to actually do this. And we know that people can reduce this quite substantially. We've had many customers that have reduced the total amount of time and energy spent on validation documentation in the 80 to 90% range, some 60%. So there's a big, big need here. And what is validation?
I love this definition from the 2002 General Principles of Software Validation guidance. I'm lucky enough to be working with one of the reviewers of this guidance, who has quite a few ideas on it. I just want to read it out loud so we're all on the same page: "Validation is the confirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and that the particular requirements implemented through software can be consistently fulfilled." There are three major ideas in order to validate a system.
One is you need to have a user need and intended use. Two is you need to have objective evidence that those are consistently fulfilled, right? And "consistently fulfilled" is that third part. You need to show that it will happen, and that it happens within the qualification limit or the intended uses you have in mind.
This idea of a qualification limit is going to become very, very critical as we discuss how to validate AI. And this is where I give the room to Gabriel to take us through a little bit of why validating AI systems is so difficult.
Gabriel Pascualy: Yeah, absolutely.
So why is validation of AI systems so difficult? I think it's kind of a funny question to pose after going through that slide where we say that 30% of all project costs go to validation. So I would say that validating AI systems is challenging to start with because validation is challenging in general. Now, where does that crop up specifically within AI? I think one of the challenges with AI is that change management is quite frequent.
Software, a production line: those are static things, but models are changing. The data that we put into the models is changing, and therefore change management needs to be frequent. The second component is that we're often using external models. We're leveraging things like OpenAI, Anthropic, Google. That means part of our supply chain, our software supply chain for the use case, can be continuously updating.
And therefore, we need to have methods to validate that change. Another big challenge is the perception that nondeterministic outputs cannot be validated. I think this is particularly relevant for those who are implementing AI and communicating to leadership and to those who come from a traditional validation background: we can, in fact, validate an AI system that has nondeterministic outputs. I like the analogy that a clinical trial in and of itself is somewhat nondeterministic in the way that we do validation of our product. And finally, I think the core challenge, and one that we'll talk about a lot today, is how quality and development are disconnected.
Oftentimes, development activities are done in one set of tools, as we can see on the diagram on the right, and then quality and documentation are done in a different set of tools. The friction that happens between these two sets of tools and life cycles is what ultimately slows the AI validation process and makes it challenging. But ultimately, the charter today at companies, and we hear this from executive teams across the space, is: how can we do more with less?
How can we leverage all of these innovations in AI to help scale our operations, maintain the same headcount, but deliver more therapies to the market? That is the charter. And it's funny: I mentioned doing this work back at Amgen years ago, and the idea of validating a single AI use case being insurmountable, and now we're moving toward a world where we have to validate 10, 50, 100 in order to achieve the goals we want, not to mention tech. And I think, Erez, you're about to jump in there.
Erez Kaminski: Yeah. I'm reminded of a conversation we had with one of our great champions, who actually transitioned into biotech from tech. He's a leader at a pretty prominent company, and he called us in after a few months on the job. He said he'd found our website, and I happened to connect with him after one or two conversations.
And he said, "When I got here, I didn't understand. I came from a large tech company. We release tens of thousands of AI models every week. I don't understand how we would be able to do even a fraction of that here, given the amount of effort around releasing an application into production in a validated state." And I think this is the real challenge.
How do you do this at massive scale, not for a one-off? And luckily, it's been solved by many medical device companies by now.
Gabriel Pascualy: And I think what we're also seeing, just from the regulatory perspective, is the shepherding: "Hey, here is some guidance." We have recent guidance that came out from the FDA on the use of AI and outputs from AI models to support regulatory decision making. We'll weave this story into the demonstration and how we talk about validating AI. But I think the charge here is that there's regulatory guidance coming out, and we have the opportunity to leverage AI models more and more.
The challenge is going to be how we accelerate the development of those internally in a safe and reliable way. So with that, I think we'll move on to talking about how you validate GxP AI, and I'll give it back to Erez.
Erez Kaminski: Yeah, thank you, Gabriel. So how do you validate GxP AI?
I think the first part of this is you want to do a lot of the things we're used to doing overall under guidance or standards like GAMP 5. Basically, we want to take this risk-based approach that everybody's talking about, that the FDA knows and loves and the EU knows and loves, and figure out the risk of the system. There's a great question that was just asked by an anonymous attendee, with a lot of detail about how you do this and how you trust it. And I'd say one of the key aspects of what you need to do with a vendor, to contract and accept, is: what kind of risk class are we in? What kind of risk class is the use case?
Is it critical for us? To answer that question, we go to a traditional risk assessment, not of the system being worked on, but of the use case. Is this going to have a lot of impact on us if it works or doesn't work? What are the different aspects of this use case?
Then we need to start talking about the system. The system uses a third-party model or a model from an open-source ecosystem: is that good or bad? Does that increase the risk or reduce the risk? And then we need to start thinking about how we would basically be able to do that validation, running through it at a much higher throughput and much higher scale.
So you have this assessment of the impact of the system. What kind of risks does it have? A lot of the risks will be top down, coming from the use case. Some of the risks, which we're going to talk about in the next few slides, are going to come from the fact that you're usually using a third-party large language model, and it's doing this type of statistical operation. Then you need to think about the severity and the probability of the issue coming up, assign the risk level, and then decide what kind of process you want to follow.
And if you really think about what the FDA is talking about, how do you define this validation? In the FDA's guidance, they give this idea of the context of use, which is related to intended use. We need to understand how you're going to use this AI system for regulatory decision making. Then you want to have a risk assessment of the AI model to understand what you're going to do with it and what risks the AI model comes with. Unlike simple, deterministic manufacturing software that's built in-house and runs on some kind of microcontroller, there are a lot of risks coming from the fact that you're using a large language model that can respond in a variety of ways, comes from a third party, and is updated very frequently.
All of those introduce a bunch of different risks that are more unique to AI. Then from this risk assessment and context of use come the user requirements and functional specification. What is this trying to do? And then that all comes into this idea of the credibility assessment plan.
Do we think the machine learning product we're using is credible? How do we assess that credibility? How do we score it? How do we know if it's working or not working? And then we want to execute this plan.
Basically, test this thing, and make sure that the product, the model, is as credible as we want it to be, validated to that credibility score. Then, do we think it's adequate to perform in this context of use? If it is, let's deploy it and monitor it. And if not, how do you go back and redefine your context of use to make sure it is adequate?
So there's this question of whether the use case is appropriate to be executed by a statistical model. Can you even validate that? And is this specific model good enough for that use case? That's going to come up a lot: this idea of the metric.
If there's one thing people can take from this entire webinar, and we'll repeat it a few times, it's this idea of the metric of the use case, the metric of what you're trying to do. And that's going to change between every use case. It's actually not that dependent on the model; it's much more related to the use case. Am I forecasting, or classifying, or generating documents?
All of those have different metrics, and companies will need to find experts to help them define them and then help scale that library up. So what does it look like to do change management for this AI system? Let's say we did all this.
We've done everything described on the previous slide and on the earlier slides that Gabriel showed, which we're going to dive into a bit more in a few slides. What happens then? I then have a validated model that has gone through this credibility plan. But now I need to either use a new model or fine-tune it. I retrain it locally, or retrain a complete model.
In most cases today, people mostly fine-tune. Sometimes they don't even do that, because the models work really well, since they've already seen a lot of the data you're going to fine-tune on. But basically, the idea is you take that model and you retrain it. Maybe that's even something you don't do, but Anthropic does for you, right?
Far away. And then they send you this new model, or you retrain this new model. Again, some biotech use cases would require very specific models. And then there's the question of the acceptance criteria and impact assessment. Is the new model good enough?
Does it pass the criteria or metric you're measuring its success on? Is it within the qualification limits of your criteria? And what's the impact of this change? Then you need to think: is that impact big? How much documentation do I need to generate?
And then you need to both document everything we just talked about and release. What's really interesting here is if you look at the bottom right, there's this five-step process where I get new data, I need to check the quality of the data, and segregate the data. I then need to get the model, retrain it, reinsert it into my validated infrastructure, and revalidate the whole thing. So now if I want to do that for 100 models a week, or even 10 models a week, I start realizing that I want to separate my models from the infrastructure that serves them, and then be able to validate them separately and deploy them separately, which is really, really critical. And that's been done in MedTech for quite a while now.
And this leads to a discussion. Now we understand the path the regulator wants to see and that we're used to taking. There are some new terms in these new guidances, but nothing is really earth-shattering. It's the same old validation strategies, just in a much more complex, fast-moving system. We saw that change management is very challenging because it's going to be done very, very fast, at a frequency of a week or a month, or at the longest a quarter, but I expect that to actually go way down.
Now the question is, what kind of strategies should I take to validate my model? What are the risk-based validation strategies that actually work? One is metrics. We need to decide on metrics. Whenever I'm faced with a new project that I'm doing or supporting for a partner, or I'm brought in to advise on, I always want to ask: okay.
This is all great. What are the metrics we're trying to measure for this particular use case? Because everything will be much easier after that. Once you decide on the metrics, the infrastructure itself, there are companies that do that. But the hardest point is: how do we all agree, as quality professionals, as subject matter experts, as operators who are going to use this, that those are the right metrics for us?
Then we need to talk about reusing reasonable validation components. You want to have a stack across your entire IT system that allows you to reuse things that have been previously validated, so you don't need to redo so much of the paperwork. Number three is that all the documentation needs to be done automatically, 100% of it. There's no real other option, given the speed at which some companies are going to start releasing these models and the need to update them.
And then the last part is how you monitor this and make sure you're tracking and basically ensuring things are safe and reliable. If we look at the left-hand side, we see these selected risks, metrics, and monitoring strategies. On the left-hand side, we see the risks: the big column. I recommend anyone take a screenshot of this. We'll send you the slides afterwards.
There are a lot of different data sources you can take this from, from really nice online resources to a standard I was part of writing, called AAMI TIR34971. It's a technical report about how to assess risk for medical devices that are driven by AI. There are a lot of other great resources as well. So the first thing you need to think about is the risk of this particular use case and this particular model. The risks of the use case are usually things we're more used to and have seen before, because they're tied, again, to a more traditional use case that we've done before, or that someone in our company is doing manually and we now want to automate.
But there are a lot of risks that are unique to data-driven software or AI software, whether that's the issue that the data has biases or is incorrect, or that we trust the model too much, or the model is overfitted, all kinds of issues like that. Then we need to agree on the metrics for this particular model for different use cases. Different models will have different metrics. Ideally, the metric is tied to the use case. For example, if you have a classification task, a very classical one is either accuracy or F1 score, but there are many more modern metrics than just an F1 score.
People have developed ideas around this quite far. In fact, there are probably hundreds of metrics for every task you can think of, because there's been so much research on it. The challenge is to find the right metric that is appropriate for your intended use and the level of error bars you're looking for. You then need to agree on a monitoring strategy. How are you going to monitor this?
How are you going to track it and make sure things are okay? What we're going to talk about, and it is my belief because I think it's both acceptable to regulators and something companies know how to do, and it works, is using statistical process control across different metrics, different error spaces, and different drift measurements of the model and the data, to check that everything is okay. And let's now compile all of that into this use case on the right.
If we look at this use case for AI-driven complaint management, which Gabriel in just a few minutes is going to show us in a live demo, including how to build it and how to validate an agent for it, with all the infrastructure, we see there are four parts on this use case scorecard, or use case label.
There's a description of the use case. There's a set of potential risks; we're just giving one example to make it easy. The model could have a bias and lead to inconsistent handling of certain complaint types.
This model is all about the triage and categorization of complaints: severe adverse event, adverse event, product feedback, or other. What if all the data is trained on people in Cambridge, Massachusetts? When you actually deploy the asset, the drug, you have people all over the world using it, but your triage system can't really understand them, whether that's their language or some syntax. Some information that maybe a human can't even notice would be inconsistent or biased, but the computer picks it up. The second part is the relevant metric. What's the right metric for this use case?
It's a triage use case, so it's a classification task. It's either bananas or oranges, apples or oranges. And I need to decide. It could be recall. It could be precision.
I think those are the metrics we have. But as part of your validation process, I'd recommend trying different metrics and seeing which ones work best for this particular use case with your particular data. And the last part is how you monitor this and make sure through SPC that things are running correctly.
Now, looking at these reusable validation components: I love this pyramid chart because it explains how complicated what we're doing is. At the top of the pyramid, there's the actual use case, the validated use case that you're going to deploy to your internal stakeholders or customers or patients.
At the bottom, there's the infrastructure it's running on, whether that's local, on-prem, private cloud, public cloud, whatever it is; there is some infrastructure it's running on. Then, from that infrastructure layer, we need to validate the infrastructure, and that widely depends on your use case. I think most teams should take a very serious risk-based approach here, because the number of things we could validate in a cloud, for example, is kind of endless, but most of them won't necessarily matter for many use cases. Then there's the specific application we're configuring: the versions of everything, its structure, the parameters of that application.
And then there's the supply chain. For most LLMs, we're actually grabbing models that exist at companies like Anthropic or OpenAI or Google, or open-source ones. We're leveraging them within our application, maybe with some fine-tuning. But there's this huge software supply chain, which is a big problem. That's why the FDA came out with the PATCH Act and is asking to see all these SBOMs.
You'll have quite a big supply chain for this. And at the top of that sits the validated application. It is really critical, especially in a large company developing this, that you agree on a strategy to validate not just this particular application, but the entire pyramid all the way down. Then you want the ability to replace the top of the pyramid, or the top two layers, with different applications and run the whole validation automatically on top of this preexisting validated infrastructure. This is the idea of component reuse, or system of systems. We need to figure out how to reuse all these components over time, as well as automate all the generation of the validation documentation, because something that will happen quite often is that the model is updated and the last model used is no longer available.
It could take you five to ten minutes to replace the model. But if you need to do all the validation by hand, that could take you months and months. If you had it all automated, you could replace it the same day, the same week. Send people reports to review if you passed all the tests, and then move forward. I think this ability to have a validation strategy that is lean, robust, fast, and automated is the core problem all pharmaceutical companies doing GxP AI will run into.
The use cases will be much simpler to identify and deploy. And here's where we want to talk for a second about monitoring. There are a lot of ways you can monitor an AI system; it's not like there's an agreed-upon path. But this is the one I've spent most of my time on, because it is so well known as a general methodology for the control of processes.
What you need to do is use a statistical process control chart and assume there's a normal distribution for the metric, the specific metric you're trying to measure. For example, the mean absolute error per batch of some measurement, the pH or whatnot.
Then you need to make sure the process is under control. We know that a process under control means 99.7% of all the data outputs are within three standard deviations of the center, the mean of the distribution. And then you need to monitor it like we monitor many other things. It could be that the Nelson rules, which are used in a lot of SPC, are not the right set of rules, but that will come with time, figuring out the right kind of anomaly detection method for you. You can even run just an anomaly detector on top of this.
But you need to see if there are any trends or any out-of-control responses. And I can tell you from work I did at MIT that you will see the error, which is what we're tracking here, the error of the model, respond much faster to issues than accuracy, for a number of reasons. This is very, very critical: how do you measure it over time and make sure things are safe? So before we get into the case study and demo, we just wanted to ask: which of these areas we talked about are critical to your AI validation strategy? And where do you see the challenge?
I just want to share one case study we did this year with a client, one of our partners, who did some amazing work. They're developing this very complex GxP AI platform. I can't share too much about it, but basically, we were able to substantially reduce the amount of documentation required to do this type of work, by about 80%. They're able to release much, much faster. Some parts of theirs are released on a weekly basis even today, some of them daily. When we started working together, one of the problems was that their AI systems were moving so fast, and they had so many sites, so many different worlds they're working in, that they just could not keep up with the amount of documentation and compliance they needed to do for all these global releases.
And it created huge bottlenecks. Most of the time a release was stuck was because they were compiling the validation documentation manually. Our solution was this ability to automatically validate everything, generate all the documents, use AI to do that, and accelerate even faster with the different types of agents we have. And there was quite a substantial reduction. At the end of the day, what they're able to do is just deploy more with a similar number of people, which I think is pretty exciting.
And that's really, really hard to do in this industry: how do you actually do this work, especially as you scale up the number of apps? Because what I'm hearing from friends, colleagues, and partners in the industry is that all the executives are saying, "I don't need to do a hundred thousand AI models a week or a month like Amazon, but let's do a thousand," some very small fraction of that. And I think it's a hard realization that validating a thousand applications a week or a month could be quite constraining in the amount of time it would take. We'll get started with this agent validation deep dive. Let's do it. Take it away.
Gabriel Pascualy: We just talked about wanting to achieve tens or hundreds of use cases. So the question is, how do we achieve that? As part of this demonstration, we're going to go through some of these strategies.
One of the core strategies, which Erez mentioned, is enabling modular validation. Here on the screen, you can see the structure of the projects we'll investigate during the demonstration. You'll see three layers. We have our use case layer, with two applications that are going to be in the environment: case insights and complaint processing.
These use cases are leveraging, or are a configured version of, an underlying GxP platform. So things like validating a fine-tuned version of a model can live at the platform layer. Then that model and the platform can be configured for a particular use case, and that configuration validated independently. So let's go into the environment and get this demonstration started.
Here we are in the Ketryx platform, and you can see the three projects, or three entities, we were just looking at: case insights, complaint processing, and the GxP AI platform. The first few things you'll see are that each of these projects is being versioned independently. We have this underlying AI platform; it's at version 1.2. And then we're validating, or incrementally improving, these two use cases that are built on top of the platform.
Within each of these projects, we're maintaining all of the requirements, functional specifications, design specifications, validation activities: all of the necessary documentation for both the use case and the underlying platform. What this system-of-systems, subsystem approach allows us to do is validate the underlying platform and then reuse that evidence in the top project. That will become very clear as we get in and talk about traceability within this particular complaint processing project. Each of these projects is connected to a set of systems. In this case, we've connected this project to Jira, along with a code repository that lives over in GitHub.
I know we've been talking a lot about validating AI, but we will also be using AI to accelerate our process here. We're kind of mid-flight. We're working on minor version 1.1, and we're building off of version 1.0.1 of this particular use case. So the first thing I can do in here is get a sense of what changes are going into this next version. I'm going to ask our assistant on the right side to show me the changed or removed items.
This will allow me to filter down and quickly see that we are resolving an anomaly that's resulting in a requirement change. It looks like we've updated a risk, and we've updated a test case. Now, the first thing this brings to mind for me is that we're making a requirement change. One of the core challenges you'll have is managing many, many requirements across your use case, and one of the challenges that comes up when managing lots of requirements is requirement conflicts.
One of the ways we can accelerate our work using the Ketryx platform is through a set of agents that run in the background and help us with our validation work. If I open up this agents module, we'll segue into looking at one of these agents that ran in the background the previous night and has a few findings for us. Going into this requirements agent and into the results, we'll see something interesting it identified the previous night. This agent ran and analyzed all of our requirements, and it flagged one of our requirements that includes our metric on recall. It identified that the metric might not be right, based on one of the requirements we wrote.
In this case, it's concerned that we haven't necessarily... let's see, let's pull out the exact sentence here. Based on our goal to minimize misclassifications, we haven't necessarily defined our metric just right. So what the AI agent has offered us is a way to update our requirement so that the metric we've selected aligns with our requirement for minimizing misclassifications. Of course, it's not going to make any changes to our requirements without human review and approval.
But once we review and approve the changes, it'll push those changes out to our requirements management system, where the rest of the team, the systems engineers and the quality engineers, are working. Alright. Now that we've made that change and updated our requirement, we can come back to the original thread we were on. The next question I have, now that we understand what items are changing across this version, is: can you tell me a little bit about the anomaly that's being resolved in this version? So it's not just doing a structured query.
It's going in and giving us a bit of background on what we're changing across this version. While that's working, I'll write my next prompt, which is: how are we going about resolving this anomaly? So here we go. It gave us a summary of what we're changing across this particular version.
The next question I might have is: how is the anomaly being resolved? It'll go ahead and process that and help me understand how we're going about this change management process on a highly complex AI use case. The final question I have before we move on and talk a little about the connection to an external system like Jira... and actually, because we're going to go to Jira, I'll open up this anomaly in Jira. So, the final question I have: we're going through and resolving this anomaly, updating a metric, maybe updating how we're testing that metric.
So ultimately, the final question I have is: show me the testing requirements... oh, typo there... show me the testing requirements affected by this anomaly. What this allows me to do is quickly understand the tests, and the changed tests, that we've implemented in order to show resolution of this anomaly in our next release. We'll give it a moment to think, and now it will filter down and show me the tests that relate to the anomaly we're resolving. Let's go over to Jira and approve this anomaly, and then we'll talk a little bit about the Git integration.
Going over to Jira: as a reminder, we want to enable you to use your existing tools, stay in your preferred tools, and only come to Ketryx for specific activity-level items: traceability, leveraging agents, the assistant, et cetera. For those familiar with Jira, you'll see we've updated Jira in two different ways. The first is that Ketryx has inserted this traceability widget, which helps us understand the relationship between a bug we're tracking in Jira and, say, a requirement we're managing in our external requirements management solution. Second, we have approvals. Wherever possible, we want to enable teams to spend more time in their preferred developer tools, even going so far as doing peer reviews and approvals directly in Jira.
Let's go ahead and approve this anomaly as being resolved, and this will allow us to go through the release when we get to it. Going back up to the All Items screen here, the other integration we're leveraging is an integration with Git. There was a question about traceability of AI models and audit trails. By integrating directly into the systems where your models are maintained, where automated testing is maintained, we can get better and more granular traceability into where those activities occur. So I clicked that link.
It brought me directly to the Git repository, where I've defined a lot of the metrics I'll be testing for my particular model. And we can also trace directly to where the model itself is defined. Now, instead of showing you that on the All Items screen, we'll go into the traceability matrix, which will give us a better view of traceability across the system. Going into the traceability view, I'll just minimize this side so we can see the full matrix. We can see a full end-to-end trace matrix from our user requirement specification all the way down through testing.
What's important to note on this traceability view is that it's not just about the use case or user requirements specifications we're validating for this particular complaint processing use case, but also the underlying documentation for the platform and for the model within the platform that we're leveraging. The columns in the center here, from design specifications and subsystem specifications through their testing, are actually being leveraged from that separate platform project. This means we don't need to redocument how we fine-tune the specific internal model at our organization for use across many different use cases. You'll also see that we have a significant amount of automated testing. So for any of the testing you're doing at the model level or the use case level... the statistic was... oh, now you're testing my memory here.
80% of all validation testing can be automated. So finding ways to integrate with where the automated testing is being done, and automatically pulling it into documentation, is really critical here. The final thing we'll do is go into our risk management. I think this part is critical, particularly from a scalability perspective. There are going to be risks that emerge at the use case level from a top-down analysis.
There are also going to be risks that emerge through an analysis of the requirements, maybe from a bottom-up perspective. The way we manage the documentation around those risks can be slightly different, enabling us to scale our development better. First, for the bottom-up cases: for example, model hallucinations leading to inaccurate or harmful insights might be a risk that's inherent to the platform or the underlying model. We may want to both document and control that risk at the platform level. The three other risks we've identified here are risks that emerge from the use case.
These are going to be maintained at the complaint processing project level, and controlled either at the complaint processing project level or by controls that exist at the platform level. If we scroll over in the risk table here, we'll see a set of test cases acting as risk controls that actually live within the platform. This allows us to reuse those risk control measures across the many use cases that can benefit from them. Finally, this brings us to the release process. Normally, we would go through a full verification and validation run.
All of our testing is automated for this project, and it's already run. So if we go into the releases area and then to version 1.1, we'll see a full summary of the items we need in order to get this use case project to release. We'll actually see that there are two uncontrolled items.
Erez Kaminski: Yeah. And I think, Gabriel, maybe it's fun to mention here: we're talking about 231 items. A lot of people have 10,000 requirements in a project, right? Some of these projects are just so complex.
They use AI within the pharmaceutical or medical device domain. There are so many systems you need to interconnect with. Many of the products people end up having are multi-stakeholder in nature, so there are a lot of different interfaces. It's surprising sometimes how complex the simplest medical devices or pharmaceutical equipment with AI or software are.
Gabriel Pascualy: That's a great point, Erez. We talk about how we use AI for our own internal development of the platform, where we have thousands of requirements. And sometimes the demo project can't fully describe that, right? We only have 31 requirements here. But this is where the assistant, and having agents that do work for you in the background, become so critical in order to accelerate the development process.
What I just did here was open up the two uncontrolled items, because we're actually going to go through a release process in the last few minutes. When I clicked that button, it brought me to the All Items screen, and this shows me that there are two requirements missing approval. I'll go ahead and do my approval here, using my fingerprint to make the approval. Once we've approved these two items, all we need to do is go back to the release screen and regenerate our documents. We haven't even talked about Word and Excel documents yet.
That's when we get to the end. We click Generate Documents. This will pull all of the latest evidence from the various platforms your teams are working across. We can actually open up one of these documents to see what that looks like. This document can be configured to your organizational template.
So the margins and the look and feel and style will all be as expected. And it's pulled in our requirements and testing information from all the different systems our teams are working across. Finally, at the release level, we can approve our release with one final signature here. That gives you a sense of the full scale: how we go from understanding the change going on in our particular use case project, our validation project, through completing the documentation of all that evidence and coordinating the release.
Erez Kaminski: Love it. We'll stay here to take some questions.