In the last lesson, you built out a validator that detects hallucinations in the output of an LLM. You use an NLI model to check whether the LLM output was actually grounded in your trusted documents. In this lesson, you actually take that validator and build a guard around it. You'll then add that to the pizza shop chatbot and see how it prevents hallucinations from being shown to your customers. Let's dive in. Awesome. In the last part of this lesson, we took a look at how to build a validator that uses NLI at its core to make a determination of whether some LLM text is hallucinated or not. So all I've done in this part of the lesson up until now is like copied over some of those imports that we need for building that validator and then copy pasted that validator that you wrote in the last lesson. We're going to do next, is use the guardrail that we built as part of a guard, to test it out on some of the examples that we've seen before, and actually use it to mitigate some of the hallucinations that we've seen in our intro lesson. So you've seen how to use a guardrail as part of a guard and by itself, as we saw in the last lesson. When would you do one versus the other? Using a guardrail directly is really useful when you're testing to see if things are working as you expect them to. This is precisely what you did at the end of the previous video. You can also use a guardrail directly if you want access to the output of the guardrail, without lots of layers of abstraction in the middle. Again, for debugging purposes. However, if you wrap your guardrail inside a guard, you get several benefits. First of all, a guard allows you to combine multiple guardrails in a single execution step. Typically, for an application, you wouldn't just want a single guardrail. You'll want to make sure that your chatbot doesn't hallucinate and also doesn't use any profanity. A guard also allows a lot of benefits, such as streaming support, where you can allow the LLM stream its output and then in real-time, validate it like chunks as they're streamed out to you so that your users end up getting really fast stream results that are also protected and validated. You can also get OpenAI compatible LLM endpoints with a guard, so that it's a single line of code change to go from using OpenAI or an LLM directly to using a protected LLM endpoint instead. You also get out of the box logging and error handling support so that every guard execution is logged and can be analyzed later to see how your application is performing overall. With that, let's go back to our notebook and then see how you can set up and test your hallucination guard. All right. So you've already seen this, to initialize a guard we basically do guard dot use and then copying the class of our hallucination validation. And we're going to use the embedding model all-MiniLM-L6-v2 same as our last default value. The entailment model we're going to use is again as our default value the guardrailsAI fine-tuned NLI provenance model, and for our first run, our first test of this guard, we're going to try it out on some of those sources from our toy example of, you know, the sun rising and setting in different directions. And how we've configured this validator is so that it throws an exception when it detects a hallucination. All right. Now that we have this initialized, let's try out this guard on the same toy example we've seen in the past of where the sun rises. So we're going to try it out on this sentence, which should not contradict the sources that we've set up here in the guard. And let's take it for a spin. So we see that for this input sentence, the sun rises in the east, validation passes successfully, and the guard doesn't throw an error. Now let's try this same guard on an example where we say the sun is a star. So interestingly, this is actually not a false statement, but it's just not a statement that is supported from the sources that are given to our guard, right? So this is factual but not grounded and very quickly we see here that validation fails with the error message that this sentence is hallucinated. Because the sources that we use were sunrise in the east, sets in the west, and the sun is hot. So even though the sentence is true, it's a non-grounded or faithful. Perfect. So now let's use the guardrail that we just created from within the guardrail server and use it for mitigating hallucinations around a chatbot. Just as a reminder for this lesson, we've actually already set up guardrails server for you to use the validator that you just implemented. If you want to use this, guard in your own server setup, you'll need to add the validator you just wrote here to the server configuration file, and you can look back at the earlier lesson for details on how to do that. All right. So now same as before, let's swap out the earlier client that we created for a guarded version of the client that uses the server that we just wrote. So once again, all we're doing is setting up the OpenAI client with a different base URL that passes the LLM API to this guard that we just created, which currently is only running the hallucination guard. Awesome. Now that we have this set up, we can basically repeat the same instructions as before where we set up our vector database and our system message. You've seen this before in the past, and we are going to set up a guarded version of our chatbot, and then try running this chatbot on the same example as our last lesson, where we pass in the same prompt we'd use that generated the hallucination. And this time around, we get this detailed error message for a validation failure. Because these sentences on how to make your own pizza, the instructions for how to set it up, etc... is hallucinated, which is exactly what we expected. By the way, for your own use cases, you don't need to rely on the validation failure messages here. You can actually try to catch these exceptions, same as what we did over here, and then add more graceful error messages to control the flow of logic in your application. So now that you've seen the pattern of setting up a guardrail or a validator, wrapping it in a guard and then serving your LLM calls through that guard on the server. Let's take a look at some of the other failure modes you saw in the first lesson. Join me in the next lesson to start with a look at how you can keep your chatbot on topic.