Using NVIDIA NeMo Guardrails with Bedrock

Last October, NVIDIA published a paper describing a new programmable way to apply safety guardrails to LLMs. The source code for NeMo Guardrails is available in GitHub. I tried using NeMo Guardrails with Amazon Bedrock via langchain, and had to figure out a couple of configuration settings to make it work. I wanted to share these in case anyone else runs into similar issues.

My example is based on Chapter 4 of the NeMo Guardrails getting started guide. First I installed the latest versions of langchain and NeMo Guardrails:

Next I imported the NeMo Guardrails and langchain modules.

Next I set up my simple prompt chain. Note that you must set streaming=True for the Bedrock model.

Now I set up the Nemo configuration.

In the config folder I have two YAML files. The contents of config.yml are where I configured NeMo to use another Bedrock model for its own use. Again note that I set streaming: True for the Bedrock model. The rest of the contents of this file, along with the contents of prompts.yml, are taken directly from the NeMo guide.

Now let's give it a try with a simple harmless prompt.

Now let's try a prompt attempts a jailbreak. (Note that Anthropic's Claude 2.1 model would not respond to this prompt directly either. I just want to illustrate that the NeMo Guardrails are working.)

And that's it! I haven't yet tried all of NeMo Guardrails' capabilities, but hopefully that helps you explore using it with Amazon Bedrock via langchain.