Using NVIDIA NeMo Guardrails with Bedrock

Using NVIDIA NeMo Guardrails with Bedrock

How to configure Bedrock langchain integration with NeMo

Randy D
Amazon Employee
Published Mar 24, 2024
Last October, NVIDIA published a paper describing a new programmable way to apply safety guardrails to LLMs. The source code for NeMo Guardrails is available in GitHub. I tried using NeMo Guardrails with Amazon Bedrock via langchain, and had to figure out a couple of configuration settings to make it work. I wanted to share these in case anyone else runs into similar issues.
My example is based on Chapter 4 of the NeMo Guardrails getting started guide. First I installed the latest versions of langchain and NeMo Guardrails:
Next I imported the NeMo Guardrails and langchain modules.
Next I set up my simple prompt chain. Note that you must set streaming=True for the Bedrock model.
Now I set up the Nemo configuration.
In the config folder I have two YAML files. The contents of config.yml are where I configured NeMo to use another Bedrock model for its own use. Again note that I set streaming: True for the Bedrock model. The rest of the contents of this file, along with the contents of prompts.yml, are taken directly from the NeMo guide.
Now let's give it a try with a simple harmless prompt.
Now let's try a prompt attempts a jailbreak. (Note that Anthropic's Claude 2.1 model would not respond to this prompt directly either. I just want to illustrate that the NeMo Guardrails are working.)
And that's it! I haven't yet tried all of NeMo Guardrails' capabilities, but hopefully that helps you explore using it with Amazon Bedrock via langchain.

Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.