Browser automation has traditionally been the domain of skilled developers, requiring detailed coding knowledge and understanding of tools like Playwright or Selenium. Amazon's Nova Act is allowing automation to be less complicated with a simple SDK that leverages natural language to explore and interact with web pages.

I created a demonstration of Nova Act handling a common but difficult task: interacting with older web pages. In particular, to find recycling schedules and requirements for an address in England. These older websites, often built without public APIs, have traditionally been difficult to interact with programmatically.

The demo showed how to chain a few simple prompts together to get Amazon's Nova Act model to control the browser, navigate the council web page, and tell us when and what we can recycle.

What's remarkable is that Nova Act handled these steps without requiring much coding knowledge from the user. The model interpreted the natural language request and translated it into precise Playwright commands behind the scenes.

See the demo for yourself here!

Getting started with Nova Act is pretty easy, you just need to set your API Key in your environment. You get this key from the Nova Act signup page.

Then we'll need to install the nova act package using your package manager (I chose uv).

Then you can setup your first local cached environment. You should make one of these for each individual project, and Nova Act will be able to keep some amount of session state between invocations.

Once you have this all setup, you can now start writing your Nova Act code! The first step is to import the Nova Act and pydantic libraries, as Nova Act uses pydantic typing for returning structured responses. Here we import the libraries, initialize Nova Act at duckduckgo, and define our response objects for later in the code.

You may have noticed on line 14 above, there's a parameter called max_steps which limits the SDK to no more than iterations to perform this action. It's highly recommended to use this to avoid consuming large amounts of compute resources.

Using browser-user applications also will consume resources on your machine, and if not carefully managed, could overwhelm your machine. Please use caution when using tools such as Nova Act without any direct supervision, and especially in parallel with other sessions!

This technology has significant implications for digital accessibility. Users who might struggle with complex web interfaces can now interact with them through simple, natural language commands. For organizations, it offers a way to automate interactions with web services that don't provide APIs, potentially streamlining many manual processes.

As models like Nova Act continue to evolve, we're likely to see more sophisticated browser automation becoming accessible to non-technical users. This could fundamentally change how we interact with web services, making digital tasks more efficient and accessible for everyone.

The preview for Nova Act is available now in the US, navigate to the Nova Act page to signup!

The use of browser-automation applications and services may expose your system to various security vulnerabilities, such as cross-site scripting (XSS), data injection, and client-side exploits along with breaking terms of usage with particular web applications. Before implementing these solutions in a production environment, it is imperative to conduct a thorough security assessment that addresses both browser-specific risks and your personal or organizational unique security requirements. Please consult the disclosures at Amazon Nova Act's GitHub for best practices on how to safely leverage this tool.

Open-source disclaimer: This blog post and related code samples make use of third party open-source technologies. If you want to make use of these code samples, make sure you check the licensing implications of all the packages involved. All code in this sample is released under Apache 2.0 licensing terms.

Housekeeping Note: After completing the experiment, it’s crucial to promptly remove or disable any environmental keys or credentials generated for the PoC. Make sure to use the max_steps parameter and/or watch your scripts closely to avoid long loops where nothing productive is happening.

Generative AI Disclaimer: The cover image for this article was generated with Amazon Nova Canvas.