Run Large Language Models with Ollama and AWS Lightsail for Research
Example demonstrating how to run Large Language Models locally or with AWS Lightsail for Research.
- Download and install Ollama: https://ollama.ai/download
- Open a terminal and start ollama:
$ ollama serve
- Check to see if it is installed:
ollama –version
- Choose and pull a large language model from the list of available models. For a local install, use
orca-mini
which is a smaller LLM:$ ollama pull orca-mini
- Run the model in the terminal.
ollama run orca-mini
- Install Docker using these instructions.
- Open a Powershell window as Administrator.
- Pull the ollama container image from Docker Hub. Copy and paste this command in the Powershell window:
powershell> docker pull ollama/ollama
- Start the ollama container. Copy and paste this command in the Powershell window:
powershell> docker run -d -v ollama:/root/.ollama -p 11434:11434 —name ollama ollama/ollama
- To run a model locally, copy and paste this command in the Powershell window:
powershell> docker exec -it ollama ollama run orca-mini
- Choose and pull a LLM from the list of available models. For a local install, use
orca-mini
which is a smaller LLM:powershell> ollama pull orca-mini
- Run a model in the Powershell console.
powershell> ollama run orca-mini
1
2
$ sudo curl https://ollama.ai/install.sh | sh
$ ollama pull llama2
1
$ ollama run llama2
1
2
$ which python3
$ sudo apt install python3.10-venv
1
2
3
$ mkdir ollama && cd ollama
$ python -m venv venv
$ source venv/bin/activate
1
2
$ pip install jupyter
$ pip install langchain
1
$ jupyter notebook
1
2
3
4
5
6
7
8
# langchain connects to Ollama - https://ollama.ai/
from langchain.llms import Ollama
from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
llm = Ollama(model="llama2",
callback_manager = CallbackManager([StreamingStdOutCallbackHandler()]))
llm("Why is the sky blue?")
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.