How To Set Up and Run a Local LLM With Ollama and Llama 3

This post was originally published on Feb 17, 2024; it has been updated.

I’ve posted about coming off the cloud, and now I’m looking at running an open source LLM locally on my MacBook. If this feels like part of some “cloud repatriation” project, it isn’t: I’m just interested in tools I can control to add to any potential workflow chain.

Assuming your machine can spare the size and memory, what are the arguments for doing this? Apart from not having to pay the running costs of someone else’s server, you can run queries on…