Running an AI agent on cloud APIs gets expensive fast. And once you’ve hit token limits or watched your monthly bill climb, the appeal of a fully local setup becomes obvious.
I’ve been running Hermes on my M1 Max Mac Studio for about a month now, with no monthly API costs and no usage limits. This guide covers exactly how to get the same setup running on your machine.
What You Need Before You Start
Before anything else, make sure you have the following ready:
- A Mac with Apple Silicon (M1, M2, M3, or M4 chip)
- Python installed: check with
python3 --versionin Terminal - Xcode Command Line Tools — run
xcode-select --installif you haven’t already - A Telegram account (this is how you talk to your agent)
Step 1: Set Up Your Local Model with omlx.ai
Before installing Hermes, you need a local inference engine running. This is what serves the AI model on your machine and replaces any cloud API like Claude, Gemini or OpenAI.
When I set up my Hermes Agent, I had problems with getting ollama running, so I tried omlx.ai. It’s relatively new, stable, and lets you pull models directly from Hugging Face without much friction. So far I had a great experience with it and it offers a few other features in the Web UI, which I haven’t seen with ollama.
You can download and install omlx from here or you can clone the GitHub repo if you want, but in this guide I will use the dmg download for an easier installation. Clicking the “Download DMG” button forwards you to the release page on GitHub. Here you can scroll down a bit and under “Assets” you can download the right version for your Mac.
After the download finished you can install it just as any other application on your Mac. When first opening the application MacOS will ask you if you want to open this application. To Open it press on Open. After that you will be prompted with the oMLX Welcome Screen. Here you can set up the location for the models, the local port and an API key. Don’t worry you don’t need a subscription, you just need to define one. Enter a key with a minimum of 4 characters and write it down so you don’t forget it since we need it later again.
Then you can start the server by clicking on “Start Server” and then click on “Open Admin Panel”. Now a window opens in your browser and after you entered your previously set API key, you are in the Web UI of oMLX.
Which Model to Run on Apple Silicon
Model choice matters here. You want something that fits comfortably in the unified memory without grinding your machine to a halt.
On my Mac Studio with 64GB of unified memory, I run Gemma 4 26B. It’s fast enough for real agent workflows and from a month of daily use roughly the ceiling of what the Mac Studio handles well at this spec. The 32B models technically run, but performance starts to drag sometimes. For 32GB machines, I’d look at models in the 14B–20B range instead.
You can go over to Models -> Downloader and search in Huggingface for your desired model. SInce it was the newest model at the time I used “lmstudio-community/gemma-4-26B-A4B-it-MLX-4bit”
Once you’ve picked your model, click on “Download” inside omlx. The app downloads it from Hugging Face and makes it available as a local API endpoint. Note the local API port omlx assigns, since you’ll need it in Step 3. It’s usually something like http://localhost:1234.
Leave omlx running in the background before continuing.
Step 2: Install Hermes Agent
Open your Terminal and run the Hermes installer:
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bashThis pulls and runs the setup script. It walks you through configuration interactively with no manual config file editing needed. The full documentation is at hermes-agent.nousresearch.com/docs if you want to go deeper on any specific option.
After pulling the script from github you can start to configure your agent. Select “Quick setup” with your arrow keys and confirm with enter.
Step 3: Connect Hermes to Your Local Model
When the installer prompts you to choose an LLM backend, do not select OpenAI. With the arrow keys go all the way down in the list and choose the local or custom endpoint option.
Point it to the omlx API, that you can find the Web UI in the Dasboard tab under API endpoints. It will look something like this:
http://127.0.0.1:8000/v1Then enter the API key you set up before and choose select “Auto detect” for the API compatibility. Hermes now sees your previously downloaded model. Write “Y” and press enter. For now you can leave the Context window blank for auto detect. Name it, so can recognize later what model is being used and choose “Keep current” or “Local” as the terminal backend. Hermes will route all inference through this endpoint. Nothing leaves your machine.
Step 4: Set Up Your Messaging Gateway
Hermes can use Telegram as its primary interface. This is how you talk to your agent from your phone, from another machine, or just without keeping a terminal window open.
Create your bot:
- Open Telegram and search for @BotFather
- Send
/newbotand follow the prompts - Copy the API token BotFather gives you and paste it into the Hermes setup when prompted
Lock it to your account:
- Search for @userinfobot on Telegram
- Start a chat it instantly returns your Telegram User ID
- Enter that ID into the Hermes setup
The User ID step matters. It means your agent only responds to you, not anyone who stumbles across the bot link.
Now go into the same terminal window on your Mac and select the option to configure a messaging service. Choose Telegram by pressing the space bar and then enter. Enter your bot token. In the next step enter your User ID. You can use it as a Home Channel by writing “Y” and pressing enter. If you want the Agent to run in the background without you needing to start it after a restart type “Y” and confirm and start the service. Now you can chat with your own personal AI assistant.
One Thing Worth Knowing Before You Start
Hermes runs a background process called the Curator that triggers roughly every 10 conversation turns. It evaluates your conversation history and cleans up stale data to prevent context bloat over time.
It also enforces strict memory limits on what it stores about you and your environment. This is intentional as it’s part of why the system stays consistent over weeks of use rather than degrading like other agent setups tend to do.
Don’t fight it. It’s one of the better-designed parts of Hermes.
You’re Up and Running
At this point, Hermes is installed, connected to your local model through omlx, accessible via Telegram, and running as a background service on your Mac Studio.
Send your first message through Telegram and you’re in.
I’ll be writing a separate post on what a month of actually using this setup looks like — the workflows, what surprised me, and where it still falls short. I’ll link it here when it’s live.
For now, the official Hermes docs are the best place to go if you hit anything this guide doesn’t cover.A
