How to Replace Claude Cowork With a Free Local AI Model

This guide walks through how to replace Claude Cowork with a free open-weight AI agent.
Key takeaways
- Hosting your own model takes three pieces: the model, an agent harness, and hardware with enough memory to run it.
- Qwen3.5 9B uses about 8GB of RAM on a 16GB Mac Mini, which leaves roughly 4GB of headroom for context.
- Expect about 17 to 18 tokens per second. That's slow next to Claude, and fine for overnight jobs that aren't customer-facing.
- Use a frontier model for planning and strategy, and the local model for the daily work.
- Your AI cost becomes a one-time hardware purchase instead of a token bill that can grow without warning.
Watch on YouTube: Replace Claude Cowork with Free AI Models
Why run your own model
A lot of the companies I work with expect Claudeflation. Their bet, and mine, is that OpenAI and Anthropic will raise token prices drastically once they go public. So they're working out now which tasks they can move to an open-weight model on their own infrastructure, and which still need a frontier model.
Most of them run a two-model setup. A frontier model from Anthropic or OpenAI does the strategic work, like planning the project and sometimes writing the first version of the codebase. Then an open-weight model hosted on their own hardware does the project work every day and keeps updating the application.
They pay for that infrastructure once, so the cost is easy to predict. Nobody has to worry about tokens running out of control.
If you build the skill set to host your own model, connect it to an agent harness, and hand it routine tasks, you'll be miles ahead of most companies in this space.
Local AI use cases for sales teams
The sales work that fits a local model is high-volume, repetitive, and can wait until morning. You hand the model a batch at the end of the day, it works through the night, and your reps review the output when they log on. The cost stays flat as volume grows, so drafting emails for 500 contacts costs the same as drafting them for 5. Your call transcripts, pipeline exports, and customer notes also stay on hardware you own.
| Use case | What you hand the model | What you get back |
|---|---|---|
| Writing cold emails | A CSV of enriched contacts, your messaging, and a few emails that have worked | A first-draft sequence for each contact, ready for a rep to edit |
| Pipeline review | A CRM export of open deals with stage, amount, close date, and last activity | Deals that are stalled, past their close date, or missing a next step, plus a question to ask the rep on each one |
| Updating CRM notes | The day's call transcripts or a rep's rough notes | Clean notes broken out by pain, stakeholders, and next steps, formatted to match your CRM fields |
| Meeting prep | Account notes, the email thread, and the last call transcript | A one-page brief on who's attending, where the deal stands, and the open questions to cover |
| Call follow-ups | A discovery or demo call transcript | A recap email with the agreed next steps for the rep to send |
| Lead list cleanup | A raw list export from your data provider | Deduplicated rows, standardized job titles, and each contact tagged by persona: economic buyer, champion, or end user |
| Inbound lead triage | The day's demo requests and form fills | Each lead scored against your ICP with a one-line reason, so reps start the morning with the best fits |
| Content repurposing | A webinar or YouTube transcript | Draft LinkedIn posts and short video scripts your reps can record |
Have a rep review anything that goes to a prospect. A 9B model's drafts need editing before they're ready to send.
What you need: the model, the harness, and the hardware
| Piece | What I use |
|---|---|
| Model | Qwen3.5, 9 billion parameters |
| Agent harness | DeepSeek's agent harness (free and open source) |
| Model server | Ollama |
| Hardware | Mac Mini with 16GB of RAM |
I picked Qwen3.5 9B because it was small enough to host on the Mac Mini and smart enough to be useful. The DeepSeek harness gives me the desktop app experience I wanted from Claude Cowork. I run the harness on my MacBook Pro, and the model runs on the Mac Mini.
Why memory headroom matters
The model takes about 8GB of the Mac Mini's 16GB. After the operating system and everything else running on the machine, I have about 4GB left, and you need that gap. Your conversation's context is stored in the leftover memory, so without it you can't hold a long conversation with the model. With almost no free memory, tokens also generate so slowly that the model is close to unusable.
When you choose a model, make sure it leaves you headroom after it loads. About 4GB has worked for me.
Step 1: Install Ollama and pick a model
Go to ollama.com/download and install Ollama on the machine that will serve the model. You can run the install command from that page in your terminal.
Next, open the Models page on ollama.com and find a model that fits both your hardware and your use case. I wanted something around 8GB, and you can look for models that are good at coding or tool calling. I ended up on the Qwen3.5 family.
Pull the model and start chatting with it from your terminal:
ollama run qwen3.5:latest
For me, that pulled the 9B version, a download of about 6.6GB. Once it finishes, the model lives on your machine and you can chat with it right in the terminal.
Step 2: Serve the model to your network
I changed three settings before connecting anything:
- Bind Ollama to a port on the Mac Mini so other machines on my network can reach it.
- Raise the context window to about 32,000 tokens. The default is 4,096.
- Turn on flash attention so Ollama makes better use of the available memory.
OLLAMA_HOST=0.0.0.0:11434 \
OLLAMA_CONTEXT_LENGTH=32768 \
OLLAMA_FLASH_ATTENTION=1 \
ollama serve
If the Ollama desktop app is already running, quit it first so the two don't fight over the port.
Step 3: Test speed and reachability
Run a quick prompt with the verbose flag:
ollama run qwen3.5:latest "hi" --verbose
The verbose output includes the eval rate, which tells you how many tokens per second the model generates on your hardware. My Mac Mini produces about 18 tokens per second.
Unless you have serious GPU infrastructure, an open-weight model will be much slower than what you get in Claude or ChatGPT. I'm fine with 18 tokens per second because none of this is customer-facing. The model can run all night on a long coding task, a website, or a slideshow, and since retries cost nothing, I can rerun a job as many times as I need. Expect the speed to drop a little once you call the model from another machine.
Then check that your laptop can reach the model. From the laptop, run:
curl http://mac-mini.local:11434/v1/models
Replace mac-mini.local with your server's hostname or IP address. If the response includes the model ID, your laptop can reach the model.
Step 4: Connect the model to the DeepSeek agent harness
DeepSeek makes an agent harness similar to Claude Cowork. It's free, open source, and installs like any other Mac app from the DeepSeek website. You can use it with DeepSeek's own models or plug in your own.
To add your local model:
- Click Settings in the bottom left.
- Go to Models, then Model provider, then Custom model API.
- Name the provider. I called mine Mac Mini.
- Set a display name you'll recognize in the model picker. I use the model's name.
- Enter the base URL: your server's address, the port, and
/v1, which is Ollama's OpenAI-compatible chat completions API. Mine looks likehttp://mac-mini.local:11434/v1. - Enter
ollamaas the API key. That's Ollama's default. - Click Add model and enter the model ID,
qwen3.5:latest. - Click Create provider.
Now select your model in any chat and send a message. I asked mine for its name, and it said it was my AI coding assistant powered by the DeepSeek harness. The reply took about 2 minutes, used about 4% of the context window, and ran at about 17 tokens per second.
What I use it for
The model runs on the Mac Mini, the harness runs on my MacBook Pro, and I don't pay for tokens. I did have to buy the Mac Mini, but that's a cost I can predict.
I use this setup for routine work that doesn't need Opus or Fable: long-running coding tasks, spreadsheets, PowerPoints, and websites. I kick off the job, let it run overnight, and come back to finished work. Planning and strategy still go to a frontier model.
FAQ
Is a local AI model really free?
There's no per-token cost. You pay once for the hardware. For me, that was a Mac Mini with 16GB of RAM.
Can a 16GB Mac Mini run a local LLM?
Yes, if you pick a model in the right size range. Qwen3.5 9B uses about 8GB of RAM and generates about 18 tokens per second on mine, with roughly 4GB left for context.
How fast is a local model compared to Claude?
Much slower. My first reply through the harness took about 2 minutes at roughly 17 tokens per second. That's fine for jobs you run overnight and a bad fit for anything a customer is waiting on.
Do I need two computers?
No. I serve the model from the Mac Mini and run the harness on my MacBook, but you can install Ollama on the same machine as the harness and point the harness at http://localhost:11434/v1.
When should I still use Claude or ChatGPT?
For planning, strategy, and writing the first version of a project. The companies I work with use frontier models for that and hand the daily work to an open-weight model.
What API key does Ollama use?
By default, ollama.
Want help putting AI to work in your sales team?
Acquisales is a GTM engineering firm that helps companies adopt AI to drive more revenue. If you're struggling to make outbound work or to generate pipeline consistently, book a free AI Sales Audit and we'll figure out how AI can scale your revenue function.
