I got tired of paying OpenAI bills just to filter spam and draft basic support replies. Today, we are building a local AI email agent using Node.js and Ollama that runs completely on your own machine.
By the end of this guide, you will have a working local Express server that accepts incoming email webhooks, classifies the content, and drafts a contextual reply using a local LLM.
Why Local LLMs Make Sense for Email Processing
Most developers default to cloud APIs when building AI agents. I did the same when I built an AI email assistant with Make and OpenAI. It worked fine, but running every incoming support ticket or junk email through a paid API gets expensive fast.
Data privacy is another major issue. Sending sensitive customer emails, invoice details, or internal queries to third-party APIs can trigger compliance headaches. When you run a model locally via Ollama, your data never leaves your server.
Using a local setup means you pay nothing per token. You can process millions of emails for the cost of the electricity running your machine. The trade-off is speed and system resource usage, but we can manage that with proper queueing and lightweight models.
Setting Up Ollama and Your Node Project
First, you need Ollama running on your machine. Download it from the official Ollama website and install it for your operating system.
Once installed, open your terminal and pull a lightweight model. I am using Llama 3 for this build, but Mistral or Phi-3 also work great for classification tasks.
ollama run llama3Keep that terminal window open or let it run in the background. Now, let’s initialize a new Node.js project. Create a directory and install the required dependencies.
mkdir local-email-agent
cd local-email-agent
npm init -y
npm install express @ollama/ollama dotenvWe are using Express to handle incoming webhooks and the official Ollama JS client to interact with our local model. Make sure you have Node.js installed on your system before running these commands.
Building the Webhook Listener for Incoming Emails
When an email comes in from a service like Mailgun, SendGrid, or GoHighLevel, it usually triggers a webhook. We need a secure endpoint to receive this payload.
If you are routing webhooks from external platforms, safety is a priority. Check out my guide on how to build a secure GoHighLevel webhook listener to see how to verify signatures and protect your endpoint.
Create a file named server.js and set up the basic Express server structure.
const express = require('express');
const { processEmail } = require('./agent'); const app = express();
app.use(express.json()); app.post('/webhook/email', async (req, res) => { const { sender, subject, body } = req.body; if (!sender || !body) { return res.status(400).json({ error: 'Missing sender or email body' }); } // Send immediate 202 accepted response to prevent webhook timeouts res.status(202).json({ status: 'Processing email in background' }); try { console.log(`Received email from ${sender}: "${subject}"`); const result = await processEmail({ sender, subject, body }); console.log('Agent processing complete:', result); } catch (error) { console.error('Failed to process email:', error.message); }
}); const PORT = process.env.PORT || 3000;
app.listen(PORT, () => { console.log(`Email agent webhook listener running on port ${PORT}`);
});Notice that we return a 202 Accepted status immediately. Local LLMs can take several seconds to generate responses. If you keep the webhook connection open while Ollama processes the prompt, the sending service will likely timeout and retry, creating an infinite loop of duplicate processes.
Writing the Ollama AI Orchestrator
Now let’s build the core AI logic. We need the model to do two things: classify the email (e.g., Support, Sales, Spam, General) and draft an appropriate reply if it is not spam.
Create a file named agent.js. We will write the prompt engineering logic here. To make the output easy to parse in JavaScript, we will instruct Ollama to return a clean JSON object.
const { ollama } = require('@ollama/ollama'); async function processEmail({ sender, subject, body }) { const systemPrompt = `You are an automated email triage assistant. Analyze the incoming email and return a JSON object with the following keys:
- "category": Must be one of "support", "sales", "spam", or "general".
- "sentiment": Must be "positive", "neutral", or "negative".
- "replyDraft": A professional email reply draft. If the category is "spam", leave this as an empty string.
- "shouldReply": Boolean (true or false) indicating if we should send the draft. Return ONLY raw JSON. Do not include markdown formatting, backticks, or conversational filler.`; const userPrompt = `Sender: ${sender}
Subject: ${subject}
Body: ${body}`; try { const response = await ollama.chat({ model: 'llama3', messages: [ { role: 'system', content: systemPrompt }, { role: 'user', content: userPrompt } ], format: 'json' }); const parsedResult = parseLLMResponse(response.message.content); return parsedResult; } catch (error) { console.error('Ollama API error:', error); throw error; }
} function parseLLMResponse(rawContent) { try { return JSON.parse(rawContent.trim()); } catch (e) { console.warn('Failed to parse raw JSON directly. Attempting regex cleanup.', e.message); // Fallback parser in case the model wraps JSON in markdown blocks const jsonRegex = /{[sS]*}/; const match = rawContent.match(jsonRegex); if (match) { try { return JSON.parse(match[0]); } catch (innerError) { throw new Error('Could not parse extracted JSON string'); } } throw new Error('Invalid JSON format returned by LLM'); }
} module.exports = { processEmail };Using the format: 'json' option in the Ollama client forces the model to output valid JSON. However, models still occasionally wrap their output in markdown blocks. The helper function parseLLMResponse uses a regular expression to extract and parse the JSON safely.
Testing the AI Agent with Real Payloads
Let’s run our server and test it using a mock payload. Start your Express server in your terminal.
node server.jsOpen another terminal window and use curl to send a mock support email to your webhook endpoint.
curl -X POST http://localhost:3000/webhook/email
-H "Content-Type: application/json"
-d '{ "sender": "customer@example.com", "subject": "Billing issue: charged twice", "body": "Hi team, I noticed two charges of $49 on my credit card statement this morning. Can you please refund one of them? Thanks!"
}'Your Express server should immediately respond with {"status":"Processing email in background"}. After a few seconds, you will see the classification and drafted reply printed in your console.
Received email from customer@example.com: "Billing issue: charged twice"
Agent processing complete: { category: 'support', sentiment: 'negative', replyDraft: 'Hi customer@example.com,nnThank you for reaching out. I apologize for the double charge on your account. I have forwarded this to our billing team to process your refund immediately. You should see the credit back on your card in 3-5 business days.nnBest regards,nSupport Team', shouldReply: true
}Let’s test it with a spam email to verify that the agent correctly identifies junk and skips drafting a reply.
curl -X POST http://localhost:3000/webhook/email
-H "Content-Type: application/json"
-d '{ "sender": "spammer@cheapdeals.com", "subject": "Buy cheap backlinks now!", "body": "Increase your website traffic overnight with our premium SEO backlinks package. Click here to buy now for only $10!"
}'The console should output a result classifying the email as spam, with an empty reply draft and shouldReply: false.
How to Handle CPU Bottlenecks and Slow Responses
If you run this on a standard development machine without a dedicated GPU, Ollama will run on your CPU. This can cause processing times to jump from 1 second to 15 seconds per email.
When you receive multiple webhooks at the same time, your CPU will spike to 100%, and Ollama will queue requests internally. This can cause your server to lag or crash.
To fix this, you should separate the webhook listener from the AI processor using a background worker queue. I recommend using BullMQ with Redis to manage this. You can read my guide on how to build a Redis-backed idempotency middleware to see how to prevent processing duplicate webhooks.
By pushing incoming emails into a Redis queue, your Express server stays fast, and your background worker can process emails one by one, keeping your CPU usage stable.
Common Gotchas and How I Fixed Them
When building this system, I hit a few annoying issues. Here is how you can avoid them.
1. The “JSON Mode” Hallucination
Even with format: 'json' enabled, older models like Llama 2 or smaller 2B parameter models sometimes add introductory text like “Here is your JSON output:” before the actual JSON block. If you encounter this, use a strict system prompt and ensure your parsing function uses the regex fallback I included in the code above.
2. High Memory Usage
If you leave Ollama running in the background, it keeps the model loaded in your system RAM. If you need to free up memory on your server when the agent is idle, you can send an empty post request to Ollama to unload the model manually.
You can read more about managing memory and API structures on the MDN Web Docs when building custom fetch wrappers for your local services.
Frequently Asked Questions
Can I run this on a cheap $5 VPS?
No. Ollama requires significant RAM and CPU. A standard 7B model needs at least 8GB of RAM to run smoothly. If you are hosting this yourself, look for a VPS with at least 4 vCPUs and 8GB of RAM, or use a GPU-enabled cloud instance.
How do I actually send the drafted email?
Once the agent generates the replyDraft, you can pass it to an email service API like Resend, Mailgun, or Nodemailer. Check the shouldReply boolean first to make sure you aren’t replying to spam.
Which local model is best for this?
Llama 3 (8B) provides the best balance of speed and accuracy for text classification and drafting. If you are extremely limited on system resources, try gemma2:2b or phi3.
Next Steps for Your Local AI Pipeline
Now that you have a local AI agent triaging your emails, you can expand its capabilities. Instead of just drafting a reply, you can connect it to other local agents to research the sender or update your CRM.
If you want to take your local automation setup to the next level, check out my guide on how to connect CrewAI to GoHighLevel API for automated lead enrichment to build multi-agent workflows that run alongside your email agent.
