I got tired of paying OpenAI every time some bot filled out my contact form with “SEO services” or “crypto investment opportunities”. When 40% of your inbound is spam, those API bills hurt. I needed a way to filter, enrich, and score these leads before they polluted my CRM, and I wanted to do it on my own hardware for free.
So I set up a local AI agent using Ollama, Llama 3, and Node.js. It runs on a spare machine in my office, intercepts incoming webhooks, scores the lead from 0 to 100, and flags the junk. If it’s hot, it goes straight to the sales queue. If it’s spam, it gets dropped immediately.
Here is how to set up the local model, write the parsing logic, force a local LLM to actually output valid JSON, and handle the performance quirks (like cold starts) that will absolutely break your webhooks if you don’t plan for them.
Why Run Lead Scoring Locally?
Paying OpenAI or Anthropic to tell you that “test@test.com” is a fake lead is a waste of money. A smaller 8-billion parameter model running locally can handle this in under a second.
Running your lead scoring pipeline locally gives you three main advantages:
- Zero API Bills: Process as many leads as you want without worrying about token usage or monthly bills.
- Data Privacy: Customer PII (names, emails, messages) never leaves your server. This makes GDPR compliance a non-issue.
- No Rate Limits: You won’t get throttled by external APIs during a traffic spike. If your hardware can handle the concurrency, you’re good.
If you’re already running a webhook listener, this fits right into your pipeline. For example, you can plug this directly into a secure GoHighLevel webhook listener to score leads before they hit your CRM.
Setting Up Ollama and Your Local Model
First, install Ollama. It’s a lightweight tool that lets you run LLMs locally without tearing your hair out. Grab the installer from the Ollama official website.
Once installed, pull the Llama 3 model. I’m using the 8B version here—it’s the sweet spot for speed and reasoning on standard consumer hardware.
Run this in your terminal:
ollama pull llama3:8bTo make sure the background service is actually running, fire a quick curl request to Ollama’s local API port:
curl http://localhost:11434/api/generate -d '{ "model": "llama3:8b", "prompt": "Why is the sky blue?", "stream": false
}'If you get a JSON response back, you’re good to go. The local server is ready for our Node.js script.
Setting Up the Node.js Project
Let’s spin up a new Node.js project. We’ll need the official Ollama JS SDK, Zod for schema validation, and dotenv. If you want to keep your files organized, check out my guide on the best project folder structure.
Initialize the project and install the dependencies:
mkdir local-lead-scorer
cd local-lead-scorer
npm init -y
npm install @ollama/ollama zod dotenv
npm install --save-dev typescript @types/node tsxWe’re using TypeScript because trying to parse unstructured LLM outputs without strict types is a recipe for runtime crashes. Let’s initialize the config:
npx tsc --initCreate a .env file in your root directory to point to your Ollama instance:
OLLAMA_HOST=http://localhost:11434
PORT=3000Defining the Lead Scoring Schema with Zod
Local models can be stubborn. Even if you tell Llama 3 to output JSON, it will occasionally wrap the response in markdown blocks or add conversational filler like, *’Sure, here is your JSON:’*.
To keep our app from crashing, we’ll use Zod to validate the model’s output. If it returns garbage, we catch the error and fall back to a safe default score.
If you’ve built webhook integrations before, you might have used a similar pattern to validate webhook payloads with Zod in Express. We’re doing the exact same thing here, but for the AI’s response.
Create schema.ts and add the validation schema:
import { z } from 'zod'; export const LeadScoreSchema = z.object({ score: z.number().min(0).max(100), category: z.enum(['HOT', 'WARM', 'COLD', 'SPAM']), reasoning: z.string(), suggestedAction: z.string(), isSpam: z.boolean()
}); export type LeadScore = z.infer<typeof LeadScoreSchema>;This ensures our application always gets a predictable object containing a score, category, reasoning, suggested action, and a spam flag.
Writing the Local AI Lead Scoring Agent
Now for the core agent logic. We need to feed the raw lead data (name, email, company, message) to Ollama with a strict system prompt. We have to explicitly tell Llama 3 to act as a sales ops agent and return *only* raw JSON.
Create agent.ts and add this code:
import { Ollama } from '@ollama/ollama';
import { LeadScoreSchema, LeadScore } from './schema';
import dotenv from 'dotenv'; dotenv.config(); const ollama = new Ollama({ host: process.env.OLLAMA_HOST || 'http://localhost:11434' }); const SYSTEM_PROMPT = `
You are an expert sales operations AI. Your job is to score incoming leads based on their quality and likelihood to buy. You must return ONLY a JSON object. Do not include any conversational text, markdown blocks, or explanations outside the JSON object. Your scoring criteria:
- SPAM (Score 0-10): Obviously fake names, spammy domains (e.g., disposable mail, random letters), messages selling SEO services, crypto, or unsolicited partnerships.
- COLD (Score 11-40): Generic personal emails (gmail, yahoo) with vague questions, students, or job seekers.
- WARM (Score 41-70): Valid business emails, mid-sized companies, asking about pricing or general capabilities.
- HOT (Score 71-100): High-intent messages, clear budget mentioned, enterprise domains, specific integration questions, or requests for a demo. Your output must exactly match this JSON structure:
{ "score": number, "category": "HOT" | "WARM" | "COLD" | "SPAM", "reasoning": "brief explanation of why this score was given", "suggestedAction": "what the sales team should do next", "isSpam": boolean
}
`; export async function scoreLead(leadData: any): Promise<LeadScore> { try { const response = await ollama.generate({ model: 'llama3:8b', system: SYSTEM_PROMPT, prompt: `Score this lead data:n${JSON.stringify(leadData, null, 2)}`, stream: false, options: { temperature: 0.1, // Low temperature keeps output predictable and structured } }); const cleanResponse = response.response.trim(); // Attempt to extract JSON if the model wrapped it in markdown code blocks const jsonMatch = cleanResponse.match(/{?[sS]*}/g); const jsonString = jsonMatch ? jsonMatch[0] : cleanResponse; const parsedData = JSON.parse(jsonString); return LeadScoreSchema.parse(parsedData); } catch (error) { console.error('Error during lead scoring execution:', error); // Safe fallback if the model fails or returns invalid JSON return { score: 10, category: 'COLD', reasoning: 'Failed to parse AI response safely. Defaulting to cold.', suggestedAction: 'Review manually.', isSpam: false }; }
}Notice I set the temperature to 0.1. Keeping the temperature low is absolutely critical. If you set it higher, the model gets ‘creative’ and starts adding conversational fluff that will break your JSON parser.
Handling the Cold Start and Timeout Gotchas
When running Ollama locally, you’ll immediately hit a major bottleneck: cold start latency. If the model hasn’t been queried in a while, Ollama has to load that 4.8GB model file from your disk into RAM/VRAM.
On a standard machine without a beefy GPU, this can take 5 to 15 seconds. If your webhook listener has a strict 3-second timeout, the connection will drop before Ollama even finishes loading the model.
You have two ways to handle this:
- Keep the model loaded: Set the
num_keepaliveparameter to-1in your Ollama API call or environment variables so the model stays in memory permanently. - Process asynchronously: Don’t score the lead inside the HTTP request lifecycle. Accept the webhook, return a
202 Acceptedimmediately, and push the job to a background queue.
If you’re dealing with high volumes, you should look into how to handle rate limits with BullMQ and Redis to queue these tasks. This keeps your local CPU from pegging at 100% when multiple leads hit your server at once.
Testing the Agent with Real Payload Examples
Let’s write a quick test script to see how this handles a high-quality enterprise lead versus an obvious spam bot selling SEO services.
Create test.ts and add the test cases:
import { scoreLead } from './agent'; const enterpriseLead = { name: 'Sarah Jenkins', email: 'sjenkins@enterprise-corp.com', company: 'Enterprise Corp', website: 'https://enterprise-corp.com', message: 'We are looking to migrate our entire infrastructure to a self-hosted setup. We have 500+ users and need to discuss custom integration options and enterprise support SLA pricing. Can we schedule a demo this week?'
}; const spamLead = { name: 'John SEO Expert', email: 'john-seo-booster99@gmail.com', company: 'SEO Growth Hacks', website: 'http://growyourtrafficnow.xyz', message: 'Hey! I noticed your website is missing some key meta tags. We can guarantee page 1 rankings on Google in 30 days. No contract required. Click here to book a call!'
}; async function runTests() { console.log('--- Testing Enterprise Lead ---'); const startEnterprise = Date.now(); const enterpriseResult = await scoreLead(enterpriseLead); console.log(`Processed in ${Date.now() - startEnterprise}ms`); console.log(JSON.stringify(enterpriseResult, null, 2)); console.log('n--- Testing Spam Lead ---'); const startSpam = Date.now(); const spamResult = await scoreLead(spamLead); console.log(`Processed in ${Date.now() - startSpam}ms`); console.log(JSON.stringify(spamResult, null, 2));
} runTests();Run the test script:
npx tsx test.tsOn my M2 MacBook Pro (16GB RAM), the enterprise lead took about 1100ms to process on the second run (once the model was already warm). It correctly flagged it as HOT with a score of 95, citing the enterprise domain and specific SLA requirements.
The spam lead took 850ms, got a score of 5, and was flagged with isSpam: true.
Integrating with Your Webhook Pipeline
Now that the agent works, let’s wrap it in an Express server. This lets you route leads from your contact forms, Typeform, or landing pages directly to your local engine.
If you’re routing webhooks from external CRMs, you might want to look at how to route webhooks to multiple endpoints in Node.js to keep your pipelines clean.
Create server.ts to handle the incoming payloads:
import express from 'express';
import { scoreLead } from './agent'; const app = express();
app.use(express.json()); const PORT = process.env.PORT || 3000; app.post('/api/leads/score', async (req, res) => { const leadData = req.body; if (!leadData || !leadData.email) { return res.status(400).json({ error: 'Missing lead email data.' }); } // Return 202 immediately to avoid webhook timeouts res.status(202).json({ status: 'processing', message: 'Lead sent to local AI agent.' }); // Process in the background try { const evaluation = await scoreLead(leadData); console.log(`[Lead Processed] Email: ${leadData.email} | Score: ${evaluation.score} | Category: ${evaluation.category}`); if (evaluation.isSpam) { console.log(`Discarding spam lead from ${leadData.email}`); return; } // Here you would make an API call to your CRM or send a Slack notification // e.g., await updateCRM(leadData.id, evaluation); } catch (error) { console.error('Background processing failed:', error); }
}); app.listen(PORT, () => { console.log(`Local lead scoring server running on port ${PORT}`);
});This setup ensures your front-end form submissions don’t hang while waiting for the local model to run. The client gets an instant response, and the heavy lifting happens in the background.
Frequently Asked Questions
Can I run this on a cheap VPS?
No. Running LLMs locally requires decent hardware. A cheap $5/month VPS with 2GB of RAM will crash the second you try to load Llama 3. You need at least 8GB of RAM (16GB is highly recommended) and a modern CPU. For a reliable local setup, a dedicated Mac Mini (M1/M2/M3) or a small form-factor PC with an Nvidia GPU works best.
What happens if Ollama crashes or stops responding?
In our agent.ts code, we wrapped the Ollama call in a try/catch block that falls back to a safe default score so your pipeline doesn’t break. For production, you should also use a process manager like PM2 to monitor and restart the Ollama service if it runs out of memory.
Can I use smaller models to make it faster?
Yes. If Llama 3 (8B) is too slow on your machine, try smaller models like Phi-3 (3.8B) or Qwen2 (1.5B). Just run ollama pull phi3 and update the model string in your code. They process much faster, though they might occasionally struggle with complex JSON formatting.
Next Steps
Now that you have a local AI agent scoring your leads, you can start chaining tasks together—like automatically drafting email replies for hot leads. To see how to handle that side of things, check out my tutorial on how to build an AI email reply agent with Ollama and Node.js.
.
.
