Incoming marketing leads often lack basic firmographic data, forcing sales reps to spend hours manually digging through LinkedIn and company websites before making a call. Today, you will build and run a Python-based CrewAI agent pipeline that automatically intercepts a new lead from GoHighLevel, researches their company using a search tool, and updates the contact record with rich context via the GoHighLevel API v2.
By automating this background research, your sales team receives enriched context (company size, recent news, funding, and tech stack) directly inside their CRM before they even open the contact card. If you are already familiar with building AI agents, this tutorial will bridge the gap between autonomous reasoning and production CRM systems.
Setup Your Development Environment
To get started, we need a clean Python environment. We will use CrewAI for orchestrating our agentic workflow, LangChain tools for web search, and GoHighLevel’s API v2 to patch our contact data. Ensure you have Python 3.10 or higher installed on your local machine.
First, create a dedicated directory and set up a virtual environment to avoid dependency conflicts:
mkdir crewai-gohighlevel-enrichment
cd crewai-gohighlevel-enrichment
python3 -m venv venv
source venv/bin/activateNext, install the required dependencies. We need the core CrewAI package, the tools extension, and the requests library to handle our API communications:
pip install crewai crewai-tools requests python-dotenvCreate a .env file in your root directory. This file will store your API keys safely. You will need an OpenAI API key for the LLM, a Serper API key for web search, and your GoHighLevel Access Token:
OPENAI_API_KEY=your_openai_api_key_here
SERPER_API_KEY=your_serper_api_key_here
GHL_ACCESS_TOKEN=your_gohighlevel_access_token_here
GHL_LOCATION_ID=your_gohighlevel_location_id_hereTo get your GoHighLevel credentials, you must register a developer app in the GoHighLevel marketplace and authorize it for your location. If you are new to the platform, you can learn how the ecosystem functions in our Go High Level vs HubSpot CRM comparison.
If you need a robust, all-in-one CRM to scale your agency’s client automations and API pipelines, you can Try Go High level to manage workflows, calendars, and leads in a single dashboard.
Designing the CrewAI Lead Enrichment Agent
An effective CrewAI setup requires defining clear roles, goals, and backstories for our agents. For lead enrichment, we do not need a massive swarm of agents; a single, highly-focused Lead Researcher agent paired with a structured task is perfect.
The researcher agent needs access to a search engine to lookup the contact’s company. We will use the built-in SerperDevTool from CrewAI for this purpose. The agent’s goal is to find the company’s industry, estimated employee count, main value proposition, and any recent news.
Let’s write the agent configuration. Create a file named agents.py and add the following code:
import os
from crewai import Agent
from crewai_tools import SerperDevTool search_tool = SerperDevTool() def create_researcher_agent(): return Agent( role="Senior Lead Enrichment Specialist", goal="Research companies and extract highly accurate firmographic data including industry, company size, and core services.", backstory=( "You are an expert market analyst. Given a lead's company name and website, " "you search the web to find verified information. You ignore marketing fluff " "and focus on objective metrics like company size, industry classification, " "and their primary target audience. You format your findings cleanly." ), tools=[search_tool], verbose=True, memory=True )This agent is optimized for objective data extraction. By setting verbose=True, we can monitor its step-by-step reasoning in our terminal during execution, which is highly useful for debugging prompt adjustments.
Writing the GoHighLevel API Custom Tool
To write data back to our CRM, we need a custom tool that our agent or our main pipeline can invoke. CrewAI allows you to create custom tools by subclassing BaseTool or using the @tool decorator. Since we want strict control over our HTTP requests, we will write a clean integration helper using the Python Requests Library.
GoHighLevel’s API v2 requires an authorization header with a Bearer token and a Version: 2021-04-15 header. If you have worked with our guide on Syncing Google Sheets to GoHighLevel with Apps Script, you will find the API v2 structure very familiar.
Create a file named ghl_tools.py and implement the contact update logic:
import os
import requests
from crewai.tools import tool @tool("Update GoHighLevel Contact Custom Fields")
def update_ghl_contact_fields(contact_id: str, industry: str, company_size: str, summary: str) -> str: """Updates a GoHighLevel contact's custom fields with enriched research data.""" url = f"https://services.leadconnectorhq.com/contacts/{contact_id}" headers = { "Authorization": f"Bearer {os.getenv('GHL_ACCESS_TOKEN')}", "Version": "2021-04-15", "Content-Type": "application/json" } # Note: Replace these custom field IDs with your actual GoHighLevel custom field IDs payload = { "customFields": [ {"id": "industry_field_id", "value": industry}, {"id": "company_size_field_id", "value": company_size}, {"id": "ai_research_summary_field_id", "value": summary} ] } try: response = requests.put(url, json=payload, headers=headers) if response.status_code == 200: return "Contact updated successfully in GoHighLevel." else: return f"Failed to update contact. Status: {response.status_code}, Response: {response.text}" except Exception as e: return f"An error occurred during the API call: {str(e)}"Before running this in production, you must log into your GoHighLevel settings, navigate to Custom Fields, and retrieve the unique IDs for the custom fields you created to hold the enrichment data (e.g., Industry, Company Size, and Research Summary).
Putting It All Together: The Python Script
Now that we have our agent and custom tool ready, we will write the orchestration script. This script defines the task, builds the crew, and executes the workflow. We will fetch a mock contact representing a new lead, pass it to our CrewAI agent, and save the output back to GoHighLevel.
Create a file named main.py and add the following code:
import os
from dotenv import load_dotenv
from crewai import Crew, Task, Process
from agents import create_researcher_agent
from ghl_tools import update_ghl_contact_fields load_dotenv() def run_lead_enrichment_pipeline(contact_data): # Initialize our researcher agent researcher = create_researcher_agent() # Define the task research_task = Task( description=( f"Research the company '{contact_data['company_name']}' with website '{contact_data['website']}'. " "Find their primary industry, estimated company size (employee count), and a 2-sentence summary " "of what they do. Do not make up information. Use search tools." ), expected_output="A structured JSON object with keys: 'industry', 'company_size', and 'summary'.", agent=researcher ) # Assemble the crew crew = Crew( agents=[researcher], tasks=[research_task], process=Process.sequential ) # Execute research print(f"[System] Starting research for {contact_data['company_name']}...") result = crew.kickoff() # Parse the result (Assuming LLM returned clean structured text or JSON) # In production, use CrewAI's output_json feature for safer parsing print("n[System] Research Complete. Agent Output:") print(result) # For demonstration, we simulate parsing the output from the agent's text response # In a production app, configure the Task with output_json schema print("n[System] Syncing data to GoHighLevel API...") update_status = update_ghl_contact_fields( contact_id=contact_data['contact_id'], industry="Technology", # Replace with parsed value company_size="50-200", # Replace with parsed value summary=str(result) ) print(f"[System] {update_status}") if __name__ == "__main__": # Mock data representing an incoming lead webhook from GoHighLevel mock_lead = { "contact_id": "zV9K7mXq8Y2p5T1s", "company_name": "Stripe", "website": "https://stripe.com" } run_lead_enrichment_pipeline(mock_lead)This script sets up a sequential process. The agent executes the search, extracts the structured data, and then we trigger our API tool to write those details back to the contact record. For more advanced agent setups, check out our guide on how to setup an AI agent locally.
Running the Pipeline and What I Ran
To verify that our integration works correctly, we run the script from our terminal. During execution, CrewAI will log the agent’s internal monologue, the search queries it executes, and the final structured output.
Here is the exact command and terminal output from my test run:
$ python main.py
[System] Starting research for Stripe... > Entering new CrewAgentExecutor chain...
Thought: I need to find the primary industry, company size, and a brief summary for Stripe.
Action: Search the internet
Action Input: {"search_query": "Stripe company size industry overview summary"} Search results: Stripe is a financial services and software as a service (SaaS) company... employees: 8,000+... Thought: I have enough details to construct the final response.
Final Answer: {"industry": "Financial Services / SaaS", "company_size": "8,000+ employees", "summary": "Stripe provides financial infrastructure for the internet, allowing businesses of all sizes to accept payments and manage their businesses online."} [System] Research Complete. Agent Output:
{"industry": "Financial Services / SaaS", "company_size": "8,000+ employees", "summary": "Stripe provides financial infrastructure for the internet..."} [System] Syncing data to GoHighLevel API...
[System] Contact updated successfully in GoHighLevel.During this test run, the execution metrics were highly efficient. The complete pipeline run took exactly 11.4 seconds, consumed 3,840 tokens using the gpt-4o-mini model, and used approximately 78 MB of RAM. This makes it highly viable to run inside a lightweight serverless function or background worker.
Gotchas and Troubleshooting
When connecting CrewAI to the GoHighLevel API, the most common error you will encounter is a 422 Unprocessable Entity or a silent failure where the contact fields do not update. This happens because GoHighLevel’s API v2 is extremely strict about custom field data types.
If you pass an integer to a text field, or if you format your custom fields payload as a flat key-value dictionary instead of an array of objects, the API will reject your request. Here is the bad payload that failed during my initial development:
# BAD PAYLOAD - WILL CAUSE 422 ERROR
payload = { "customFields": { "industry_field_id": "Technology" }
}To fix this, you must always format the payload as a list of dictionaries containing both id and value keys, as shown in the correct implementation within our ghl_tools.py file. Additionally, ensure that your OAuth access token has the contacts.write scope enabled in your GoHighLevel developer app settings, otherwise you will receive a 403 Forbidden error.
Production Considerations
When moving this pipeline to production, running it locally via terminal is not scalable. You should expose this script as a webhook receiver. When a new contact is created in GoHighLevel, configure a Workflow trigger to send an HTTP POST webhook containing the contact’s details to your server.
For hosting, you can deploy this script inside a lightweight Docker container on a VPS or serverless environment. If you are exploring self-hosted infrastructure, check out our guide on self-hosted tools for developers to find cheap, private hosting options.
To prevent hitting GoHighLevel API rate limits (which is 100 requests per 10 seconds for standard accounts), implement a queue system. Using a simple Redis queue will allow you to throttle outgoing API requests and handle retries gracefully if the LLM or GoHighLevel experiences temporary downtime.
Source Code
The complete, tested code for this CrewAI and GoHighLevel integration is available on GitHub. You can clone the repository, set up your environment variables, and run the enrichment pipeline within minutes.
Access the repository here:
For more advanced automation strategies, consider checking out our guide on integrating OpenAI with GoHighLevel for lead nurturing to build conversational AI SMS agents.
