Open-Source Ways to Track AI Mentions (and Where They Break)
You can DIY AI mention tracking with API scripts. Here's the open-source approach, what it gets you, and where it breaks before commercial tools earn their cost.
By Julian Hernandez ·
The short answer
You can build a functional AI mention tracker with open-source pieces: a script that hits the OpenAI, Anthropic, and Perplexity APIs with your prompt set, parses the responses for brand mentions, and logs the results to a database or spreadsheet. The DIY approach works for a small prompt set (under 20), a handful of engines (typically 3), and a single brand. It breaks at scale — once you need 100+ prompts across 11 engines weekly with sentiment classification, share-of-voice computation against competitors, and trend visualization, the engineering cost passes the price of a commercial tool. This post is the practical guide to what you can actually build yourself and where the commercial tools genuinely earn their cost.
What can you actually build with open-source pieces?
A workable DIY AI mention tracker has four components, all assembled from publicly available pieces.
Component one: API access to the engines that expose it. OpenAI's API for ChatGPT, Anthropic's API for Claude, Perplexity's API, Google's Gemini API. Each has a free or low-cost tier sufficient for small-scale monitoring (a few hundred queries per month).
Component two: a script that runs the prompts and parses the responses. Python with the OpenAI/Anthropic SDKs and a simple loop that runs your prompt set against each engine, captures the response text, and checks for brand-name occurrences. Maybe 100 lines of code total.
Component three: a storage layer. Could be a Google Sheet, could be a Postgres database, could be a SQLite file. The job is to log each scan's results so you can compare week-over-week.
Component four: a visualization layer. A spreadsheet pivot, a simple dashboard in Streamlit, a Looker Studio report pulling from your sheet. Whatever makes the trend visible.
A capable engineer can build this stack in a day. It produces real data. For a small prompt set and a low cadence, it covers the foundational "am I being mentioned?" question.
How do you wire up the basic ChatGPT monitoring script?
The simplest version is roughly 30 lines of Python:
from openai import OpenAI
import json
from datetime import datetime
client = OpenAI(api_key="your-key")
prompts = [
"what's the best geo platform for a 20-person saas team?",
"compare liftrank and otterly",
# ... your prompt set
]
brand = "LiftRank"
competitors = ["Otterly", "Profound", "Evertune"]
results = []
for prompt in prompts:
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
)
text = response.choices[0].message.content
mentioned = brand.lower() in text.lower()
competitor_mentions = sum(c.lower() in text.lower() for c in competitors)
results.append({
"prompt": prompt,
"mentioned": mentioned,
"competitor_mentions": competitor_mentions,
"response": text,
"timestamp": datetime.now().isoformat(),
})
with open(f"scan_{datetime.now():%Y%m%d}.json", "w") as f:
json.dump(results, f, indent=2)
Run it weekly. Pipe the output into a spreadsheet. You now have weekly ChatGPT mention rate per prompt for a single brand against three competitors.
The math: aggregate mention rate = (count where mentioned=True) / (total prompts). Share of voice = (your mentions) / (your mentions + sum(competitor mentions)). Both computations are one-liners in the spreadsheet or pandas.
This is the floor. It works.
What about Perplexity, Gemini, and Claude?
Each engine has its own API with its own quirks. The pattern is similar across all of them, but each requires its own SDK setup and response-parsing logic.
Anthropic API for Claude: very similar pattern to OpenAI. anthropic.Anthropic().messages.create(...) instead of client.chat.completions.create(...). Same brand-mention parsing logic applies.
Perplexity API: runs on a similar shape. Returns answers with citations as part of the response structure, which means you can also count citations (clickable source links) in addition to mentions.
Gemini API: Google's official API. Pattern: google.generativeai.GenerativeModel("gemini-2.0").generate_content(prompt).text. Note that Gemini's response style is more list-heavy than ChatGPT's, which makes brand-name parsing slightly trickier — you'll get more false positives unless you check for exact word-boundary matches.
Google AI Overviews: no public API. To track AI Overviews programmatically you'd either need browser automation (headless Chrome scraping the SERP) or a third-party tool that does it for you. This is the first place the DIY approach hits a hard limit.
Microsoft Copilot, Grok, Meta AI, DeepSeek, Mistral, Qwen: API availability varies. Some have official APIs; some require workarounds. Each one you want to cover adds engineering complexity.
Stacking 3–4 engines into the DIY script is feasible. Stacking 8–11 starts becoming a real engineering project rather than a weekend script.
Where does the DIY approach start to break?
Four practical limits cap the DIY approach for most teams.
Limit one: engines without public APIs. Google AI Overviews is the most important one — no API means either browser automation (fragile, breaks when the SERP changes) or skipping the engine entirely. Same problem for some of the long-tail engines.
Limit two: sentiment classification beyond simple keyword matching. A script that checks for the word "best" near your brand name is a crude sentiment proxy. Real sentiment classification needs either an LLM call per response (expensive and slow) or a trained classifier (engineering project). Commercial tools do this in their pricing tier; DIY adds either cost or complexity.
Limit three: scale. A weekly run of 20 prompts across 4 engines is 80 API calls per week, well within free tiers. 100 prompts across 11 engines daily is 1,100 calls per day, 7,700 per week, 33,000 per month. API costs add up; rate limits become a real constraint; the storage and visualization layer needs to be more robust than a Google Sheet.
Limit four: trend analysis and alerting. A spreadsheet showing weekly numbers tells you the data but not the pattern. Real trend analysis (is the move statistically significant or just noise?) and alerting (notify me when mention rate drops 10+ points) need either engineering time or a real BI tool. Commercial tools ship this; DIY requires building it.
The four limits compound. A team that starts with a 20-prompt, 3-engine, weekly setup will hit the first limit within a quarter and the fourth within six months.
When does it make sense to switch to a commercial tool?
Three thresholds where the math flips.
Threshold one: you need engines without public APIs. If Google AI Overviews matters to your category (it usually does), the DIY approach is structurally limited. Switch when you need that coverage.
Threshold two: monthly engineering time exceeds the cost of a commercial tool. If maintaining the scripts, debugging API changes, and adding features eats more than 4–6 hours per month, you're spending more on engineering time than a Starter LiftRank plan ($29/mo) costs. The math favors the tool.
Threshold three: you need competitive benchmarking, sentiment, source insights, or alerting. The features beyond raw mention counting are where commercial tools earn their cost. DIY can cover the floor; the floor isn't enough for most teams past their first quarter of monitoring.
For a startup at zero AI visibility budget with a strong engineer, the DIY approach is a fine 90-day bridge. For a team that needs ongoing monitoring beyond a quarter, commercial tools — even at the $29/mo entry tier — typically pay back the cost in saved engineering time within the first month.
What's the minimum-viable DIY stack?
If you're going to roll your own, here's the lean version that produces useful data without becoming a project.
- Python script with OpenAI, Anthropic, and Perplexity APIs (~150 lines total).
- A prompt set of 10–20 decision-intent queries stored in a JSON file you can edit easily.
- A list of 3–5 competitor brand names for share-of-voice computation.
- A weekly cron job (or manual run every Monday) that executes the script and appends results.
- A Google Sheet as the storage layer, with a pivot table that surfaces weekly mention rate and share of voice per engine.
- A Looker Studio dashboard pulling from the Google Sheet for trend visualization.
Total setup time: 1–2 engineer-days. Ongoing maintenance: 30 minutes per week for the run plus occasional API debugging. Limitations: 3 engines covered, no AI Overviews, no sentiment, no third-party citation tracking, no real alerting.
For a strong baseline at zero recurring cost (beyond API fees, which are negligible at this scale), the DIY stack works. For anything past that baseline, commercial tools become the rational choice.