Keyword Research Tools: Where the Numbers Come From, and Which Ones Show AI Fan-Out

Two tools report two volumes for the same keyword and neither is the truth, because only one source in this market is not an estimate. What Google's own figure actually is, why Trends numbers are not volumes, why keyword difficulty is every vendor's own invention, which data is weakest for Indian and Indian-language queries, and which tools expose the sub-questions AI assistants decompose a query into.

A tool says 2,400 a month. Another says 880. A third says 1,900. The client asks which is right and the honest answer is none of them, because with one exception every search volume in every keyword tool is a modelled estimate, and the models disagree most on exactly the queries Indian businesses care about.

This piece is about where the numbers come from, which of them mean what they appear to mean, and how to pick tools on that basis rather than on feature lists. Everything attributed to documentation was read on 2 November 2026, with sources at the end.

The one source that is not an estimate

Search Console. It reports the queries that actually brought impressions and clicks to your site, measured rather than modelled, for free, with no sampling you need to worry about for this purpose.

It has two limits worth stating plainly. It only shows queries you already appear for, so it cannot tell you about demand you are invisible to. And it anonymises low-volume queries, so the long tail is partly withheld.

Within those limits it beats every paid estimate for your own site, and most teams underuse it badly. The commonest missed opportunity is the set of queries where you have impressions and almost no clicks, which is a title and snippet problem rather than a content problem, and the second is the set where you rank on page two for something you never targeted. Both are visible in the export, and the pipeline for getting at them properly is in how to export all your Search Console data with Python.

Start every keyword project here, not in a paid tool. It costs nothing and it is the only data about your own market that is not someone’s model.

What Google’s own volume figure actually is

Keyword Planner, free with a Google Ads account, is the closest thing to an authoritative estimate, and its documentation makes two things clear that most people using it do not know.

First, the headline number is an average. The field is avg_monthly_searches and it is a 12-month average, with the date range defaulting to the past 12 months. So a keyword that does 12,000 searches in October and almost nothing for the rest of the year reports as a modest monthly figure, and a keyword with steady demand at the same annual total looks identical. For anything seasonal, which in India means most festival, wedding, exam and monsoon-related demand, the headline figure actively misleads. The fix is in the same API: monthly_search_volumes returns a count per year and month, and year_month_range is available for the past four years. Pull the monthly series, not the average.

Second, the competition figure is not SEO difficulty. Keyword Planner reports competition as LOW, MEDIUM or HIGH plus a competition_index from 0 to 100, and these describe advertiser competition in the ad auction. The bid fields alongside them, low_top_of_page_bid_micros and high_top_of_page_bid_micros, are the 20th and 80th percentiles of what advertisers pay. None of it says anything about how hard the organic results are. Teams read LOW competition as an easy ranking opportunity every day, and it means no such thing.

Trends is the most misquoted data source in this subject. Its documentation is explicit: each data point is “divided by the total searches of the geography and time range it represents”, then “scaled on a range of 0 to 100 based on a topic’s proportion to all searches on all topics”.

So the number is relative interest within the place and period you chose, not a count. Google states directly that “Different regions that show the same search interest for a term don’t always have the same total search volumes”. A score of 100 in one state and 100 in another tells you nothing about which has more demand.

Three further cautions from the same page, all of which matter in practice. Low-volume terms display as 0, so a zero may mean too few searches to show rather than none. Statistical noise is “most noticeable on queries with low or no search interest”, so a spike on an obscure term may not be real. And Google advises treating it as “one data point among others before drawing conclusions”.

What it is genuinely good for: comparing two terms against each other, establishing seasonality shape, catching a rising topic early, and comparing regions for relative interest. What it cannot do is tell you how many people searched.

Where the paid tools get their numbers

No third party has Google’s query logs. Every paid volume figure is built from some combination of clickstream data bought from panel providers and browser extensions, Keyword Planner data, the tool’s own crawling, and a model that extrapolates from all of it. That is why they disagree, and the disagreement is not random: it is largest where the panel data is thinnest.

Which, for our market, means Indian queries are among the least reliable in these tools. Panel coverage in India is smaller relative to search volume than in the United States or United Kingdom. Queries in Hindi, Marathi, Tamil, Telugu, Bengali and the rest are worse again. Transliterated queries, the ones typed in Latin script but in an Indian language, are the weakest of all, and they are a large share of real demand. The content side of that problem is in Hindi and Marathi content SEO.

The practical consequence: for Indian-language and transliterated research, treat tool volumes as a ranking of relative interest at best, and validate against Search Console, Google’s own autocomplete and related searches, and what your customers actually say. Do not put a tool’s Hindi volume figure in a client deck as a number.

Keyword difficulty is every vendor’s own invention

There is no standard. Each tool computes a difficulty or competition score from its own backlink index and its own weighting, scaled to its own 0 to 100. A 34 in one tool and a 34 in another are not the same claim, and neither is a measurement of anything Google publishes.

They are still useful, within their own universe, for sorting a list from easier to harder. They are not useful as an absolute threshold, and “we only target keywords under 30” is a rule built on a number the vendor made up. Use difficulty to order a list, then look at the actual results page, which tells you more in thirty seconds than any score.

Pricing, for the ones we could verify

As published on each vendor’s own page, read 2 November 2026.

Ahrefs: Starter $29 a month, Lite $129, Standard $249, Advanced $449, Enterprise $1,499 with an annual commitment required, plus a free tier. Its AI product, Brand Radar, is listed separately from $199 a month.

SE Ranking: Core at $103.20 a month shown struck through from $129.00, Growth at $223.20 from $279.00, with an annual discount toggle on the page, and a custom Enterprise tier. AI tracking is included at 25 prompts daily across 5 engines, with a standalone AI visibility add-on from $71 a month.

Semrush: not verified. Its pricing page returned a bot challenge rather than content when we tried to read it on 2 November 2026, and we will not repeat figures from a third party post that may be stale.

Google Keyword Planner and Search Console: free. Keyword Planner requires a Google Ads account.

The fuller comparison of these suites on the tracking side, including the AI prompt allowance arithmetic, is in rank trackers and AI answer trackers.

The fan-out question

This is the part of keyword research that has actually changed, and most tool lists have not caught up.

When someone asks an assistant a question, the assistant does not run that one query. It decomposes the question into sub-questions and retrieves against those, then assembles an answer. So the unit of research is shifting from the keyword to the set of sub-questions a topic generates, and a page that answers only the headline query can be absent from an answer assembled from six others. The mechanism is explained in our query fan-out glossary entry, and the research method in query fan-out keyword research.

What the tools offer today, honestly assessed. The suites are adding AI visibility modules that track whether you appear for a prompt, which is measurement rather than research. Some now expose related-question clusters, which are closer. But no tool we are aware of reliably publishes the actual decomposition a given assistant performs, because the assistants do not expose it.

So the practical method remains manual and cheap: take the question, ask three assistants, read what they actually answered and which sources they cited, and treat the sub-questions visible in those answers as your research output. Google’s People Also Ask and related searches, and the questions in your own support tickets, cover much of the same ground. Using support data properly is in how to use customer support data to plan content, and it is consistently the best source of real questions we find on any engagement.

The comparison

Source Data Cost Best for Trap
Search Console Measured, your site only Free Queries you already touch, snippet opportunities Blind to demand you miss; long tail anonymised
Keyword Planner Google’s own estimate Free with Ads account Volume with monthly series, ad economics Headline figure is a 12-month average; competition is advertiser competition
Google Trends Normalised relative interest Free Seasonality shape, comparing terms and regions Not a volume; 0 can mean too few to show
Ahrefs Clickstream plus model $29 to $1,499 a month Backlink-informed difficulty, large index Difficulty is proprietary; weak India-language data
SE Ranking Clickstream plus model $103.20 to custom Daily tracking, AI prompts included, price Same estimate caveats
Semrush Clickstream plus model Not verified Not assessed Could not read the pricing page
Autocomplete and related searches Google’s own suggestions Free Real phrasings, including transliterated No volume at all
Assistant answers read by hand Actual sub-questions and citations Free Fan-out research Manual, small sample, varies per run
Your support tickets and sales calls Real customer questions Free The questions that convert Nobody has collated them

A research order that does not produce a useless spreadsheet

  1. Export Search Console for twelve months and find the impressions you are wasting and the page-two accidents.
  2. Collect the real questions from support, sales and the communities your buyers use. These are phrasings, not volumes, and they are the most valuable input.
  3. Pull Keyword Planner for volume, taking the monthly series rather than the average, and ignore its competition column entirely for organic purposes.
  4. Use Trends to establish the seasonality shape and to compare candidate terms against each other.
  5. Use a paid tool to expand the list and to order it by difficulty within that tool’s own scale.
  6. For every shortlisted term, open the actual results page. Thirty seconds there beats any difficulty score.
  7. For the topics that matter, ask three assistants and record the sub-questions and cited sources. That is your fan-out research.
  8. Write down, next to every number in the final sheet, which source it came from and when, so nobody treats an estimate as a measurement six months later.

What is a waste of effort

Reading LOW competition in Keyword Planner as an easy organic opportunity.

Quoting a Trends score as a search volume.

Putting a tool’s Hindi or transliterated volume figure in a client deck as a number.

Comparing difficulty scores across two tools, or setting a company-wide threshold on one vendor’s invented scale.

Buying a suite before exporting Search Console, which is free and is about your own site.

A thousand-row keyword sheet nobody will ever write against. Most businesses act on twenty to fifty terms a year.

Treating the headline average as the demand for a festival or exam-season keyword.

If your keyword sheet has three different volumes for the same term and nobody can say which source each came from, get in touch.

Last verified: 2 November 2026. We review this page quarterly, and pricing in this category moves faster than that.

Sources, all read 2 November 2026. Google Ads API documentation, “Generate keyword ideas”, for the avg_monthly_searches, monthly_search_volumes, year_month_range, competition_index and bid percentile fields. Google Trends Help, “FAQ about Google Trends data”, for the normalisation and 0 to 100 scale. ahrefs.com/pricing and seranking.com pricing page. semrush.com/prices returned a bot challenge and is recorded as unverified.