Why One Page Ranks and the Other Doesn't: How Google Actually Judges Text Relevance in 2026

Two sites. Same topic. Both have a clean Title, a sensible H1, the keyword in all the "right" places. One sits in the top three. The other is stranded on page two.

Why?

The lazy answer is "the first one has more backlinks or better engagement." That's true as far as it goes — but it's not the whole story, and the whole story is the part that actually matters. It explains why you can't just sprinkle more keywords into a page and expect it to climb, and it explains why a page that reads fine to your editor can still be invisible to Google.

So let's walk through what happens between the moment someone types a query and the moment a page shows up in the results — and what you, as someone who has to make websites earn their keep, can do about each step.

A note before we start: nobody outside Google has the exact ranking formula. But between Google's own documentation, the sworn testimony from the 2023–2025 U.S. antitrust trial, and the May 2024 leak of internal Search API documentation, we now know far more about the real machinery than we did even two years ago. This article leans on all three. Where something is a model-for-understanding rather than the literal production system, I'll say so.

What "relevance" even means now

Google frames ranking as roughly five things happening at once: understanding what the query means, finding content that matches it, judging the quality of the source, checking that the page is usable, and factoring in context like the searcher's location and settings. That's the public version, and it's a useful mental model.

The important shift hiding inside it is this: search stopped asking "what is this text about?" years ago. It now asks "will this page actually help the person who typed the query?" Those sound similar. They are not. A page can be unmistakably "about" your topic and still fail to help anyone — and Google has built an increasingly good nose for the difference.

Everything below is just different approximations of that one question, layered on top of each other.

Layer 1: The words

Start with the foundation, because it hasn't gone anywhere — and the antitrust trial confirmed it's more central than most SEOs assumed.

From TF-IDF to BM25

Before BM25 there was TF-IDF, and understanding it makes clear why BM25 replaced it.

TF (term frequency): how often a word appears in a document.
IDF (inverse document frequency): how rare that word is across the whole collection.

The score is basically the product of the two. The word "buy" appears in half the documents on the web — its IDF is near zero, so it barely moves the needle. A rare, specific term has a high IDF and carries real weight. Sensible model. But it has a flaw: TF grows linearly. The more times a word appears, the higher the score climbs, with no ceiling. That's an open invitation to keyword stuffing.

BM25 (Best Match 25) fixes exactly that. Without the algebra, three things fall out of its formula, and all three matter to you:

Frequency helps, but with diminishing returns. The second mention of a term gives you a bump. The twentieth barely registers. The curve flattens — it saturates.
Rare terms weigh more. A precise, uncommon term is worth orders of magnitude more than a generic one.
Long documents get a "dilution" penalty. A long page needs proportionally more genuinely relevant mentions to look focused, because length is normalized away.

Put those together and you see why keyword stuffing is dead on arrival: BM25 reads it as high term frequency on a long-ish document, saturation eats the gains almost immediately, and you've spent your reader's patience for nothing.

This isn't ancient history — it's Google's first gate

Here's the part the antitrust trial made concrete. Under oath, a senior Google Search executive described the first stage of retrieval as an old-school inverted index and postings lists — the same information-retrieval methods that predate modern AI by decades. Court exhibits from the remedies phase explicitly reference "Okapi BM25" as the lexical algorithm Google's system evolved from.

Read that again, because it reframes everything: the first gate your content has to pass through is not a neural network. It's word matching. Google runs cheap lexical operations to cast a wide net across billions of pages, and only then spends expensive AI on the small catch that survives. If your page doesn't clear the lexical gate, none of the clever stuff downstream ever gets a chance to help you.

Position, depth, and breadth

Beyond raw frequency, placement matters. Title, H1, the opening of the body, subheadings — these carry more weight than the third paragraph from the bottom. The title in particular is one of the first things the system reads to form its picture of what the page is about.

Two informal but useful ideas: depth and breadth.

Depth is how thoroughly you develop the main topic — roughly, how much meaningful term frequency your core terms accumulate.
Breadth is how many related subtopics and entities you cover — how many distinct relevant terms make it into the document and contribute to the score at all.

These aren't official metrics. But they describe real behavior. If an article about "relevance" mentions the word "relevance" once in 10,000 characters, the term frequency is minimal and the system struggles to tell that's the main theme. And if you cover only two or three of the ten entities a full answer would touch, a competitor with broader coverage beats you, all else equal.

Think of the text as a map. Depth is how detailed one district is drawn. Breadth is how many districts made it onto the map at all. Hold onto that image — in the next layer, "breadth" takes on a second, sharper meaning.

Layer 2: The meaning

BM25 doesn't understand anything. It's a very well-calibrated counter, but it's still a counter.

A page about "renting a property long-term" contains none of the words in the query "how to lease an apartment" — BM25 shrugs. But to a human, that's the same document for the same need. Closing that gap requires understanding meaning, not matching strings.

Embeddings, in plain terms

The fix is embeddings. Text gets turned into a numeric vector in a high-dimensional space, arranged so that things close in meaning end up close together geometrically. Similarity is measured by the angle (or dot product) between vectors. Semantically similar content lands nearby even with zero shared keywords.

Two architectures do this, and they solve different problems:

Bi-encoder. Encodes the query and the document separately, each into its own vector, then compares them in one cheap operation. Because the document vectors can be computed ahead of time and stored, this is fast enough to sweep billions of pages. This is roughly how Google's RankEmbed works, which is why it lives at the candidate-selection stage.
Cross-encoder. Feeds the query and document in together as one sequence and judges relevance in a single pass, letting every query word interact with every document word. Far more accurate — and far more expensive. You can't precompute it; every pair runs fresh. So it only makes sense for re-ranking, when candidates are down to hundreds, not billions, and precision beats speed.

Bi-encoder to gather candidates; cross-encoder-style models to re-rank the finalists. Google's pipeline follows exactly that shape.

Google's neural stack, as we now know it

Google started talking publicly about embedding-based search back in 2018, under the name Neural Matching. Since then the trial and the API leak have filled in the org chart of models:

RankBrain (2016) — the first machine-learning ranking signal.
DeepRank — a BERT-based language-understanding model, applied late in the pipeline (roughly the final 20–30 results) because it's too expensive to run at scale.
RankEmbed / RankEmbed BERT — dense-vector models that pull in relevant candidates that pure keyword matching missed, trained on a mix of search logs and human quality-rater judgments.

Crucially, Google found that these neural signals could be decomposed into signals resembling the traditional ones — and that combining both types beat either alone. That's not a philosophical stance; it's a benchmark result.

The limit nobody talks about — and why hybrid retrieval wins

Here's a piece that rarely makes it into SEO articles. Embeddings are not magic; they have a hard ceiling. A 2025 Google DeepMind paper (the "LIMIT" work) showed that a single-vector embedding of typical size can only cleanly distinguish relevance across roughly 1.7 million documents before the distinctions start to blur — a tiny fraction of Google's index. A fixed-length vector can only hold so much information.

That's why Google can't just throw everything at embeddings and call it a day, and it's why hybrid retrieval — BM25 plus dense vectors — consistently outperforms either method alone on standard benchmarks. BM25 gives you precision on the exact words. Vectors give you reach across meaning. Each covers the other's blind spot. The counter and the neural net are partners, not rivals.

"Breadth," version two

Now that second meaning of breadth. In the lexical world, breadth was about term coverage. In the vector world, breadth is how many relevant subtopics and entities of the page make it into the embedding and shrink its distance to real queries. A page that touches only one facet of a topic produces a vector that sits far from queries about the other facets. Google literally calls this topical coverage.

So the same instinct — cover the topic like someone who actually knows it — pays off twice, once for the counter and once for the neural model.

Layer 3: The user

A perfectly optimized page can still rot on page two if people don't stay on it. This is the layer where the antitrust trial dropped the biggest bombshell, because for years Google publicly downplayed exactly this.

Navboost: the giant table of clicks

Meet Navboost. For a long time the SEO community argued about whether clicks affect rankings, with Google's public statements ranging from evasive to flatly dismissive. The trial ended the argument.

Navboost turned out to be, in one Google engineer's own description, essentially a big table: for a given query, this document got these clicks, that document got those clicks, aggregated with a bit of extra data. It's not even a machine-learning system — it's a spreadsheet of user behavior. And it's been running since roughly 2005, holding a rolling 13-month window of interaction data. A senior Google VP called it under oath one of the strongest ranking signals they have.

The mechanic that matters to you is the click quality, not the click count:

Long clicks — the user goes to your page and doesn't come bouncing back to the results for a while — read as satisfaction.
Short clicks / pogo-sticking — the user clicks, immediately returns to the SERP, and clicks something else — read as failure.

Related to Navboost is Glue, which does the same kind of thing for the non-blue-link elements of the page (images, "People also ask," and so on), positioning those universal-search features based on interaction data.

Why this doesn't mean "go buy clicks"

Google is the first to say clicks are a lousy standalone metric — they're trivially manipulable. So this was never a naive "higher CTR = higher rank" lever. Behavior is one input into a complex learning system, cross-checked against everything else. Trying to spoof it tends to get detected, and a burst of fake short-visit traffic is as likely to hurt as help.

The honest takeaway: the two things you can control are (1) whether your snippet earns the click, and (2) whether the page immediately delivers on the promise that snippet made. Nail both and Navboost works in your favor over time. Fail the second one — even after a great snippet — and you're training the system to bury you.

A concrete example of "text noise"

There's a well-documented pattern where a bloated site-wide menu — dozens of links repeated on every page — dumps so many extra words into each page's text model that it muddies what any individual page is actually about. Teams that strip that noise (or stop rendering a heavy mega-menu for crawlers) routinely see visibility jump, sometimes dramatically. The lesson generalizes far beyond menus: noise in the page dilutes the topic signal. If a crawler read your page as a single document, what would it decide the page is about? If the answer is "hard to say," you have a noise problem.

Layer 4: The quality and the trust

Relevance gets you considered. Trust decides whether you win — especially on topics where being wrong hurts people.

Google's public framing here is E-E-A-T: Experience, Expertise, Authoritativeness, Trustworthiness. But note the crucial caveat baked into Google's own documentation, and reinforced by the trial and leak: there is no single "E-E-A-T score." The term never appears in a patent, the leaked API docs, or the court evidence. It's a guiding principle expressed through dozens of indirect signals — quality modules, site-level authority measures, PageRank-derived quality scores — that the systems learned to read automatically.

That distinction is practical, not pedantic. Google doesn't "sense the author's voice." It detects the presence or absence of concrete signals that tend to correlate with expertise: a named author with a real bio, cited primary sources, a stated methodology, contact and business details, an honest acknowledgment of limitations.

For YMYL topics — Your Money or Your Life: health, finance, law — the bar is higher and the penalty for missing signals is steeper. A law firm's page with no author, no credentials, and no real markers of expertise is simply weaker than one that has all three. Not because a machine "respects lawyers," but because the trust signals are either there in the text and structure, or they aren't.

This is also the clean rebuttal to the fashionable claim that "in semantic search, keywords don't matter anymore." They do. The winning formula for competitive commercial and YMYL work is a correct relevance map (words + meaning + coverage) plus genuine, demonstrable expertise. Neither half is optional.

The new layer on top of everything: AI answers

Since 2025, a whole new question sits above the classic stack: will your page get pulled into Google's AI Overview (or AI Mode), and will it get cited?

This is not a minor tweak to the SERP. It's a structural change to what "ranking well" even means, and the data is stark.

The click math has changed

Multiple independent studies, using different methods, all point the same direction:

Ahrefs (300,000 keywords, December 2025 data): when an AI Overview is present, the top-ranking page's average click-through rate is about 58% lower than it would otherwise be — up sharply from the ~34.5% they measured a year earlier. Their blunt framing: for every 100 clicks a top page used to earn, Google now keeps 58.
Pew (real browsing data): only ~8% of users click a traditional result when an AI Overview appears, versus ~15% without one.
Seer Interactive (long time series): organic CTR on AI-Overview queries fell from ~1.76% to as low as ~0.61%, with a tentative stabilization in early 2026 rather than a recovery.

Different datasets, different numbers, one unmistakable trend. When Google answers the query itself at the top of the page, fewer people scroll down to your link. Zero-click search — where the user's need is met without leaving Google — is now the default for a large share of informational queries.

**But being cited flips the sign**

Here's the hopeful part, and it should reshape your strategy. Pages cited inside an AI Overview earn higher organic CTR than uncited pages sitting in the same results — one analysis found roughly 35% more organic clicks for cited sources. Being in the answer beats being merely next to it.

So the metric quietly moved. It used to be "how many clicks." It's becoming "how often are you the source the machine quotes." That discipline now has a name in the industry — GEO (Generative Engine Optimization) or answer-engine optimization — but the good news is that there's no separate secret recipe. To be eligible, a page needs to be indexable, high quality, and genuinely matched to intent. If you've been doing real SEO, you're already most of the way there. The shift is in what you measure, not in the fundamentals.

Two practical wrinkles for your reporting

Google Search Console now reports AI surfaces. As of 2026 there's dedicated visibility for AI Overviews and AI Mode. Watch it, because it changes how you read your own numbers.
Impressions are being double-counted. When you rank on page one and appear in the AI Overview for the same query, GSC can log two impressions. Clicks don't double with them. So a page can show rising impressions and improving average position while clicks and CTR fall — a pattern that's now widespread. Don't panic-diagnose a content problem when what you're seeing is the AI-Overview accounting effect.
The CTR curve is flattening. Positions 6–10 have actually gained click share in some studies, as users who scroll past the AI Overview engage with lower results. The old "only positions 1–3 matter" assumption is weaker than it was.

Commercial vs. informational pages: different kinds of relevance

Everything above applies to both a blog post and a product page. But for commercial pages, text relevance is only part of the score.

For a query like "buy a road bike with delivery," the job isn't to read an article about road bikes — it's to buy a specific bike. Different intent, different expected answer. So:

An informational page must explain, compare, teach. Its relevance is judged on how completely it covers the topic and how much expertise it shows.
A commercial page must enable a purchase. Its relevance is text plus the signals that a real transaction can happen here — price, stock, delivery options, trust markers, reviews, the buy button where the eye expects it.

This explains a common frustration: great prose on a commercial page that stubbornly won't rank. The prose is solving an informational page's job. The commercial page needed to prove "you can buy, here, now" — not narrate.

And a hard rule: don't mix commercial and informational intent on one page. Trying to sell a product and explain what the product is, in one place, smears the topical signal. Google struggles to tell which query the page targets; the visitor arrives with one goal and hits the other; short return, weak engagement signal. For informational queries, write the long, thorough piece. For commercial queries, build dedicated landing pages with the full set of commercial signals — the copy can still be substantial, just distributed through real commercial blocks rather than dumped as a wall of text.

What actually follows from all this

Controlled experiments back up the theory in useful ways. SearchPilot, which runs split tests on live sites, has found patterns like these on real e-commerce pages:

Adding genuinely useful information to product pages: a meaningful traffic lift.
Pulling content out of hidden tabs into the visible area: another lift.
Adding a few more keywords to the title: no significant effect.
Removing boilerplate SEO text from category pages: a statistically significant increase — the filler was diluting the topic and hurting usability.

The through-line: Google finds it easier to judge a page relevant when the topic signals are visible, structured, and not drowned in noise. Understanding intent beats matching keywords. Breadth of topic coverage beats density of any single keyword.

There's an adjacent truth worth stating plainly, straight from the trial-era analysis: content scoring tools get you through the first gate, but they don't win the race. Push a page's on-page score from 60 to 95 and you may see nothing move — not because the tool failed, but because backlinks, domain authority, and click signals are doing more work for your competitors than content alone can overcome. If a DR 15 site is up against DR 70+ sites, perfect on-page work is necessary but not sufficient. Be honest with yourself (and your clients) about which part of the problem content can actually solve.

One more honest caveat: the exact candidate-retrieval formulas at Google are not published. BM25 is a correct teaching model, not the literal production system. The precise weights and the architecture of the neural layers stay hidden. Treat everything here as a well-supported map, not the territory.

A checklist: pressure-test a page before you publish

Formulas and embeddings are interesting. Here's what to actually do with them. These are averages — some apply to your project, some don't, so filter through your own site's job.

1. Write down the real intent. Does the person want to learn, buy, compare, or solve a specific problem? "How to move from QA into development" isn't a request to learn about the QA profession — it's a request for a transition plan: what to learn, how long it takes, where to start. If your page answers a different question, it's irrelevant no matter how many keywords you add.

2. Check that Title and H1 hit the intent, not just contain the keyword. "The QA Profession: Duties and Salaries" and "How to Move From QA Into Development: A Step-by-Step Plan" are both "about QA" — and answer completely different questions. Test: if you deleted the keyword from the headline, would it still be clear what the reader gets?

3. Compare the top 10 by intent — in a real browser, not a tool. Open Google's results for your target query. What subtopics do all the strong pages share? Which blocks show up in most of them? What does the top 3 have that the top 10–20 don't? You're not copying structure for its own sake (though frankly that's a workable tactic) — you're reverse-engineering what the algorithm treats as a complete answer for this query. If everyone in the top has a concrete action plan and you have only vague musings, that's a relevance signal working against you.

4. Strip the boilerplate noise. A giant site-wide menu with dozens of links (context-dependent — sometimes you need it for coverage), a 500-word SEO block in the footer, "we've been the best since 2005" filler — all of it is text noise diluting the page's topical signal. Ask yourself, or better yet run a tool: if Google read this page as one document, what's it about? Your mega-menu might be stuffing so many query variations into every page that it blurs them all.

5. Add entities and specifics. Dates, numbers, definitions, examples, tables. Not "many companies use…" but "per Google's own December 2025 disclosures…". Not "this matters for SEO" but "in a 300,000-keyword study, Ahrefs found…". Specifics serve both audiences at once: the algorithm sees more topical entities, the human gets real value.

6. Verify the expertise signals. Can a reader tell who wrote this and why they should trust it? For Google that's E-E-A-T, with the emphasis on trust. Is there a named author with a bio, links to primary sources, a stated method, an honest note about limits ("this data is for the US market; other regions may differ")?

7. Confirm the important content is actually visible. Are key blocks hidden in tabs, accordions, or JS components that don't render at crawl time? (Tabs are fine if the content renders immediately.) The simple test: load the page with JavaScript disabled and see what the crawler sees.

8. After publishing, watch the right metrics. Position isn't the only signal, or even the main one — especially now, with AI Overviews reshaping the SERP. From Search Console: which queries you show for (coverage), snippet CTR, the 2–4-week trend, and now your AI-surface visibility. From analytics: bounce, scroll depth, time on page. And remember the impression double-counting before you read a falling CTR as a content failure.

What this looks like in practice

One worked example. Query: "how to find a job as an IT specialist." Intent is informational — the person wants to know where to start and what actually works. Here's the opening paragraph in three versions, worst to best.

Version 1 — the obviously bad, old-school stuff:

Finding a job as an IT specialist is an important and relevant question in today's world of information technology. The job search for an IT specialist plays a key role in building a career. If you're wondering how to find a job as an IT specialist, in this article we'll tell you how to find a job as an IT specialist quickly and effectively.

Formally "correct": the phrase "find a job as an IT specialist" appears three times in three sentences. And that's exactly why it fails. BM25 sees extreme term frequency on a short document — the classic over-optimization flag. The text answers no real question. Its embedding is impoverished, spinning around one phrase. The reader is back on the SERP in three seconds; the engagement signal logs a failure. Nobody writes quite this badly anymore, but it's the clearest illustration.

Version 2 — the dangerous one, because it looks fine:

The IT job market today is going through a period of active development. The industry offers many opportunities for specialists at every level. Competition for good positions remains high, so candidates need to approach the process thoughtfully. A well-planned search strategy and solid preparation can significantly improve your chances of success in this dynamic and promising field.

This is the common, dangerous case. No keyword stuffing, grammatical, reads smoothly. A tasteless editor waves it through. And it's just as irrelevant as version 1. Look at what it actually says: "the market is developing," "lots of opportunities," "competition is high," "approach it thoughtfully." Four sentences that fit any profession — swap "IT" for "law" or "logistics" and nothing breaks. For the lexical model there are almost no meaningful terms, just high-frequency, low-IDF filler. For the vector model it's worse: the embedding is smeared, close to no specific query, because it's not about a specific task. The searcher gets zero answers and leaves. The difference from version 1 is only that this one dies at the user layer instead of the lexical one. Technical competence and relevance are not the same thing — that's the whole trap.

Version 3 — the relevant one:

Finding work in IT usually comes down to precision, not volume of applications. A junior with no commercial experience is better off building a couple of portfolio projects on GitHub and applying to internships than chasing "3+ years required" roles. A mid-level engineer should do the opposite — fewer applications, but each one with a résumé rewritten for the specific job and a real cover letter, which beats a hundred identical submissions. Search in several places at once — job boards, specialist Telegram channels, and through your network: recruiters report that referrals fill a large share of IT roles before they're ever posted.

The key phrase barely appears in its original form. Instead the paragraph densely covers related entities: junior, mid-level, résumé, cover letter, portfolio projects, GitHub, internships, job boards, referrals. That's breadth in action — the vector model sees a document close not to one query but to a whole cluster: "first IT job for juniors," "how to write an IT résumé," "where developers find openings." The human gets specifics immediately and stays to read. Same topic, same intent — fundamentally different relevance, because the page solves the task instead of repeating the query.

The whole pipeline, in one picture

If you want the mental model that ties it together, here's the path your content actually travels, from the antitrust-era descriptions:

Inverted index + BM25 — the cheap lexical first gate, sweeping billions of pages. Clear this or nothing else matters.
RankEmbed — dense vectors add relevant candidates that keyword matching missed.
Mustang — roughly 100+ signals applied to the surviving set: topicality, quality scores, and Navboost click data.
DeepRank — expensive BERT-based understanding, applied only to the final ~20–30 results.
Re-ranking + AI Overview generation — trust signals, personalization, and the question of whether you get summarized and cited.

The practical implication is bracing: each stage is a filter, and you can be eliminated at any of them. No amount of authority saves a page that never clears the lexical gate. No amount of keywords saves a page that clears the gate but bores every reader who lands on it. You have to survive all of it.

The unsurprising conclusion

Write good text that helps people and answers the question. That's it.

BM25, neural models, click signals, quality assessment, AI summaries — they're all different approximations of one question the algorithm couldn't ask for most of its history, and that a human could always ask instantly: "okay, but did that actually help you or not?"

Optimizing for the algorithm and writing for the human used to be two different strategies you had to choose between. Ten years ago you did have to choose, and — let's be honest — the person who optimized for the algorithm usually won. That era is over. The two jobs have converged close enough that they're nearly the same job. The algorithm has gotten good enough (which, to be clear, doesn't stop people from trying to fool it) to tell the difference between text that genuinely answers a question and text that merely contains the right words.

Build for the second kind. Then measure whether you got cited, not just whether you got clicked — because in 2026, that's increasingly where the game is decided.