The Taxonomy Problem: Why Your Category Tree Hides Content from AI Search

TL;DR.

• LLMs retrieve content through three distinct pathways — training ingestion, RAG retrieval, and live web grounding — and your taxonomy affects each differently.
• Deep hierarchies generate compound terms that don't match high-frequency vocabulary in LLM training corpora; flat, entity-aligned tags do.
• Faceted classification (orthogonal dimensions: topic × content-type × audience × maturity) lets a single post match multiple LLM query shapes simultaneously.
• In 2026, structured JSON-LD with schema:sameAs pointing to Wikidata or Wikipedia entities is the clearest trust signal to LLM grounding pipelines.
• If you only remember one thing: design your taxonomy against external vocabulary frequency, not internal information architecture logic.

Content taxonomies were designed for humans browsing catalogs. The goal was faceted navigation: filter by category, sub-category, content type, date. That hierarchy reflected organizational logic — Product > Category > Subcategory — and it worked well enough for sitemap generation and breadcrumb trails.

LLMs don't browse. They retrieve, synthesize, and cite. When Perplexity, ChatGPT with web browsing, or Gemini grounds a response in your content, the mechanism is fundamentally different from a user clicking through a navigation tree. The taxonomy decisions you make in your CMS — how deep, which vocabulary, how tags co-occur — determine whether your content surfaces or disappears in AI-generated answers.

By 2026, the majority of informational queries are being partially or fully answered by LLM-powered search interfaces rather than generating traditional SERP clicks. The traffic mechanics are shifting. Getting your content cited is increasingly about how well your taxonomy maps to the conceptual vocabulary LLMs use to retrieve and synthesize — not about keyword density or backlink profiles. If your taxonomy was designed for a navigation sidebar, it was designed for the wrong reader.

What "LLM-Discoverable" Means Mechanically

LLM discoverability is not a single channel. It operates through at least three distinct pathways, each with different sensitivity to taxonomy design.

Training-time ingestion. During pretraining, LLMs ingest web corpora. Content that appears frequently, is linked from high-authority pages, and uses vocabulary consistent with the model's existing representation of a topic gets more weight in training distributions. Your taxonomy signals topical clustering — if your 30 posts about knowledge graphs share a coherent, externally anchored vocabulary, they collectively reinforce each other's signal.

RAG-augmented retrieval. Enterprise deployments and many consumer LLM products retrieve documents at inference time from vector stores, search indexes, or CMS APIs. The retrieval step embeds your content alongside the query. If your taxonomy terms — in titles, descriptions, tags — map poorly to how a user phrases a query, cosine similarity drops and the retrieval miss happens before the LLM ever sees your content.

Live web grounding. Perplexity, Bing Copilot, SearchGPT, and Google's AI Mode crawl the web in response to queries. They parse structured data. They follow entity resolution signals. A page with well-structured JSON-LD that declares sameAs links to canonical entities gets parsed with much higher fidelity than a page where taxonomy only exists as HTML <a class="tag"> nodes in the DOM.

Most content teams optimize for traditional SEO and ignore RAG retrieval entirely. The result: content that ranks well in blue-link search but gets skipped when LLMs retrieve context for a related answer. These are different retrieval problems with different structural requirements.

Why Navigation-First Hierarchies Fail LLMs

The standard CMS taxonomy is a tree. It has depth because depth reflects editorial logic: a "Recommendations" category has children "Collaborative Filtering", "Content-Based", "Hybrid", and each child has grandchildren for specific use cases. The tree is internally coherent and useful for breadcrumb navigation.

The problem is that deep trees produce compound terms that are sparse in external corpora. "Personalization > Recommendations > Collaborative Filtering > Cold-Start > Implicit Feedback Handling" is a valid internal taxonomy node. But an LLM's representation of this topic was trained on thousands of papers and blog posts that use "cold-start collaborative filtering" — a flat, two-concept phrase — not your five-level path.

The practical consequence: when you serialize your taxonomy into tags or metadata, deep taxonomy paths become hapax legomena. Terms that appear once in your corpus. Terms the LLM has no strong associations for. They don't match query vocabulary. They don't match entity vocabulary. They're invisible to a retrieval system that pattern-matches against high-frequency training concepts.

Shallow taxonomies — ≤2 levels deep, with the second level specific enough to carry precise meaning — consistently outperform deep hierarchies in retrieval tasks. Not because shallow is inherently better, but because shallow maps more cleanly to the way concepts are referenced in the external corpora LLMs train on. "collaborative filtering" and "cold start" are separate, flat tags. Let the co-occurrence of both on a post express their relationship — you don't need a hierarchy to encode it.

The hierarchy instinct is also a wrong-level abstraction for LLM retrieval. LLMs don't traverse trees — they embed everything into a flat semantic space. A tree node 3 levels deep is not "closer to" a leaf node in embedding space just because they share a path. The semantic relationship is encoded in the term's meaning, not its position in your CMS tree. Hierarchies optimize for browsing. Flat, precise vocabularies optimize for retrieval.

Entity Alignment: The sameAs Trust Signal

The single highest-leverage taxonomy improvement is aligning your terms to canonical knowledge graph entities.

Every concept worth tagging already has a Wikipedia article and a Wikidata entry with a stable identifier (Wikidata identifiers). "Collaborative filtering", "knowledge graph", "ontology" — each has a canonical entity with a defined semantic neighborhood: related concepts, broader/narrower terms, equivalences across languages and knowledge bases. When your taxonomy term resolves to a canonical entity, you inherit that neighborhood.

In practice, entity alignment means two things.

First, audit your vocabulary against Wikipedia coverage. Before finalizing a taxonomy term, check whether a Wikipedia article exists for it. Not "does something vaguely similar exist" — does a specific article exist for this exact concept, with enough depth (3,000+ words is a rough threshold for "established consensus vocabulary"). If no article exists, your term is probably either too internal (rename to the canonical external phrasing) or too vague (split into constituent concepts that do have coverage). Expect to collapse your tag vocabulary by 30–50% in this audit.

Second, use sameAs in your structured data. In your JSON-LD, for each concept your content is about, declare a sameAs link to the Wikipedia article or Wikidata entry. This is how LLM grounding pipelines — particularly those built on knowledge graph backends — establish that your content about "knowledge graphs" is the same conceptual entity as the thousands of other documents about knowledge graphs they've indexed. Without sameAs, you're relying on string matching between your vocabulary and theirs.

{
  "@context": "https://schema.org",
  "@type": "Article",
  "about": [
    {
      "@type": "Thing",
      "name": "Knowledge graph",
      "sameAs": [
        "https://en.wikipedia.org/wiki/Knowledge_graph",
        "https://www.wikidata.org/wiki/Q33002955"
      ]
    },
    {
      "@type": "Thing",
      "name": "Collaborative filtering",
      "sameAs": "https://en.wikipedia.org/wiki/Collaborative_filtering"
    }
  ]
}

The ISO 25964 standard for thesauri and interoperability (ISO 25964, NISO) formalizes the broader principle: controlled vocabularies gain power when they declare equivalence relationships with other vocabularies. LLM grounding pipelines are operationalizing this at scale without calling it by that name.

Faceted Classification Over Strict Hierarchy

A strict hierarchy puts every post in exactly one place in the tree. A faceted taxonomy applies multiple orthogonal classification dimensions simultaneously. The canonical theoretical grounding is Ranganathan's Colon Classification and its modern descendants (Wikipedia: Faceted classification), but the practical implementation for content teams is straightforward: define 4–6 orthogonal facets and tag every post against each facet independently.

For a technical content site, a working facet set:

• Topic: the primary concept (knowledge-graph, collaborative-filtering, cold-start-problem)
• Content-type: the format and purpose (tutorial, architecture-guide, benchmark, case-study, explainer)
• Audience: who it's for (ml-engineer, product-manager, data-scientist, founder)
• Maturity: where in the build it applies (research, prototyping, production, scale)

A post tagged [knowledge-graph] × [tutorial] × [ml-engineer] × [prototyping] is now discoverable by at least four distinct query shapes: queries about knowledge graphs (topic), queries about tutorials for engineers (content-type + audience), queries about prototyping data infrastructure (topic + maturity), and queries about knowledge graph prototyping specifically (topic + maturity intersection).

Under a strict hierarchy, the same post lives at one node: /topic/knowledge-graph/tutorials/. It's discoverable by queries that hit that exact node. The other query shapes miss it.

The key insight is that LLMs generate answers by query intent, not tree position. A user asking "how do I start building a knowledge graph?" is expressing intent that maps to [content-type: tutorial] × [audience: engineer] × [maturity: prototyping]. If your post doesn't declare those facets explicitly — in metadata, tags, JSON-LD — the retrieval system has to infer them from body text alone. Inference adds noise. Explicit facets remove it.

Research on generative engine optimization confirms that structural content signals — explicit categorization, clear entity mentions, well-formed metadata — significantly increase citation rates in LLM-generated answers compared to high-keyword-density content that lacks structural clarity (Aggarwal et al., "GEO: Generative Engine Optimization", arXiv 2024). The signal isn't just body text. Structure is legible.

Co-occurrence Engineering for Topical Authority

Topical authority — the signal that a domain is a reliable source on a topic cluster — is not a property of individual pages. It emerges from the coherence of tag vocabulary across pages.

When 20 posts in your content catalog all share the tags knowledge-graph and entity-linking, you build a co-occurrence signal: this domain treats knowledge graphs and entity linking as related concepts, and produces consistent content at their intersection. LLM training pipelines and grounding systems that use quality signals — links, citations, thematic coherence — learn that your domain has depth in this cluster. They surface it more often for queries that sit in this cluster.

The failure mode is tag inflation: adding every conceivable tag to every post to "maximize coverage." If a post about knowledge graphs is also tagged machine-learning, database, software-architecture, system-design, api-design, and performance-optimization, the co-occurrence signal is diluted. Those tags might all be technically accurate, but they're noise — they're true of half the posts on the site. The post is about knowledge graphs specifically.

Discipline your tag breadth. 5–8 tags per post is a working constraint. 3–4 high-precision topic tags, 1–2 content-type/audience facets. Resist adding accurate-but-generic tags. software-engineering is almost always too broad to be useful. graph-traversal-algorithms is specific enough to carry topical signal.

Deliberately engineer co-occurrence for the clusters you want authority in. If you want your site to be the authority on "personalization + knowledge graphs", make sure a substantial fraction of your knowledge graph posts also tag personalization concepts, and vice versa. Don't leave this to chance — map your planned content against the co-occurrence matrix you want to build, then commission content to fill the gaps.

The effect compounds over time. A taxonomy designed for topical authority coherence produces a content catalog that LLMs treat as a reliable cluster source, which increases citation frequency across the cluster, not just for individual posts. This is the compounding return that makes taxonomy design worth the upfront effort.

JSON-LD: Making Taxonomy Legible to Grounding Pipelines

Most CMS taxonomy implementations are navigation-first at the code level too. Tags appear in HTML as:

<a href="/tags/knowledge-graph" class="tag">knowledge-graph</a>

This is invisible to structured data parsers. The tag string is in the DOM, but there's no machine-readable declaration of its meaning, its type, or its relationship to canonical entities. A grounding pipeline that parses structured data won't find it — not because it's hidden, but because it's semantically undeclared.

The fix is JSON-LD embedded in the page's <head>, declaring the full schema.org/Article markup including the keywords array and the about array with entity references (schema.org/Article):

{
  "@context": "https://schema.org",
  "@type": "Article",
  "headline": "Your Post Title",
  "author": {
    "@type": "Person",
    "name": "Alex Shrestha"
  },
  "keywords": ["knowledge graph", "entity linking", "personalization", "taxonomy design"],
  "about": [
    {
      "@type": "Thing",
      "name": "Knowledge graph",
      "sameAs": [
        "https://en.wikipedia.org/wiki/Knowledge_graph",
        "https://www.wikidata.org/wiki/Q33002955"
      ]
    }
  ],
  "audience": {
    "@type": "Audience",
    "audienceType": "Machine learning engineers"
  }
}

The keywords field is a flat string array — your taxonomy vocabulary serialized into a form structured data parsers understand. The about field with sameAs is entity alignment in machine-readable form. Together they give grounding pipelines everything they need to link your content to canonical entities without relying on string matching against your internal vocabulary.

The schema.org/Article spec supports additional properties worth using: articleSection for your hierarchy level if you keep one, citation for references you cite, and isPartOf for series or collections. Each property is an additional hook for structured retrieval.

One hard constraint: don't emit JSON-LD that contradicts visible content. A grounding pipeline that finds about: knowledge-graph in structured data but body text that never substantively addresses the topic will flag the page as low-quality signal. The structured data should amplify what's already in the content, not manufacture a false classification.

Where to Start

This is a 4-step process, roughly in order of impact-to-effort ratio.

1. Audit existing taxonomy against Wikipedia coverage

Export your current tag vocabulary. For each term, check whether a Wikipedia article exists with substantive depth (3,000+ words is a rough heuristic for "established concept"). Terms without Wikipedia coverage fall into two categories: too internal (rename to canonical external vocabulary) or too vague (split into constituent concepts that do have individual coverage). Budget for collapsing your tag vocabulary by 30–50%. The terms you cut weren't generating retrieval signal anyway.

2. Flatten hierarchy depth to ≤2 levels

If your CMS supports nested categories, flatten to parent + child. The parent should be broad enough to have a Wikipedia article. The child should be specific enough to be a distinct entity. Cut the grandchild level entirely — encode that specificity through co-occurring flat tags. This doesn't lose information; it relocates the relationship from the tree structure (invisible to retrievers) into co-occurrence patterns (visible and meaningful).

3. Implement faceted tagging across your catalog

Define your 4–6 facet dimensions. Apply them consistently. This usually requires a tag migration — existing tags are likely all "topic" tags without content-type or audience facets. Retrofitting facets to existing content is worth the effort; it opens retrieval pathways that don't exist in a topic-only taxonomy. Track which facet combinations you have and which you're missing — those gaps are your content roadmap.

4. Deploy JSON-LD with sameAs

Once your taxonomy vocabulary is stable, build or configure JSON-LD output for every page. Map each topic tag to its Wikidata entity and Wikipedia article URL. Validate with Google's Rich Results Test and monitor structured data errors in Search Console. For any grounding pipeline that parses JSON-LD, this step converts your internal classification into machine-readable entity alignment — it's the difference between your taxonomy being visible to automated systems or not.

The underlying principle across all four steps: your taxonomy is not an organizational tool for editors. It's the primary signal your content sends to automated retrieval systems about what it is, who it's for, and how it relates to canonical knowledge. Design it for the retrieval system, then verify it still makes sense for editors as a secondary constraint.

If you're building a personalization layer over a content catalog, taxonomy quality has a second downstream effect: entity resolution in your knowledge graph. A catalog with entity-aligned, faceted taxonomy produces cleaner training signal for the knowledge graph, which improves personalization quality for the same underlying content. The taxonomy investment compounds in both directions — outbound discoverability and inbound personalization accuracy.