SWAG: Semantic Weighted Augmented Generation

Most businesses deploying retrieval-augmented generation (RAG) hit the same wall: vector search returns similar text, not necessarily the right text. A query about “three-way matching in accounts payable” may rank a general AP overview above the paragraph that actually explains the matching process — because both chunks share vocabulary and sit close together in embedding space.

SWAG — Semantic Weighted Augmented Generation — is a retrieval enhancement layer that addresses that gap. It does not replace your documents, rewrite chunks with synonyms, or trigger corrective web searches. It generates structured semantic metadata for each chunk, scores query-to-chunk alignment using weighted term relationships, and reranks vector search results before generation.

What SWAG is (and is not)

SWAG is a reranker: vector search still retrieves candidates; SWAG scores query-to-chunk semantic alignment and reorders them. It does not replace embeddings, vector stores, CRAG grading, or web fallback. In a combined pipeline, SWAG runs before CRAG — better first-pass ranking can mean fewer rewrite and fallback cycles, but CRAG still owns correction when confidence is low.

SWAG core vs reference-build retrieval stack

Two related layers. Keeping them separate avoids overstating what the reranker alone does.

Benchmarks below label which layer each number measures. Rank-1 body vs title results on lesson HTML often involve both reranking and the retrieval stack’s body filter — we report them separately where possible.

What’s in the rerank layer

Capabilities we’ve refined and tested in our reference build:

• Adaptive blend weights — when vector scores cluster or the query is evaluative (“how”, “compare”, “evaluate”), SWAG weight increases; title/nav chunks are penalized at score time
• Noise gate — drops low-confidence SWAG matches that would pollute ranking
• Chunk-kind multipliers — title, nav, and boilerplate down-ranked during rerank (not only after)
• Grounding checks — profile terms not supported by chunk text are discounted; short-token substring traps (e.g. ai → mail) are blocked
• Contrastive negatives — negated phrases in chunk text do not penalize the standalone term (e.g. “without retrieval” vs query “retrieval”)
• Near-duplicate collapse and per-source diversity caps
• Text-level negative penalties — query negative contexts (e.g. “incident reporting” on a “basics” query) penalize chunk body text, not just profile terms
• Always-on filter path — SWAG rerank + noise gate runs on multi-candidate pools (no “clear body winner” skip that bypassed filtering)
• README chrome classification — badges, table-of-contents, and link-heavy nav chunks tagged nav at rerank time
• Heuristic or LLM profiles — structured metadata generated at ingest; query profiles per search

Strict filtering defaults (v0.3.2)

SWAG’s job is to rank and filter, not pad results to top-K. Defaults on the reference build:

• One chunk per document in top-K (SWAG_MAX_CHUNKS_PER_SOURCE=1)
• No noise backfill — if only 2 chunks pass the gate, return 2 (not 8)
• Tighter vector noise ratio (0.62) — weak vector matches drop unless SWAG score justifies them
• Chrome hard drop — nav/title chunks need higher SWAG floor; boilerplate never passes the gate
• CRAG grade cap — at most 5 chunks sent to the Haiku grader per request (reference build)

Installable product: pip install -e . and the swag CLI. Showcase regression: validation tuning 10/10, holdout 6/6. Product harness: 5 ugly corpora, strong baselines (BM25, vector, hybrid RRF, hybrid rerank), brutal thresholds — run ./scripts/run-eval-harness.sh (add --generate for answer behavior).

What side-by-side compare shows (honest)

On the live reference showcase, the Search tab defaults to Vector → SWAG with baseline compare on. That is the product demo — same query, raw vector hits vs reranked hits. Use the scripted demo paths (frozen from the validation suite) for chrome traps and topic lanes; do not rely on random mixed-corpus queries to prove SWAG.

• Vector only — classic RAG (embed + cosine). README badges, table-of-contents lines, and wrong-domain neighbors often rank high. That junk is real; it is why naive RAG frustrates teams.
• Vector → SWAG — primary proof. Rerank + noise filter promotes body chunks, drops duplicates, caps one chunk per document. Lift cards (Top-1 quality, noise filtered) are the SWAG story.
• Vector → CRAG (optional) — Haiku grades whatever vector search already returned. Useful strict gating; weaker alone on messy HTML corpora.
• Vector → SWAG → CRAG (optional) — agent stack: SWAG cleans candidates first, then CRAG grades ≤5 survivors. Empty after CRAG does not mean SWAG failed — check baseline compare for what SWAG surfaced.

We do not claim SWAG fixes every corpus. It is strongest on structured markdown (nav vs body) and mixed topic indexes where similarity alone picks the wrong paragraph. The validation tuning suite (10/10) and holdout (6/6) plus the showcase compare panel are the proof — not ad-hoc free-form queries.

All retrieval knobs (phrase backfill for exact multi-word matches, cross-domain CRAG gates, grade caps, sibling expand) live in ingestion config — tune per corpus without forking SWAG core.

When the optional full stack returns zero hits with CRAG web_fallback, that can be valid: the grader rejected low confidence. Prefer that to feeding garbage — after running the SWAG compare first.

Reference-build additions (alongside SWAG, not instead of RAG/CRAG)

• Wider candidate pool — retrieve top-N (N > K) before rerank so better body chunks can be promoted into top-K
• Vector confidence floor — off-corpus queries with weak top vector scores return no hits (unless CRAG fallback is enabled)
• Same-document sibling expansion — optional; default off. When enabled, pulls related body chunks from the same source. Most UIs only need the best chunk plus a doc link.
• Body-only result pruning — final hits prefer body chunks over title/nav chrome
• Lift metrics — per-query compare panel with mode-scoped cards: SWAG-only shows rerank vs full stack; SWAG→CRAG adds a third card; CRAG-only modes omit SWAG lift
• Symmetric baseline — when benchmarking, the same post-filters apply to baseline and SWAG path so post-filters are not credited to the reranker alone
• Topic filter — optional metadata filter before retrieve (e.g. isolate ap-procurement in a mixed corpus)
• CRAG-aware finalize — after CRAG keeps hits, no cross-document pool backfill. When CRAG clears all hits, results stay empty (no refill from the vector pool)

The problem with vector-only RAG

Traditional RAG: chunk documents, embed raw text, store vectors, retrieve top-K by cosine similarity, pass chunks to an LLM. That works until:

• Multiple chunks share overlapping vocabulary but answer different questions
• Domain terms collide across unrelated contexts (“Apple silicon” vs. orchard management)
• Users ask intent-driven questions that do not lexically match the source paragraph
• Teams inject synonym expansion into chunk text — improving recall but polluting the knowledge base

CRAG (Corrective RAG) addresses retrieval failures after the fact by evaluating confidence and triggering fallback retrieval or web search. SWAG is complementary: it improves initial candidate ranking so fewer queries reach the correction step. In combined pipelines, SWAG sits before the CRAG confidence evaluator.

Semantic profiles

During ingestion, SWAG generates a structured profile stored separately from chunk text:

{
  "detected_domain": "finance",
  "primary_concepts": ["three-way matching", "invoice reconciliation"],
  "domain_terms": ["accounts payable", "purchase order", "goods receipt"],
  "direct_synonyms": ["PO-GR-IR matching"],
  "adjacent_concepts": ["exception workflows", "straight-through processing"],
  "business_goals": ["reduce manual AP review"],
  "user_intents": ["how does three-way matching work"],
  "negative_contexts": ["inventory forecasting", "general ledger reporting"],
  "confidence": 0.91,
  "weighted_terms": [
    {
      "term": "vendor invoice matching",
      "category": "adjacent_concept",
      "weight": 0.86,
      "reason": "Closely related to accounts payable reconciliation"
    }
  ]
}

Every term carries a weight from 0.0 to 1.0. At query time, SWAG generates a matching profile and scores alignment — with penalties for negative-context matches, domain mismatches, and ungrounded terms.

SWAG core rerank pipeline

In the reference build, optional steps after rerank: confidence floor → (optional) sibling expand → body prune → CRAG grade (capped). Sibling expansion is off by default — return the best chunk, link to the source document.

How SWAG fits into RAG and CRAG

Our reference build runs SWAG reranking before CRAG filtering — vectors and semantic profile metadata stored together in each chunk payload.

Scoring approach

Each candidate gets a blended score: vector similarity, semantic alignment, and a small metadata boost — then a quality multiplier for title, nav, or boilerplate chunks. Vector similarity carries the most weight by default; semantic alignment matters more when vector scores cluster or the query is evaluative.

Guardrails

• No isolated keyword expansion — ambiguous words require full chunk context
• Domain detection first — every profile includes detected_domain
• Negative contexts — block false matches across domains (with contrastive phrase handling)
• Grounding checks — profile terms not supported by chunk text are discounted
• Synonyms are retrieval hints only — never appended to chunk text
• Noise gate — weak SWAG signals cannot override strong vector matches without justification
• Chunk-kind awareness — title/nav/boilerplate down-ranked at rerank time; reference build can also filter them from final hits

What we measured (and what we did not)

These are primarily retrieval-ranking tests on small, labeled query sets. Preliminary end-to-end answer auditing on early corpora has been positive; evaluation on larger, more diverse corpora is underway and findings will be published. We report rank changes and body-at-rank-1 where we have checks; we do not claim universal precision gains on every corpus.

1. Scoring integrity (SWAG core only, offline)

Four synthetic adversarial cases (substring traps, query negatives, ungrounded profile terms, one clean match). Compares legacy vs hardened scoring. Rank order barely changed — vector weight still dominates when traps have slightly higher vector scores — but false-positive SWAG scores on trap chunks dropped sharply.

2. Demo AP corpus (SWAG core rerank, offline)

Eight chunks (accounts payable guide + SWAG overview), eight queries, heuristic profiles. Ground truth by content substring — a small smoke test, not a customer corpus. One query showed a clear rank-1 improvement; others already ranked acceptably or share overlapping chunk text.

3. RAG lesson corpus (reference build, live eval)

112 public RAG lesson chunks, 10 fixed queries, custom eval sheet (body-at-rank-1, phrase checks, beats-baseline-title on six cases). Full retrieval path: SWAG rerank plus reference-build body pruning and expansion — compared against unpruned vector baseline. Last run: 10/10 checks passed (80% pass bar).

Example (AP, SWAG rerank only): “How does three-way matching work in accounts payable?” — baseline ranked the overview chunk first (vector 0.732). SWAG promoted the next section (vector 0.715) by semantic alignment. Both chunks sit in a small, overlapping guide — good chunking at ingest would also help here.

Example (RAG lesson, full path): “what is rag” — baseline put # RAG From Scratch at rank 1. After rerank plus body filtering, the explanatory body chunk (“LLMs are trained on a large but fixed corpus…”) surfaced first.

4. Cross-domain corpus (reference build, live on langrag.automatico.llc)

After batch-seeding ~23 public URLs across 6 topics into one collection (~310 points) — RAG foundations, software READMEs, accounts-payable guides, cafeteria/food ops, platform/Kubernetes docs, security/compliance — we stress-tested mixed-folder retrieval: vector search sees everything; topic metadata and CRAG grading must disambiguate.

SWAG Evaluation Harness — four-pillar spec

We can now define, in one runnable command, what “good” means for SWAG — not just demo vibes:

Baselines are strong: BM25, vector-only, hybrid RRF, hybrid+BM25 rerank — not toy comparisons. Brutal pass gates: +10% recall vs hybrid on 4/5 domains, +15% on hard categories, SWAG wrong-version ≤ hybrid, latency ≤ 3× vector. Early pilot runs report honestly when SWAG is not yet product-worthy — that is the point of the harness.

What's still in validation

• Closing the gap on the product harness (pilot SWAG recall trails hybrid on lexical-heavy qrels — engineering grind, not theory)
• Scaling qrels to 100/corpus and optional LLM answer mode (--generate-mode llm)
• Head-to-head vs dedicated cross-encoder rerankers
• Customer-private corpora at scale

How it fits together

Embeddings always run on original chunk text. Semantic profiles live in chunk payload metadata, separate from text used for generation.

• Ingestion: common document formats — chunk size, overlap, and body vs title/nav classification at index time
• Typical query path: widen pool → vector search → SWAG rerank + noise filter → (optional) confidence floor → (optional) sibling expand → body prune → CRAG grade (capped) → generate

When SWAG earns its place

• Dense, overlapping chunks in the same domain (SOPs, policies, AP workflows)
• HTML/README corpora where title and nav chunks pollute vector top-K
• Intent-driven questions that do not mirror source phrasing
• CRAG pipelines where you want fewer corrective cycles — SWAG improves the first retrieval pass; CRAG still handles low confidence
• You want disambiguation guardrails without editing source documents

Strengthening results without replacing RAG or CRAG

SWAG is one layer. These changes compound with it — none of them require swapping out your vector store, generator, or CRAG grader.

Practical order: chunk and label → widen pool + SWAG rerank + strict filter → demo with Vector → SWAG compare → optional CRAG as safety net (grade few) → measure on your frozen query set. Enable sibling expansion only if your UI truly needs multi-chunk context per document.

At automatico.llc, we start with the operational bottleneck, measure what changes, and expand from there. SWAG is one layer we’re evaluating in that stack — structure and honest benchmarks first, deeper models where they earn their place.

By Michael Thompson · June 19, 2026