Module 4: Vector Store Design

🧠 Free Recall ❓ Quiz 🔍 Cues 🔮 Simulation 🥷 ShadowBox 📐 Encode Cards

Module 4 — Vector Store Design
Tier: Deep. Covers Skills 1.4.1 (advanced vector database architectures), 1.4.2 (metadata frameworks), 1.4.3 (high-performance architectures at scale), 1.4.4 (integration components), and 1.4.5 (data maintenance systems).

4.0 Why this module exists
Module 3 ended at "chunk and embed." This module is about where those embeddings live, and it is the point at which retrieval architecture stops being conceptual and starts being an infrastructure decision with cost, latency, and compliance consequences that are expensive to reverse.
The reason it warrants Deep treatment is that vector store decisions are unusually sticky. A bad chunking strategy can be fixed with a reindex. A bad prompt can be fixed in an afternoon. But an index topology that cannot express your tenant isolation requirement, a store that does not support the metadata filtering you turn out to need, or a dimensionality choice that makes your memory footprint untenable at scale — these require rebuilding and remigrating, often while the system is in production.
There is also a specific gap this module fills. Most practitioners can describe vector search at the level of "it finds semantically similar things." Very few can explain why increasing ef_search improves recall at the cost of latency, or why a metadata filter that matches 0.1% of documents can make an ANN index perform worse than no index at all. Those mechanics are exactly what professional-level scenario questions probe, because they are what distinguishes someone who has operated a vector store from someone who has read about one.

4.1 What a vector store actually is
4.1.1 The embedding space
An embedding model maps text to a point in a high-dimensional space, arranged so that semantically related texts land near one another. A vector store holds those points and answers one question efficiently: given a query point, which stored points are nearest?
That is the whole abstraction. Everything else — indexes, filters, shards — is machinery for answering that question fast enough, at scale, with the right subset of the data.
Two properties of the space matter architecturally:
The space is defined by the model, not by the store. Vectors from different embedding models occupy different, incompatible spaces. This is the single most consequential fact in the module: queries and documents must be embedded by the same model, and changing the model requires a full reindex. There is no partial migration, no backfill, no dual-read that works without careful design. Section 4.8.4 covers how to execute that migration.
Proximity encodes similarity, not truth. Two chunks can be close in the space because they discuss the same topic while stating opposite things. A chunk saying "the policy covers flood damage" and one saying "the policy excludes flood damage" are near neighbours. Retrieval will surface both, and it is the model's job — not the store's — to distinguish them. This is why retrieval quality and answer quality are separate problems requiring separate measurement, a theme developed fully in Module 5.

4.1.2 Distance metrics
Three are in common use, and choosing wrongly produces silently degraded retrieval.
Metric / Computes / Sensitive to magnitude? / Typical use

Cosine similarity
The angle between vectors
No — normalizes away length
The default for text embeddings. Two documents about the same subject are similar regardless of length.

Dot product (inner product)
Angle and magnitude
Yes
Used when the model is trained for it, or when magnitude carries meaning. 
Equivalent to cosine when vectors are unit-normalized.

Euclidean (L2)
Straight-line distance
Yes
Common in image and some multimodal embeddings; less common for text.

The rule that prevents a class of quiet failure: match the metric to what the embedding model was trained with. Model documentation states this. Using L2 with a model trained for cosine produces results that look plausible and are measurably worse — you will not get an error, you will get a lower recall number that you may attribute to chunking.

The normalization subtlety. If vectors are unit-normalized (length 1), cosine similarity and dot product produce identical rankings. Many models output normalized vectors; some do not. If you are unsure, check — a store configured for dot product fed unnormalized vectors will rank long documents higher simply because they have larger magnitude, which is a real and frequently-undiagnosed retrieval bug.

4.1.3 Why "nearest" is computationally hard
The naive approach — compare the query to every stored vector — is exact and simple. It is also O(n·d): for 50 million vectors of 1,024 dimensions, that is roughly 51 billion multiply-add operations per query. At interactive latency, this is not viable beyond modest corpus sizes.
  EXACT SEARCH (brute force / flat)
  +--------------------------------------------------------------+
  |  Compare query to EVERY vector.                               |
  |  Recall: 1.0 (perfect, by definition)                         |
  |  Latency: O(n * d) -- linear in corpus size                   |
  |  Viable up to roughly tens of thousands of vectors at         |
  |  interactive latency; more in batch.                          |
  +--------------------------------------------------------------+

APPROXIMATE NEAREST NEIGHBOUR (ANN)
  +--------------------------------------------------------------+
  |  Build a structure that lets you skip most comparisons.       |
  |  Recall: < 1.0 -- you WILL miss some true neighbours          |
  |  Latency: sub-linear -- often near-logarithmic                |
  |  This is what every production vector store uses.             |
  +--------------------------------------------------------------+
The trade is explicit and must be understood as a design parameter, not a defect. ANN search is approximate: it will sometimes fail to return a document that is genuinely among the true nearest neighbours. How often it does so is tunable, and the tuning knob trades directly against latency and memory. An architect who does not know this will treat retrieval misses as an embedding problem and tune the wrong thing.

4.2 Approximate nearest neighbour search, mechanically
4.2.1 HNSW: hierarchical navigable small world
The dominant algorithm in current vector stores. The intuition is worth building carefully because its parameters are the ones you will actually tune.
The structure: a multi-layer graph. Each vector is a node. Nodes are connected to their approximate neighbours. Higher layers are sparse and contain long-range links; lower layers are dense and contain short-range links. The bottom layer contains every vector.
  Layer 2 (sparse, long hops)
     A -------------------- F -------------------- K

Layer 1 (denser)
     A ------ C ------ F ------ H ------ K

Layer 0 (every vector, short hops)
     A - B - C - D - E - F - G - H - I - J - K
The search: enter at the top layer, greedily move toward the query, descend a layer when you can get no closer, repeat. You arrive at the bottom layer already near the target and explore locally.
This is why it is fast: the top layers cover enormous distance in a few hops, so you skip the overwhelming majority of the corpus without ever comparing against it.

The parameters that matter:

Parameter / When set / Controls / Effect of increasing

m
Build time / 
Number of connections per node /
Better recall, more memory, slower build. Changing it requires a rebuild.

ef_construction
Build time
How thoroughly neighbours are searched while building
Better graph quality, therefore better recall. 
Slower build. Requires rebuild to change.

ef_search
Query time
Size of the candidate list explored per query
Better recall, higher latency. Adjustable live, no rebuild.

ef_search is the single most operationally important parameter in this module. It is the one dial you can turn in production, without reindexing, to trade latency for recall. When retrieval latency spikes, lowering it is the fastest mitigation available. When recall is inadequate, raising it is the first thing to try — before touching chunking, before changing embedding models, before adding a reranker.

Memory characteristics. HNSW graphs are held in memory for query performance. Memory footprint is driven by vector count, dimensionality, and m. This is why memory, not compute, is usually the binding constraint on vector store sizing — the same conclusion Module 1 reached about model serving, arrived at by a different route.

4.2.2 IVF: inverted file index
The main alternative, and the mechanism behind pgvector's ivfflat index type.
The structure: cluster the vectors into nlist groups during index build, each with a centroid. At query time, compare the query against the centroids only, then search exhaustively within the closest nprobe clusters.
  Corpus partitioned into clusters:

[C1]      [C2]      [C3]      [C4]      [C5]
      ***       ***       ***       ***       ***
      ***       ***       ***       ***       ***

Query arrives -> compare to 5 centroids (cheap)
                -> search exhaustively inside nearest nprobe=2
                -> skip the other 3 clusters entirely
The parameters:
Parameter / When set / Controls

nlist
Build time
Number of clusters. 
More clusters means finer partitioning and faster search, but higher risk of the true neighbour sitting in an unsearched cluster.
nprobe
Query time
How many clusters to search. 
The recall/latency dial, equivalent in role to ef_search.

The characteristic IVF failure: a true nearest neighbour sitting just across a cluster boundary from the query. If that cluster is not among the nprobe searched, the result is missed entirely. Raising nprobe mitigates it; at nprobe = nlist you are back to exhaustive search.

4.2.3 HNSW versus IVF

HNSW:  / IVF:

Recall at equivalent latency
HNSW:  Generally better
IVF: Generally lower

Memory footprint
HNSW: Higher (graph structure)
IVF: Lower

Build time
HNSW: Slower
IVF: Faster

Incremental insert
HNSW: Handles well
IVF: Degrades as data drifts from the original clustering; benefits from periodic rebuild

Rule of thumb, to validate against your own corpus: HNSW is the default choice for most GenAI retrieval workloads, because recall matters more than memory in typical corpus sizes, and because incremental inserts are common in continuously-updated knowledge bases. IVF becomes attractive at very large scale where memory cost dominates, or where the corpus is static and rebuilt in batches.

4.2.4 The triangle
Every ANN decision is a position on a three-way trade.
                        RECALL
                       (quality)
                          /\
                         /  \
                        /    \
                       /      \
                      /        \
                     /          \
              LATENCY ---------- MEMORY
              (speed)            (cost)

Raise ef_search / nprobe  -> recall UP,  latency UP
   Raise m / ef_construction -> recall UP,  memory UP, build slower
   Quantize vectors          -> memory DOWN, recall DOWN slightly
   Reduce dimensionality     -> memory DOWN, latency DOWN, recall DOWN
You cannot optimize all three. The architect's job is knowing which one the requirement actually binds on, and the requirement usually states it: a strict latency SLO binds latency; a very large corpus on a cost ceiling binds memory; a compliance use case where a missed document is a problem binds recall.

4.2.5 Quantization
Reducing the precision of stored vectors to save memory.
Approach / Mechanism / Memory saving / Recall cost

Scalar quantization
Store each dimension in fewer bits (e.g. 8-bit rather than 32-bit float)
Roughly 4x
Small

Product quantization
Split the vector into sub-vectors, encode each against a learned codebook
Large — often 10x or more
Moderate; tunable

Binary quantization
Reduce each dimension to a single bit
Very large — up to 32x
Substantial, but often recoverable with a rescoring pass over full-precision vectors

The rescoring pattern is what makes aggressive quantization viable. Search the quantized index to retrieve a generous candidate set — say top 100 — then rescore those candidates against full-precision vectors and return the top 10. You get most of the memory saving with most of the recall, at the cost of retaining full-precision vectors somewhere and one extra comparison pass.

Worth noting for the AWS surface in section 4.6: OpenSearch Serverless and OpenSearch managed clusters are the only Bedrock Knowledge Bases vector stores that support storing binary vectors. If binary quantization is part of your cost strategy, that constrains your store choice.

4.3 Dimensionality and storage economics
4.3.1 The arithmetic
Vector storage cost is one of the few things in this material you can compute exactly, and doing so early prevents expensive surprises.
  Raw vector storage (before index overhead):

bytes = n_vectors * n_dimensions * bytes_per_dimension

At 32-bit float (4 bytes per dimension):

1,024-dim vector  =  4,096 bytes  ~= 4 KB per vector

1 million vectors    ->  ~4 GB
    10 million vectors   ->  ~40 GB
    100 million vectors  ->  ~400 GB

HNSW graph overhead adds roughly 30-100% on top of this,
  depending on `m`. Treat raw vector size as a floor, not an
  estimate.
Why this matters: HNSW performance depends on the graph being resident in memory. A 400 GB index does not fit on a single node. That drives sharding, which drives cluster sizing, which drives cost — and all of it follows from a dimensionality choice made when someone picked an embedding model.
4.3.2 Dimensionality as a design decision
Higher dimensionality generally means richer representation and better retrieval quality, with diminishing returns. It also means proportionally more memory, more computation per comparison, and slower index builds.

Dimensionality / Bytes/vector (fp32)10M vectorsTypical positioning256~1 KB~10 GBFast, cheap, adequate for many tasks384~1.5 KB~15 GBCommon for smaller models768~3 KB~30 GBCommon mid-range1,024~4 KB~40 GBCommon for larger models1,536~6 KB~60 GBHigher-end3,072~12 KB~120 GBLarge; rarely justified by measurement

Matryoshka-style truncation. Some embedding models are trained so that a prefix of the full vector is itself a usable embedding — you can take the first 256 dimensions of a 1,024-dimension output and retain most of the retrieval quality. Where a model supports this, it is a genuine free parameter: you can trade quality for memory without changing models, and you can even store truncated vectors for the index and full vectors for rescoring.
The measurement discipline: do not assume higher dimensionality is better for your corpus. Run the retrieval evaluation from Module 5 at two or three dimensionalities and look at the recall difference. It is frequently smaller than the cost difference, and at 100 million vectors that gap is a material budget line.

4.4 Metadata frameworks
Skill 1.4.2. Metadata is what turns a semantic index into a system that can answer "what does the current version of the EU policy say" rather than "what documents are semantically similar to this question." It is also the mechanism for permissions, freshness, and provenance.
4.4.1 What to capture
Module 3 established that metadata must be captured at intake because it cannot be recovered later. Here is what to capture.

Strategy / Ingestion cost / Chunk size predictability / Preserves structure / Context per chunk / Best for

Fixed-size + overlap
- Ingestion cost: Lowest
- Chunk size predictability: High
- Preserves structure: No
- Context per chunk: Low
- Best for: Baseline, dev, uniform prose

Recursive character
- Ingestion cost: Low
- Chunk size predictability: Moderate
- Preserves structure: Partially
- Context per chunk: Low
- Best for: General prose, a free improvement over fixed

Semantic
- Ingestion cost: High
- Chunk size predictability: Low
- Preserves structure: No
- Context per chunk: Moderate
- Best for: Dense uniform prose, moderate file sizes

Hierarchical
- Ingestion cost: Moderate
- Chunk size predictability: High
- Preserves structure: Partially
- Context per chunk: High
- Best for: Long structured documents — robust default

Structure-aware
- Ingestion cost: Moderate (custom)
- Chunk size predictability: Low
- Preserves structure: Yes
- Context per chunk: High
- Best for: Technical docs, legal, anything with headings

None
- Ingestion cost: Lowest
- Chunk size predictability: N/A
- Preserves structure: N/A
- Context per chunk: Complete
- Best for: Short self-contained documents

Two of these carry disproportionate weight:
Tenancy is a security boundary, and section 4.4.4 treats it separately.
Temporal fields separate "modified" from "effective." A policy document edited yesterday to fix a typo has a recent modification date but an effective date from 2019. A document effective next quarter exists now but should not be retrieved as current. Systems that conflate these will confidently cite future or superseded policy. If your domain has effective dates, model them explicitly — this is a common and entirely avoidable class of confident error.

4.4.2 Filterable versus stored metadata
A distinction that has real cost consequences and that stores handle differently.
Filterable metadata participates in query-time filtering. It must be indexed, which costs memory and constrains what values it can hold.
Stored metadata is returned with results but cannot be filtered on. Cheaper; use it for anything you only need to display or cite.
Store limits vary and are worth checking against your design. As one concrete example: when using S3 Vectors with Bedrock Knowledge Bases, you can attach up to 1 KB of custom metadata per vector, across both filterable and non-filterable metadata, with a maximum of 35 metadata keys per vector. That is generous for most designs and genuinely constraining for some — a system carrying twenty classification tags plus provenance plus structural context can hit it.
The design implication: decide early which fields need filtering. Adding a filterable field later may require reindexing.
4.4.3 Pre-filtering versus post-filtering
This is the mechanic most often misunderstood, and it has a counterintuitive failure mode worth internalizing.
  POST-FILTERING
  +---------------------------------------------------------------+
  |  1. ANN search for top-k by vector similarity                  |
  |  2. Discard results that fail the metadata filter              |
  |                                                                |
  |  PROBLEM: if the filter is selective, you may discard          |
  |  nearly everything and return 2 results when you asked         |
  |  for 10. Recall collapses silently.                            |
  +---------------------------------------------------------------+

PRE-FILTERING
  +---------------------------------------------------------------+
  |  1. Apply the metadata filter to determine the eligible set    |
  |  2. Search for nearest neighbours within that set only         |
  |                                                                |
  |  PROBLEM: ANN indexes are built over the WHOLE corpus. A       |
  |  filter matching a tiny fraction can force the search to       |
  |  traverse the graph extensively, or fall back to brute force   |
  |  over the filtered subset. Latency can INCREASE dramatically.  |
  +---------------------------------------------------------------+
The counterintuitive part: a highly selective filter — matching, say, 0.1% of documents — can make a query slower than an unfiltered one, because the ANN structure's efficiency comes from navigating a densely connected graph, and filtering breaks that density. In the worst case, the store falls back to scanning the filtered subset exhaustively.
Modern stores implement adaptive strategies — choosing pre- or post-filtering based on estimated filter selectivity — but the failure mode still exists and still surfaces as unexplained latency spikes on specific queries.
The architect's decision framework:
  How selective is the filter?

Broadly selective (matches >10% of corpus)
       -> Either strategy works. Post-filter with over-fetch is
          simple and adequate.

Moderately selective (1-10%)
       -> Pre-filter. Over-fetch on post-filter would be excessive.

Highly selective (<1%)
       -> Consider a SEPARATE INDEX rather than a filter.
          If the filter value partitions your corpus naturally
          (tenant, region, product line), partitioning the index
          is usually better than filtering one large index.

Is the filter a SECURITY boundary (tenant, classification)?
       -> Do NOT rely on post-filtering. A post-filter is applied
          after retrieval, which means the wrong data was
          retrieved and merely discarded. Pre-filter, or better,
          partition the index. See 4.4.4.
4.4.4 Multi-tenancy: three strategies
The most consequential metadata decision, because getting it wrong is a data breach rather than a performance issue.

Strategy / Mechanism / Isolation strength / Cost / Best for

Shared index, metadata filter
- Mechanism: One index; every query filters on tenant ID
- Isolation strength: Weakest — depends on every query applying the filter correctly
- Cost: Lowest
- Best for: Many small tenants, low sensitivity

Shared index, separate namespaces
- Mechanism: One store, logically partitioned
- Isolation strength: Moderate
- Cost: Low
- Best for: Middle ground where the store supports it well

Index per tenant
- Mechanism: Physically separate indexes
- Isolation strength: Strongest
- Cost: Highest — per-index overhead multiplies
- Best for: Few large tenants, high sensitivity, regulated data

The critical principle, which recurs throughout Domains 2 and 3:
The tenant identifier must be derived from the authenticated session, before the model or the user can influence it — and it must never be a parameter the model populates.
If a tenant ID can be set by anything downstream of user input, a prompt injection or a parameter-handling bug becomes cross-tenant data access. The filter must be applied by infrastructure that sits above the application logic, not asserted by the application as a convention.
Defense in depth for high-sensitivity tenancy combines layers so that no single bug is sufficient:
Physical index separation per tenant, or per tenant group
IAM scoping so the credential used by a tenant's request cannot reach another tenant's index at all
Application-level filtering as a third layer
Monitoring that alarms on any query whose result set contains a foreign tenant ID
Layer 2 is the one that converts this from a code-correctness problem to an infrastructure guarantee, and it is the layer most often missing.

4.4.5 The permissions problem in retrieval
A structural difficulty worth naming explicitly, because it has no clean solution and appears in scenario questions.

Embeddings flatten permissions. Once a document is chunked and indexed, the vector index does not inherently know who is allowed to see it. The source system may have had rich row-, folder-, or document-level permissions; the index has vectors and metadata.

Three approaches, none perfect:
Approach / How / Weakness

Permission metadata + pre-filter
- How: Store an access-control list or group identifiers as filterable metadata; filter per user
- Weakness: Permissions change; the index goes stale. Requires re-sync on every permission change, not just content change.

Index per permission set
- How: Separate indexes for each distinct access group
- Weakness: Explodes combinatorially with fine-grained permissions

Post-retrieval permission check
- How: Retrieve broadly, then check each result against the live source system
- Weakness: Correct and current, but slow, and recall degrades because you may discard most of what you retrieved

The hybrid that works in practice: coarse pre-filtering on stable, low-cardinality attributes (department, classification level, region), followed by a post-retrieval check against the live authority for anything sensitive. Over-fetch to compensate for the post-check discard rate.

What you should say in an architecture review: permission synchronization is an ongoing obligation, not a one-time configuration. If the source system's permissions change frequently, the sync mechanism (section 4.8) must handle permission-only changes, not just content changes — and most implementations do not, which is how stale access ends up embedded in an index.

4.5 Index topology at scale
Skill 1.4.3. How you lay out indexes, shards, and replicas determines whether the system performs at scale and whether it can be operated.
4.5.1 Single index versus multiple indexes
  SINGLE INDEX
  +-------------------------------------------------------------+
  |  All content in one index; distinctions expressed as         |
  |  metadata filters.                                            |
  |                                                               |
  |  +  Simple. One place to query. Cross-domain retrieval        |
  |     works naturally.                                          |
  |  -  Filters may be expensive (4.4.3). Weak isolation.         |
  |     One index's problems affect everyone.                     |
  +-------------------------------------------------------------+

MULTIPLE INDEXES
  +-------------------------------------------------------------+
  |  Content partitioned across indexes by tenant, domain,        |
  |  language, or classification.                                 |
  |                                                               |
  |  +  Strong isolation. Smaller indexes search faster.          |
  |     Independent lifecycle -- reindex one without the others.  |
  |     Per-index tuning.                                         |
  |  -  Requires ROUTING. Cross-index queries need fan-out and    |
  |     score fusion. Per-index overhead multiplies.              |
  +-------------------------------------------------------------+
The partition test. Split into multiple indexes when the partitioning attribute is:
Stable — a document does not move between partitions often
Naturally selective — queries usually target one partition
Operationally meaningful — you would want to reindex, tune, or secure partitions independently
Tenant, region, and language usually pass all three. Topic and document type usually fail the second, because queries cross them freely.
The cost that catches people out: managed vector stores frequently have a per-index or per-collection floor — a minimum capacity allocation that bills whether or not the index is busy. Two hundred small tenant indexes on a store with a meaningful capacity floor is an enormous bill for a small amount of data. Always check the minimum billable unit before designing a per-tenant index topology. This single check has reversed many index-per-tenant designs.
4.5.2 Sharding
Sharding splits one index across multiple nodes. Each shard holds a portion of the vectors; a query fans out to all shards and results are merged.
  Query
    |
    +--> Shard 1 -> local top-k --+
    +--> Shard 2 -> local top-k --+--> merge -> global top-k
    +--> Shard 3 -> local top-k --+
Why you shard: the index no longer fits in one node's memory, or query throughput exceeds what one node can serve.
Sharding facts that matter:
Query latency is bounded by the slowest shard. Fan-out means every query waits for all shards. An unbalanced cluster performs at the speed of its worst node.
Recall can degrade slightly. Each shard returns its local top-k; the true global top-k might include an item that ranked eleventh in one shard and was never returned. Over-fetching per shard mitigates this.
Shard count is usually fixed at index creation. Changing it means reindexing. Size for growth deliberately.
Too many shards is as bad as too few. Each shard adds fan-out overhead and per-shard fixed cost. A common error is over-sharding a small index, which adds coordination cost with no benefit.
Replicas are copies of shards. They add query throughput and availability; they do not reduce per-query latency for a single query and they multiply memory cost.
Rule of thumb, to validate against your workload: size shards so each fits comfortably in node memory with headroom for the graph, then add replicas for throughput and availability. Vector workloads are memory-bound, so the memory-per-shard calculation drives the whole cluster shape.
4.5.3 Hierarchical and routed retrieval
For very large or heterogeneous corpora, a two-stage architecture often outperforms a single flat index.
  Stage 1: ROUTE
    Determine which index or partition is relevant.
    Mechanisms: a classifier, a lightweight model call,
    metadata on the query, or a small "index of summaries."

Stage 2: RETRIEVE
    Search only the selected partition(s), at full depth.

+----------------------------------------------------------------+
  |  Benefits: each search is over a smaller, more homogeneous      |
  |  index -- faster and often MORE accurate, because irrelevant    |
  |  domains cannot produce false neighbours.                       |
  |                                                                 |
  |  Risk: routing errors are unrecoverable. If stage 1 picks the   |
  |  wrong partition, stage 2 cannot find the answer no matter      |
  |  how well it searches.                                          |
  +----------------------------------------------------------------+
Mitigating routing risk: route to the top n partitions rather than one, and fan out. This trades some of the efficiency benefit for a large reduction in catastrophic misses, and it is usually the right default.
Parent-child (hierarchical chunking) is a related but distinct pattern, and the distinction is worth keeping clear:
Routing selects which index to search.
Parent-child searches small chunks for precision, then returns the larger parent chunk or document for context.
Parent-child solves a genuine tension: small chunks embed precisely but lack context; large chunks carry context but embed diffusely. Searching small and returning large gets both. It requires storing the parent-child relationship in metadata, which is a Module 3 intake concern.
4.5.4 Sizing a cluster: the procedure
  1. COUNT vectors.
     documents x average chunks per document.
     Project 12-24 months forward, not just today.

2. COMPUTE raw vector memory.
     n_vectors x dimensions x bytes_per_dimension.

3. ADD index overhead.
     HNSW: budget 30-100% on top, depending on `m`.

4. ADD headroom.
     Query-time working memory, merges, and growth.
     Do not size to 100% utilization.

5. DIVIDE by node memory available for the index.
     -> shard count.

6. MULTIPLY by replica count for throughput and availability.
     -> total node count.

7. VALIDATE against latency requirements with a load test at
     realistic ef_search, on realistic queries.
     A calculation is a hypothesis; the load test is the evidence.
Step 7 is not optional. The arithmetic gives you a starting shape. Whether that shape meets a p99 latency target under concurrent load with your actual filter patterns is an empirical question, and filter patterns in particular can invalidate a clean calculation (section 4.4.3).

4.6 The AWS vector store landscape
Skill 1.4.1. Several genuinely different options, chosen by genuinely different criteria.
4.6.1 Amazon Bedrock Knowledge Bases
Knowledge Bases is not itself a vector store — it is a managed RAG layer that handles ingestion, chunking, embedding, storage, and retrieval, and it sits on top of a vector store you choose.
Supported vector stores include: Amazon OpenSearch Serverless, Amazon OpenSearch Service managed clusters, Amazon Aurora PostgreSQL (with pgvector), Amazon Neptune Analytics, Amazon S3 Vectors, Pinecone, MongoDB Atlas, and Redis Enterprise Cloud.
Three constraints worth committing to memory, because they are exactly the kind of detail that decides a scenario question:
Standard Amazon RDS PostgreSQL with pgvector is not a supported Knowledge Bases vector store — only Aurora PostgreSQL is. Teams running RDS PostgreSQL and hoping to point Knowledge Bases at it will discover this late.
If your data source is Confluence, Microsoft SharePoint, or Salesforce, the only supported vector store is Amazon OpenSearch Serverless. This is a hard constraint that can dictate your entire store choice from a data-source decision made elsewhere.
OpenSearch Serverless and OpenSearch managed clusters are the only supported stores that can hold binary vectors. If binary quantization is part of your memory strategy, this narrows the field.
Aurora also offers a "quick create" path that provisions a preconfigured Aurora Serverless cluster with pgvector in minutes, which makes it a low-friction option for development and smaller production workloads.
What Knowledge Bases gives you: ingestion job orchestration, several chunking strategies, embedding generation, vector store provisioning, metadata filtering, reranking integration, and Retrieve / RetrieveAndGenerate APIs with citations.
What you give up: fine-grained control over the ingestion pipeline, custom chunking logic beyond the provided strategies, and direct index tuning. You are also bound to its sync model.
The default recommendation: start with Knowledge Bases. The Module 2 principle applies — proving retrieval viability quickly is the point of early work, and hand-building an ingestion and indexing pipeline to answer that question is a category error. Move to self-managed when a specific requirement forces it.
4.6.2 Amazon OpenSearch Service
The most capable and most operationally demanding option, in two deployment forms.
Serverless (collections)Managed cluster (domains)Capacity modelOCUs, auto-scalingYou choose instance types and countsMinimum costA capacity floor applies even when idleInstance hours; you control the floorControl over shardingLimitedFullIndex parameter tuningLimitedFull (m, ef_construction, ef_search, engine choice)Operational burdenLowHigherBest forVariable load, teams without OpenSearch expertiseLarge scale, strict latency targets, cost control at steady high volume
Capabilities relevant here:
k-NN with multiple engines (Lucene, nmslib, faiss), supporting HNSW and IVF with full parameter control
Hybrid search — combining BM25 keyword scoring with vector similarity, via search pipelines with normalization and fusion processors. Module 5 develops this; it matters here because it is a store capability, not something you can add later at the application layer.
The Neural plugin — generates embeddings in-cluster at ingest and query time, so your application sends text rather than vectors. It simplifies the pipeline and couples the embedding step to the cluster.
Rich filtering, aggregations, and the full search feature set alongside vector search
Sharding and replica control, which is why it is the answer at large scale
The cost consideration to check early: OpenSearch Serverless allocates a minimum number of OCUs, which establishes a monthly floor regardless of usage. For a small index this floor can dominate the bill and make Serverless a poor fit for low-volume workloads — precisely the workloads where its operational simplicity is most attractive. This tension is a common and legitimate source of frustration; check the current minimum before committing.
4.6.3 Amazon Aurora PostgreSQL with pgvector
When it is the right answer: you already run Aurora PostgreSQL, your vector count is moderate, and — most importantly — you need vectors and relational data to be transactionally consistent.
That last point is the genuine differentiator. If a document's metadata, its permissions, and its embedding must all update atomically, a relational store with vector support does something no dedicated vector store does. Everything else is a tradeoff you could make either way.
Index types: ivfflat (IVF, tuned with lists and probes) and hnsw (tuned with m, ef_construction, and hnsw.ef_search).
Practical strengths: SQL filtering combines naturally with vector search; you get transactions, joins, and familiar operational tooling; EXPLAIN ANALYZE gives real query plans, which is a genuine debugging advantage over most vector stores.
Practical limits: it is a relational database doing vector search, not a purpose-built vector engine. At very large vector counts, performance and memory pressure become limiting, and you will be tuning PostgreSQL rather than tuning a search engine.
4.6.4 Amazon S3 Vectors
The newest shape in the landscape and the one that changes the economics.
What it is: native vector storage and querying in S3 — an object store with vector support, positioned as purpose-built, cost-optimized storage for large vector datasets, with the stated aim of substantially reducing the cost of storing and querying vectors relative to alternatives.
The tradeoff is explicit: cost versus latency. It is object-store-backed rather than memory-resident, so query latency is higher than an in-memory HNSW index. In exchange, storage cost for very large vector sets is dramatically lower.
Where it fits:
Very large vector corpora where in-memory indexing is cost-prohibitive
Workloads tolerant of higher retrieval latency — batch processing, asynchronous enrichment, agent long-term memory
Archival or infrequently-queried vector tiers
Tiered architectures: hot vectors in OpenSearch, cold vectors in S3 Vectors
The metadata constraint noted earlier applies here: with Knowledge Bases, up to 1 KB of custom metadata and 35 metadata keys per vector.
4.6.5 Amazon Neptune Analytics
Graph-based retrieval, supported as a Knowledge Bases vector store. The value is in combining graph traversal with vector similarity — retrieving not just semantically similar chunks but chunks connected to them through explicit relationships.
Where it earns its complexity: corpora with rich, meaningful entity relationships where multi-hop questions are common. "Which suppliers are affected by a regulation that applies to components used in product X" is a graph question that pure vector similarity answers poorly.
Where it does not: ordinary document Q&A. Graph RAG adds real modelling and operational cost and should be chosen because relationships matter, not because it sounds sophisticated.
4.6.6 Amazon Kendra
Managed intelligent search with its own relevance engine, built-in connectors to enterprise systems, and native support for document-level access control.
When it is the right answer: you want enterprise search over many source systems with permission awareness, and you do not want to own embeddings, chunking, or index tuning at all. Its connector ecosystem and ACL handling are the differentiators, and its ACL handling in particular addresses the permissions problem from section 4.4.5 more directly than a raw vector store does.
When it is not: you need control over embedding models, chunking, or retrieval mechanics, or you need vector search as a component in a larger custom pipeline.
4.6.7 DynamoDB alongside a vector store
DynamoDB is not a vector store, but it appears constantly in these architectures as the metadata and state layer:
Document and chunk registry — what has been ingested, at what version, with what content hash
Ingestion job state and idempotency records
Permission mappings that change more often than content
Sync watermarks and change-detection cursors
Parent-child chunk relationships
The pattern is: vectors and filterable metadata in the vector store; everything operational in DynamoDB, keyed for fast lookup. Section 4.8 depends on this layer.
4.6.8 The selection framework
  START
    |
    +-- Is this a PoC or early validation?
    |     -> Bedrock Knowledge Bases with the default store.
    |        Answer the "is retrieval viable" question first.
    |
    +-- Is your data source Confluence, SharePoint, or Salesforce
    |   (via Knowledge Bases)?
    |     -> OpenSearch Serverless. The constraint decides for you.
    |
    +-- Do vectors and relational data need transactional
    |   consistency?
    |     -> Aurora PostgreSQL with pgvector.
    |
    +-- Do entity RELATIONSHIPS drive retrieval quality
    |   (multi-hop, connected questions)?
    |     -> Neptune Analytics.
    |
    +-- Do you need enterprise connectors and document-level ACLs
    |   without owning retrieval mechanics?
    |     -> Kendra.
    |
    +-- Is the corpus very large and is COST the binding
    |   constraint, with latency tolerance?
    |     -> S3 Vectors, possibly as the cold tier of a
    |        tiered architecture.
    |
    +-- Do you need strict latency at scale, hybrid search,
    |   shard control, or binary vectors?
    |     -> OpenSearch: Serverless if load is variable and
    |        the capacity floor is acceptable; managed cluster
    |        if you need control or steady-state cost efficiency.
    |
    +-- Otherwise
          -> Knowledge Bases on OpenSearch Serverless is the
             reasonable default.

ALWAYS CHECK FIRST:
    - Regional availability against residency requirements
    - Minimum billable capacity against your actual data volume
    - Metadata limits against your metadata design
4.7 Integration components
Skill 1.4.4. Connecting the vector store to the systems where content actually lives.
4.7.1 The connector problem
Enterprise content does not sit in an S3 bucket waiting to be indexed. It sits in Confluence, SharePoint, Google Drive, Salesforce, ServiceNow, Jira, Notion, network file shares, and a document management system nobody has fully documented. Each has its own authentication, its own change-notification model (or none), its own permission model, and its own pagination quirks.
Three integration approaches:
Approach / Mechanism / Strengths / Weaknesses

Managed connector
- Mechanism: Knowledge Bases or Kendra data source connectors
- Strengths: Handles auth, crawling, incremental sync, and often permissions. Fastest path.
- Weaknesses: Limited to supported systems; limited customization; and note the vector store constraint for some sources

Export to S3, then ingest
- Mechanism: Periodic export from the source into a staging bucket
- Strengths: Simple, decoupled, works for anything
- Weaknesses: Loses permission context; freshness limited by export cadence; full exports are wasteful

Custom connector
- Mechanism: Your code against the source API
- Strengths: Full control over what is extracted, how permissions map, and how changes are detected
- Weaknesses: You own auth refresh, rate limits, pagination, error handling, and change detection

The default: use a managed connector where one exists and satisfies your permission requirements. Custom connectors are a surprising amount of ongoing maintenance — source APIs change, tokens expire, rate limits shift — and that maintenance is invisible in the initial estimate.

4.7.2 What a connector must actually handle
Whether managed or custom, these are the requirements. Reviewing a custom connector proposal against this list surfaces omissions quickly.
Authentication and refresh — including service accounts, token expiry, and what happens when credentials rotate
Full crawl and incremental crawl — a full crawl for initial load, incremental thereafter
Change detection — how do you know something changed? (section 4.8.2)
Deletion detection — the hardest one. How do you know something was removed? Many sources do not tell you.
Permission extraction — capturing the source's ACLs into metadata, and re-syncing when permissions change independently of content
Rate limiting and backoff — enterprise systems throttle, and a crawl that ignores this gets blocked
Error handling and resumption — a crawl that fails at document 40,000 of 60,000 should resume, not restart
Content type handling — routing to the extraction paths from Module 3
Deletion detection deserves emphasis because it is the most common gap. If a document is deleted from the source and your connector only processes changes, the document remains in the index indefinitely and remains retrievable. Users get answers citing content that no longer exists — and in a compliance context, content that was deliberately removed. The standard solution is a periodic reconciliation pass: enumerate source identifiers, compare against the index registry, and remove orphans.

4.8 Maintenance and freshness
Skill 1.4.5. A vector store is not a one-time build. Keeping it current is an ongoing system with its own failure modes.
4.8.1 The freshness requirement drives the architecture
  How stale can the index be before it causes harm?

Minutes      -> real-time change detection, event-driven
                    ingestion. Highest complexity and cost.

Hours        -> frequent incremental sync on a schedule
                    or triggered by change events. The common case.

Days         -> scheduled incremental sync. Simple.

Weeks/never  -> periodic full rebuild. Simplest of all;
                    do not over-engineer.
Ask this question explicitly during requirements elicitation (Module 2). Teams frequently build real-time ingestion for corpora that change monthly, and the complexity is entirely wasted.
4.8.2 Change detection mechanisms
Mechanism / How / Latency / Reliability

Event notification
- How: Source emits an event on change (S3 Event Notifications, webhooks)
- Latency: Seconds
- Reliability: High, where supported. Not all sources emit events.

Change data capture
- How: Read a database change stream (DynamoDB Streams, database CDC)
- Latency: Seconds
- Reliability: High for database sources

Polling with a watermark
- How: Query for items modified since the last run
- Latency: Minutes to hours
- Reliability: Good, but misses deletions and depends on reliable modification timestamps

Full enumeration and diff
- How: List everything, compare against a registry
- Latency: Whatever the schedule allows
- Reliability: Highest — the only method that reliably catches deletions and out-of-band changes

The pattern that works in practice combines two of these: event-driven or polled incremental sync for freshness, plus a scheduled full reconciliation — nightly or weekly — to catch deletions, missed events, and drift. Relying on incremental sync alone produces an index that slowly diverges from the source in ways nobody notices until an auditor asks.
4.8.3 The update operations
Vector stores handle these differently and the differences matter operationally.
Insert — straightforward. New chunks, new vectors, new metadata.
Update — a document changed. The correct handling is usually delete all chunks for that document, then re-chunk and re-insert, because a content edit may change chunk boundaries entirely. Attempting to update chunks in place produces orphaned and duplicated fragments. This is why a document-to-chunk registry (section 4.6.7) is not optional.
Delete — and here is the operational subtlety. In many index structures, deletion is a tombstone: the vector is marked deleted but remains in the graph until a merge or compaction removes it. Consequences:
Deleted vectors still consume memory
They still participate in graph traversal, degrading search efficiency
Search results must filter them out, adding overhead
High-churn corpora accumulate tombstones and degrade progressively, which is a slow, silent performance decay
The remedy is periodic compaction or force-merge, scheduled during low-traffic windows. If your corpus has high churn — a wiki with hundreds of daily edits — this is a required maintenance job, not an optimization.
Metadata-only update — a permission change or a reclassification with no content change. Some stores support updating metadata without re-embedding, which is dramatically cheaper. If your permissions change frequently, check whether your chosen store supports this; if it does not, every permission change costs a re-embedding.
4.8.4 Migrating embedding models
The hardest maintenance operation, and one that will eventually be necessary — models improve, are deprecated, or turn out to fit your domain poorly.
The constraint: vectors from different models are not comparable. There is no incremental migration. Both indexes must be complete before you can switch.
  ZERO-DOWNTIME EMBEDDING MIGRATION

1. PROVISION a second index with the new model's dimensions.

2. DUAL-WRITE: new and updated documents go to BOTH indexes.
     Start this before backfilling, so the new index does not
     fall behind while you backfill.

3. BACKFILL the new index from source content. This is a full
     re-embedding of the corpus -- budget the cost and the time.
     Use batch inference; on-demand for a full corpus re-embed
     is a common and avoidable overspend.

4. VALIDATE: run the retrieval evaluation set (Module 5)
     against BOTH indexes. Compare recall@k and NDCG.
     Do not assume the new model is better -- measure it.
     A newer model can be worse on your specific domain.

5. SHADOW: send production queries to both, log both result
     sets, compare. Catches distribution differences that a
     fixed eval set misses.

6. CUT OVER gradually -- a traffic percentage with the ability
     to revert instantly.

7. DECOMMISSION the old index only after a stability period.

Throughout: keep the SOURCE content, not just the vectors.
  You cannot re-embed from vectors.
Step 4 is the one teams skip, and it is the one that matters. Migrating to a newer model without measuring is a change with unknown direction. The Domain 5 book, Module 4, provides the evaluation method.
4.8.5 Index health monitoring
Vector indexes degrade in ways that do not surface as errors. Monitor:
Signal / Watch for / Indicates

Search latency p50/p99
- Watch for: Gradual rise
- Indicates: Fragmentation, tombstone accumulation, memory pressure

Memory utilization
- Watch for: Approaching capacity
- Indicates: Time to shard, quantize, or prune

Segment or shard count
- Watch for: Growth
- Indicates: Merge pressure; force-merge candidate

Deleted-document ratio
- Watch for: Rising
- Indicates: Tombstone accumulation; compaction needed

Index vs. source document count
- Watch for: Divergence
- Indicates: Sync drift — orphans or missed ingests

Ingestion job failure rate
- Watch for: Any increase
- Indicates: Connector or extraction problems

Retrieval score distribution
- Watch for: Downward shift
- Indicates: Corpus drift, embedding mismatch, or content quality decline
The last two are the ones that catch real problems. A divergence between the source document count and the index count is the clearest possible signal that your sync is broken, and it is trivially cheap to compute. A downward shift in retrieval score distribution is the earliest available warning that retrieval quality is degrading, and it precedes user complaints by weeks.

4.9 Consolidated decision reference
  +-----------------------------------------------------------------+
  |  ANN TUNING: which dial?                                         |
  |                                                                  |
  |  Recall too low        -> raise ef_search / nprobe FIRST         |
  |                           (live, no rebuild). Only then consider |
  |                           chunking, embeddings, reranking.       |
  |  Latency too high      -> lower ef_search / nprobe FIRST         |
  |  Memory constrained    -> quantize; reduce dimensionality;       |
  |                           lower `m` (requires rebuild)           |
  |  Build too slow        -> lower ef_construction (costs recall)   |
  +-----------------------------------------------------------------+

+-----------------------------------------------------------------+
  |  FILTERING: how selective?                                       |
  |    >10% of corpus     -> post-filter with over-fetch is fine     |
  |    1-10%              -> pre-filter                              |
  |    <1%                -> consider a SEPARATE INDEX               |
  |    security boundary  -> partition + IAM. NOT post-filtering.    |
  +-----------------------------------------------------------------+

+-----------------------------------------------------------------+
  |  MULTI-TENANCY: how much isolation?                              |
  |    many small, low sensitivity -> shared index + filter          |
  |    middle ground               -> namespaces                     |
  |    few large, regulated        -> index per tenant + IAM scoping |
  |                                                                  |
  |    ALWAYS: tenant ID derived from the authenticated session,     |
  |    never a parameter the model can populate.                     |
  |    CHECK: minimum billable capacity per index before choosing    |
  |    index-per-tenant.                                             |
  +-----------------------------------------------------------------+

+-----------------------------------------------------------------+
  |  FRESHNESS: how stale is tolerable?                              |
  |    minutes  -> event-driven ingestion                            |
  |    hours    -> frequent incremental sync                         |
  |    days     -> scheduled incremental                             |
  |    weeks    -> periodic full rebuild (do not over-engineer)      |
  |                                                                  |
  |    ALWAYS pair incremental sync with periodic full               |
  |    reconciliation, or deletions and missed events accumulate.    |
  +-----------------------------------------------------------------+
4.10 Checkpoint
1. Why can queries and documents never be embedded with different models? (4.1.1)
Vectors from different models occupy different, incompatible spaces. Distances between them are meaningless. Changing the embedding model therefore requires a full reindex — there is no partial migration.
2. You configured your store for dot product but your model outputs unnormalized vectors. What happens? (4.1.2)
Longer documents rank higher simply because their vectors have greater magnitude. Results look plausible and are systematically wrong. Match the metric to the model, and check whether output is normalized.
3. Which ANN parameter can you change in production without a rebuild, and what does it trade? (4.2.1)
ef_search in HNSW, or nprobe in IVF. It trades recall against latency. It is the first dial to reach for in either direction — before changing chunking, embeddings, or adding a reranker.
4. A metadata filter matches 0.1% of your corpus. Why might the query get slower? (4.4.3)
ANN efficiency depends on navigating a densely connected graph. A highly selective filter breaks that density, forcing extensive traversal or a fallback to exhaustive scan over the filtered subset. Below roughly 1% selectivity, a separate index is usually better than a filter.
5. Why is post-filtering unacceptable for a tenant boundary? (4.4.3, 4.4.4)
Post-filtering applies after retrieval, meaning the other tenant's data was retrieved and then discarded. The isolation depends entirely on the discard step being correct. Partition the index and scope with IAM so the credential cannot reach foreign data at all.
6. Name the structural difficulty with permissions in retrieval. (4.4.5)
Embeddings flatten permissions. The index holds vectors and metadata; it does not inherit the source system's access model. Permission changes must therefore trigger re-sync independently of content changes, which most implementations do not handle.
7. What is the most commonly missed connector requirement? (4.7.2)
Deletion detection. Sources often do not report deletions, so removed documents remain retrievable indefinitely. The remedy is periodic reconciliation: enumerate source IDs, compare against the index registry, remove orphans.
8. Why do high-churn corpora degrade over time even with correct sync? (4.8.3)
Deletion is typically a tombstone. Deleted vectors still consume memory and still participate in graph traversal until compaction removes them. High churn accumulates tombstones and progressively degrades search efficiency, silently. Schedule compaction.
9. A document is edited. Why update rather than patch the affected chunk? (4.8.3)
A content edit can change chunk boundaries entirely. Patching produces orphaned and duplicated fragments. Delete all chunks for the document, then re-chunk and re-insert — which requires a document-to-chunk registry.
10. Which step of an embedding-model migration do teams most often skip, and why does it matter? (4.8.4)
Validating the new index against a retrieval evaluation set before cutting over. A newer model is not automatically better on your specific domain. Without measurement, the migration is a change of unknown direction.
11. What three things should you always check before committing to a vector store? (4.6.8)
Regional availability against residency requirements; minimum billable capacity against your actual data volume; and metadata limits against your metadata design.
12. Your data source is SharePoint, via Knowledge Bases. What does that determine? (4.6.1)
Your vector store. OpenSearch Serverless is the only supported option for Confluence, SharePoint, and Salesforce sources — a data-source decision made elsewhere dictates your store choice.
4.11 Practice cases
Case 1.14 — Sizing for fifty million chunks
Scenario. A media company is building retrieval over its full editorial archive.
~2.5 million articles, averaging 20 chunks each — approximately 50 million chunks
Embedding model outputs 1,024 dimensions
Requirement: p99 retrieval latency under 50 ms
Strict per-tenant isolation: content is licensed to 12 corporate clients with different, overlapping access rights
Archive grows by ~40,000 articles monthly
Queries frequently filter by publication date range and by section
Produce: a store selection, index topology, and sizing analysis.

Discussion.
Start with the arithmetic, because it eliminates options immediately.
  Raw vectors:  50,000,000 x 1,024 x 4 bytes  =  ~205 GB
  HNSW overhead (budget ~50%):                =  ~100 GB
  Working headroom:                            =  ~60 GB
  ------------------------------------------------------
  Total memory footprint:                      =  ~365 GB
That number decides several things at once. It does not fit on one node, so sharding is mandatory. It rules out Aurora pgvector as the primary store at this scale — you would be fighting PostgreSQL memory management rather than tuning a search engine. It also makes the 50 ms p99 requirement demanding but achievable, since a memory-resident sharded HNSW index is exactly the configuration that meets that target.
Store selection: OpenSearch managed cluster. The reasoning:
50 ms p99 requires memory-resident HNSW with tuned parameters, which needs full control over m, ef_construction, and ef_search — Serverless does not give you this.
365 GB across shards requires deliberate shard sizing, which needs a managed cluster.
At steady high volume, managed cluster instance-hour cost beats Serverless OCU scaling.
S3 Vectors is disqualified by the latency requirement — its economics are compelling and its latency profile is not compatible with 50 ms p99.
Index topology: the interesting decision, and the one to reason about carefully.
The instinct is index-per-tenant for isolation. Resist it here. Twelve tenants with overlapping access rights means content is not partitionable by tenant — the same article may be licensed to seven of them. Index-per-tenant would require storing each article up to twelve times, multiplying a 365 GB footprint to potentially several terabytes for no isolation benefit that a filter cannot provide.
The correct topology partitions on something that is genuinely disjoint. Options:
By date range — articles belong to exactly one period, queries frequently filter by date, and old content is queried less. This enables a hot/cold tier: recent years in a high-performance cluster, archive in a cheaper tier or S3 Vectors.
By section — if queries usually target one section.
Date is the stronger choice given the stated query pattern and the natural access skew.
Licensing handled as pre-filtered metadata. Each chunk carries the set of tenants licensed to see it. Queries pre-filter on the requesting tenant. Because each tenant sees a substantial fraction of the corpus, this filter is broadly selective — comfortably in the range where filtering performs well (section 4.4.3).
Critically: the tenant identifier comes from the authenticated session, never from request content. And because licensing is a commercial boundary rather than a regulatory one, metadata filtering with strong session-derived tenant resolution is defensible here — a judgment that would change if this were regulated personal data.
Sizing:
  Target per-shard index size: ~40 GB (fits comfortably with
  headroom on a node with 64+ GB of usable memory)

365 GB / 40 GB  =  ~9 primary shards  (round to 10)
  x 2 replicas for throughput and availability
  =  ~30 shard copies

-> node count follows from instance memory and the shard
     copies per node you are willing to co-locate.

Then LOAD TEST at realistic ef_search with realistic filters.
  The date-range filter in particular must be tested, because
  filter selectivity can invalidate the calculation.
Growth planning. 40,000 articles monthly is ~800,000 chunks — roughly 1.6% growth per month, or about 21% annually. Shard count is fixed at creation, so size for 24 months, not for today. Alternatively, date-based partitioning gives a natural answer: create a new index per year, which sidesteps the fixed-shard-count problem entirely and is a strong argument for the date-partitioned topology.
What a strong answer includes: doing the memory arithmetic first and letting it eliminate options; recognizing that overlapping tenant rights make index-per-tenant actively wrong; choosing a partition key that is genuinely disjoint and matches the query pattern; noting that date partitioning also solves the shard-count-growth problem; and insisting on a load test with realistic filters.
The reasoning being tested: can you compute the footprint and let it drive the design; can you recognize when the intuitive isolation strategy is inappropriate for the access model; and do you size for growth given that shard count is not easily changed?

Case 1.15 — The store that got slower
Scenario. A retrieval system has been in production for eight months over a corporate wiki.
~180,000 chunks, OpenSearch, HNSW
At launch: p99 retrieval 35 ms
Now: p99 retrieval 240 ms
Corpus has grown roughly 15% in that time
No configuration changes, no code changes
The wiki is actively edited — roughly 400 page edits per day
Recall on a fixed evaluation set has also declined, from 0.91 to 0.86
Produce: a diagnostic sequence and the most likely cause.

Discussion.
Two facts constrain the diagnosis sharply. A 15% corpus growth cannot explain a nearly 7x latency increase — HNSW search is roughly logarithmic in corpus size, so 15% growth should be nearly invisible. And recall declining alongside latency rules out simple resource contention, which would slow things down without making results worse.
Something is wrong with the index structure itself, and the edit rate is the clue.
The diagnostic sequence:
Step 1: Check the deleted-document ratio. 400 edits per day over eight months is roughly 96,000 edits against a 180,000-chunk corpus. If each edit triggers delete-and-reinsert of that page's chunks (which section 4.8.3 says it should), the index has processed an enormous volume of deletions. If tombstones have not been compacted, the index may contain a substantial proportion of deleted vectors that still consume memory and still participate in graph traversal.
This is the leading hypothesis, and it explains both symptoms. Tombstones degrade latency by inflating the graph traversed. They degrade recall because HNSW graph quality assumes the graph reflects the live data — as deleted nodes accumulate, connectivity degrades and the greedy search finds worse paths.
Step 2: Check segment count. Many small segments from continuous writes, without merging, add per-query overhead across segments.
Step 3: Check memory utilization and JVM pressure. If graph memory has grown past comfortable capacity — plausibly because of tombstones — you get garbage collection pressure and paging, both of which inflate p99 specifically.
Step 4: Check index-vs-source document count. This is cheap and diagnostic in its own right. Divergence indicates sync drift and would explain part of the recall decline independently.
Step 5: Confirm no upstream change. Verify the embedding model version has not changed and that ingestion is not silently truncating chunks (Module 3, section 3.5.1). "No code changes" does not rule out a dependency version change.
The remediation, in order:
Force-merge / compaction during a low-traffic window to purge tombstones and consolidate segments. This is likely to recover most of the latency and some of the recall immediately.
Schedule compaction as a recurring job. This is the actual fix — the problem will recur otherwise. High-churn corpora require compaction as routine maintenance, not as an incident response.
Add index health monitoring (section 4.8.5): deleted-document ratio, segment count, memory utilization, and search latency percentiles, with alarms on trend rather than absolute value.
Reconcile against the source to correct any drift.
Re-measure recall after compaction. If it has not recovered to near 0.91, there is a second cause — most likely sync drift or content quality decline — and it should be pursued separately.
A design question worth raising afterward. With 400 edits per day, is delete-and-reinsert of all chunks per edit the right model? If most edits are small, a content-hash comparison at chunk level would let you re-embed only changed chunks, dramatically reducing churn. This is an optimization, not a fix, but it addresses the root pressure.
What a strong answer includes: recognizing that 15% growth cannot explain 7x latency; using the combination of latency and recall degradation to rule out simple resource contention; identifying edit rate as the causal driver; distinguishing the immediate remediation (compaction) from the actual fix (scheduled compaction plus monitoring); and re-measuring to check whether one cause explains everything.
The reasoning being tested: can you reason quantitatively about whether a proposed cause is sufficient to explain an observed magnitude, and do you distinguish incident remediation from the systemic fix?

Case 1.16 — The permission problem
Scenario. A professional services firm wants retrieval over its internal knowledge base: client deliverables, methodologies, and case studies.
~90,000 documents in a document management system
Access is granular: each document has an ACL, driven by client engagement teams. A consultant sees documents from engagements they worked on, plus firm-wide material.
ACLs change frequently — people join and leave engagements weekly
Some documents are client-confidential; exposure to the wrong consultant is a contractual breach
~1,200 consultants
Freshness requirement: new deliverables searchable within a day
Produce: an architecture addressing the permissions problem, being explicit about the residual risk.

Discussion.
This is section 4.4.5 as a live problem, and the honest answer acknowledges there is no clean solution.
Rule out the obvious approaches first, with reasons:
Index per permission set is combinatorially impossible. With 1,200 consultants and engagement-based access, the number of distinct permission sets approaches the number of consultants. This is not a viable topology.
ACL as filterable metadata alone fails on the change rate. ACLs change weekly; content changes on a different cadence entirely. If the ACL lives in the index, every permission change requires an index update, and a missed update means a consultant retains access to a document after leaving an engagement. Given that exposure is a contractual breach, an eventually-consistent permission model is not adequate on its own.
The recommended architecture is layered, with the live authority check as the actual security boundary:
  Query from consultant C
      |
      v
  1. RESOLVE C's access groups from the live source of truth
     (engagement memberships, firm-wide roles).
     Cached briefly -- minutes, not hours.
      |
      v
  2. PRE-FILTER retrieval on COARSE, STABLE attributes:
     - firm-wide vs. client-confidential
     - practice area
     - the engagement IDs C currently belongs to
     Over-fetch: request top-30 rather than top-10.
      |
      v
  3. POST-RETRIEVAL AUTHORIZATION CHECK against the document
     management system for every candidate.
     <-- THIS is the security boundary. Live, authoritative,
         and it cannot be stale.
      |
      v
  4. Return top-10 of what survives, with citations.
Why this layering: step 2 makes retrieval efficient and removes the bulk of ineligible content cheaply. Step 3 makes it correct, because it consults the live authority rather than an index snapshot. Step 2 alone is fast and unsafe; step 3 alone is safe and slow.
The engagement-ID filter is the key design element. Engagement membership is relatively low-cardinality per consultant — a consultant is on a handful of engagements — and it maps directly to how access is granted. Filtering on the union of a consultant's engagements plus firm-wide content is broadly selective enough to perform acceptably, while eliminating the large majority of documents they cannot see.
Residual risks, stated explicitly — this is the part of the answer that matters most:
Metadata staleness between sync runs. If a document's classification changes and the index has not caught up, step 2 may filter incorrectly. Because step 3 is authoritative, this causes under-retrieval (missing documents the consultant may see), not over-exposure. That is the correct direction for the failure to point, and it is a deliberate design property worth stating.
The post-check adds latency, proportional to the candidate count. Batch the authorization calls; cache per-consultant results for a short window.
Recall degrades because some retrieved candidates are discarded. Over-fetching compensates. Measure the discard rate in production and tune the over-fetch factor against it.
Chunk-level versus document-level permissions. If the DMS grants at document level, the check is per source document, not per chunk — cheaper, and worth confirming.
The index still contains the content. Anyone with direct index access sees everything. This means index access itself must be tightly IAM-controlled, and it should be stated in the design rather than assumed.
Permission-change synchronization. Even though step 3 is authoritative, the coarse metadata should be re-synced when engagements change — otherwise step 2's filter degrades over time. This requires the connector to handle permission-only changes, which section 4.7.2 flags as a common gap. A nightly reconciliation of engagement membership into chunk metadata is adequate given that step 3 backstops it.
What a strong answer includes: ruling out the two obvious approaches with specific reasons; making the live authority check the actual boundary rather than the index; noting that the failure mode points toward under-retrieval rather than over-exposure and identifying that as deliberate; over-fetching to compensate for discard; and naming that the index itself holds all content and must be IAM-protected independently.
The reasoning being tested: can you design a layered permission architecture where the security boundary sits at the authoritative source rather than in a cached index, and can you state residual risk honestly rather than presenting the design as complete?

Case 1.17 — The store selection review
Scenario. Four teams have submitted vector store proposals. Review each.
TeamCorpusRequirementProposalA12,000 chunks of product docsInternal search assistant, no latency SLOOpenSearch managed cluster, 3 nodes, custom HNSW tuningB40M chunks of log-derived summariesBatch enrichment overnight; cost is the binding constraintOpenSearch ServerlessC500,000 chunks from ConfluenceEmployee assistant via Bedrock Knowledge BasesAurora PostgreSQL with pgvectorD3M chunks of contracts and their entity relationshipsMulti-hop questions about which agreements affect which entitiesOpenSearch with metadata filters
Produce: a verdict and recommendation for each.

Discussion.
Team A — massively over-engineered. 12,000 chunks is roughly 50 MB of vectors at typical dimensionality. A three-node managed cluster with custom HNSW tuning is an operational burden with no corresponding benefit, and at this scale even exact search would perform acceptably. There is no latency SLO to justify tuning.
Recommendation: Bedrock Knowledge Bases with the default store, or Aurora if they already run it. Revisit if the corpus grows by an order of magnitude. The Module 2 principle applies — do not build infrastructure to answer a question that a managed service answers in an afternoon.
Team B — wrong store for the stated constraint. Cost is explicitly binding and the workload is overnight batch with no latency requirement. OpenSearch Serverless is optimized for exactly the opposite profile: it carries a capacity floor and delivers low latency they do not need. At 40 million chunks they would be paying substantial memory-resident cost for a workload that runs a few hours a night.
Recommendation: S3 Vectors. This is precisely its case — very large corpus, cost-binding, latency-tolerant. The higher query latency is irrelevant for overnight batch. If they later add an interactive path, a tiered architecture with a hot subset in OpenSearch is the natural evolution.
Team C — will not work, and the reason is a specific constraint. They are using Bedrock Knowledge Bases with a Confluence data source. For Confluence, SharePoint, and Salesforce sources, the only supported Knowledge Bases vector store is OpenSearch Serverless. Aurora is not an option in this configuration.
Recommendation: OpenSearch Serverless, as the constraint requires. If Aurora is strongly preferred for organizational reasons, the alternative is to export Confluence content to S3 and use S3 as the data source — but that trades away the connector's incremental sync and permission handling, which is usually a bad trade. Take the constraint.
Worth noting: this is the kind of dependency that should be discovered during design, not during implementation. It is a five-minute documentation check.
Team D — right store, wrong mechanism for the requirement. "Multi-hop questions about which agreements affect which entities" is a relationship traversal problem. Metadata filters express attributes of a chunk; they do not express connections between chunks. A question like "which contracts are affected by a clause in the master agreement that governs entity X" requires following relationships, and vector similarity plus attribute filtering answers it poorly.
Recommendation: evaluate Neptune Analytics as a Knowledge Bases vector store, giving graph traversal alongside vector similarity.
But add a caution: graph RAG carries real modelling and operational cost. Before committing, they should verify that multi-hop questions are actually a substantial share of real usage rather than an anticipated one. If most queries turn out to be ordinary single-hop document lookup, OpenSearch with well-designed metadata is simpler and adequate. Measure the query distribution before adopting the more complex architecture — this is the Module 2 proof-of-concept discipline applied to a store decision.
The pattern across all four: each team selected a store by familiarity or by perceived sophistication rather than by matching the store's characteristics to the binding constraint. The selection framework in section 4.6.8 resolves all four, and in three of the four cases the binding constraint was stated plainly in the requirement.
The reasoning being tested: can you match store characteristics to the actual binding constraint, do you know the specific compatibility constraints that override preference, and can you recommend the more complex option while insisting it be justified by measurement?

4.12 Skill coverage audit
Skill statementWhere discharged1.4.1 — Create advanced vector database architectures for FM augmentation, enabling efficient semantic retrieval beyond traditional search (Bedrock Knowledge Bases, OpenSearch with the Neural plugin, RDS with S3 document repositories, DynamoDB with vector databases for metadata and embeddings)What a vector store is and the embedding space: 4.1. Knowledge Bases as a managed RAG layer, its supported stores, and its three hard constraints: 4.6.1. OpenSearch Serverless vs. managed cluster, including the Neural plugin and hybrid search: 4.6.2. Aurora pgvector and the transactional-consistency differentiator: 4.6.3. S3 Vectors and the cost/latency trade: 4.6.4. Neptune Analytics for relationship-driven retrieval: 4.6.5. Kendra: 4.6.6. DynamoDB as the metadata and state layer: 4.6.7. Full selection framework: 4.6.8. Applied in Cases 1.14 and 1.17.1.4.2 — Develop comprehensive metadata frameworks to improve search precision and context awareness (S3 object metadata for timestamps, custom attributes for authorship, tagging systems for domain classification)What to capture and why, including the modified-vs-effective distinction: 4.4.1. Filterable vs. stored metadata and store limits: 4.4.2. Pre- vs. post-filtering mechanics and the selectivity failure mode: 4.4.3. Multi-tenancy strategies and the session-derived tenant principle: 4.4.4. The permissions-flattening problem and its three imperfect solutions: 4.4.5. Applied in Case 1.16.1.4.3 — Implement high-performance vector database architectures to optimize semantic search at scale (OpenSearch sharding strategies, multi-index approaches, hierarchical indexing)ANN mechanics — HNSW, IVF, their parameters, and the recall/latency/memory triangle: 4.2. Quantization and the rescoring pattern: 4.2.5. Dimensionality and storage economics: 4.3. Single vs. multiple indexes and the partition test: 4.5.1. Sharding mechanics, fan-out latency, and shard-count immutability: 4.5.2. Hierarchical routing and parent-child retrieval: 4.5.3. The cluster sizing procedure: 4.5.4. Applied in Case 1.14.1.4.4 — Use AWS services to create integration components connecting with resources (document management systems, knowledge bases, internal wikis)The connector problem and three integration approaches: 4.7.1. The eight requirements any connector must handle, with deletion detection called out as the common gap: 4.7.2. Store-specific data source constraints: 4.6.1. Applied in Cases 1.16 and 1.17.1.4.5 — Design and deploy data maintenance systems ensuring vector stores contain current and accurate information (incremental update mechanisms, real-time change detection, automated synchronization, scheduled refresh pipelines)Freshness requirement driving architecture: 4.8.1. Change detection mechanisms and their reliability: 4.8.2. Insert, update, delete, and the tombstone degradation problem: 4.8.3. Zero-downtime embedding model migration: 4.8.4. Index health monitoring signals: 4.8.5. Applied in Case 1.15.4.13 Module summary
Mechanics:
A vector store answers one question — which stored points are nearest to this query point. Everything else is machinery for answering it fast, at scale, over the right subset.
Proximity encodes similarity, not truth. Two chunks stating opposite things are near neighbours. Distinguishing them is the model's job, not the store's.
Match the distance metric to what the embedding model was trained with, and check whether output vectors are normalized. Mismatches produce plausible, measurably worse results with no error.
ANN search is approximate by design. It will miss true neighbours; how often is a tunable parameter.
ef_search (HNSW) and nprobe (IVF) are query-time dials — the only recall/latency adjustment available without a rebuild, and the first thing to reach for in either direction.
Memory, not compute, is the binding constraint on vector store sizing.
Metadata:
Capture identity, provenance, temporal, classification, domain, tenancy, quality, and structural fields at intake. It cannot be recovered later.
Separate "modified" from "effective" dates, or the system will confidently cite superseded or future policy.
Filter selectivity determines strategy: above 10%, post-filter; 1–10%, pre-filter; below 1%, consider a separate index. Highly selective filters can make queries slower.
Never use post-filtering as a security boundary. The data was retrieved and then discarded.
Tenant identity must be derived from the authenticated session, never populated by the model.
Embeddings flatten permissions. The workable pattern is coarse pre-filtering plus a live post-retrieval authorization check, with over-fetch to compensate.
Topology:
Partition into multiple indexes when the attribute is stable, naturally selective, and operationally meaningful. Tenant, region, and language usually qualify; topic and document type usually do not.
Check the minimum billable capacity per index before designing index-per-tenant.
Query latency is bounded by the slowest shard. Shard count is usually fixed at creation — size for growth.
Routing selects which index to search; parent-child searches small and returns large. Different patterns solving different problems.
AWS surface — the constraints worth memorizing:
Knowledge Bases supports OpenSearch Serverless and managed clusters, Aurora PostgreSQL, Neptune Analytics, S3 Vectors, and several third-party stores.
Standard RDS PostgreSQL with pgvector is not supported — only Aurora.
Confluence, SharePoint, and Salesforce data sources require OpenSearch Serverless.
Only OpenSearch Serverless and managed clusters support binary vectors.
S3 Vectors trades latency for dramatically lower cost at large scale.
Always check regional availability, minimum billable capacity, and metadata limits before committing.
Maintenance:
Let the freshness requirement drive the architecture. Do not build real-time ingestion for a monthly-changing corpus.
Pair incremental sync with periodic full reconciliation, or deletions and missed events accumulate silently.
On update, delete and re-chunk the whole document — edits change chunk boundaries.
Deletion is a tombstone. High-churn corpora degrade progressively until compaction. Schedule it.
Embedding migration requires dual-write, backfill, measurement, shadow, gradual cutover. A newer model is not automatically better on your domain.
Monitor index-vs-source document count and retrieval score distribution. They are cheap and they catch the problems that do not raise errors.
Carry into Module 5: this module built the store. Module 5 addresses what you put in it and how you query it — chunking strategy, embedding selection, hybrid search, reranking, query transformation, and the retrieval interfaces that connect to foundation models. The metadata framework from section 4.4 and the ANN parameters from section 4.2 are the substrate that Module 5's techniques operate on.

End of Module 4. Module 5 — Retrieval Mechanisms — follows.

──────────────────────
Q: What do these metadata strategies (fixed-size, Recursive, Semantic, Hierarchical) look like in practice?

Chunking Strategies in Practice
These are actually chunking strategies, not metadata strategies — they're about how you split source documents into pieces before embedding. Let me walk through each one concretely, showing what the input and output actually look like.

The Source Document We'll Use
Let's say we have a product manual with this structure:
Chapter 3: Installation

3.1 Prerequisites

Before installing, ensure your system meets the following requirements:
- Operating system: Windows 10 or later
- RAM: 8 GB minimum, 16 GB recommended
- Disk space: 20 GB free

3.2 Installation Steps

Download the installer from the official website. Run the installer 
as administrator. Accept the license agreement. Choose your installation 
directory. Click Install and wait for completion.

3.3 Troubleshooting

If the installer fails, check that you have sufficient disk space. 
Antivirus software may block the installer — add an exception if needed.
Fixed-Size + Overlap
What it does: Counts characters (or tokens), cuts at that count, then backs up by the overlap amount before starting the next chunk.
Chunk 1 (chars 0-400):
"Chapter 3: Installation\n\n3.1 Prerequisites\n\nBefore installing, 
ensure your system meets the following requirements:\n- Operating 
system: Windows 10 or later\n- RAM: 8 GB minimum, 16 GB recommended
\n- Disk space: 20 GB free\n\n3.2 Installation Steps\n\nDownload 
the installer from the official website. Run the"

Chunk 2 (chars 350-750):  ← 50-char overlap with Chunk 1
"the installer from the official website. Run the installer as 
administrator. Accept the license agreement. Choose your installation 
directory. Click Install and wait for completion.\n\n3.3 Troubleshooting
\n\nIf the installer fails, check that you have sufficient disk space.
Antivirus software may block"
Notice what happened: The chunk boundary fell in the middle of a sentence ("Run the..."), and the section header "3.2 Installation Steps" got split from its content. The overlap helps — if someone asks "how do I install?", both chunks will surface because the key sentence spans them — but neither chunk is coherent on its own.
Why it's the baseline: Zero intelligence required. No parsing, no model calls. Works on any text. You can implement it in five lines of Python. That's genuinely valuable for prototyping.
The real failure mode: Dense technical content where a single concept spans 600 characters gets sliced in half. You retrieve the second half, it lacks context, the LLM produces a confused answer. The user blames the AI.

Recursive Character Splitting
What it does: Tries a hierarchy of separators in order — typically ["\n\n", "\n", ". ", " ", ""] — choosing the largest one that keeps the chunk under the size limit.
Attempt: split on "\n\n" (paragraph breaks)
→ Paragraphs are 150-300 chars each. All fit under 400-char limit.

Chunk 1: "Chapter 3: Installation"
Chunk 2: "3.1 Prerequisites\n\nBefore installing, ensure your system 
meets the following requirements:\n- Operating system: Windows 10 
or later\n- RAM: 8 GB minimum, 16 GB recommended\n- Disk space: 
20 GB free"
Chunk 3: "3.2 Installation Steps\n\nDownload the installer from the 
official website. Run the installer as administrator. Accept the 
license agreement. Choose your installation directory. Click Install 
and wait for completion."
Chunk 4: "3.3 Troubleshooting\n\nIf the installer fails, check that 
you have sufficient disk space. Antivirus software may block the 
installer — add an exception if needed."
Notice what happened: Section headers stay with their content. Sentences aren't broken mid-stream. This is meaningfully better than fixed-size for almost no extra cost — hence the document calling it "a free improvement over fixed."
Where it still fails: If a paragraph is 2,000 characters long, the algorithm recurses down to sentence splitting, then word splitting. You lose the paragraph boundary preference. Also, it has no understanding of what the text means — it's purely syntactic.

Semantic Chunking
What it does: Embeds sentences (or small groups of sentences), then finds points where the embedding shifts significantly — where the topic changes. It cuts there.
Sentence embeddings computed:
S1: "Before installing, ensure your system meets requirements." → [0.2, 0.8, ...]
S2: "Operating system: Windows 10 or later."                   → [0.21, 0.79, ...]
S3: "RAM: 8 GB minimum, 16 GB recommended."                    → [0.19, 0.81, ...]
S4: "Download the installer from the official website."        → [0.6, 0.3, ...]
                                                                    ↑
                           Cosine distance between S3 and S4 is HIGH
                           → CUT HERE

Chunk 1: "Before installing, ensure your system meets requirements.
Operating system: Windows 10 or later. RAM: 8 GB minimum, 16 GB 
recommended. Disk space: 20 GB free."

Chunk 2: "Download the installer from the official website. Run the 
installer as administrator. Accept the license agreement. Choose 
your installation directory. Click Install and wait for completion."
Notice what happened: The boundary fell at a meaningful topic shift — from prerequisites to steps. No arbitrary character count. The chunks are semantically coherent units.
The cost: You're embedding every sentence during ingestion, not just the final chunks. For a 10-million-document corpus, this is expensive. It's also why chunk size is unpredictable — some topics run for 50 words, some for 500.
Where it struggles: Highly technical content where every sentence is dense and distinct. The distance threshold is a tunable hyperparameter that requires calibration for your domain. Set it too low, you get micro-chunks. Too high, you get enormous chunks that blend multiple concepts.

Hierarchical Chunking
What it does: Creates chunks at multiple granularities simultaneously, preserving parent-child relationships. A common implementation: store paragraph-level chunks for retrieval (precise embedding) but attach the full section as the "parent" to return for context.
PARENT (stored for context retrieval):
{
  "id": "doc-001-section-3.2",
  "content": "3.2 Installation Steps\n\nDownload the installer from 
  the official website. Run the installer as administrator. Accept 
  the license agreement. Choose your installation directory. Click 
  Install and wait for completion.",
  "type": "section"
}

CHILDREN (stored for embedding and search):
{
  "id": "doc-001-chunk-3.2a",
  "content": "Download the installer from the official website. 
  Run the installer as administrator.",
  "parent_id": "doc-001-section-3.2",
  "type": "chunk"
}
{
  "id": "doc-001-chunk-3.2b", 
  "content": "Accept the license agreement. Choose your installation 
  directory. Click Install and wait for completion.",
  "parent_id": "doc-001-section-3.2",
  "type": "chunk"
}
At query time:
User asks: "How do I install the software?"
Vector search finds doc-001-chunk-3.2a (small chunk, precise embedding match)
System fetches doc-001-section-3.2 (the parent) to send to the LLM
LLM gets full section context, not just the sentence fragment
Why this is the "robust default" for long structured documents: It solves the fundamental tension — small chunks embed precisely but lack context; large chunks carry context but embed diffusely. You get the precision of small embeddings and the context of large passages. The tradeoff is metadata complexity: you must store and maintain the parent-child relationships.

Structure-Aware Chunking
What it does: Parses the document's actual structure — headings, tables, code blocks, lists — and respects those boundaries explicitly, often

Unstarred

Dunk (exclude from study)

Module 4: Vector Store Design

Review Questions

Todos

Sensemaker