Autocomplete & Suggestions

What it is

Autocomplete (also called search-as-you-type or type-ahead) is the feature that displays a dropdown of suggested completions, popular queries, or matching documents as a user types each character. It dramatically reduces the number of keystrokes needed to reach a successful search, surfaces content users didn't know existed, and corrects typos before an empty-results page is shown. The core technical challenge is delivering relevant suggestions in under 100ms — faster than a user can type the next character.

There are two primary architectural approaches. Index-time approaches pre-expand every prefix into tokens at write time (using an edge n-gram tokenizer or a dedicated completion field type), so queries are simple lookups. Query-time approaches run prefix queries, fuzzy queries, or phrase-suggest queries at read time against the normal index. Index-time approaches are faster at query time but inflate index size; query-time approaches require no special index structure but consume more CPU per request. Most production systems combine both: a completion suggester for the happy path and a fuzzy fallback for typos.

Why it exists

Users abandon searches when they can't formulate the right query. Studies show that sites with working autocomplete see higher search engagement, lower zero-result rates, and higher conversion. Autocomplete also solves the "vocabulary mismatch" problem: a user typing "footwear" may not realise the catalogue uses the term "shoes". By surfacing matching product names and categories as they type, autocomplete guides users to the vocabulary the system understands.

At scale, the latency requirement (sub-100ms, end-to-end) makes standard BM25 full-text queries insufficient: they do too much work per character change. The completion suggester and edge n-gram approaches trade index storage for query speed, using purpose-built FST (Finite State Transducer) data structures that can traverse millions of completions in microseconds — a data structure that cannot be replicated in a general-purpose database.

When to use

• Any search interface where users type free-form queries and expect instant feedback while typing.
• E-commerce where suggesting matching product names, categories, and brands reduces abandonment.
• Address and location search where partial postal codes or street names need fast completion.
• Mobile interfaces where the keyboard takes up half the screen and users want to type as little as possible.
• Enterprise search portals where surfacing document titles as suggestions shortcuts navigation.

When not to use

• Highly sensitive data fields: Autocomplete over PII (user names, email addresses) can inadvertently expose personal data to other users — ensure strict access controls on the completion endpoint.
• Very dynamic catalogues: If products are added and removed at high frequency, the index-time completion approach requires frequent rebuilds; a simpler prefix query on a cache may suffice.
• Extremely small datasets: A simple in-memory prefix tree in the application layer is simpler than a search cluster for fewer than ~10,000 entities.
• Command-line interfaces: Terminal tab-completion is a different problem domain with different trade-offs.

Typical architecture

User types: "wir" (wants "wireless headphones")

── Approach 1: Edge N-Gram (index-time) ──────────────────
Index mapping:
  title: {
    type: text,
    analyzer: "autocomplete_index",   // generates: w, wi, wir, wire...
    search_analyzer: "standard"       // searches exact prefix
  }

Query: { match: { "title": "wir" } }
→ Matches docs containing token "wir" (edge n-gram of "wireless")
→ Very fast, larger index size

── Approach 2: Completion Suggester ─────────────────────
Index mapping:
  suggest: {
    type: completion,
    analyzer: standard,
    contexts: [{ name: "category", type: "category" }]
  }

Query: _search/suggest
  { "product_suggest": { "prefix": "wir",
      "completion": { "field": "suggest", "size": 5,
        "fuzzy": { "fuzziness": 1 } } } }
→ FST traversal, very fast, supports fuzzy matching

── Approach 3: search_as_you_type field ─────────────────
Mapping:
  title: { type: search_as_you_type }
  (auto-creates: title._2gram, title._3gram, title._index_prefix)

Query: { multi_match: { query: "wir",
    type: bool_prefix,
    fields: ["title","title._2gram","title._3gram"] } }

Pros and cons

Implementation notes

For most cases, the search_as_you_type field type (Elasticsearch 7.2+) offers the best balance: it automatically creates shingle sub-fields for 2-gram and 3-gram tokens alongside a prefix index, and the bool_prefix multi-match query type efficiently combines these. This approach keeps memory in the JVM heap lower than the completion suggester while delivering latency comparable to edge n-gram. Reserve the completion type for curated suggestion lists (e.g., top 1,000 most-searched queries) where you need context filtering and weight-based ordering.

For popularity-based ranking, store a weight field in the completion document equal to the search frequency (updated by a daily batch job from query logs). Combine this with context filtering to serve different suggestions to different user segments or on different pages. Implement a debounce of 150–300ms on the client side to avoid firing a request per keystroke — this halves server load with negligible UX impact. Always cache autocomplete responses in a shared cache (Redis) with a short TTL (5–30 seconds) since the suggestion list changes slowly relative to query volume.

Common failure modes

• Completion suggester heap exhaustion: Loading millions of completion terms into heap with high-weight contexts causes OutOfMemoryError on Elasticsearch data nodes.
• Leaking PII through suggestions: Indexing user-generated text (names, emails) into a shared completion index allows any user to enumerate other users' data character by character.
• No debounce on the client: Without debouncing, a user typing "laptop" fires 6 requests; at scale this creates a 6× amplification of QPS compared to final searches.
• Stale suggestions after catalogue update: Completion suggesters are not updated by document updates; a full index rebuild is required to remove deleted products from suggestions.
• Language mismatch: Using a language-specific analyzer (e.g., English stemmer) on the autocomplete field corrupts prefix matching — use standard/simple analyzer for the autocomplete field.

Decision checklist

• Have you chosen between edge n-gram, completion suggester, and search_as_you_type based on cardinality and latency needs?
• Is autocomplete data isolated from PII or sensitive user-generated content?
• Have you implemented client-side debouncing (150–300ms)?
• Is there a caching layer (Redis) in front of the autocomplete endpoint?
• Have you defined how suggestions are ranked (popularity, revenue, recency)?
• Is there a refresh/rebuild strategy when the catalogue changes significantly?

Example use cases

• E-commerce type-ahead: As a user types "blue", suggestions appear for "Blue Jeans", "Bluetooth Headphones", "Blue Light Glasses" weighted by sales volume.
• Address autocomplete: Postal address lookup for checkout — edge n-gram over structured address data returns matching streets and postcodes within 50ms.
• Internal wiki search: Page title suggestions using completion suggester with context filtering by team namespace, so engineering sees engineering pages first.

Related patterns

• Full-Text Search — The full query executed once the user submits the autocomplete suggestion.
• Faceted Search — Often combined with autocomplete, where suggestions also pre-filter by category.
• Search Relevance Tuning — Popularity-weighted suggestions require the same feedback loop as relevance tuning.

Further reading

• Elasticsearch Suggesters Reference
• Elasticsearch search_as_you_type field type
• Lucene Completion Postings Format (FST internals)