Backend & AI Engineer Guide: Hybrid Search with Elasticsearch, BM25 & k-NN Vectors

When users search an e-commerce catalog, enterprise document repository, or SaaS knowledge base, slow ILIKE '%query%' SQL queries fail on three fronts: they cannot handle typos, they ignore field-level boosting (e.g., matching a product title vs. a footer note), and they miss conceptual synonyms. Conversely, pure vector search struggles with exact SKU numbers and part codes. As a Senior Backend Engineer and AI Application Engineer, I architect sub-20ms hybrid search engines using Elasticsearch, OpenSearch, BM25 lexical scoring, and HNSW k-NN vector retrieval.

1. Hybrid BM25 + Dense Vector k-NN Querying in Elasticsearch

By combining a boosted multi-match BM25 query (handling exact SKU/code matches, prefix autocomplete, and fuzzy edit distance) with an approximate knn vector search in a single Elasticsearch request using Reciprocal Rank Fusion (RRF), users get both pinpoint lexical precision and semantic recall:

def build_hybrid_product_search_body(
    query_text: str,
    query_vector: list[float],
    tenant_id: str,
    category_slug: str | None = None,
) -> dict:
    filters = [{"term": {"tenant_id": tenant_id}}, {"term": {"is_active": True}}]
    if category_slug:
        filters.append({"term": {"category_slug": category_slug}})

    return {
        "size": 20,
        "query": {
            "bool": {
                "filter": filters,
                "must": [
                    {
                        "multi_match": {
                            "query": query_text,
                            "fields": ["sku_code^6", "title^3", "technical_specs^1.5", "description"],
                            "type": "best_fields",
                            "fuzziness": "AUTO",
                        }
                    }
                ],
            }
        },
        "knn": {
            "field": "semantic_embedding",
            "query_vector": query_vector,
            "k": 20,
            "num_candidates": 100,
            "filter": {"bool": {"filter": filters}},
            "boost": 0.4,
        },
    }

2. Zero-Downtime Schema Migrations via Index Aliases

In Elasticsearch and OpenSearch, you cannot change the analyzer or vector dimension of an existing field in-place without reindexing. If your backend application hardcodes the index name products, reindexing causes search downtime. I implement atomic alias swapping: the application always queries the read alias products_read while pointing writes to products_write. When mappings change, a background worker builds products_v2026_05, backfills documents via bulk streaming, and executes an atomic _aliases swap in zero milliseconds of downtime.

3. Hire a Senior Backend & AI Engineer for Search Relevance

Need to optimize search relevance, reduce Elasticsearch cluster memory costs, or deploy hybrid vector search? Explore my Backend Engineer Pillar Guide, read my AI Application Engineer Guide, or book a 1-Day Search Architecture Sprint.


About the Author — Hire Aldian Fazrihady (Cloud Engineer, GCP Professional Cloud Architect, GCP Professional Data Engineer, Backend, AI & Full Stack Engineer)

Aldian Fazrihady is a Top-Rated Senior Cloud Engineer, GCP Professional Cloud Architect (Google Cloud Certified), GCP Professional Data Engineer (Google Cloud Certified), Backend Engineer (Python, Django, FastAPI, Go, PostgreSQL), AI Application Engineer (LLMs, RAG, MCP, Google ADK, LangGraph), and Full Stack Engineer (React, Next.js, TypeScript) with 15+ years of production software engineering experience.

Hire Aldian Fazrihady on Upwork | Book 1-Day Programming Service ($100 USD / Rp 1.000.000) | View Live AI Portfolio | Send a Direct Message

Related Engineering Articles

Engineering Role Pillar Guides & Category Hubs