Labs — Solutions

This file contains reference solutions for every lab across all chapters (01–08). Solutions are written for the standard course environment: single-node Docker cluster on localhost:9200, with the shop-orders-demo index loaded. Expected outputs reflect a fresh cluster; counts will differ if you have run other exercises in the same session.

Chapter 01 — Introduction and architecture

Lab 1 — Explore the cluster

GET /
GET /_cluster/health
GET /_cat/nodes?v
GET /_cat/indices?v
GET /_cat/shards?v

Expected answers:

Cluster name: docker-cluster (set in docker-compose.yml via discovery.type=single-node defaults).
Cluster health status: yellow — because there is only one node and replica shards cannot be placed on the same node as their primary.
Node count and roles: 1 node named es01. The role column shows cdfhilmrstw (or similar) — the node holds all roles (master, data, ingest, etc.).
Unassigned shards: Yes, the replica shards for any index with number_of_replicas ≥ 1 will show as UNASSIGNED. On system indices (.kibana_*) you will see unassigned replicas.

Lab 2 — Index a document and observe near real-time refresh

PUT /nrt-demo
{
  "settings": {
    "index": {
      "refresh_interval": "30s",
      "number_of_replicas": 0
    }
  }
}

POST /nrt-demo/_doc/1
{
  "product": "Wireless Mouse Logitech",
  "price": 21
}

Step 3 — Search immediately (0 hits expected):

GET /nrt-demo/_search
{
  "query": { "match_all": {} }
}

Expected: "hits": { "total": { "value": 0 } } — the document is in the in-memory buffer but not yet visible.

Step 4 — Trigger refresh:

POST /nrt-demo/_refresh

Step 5 — Search again (1 hit expected):

GET /nrt-demo/_search
{
  "query": { "match_all": {} }
}

Expected: "total": { "value": 1 }, with the Wireless Mouse document in hits.hits.

Step 6 — Cleanup:

DELETE /nrt-demo

Key insight: The refresh interval of 30 s means newly indexed documents are invisible until _refresh is called (or the interval elapses). In tests, always call POST /<index>/_refresh after indexing before asserting on search results.

Lab 3 — Understand shard and replica distribution

PUT /shard-demo
{
  "settings": {
    "number_of_shards": 2,
    "number_of_replicas": 2
  }
}

GET /_cluster/health

Expected: "status": "yellow" — primaries are allocated, replicas are not (only one node available).

GET /_cat/shards/shard-demo?v

Expected output (abbreviated):

index       shard prirep state      node
shard-demo  0     p      STARTED    es01
shard-demo  0     r      UNASSIGNED
shard-demo  0     r      UNASSIGNED
shard-demo  1     p      STARTED    es01
shard-demo  1     r      UNASSIGNED
shard-demo  1     r      UNASSIGNED

Primary shards (p) are STARTED, replica shards (r) are UNASSIGNED.

PUT /shard-demo/_settings
{
  "index": {
    "number_of_replicas": 0
  }
}

GET /_cluster/health

Expected: "status": "green" — with 0 replicas there is nothing to assign, so all shards are accounted for.

DELETE /shard-demo

Lab 4 — Preview the course dataset

GET /_cat/indices/shop-orders-demo?v

Expected: the index is listed with a document count (e.g., ~5,000 docs), a store size, and health green (if replicas = 0) or yellow.

GET /shop-orders-demo/_mapping

Note the nested items field, delivery.location as geo_point, and the czech analyzer on notes.

GET /shop-orders-demo/_search
{
  "size": 1,
  "query": { "match_all": {} }
}

Inspect _source on the returned document — you should see nested customer, nested items, delivery, total_amount, channel, etc.

Chapter 02 — Data model and CRUD

Lab 2-A — Index, read, and verify your first document

Step 1 — Index with PUT:

PUT /my-first-orders/_doc/LAB-001
{
  "order_id": "LAB-001",
  "customer_name": "Test User",
  "total_amount": 299.00,
  "order_status": "pending",
  "created_at": "2026-05-30T10:00:00Z"
}

Expected response: "result": "created", "_version": 1.

Step 2 — Verify:

GET /my-first-orders/_doc/LAB-001

Expected: "found": true, _source contains all five fields.

Step 3 — Try PUT _create on existing ID:

PUT /my-first-orders/_create/LAB-001
{
  "order_id": "LAB-001",
  "customer_name": "Different User",
  "total_amount": 999.00
}

Expected: 409 Conflict — "type": "version_conflict_engine_exception". The document is unchanged.

Step 4 — HEAD and _source:

HEAD /my-first-orders/_doc/LAB-001

Expected: 200 OK, no body.

GET /my-first-orders/_source/LAB-001

Expected: the raw _source JSON without the metadata envelope (_index, _id, _version are absent).

Lab 2-B — Partial update and scripted update

Step 1 — Index:

PUT /my-first-orders/_doc/LAB-002
{
  "order_id": "LAB-002",
  "order_status": "processing",
  "rating": 3.5,
  "customer_name": "Lab User"
}

Step 2 — Partial update (only order_status):

POST /my-first-orders/_update/LAB-002
{
  "doc": {
    "order_status": "delivered"
  }
}

Expected: "result": "updated", "_version": 2.

Step 3 — Verify rating unchanged:

GET /my-first-orders/_source/LAB-002

Expected: "order_status": "delivered" AND "rating": 3.5 — the partial update did not overwrite unmentioned fields.

Step 4 — Scripted update to increment rating:

POST /my-first-orders/_update/LAB-002
{
  "script": {
    "source": "ctx._source.rating += params.increment",
    "params": { "increment": 1.0 }
  }
}

Expected: "result": "updated", "_version": 3. GET /my-first-orders/_source/LAB-002 should show "rating": 4.5.

Step 5 — Noop check:

Run the same _update with "doc": {"order_status": "delivered"} again (no change). Expected: "result": "noop", _version remains 3.

Lab 2-C — Bulk ingest and error handling

Step 1 — First bulk run:

POST /my-first-orders/_bulk
{ "index": { "_id": "LAB-010" } }
{ "order_id": "LAB-010", "order_status": "pending", "total_amount": 100.00 }
{ "create": { "_id": "LAB-011" } }
{ "order_id": "LAB-011", "order_status": "processing", "total_amount": 200.00 }
{ "index": { "_id": "LAB-012" } }
{ "order_id": "LAB-012", "order_status": "delivered", "total_amount": 300.00 }

Expected: "errors": false, all three items show "result": "created" with status 201.

Second run of the same body:

Expected: "errors": true. - index for LAB-010 → "result": "updated", status 200 (index is create-or-replace, so it succeeds). - create for LAB-011 → 409, "type": "version_conflict_engine_exception" — create fails if document already exists. - index for LAB-012 → "result": "updated", status 200.

Step 2 — Find the failure: Check items[1].create.status — it will be 409 with an error key.

Step 3 — mget:

GET /_mget
{
  "docs": [
    { "_index": "my-first-orders", "_id": "LAB-010" },
    { "_index": "my-first-orders", "_id": "LAB-011" },
    { "_index": "my-first-orders", "_id": "LAB-012" }
  ]
}

Expected: all three documents returned, each with "found": true.

Lab 2-D — Mapping inspection and type pitfall

Step 1 — Index string into new index:

POST /my-type-test/_doc
{
  "price": "not-a-number"
}

Step 2 — Check inferred mapping:

GET /my-type-test/_mapping

Expected: "price": { "type": "text", "fields": { "keyword": { "type": "keyword" } } } — Elasticsearch guessed text from the string value.

Step 3 — Index a number:

POST /my-type-test/_doc
{
  "price": 99.90
}

Expected: 400 Bad Request — "mapper_parsing_exception" — 99.90 cannot be indexed into a text field.

Step 4 — Explicit mapping:

DELETE /my-type-test

PUT /my-type-test
{
  "mappings": {
    "properties": {
      "price": { "type": "double" }
    }
  }
}

Now indexing "price": "not-a-number" will fail immediately with a type error (cannot parse "not-a-number" as double), and indexing 99.90 succeeds cleanly.

DELETE /my-type-test
DELETE /my-first-orders

Chapter 03 — Basic search

Lab 1 — Explore the response envelope

GET /shop-orders-demo/_search
{
  "query": { "match_all": {} }
}

Note: took (ms), hits.total.value (total matching docs — will be the full index size), hits.total.relation ("eq" or "gte"), hits.max_score (1.0 for match_all).

GET /shop-orders-demo/_search
{
  "size": 1,
  "query": { "match_all": {} }
}

hits.total.value is unchanged — still the full count. hits.hits has exactly 1 document.

GET /shop-orders-demo/_search
{
  "size": 1,
  "_source": ["order_id", "total_amount", "order_status"],
  "query": { "match_all": {} }
}

_source contains only the three requested fields.

Lab 2 — `match` vs `term` in practice

Term on keyword (exact, case-sensitive):

GET /shop-orders-demo/_search
{
  "query": { "term": { "order_status": "delivered" } }
}

Expected: returns matching documents (non-zero count).

GET /shop-orders-demo/_search
{
  "query": { "term": { "order_status": "Delivered" } }
}

Expected: 0 results — keyword fields are case-sensitive.

Match on text field:

GET /shop-orders-demo/_search
{
  "query": { "match": { "customer.name": "Jana" } },
  "size": 3
}

Expected: returns documents where customer.name contains the token jana (case-insensitive due to analysis).

Term on text field (wrong):

GET /shop-orders-demo/_search
{
  "query": { "term": { "customer.name": "Jana Nováková" } }
}

Expected: 0 results — customer.name is a text field; the stored tokens are lowercase single words, not the full phrase.

Term on keyword sub-field (correct):

GET /shop-orders-demo/_search
{
  "query": { "term": { "customer.name.keyword": "Jana Nováková" } }
}

Expected: returns exact matches for that full name string.

Lab 3 — Build a `bool` query step by step

Final complete query:

GET /shop-orders-demo/_search
{
  "query": {
    "bool": {
      "filter": [
        { "term":  { "delivery.country_code": "DE" } },
        { "range": { "total_amount": { "gt": 2000 } } }
      ],
      "must": [
        { "match": { "notes": "package" } }
      ],
      "must_not": [
        { "term": { "order_status": "cancelled" } }
      ],
      "should": [
        { "term": { "shipping_method": "express" } }
      ]
    }
  },
  "size": 10
}

What to observe: - Each additional filter clause reduces the result count. - Adding the must (match) clause introduces _score — documents are now ranked by relevance of “package” in notes. - The should (express shipping) does not change the result count but reorders results: express orders float to the top.

Lab 4 — Pagination and sorting

GET /shop-orders-demo/_search
{
  "size": 5,
  "sort": [{ "total_amount": "desc" }],
  "_source": ["order_id", "total_amount", "customer.name", "delivery.city"],
  "query": { "term": { "order_status": "delivered" } }
}

Page 2:

GET /shop-orders-demo/_search
{
  "from": 5,
  "size": 5,
  "sort": [{ "total_amount": "desc" }],
  "_source": ["order_id", "total_amount"],
  "query": { "term": { "order_status": "delivered" } }
}

The first document on page 2 has a total_amount ≤ the last document on page 1.

Deep pagination (note the latency):

GET /shop-orders-demo/_search
{
  "from": 500,
  "size": 5,
  "sort": [{ "total_amount": "desc" }],
  "query": { "term": { "order_status": "delivered" } }
}

This works but Elasticsearch must collect 505 documents per shard and discard 500. The cost grows linearly with from.

Chapter 04 — Mapping and analyzers

Lab 1 — Inspect and compare mapping behavior

GET /shop-orders-demo/_mapping

keyword fields: order_id, channel, order_status, payment_method, delivery.country_code, delivery.city, customer.id, customer.loyalty_tier, items.category, tags, categories
text fields with analyzers: notes (czech + standard multi-fields), customer.name (czech), items.name (czech)
Multi-fields: notes.std, notes.keyword, customer.name.keyword, items.name.keyword

Attempt to aggregate on text (fails):

GET /shop-orders-demo/_search
{
  "size": 0,
  "aggs": {
    "by_customer_name": {
      "terms": { "field": "customer.name" }
    }
  }
}

Expected: error — "Fielddata is disabled on text fields by default".

Aggregate on .keyword (succeeds):

GET /shop-orders-demo/_search
{
  "size": 0,
  "aggs": {
    "by_customer_name": {
      "terms": { "field": "customer.name.keyword", "size": 5 }
    }
  }
}

Why .keyword exists: The text field stores analyzed tokens for full-text search. The .keyword sub-field stores the original string as-is using doc_values, enabling sorting and aggregation without loading everything into heap.

Lab 2 — Explore analysis with `_analyze`

Standard:

POST /_analyze
{
  "analyzer": "standard",
  "text": "Wireless Headphones, 2-pack!"
}

Tokens: ["wireless", "headphones", "2", "pack"]

Whitespace:

POST /_analyze
{
  "analyzer": "whitespace",
  "text": "Wireless Headphones, 2-pack!"
}

Tokens: ["Wireless", "Headphones,", "2-pack!"] — case preserved, punctuation attached.

English:

POST /_analyze
{
  "analyzer": "english",
  "text": "Wireless Headphones, 2-pack!"
}

Tokens: ["wireless", "headphon", "2", "pack"] — headphones is stemmed to headphon.

Czech vs standard on same text:

POST /shop-orders-demo/_analyze
{
  "field": "notes",
  "text": "Bezdrátová sluchátka nebyla součástí balení."
}

Czech analyzer: fewer tokens, stems applied, stopwords removed.

POST /shop-orders-demo/_analyze
{
  "field": "notes.std",
  "text": "Bezdrátová sluchátka nebyla součástí balení."
}

Standard analyzer: more tokens, no Czech stemming, stopwords kept.

Keyword (single token):

POST /_analyze
{
  "analyzer": "keyword",
  "text": "Order-2024-XYZ"
}

One token: ["Order-2024-XYZ"] — the entire string, unchanged.

Lab 3 — Dynamic vs strict mapping

Dynamic mapping type conflict:

POST /test-dynamic/_doc
{
  "delivery_note": "leave at door"
}

GET /test-dynamic/_mapping

delivery_note is mapped as text + keyword. Now:

POST /test-dynamic/_doc
{
  "delivery_note": 42
}

Expected: 400 Bad Request — "mapper_parsing_exception": 42 cannot be stored in a text field.

Strict mapping — rejecting unknown fields:

PUT /test-strict
{
  "mappings": {
    "dynamic": "strict",
    "properties": {
      "order_id": { "type": "keyword" }
    }
  }
}

POST /test-strict/_doc
{
  "order_id": "ORD-001",
  "unexpected_field": "value"
}

Expected: 400 Bad Request — "strict_dynamic_mapping_exception": unexpected_field is not in the mapping.

Fix — add the field:

PUT /test-strict/_mapping
{
  "properties": {
    "unexpected_field": { "type": "keyword" }
  }
}

POST /test-strict/_doc
{
  "order_id": "ORD-001",
  "unexpected_field": "value"
}

Expected: "result": "created" — the field is now in the mapping.

DELETE /test-dynamic
DELETE /test-strict

Lab 4 — Multi-field design

Text-only field (aggregation fails):

PUT /multifield-test-v1
{
  "settings": { "number_of_replicas": 0 },
  "mappings": {
    "properties": {
      "customer_name": { "type": "text" }
    }
  }
}

POST /multifield-test-v1/_bulk
{ "index": {} }
{ "customer_name": "Jana Nováková" }
{ "index": {} }
{ "customer_name": "Petr Svoboda" }
{ "index": {} }
{ "customer_name": "Jana Horáková" }
{ "index": {} }
{ "customer_name": "Martin Novák" }
{ "index": {} }
{ "customer_name": "Jana Nováková" }

GET /multifield-test-v1/_search
{
  "size": 0,
  "aggs": { "by_name": { "terms": { "field": "customer_name" } } }
}

Expected: error — "Fielddata is disabled on text fields by default".

Multi-field version (both work):

PUT /multifield-test-v2
{
  "settings": { "number_of_replicas": 0 },
  "mappings": {
    "properties": {
      "customer_name": {
        "type": "text",
        "fields": {
          "keyword": { "type": "keyword" }
        }
      }
    }
  }
}

POST /multifield-test-v2/_bulk
{ "index": {} }
{ "customer_name": "Jana Nováková" }
{ "index": {} }
{ "customer_name": "Petr Svoboda" }
{ "index": {} }
{ "customer_name": "Jana Horáková" }
{ "index": {} }
{ "customer_name": "Martin Novák" }
{ "index": {} }
{ "customer_name": "Jana Nováková" }

Full-text match (on text field):

GET /multifield-test-v2/_search
{
  "query": { "match": { "customer_name": "Jana" } }
}

Returns all 3 “Jana” documents.

Exact aggregation (on .keyword sub-field):

GET /multifield-test-v2/_search
{
  "size": 0,
  "aggs": {
    "by_name": {
      "terms": { "field": "customer_name.keyword", "size": 10 }
    }
  }
}

Returns: Jana Nováková (count 2), Jana Horáková (count 1), Petr Svoboda (count 1), Martin Novák (count 1).

DELETE /multifield-test-v1
DELETE /multifield-test-v2

Chapter 05a — Custom analyzers

Lab — Custom analyzer with synonyms and diacritics

Step 1 — Create the index:

PUT /notes-analyzer-lab
{
  "settings": {
    "analysis": {
      "char_filter": {
        "html_strip_filter": { "type": "html_strip" }
      },
      "filter": {
        "ascii_fold": {
          "type": "asciifolding",
          "preserve_original": true
        },
        "order_synonyms": {
          "type": "synonym",
          "synonyms": [
            "refund, return => return",
            "cancelled, canceled => cancelled",
            "broken, damaged => defective"
          ]
        }
      },
      "analyzer": {
        "notes_index_analyzer": {
          "type": "custom",
          "char_filter": ["html_strip_filter"],
          "tokenizer": "standard",
          "filter": ["lowercase", "ascii_fold", "order_synonyms"]
        },
        "notes_search_analyzer": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": ["lowercase", "ascii_fold", "order_synonyms"]
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "notes": {
        "type": "text",
        "analyzer": "notes_index_analyzer",
        "search_analyzer": "notes_search_analyzer"
      }
    }
  }
}

Step 2 — Verify HTML stripping and synonym expansion:

POST /notes-analyzer-lab/_analyze
{
  "analyzer": "notes_index_analyzer",
  "text": "<b>The item was broken</b> and customer requested a refund."
}

Expected tokens (order may vary): broken → defective, refund → return, HTML tags stripped, the, item, was, and, customer, requested, a lowercased.

Answers: 1. Yes —  and  are stripped. 2. Yes — broken is normalized to defective. 3. Yes — refund is normalized to return.

Step 3 — explain: true:

POST /notes-analyzer-lab/_analyze
{
  "analyzer": "notes_index_analyzer",
  "explain": true,
  "text": "damaged packaging"
}

The explain output shows each token filter in the chain and what transformation it applied. For damaged, you will see order_synonyms converting it to defective.

Step 4 — Index documents:

POST /notes-analyzer-lab/_doc/1
{
  "notes": "Customer received broken item and wants a refund."
}

POST /notes-analyzer-lab/_doc/2
{
  "notes": "Item was defective. Issued a return."
}

Step 5 — Search with target synonym term:

GET /notes-analyzer-lab/_search
{
  "query": {
    "match": { "notes": "defective" }
  }
}

Expected: both documents match. - Doc 1 matches because broken was indexed as defective (synonym expansion at index time). - Doc 2 matches because it explicitly contains defective.

Step 6 — Czech diacritics:

POST /notes-analyzer-lab/_doc/3
{
  "notes": "Zákazník vrátil poškozené zboží."
}

GET /notes-analyzer-lab/_search
{
  "query": {
    "match": { "notes": "poskozene zbozi" }
  }
}

Expected: Doc 3 matches. The asciifolding filter (with preserve_original: true) indexes both poškozené and poskozene as tokens — so searching without diacritics finds the Czech original.

Cleanup:

DELETE /notes-analyzer-lab

Chapter 05b — Modeling relationships

Lab Part A — nested query with inner_hits and sorting

Step 1 — Naïve flat query (false positives expected):

GET /shop-orders-demo/_search
{
  "query": {
    "bool": {
      "must": [
        { "term":  { "items.category": "Elektronika" } },
        { "range": { "items.unit_price": { "gt": 500 } } }
      ]
    }
  },
  "_source": ["order_id"],
  "size": 5
}

Note the result count — this may return orders where one item has Elektronika category but a different item exceeds 500 in price (false positive).

Step 2 — Correct nested query:

GET /shop-orders-demo/_search
{
  "query": {
    "nested": {
      "path": "items",
      "query": {
        "bool": {
          "must": [
            { "term":  { "items.category": "Elektronika" } },
            { "range": { "items.unit_price": { "gt": 500 } } }
          ]
        }
      },
      "inner_hits": {
        "name": "matching_items",
        "size": 3,
        "_source": ["items.name", "items.unit_price", "items.category"]
      }
    }
  },
  "_source": ["order_id", "customer.name", "order_status"],
  "size": 5
}

Answers: 1. The nested query returns fewer or equal results — it eliminates the false positives where Elektronika and price > 500 belonged to different items. 2. inner_hits.matching_items shows only the items that satisfied both conditions within the same object.

Step 3 — Sorted by max unit_price of matching items:

GET /shop-orders-demo/_search
{
  "query": {
    "nested": {
      "path": "items",
      "query": {
        "bool": {
          "must": [
            { "term":  { "items.category": "Elektronika" } },
            { "range": { "items.unit_price": { "gt": 500 } } }
          ]
        }
      },
      "inner_hits": {
        "name": "matching_items",
        "size": 3,
        "_source": ["items.name", "items.unit_price"]
      }
    }
  },
  "sort": [
    {
      "items.unit_price": {
        "order": "desc",
        "mode": "max",
        "nested": {
          "path": "items",
          "filter": { "term": { "items.category": "Elektronika" } }
        }
      }
    }
  ],
  "_source": ["order_id", "customer.name"],
  "size": 5
}

Orders with the most expensive Elektronika item appear first.

Lab Part B — join field (parent-child)

Setup:

PUT /orders-with-events
{
  "settings": { "number_of_replicas": 0 },
  "mappings": {
    "properties": {
      "join_field": {
        "type": "join",
        "relations": { "order": "event" }
      },
      "order_id":   { "type": "keyword" },
      "event_type": { "type": "keyword" },
      "event_at":   { "type": "date" }
    }
  }
}

PUT /orders-with-events/_doc/order-1?routing=order-1
{ "order_id": "order-1", "join_field": "order" }

PUT /orders-with-events/_doc/order-2?routing=order-2
{ "order_id": "order-2", "join_field": "order" }

PUT /orders-with-events/_doc/event-1?routing=order-1
{
  "order_id": "order-1",
  "event_type": "payment_received",
  "event_at": "2024-03-15T08:00:00Z",
  "join_field": { "name": "event", "parent": "order-1" }
}

PUT /orders-with-events/_doc/event-2?routing=order-1
{
  "order_id": "order-1",
  "event_type": "shipped",
  "event_at": "2024-03-15T14:00:00Z",
  "join_field": { "name": "event", "parent": "order-1" }
}

PUT /orders-with-events/_doc/event-3?routing=order-2
{
  "order_id": "order-2",
  "event_type": "payment_received",
  "event_at": "2024-03-16T09:00:00Z",
  "join_field": { "name": "event", "parent": "order-2" }
}

Find orders with a “shipped” event:

GET /orders-with-events/_search
{
  "query": {
    "has_child": {
      "type": "event",
      "query": { "term": { "event_type": "shipped" } },
      "inner_hits": {}
    }
  }
}

Expected: returns order-1 only (it has a shipped event). inner_hits shows event-2.

Update child without touching parent:

POST /orders-with-events/_update/event-2?routing=order-1
{
  "doc": { "event_type": "delivered" }
}

GET /orders-with-events/_doc/order-1
GET /orders-with-events/_doc/event-2?routing=order-1

order-1’s _version is unchanged. event-2’s version incremented to 2 and event_type is now delivered.

Cleanup:

DELETE /orders-with-events

Chapter 05c — Highlighting and autocomplete

Lab Part A — Highlighting

Step 1 — Default highlight:

GET /shop-orders-demo/_search
{
  "query": { "match": { "notes": "payment" } },
  "highlight": { "fields": { "notes": {} } },
  "size": 3
}

Default tags: .... Default: up to 5 fragments of ~100 characters each.

Step 2 — Custom tags, 1 fragment:

GET /shop-orders-demo/_search
{
  "query": { "match": { "notes": "payment" } },
  "highlight": {
    "pre_tags": ["<mark>"],
    "post_tags": ["</mark>"],
    "fields": {
      "notes": {
        "fragment_size": 120,
        "number_of_fragments": 1,
        "no_match_size": 80
      }
    }
  },
  "size": 3
}

 wraps matched terms. Documents without a matching fragment still return 80 characters from the start of the field.

Step 4 — Nested inner_hits highlighting:

GET /shop-orders-demo/_search
{
  "query": {
    "nested": {
      "path": "items",
      "query": { "match": { "items.name": "kabel" } },
      "inner_hits": {
        "highlight": {
          "fields": {
            "items.name": {
              "fragment_size": 80,
              "number_of_fragments": 1
            }
          }
        }
      }
    }
  },
  "_source": ["order_id"],
  "size": 3
}

Each inner_hit in the response contains a highlight.items.name array with the matched fragment.

Lab Part B — Completion suggester

Setup:

PUT /product-suggestions
{
  "settings": { "number_of_replicas": 0 },
  "mappings": {
    "properties": {
      "suggest": { "type": "completion", "analyzer": "simple" },
      "product_id": { "type": "keyword" },
      "name":       { "type": "text" }
    }
  }
}

Index products (bulk):

POST /product-suggestions/_bulk
{ "index": {} }
{ "product_id": "EL-001", "name": "USB-C Charging Cable", "suggest": { "input": ["USB-C Charging Cable", "USB Cable", "Charging Cable"], "weight": 95 } }
{ "index": {} }
{ "product_id": "EL-002", "name": "USB-A to USB-C Adapter", "suggest": { "input": ["USB-A to USB-C Adapter", "USB Adapter"], "weight": 60 } }
{ "index": {} }
{ "product_id": "EL-003", "name": "USB Hub 4-port", "suggest": { "input": ["USB Hub", "4-port Hub"], "weight": 40 } }
{ "index": {} }
{ "product_id": "EL-006", "name": "USB Microphone", "suggest": { "input": ["USB Microphone", "Microphone"], "weight": 50 } }
{ "index": {} }
{ "product_id": "EL-008", "name": "USB-C Docking Station", "suggest": { "input": ["USB-C Docking Station", "Docking Station", "USB Dock"], "weight": 30 } }

Prefix query for “usb”:

GET /product-suggestions/_search
{
  "suggest": {
    "product_autocomplete": {
      "prefix": "usb",
      "completion": {
        "field": "suggest",
        "size": 5,
        "skip_duplicates": true
      }
    }
  }
}

Expected order: EL-001 (weight 95), EL-002 (weight 60), EL-006 (weight 50), EL-003 (weight 40), EL-008 (weight 30) — ordered by weight descending.

Fuzzy prefix for “uab”:

GET /product-suggestions/_search
{
  "suggest": {
    "product_autocomplete": {
      "prefix": "uab",
      "completion": {
        "field": "suggest",
        "size": 5,
        "skip_duplicates": true,
        "fuzzy": {
          "fuzziness": "AUTO",
          "min_length": 4,
          "prefix_length": 1
        }
      }
    }
  }
}

Expected: USB products appear despite the one-character typo (u matches, a→s is one edit, b matches).

Cleanup:

DELETE /product-suggestions
DELETE /product-sat

Chapter 05d — Custom relevance scoring

Lab — `function_score` and `script_score`

Step 1 — function_score with boost_mode: sum:

GET /shop-orders-demo/_search
{
  "query": {
    "function_score": {
      "query": { "match": { "notes": "delayed" } },
      "functions": [
        {
          "field_value_factor": {
            "field": "rating",
            "factor": 2.0,
            "modifier": "log1p",
            "missing": 1
          }
        },
        {
          "gauss": {
            "created_at": {
              "origin": "now",
              "scale":  "14d",
              "offset": "2d",
              "decay":  0.5
            }
          }
        },
        {
          "filter": { "term": { "tags": "priority" } },
          "weight": 5.0
        }
      ],
      "score_mode": "sum",
      "boost_mode": "sum"
    }
  },
  "size": 5
}

Note the _id of the top result.

Step 2 — boost_mode: multiply:

Change "boost_mode": "sum" to "boost_mode": "multiply". With multiply, the function scores are multiplied against the base BM25 score — a document with a low BM25 score (weak text match) will be penalized more heavily even if it has a high rating. With sum, the business signals add to relevance rather than amplify/suppress it.

**Step 3 — _explain for function_score:**

GET /shop-orders-demo/_explain/<_id-from-step-1>
{
  "query": {
    "function_score": {
      "query": { "match": { "notes": "delayed" } },
      "functions": [
        { "field_value_factor": { "field": "rating", "factor": 2.0, "modifier": "log1p", "missing": 1 } },
        { "gauss": { "created_at": { "origin": "now", "scale": "14d", "offset": "2d", "decay": 0.5 } } },
        { "filter": { "term": { "tags": "priority" } }, "weight": 5.0 }
      ],
      "score_mode": "sum",
      "boost_mode": "sum"
    }
  }
}

The _explain response shows: - The base BM25 _score from the match clause. - The output of each function (field_value_factor value, gauss decay value, weight if priority matched). - The combined function score (sum of function outputs). - The final score (base + combined, because boost_mode: sum).

Step 4 — Equivalent script_score:

GET /shop-orders-demo/_search
{
  "query": {
    "script_score": {
      "query": { "match": { "notes": "delayed" } },
      "script": {
        "source": """
          double ratingBoost = doc['rating'].size() > 0
            ? Math.log1p(doc['rating'].value * params.rating_factor)
            : Math.log1p(params.default_rating * params.rating_factor);

          double decayBoost = 1.0;
          if (doc['created_at'].size() > 0) {
            long ageMs = System.currentTimeMillis() - doc['created_at'].value.toInstant().toEpochMilli();
            long ageDays = ageMs / 86400000L;
            if (ageDays > 2) {
              decayBoost = Math.pow(0.5, (ageDays - 2.0) / 14.0);
            }
          }
          return Math.max(0, _score + ratingBoost + decayBoost);
        """,
        "params": { "rating_factor": 2.0, "default_rating": 1.0 }
      }
    }
  },
  "size": 5
}

The orderings between function_score and script_score are similar but not identical — the decay formulas are mathematically equivalent but floating-point arithmetic and the exact now timestamp sampled at execution time can differ slightly.

The _explain for script_score shows the script as a single "Script Score" node with the final computed value — less informative than function_score’s per-function breakdown.

Chapter 06a — Bucket and metric aggregations

Lab — Bucket and metric aggregations on shop-orders-demo

Step 1 — Orders by channel:

GET /shop-orders-demo/_search
{
  "size": 0,
  "aggs": {
    "by_channel": {
      "terms": { "field": "channel", "size": 10 }
    }
  }
}

Expected: buckets for web, mobile, api (and possibly store). web is typically the largest. sum_other_doc_count > 0 only if there are more than size distinct values; with a small cardinality field it will be 0.

Step 2 — Revenue per channel:

GET /shop-orders-demo/_search
{
  "size": 0,
  "aggs": {
    "by_channel": {
      "terms": { "field": "channel", "size": 10 },
      "aggs": {
        "avg_order_value": { "avg": { "field": "total_amount" } },
        "total_revenue":   { "sum": { "field": "total_amount" } }
      }
    }
  }
}

Each channel bucket now contains avg_order_value and total_revenue values.

Step 3 — Monthly histogram:

GET /shop-orders-demo/_search
{
  "size": 0,
  "aggs": {
    "orders_per_month": {
      "date_histogram": {
        "field": "created_at",
        "calendar_interval": "month",
        "min_doc_count": 0
      }
    }
  }
}

Every month in the data’s range has a bucket. Without min_doc_count: 0, months with 0 orders would be absent.

Step 4 — Revenue range buckets:

GET /shop-orders-demo/_search
{
  "size": 0,
  "aggs": {
    "order_size_distribution": {
      "range": {
        "field": "total_amount",
        "ranges": [
          { "to": 50,   "key": "micro" },
          { "from": 50,  "to": 200, "key": "small" },
          { "from": 200, "to": 500, "key": "medium" },
          { "from": 500,            "key": "large" }
        ]
      }
    }
  }
}

Step 5 — Cardinality and extended stats:

GET /shop-orders-demo/_search
{
  "size": 0,
  "query": { "term": { "order_status": "completed" } },
  "aggs": {
    "unique_customers": {
      "cardinality": { "field": "customer.id", "precision_threshold": 3000 }
    },
    "order_value_stats": {
      "extended_stats": { "field": "total_amount" }
    }
  }
}

unique_customers.value = approximate count of distinct customer IDs. order_value_stats.std_deviation shows spread — a large value means wide variance in order sizes.

Step 6 — Delivery percentiles:

GET /shop-orders-demo/_search
{
  "size": 0,
  "aggs": {
    "delivery_days_percentiles": {
      "percentiles": { "field": "delivery.days", "percents": [50, 75, 90, 95, 99] }
    },
    "sla_compliance": {
      "percentile_ranks": { "field": "delivery.days", "values": [3, 5, 7] }
    }
  }
}

delivery_days_percentiles.values.50 = median delivery time. sla_compliance.values.5 = percentage of orders delivered within 5 days (e.g., 72.5 means 72.5% met the 5-day SLA).

Chapter 06b — Nested and multi-level aggregations

Lab Part A — Multi-level buckets

Step 1 — Revenue by loyalty tier × country:

GET /shop-orders-demo/_search
{
  "size": 0,
  "query": { "range": { "created_at": { "gte": "now-12M" } } },
  "aggs": {
    "by_loyalty_tier": {
      "terms": { "field": "customer.loyalty_tier", "size": 10 },
      "aggs": {
        "by_country": {
          "terms": { "field": "delivery.country_code", "size": 5 },
          "aggs": {
            "total_revenue":    { "sum":         { "field": "total_amount" } },
            "avg_order_value":  { "avg":         { "field": "total_amount" } },
            "unique_customers": { "cardinality": { "field": "customer.id"  } }
          }
        }
      }
    }
  }
}

Typically gold or platinum tier generates the most revenue per order. The country with the highest average depends on the dataset. When by_country size is small (e.g., 5), sum_other_doc_count accumulates orders from countries outside the top 5 — those are not visible in the response.

Step 2 — High-value orders filter:

GET /shop-orders-demo/_search
{
  "size": 0,
  "aggs": {
    "by_country": {
      "terms": { "field": "delivery.country_code", "size": 10 },
      "aggs": {
        "avg_all": { "avg": { "field": "total_amount" } },
        "high_value_orders": {
          "filter": { "range": { "total_amount": { "gte": 500 } } },
          "aggs": {
            "avg_high_value": { "avg": { "field": "total_amount" } },
            "count":          { "value_count": { "field": "total_amount" } }
          }
        }
      }
    }
  }
}

Lab Part B — Nested and reverse_nested

Step 3 — Item category totals:

GET /shop-orders-demo/_search
{
  "size": 0,
  "aggs": {
    "into_items": {
      "nested": { "path": "items" },
      "aggs": {
        "by_category": {
          "terms": { "field": "items.category", "size": 15 },
          "aggs": {
            "total_units_sold":   { "sum": { "field": "items.quantity" } },
            "total_item_revenue": { "sum": { "field": "items.unit_price" } }
          }
        }
      }
    }
  }
}

Elektronika or Electronics typically has the highest revenue. Category with most units may differ.

Step 4 — reverse_nested for order-level metrics:

GET /shop-orders-demo/_search
{
  "size": 0,
  "aggs": {
    "into_items": {
      "nested": { "path": "items" },
      "aggs": {
        "by_category": {
          "terms": { "field": "items.category", "size": 10 },
          "aggs": {
            "total_units_sold": { "sum": { "field": "items.quantity" } },
            "back_to_order": {
              "reverse_nested": {},
              "aggs": {
                "avg_order_total":  { "avg":         { "field": "total_amount" } },
                "unique_customers": { "cardinality": { "field": "customer.id"  } }
              }
            }
          }
        }
      }
    }
  }
}

reverse_nested is needed because after descending into items, total_amount and customer.id live on the parent document. Without reverse_nested, those fields are not accessible in the nested context.

Step 5 — Flat terms on nested field:

GET /shop-orders-demo/_search
{
  "size": 0,
  "aggs": {
    "flat_attempt": {
      "terms": { "field": "items.category", "size": 10 }
    }
  }
}

This will either return an error or produce incorrect results. The items field is mapped as nested — you cannot reach into it with a regular top-level aggregation. You must use the nested aggregation context.

Chapter 06c — Pipeline aggregations

Lab Part A — Monthly sales trend

GET /shop-orders-demo/_search
{
  "size": 0,
  "aggs": {
    "monthly": {
      "date_histogram": {
        "field": "created_at",
        "calendar_interval": "month",
        "min_doc_count": 0
      },
      "aggs": {
        "revenue": { "sum": { "field": "total_amount" } },
        "mom_growth": {
          "derivative": { "buckets_path": "revenue", "gap_policy": "skip" }
        },
        "running_total": {
          "cumulative_sum": { "buckets_path": "revenue", "gap_policy": "insert_zeros" }
        },
        "smoothed_3m": {
          "moving_fn": {
            "buckets_path": "revenue",
            "window": 3,
            "script": "MovingFunctions.unweightedAvg(values)",
            "gap_policy": "skip"
          }
        }
      }
    },
    "peak_month":   { "max_bucket": { "buckets_path": "monthly>revenue" } },
    "avg_monthly":  { "avg_bucket": { "buckets_path": "monthly>revenue" } }
  }
}

Question answers: 1. peak_month.keys shows the month with the highest revenue. 2. Months where mom_growth.value < 0 had declining revenue vs the prior month. 3. smoothed_3m dampens spikes and shows the trend direction more clearly than raw monthly values. 4. The first bucket has no mom_growth value (null) — the derivative needs a previous bucket to compute a difference.

Removing min_doc_count: If a month has 0 orders, it disappears from the histogram. cumulative_sum skips the gap and the total stays flat for that month — but it looks like a plateau rather than the correct running total continuing. Adding min_doc_count: 0 ensures a bucket for every month even with 0 docs, giving the pipeline a complete series.

Lab Part B — Discount ratios and bucket selector

Step 3 — Discount rate by country:

GET /shop-orders-demo/_search
{
  "size": 0,
  "aggs": {
    "by_country": {
      "terms": { "field": "delivery.country_code", "size": 20 },
      "aggs": {
        "total_revenue":   { "sum": { "field": "total_amount"    } },
        "total_discounts": { "sum": { "field": "discount_amount" } },
        "avg_value":       { "avg": { "field": "total_amount"    } },
        "discount_rate": {
          "bucket_script": {
            "buckets_path": { "rev": "total_revenue", "disc": "total_discounts" },
            "script": "params.rev > 0 ? params.disc / params.rev : 0"
          }
        }
      }
    }
  }
}

Step 4 — bucket_selector:

GET /shop-orders-demo/_search
{
  "size": 0,
  "aggs": {
    "by_country": {
      "terms": { "field": "delivery.country_code", "size": 20 },
      "aggs": {
        "avg_value": { "avg": { "field": "total_amount" } },
        "only_premium_markets": {
          "bucket_selector": {
            "buckets_path": { "avgVal": "avg_value" },
            "script": "params.avgVal >= 120"
          }
        }
      }
    }
  }
}

bucket_selector is a parent pipeline agg — it runs after all bucket metrics (avg_value) have been computed, then removes buckets that fail the condition. Buckets below the threshold simply vanish from the response.

Chapter 07a — Shard strategy

Lab — Shard strategy and alias-based reshard

PUT /shop-orders-v1
{
  "settings": { "number_of_shards": 2, "number_of_replicas": 0 },
  "aliases": { "shop-orders": {} }
}

POST /shop-orders/_bulk
{ "index": {} }
{ "order_id": "ord-001", "channel": "web",    "total_amount": 150.00, "order_status": "completed" }
{ "index": {} }
{ "order_id": "ord-002", "channel": "mobile", "total_amount": 89.90,  "order_status": "pending" }
{ "index": {} }
{ "order_id": "ord-003", "channel": "web",    "total_amount": 320.00, "order_status": "completed" }

GET /shop-orders/_count

Expected: 3 documents.

GET /_cat/shards/shop-orders-v1?v

Shows 2 primary shards on es01, no replicas.

PUT /shop-orders-v2
{
  "settings": { "number_of_shards": 3, "number_of_replicas": 0 }
}

POST /_reindex?wait_for_completion=false
{
  "source": { "index": "shop-orders-v1" },
  "dest":   { "index": "shop-orders-v2" }
}

Monitor: GET /_tasks?actions=*reindex&detailed

After completion: GET /shop-orders-v2/_count → 3 documents.

POST /_aliases
{
  "actions": [
    { "remove": { "index": "shop-orders-v1", "alias": "shop-orders" } },
    { "add":    { "index": "shop-orders-v2", "alias": "shop-orders" } }
  ]
}

GET /_cat/aliases/shop-orders?v

Expected: shop-orders points to shop-orders-v2.

GET /shop-orders/_search

Returns 3 documents from shop-orders-v2.

GET /_cat/shards/shop-orders-v2?v

Expected: 3 primary shards.

Cleanup:

DELETE /shop-orders-v1
DELETE /shop-orders-v2

Chapter 07b — ILM and data streams

Lab Part A — Classic ILM rollover pipeline

PUT _ilm/policy/lab-logs-policy
{
  "policy": {
    "phases": {
      "hot":    { "actions": { "rollover": { "max_primary_shard_size": "1mb", "max_age": "7d" } } },
      "delete": { "min_age": "30d", "actions": { "delete": {} } }
    }
  }
}

PUT _index_template/lab-logs-template
{
  "index_patterns": ["lab-logs-*"],
  "template": {
    "settings": {
      "number_of_shards": 1,
      "number_of_replicas": 0,
      "index.lifecycle.name": "lab-logs-policy",
      "index.lifecycle.rollover_alias": "lab-logs-write"
    },
    "mappings": {
      "properties": {
        "@timestamp": { "type": "date" },
        "service":    { "type": "keyword" },
        "level":      { "type": "keyword" },
        "message":    { "type": "text" }
      }
    }
  },
  "priority": 200
}

PUT lab-logs-000001
{
  "aliases": {
    "lab-logs-write": { "is_write_index": true },
    "lab-logs-read":  {}
  }
}

Index documents, then trigger rollover:

POST lab-logs-write/_rollover
{
  "conditions": { "max_primary_shard_size": "1mb" }
}

Expected response: "rolled_over": true, "old_index": "lab-logs-000001", "new_index": "lab-logs-000002".

GET /_cat/aliases/lab-logs-*?v

lab-logs-write now points to lab-logs-000002 with is_write_index=true. lab-logs-read spans both indices.

GET lab-logs-read/_search
{
  "query": { "match_all": {} },
  "sort": [{ "@timestamp": "asc" }],
  "size": 20
}

Documents from both generations appear in timestamp order.

Lab Part B — Data stream

PUT _index_template/lab-stream-template
{
  "index_patterns": ["lab-stream*"],
  "data_stream": {},
  "template": {
    "settings": {
      "number_of_shards": 1,
      "number_of_replicas": 0,
      "index.lifecycle.name": "lab-logs-policy"
    },
    "mappings": {
      "properties": {
        "@timestamp": { "type": "date" },
        "service":    { "type": "keyword" },
        "level":      { "type": "keyword" },
        "message":    { "type": "text" }
      }
    }
  },
  "priority": 200
}

POST lab-stream/_doc
{ "@timestamp": "2026-05-30T10:00:00Z", "service": "api", "level": "info", "message": "Data stream start" }

GET _data_stream/lab-stream

Expected: generation: 1, write index is .ds-lab-stream-<date>-000001.

POST lab-stream/_rollover
GET _cat/indices/.ds-lab-stream*?v

Expected: two backing indices — the original (now read-only) and 000002 (current write index).

**Update by _id attempt (blocked):**

POST lab-stream/_update/some-fake-id
{ "doc": { "level": "warn" } }

Expected: 400 Bad Request — data streams are append-only; direct updates by _id are not supported.

Chapter 07c — Caching and query profiling

Lab — Caching and profiling slow queries

Step 1 — must vs filter comparison:

Run with must first, note time_in_nanos:

GET /shop-orders-demo/_search
{
  "profile": true,
  "query": {
    "bool": {
      "must": [
        { "term":  { "order_status": "completed" } },
        { "term":  { "channel": "web" } },
        { "match": { "notes": "payment" } }
      ]
    }
  },
  "size": 0,
  "aggs": { "by_channel": { "terms": { "field": "channel" } } }
}

Then with filter:

GET /shop-orders-demo/_search
{
  "profile": true,
  "query": {
    "bool": {
      "filter": [
        { "term": { "order_status": "completed" } },
        { "term": { "channel": "web" } }
      ],
      "must": [
        { "match": { "notes": "payment" } }
      ]
    }
  },
  "size": 0,
  "aggs": { "by_channel": { "terms": { "field": "channel" } } }
}

On the second run of the filter version, build_scorer time decreases — the filter bitset is served from the query cache.

Questions: 1. must clauses compute BM25 scores. filter clauses do not compute scores, so they do not appear in the scoring tree. 2. filter context results are stored as bitsets and reused across queries on the same shard — this is the node-level query cache.

Step 3 — Profile a wildcard:

GET /shop-orders-demo/_search
{
  "profile": true,
  "query": {
    "bool": {
      "must": [
        { "wildcard": { "order_status": "*end*" } }
      ]
    }
  },
  "aggs": { "by_status": { "terms": { "field": "order_status", "size": 50 } } }
}

The wildcard clause will show the highest time_in_nanos. Its Lucene description will be something like WildcardQuery — it scans the term dictionary rather than using a direct lookup.

Step 4 — Fixed query:

GET /shop-orders-demo/_search
{
  "profile": true,
  "query": {
    "bool": {
      "filter": [
        { "term": { "order_status": "pending" } }
      ]
    }
  },
  "size": 0,
  "aggs": { "by_status": { "terms": { "field": "order_status", "size": 10 } } }
}

Expected: much lower time_in_nanos — a term query on a keyword field is a direct dictionary lookup, and in filter context the result is cached.

Chapter 07d — Performance anti-patterns and templates

Lab Part A — Anti-patterns

Step 1 — PIT + search_after:

POST /shop-orders-demo/_pit?keep_alive=1m

Take the returned id, then:

GET /shop-orders-demo/_search
{
  "size": 10,
  "sort": [{ "created_at": "desc" }, { "_id": "asc" }],
  "pit": { "id": "<pit-id>", "keep_alive": "1m" }
}

For the next page, use the sort values from the last hit in search_after:

GET /shop-orders-demo/_search
{
  "size": 10,
  "sort": [{ "created_at": "desc" }, { "_id": "asc" }],
  "pit": { "id": "<pit-id>", "keep_alive": "1m" },
  "search_after": ["2024-03-15T12:00:00Z", "order-abc"]
}

Delete the PIT:

DELETE /_pit
{ "id": "<pit-id>" }

Step 2 — Source filtering:

The profile output’s fetch phase will show lower time when _source is restricted — fewer bytes to decompress and serialize.

Step 3 — Dynamic mapping:

GET /dynamic-strict-demo/_mapping

metadata.arbitrary_key_1 is not in the mapping (because "dynamic": false on the metadata object). The field is stored in _source but not indexed — you cannot search or aggregate on it:

GET /dynamic-strict-demo/_search
{
  "query": { "term": { "metadata.arbitrary_key_1": "value" } }
}

Returns 0 results — the field was not indexed.

Lab Part B — Component templates

Simulate before creating:

POST _index_template/_simulate_index/lab-logs-2024-test

Expected response includes: - All properties from both component templates merged (@timestamp, env, service, version, level, message, trace_id). - "dynamic": "strict" applied from the lab-log-fields component template. - Settings: 1 shard, 0 replicas.

After changing lab-log-fields to add duration_ms:

POST _index_template/_simulate_index/lab-logs-2024-test

The simulation shows duration_ms in the merged mappings.

GET /lab-logs-2024-test/_mapping

The existing index does NOT have duration_ms — template changes only apply to new indices. This is expected behavior.

Chapter 08a — Cluster health and monitoring

Lab Part A — Health diagnosis

Intentional yellow state:

PUT /health-demo
{
  "settings": { "number_of_shards": 2, "number_of_replicas": 1 }
}

GET /_cluster/health

Expected: "status": "yellow", "unassigned_shards": 2.

GET /_cat/shards/health-demo?v

Both replica shards (prirep: r) show UNASSIGNED.

GET /_cluster/allocation/explain

Expected: the explanation says the replica cannot be placed on the same node as its primary — there is only one node. The blocking cause will be something like "the shard cannot be allocated to the same node on which a copy of the shard already exists".

Fix:

PUT /health-demo/_settings
{ "index": { "number_of_replicas": 0 } }

GET /_cluster/health

Expected: "status": "green".

DELETE /health-demo

Lab Part B — Monitoring signals baseline

Step 6 — Node stats:

GET /_nodes/stats/indices,jvm

Find: nodes.<node-id>.jvm.mem.heap_used_percent (should be < 75% on a healthy dev cluster). Find: nodes.<node-id>.indices.search.query_time_in_millis / nodes.<node-id>.indices.search.query_total → average latency in ms per query.

Step 7 — Per-index stats:

GET /shop-orders-demo/_stats

Find: _all.primaries.docs.count, _all.primaries.store.size_in_bytes, _all.primaries.search.query_time_in_millis. request_cache.hit_count / (hit_count + miss_count) = cache hit rate. Low hit rate on an aggregation-heavy index suggests time range is not being rounded.

Step 8 — Largest indices:

GET /_cat/indices?v&s=store.size:desc&h=index,docs.count,store.size,health

System indices (.kibana_*, .ds-*) are typically small. shop-orders-demo should appear near the top.

Chapter 08b — Snapshots and backups

Lab — Snapshot and restore drill

PUT /_snapshot/lab-repo
{
  "type": "fs",
  "settings": { "location": "/tmp/es-snapshots" }
}

POST /_snapshot/lab-repo/_verify

Expected: { "nodes": { "<node-id>": { "name": "es01" } } } — the path is reachable.

PUT /_snapshot/lab-repo/snap-lab-1?wait_for_completion=true
{
  "indices": "shop-orders-demo",
  "include_global_state": false
}

Expected response: - "state": "SUCCESS" - "shards.failed": 0 - "duration_in_millis": < 5000 ms typically

GET /_snapshot/lab-repo/snap-lab-1

Confirms state: SUCCESS.

POST /_snapshot/lab-repo/snap-lab-1/_restore
{
  "indices": "shop-orders-demo",
  "include_global_state": false,
  "rename_pattern": "(.+)",
  "rename_replacement": "restored-$1"
}

Monitor: GET /_cat/recovery/restored-shop-orders-demo?v — wait for stage done.

GET /restored-shop-orders-demo/_count
GET /shop-orders-demo/_count

Both counts must match.

Second snapshot (incremental):

PUT /_snapshot/lab-repo/snap-lab-2?wait_for_completion=true
{
  "indices": "shop-orders-demo",
  "include_global_state": false
}

snap-lab-2’s "stats.incremental.size_in_bytes" will be near 0 — no segments changed, so nothing new was copied.

SLM:

PUT /_slm/policy/lab-daily
{
  "schedule": "0 0 1 * * ?",
  "name": "<lab-snap-{now/d}>",
  "repository": "lab-repo",
  "config": { "indices": ["shop-orders-demo"], "include_global_state": false },
  "retention": { "expire_after": "7d", "min_count": 2, "max_count": 10 }
}

POST /_slm/policy/lab-daily/_execute

GET /_slm/policy/lab-daily

last_success.snapshot_name shows the name of the snapshot just taken.

Cleanup:

DELETE /restored-shop-orders-demo
DELETE /_slm/policy/lab-daily
DELETE /_snapshot/lab-repo/snap-lab-1
DELETE /_snapshot/lab-repo/snap-lab-2
DELETE /_snapshot/lab-repo

Chapter 08c — Ingest pipelines and best practices

Lab — Build and test an ingest pipeline

Create the pipeline:

PUT /_ingest/pipeline/normalize-orders-v1
{
  "description": "Normalize order documents on ingest",
  "processors": [
    { "set":      { "field": "@timestamp", "value": "{{created_at}}", "ignore_failure": true } },
    { "date":     { "field": "@timestamp", "formats": ["ISO8601"], "ignore_failure": true } },
    { "lowercase":{ "field": "customer.name", "ignore_missing": true } },
    { "convert":  { "field": "amount", "type": "double", "ignore_missing": true, "ignore_failure": true } },
    { "remove":   { "field": ["internal_debug_field"], "ignore_missing": true } }
  ],
  "on_failure": [
    { "set": { "field": "ingest.error", "value": "{{_ingest.on_failure_message}}" } }
  ]
}

Simulate — document 1 (valid):

POST /_ingest/pipeline/normalize-orders-v1/_simulate
{
  "docs": [
    {
      "_index": "orders-test",
      "_source": {
        "created_at": "2026-05-30T09:15:00Z",
        "customer": { "name": "Alice SMITH" },
        "amount": "149.99",
        "internal_debug_field": "drop-me"
      }
    }
  ]
}

Expected result: - @timestamp: 2026-05-30T09:15:00.000Z (parsed date, not string) - customer.name: "alice smith" (lowercased) - amount: 149.99 (double, not string) - internal_debug_field: absent from _source

Simulate — document 2 (bad amount):

POST /_ingest/pipeline/normalize-orders-v1/_simulate
{
  "docs": [
    {
      "_index": "orders-test",
      "_source": {
        "created_at": "2026-05-30T10:00:00Z",
        "customer": { "name": "BOB JONES" },
        "amount": "not-a-number",
        "internal_debug_field": "drop"
      }
    }
  ]
}

Because convert has ignore_failure: true, the document is processed successfully. amount stays as the string "not-a-number" (the conversion was silently skipped). ingest.error is NOT set — on_failure is only triggered when a processor fails without ignore_failure.

on_failure in action (strict version):

PUT /_ingest/pipeline/normalize-orders-v1-strict
{
  "description": "Strict version to test on_failure",
  "processors": [
    { "convert": { "field": "amount", "type": "double" } }
  ],
  "on_failure": [
    { "set": { "field": "ingest.error", "value": "{{_ingest.on_failure_message}}" } }
  ]
}

POST /_ingest/pipeline/normalize-orders-v1-strict/_simulate
{
  "docs": [
    { "_index": "orders-test", "_source": { "amount": "not-a-number" } }
  ]
}

Expected: ingest.error is set with the conversion error message. The document was still indexed (the on_failure handler ran instead of aborting).

Attach to index and verify auto-run:

PUT /orders-pipeline-test
{
  "settings": { "number_of_replicas": 0, "index.default_pipeline": "normalize-orders-v1" },
  "mappings": {
    "properties": {
      "@timestamp": { "type": "date" },
      "created_at": { "type": "keyword" },
      "customer":   { "type": "object" },
      "amount":     { "type": "double" }
    }
  }
}

POST /orders-pipeline-test/_doc
{
  "created_at": "2026-05-30T11:00:00Z",
  "customer": { "name": "CAROL NOVAK" },
  "amount": "299.50",
  "internal_debug_field": "should-be-dropped"
}

GET /orders-pipeline-test/_search

Expected _source: - @timestamp: parsed date - customer.name: "carol novak" - amount: 299.5 (double) - internal_debug_field: absent

Trade-off: ignore_failure: true means bad documents are accepted but their field may have the wrong type. on_failure without ignore_failure means every conversion failure results in the error being recorded — bad documents are visible but not lost. Choose based on whether you prefer silent skips or explicit error tracking.

Cleanup:

DELETE /orders-pipeline-test
DELETE /_ingest/pipeline/normalize-orders-v1
DELETE /_ingest/pipeline/normalize-orders-v1-strict

Labs — Solutions

Chapter 01 — Introduction and architecture

Lab 1 — Explore the cluster

Lab 2 — Index a document and observe near real-time refresh

Lab 3 — Understand shard and replica distribution

Lab 4 — Preview the course dataset

Chapter 02 — Data model and CRUD

Lab 2-A — Index, read, and verify your first document

Lab 2-B — Partial update and scripted update

Lab 2-C — Bulk ingest and error handling

Lab 2-D — Mapping inspection and type pitfall

Chapter 03 — Basic search

Lab 1 — Explore the response envelope

Lab 2 — match vs term in practice

Lab 3 — Build a bool query step by step

Lab 4 — Pagination and sorting

Chapter 04 — Mapping and analyzers

Lab 1 — Inspect and compare mapping behavior

Lab 2 — Explore analysis with _analyze

Lab 3 — Dynamic vs strict mapping

Lab 4 — Multi-field design

Chapter 05a — Custom analyzers

Lab — Custom analyzer with synonyms and diacritics

Chapter 05b — Modeling relationships

Lab Part A — nested query with inner_hits and sorting

Lab Part B — join field (parent-child)

Chapter 05c — Highlighting and autocomplete

Lab Part A — Highlighting

Lab Part B — Completion suggester

Chapter 05d — Custom relevance scoring

Lab — function_score and script_score

Chapter 06a — Bucket and metric aggregations

Lab — Bucket and metric aggregations on shop-orders-demo

Chapter 06b — Nested and multi-level aggregations

Lab Part A — Multi-level buckets

Lab Part B — Nested and reverse_nested

Chapter 06c — Pipeline aggregations

Lab Part A — Monthly sales trend

Lab Part B — Discount ratios and bucket selector

Chapter 07a — Shard strategy

Lab — Shard strategy and alias-based reshard

Chapter 07b — ILM and data streams

Lab Part A — Classic ILM rollover pipeline

Lab Part B — Data stream

Chapter 07c — Caching and query profiling

Lab — Caching and profiling slow queries

Chapter 07d — Performance anti-patterns and templates

Lab Part A — Anti-patterns

Lab Part B — Component templates

Chapter 08a — Cluster health and monitoring

Lab Part A — Health diagnosis

Lab Part B — Monitoring signals baseline

Chapter 08b — Snapshots and backups

Lab — Snapshot and restore drill

Chapter 08c — Ingest pipelines and best practices

Lab — Build and test an ingest pipeline

Lab 2 — `match` vs `term` in practice

Lab 3 — Build a `bool` query step by step

Lab 2 — Explore analysis with `_analyze`

Lab — `function_score` and `script_score`