08b — Snapshots and backups

“Backup” only exists if you have practiced the restore. This part covers the full snapshot lifecycle — from registering a repository, taking incremental snapshots, and automating with SLM, to restoring safely without clobbering live data.

Goals for this part

Register a snapshot repository (filesystem and S3)
Take, list, and verify snapshots
Restore to a renamed index to validate without risk
Automate snapshots with SLM (Snapshot Lifecycle Management)
Know when a snapshot vs. a source-of-truth rebuild is the right recovery path

8.3 Snapshots and restore — backup strategies

Snapshots are the only supported backup mechanism

Elasticsearch snapshots are incremental and segment-based: the first snapshot captures all segments; subsequent snapshots only copy new or changed segments. This makes them efficient for daily/hourly schedules.

Snapshots capture: - Index data (shards → segments) - Index metadata (mappings, settings) - Optionally: cluster state (templates, ILM policies, etc.)

Know your source of truth. Elasticsearch is frequently a derived store — data originally lives in a database or event stream and is indexed into ES for search. In that case, a full data rebuild from source may be more appropriate than a restore, especially after mapping changes. Snapshots are still critical for disaster recovery and for data that only exists in ES.

Step 1: Register a repository

File system (useful for local or NFS mounts):

PUT /_snapshot/my-repo
{
  "type": "fs",
  "settings": {
    "location": "/mnt/snapshots"
  }
}

S3-compatible (common in production):

PUT /_snapshot/my-repo
{
  "type": "s3",
  "settings": {
    "bucket": "my-es-snapshots",
    "region": "eu-west-1"
  }
}

Step 2: Take a snapshot

PUT /_snapshot/my-repo/snap-shop-orders-2026-05-30?wait_for_completion=true
{
  "indices": "shop-orders-demo",
  "include_global_state": false
}

wait_for_completion=true blocks until done (fine for ad-hoc; use async for automation).

Step 3: List and verify

GET /_snapshot/my-repo/_all
GET /_snapshot/my-repo/snap-shop-orders-2026-05-30

Check state: SUCCESS and shards.failed: 0.

Step 4: Restore

Restore to a renamed index to avoid clobbering the live index:

POST /_snapshot/my-repo/snap-shop-orders-2026-05-30/_restore
{
  "indices": "shop-orders-demo",
  "include_global_state": false,
  "rename_pattern": "(.+)",
  "rename_replacement": "restored-$1"
}

This restores shop-orders-demo as restored-shop-orders-demo. Validate doc counts and run a couple of queries before treating the restore as good.

Snapshot Lifecycle Management (SLM)

SLM schedules snapshots automatically. Create a policy:

PUT /_slm/policy/daily-shop-orders
{
  "schedule": "0 30 1 * * ?",
  "name": "<shop-orders-{now/d}>",
  "repository": "my-repo",
  "config": {
    "indices": ["shop-orders-*"],
    "include_global_state": false
  },
  "retention": {
    "expire_after": "30d",
    "min_count": 5,
    "max_count": 50
  }
}

Trigger it manually to test:

POST /_slm/policy/daily-shop-orders/_execute

Check SLM status:

GET /_slm/policy/daily-shop-orders
GET /_slm/stats

Snapshot best practices

Always restore into a separate index (or separate cluster) to validate — never overwrite a live index with an untested restore.
Take a snapshot before any reindex, bulk update, or upgrade operation.
Test restores on a schedule, not just when you need them. A snapshot that cannot restore is not a backup.
Monitor SLM policy execution and alert on failures.

Lab — Snapshot and restore drill

Goal: Practice the full snapshot lifecycle including a safe restore with rename.

Note on the filesystem repository: The location path must be writable by the Elasticsearch process. For the Docker Compose setup from chapter 01, mount a volume or use a path inside the container. If you cannot configure a filesystem mount, skip Step 1 and use a pre-configured repository your instructor has set up.

Step 1 — Register a filesystem snapshot repository:

PUT /_snapshot/lab-repo
{
  "type": "fs",
  "settings": {
    "location": "/tmp/es-snapshots"
  }
}

Verify the repository is accessible:

POST /_snapshot/lab-repo/_verify

A 200 OK with a nodes object means the path is reachable from all nodes.

Step 2 — Take a snapshot of shop-orders-demo. Use wait_for_completion=true so you can immediately check the result:

PUT /_snapshot/lab-repo/snap-lab-1?wait_for_completion=true
{
  "indices": "shop-orders-demo",
  "include_global_state": false
}

Questions from the response: 1. What is the state of the snapshot? 2. How many shards were snapshotted successfully? 3. How long did it take (duration_in_millis)?

Step 3 — List all snapshots in the repository and verify:

GET /_snapshot/lab-repo/_all

GET /_snapshot/lab-repo/snap-lab-1

Confirm state: SUCCESS and shards.failed: 0.

Step 4 — Restore the snapshot as a renamed index (so you don’t overwrite the live shop-orders-demo):

POST /_snapshot/lab-repo/snap-lab-1/_restore
{
  "indices": "shop-orders-demo",
  "include_global_state": false,
  "rename_pattern": "(.+)",
  "rename_replacement": "restored-$1"
}

Monitor the restore:

GET /_cat/recovery/restored-shop-orders-demo?v

Wait until the stage shows done for all shards.

Step 5 — Validate the restored index:

GET /restored-shop-orders-demo/_count

Compare to the original:

GET /shop-orders-demo/_count

Do the counts match? Run a sample query on the restored index:

GET /restored-shop-orders-demo/_search
{
  "query": { "term": { "order_status": "completed" } },
  "size": 3
}

Does the data look correct?

Step 6 — Take a second snapshot (to observe incremental behavior):

PUT /_snapshot/lab-repo/snap-lab-2?wait_for_completion=true
{
  "indices": "shop-orders-demo",
  "include_global_state": false
}

Compare size_in_bytes of snap-lab-1 vs snap-lab-2. The second snapshot should be much smaller because only changed segments are copied (most segments are shared with the first snapshot).

GET /_snapshot/lab-repo/snap-lab-1
GET /_snapshot/lab-repo/snap-lab-2

Step 7 — Create an SLM policy and execute it manually:

PUT /_slm/policy/lab-daily
{
  "schedule": "0 0 1 * * ?",
  "name": "<lab-snap-{now/d}>",
  "repository": "lab-repo",
  "config": {
    "indices": ["shop-orders-demo"],
    "include_global_state": false
  },
  "retention": {
    "expire_after": "7d",
    "min_count": 2,
    "max_count": 10
  }
}

Execute immediately:

POST /_slm/policy/lab-daily/_execute

Check the result:

GET /_slm/policy/lab-daily

Inspect last_success in the response.

Step 8 — Check SLM stats:

GET /_slm/stats

Step 9 — Clean up:

DELETE /restored-shop-orders-demo
DELETE /_slm/policy/lab-daily
DELETE /_snapshot/lab-repo/snap-lab-1
DELETE /_snapshot/lab-repo/snap-lab-2
DELETE /_snapshot/lab-repo

Takeaway: “Backup” only exists if you have practiced the restore. Always restore to a renamed index — never overwrite a live index with an untested restore. Incremental snapshots share unchanged segments, making subsequent snapshots fast and storage-efficient. SLM automates the schedule and retention; pair it with an alert on last_failure so you know when snapshots silently stop working.