Setting up a secure logging and metrics platform
A practical, hands-on guide to a production-grade observability stack on Kubernetes — ECK, Falco, Prometheus, Thanos — secured end-to-end with mTLS.
By Werner Dijkerman — published on Leanpub
About the book
This practical, hands-on guide shows you exactly how to build a production-grade logging and metrics platform on Kubernetes using the Elastic Cloud on Kubernetes (ECK) operator. You will deploy a fully integrated Elasticsearch cluster with Kibana, Logstash, Filebeat, and Metricbeat, secured end-to-end with mTLS and zero-trust principles. Runtime security events from the eBPF-powered Falco are streamed directly into Elasticsearch, giving you real-time threat visibility and audit trails.
On the metrics side, you will stand up a highly available Prometheus, Grafana, and Alertmanager stack, fronted by Thanos Query for global query federation and long-term retention, again with full TLS encryption and certificate management.
At the end of the book you will have a highly secure and available logging and metrics platform, ready to run on any production environment. And all the code that is needed to make this happen too.
Who is this book for?
- DevOps and Platform Engineers building production observability on Kubernetes.
- Site Reliability Engineers who need a fully secured logging and metrics stack from day one.
- Security teams looking to stream Falco runtime events into Elasticsearch for real-time threat detection.
- Engineers using or evaluating ECK (Elastic Cloud on Kubernetes).
- Anyone implementing Prometheus and Thanos at scale with TLS and certificate management built in.
Table of contents
Chapter 4 is currently being written.
Chapter 1 — Introduction
- Welcome and guide overview
- Tooling
- Who and why
- AI
- Why monitoring is important
Chapter 2 — Building a secure logging solution with the ECK operator
- Prerequisites and architecture overview
- Installing the ECK operator
- Deploying the core Elastic Stack (Elasticsearch, Kibana, Logstash)
- Data collection layer (Filebeat, Metricbeat)
- Security foundations: mutual TLS for every component
- Network policies (Reflector, eck-operator, Elastic Stack)
- Extending with runtime security events from Falco
- Configuring automated snapshots and backups
- Production-ready configurations for each component
- What we deliberately left out
Chapter 3 — Metrics setup with Prometheus and Thanos Query
- Basic setup for Prometheus and Thanos
- Network policies
- Configuring TLS for Prometheus and Thanos
- Adding metrics: Cilium and Falco
- Configure for production
- What did we not do? (Central Thanos, scaling, AuthN/AuthZ)
Appendix 1 — Setting up a Kind cluster
- Kind, Gateway API CRDs, Cilium, Cert Manager, CSI driver, Reflector
- CA certificate and Gateway configuration
Read a free sample
A free preview is available on Leanpub. Read the first chapters before you decide.
Get the sample on LeanpubAbout the author
I'm Werner Dijkerman, a freelance DevOps and Platform Engineer based in Utrecht, the Netherlands. I work daily with the technologies covered in this book, and I'm a technical reviewer for Packt, O'Reilly, Manning and BPB Online. I also maintain several open-source projects in the observability space.