Skip to content
h8lio

Grafana

Two Grafanas are in play on h8lio, and they answer different questions. The one we provision shows what the platform measures about your workloads, with no setup on your side. The one you run yourself shows what your applications report, and is where your own dashboards and logs live.

Start with ours: it already holds the container view, which is the half most teams discover late.

Your organization gets its own space at https://monitoring.h8l.io, created with the organization itself. Sign in with your h8lio account: no invitation, no datasource to configure, no dashboard to import.

Nine dashboards are provisioned, all scoped to your namespaces.

DashboardWhat it answers
Compute Resources / PodCPU, CPU throttling, memory working set, disk IOPS and throughput, per container
Compute Resources / Namespace (Pods)CPU and memory of every pod, as a share of both requests and limits
Compute Resources / Namespace (Workloads)the same, grouped by deployment, statefulset or daemonset
Compute Resources / Workloadone workload in detail, pod by pod
Networking / Podbandwidth, packet rate, dropped packets for one pod
Networking / Namespace (Pods)the same across every pod of a namespace
Networking / Namespace (Workload)the same, grouped by workload
Networking / Workloadone workload, with per-pod averages
Persistent Volumesspace and inode usage per volume

Your Grafana role follows your role in the organization:

Your h8lio roleGrafana roleWhat you get
Owner, Admin, OperatorEditorthe nine dashboards, plus Explore for your own queries, your own dashboards and your own alert rules
DeveloperViewerthe nine dashboards

Every query only sees your organization’s namespaces, whether it comes from a dashboard, from Explore or from an alert rule. The boundary is enforced by the platform beneath the datasource, not by the way a query is written, so you can write PromQL freely. The datasource itself cannot be modified.

The alert rules you write need somewhere to go: your organization comes with an email contact point addressed to its owner, and the default notification policy routes to it. You can edit it, add recipients, or create other contact points.

The container view is not the runtime view

Section titled “The container view is not the runtime view”

Compute Resources / Pod is the one to open first when a pod restarts without an obvious cause. It reports what the kernel counts for the container: everything the process holds, not just what your runtime accounts for. A JVM heap, a Node.js heap or a Python allocator can look perfectly healthy while the container is over its limit, because thread stacks, compiled code, metaspace and native buffers all sit outside the runtime’s own numbers and inside the container’s.

Two panels are worth knowing before you need them:

  • CPU throttling tells you whether the kernel is capping your container. A container limited to half a core gets 50 ms of CPU per 100 ms period, and a runtime that wants more during startup spends the rest of each period waiting. High throttling with low average CPU is the signature.
  • Memory usage (WSS) against the limit is what decides an out-of-memory kill. Sampled panels show a floor, never the instant allocation that triggers the kill, so a curve that stays just under the limit is already a finding.

Reading both views together, ours and your runtime’s, is what makes the difference between the two measurable rather than mysterious.

They are deliberately narrow, and knowing the edges saves you a wasted search:

  • The nine dashboards stay as provided. Nobody edits them, so they keep working the same way for every member. To change a panel, build your own dashboard next to them.
  • Metrics only, no logs. See Loki for logs.
  • 30 days of history. The dashboards reach back 30 days; older metrics are no longer kept. For a longer history of your own metrics, set the retention of your own Prometheus.
  • Platform metrics, not application metrics. CPU, memory, network, storage and volumes are covered. Your request rates, queue depths and business counters are not: they come from your own Prometheus, see Prometheus.
  • Namespaces are matched by name prefix. Your dashboards select namespaces starting with your organization name, so acme-prod, acme-staging and acme-monitoring all appear. A namespace that does not carry the prefix will not show up.
  • No exit reason, no restart count. To know why a pod stopped, ask the cluster directly (kubectl get pod <name> -o jsonpath='{.status.containerStatuses[*].lastState}') or open a support ticket: we keep a longer metric history than a pod does.

Ours answers questions about containers. Yours answers questions about applications: request latency, queue depth, business counters, plus your logs, side by side with them.

The usual layout is a dedicated monitoring namespace for your organization (an h8lio cluster such as acme-monitoring) hosting Grafana next to your Prometheus instance and, if you collect logs, your Loki instance. All namespaces of an organization share the label tenant: <organization>, which makes cross-namespace selection a one-liner.

Install it as you would any workload in your namespace, then add the datasources. Both live in the same namespace, so their in-cluster addresses are enough:

apiVersion: v1
kind: ConfigMap
metadata:
name: grafana-datasources
namespace: acme-monitoring
data:
datasources.yaml: |
apiVersion: 1
datasources:
- name: Prometheus
type: prometheus
# Service exposed by the Prometheus Operator for your instance
url: http://prometheus-operated:9090
access: proxy
isDefault: true
- name: Loki
type: loki
url: http://loki-gateway:80
access: proxy

From there you can import community dashboards, build your own, and put metrics and logs side by side on the same page.

Two things to keep in mind when you size it: everything you run in your namespace is billed on the CPU and memory limits you declare, summed over every container of every pod, so a modest Grafana is worth more than a generous one. And a log collector deployed as a sidecar is paid once per pod: a single collector for the whole namespace costs the same whether you run five pods or fifty.