Skip to content
h8lio

Object storage

An object store gives your applications an S3 API inside your cluster: buckets, access keys, presigned URLs, the same client libraries you would use against a cloud provider, with the data on a volume you own. The platform does not provide a shared one. You run yours, in your cluster, from a chart the platform publishes and runs itself.

The platform’s own object store is Silo, the maintained community fork of MinIO. MinIO’s community line stopped receiving updates in early 2026, security fixes included, while the project no longer ships a chart repository of its own. Silo keeps the S3 API, the MINIO_* environment variables, the client and the on-disk format, so it is a drop-in successor, and it is what the platform migrated its own buckets to.

The command line client follows the same path. MinIO’s mc is archived too; Silo maintains it as mcli, same commands, same flags, same configuration, only the binary name changes. The examples below use it. Your applications do not need it: they speak S3 through any SDK, and any S3 tool works against the endpoint, the aws CLI with --endpoint-url or rclone included. mcli is what you reach for when the operation is administrative: users, policies, versioning, replication.

Silo publishes no Helm repository either, so h8lio publishes the chart it runs, unchanged, at the version it runs:

oci://registry.h8l.io/charts/silo

Pulling it needs no account. A published version never changes: the same version always installs the same chart, which is what lets you pin it in your values and in your GitOps repository. See Helm if the CLI is not set up yet.

Start from a values file rather than from --set flags. Everything not listed is the chart default, and the defaults are sized for a sixteen node distributed deployment, which is not what a first object store in a cluster looks like.

silo.yaml
mode: standalone # one pod, one volume; the default is a 16 replica distributed cluster
persistence:
size: 50Gi
storageClass: eu-west-fr-gra-block-nvme-ec-xfs
# The chart default requests 16Gi of memory. Size it for your cluster's quota.
resources:
requests:
cpu: 100m
memory: 512Mi
limits:
memory: 2Gi
# Root credentials. Prefer a secret you manage over inline values: the secret must carry the
# keys `rootUser` and `rootPassword`.
existingSecret: silo-root
# Buckets created at startup, and a service user your applications will use.
buckets:
- name: app-assets
policy: none
users:
- accessKey: app
existingSecret: silo-app # key `secretKey` holds the secret key
existingSecretKey: secretKey
policy: readwrite

storageClass values and what they mean are listed in Volumes. Then:

Terminal window
kubectl create secret generic silo-root -n my-cluster \
--from-literal=rootUser=admin --from-literal=rootPassword='<a long random password>'
kubectl create secret generic silo-app -n my-cluster \
--from-literal=secretKey='<another long random secret>'
helm install silo oci://registry.h8l.io/charts/silo -n my-cluster -f silo.yaml

Without --version, Helm installs the newest published version. A version is published once the platform has qualified it and runs it in production, so the newest one is the one to start from. The server image is a separate matter: a chart version ships with the server release it was cut with, and the platform often runs a newer one, since Silo publishes server releases more often than chart versions. Pin the newest server release in your values from the first install, so you start where the platform is rather than two upgrades behind it (see Upgrade for the two keys). Read which version you got, and keep it in your notes or your GitOps repository:

Terminal window
helm list -n my-cluster # CHART column, e.g. silo-7.0.3
helm show chart oci://registry.h8l.io/charts/silo # newest published version, and its image

From then on, pass that version explicitly with --version on every helm upgrade. Without it, an upgrade run for a configuration change also moves you to the newest chart, silently.

Helm cannot list the versions of an OCI repository. Any OCI client can, anonymously:

Terminal window
oras repo tags registry.h8l.io/charts/silo
curl -s 'https://registry.h8l.io/api/v2.0/projects/charts/repositories/silo/artifacts?with_tag=true' \
| jq -r '.[].tags[]?.name'

oras is a single binary, nothing to configure.

The chart publishes two services in your namespace:

ServicePortUse it for
silo9000the S3 API, http://silo.my-cluster.svc:9000
silo-console9001the web console, sign in with the root credentials

Point your applications at the service name, with path-style addressing (the bucket in the path, not in the host name), which is what any S3 SDK does when you give it an explicit endpoint:

env:
- name: S3_ENDPOINT
value: http://silo.my-cluster.svc:9000
- name: S3_ACCESS_KEY
value: app
- name: S3_SECRET_KEY
valueFrom:
secretKeyRef:
name: silo-app
key: secretKey

To reach the API or the console from outside the cluster, pick the door that matches how much you want to expose:

  • A port-forward, when nothing should be published. It is the right door for the console and for hand operations: kubectl port-forward svc/silo 9000:9000 -n my-cluster, then the client talks to it as if it were local:

    Terminal window
    mcli alias set my-silo http://localhost:9000 admin '<root password>'
    mcli ls my-silo
    mcli mirror ./local-dir my-silo/app-assets
  • A route, when applications outside the cluster need the API: an IngressRoute on the silo service, with a certificate, exactly as the platform publishes its own. The recipe is in Certificates. Publish the API on its own host name and leave the console unpublished unless you protect it: it is the administration surface of your data.

  • A gateway, only when the traffic has to enter through a server you operate outside the cluster (Gateway). It is the heaviest of the three and rarely needed for an object store.

Back up the data: replicate it, do not copy the volume

Section titled “Back up the data: replicate it, do not copy the volume”

An object store grows into the terabytes, and the platform’s backup tools are built around volumes: a cold backup copies every byte of the volume at each run, a hot snapshot freezes it in the storage cluster. Neither is the right shape for a large store, and the S3 protocol already carries the right one. Replicate your buckets to an object store outside the platform: a bucket at OVHcloud, Scaleway, AWS or any S3 endpoint you hold an account on. The copy is incremental and continuous rather than a nightly full pass, it costs the platform no backup traffic, and it doubles as a failover site your applications can be pointed at.

Two ways to run it, both with tools you already have:

Terminal window
# One-shot or scheduled mirror (a CronJob, or your CI): copies what changed, removes what went
mcli alias set offsite https://s3.<provider>.example '<access key>' '<secret key>'
mcli mirror --remove --overwrite my-silo/app-assets offsite/app-assets-replica
# Server-side bucket replication: the store pushes every change itself, as it happens.
# Both buckets need versioning enabled.
mcli version enable my-silo/app-assets
mcli version enable offsite/app-assets-replica
mcli replicate add my-silo/app-assets --remote-bucket offsite/app-assets-replica

Then treat the replica as you would any backup: check that it is complete, and try a restore in the other direction once.

The platform’s cold backups are not the tool for an object store, and the platform keeps its own stores out of them. A file-system backup copies the volume as it is at that moment, so an object being written during the pass can land half-copied, and a store that grows past a few tens of gigabytes turns each nightly pass into hours of pressure on the backup chain. The volume can still be opted in for a small store, through the podAnnotations value and the annotation described in the Backups guide (export is the name the chart gives its data volume, not the name of your claim); if you do, open the first backup and check that the export volume is listed with a non-zero size, and drop the annotation the day the store grows. The replica above remains the recommendation, at any size.

Silo reads MinIO’s on-disk format as is, its IAM included: users, policies and service accounts stored in the volume come across untouched. The migration is therefore a change of image on the same volume, and the cleanest way to do it is a new release reusing the existing claim.

  1. Back up the buckets first, completely, outside the volume: mc mirror to another store, or the replication described in Back up the data. The steps below keep the volume, and the platform rehearsed the switch in both directions, including after Silo had written to it. A full copy is still the only net that does not depend on getting every step right.

  2. Find out who owns the claim and the root credentials. If you installed MinIO with persistence.existingClaim and existingSecret, neither belongs to the release: skip to step 5. Otherwise the MinIO chart created them, and uninstalling the release deletes them, data included. Check what the release holds:

    Terminal window
    helm get manifest minio -n my-cluster | grep -E '^kind: (PersistentVolumeClaim|Secret)'
  3. Keep the claim. Helm decides what to spare at uninstall from the manifest it stored for the release, never from the live object, so annotating the claim with kubectl annotate has no effect. The annotation has to go through the chart, with an upgrade that changes nothing else: same chart version (the CHART column of helm list), same image, current values.

    Terminal window
    helm get values minio -n my-cluster -o yaml > minio-current.yaml
    helm upgrade minio minio/minio --version 5.4.0 -n my-cluster -f minio-current.yaml \
    --set-string 'persistence.annotations.helm\.sh/resource-policy=keep'
    helm get manifest minio -n my-cluster | grep -A12 '^kind: PersistentVolumeClaim'

    The last command must show helm.sh/resource-policy: keep under the claim before you go on. It reads what Helm will read at uninstall, which is the whole point. The MinIO pod does not restart: only the claim’s annotations changed. If your chart offers no annotations for the claim, stop here and ask support: the platform can hold the volume itself while the claim changes hands.

  4. Copy the root credentials to a secret the release does not own. The MinIO chart gives its secret no annotations, so it cannot be kept the way the claim is:

    Terminal window
    kubectl get secret minio -n my-cluster -o json \
    | jq '{apiVersion, kind, type, data, metadata: {name: "silo-root"}}' \
    | kubectl apply -n my-cluster -f -
  5. Uninstall the MinIO release. The Deployment, the services and the chart’s secret go, the claim stays. Two pods cannot mount the same claim, which is why the old release goes before the new one comes.

  6. Install Silo on the same claim, with the same credentials and the same names. Start from your MinIO values file rather than from scratch: carry over resources, environment and clusterDomain, since the Silo defaults describe a sixteen pod distributed cluster. Then add:

    mode: standalone
    persistence:
    existingClaim: minio # the claim MinIO was using
    existingSecret: silo-root # the copy made at step 4 (or your own secret), keys rootUser and rootPassword
    fullnameOverride: minio # keep the service names, and your applications untouched
    nameOverride: minio # keep the app=minio label, for anything that selects the pods
    image:
    tag: RELEASE.<date> # the newest Silo server release, not the one the chart was cut with
    mcImage:
    tag: RELEASE.<date> # its matching client, same date

    Pinning the server release here spares a second restart right after the migration: the chart’s bundled image is the release the chart was cut with, and the platform usually runs a newer one. Both keys take the same date, from the Silo releases.

    helm template with the same values shows the render before anything is installed.

  7. Check that the buckets are all there, then that a write and a read work. Your applications never noticed: same service name, same credentials, same data.

To roll back, reinstall your MinIO release with persistence.existingClaim: minio and existingSecret: silo-root: same volume, same credentials.

If you run MinIO from your own manifests rather than from the chart, switching the image to pgsty/silo on the same volume also works, depending on how far behind your MinIO is. One detail decides whether the pod comes up: the image ships a binary named silo and no minio alias, so a manifest whose command runs minio server must run silo server instead. The platform tested the swap and still preferred a clean install on the existing claim, which is the recipe above.

Two traps, both measured during the platform’s own migration:

  • Bucket policies need the bare bucket ARN. A policy that only names arn:aws:s3:::bucket/* still reads but fails to write, silently, at the first request rather than at startup. Add arn:aws:s3:::bucket next to it before migrating.
  • OpenID tokens must be signed asymmetrically. If you authenticate to the console or the API with an OpenID provider, HMAC-signed tokens are no longer accepted; the provider must publish a JWKS.

New versions land on the same address. Check what the platform runs before you move, then upgrade with the values file you already have:

Terminal window
helm show chart oci://registry.h8l.io/charts/silo # newest published version
oras repo tags registry.h8l.io/charts/silo # every published version
helm upgrade silo oci://registry.h8l.io/charts/silo --version <new> \
-n my-cluster -f silo.yaml --history-max 3

A Silo release does not always move the chart. When only the server image changes, no new chart version is published, and the chart keeps pulling the image it was released with. To take the newer server, set its tag in your values and upgrade at the version you already run. Set the client image to the same release: the chart runs it in the job that creates the buckets, users and policies declared in your values, and Silo pairs each server release with its client.

silo.yaml
image:
tag: RELEASE.<date> # from https://github.com/pgsty/silo/releases
mcImage:
tag: RELEASE.<date> # the client of the same release
Terminal window
helm upgrade silo oci://registry.h8l.io/charts/silo --version <current> \
-n my-cluster -f silo.yaml --history-max 3

Keep the two tags in your values once set: a later chart upgrade does not override them, so remove them when the chart’s own images catch up.

In standalone mode the pod is recreated, not rolled: a single claim cannot be mounted twice, so expect a few seconds without the API. Plan it outside the hours where your applications write.