h8lio

Gateway

Tutorial to create an instance to be used as a bidirectional Gateway proxy by using Traefik and kubectl port-forward. The Gateway allows to provide an egress traffic static public IP and/or a way to access Kubernetes private services.

The diagram below represents the Gateway role : Gateway Diagram

The following documentation presents how to create a Gateway instance on an Ubuntu 20.04 server. You may need to adapt it to your needs.

Instance

  • Minimum Requirement:
    • 1 vCPU
    • 2GB RAM
    • 100Gbps Bandwidth
    • 20GB Disk
  • OS: Unbuntu 20.04

All the following commands have been executed as root user.

Web Proxy

We will use Traefik as reverse proxy. You can use any reverse proxy you like.

Installation

  1. go to https://github.com/traefik/traefik/releases and download the latest binary. The documentation has been tested with the 2.6.0 version.
  2. extract the binary and test the installation
tar -zxvf traefik_[VERSION]_linux_[ARCH].tar.gz
cp /path/to/traefik /usr/local/bin
chown root:root /usr/local/bin/traefik
chmod 755 /usr/local/bin/traefik
traefik --help
  1. create traefik user and group
groupadd -g 321 traefik
useradd \
  -g traefik --no-user-group \
  --home-dir /var/www --no-create-home \
  --shell /usr/sbin/nologin \
  --system --uid 321 traefik
id traefik

Output:

uid=321(traefik) gid=321(traefik) groups=321(traefik)

Configuration

  1. create an admin user with password
apt install apache2-utils
htpasswd -nb admin [PASSWORD]

Output:

admin:$apr1$ruca84Hq$mbjdMZBAG.KWn7vfN/SNK/

  1. traefik configuration file
mkdir /etc/traefik
vi /etc/traefik/traefik.toml

traefik.toml

[global]
  checkNewVersion = true
  sendAnonymousUsage = true

[api]
  dashboard = true

[log]
  level = "INFO"
  filePath = "/var/log/traefik.log"

[entryPoints]
  [entryPoints.http]
    address = ":80"
    [entryPoints.http.http.redirections.entryPoint]
      to = "https"
      scheme = "https"
  [entryPoints.https]
    address = ":443"

[certificatesResolvers.lets-encrypt.acme]
  email = "[EMAIL_TO_CHANGE]"
  storage = "/etc/traefik/acme.json"
  [certificatesResolvers.lets-encrypt.acme.tlsChallenge]

[providers.file]
  filename = "/etc/traefik/routes.toml"
  1. routes configuration file
vi /etc/traefik/routes.toml

routes.toml

[http.middlewares.users.basicAuth]
  users = [
    "admin:[PASSWORD_HASH_TO_CHANGE]"
  ]

[http.routers]
  [http.routers.api]
    rule = "Host(`domain.to.change.org`)"
    entrypoints = ["https"]
    middlewares = ["users"]
    service = "api@internal"
    [http.routers.api.tls]
      certResolver = "lets-encrypt"
  1. TLS certificates and logs file
touch /etc/traefik/acme.json
chown traefik:traefik /etc/traefik/acme.json
chmod 600 /etc/traefik/acme.json
touch /var/log/traefik.log
chown traefik:traefik /var/log/traefik.log

System Service

  1. create the Traefik service
vi /lib/systemd/system/traefik.service

traefik.service

[Unit]
Description=Traefik
Documentation=https://docs.traefik.io
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/traefik
AssertPathExists=/etc/traefik/traefik.toml

[Service]
User=traefik
Group=traefik
AmbientCapabilities=CAP_NET_BIND_SERVICE
CapabilityBoundingSet=CAP_NET_BIND_SERVICE

# configure service behavior
Type=notify
ExecStart=/usr/local/bin/traefik --configFile=/etc/traefik/traefik.toml
Restart=on-abnormal
WatchdogSec=1s

# lock down system access
# prohibit any operating system and configuration modification
ProtectSystem=full
# create separate, new (and empty) /tmp and /var/tmp filesystems
PrivateTmp=true
# make /home directories inaccessible
ProtectHome=true
# turns off access to physical devices (/dev/...)
PrivateDevices=false
# make kernel settings (procfs and sysfs) read-only
#ProtectKernelTunables=true
# make cgroups /sys/fs/cgroup read-only
#ProtectControlGroups=true

# allow writing of acme.json
ReadWritePaths=/etc/traefik/acme.json
# depending on log and entrypoint configuration, you may need to allow writing to other paths, too

# limit number of processes in this unit
#LimitNPROC=1

[Install]
WantedBy=multi-user.target
  1. enable and start the service
systemctl enable traefik.service
systemctl daemon-reload
systemctl start traefik.service
systemctl status traefik.service
journalctl --boot -u traefik.service
  1. Access your Traefik dashboard https://domain.to.change.org.

Port Proxy

We use kubectl to securely access to your application services.

kubectl is only required for the Ingress Services.

Check the documentation kubectl for more details

Installation 

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client

Configuration

Configure the ~/.kube/config to access your h8lio cluster where is located the service to expose.

mkdir ~/.kube
vi ~/.kube/config

Copy the content of your .kube/config file. You should use a specific user for the Gateway. Test the access:

kubectl get service

Service

The Gateway provide ingress and egress traffic to service:

  • Egress: a Kubernetes internal service calls an external service through the Gateway to provide to the client a static IP
  • Ingress: a Client calls an interanl Kubernetes services through the Gateway which uses kubectl port forward to act as local to the Gateway service.

You can repeat the following configurations for each service you want to expose to the Gateway.

Egress Service

The Gateway redirects internal service calls to the external service.

If you need to redirect TCP or UDP protocols check the Traefik routers

  1. configure Traefik router and service
vi /etc/traefik/routes.toml

routes.toml

[http.services]
  [http.services.client-service.loadBalancer]
    [[http.services.client-service.loadBalancer.servers]]
      url = "https://service.client.com/"

[http.routers]
  [http.routers.my-service]
    rule = "Host(`my-service.my-gateway.com`)"
    entrypoints = ["https"]
    service = "client-service"
    [http.routers.my-service.tls]
      certResolver = "lets-encrypt"
  1. update your Kubernetes service to call the Gateway https://my-service.my-gateway.com instead of the client address https://service.client.com/.

Ingress Service

Create an instance failover ubuntu service to connect to your Kubernetes internal service. To do so, we are using the kubectl port-forward command.

You may also consider to use socat as a lower level relay

  1. create a bash file to run the port forward command to connect to the service
vi /etc/traefik/myservice.sh
chmod +x /etc/traefik/myservice.sh

myservice.sh

#/bin/bash
kubectl -n [namespace] port-forward service/[myserice] [local-port]:[local-port-in]

Add the argument --address 0.0.0.0 to listen to the local port on any local address and forward to the resource port

  1. create the system service
vi /lib/systemd/system/myservice.service

myservice.service

[Unit]
Description=MyService
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/kubectl
AssertPathExists=/etc/traefik/myservice.sh

[Service]
User=root
ExecStart=/bin/bash /etc/traefik/myservice.sh
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
  1. enable and start the service
systemctl enable myserice.service
systemctl daemon-reload
systemctl start myserice.service
systemctl status myserice.service
journalctl --boot -u myserice.service
  1. add your router and service to Traefik configuration
vi /etc/traefik/routes.toml

routes.toml

[http.services]
  [http.services.my-service.loadBalancer]
    [[http.services.my-service.loadBalancer.servers]]
      url = "http://127.0.0.1:[local-port]/"

[http.routers]
  [http.routers.my-service]
    rule = "Host(`my-service.com`)"
    entrypoints = ["https"]
    service = "my-service"
    [http.routers.my-service.tls]
      certResolver = "lets-encrypt"

Check the Traefik routing & load balancing documentation for more routing options.

  1. Access to your gateway service https://my-service.com.
March 23, 2024
Certificates
Helm