Ready/Set/Go Kubernetes+Traefik+LetsEncrypt on ARM at Oracle OCI
A few weeks ago Oracle announce the general availability of ARM CPUs at Oracle cloud, here the announcement:
The demo which I will show you is available using Free Tier resources!!!
Ready — Create an Oracle Kubernetes Cluster (OKE)
Set — Choose ARM Ampere CPU nodes
Go — Confirm and create the cluster
Once you do above three steps a cluster creation will take a few minutes, after that you can see three VMs, as We choose in my demo, at the Instances pane
And that's all, you have a Kubernetes cluster running using ARM processors, lets deploy now a Traefik Ingress Controller and a sample app. To connect to the cluster there are several options, I am using VSCode Kubernetes extension and getting local access from OCI information as is available at the Quick Start link (bottom left menu of cluster information), required tools are oci and kubectl command line.
Once I have access to my cluster using VSCode I can see all the information about it as well I can use a command line to send commands
Lets deploy Traefik using Helm from command line:
$ helm repo add traefik https://helm.traefik.io/traefik
$ helm repo update
$ helm install traefik traefik/traefik --values traefik-values.yaml
$ kubectl port-forward $(kubectl get pods --selector "app.kubernetes.io/name=traefik" --output=name) 9000:9000Last step allow you to see a fancy Traefik Dashboard with the Ingress Controller status as is:
But what happen under the hood, first lets analyze traefik-values.yaml
podSecurityContext:
fsGroup: 65532
is required to avoid permission problem on persistent volume, see Traefik forum
Apart from parameters for using LetsEncrypt certificates, OKE cluster automatically detect that you deploy a new Ingress Controller and configure a Load Balancer to send HTTP/HTTPs traffic to your cluster, you can see it at Load Balancer pane:
OK, We have Traefik working and a Load Balancer configured, check if it works using HTTP deploying a WhoAmi app:
Note that I am exposing my App using DNS entry provided by Free DNS service NoIP, you could use whatever a DNS service you want or directly editing your localhost file, but for testing a LetsEncrypt generated cert a public DNS entry is required with A entry associated to the Load Balancer public IP.
$ kubectl apply -f whoami.yml
$ kubectl get pods -n default
NAME READY STATUS RESTARTS AGE
app-v1-77c7d7c577-jwfz5 1/1 Running 0 14h
traefik-89984f845-pd29m 1/1 Running 0 14h
$ curl -v http://whoami-oci.hopto.org
* Trying 129.159.86.238:80...
* TCP_NODELAY set
* Connected to whoami-oci.hopto.org (129.159.86.238) port 80 (#0)
> GET / HTTP/1.1
> Host: whoami-oci.hopto.org
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Length: 385
< Content-Type: text/plain; charset=utf-8
< Date: Tue, 22 Jun 2021 13:53:20 GMT
<
Hostname: app-v1-77c7d7c577-jwfz5
IP: 127.0.0.1
IP: 10.244.0.131
RemoteAddr: 10.244.1.4:57122
GET / HTTP/1.1
Host: whoami-oci.hopto.org
User-Agent: curl/7.68.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.244.0.0
X-Forwarded-Host: whoami-oci.hopto.org
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-89984f845-pd29m
X-Real-Ip: 10.244.0.0* Connection #0 to host whoami-oci.hopto.org left intact
Finally check if LetsEncrypt Certificate is generated for our App using whoami-tls.yaml Ingress Route:
$ kubectl apply -f whoami-tls.yml
$ kubectl exec -ti $(kubectl get pods --selector "app.kubernetes.io/name=traefik" --output=name) -- /bin/sh
/ $ grep main /data/acme.json
"domain": {
"main": "whoami-oci.hopto.org"
/ $ exit
$ curl -v https://whoami-oci.hopto.org
* Trying 129.159.86.238:443...
* TCP_NODELAY set
* Connected to whoami-oci.hopto.org (129.159.86.238) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=whoami-oci.hopto.org
* start date: Jun 21 22:08:00 2021 GMT
* expire date: Sep 19 22:07:59 2021 GMT
* subjectAltName: host "whoami-oci.hopto.org" matched cert's "whoami-oci.hopto.org"
* issuer: C=US; O=Let's Encrypt; CN=R3
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55b517417e10)
> GET / HTTP/2
> Host: whoami-oci.hopto.org
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200
< content-type: text/plain; charset=utf-8
< date: Tue, 22 Jun 2021 14:10:58 GMT
< content-length: 387
<
Hostname: app-v1-77c7d7c577-jwfz5
IP: 127.0.0.1
IP: 10.244.0.131
RemoteAddr: 10.244.1.4:60382
GET / HTTP/1.1
Host: whoami-oci.hopto.org
User-Agent: curl/7.68.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.244.1.1
X-Forwarded-Host: whoami-oci.hopto.org
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: traefik-89984f845-pd29m
X-Real-Ip: 10.244.1.1* Connection #0 to host whoami-oci.hopto.org left intact
Intermediate step logging into Traefik pod is not necessary, but it help you if want to check if LetsEncrypt cert generation works OK.
It works perfect, lets rock now deployment other apps using ARM Free Tier!!!