Identity Server - Kubernetes

10 Feb 2022

This example is for local (dev) environment ONLY, not for production environment

In Kubernetes branch (based on Docker branch), there are 3 main changes:

Ingress Controller will be responsible for routing the requests to the corresponding containers
Service endpoint will become [x].default.svc.cluster.local e.g. identity-core-api-service.default.svc.cluster.local
In-Memory Resources, Clients and Users will be used in Identity Server

Similar to dockerization, it is critical to figure out each individual DNS from all the Kubernetes Service, then generate self-signed certificate. Otherwise System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure will be thrown.

Install Docker Desktop
Enable Kubernetes in Docker Desktop Settings

Instal NGINX Ingress Controll

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.1/deploy/static/provider/cloud/deploy.yaml

A few pods should start in the ingress-nginx namespace:

kubectl get pods --namespace=ingress-nginx

Generate self-signed certificate

Add configuration file e.g. “is4-container-cert.conf” to ./Certificates folder

 [req]
 distinguished_name = req_distinguished_name
 req_extensions     = req_ext
 x509_extensions    = v3_ca
 [req_distinguished_name]
 commonName                  = jaylin
 commonName_default          = localhost
 commonName_max              = 64
 [req_ext]
 subjectAltName = @alt_names
 1.3.6.1.4.1.311.84.1.1=ASN1:UTF8String:Something
 [v3_ca]
 subjectAltName = @alt_names
 basicConstraints = critical, CA:false
 keyUsage = keyCertSign, cRLSign, digitalSignature,keyEncipherment
 [alt_names]
 # Local domains
 DNS.1 = api.local
 DNS.2 = client.local
 DNS.3 = is4.local
 # Kubernetes
 DNS.4 = *.default.svc.cluster.local
 # docker-compose
 DNS.5 = localhost
 DNS.6 = 127.0.0.1
 DNS.7 = is4
 DNS.8 = api

Open bash command and navigate to Certificates folder

Run openssl command to generate Linux certificate

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./is4-container-cert.key -out ./is4-container-cert.crt -config ./is4-container-cert.conf

Run openssl command to generate Windows certificate

openssl pkcs12 -export -out ./is4-container-cert.pfx -inkey ./is4-container-cert.key -in ./is4-container-cert.crt

Create TLS secret named k8s-tls-secret in Kubernetes

kubectl create secret tls k8s-tls-secret --key is4-container-cert.key --cert is4-container-cert.crt

To re-create the secret, it needs deleting first, then create again:

kubectl delete secret k8s-tls-secret

Update Dockerfile to copy the pfx to app folder in container

FROM base AS final
......
WORKDIR /app
COPY --from=publish /app/publish .
COPY ["./Certificates/is4-container-cert.pfx", "./is4-container-cert.pfx"]
ENTRYPOINT ["dotnet", "API.dll"]

Add api yml file

 apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: identity-core-api
   labels:
     app: identity-core-api
 spec:
   replicas: 1
   selector:
     matchLabels:
       app: identity-core-api
   template:
     metadata:
       labels:
         app: identity-core-api
     spec:
       containers:
       - name: identity-core-api
         image: api:latest
         imagePullPolicy: IfNotPresent
         env:
           - name: ASPNETCORE_ENVIRONMENT
             value: Development
           - name: ASPNETCORE_URLS
             value: https://+:443
           - name: IdentityServer__Audience
             value: api1
           - name: IdentityServer__Authority
             value: https://identity-core-is4-service.default.svc.cluster.local
           - name: ASPNETCORE_Kestrel__Certificates__Default__Password
             value: P@ssword!
           - name: ASPNETCORE_Kestrel__Certificates__Default__Path
             value: ./is4-container-cert.pfx

 ---

 apiVersion: v1
 kind: Service
 metadata:
   name: identity-core-api-service
 spec:
   type: ClusterIP
   ports:
   - port: 443
     targetPort: 443
     protocol: TCP
     name: https
   selector:
     app: identity-core-api

Environment variables will be almost the same as docker-compose.override.yml, Kestrel Path comes from Dockerfile at #6

Apply to Kubernetes
```
kubectl apply -f api.yml
```
Continue #6, #7 and #8 for Client and IdentityProvider project

Add Ingress Controller to route traffic

In below yml file, a few things need highlighting:

rules.host must exist in tls.hosts array
tls.hosts.secretName comes from #5
annotations.backend-protocol is used to enforce container protocal e.g. HTTPS
annotations.proxy-buffer-size and annotations.proxy-buffer-number are used to resolve 502 Bad Gateway issue from NGINX when redirecting to /signin-oidc from Identity Server

 apiVersion: networking.k8s.io/v1
 kind: Ingress
 metadata:
   name: identity-core-api-ingress
   labels:
     app: identity-core-api-ingress
   annotations:
     kubernetes.io/ingress.class: nginx
     nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
     nginx.ingress.kubernetes.io/proxy-buffer-size: "128k"
     nginx.ingress.kubernetes.io/proxy-buffers-number: "4"
     nginx.ingress.kubernetes.io/use-regex: "true"
     nginx.ingress.kubernetes.io/rewrite-target: /$1
 spec:
   tls:
   - hosts:
     - api.local
     - client.local
     - is4.local
     secretName: k8s-tls-secret
   rules:
   - host: api.local
     http:
       paths:
       - path: /(.*)
         pathType: Prefix
         backend:
           service:
             name: identity-core-api-service
             port:
               number: 443
   - host: client.local
     http:
       paths:
       - path: /(.*)
         pathType: Prefix
         backend:
           service:
             name: identity-core-client-service
             port:
               number: 443
   - host: is4.local
     http:
       paths:
       - path: /(.*)
         pathType: Prefix
         backend:
           service:
             name: identity-core-is4-service
             port:
               number: 443

Apply to Kubernetes
```
   kubectl apply -f ingress.yml
```
Add 3 domains in hosts
- api.local 127.0.0.1
- client.local 127.0.0.1
- is4.local 127.0.0.1
Browse https://client.local
Navigate to Privacy, it will redirect to Identity Server to login (username / password: scott / Password123!)
After login successfully, it will redirect back to Privacy page
Click on Call Api button
API access authorized! should return

Jay Lin

Identity Server - Kubernetes