Table of Contents

Service discovery and distributed configuration with Oracle Cloud Container Engine for Kubernetes (OKE)

Service discovery and distributed configuration in a Micronaut application with Oracle Cloud Container Engine for Kubernetes (OKE)

Authors: Nemanja Mikic, Nemanja Mikic

Micronaut Version: 4.6.3

1. Getting Started

In this guide, we will deploy three microservices on the Oracle Cloud Container Engine for Kubernetes (OKE). We will use Kubernetes Service discovery and Distributed configuration to wire up our microservices.

You will discover how the Micronaut framework eases Kubernetes integration and deployment to OKE.

2. What you will need

To complete this guide, you will need the following:

  • Some time on your hands

  • A decent text editor or IDE (e.g. IntelliJ IDEA)

  • JDK 17 or greater installed with JAVA_HOME configured appropriately

  • Docker.

  • A paid or free trial Oracle Cloud account (create an account at signup.oraclecloud.com)

Your Oracle Cloud account must be a paid account or trial with credits available because there isn’t currently a free-tier option for OKE.

  • We’ll use the OCI command line to authenticate ourselves. If you don’t have it already, install the Oracle Cloud CLI and run oci setup config.

Some of the following commands use jq

jq is a lightweight and flexible command-line JSON processor

3. The Application

Download the complete solution of the Kubernetes and the Micronaut Framework guide. You will use same three microservices (users, orders, and api) as a starting point.

4. Create Oracle Cloud Kubernetes Cluster

We will use Quick create cluster option on OKE to create Kubernetes Cluster. To start browse Quick Kubernetes Clusters (OKE).

create cluster wizard

Choose name for kubernetes cluster, we chose to name it micronaut-k8s.


finish creating cluster

Wait for all resources to be created.


micronaut k8s cluster

Copy Cluster Id (you will need it later).

5. Prepare and Deploy Microservices

We will define some environment variables to make deploying process easier. In OCI_USER_ID store your user OCI, which you can find in your oci configuration file. We will get tenancy namespace from the oci commandline tool and store it in OCI_TENANCY_NAMESPACE. In OCIR_USERNAME store your username in the format <tenancy_namespace>/<username>. We can reuse OCI_TENANCY_NAMESPACE and only edit the <username> part. Store your region code in OCI_REGION for an example "us-phoenix-1". In OCI_CLUSTER_ID store the Cluster Id that you copied earlier. Put your compartment id inside OCI_COMPARTMENT_ID variable.

export OCI_USER_ID="ocid1.user.oc1..aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
export OCI_TENANCY_NAMESPACE=$(oci os ns get | jq .data -r)
export OCIR_USERNAME="$OCI_TENANCY_NAMESPACE/<username>"
export OCI_REGION="<region-key>"
export OCI_CLUSTER_ID="ocid1.cluster.oc1.iad.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
export OCI_COMPARTMENT_ID="ocid1.compartment.oc1..aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"

We have to create AUTH_TOKEN to be able to authenticate to Oracle Cloud Container Registry. Run the following command:

export AUTH_TOKEN=$(oci iam auth-token create --user-id $OCI_USER_ID --description k8s-micronaut | jq -r '.data.token')
Oracle Cloud allows you to have only 2 auth tokens in same time. If you already have 2 of them, please use existing one by exporting it in AUTH_TOKEN variable or delete one that you are not using.

Run the next command to log in to ocir.io (Oracle Cloud Container Registry):

docker login $OCI_REGION.ocir.io -u $OCIR_USERNAME -p $AUTH_TOKEN

5.1. Users Microservice

Edit k8s.yml inside users service.

/users/k8s.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: micronaut-k8s
  name: "users"
spec:
  selector:
    matchLabels:
      app: "users"
  template:
    metadata:
      labels:
        app: "users"
    spec:
      serviceAccountName: micronaut-service
      containers:
        - name: "users"
          image: <region-key>.ocir.io/<tenancy-namespace>/micronaut-k8s/users:latest (1)
          imagePullPolicy: Always (2)
          ports:
            - name: http
              containerPort: 8080
          readinessProbe:
            httpGet:
              path: /health/readiness
              port: 8080
            initialDelaySeconds: 5
            timeoutSeconds: 3
          livenessProbe:
            httpGet:
              path: /health/liveness
              port: 8080
            initialDelaySeconds: 5
            timeoutSeconds: 3
            failureThreshold: 10
      imagePullSecrets:
        - name: ocirsecret (3)
---
apiVersion: v1
kind: Service
metadata:
  namespace: micronaut-k8s
  name: "users"
spec:
  selector:
    app: "users"
  type: NodePort
  ports:
    - protocol: "TCP"
      port: 8080
1 Image name that exists in OCI container registry. Change the <region-key> to your region and change the <tenancy-name> to your tenancy name.
2 Change imagePullPolicy to Always
3 Secret needed to pull images from Oracle container registry

Create container repository in your compartment.

export USERS_REPOSITORY=$(oci artifacts container repository create --display-name micronaut-k8s/users --compartment-id $OCI_COMPARTMENT_ID | jq .data.id -r)

Build a docker image of the users service with the name users.

Ensure that Docker images are constructed for the correct CPU architecture. For instance, if you’re utilizing Apple Silicon (aarch64), you can consider modifying the DOCKER_DEFAULT_PLATFORM environment variable to the value linux/amd64. Alternatively, you have the option to use ARM (arch64) instances within your Kubernetes cluster.

Tag an existing users microservice image.

docker tag users:latest $OCI_REGION.ocir.io/$OCI_TENANCY_NAMESPACE/micronaut-k8s/users:latest

Push tagged users microservice image to remote repository.

docker push $OCI_REGION.ocir.io/$OCI_TENANCY_NAMESPACE/micronaut-k8s/users:latest

5.2. Orders Microservice

Edit k8s.yml inside orders service.

/orders/k8s.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: micronaut-k8s
  name: "orders"
spec:
  selector:
    matchLabels:
      app: "orders"
  template:
    metadata:
      labels:
        app: "orders"
    spec:
      serviceAccountName: micronaut-service
      containers:
        - name: "orders"
          image: <region-key>.ocir.io/<tenancy-namespace>/micronaut-k8s/orders:latest (1)
          imagePullPolicy: Always (2)
          ports:
            - name: http
              containerPort: 8080
          readinessProbe:
            httpGet:
              path: /health/readiness
              port: 8080
            initialDelaySeconds: 5
            timeoutSeconds: 3
          livenessProbe:
            httpGet:
              path: /health/liveness
              port: 8080
            initialDelaySeconds: 5
            timeoutSeconds: 3
            failureThreshold: 10
      imagePullSecrets:
        - name: ocirsecret (3)
---
apiVersion: v1
kind: Service
metadata:
  namespace: micronaut-k8s
  name: "orders"
spec:
  selector:
    app: "orders"
  type: NodePort
  ports:
    - protocol: "TCP"
      port: 8080
1 Image name that exists in OCI container registry. Change the <region-key> to your region and change the <tenancy-name> to your tenancy name.
2 Change imagePullPolicy to Always
3 Secret needed to pull images from Oracle container registry

Create container repository in your compartment.

export ORDERS_REPOSITORY=$(oci artifacts container repository create --display-name micronaut-k8s/orders --compartment-id $OCI_COMPARTMENT_ID | jq .data.id -r)

Build a docker image of the orders service with the name orders.

Ensure that Docker images are constructed for the correct CPU architecture. For instance, if you’re utilizing Apple Silicon (aarch64), you can consider modifying the DOCKER_DEFAULT_PLATFORM environment variable to the value linux/amd64. Alternatively, you have the option to use ARM (arch64) instances within your Kubernetes cluster.

Tag an existing orders microservice image.

docker tag orders:latest $OCI_REGION.ocir.io/$OCI_TENANCY_NAMESPACE/micronaut-k8s/orders:latest

Push tagged orders microservice image to remote repository.

docker push $OCI_REGION.ocir.io/$OCI_TENANCY_NAMESPACE/micronaut-k8s/orders:latest

5.3. API Microservice

Edit k8s.yml inside api service.

/api/k8s.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: micronaut-k8s
  name: "api"
spec:
  selector:
    matchLabels:
      app: "api"
  template:
    metadata:
      labels:
        app: "api"
    spec:
      serviceAccountName: micronaut-service
      containers:
        - name: "api"
          image: <region-key>.ocir.io/<tenancy-namespace>/micronaut-k8s/api:latest (1)
          imagePullPolicy: Always (2)
          ports:
            - name: http
              containerPort: 8080
          readinessProbe:
            httpGet:
              path: /health/readiness
              port: 8080
            initialDelaySeconds: 5
            timeoutSeconds: 3
          livenessProbe:
            httpGet:
              path: /health/liveness
              port: 8080
            initialDelaySeconds: 5
            timeoutSeconds: 3
            failureThreshold: 10
      imagePullSecrets:
        - name: ocirsecret (3)
---
apiVersion: v1
kind: Service
metadata:
  namespace: micronaut-k8s
  name: "api"
  annotations: (4)
    oci.oraclecloud.com/load-balancer-type: "lb"
    service.beta.kubernetes.io/oci-load-balancer-shape: "flexible"
    service.beta.kubernetes.io/oci-load-balancer-shape-flex-min: "10"
    service.beta.kubernetes.io/oci-load-balancer-shape-flex-max: "10"
spec:
  selector:
    app: "api"
  type: LoadBalancer
  ports:
    - protocol: "TCP"
      port: 8080
1 Image name that exists in OCI container registry. Change the <region-key> to your region and change the <tenancy-name> to your tenancy name.
2 Change imagePullPolicy to Always
3 Secret needed to pull images from Oracle container registry
4 Metadata annotations for OCI Load Balancer

Create container repository in your compartment.

export API_REPOSITORY=$(oci artifacts container repository create --display-name micronaut-k8s/api --compartment-id $OCI_COMPARTMENT_ID | jq .data.id -r)

Build a docker image of the api service with the name api.

Ensure that Docker images are constructed for the correct CPU architecture. For instance, if you’re utilizing Apple Silicon (aarch64), you can consider modifying the DOCKER_DEFAULT_PLATFORM environment variable to the value linux/amd64. Alternatively, you have the option to use ARM (arch64) instances within your Kubernetes cluster.

Tag an existing api microservice image.

docker tag api:latest $OCI_REGION.ocir.io/$OCI_TENANCY_NAMESPACE/micronaut-k8s/api:latest

Push tagged api microservice image to remote repository.

docker push $OCI_REGION.ocir.io/$OCI_TENANCY_NAMESPACE/micronaut-k8s/api:latest

5.4. Deploy Services to OKE

Create a directory for kubectl configuration.

mkdir -p $HOME/.kube

Generate kubectl configuration for authentication to OKE.

oci ce cluster create-kubeconfig --cluster-id $OCI_CLUSTER_ID --file $HOME/.kube/config --region $OCI_REGION --token-version 2.0.0  --kube-endpoint PUBLIC_ENDPOINT

Set KUBECONFIG to the created config file. This variable is consumed by kubectl.

export KUBECONFIG=$HOME/.kube/config

Deploy the auth.yml file that we created in the Kubernetes and the Micronaut Framework guide.

kubectl apply -f auth.yml

Create ocirsecret secret that will be used for authentication to Oracle Cloud Container Registry. This is needed because OKE needs credentials to be able to pull microservices images.

kubectl create secret docker-registry ocirsecret --docker-server=$OCI_REGION.ocir.io --docker-username=$OCIR_USERNAME --docker-password=$AUTH_TOKEN --namespace=micronaut-k8s

Run the next command to deploy for users microservice:

kubectl apply -f users/k8s.yml

Run the next command to deploy for orders microservice:

kubectl apply -f orders/k8s.yml

Run the next command to deploy for api microservice:

kubectl apply -f api/k8s.yml

6. Test integration between applications deployed on OKE

Run the next command to check status of the pods and make sure that all of them have the status "Running":

kubectl get pods -n=micronaut-k8s
NAME                      READY   STATUS    RESTARTS   AGE
api-6fb4cd949f-kxxx8      1/1     Running   0          2d1h
orders-595887ddd6-6lzp4   1/1     Running   0          2d1h
users-df6f78cd7-lgnzx     1/1     Running   0          2d1h

Run the next command to check the status of the microservices:

kubectl get services -n=micronaut-k8s
NAME         TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)             AGE
api          LoadBalancer   10.96.70.48    129.159.92.209   8080:31690/TCP      2d1h
orders       NodePort       10.96.94.130   <none>           8080:31245/TCP      2d1h
users        NodePort       10.96.34.174   <none>           8080:30790/TCP      2d1h
If EXTERNAL-IP is in <pending> state wait a couple of seconds and then run command again.

Run the next command to retrieve the URL of the api microservice:

export API_URL=http://$(kubectl get svc api -n=micronaut-k8s -o json | jq -r '.status.loadBalancer.ingress[0].ip'):8080

Run a cURL command to create a new user via the api microservice:

curl -X "POST" "$API_URL/api/users" -H 'Content-Type: application/json; charset=utf-8' -d '{ "first_name": "Nemanja", "last_name": "Mikic", "username": "nmikic" }'
{"id":1,"username":"nmikic","first_name":"Nemanja","last_name":"Mikic"}

Run a cURL command to a new order via the api microservice:

curl -X "POST" "$API_URL/api/orders" -H 'Content-Type: application/json; charset=utf-8' -d '{ "user_id": 1, "item_ids": [1,2] }'
{"id":1,"user":{"first_name":"Nemanja","last_name":"Mikic","id":1,"username":"nmikic"},"items":[{"id":1,"name":"Banana","price":1.5},{"id":2,"name":"Kiwi","price":2.5}],"total":4.0}

Run a cURL command to list created orders:

curl "$API_URL/api/orders" -H 'Content-Type: application/json; charset=utf-8'
[{"id":1,"user":{"first_name":"Nemanja","last_name":"Mikic","id":1,"username":"nmikic"},"items":[{"id":1,"name":"Banana","price":1.5},{"id":2,"name":"Kiwi","price":2.5}],"total":4.0}]

We can try to place an order for a user who doesn’t exist (with id 100). Run a cURL command:

curl -X "POST" "$API_URL/api/orders" -H 'Content-Type: application/json; charset=utf-8' -d '{ "user_id": 100, "item_ids": [1,2] }'
{"message":"Bad Request","_links":{"self":[{"href":"/api/orders","templated":false}]},"_embedded":{"errors":[{"message":"User with id 100 doesn't exist"}]}}

7. Cleaning Up

To delete all resources that were created in this guide run next command.

kubectl delete namespaces micronaut-k8s

Run next command to delete OKE cluster.

oci ce cluster delete --cluster-id $OCI_CLUSTER_ID --force

Run the next command to delete micronaut-k8s/users artifacts container repository:

oci artifacts container repository delete --repository-id $USERS_REPOSITORY --force

Run the next command to delete micronaut-k8s/orders artifacts container repository:

oci artifacts container repository delete --repository-id $ORDERS_REPOSITORY --force

Run the next command to delete micronaut-k8s/api artifacts container repository:

oci artifacts container repository delete --repository-id $API_REPOSITORY --force

8. Next steps

Explore more features with Micronaut Guides.

Read more about Micronaut Kubernetes module.

Read more about Oracle Container Engine for Kubernetes (OKE)

9. License

All guides are released with an Apache license 2.0 license for the code and a Creative Commons Attribution 4.0 license for the writing and media (images…​).