Service discovery and distributed configuration with Google Kubernetes Engine (GKE)

Service discovery and distributed configuration in a Micronaut application with Google Kubernetes Engine (GKE)

Authors: Nemanja Mikic, Nemanja Mikic

Micronaut Version: 4.6.3

1. Getting Started

In this guide, we will deploy three microservices on the Google Kubernetes Engine (GKE). We will use Kubernetes Service discovery and Distributed configuration to wire up our microservices.

You will discover how the Micronaut framework eases Kubernetes integration and deployment to GKE.

2. Costs

This guide uses paid services; you may need to enable Billing in Google Cloud to complete some steps in this guide.

3. What you will need

To complete this guide, you will need the following:

Some time on your hands
A decent text editor or IDE (e.g. IntelliJ IDEA)
JDK 17 or greater installed with JAVA_HOME configured appropriately
Docker. Some of the following commands use jq

jq is a lightweight and flexible command-line JSON processor

You need a Google Cloud Platform (GCP) account and a GCP project.

4. The Application

Download the complete solution of the Kubernetes and the Micronaut Framework guide. You will use same three microservices (users, orders, and api) as a starting point.

5. Google Cloud Platform

Signup for the Google Cloud Platform

5.1. Cloud SDK

Install the Cloud SDK CLI for your operating system.

Cloud SDK includes the gcloud command-line tool. Run the init command in your terminal:

gcloud init

gcloud auth login

5.2. Google Cloud Platform Project

Create a new project with a unique name (replace xxxxxx with alphanumeric characters of your choice):

gcloud projects create micronaut-guides-xxxxxx

In GCP, project ids are globally unique, so the id you used above is the one you should use in the rest of this guide.

Change your project:

gcloud config set project micronaut-guides-xxxxxx

If you forget the project id, you can list all projects:

gcloud projects list

5.3. Enable billing

Ensure you have a billing account created, and if not create one via the Google Cloud Platform Console.

To get a list of your billing accounts, run:

❯ gcloud beta billing accounts list
ACCOUNT_ID            NAME                  OPEN   MASTER_ACCOUNT_ID
XXXXXX-XXXXXX-XXXXXX  My Billing Account    True

You can then attach a billing account to your project:

> gcloud beta billing projects link micronaut-guides-xxxxxxx --billing-account=XXXXXX-XXXXXX-XXXXXX
billingAccountName: billingAccounts/XXXXXX-XXXXXX-XXXXXX
billingEnabled: true
name: projects/micronaut-guides-xxxxxxx/billingInfo
projectId: micronaut-guides-xxxxxxx

5.4. Enable the Google Cloud Container Registry API

We need somewhere to store our docker images, so we need to enable the Google Cloud Container Registry API for your project via the Google Cloud CLI:

gcloud services enable containerregistry.googleapis.com

5.5. Configure Google Cloud Docker

Run auth configure-docker via the Google Cloud CLI:

> gcloud auth configure-docker

Adding credentials for all GCR repositories.
WARNING: A long list of credential helpers may cause delays running 'docker build'. We recommend passing the registry name to configure only the registry you are using.
After update, the following will be written to your Docker config file located at [~/.docker/config.json]:
 {
  "credHelpers": {
    "gcr.io": "gcloud",
    "us.gcr.io": "gcloud",
    "eu.gcr.io": "gcloud",
    "asia.gcr.io": "gcloud",
    "staging-k8s.gcr.io": "gcloud",
    "marketplace.gcr.io": "gcloud"
  }
}

5.6. Enable the Google Cloud Kubernetes Engine API

To use Kubernetes Engine API we have to enable it for your project via the Google Cloud CLI:

gcloud services enable container.googleapis.com

5.7. Create GKE cluster

In this guide we will create a cluster named micronaut-k8s and we will use two n1-standard-2 instances as GKE nodes. Everything will be deployed on your default Google Cloud region. You can check your default region by running:

> gcloud config get-value compute/region

Create micronaut-k8s kubernetes cluster:

gcloud container clusters create micronaut-k8s --machine-type n1-standard-2 --num-nodes 2 --disk-type pd-standard

6. Prepare and Deploy Microservices

Set the GCP_PROJECT_ID environment variable to store your GCP project id.

export GCP_PROJECT_ID="$(gcloud config get-value project)"

6.1. Users Microservice

Export users microservice image repository to the USERS_REPOSITORY environment variable.

export USERS_REPOSITORY="gcr.io/$GCP_PROJECT_ID/users"

Edit k8s.yml inside users service.

/users/k8s.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: micronaut-k8s
  name: "users"
spec:
  selector:
    matchLabels:
      app: "users"
  template:
    metadata:
      labels:
        app: "users"
    spec:
      serviceAccountName: micronaut-service
      containers:
        - name: "users"
          image: gcr.io/<gcp-project-id>/users:latest (1)
          imagePullPolicy: Always (2)
          ports:
            - name: http
              containerPort: 8080
          readinessProbe:
            httpGet:
              path: /health/readiness
              port: 8080
            initialDelaySeconds: 5
            timeoutSeconds: 3
          livenessProbe:
            httpGet:
              path: /health/liveness
              port: 8080
            initialDelaySeconds: 5
            timeoutSeconds: 3
            failureThreshold: 10
      imagePullSecrets:
        - name: gcpsersecret (3)
---
apiVersion: v1
kind: Service
metadata:
  namespace: micronaut-k8s
  name: "users"
spec:
  selector:
    app: "users"
  type: NodePort
  ports:
    - protocol: "TCP"
      port: 8080

1	Repository URI that we created in GCR container registry. Change `<project-id>` to your google project id. If you are using UNIX based OS you can achieve this by running `sed -i'' -e "s/<gcp-project-id>/$GCP_PROJECT_ID/" users/k8s.yml`
2	Change imagePullPolicy to Always
3	Secret needed to pull images from Google container registry

Build a docker image of the users service with the name users.

Ensure that Docker images are constructed for the correct CPU architecture. For instance, if you’re utilizing Apple Silicon (aarch64), you can consider modifying the DOCKER_DEFAULT_PLATFORM environment variable to the value linux/amd64. Alternatively, you have the option to use ARM (arch64) instances within your Kubernetes cluster.

Tag an existing users microservice image.

docker tag users:latest ${USERS_REPOSITORY}:latest

Push tagged users microservice image to remote repository.

docker push ${USERS_REPOSITORY}:latest

6.2. Orders Microservice

Export orders microservice image repository to the ORDERS_REPOSITORY environment variable.

export ORDERS_REPOSITORY="gcr.io/$GCP_PROJECT_ID/orders"

Edit k8s.yml inside orders service.

/orders/k8s.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: micronaut-k8s
  name: "orders"
spec:
  selector:
    matchLabels:
      app: "orders"
  template:
    metadata:
      labels:
        app: "orders"
    spec:
      serviceAccountName: micronaut-service
      containers:
        - name: "orders"
          image: gcr.io/<gcp-project-id>/orders:latest (1)
          imagePullPolicy: Always (2)
          ports:
            - name: http
              containerPort: 8080
          readinessProbe:
            httpGet:
              path: /health/readiness
              port: 8080
            initialDelaySeconds: 5
            timeoutSeconds: 3
          livenessProbe:
            httpGet:
              path: /health/liveness
              port: 8080
            initialDelaySeconds: 5
            timeoutSeconds: 3
            failureThreshold: 10
      imagePullSecrets:
        - name: gcpsersecret (3)
---
apiVersion: v1
kind: Service
metadata:
  namespace: micronaut-k8s
  name: "orders"
spec:
  selector:
    app: "orders"
  type: NodePort
  ports:
    - protocol: "TCP"
      port: 8080

1	Repository URI that we created in GCR container registry. Change `<project-id>` to your google project id. If you are using UNIX based OS you can achieve this by running `sed -i'' -e "s/<gcp-project-id>/$GCP_PROJECT_ID/" orders/k8s.yml`
2	Change imagePullPolicy to Always
3	Secret needed to pull images from Google container registry

Build a docker image of the orders service with the name orders.

Tag an existing orders microservice image.

docker tag orders:latest ${ORDERS_REPOSITORY}:latest

Push tagged orders microservice image to remote repository.

docker push ${ORDERS_REPOSITORY}:latest

6.3. API Microservice

Export api microservice image repository to the API_REPOSITORY environment variable.

export API_REPOSITORY="gcr.io/$GCP_PROJECT_ID/api"

Edit k8s.yml inside api service.

/api/k8s.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: micronaut-k8s
  name: "api"
spec:
  selector:
    matchLabels:
      app: "api"
  template:
    metadata:
      labels:
        app: "api"
    spec:
      serviceAccountName: micronaut-service
      containers:
        - name: "api"
          image: gcr.io/<gcp-project-id>/api:latest (1)
          imagePullPolicy: Always (2)
          ports:
            - name: http
              containerPort: 8080
          readinessProbe:
            httpGet:
              path: /health/readiness
              port: 8080
            initialDelaySeconds: 5
            timeoutSeconds: 3
          livenessProbe:
            httpGet:
              path: /health/liveness
              port: 8080
            initialDelaySeconds: 5
            timeoutSeconds: 3
            failureThreshold: 10
      imagePullSecrets:
        - name: gcpsersecret (3)
---
apiVersion: v1
kind: Service
metadata:
  namespace: micronaut-k8s
  name: "api"
spec:
  selector:
    app: "api"
  type: LoadBalancer
  ports:
    - protocol: "TCP"
      port: 8080

1	Repository URI that we created in GCR container registry. Change `<project-id>` to your google project id. If you are using UNIX based OS you can achieve this by running `sed -i'' -e "s/<gcp-project-id>/$GCP_PROJECT_ID/" api/k8s.yml`
2	Change imagePullPolicy to Always
3	Secret needed to pull images from Google container registry

Build a docker image of the api service with the name api.

Tag an existing api microservice image.

docker tag api:latest ${API_REPOSITORY}:latest

Push tagged api microservice image to remote repository.

docker push ${API_REPOSITORY}:latest

6.4. Deploy Services to GKE

Create a directory for kubectl configuration.

mkdir -p $HOME/.kube

Install kubectl component.

gcloud components install kubectl

Generate kubectl configuration for authentication to GKE.

gcloud container clusters get-credentials micronaut-k8s

Set the value of KUBECONFIG to the created config file. This variable is consumed by kubectl.

export KUBECONFIG=$HOME/.kube/config

Deploy the auth.yml file that we created in the Kubernetes and the Micronaut Framework guide.

kubectl apply -f auth.yml

Create a secret named gcpsersecret that will be used for authentication to Google Cloud Container Registry. This is needed because GKE needs credentials to be able to pull microservices images.

kubectl create secret docker-registry gcpsersecret --docker-server=gcr.io \
--docker-username=oauth3accesstoken \
--docker-password="$(gcloud auth print-access-token)" \
--docker-email=your@email.com

Run the next command to deploy the users microservice:

kubectl apply -f users/k8s.yml

Run the next command to deploy the orders microservice:

kubectl apply -f orders/k8s.yml

Run the next command to deploy the api microservice:

kubectl apply -f api/k8s.yml

7. Test integration between applications deployed on GKE

Run the next command to check status of the pods and make sure that all of them have the status "Running":

kubectl get pods -n=micronaut-k8s

NAME                      READY   STATUS    RESTARTS   AGE
api-6b884d9c88-hxsdk      1/1     Running   0          21s
orders-54465f845c-k7z5t   1/1     Running   0          24s
users-d8b46cf48-wq6sm     1/1     Running   0          29s

Run the next command to check the status of the microservices:

kubectl get services -n=micronaut-k8s

NAME     TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)          AGE
api      LoadBalancer   10.112.14.49   <redacted>       8080:31299/TCP   37s
orders   NodePort       10.112.2.71    <none>           8080:30613/TCP   40s
users    NodePort       10.112.2.227   <none>           8080:32718/TCP   44s

If EXTERNAL-IP is in <pending> state wait a couple of seconds and then run command again.

Run the next command to retrieve the URL of the api microservice:

export API_URL=http://$(kubectl get svc api -n=micronaut-k8s -o json | jq -r '.status.loadBalancer.ingress[0].ip'):8080

Run a cURL command to create a new user via the api microservice:

curl -X "POST" "$API_URL/api/users" -H 'Content-Type: application/json; charset=utf-8' -d '{ "first_name": "Nemanja", "last_name": "Mikic", "username": "nmikic" }'

{"id":1,"username":"nmikic","first_name":"Nemanja","last_name":"Mikic"}

Run a cURL command to a new order via the api microservice:

curl -X "POST" "$API_URL/api/orders" -H 'Content-Type: application/json; charset=utf-8' -d '{ "user_id": 1, "item_ids": [1,2] }'

{"id":1,"user":{"first_name":"Nemanja","last_name":"Mikic","id":1,"username":"nmikic"},"items":[{"id":1,"name":"Banana","price":1.5},{"id":2,"name":"Kiwi","price":2.5}],"total":4.0}

Run a cURL command to list created orders:

curl "$API_URL/api/orders" -H 'Content-Type: application/json; charset=utf-8'

[{"id":1,"user":{"first_name":"Nemanja","last_name":"Mikic","id":1,"username":"nmikic"},"items":[{"id":1,"name":"Banana","price":1.5},{"id":2,"name":"Kiwi","price":2.5}],"total":4.0}]

We can try to place an order for a user who doesn’t exist (with id 100). Run a cURL command:

curl -X "POST" "$API_URL/api/orders" -H 'Content-Type: application/json; charset=utf-8' -d '{ "user_id": 100, "item_ids": [1,2] }'

{"message":"Bad Request","_links":{"self":[{"href":"/api/orders","templated":false}]},"_embedded":{"errors":[{"message":"User with id 100 doesn't exist"}]}}

8. Cleaning Up

To delete all resources that were created in this guide run the next command.

kubectl delete namespaces micronaut-k8s

Run the next command to delete users artifacts container repository:

gcloud container images delete ${USERS_REPOSITORY} --force-delete-tags --quiet

Run the next command to delete orders artifacts container repository:

gcloud container images delete ${ORDERS_REPOSITORY} --force-delete-tags --quiet

Run the next command to delete api artifacts container repository:

gcloud container images delete ${API_REPOSITORY} --force-delete-tags --quiet

Run the next command to delete the micronaut-guide GKE cluster:

gcloud container clusters delete micronaut-k8s

8.1. Deleting the project

The easiest way to eliminate billing is to delete the project you created for the tutorial.

Deleting a project has the following consequences:

If you used an existing project, you’ll also delete any other work you’ve done in the project.
You can’t reuse the project ID of a deleted project. If you created a custom project ID that you plan to use in the future, you should delete the resources inside the project instead. This ensures that URLs that use the project ID, such as an appspot.com URL, remain available.
If you are exploring multiple tutorials and quickstarts, reusing projects instead of deleting them prevents you from exceeding project quota limits.

8.1.1. Via the CLI

To delete the project using the Cloud SDK, run the following command, replacing YOUR_PROJECT_ID with the project ID:

gcloud projects delete YOUR_PROJECT_ID

8.1.2. Via the Cloud Platform Console

In the Cloud Platform Console, go to the Projects page.

In the project list, select the project you want to delete and click Delete project. After selecting the checkbox next to the project name, click Delete project

In the dialog, type the project ID, and then click Shut down to delete the project.

Deleting or turning off specific resources

You can individually delete or turn off some of the resources that you created during the tutorial.

9. Next Steps

Explore more features with Micronaut Guides.

Google Kubernetes Engine (GKE)
Google Cloud Container Registry
Google Cloud Artifact Registry
Micronaut Google Cloud Platform (GCP) integration.
Read more about Micronaut Kubernetes module.

10. License

All guides are released with an Apache license 2.0 license for the code and a Creative Commons Attribution 4.0 license for the writing and media (images…).

Table of Contents