Microservices Distributed Tracing with Jaeger and the Micronaut Framework

Use Jaeger distributed tracing to investigate the behaviour of your Micronaut applications.

Authors: Sergio del Amo

Micronaut Version: 3.2.7

1. Getting Started

In this guide, we will integrate Jaeger in a Micronaut application composed of three microservices.

As on-the-ground microservice practitioners are quickly realizing, the majority of operational problems that arise when moving to a distributed architecture are ultimately grounded in two areas: networking and observability. It is a much larger problem to network and debug a set of intertwined distributed services versus a single monolithic application.

You will discover how the Micronaut framework eases Jaeger integration.

2. What you will need

To complete this guide, you will need the following:

  • Some time on your hands

  • A decent text editor or IDE

  • JDK 1.8 or greater installed with JAVA_HOME configured appropriately

3. Solution

We recommend that you follow the instructions in the next sections and create the application step by step. However, you can go right to the completed example.

4. Writing the Application

To learn more about this sample application, read the Consul and the Micronaut Framework - Microservices Service Discovery guide. The application contains three microservices.

  • bookcatalogue - This returns a list of books. It uses a domain consisting of a book name and an ISBN.

  • bookinventory - This exposes an endpoint to check whether a book has sufficient stock to fulfil an order. It uses a domain consisting of a stock level and an ISBN.

  • bookrecommendation - This consumes previous services and exposes an endpoint, which recommends book names that are in stock.

The bookcatalogue service consumes endpoints exposed by the other services. The following image illustrates the application flow:

flow

A request to bookrecommendation (http://localhost:8080/books) triggers several requests through our microservices mesh.

4.1. Enable annotation Processing

If you use Java or Kotlin and IntelliJ IDEA, make sure to enable annotation processing.

annotationprocessorsintellij

5. Jaeger and the Micronaut Framework

5.1. Install Jaeger via Docker

The quickest way to start Jaeger is via Docker:

docker run -d --name jaeger \
  -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
  -p 5775:5775/udp \
  -p 6831:6831/udp \
  -p 6832:6832/udp \
  -p 5778:5778 \
  -p 16686:16686 \
  -p 14268:14268 \
  -p 14250:14250 \
  -p 9411:9411 \
  jaegertracing/all-in-one:latest

5.2. Book catalogue

Add tracing dependency.

pom.xml
<dependency>
    <groupId>io.micronaut</groupId>
    <artifactId>micronaut-tracing</artifactId>
    <scope>compile</scope>
</dependency>

To send tracing spans to Jaeger, the minimal configuration requires you to add the following dependency:

pom.xml
<dependency>
    <groupId>io.jaegertracing</groupId>
    <artifactId>jaeger-thrift</artifactId>
    <scope>runtime</scope>
</dependency>

Append to bookcatalogue service application.yml the following snippet:

bookcatalogue/src/main/resources/application.yml
tracing:
  jaeger:
    enabled: true
    sampler:
      probability: 1 (1)
1 Trace 100% of requests.

In production, you will probably want to trace a smaller percentage of the requests. However, in order to keep this guide simple, we set it to trace 100%.

Disable distributed tracing in tests:

bookcatalogue/src/test/resources/application-test.yml
tracing:
  jaeger:
    enabled: false

5.3. Book inventory

Add tracing dependency.

pom.xml
<dependency>
    <groupId>io.micronaut</groupId>
    <artifactId>micronaut-tracing</artifactId>
    <scope>compile</scope>
</dependency>

To send tracing spans to Jaeger, the minimal configuration requires you to add the following dependency:

pom.xml
<dependency>
    <groupId>io.jaegertracing</groupId>
    <artifactId>jaeger-thrift</artifactId>
    <scope>runtime</scope>
</dependency>

Append to bookinventory service application.yml the following snippet:

bookinventory/src/main/resources/application.yml
tracing:
  jaeger:
    enabled: true
    sampler:
      probability: 1 (1)
1 Trace 100% of requests.

In production, you will probably want to trace a smaller percentage of the requests. However, in order to keep this guide simple, we set it to trace 100%.

Disable distributed tracing in tests:

bookinventory/src/test/resources/application-test.yml
tracing:
  jaeger:
    enabled: false

Annotate the method with @ContinueSpan and the parameter with @SpanTag:

bookinventory/src/main/kotlin/example/micronaut/BooksController.kt
package example.micronaut

import io.micronaut.http.MediaType
import io.micronaut.http.annotation.Controller
import io.micronaut.http.annotation.Get
import io.micronaut.http.annotation.Produces
import io.micronaut.tracing.annotation.ContinueSpan
import io.micronaut.tracing.annotation.SpanTag
import java.util.Optional
import javax.validation.constraints.NotBlank

@Controller("/books")
open class BooksController {

    @Produces(MediaType.TEXT_PLAIN)
    @Get("/stock/{isbn}")
    @ContinueSpan (1)
    open fun stock(@SpanTag("stock.isbn") @NotBlank isbn: String): Boolean? {  (2)
        return bookInventoryByIsbn(isbn).map { (_, stock) -> stock > 0 }.orElse(null)
    }

    private fun bookInventoryByIsbn(isbn: String): Optional<BookInventory> {
        if (isbn == "1491950358") {
            return Optional.of(BookInventory(isbn, 4))

        } else if (isbn == "1680502395") {
            return Optional.of(BookInventory(isbn, 0))
        }
        return Optional.empty()
    }
}
1 The @ContinueSpan annotation will continue an existing span, wrapping the method call or reactive type.
2 The @SpanTag annotation can be used on method arguments to include the value of each argument within a Span’s tags. When you use @SpanTag you need either to annotate the method with @NewSpan or @ContinueSpan.

5.4. Book recommendation

Add tracing dependency.

pom.xml
<dependency>
    <groupId>io.micronaut</groupId>
    <artifactId>micronaut-tracing</artifactId>
    <scope>compile</scope>
</dependency>

To send tracing spans to Jaeger, the minimal configuration requires you to add the following dependency:

pom.xml
<dependency>
    <groupId>io.jaegertracing</groupId>
    <artifactId>jaeger-thrift</artifactId>
    <scope>runtime</scope>
</dependency>

Append to bookrecommendation service application.yml the following snippet:

bookrecommendation/src/main/resources/application.yml
tracing:
  jaeger:
    enabled: true
    sampler:
      probability: 1 (1)
1 Trace 100% of requests.

In production, you will probably want to trace a smaller percentage of the requests. However, in order to keep this guide simple, we set it to trace 100%.

Disable distributed tracing in tests:

bookrecommendation/src/test/resources/application-test.yml
tracing:
  jaeger:
    enabled: false

6. Running the Application

Run bookcatalogue microservice:

To run the application, execute ./mvnw mn:run.

...
14:28:34.034 [main] INFO  io.micronaut.runtime.Micronaut - Startup completed in 499ms. Server Running: http://localhost:8081

Run bookinventory microservice:

To run the application, execute ./mvnw mn:run.

...
14:31:13.104 [main] INFO  io.micronaut.runtime.Micronaut - Startup completed in 506ms. Server Running: http://localhost:8082

Run bookrecommendation microservice:

To run the application, execute ./mvnw mn:run.

...
14:31:57.389 [main] INFO  io.micronaut.runtime.Micronaut - Startup completed in 523ms. Server Running: http://localhost:8080

You can run a cURL command to test the whole application:

curl http://localhost:8080/books
[{"name":"Building Microservices"}

You can then navigate to http://localhost:16686 to access the Jaeger UI.

The previous request generates a traces composed by 9 spans.

jaegerui

In the previous image, you can see that:

  • Whenever a Micronaut HTTP client executes a new network request, it creates a new span.

  • Whenever a Micronaut server receives a request, it creates a new span.

The stock.isbn tags that we configured with @SpanTag are present.

Moreover, you can see the requests to bookinventory are done in parallel.

7. Next Steps

As you have seen in this guide, without any annotations, you get distributed tracing up and running fast with the Micronaut framework.

The Micronaut framework includes several annotations to give you more flexibility. We introduced the @ContinueSpan and @SpanTag annotations. Also, you have at your disposal the @NewSpan annotation, which will create a new span, wrapping the method call or reactive type.

Make sure to read more about Tracing with Jaeger in the Micronaut framework.

8. Help with the Micronaut Framework

Object Computing, Inc. (OCI) sponsored the creation of this Guide. A variety of consulting and support services are available.