Table of Contents

Publish Micronaut application logs to Microsoft Azure Monitor Logs

Learn how to send your Micronaut application Logback logs to Microsoft Azure Monitor Logs.

Authors: Burt Beckwith

Micronaut Version: 4.6.3

1. Getting Started

In this guide, we will create a Micronaut application written in Groovy.

This guide describes how to create a Micronaut application that publishes logs to Azure Monitor Logs. Typically, application logs are written to stdout and to files, but the Micronaut® Logging module supports multiple logging frameworks such as Logback, and logging to various other appenders, to email, a database, or other destinations.

2. What you will need

To complete this guide, you will need the following:

  • Some time on your hands

  • A decent text editor or IDE (e.g. IntelliJ IDEA)

  • JDK 21 or greater installed with JAVA_HOME configured appropriately

  • Azure CLI installed with local access to Azure configured by running az login.

  • A paid or free trial Microsoft Azure account (create an account at Azure account signup) NOTE: Paid services are used as part of this guide. To follow the tutorial, you will need a paid Azure account or a trial account with credits available.

3. Solution

We recommend that you follow the instructions in the next sections and create the application step by step. However, you can go right to the completed example.

  • Download and unzip the source Create an application using the Micronaut Command Line Interface or with Micronaut Launch. NOTE: If you don’t specify the --build argument, Gradle with the Kotlin DSL is used as the build tool.
    If you don’t specify the --lang argument, Java is used as the language.
    If you don’t specify the --test argument, JUnit is used for Java and Kotlin, and Spock is used for Groovy. The previous command creates a Micronaut application with the default package example.micronaut in a directory named micronautguide. == Writing the Application

Create an application using the Micronaut Command Line Interface or with Micronaut Launch.

mn create-app example.micronaut.micronautguide --build=maven --lang=groovy
If you don’t specify the --build argument, Gradle with the Kotlin DSL is used as the build tool.
If you don’t specify the --lang argument, Java is used as the language.
If you don’t specify the --test argument, JUnit is used for Java and Kotlin, and Spock is used for Groovy.

The previous command creates a Micronaut application with the default package example.micronaut in a directory named micronautguide.

Setup IntelliJ IDEA to develop Micronaut Applications.

3.1. Micronaut Azure Logging Dependency

Add this dependency to your build to add Azure logging support:

pom.xml
<dependency>
    <groupId>io.micronaut.azure</groupId>
    <artifactId>micronaut-azure-logging</artifactId>
    <scope>compile</scope>
</dependency>

4. LogController

Create a controller which enables you to send POST requests to log messages:

src/main/groovy/example/micronaut/LogController.groovy
package example.micronaut

import groovy.util.logging.Slf4j
import io.micronaut.http.annotation.Body
import io.micronaut.http.annotation.Controller
import io.micronaut.http.annotation.Post

@Slf4j('LOG')
@Controller
class LogController {

    @Post('/log')
    void log(@Body String message) {
        LOG.info(message)
    }
}

5. Create Azure Cloud Resources

You will create a resource group, a Log Analytics Workspace and a table in the workspace to hold log event data, a Data Collection Endpoint, and a Data Collection Rule.

5.1. Create a Resource Group

We recommend that you create a new resource group for this guide, but you can use an existing resource group instead.

Run the az group create command to create a resource group named micronautguides in the eastus region:

az group create --location eastus --name micronautguides

If you prefer using a region geographically closer to you, run az account list-locations to list all available regions.

5.2. Add the monitor-control-service CLI Extension

Run the az extension add command to add the monitor-control-service CLI Extension:

az extension add -n monitor-control-service

5.3. Add the log-analytics CLI Extension

Run the az extension add command to add the log-analytics CLI Extension:

az extension add -n log-analytics

5.4. Create a Log Analytics Workspace

Run the az monitor log-analytics workspace create command to create a Log Analytics Workspace:

az monitor log-analytics workspace create \
   --name micronautworkspace \
   --resource-group micronautguides

The response should look like this:

{
   "createdDate": "2024-08-19T20:11:18.4002398Z",
   "customerId": "c222d080-5b14-43e7-b648-71c2b358dc74",
   ...
   "name": "micronautworkspace",
   "provisioningState": "Creating",
   ...
}

Save the value of the customerId attribute. This is your workspace GUID, and it will be needed later.

5.5. Create a Data Collection Endpoint

Run the az monitor data-collection endpoint create command to create a Data Collection Endpoint:

az monitor data-collection endpoint create \
   --data-collection-endpoint-name micronautCollectionEndpoint \
   --public-network-access Enabled \
   --resource-group micronautguides

The response should look like this:

{
  ...
  "logsIngestion": {
    "endpoint": "https://micronautcollectionendpoint-xxxx.eastus-1.ingest.monitor.azure.com"
  },
  ...
  "name": "micronautCollectionEndpoint",
  ...
  "type": "Microsoft.Insights/dataCollectionEndpoints"
}

Save the value of the logsIngestion.endpoint attribute. This is the logs ingestion endpoint URL, and it will be needed later.

5.6. Create a table in your Log Analytics workspace

Run the az monitor log-analytics workspace table create command to create a table in your Log Analytics workspace to hold the log records:

az monitor log-analytics workspace table create \
   --name MicronautTable_CL \
   --workspace-name micronautworkspace \
   --columns EventTimestamp=long Source=string Subject=string Data=string TimeGenerated=datetime \
   --resource-group micronautguides

5.7. Create a Data Collection Rule

Create a file named "dcr.json" with this content:

{
  "location": "<region>",
  "properties": {
    "streamDeclarations": {
      "Custom-MicronautTable": {
        "columns": [
          {
            "name": "TimeGenerated",
            "type": "datetime"
          },
          {
            "name": "EventTimestamp",
            "type": "long"
          },
          {
            "name": "Source",
            "type": "string"
          },
          {
            "name": "Subject",
            "type": "string"
          },
          {
            "name": "Data",
            "type": "string"
          }
        ]
      }
    },
    "destinations": {
      "logAnalytics": [
        {
          "workspaceResourceId": "/subscriptions/<subscription-id>/resourceGroups/micronautguides/providers/microsoft.operationalinsights/workspaces/micronautworkspace",
          "name": "micronautLogDestination"
        }
      ]
    },
    "dataFlows": [
      {
        "streams": [
          "Custom-MicronautTable"
        ],
        "destinations": [
          "micronautLogDestination"
        ],
        "transformKql": "source | extend TimeGenerated = now()",
        "outputStream": "Custom-MicronautTable_CL"
      }
    ]
  }
}

Replace <region> with the name of the region you’re using, e.g. "eastus", and replace <subscription-id> with your Azure Subscription ID.

Run the az monitor data-collection rule create command to create a data collection rule:

az monitor data-collection rule create \
   --name micronautCollectionRule \
   --location <region> \
   --endpoint-id /subscriptions/<subscription-id>/resourceGroups/micronautguides/providers/Microsoft.Insights/dataCollectionEndpoints/micronautCollectionEndpoint \
   --rule-file "path/to/dcr.json" \
   --resource-group micronautguides

Replace <region> with the name of the region you’re using, e.g. "eastus", replace <subscription-id> with your Azure Subscription ID, and replace "path/to/dcr.json" with the file location of the file you created.

The response should look like this:

{
  "dataCollectionEndpointId": "/subscriptions/fe053...",
  "dataFlows": [
    {
      "destinations": [
        "micronautLogDestination"
      ],
      "outputStream": "Custom-MicronautTable_CL",
      "streams": [
        "Custom-MicronautTable"
      ],
      "transformKql": "source | extend TimeGenerated = now()"
    }
  ],
  ...
  "immutableId": "dcr-e7ebfceb7df24631b64d7ae880eb8ada",
  "location": "eastus",
  "name": "micronautCollectionRule",
  ...
  "type": "Microsoft.Insights/dataCollectionRules"
}

Save the value of the immutableId attribute. This is the rule ID, and it will be needed later.

5.8. Authorize Sending Logs

Authorize sending logs by assigning the Monitoring Metrics Publisher role to yourself:

az role assignment create \
   --role "Monitoring Metrics Publisher" \
   --assignee <email> \
   --scope "/subscriptions/<subscription-id>/resourceGroups/micronautguides/providers/Microsoft.Insights/dataCollectionRules/micronautCollectionRule"

replacing <email> with the email address associated with your account, and <subscription-id> with your Azure Subscription ID.

Note that it can take a few minutes for the role grant to propagate.

6. Configure Appender and Application Configuration

6.1. Logback Appender

Edit logback.xml to look like the following:

src/main/resources/logback.xml
<configuration debug='false'>

    <!--
    You can un-comment the STDOUT appender and <appender-ref ref='STDOUT'/> in
    the cloud appender to log to STDOUT as the 'emergency' appender.
    -->

    <!--
    <appender name='STDOUT' class='ch.qos.logback.core.ConsoleAppender'>
        <encoder>
            <pattern>%cyan(%d{HH:mm:ss.SSS}) %gray([%thread]) %highlight(%-5level) %magenta(%logger{36}) - %msg%n</pattern>
        </encoder>
    </appender>
    -->

    <appender name='AZURE' class='io.micronaut.azure.logging.AzureAppender'>
        <!-- <appender-ref ref='STDOUT'/> -->
        <!-- <blackListLoggerName>com.foo.Bar</blackListLoggerName> -->
        <encoder class='ch.qos.logback.core.encoder.LayoutWrappingEncoder'>
            <layout class='ch.qos.logback.contrib.json.classic.JsonLayout'>
                <jsonFormatter class='io.micronaut.azure.logging.AzureJsonFormatter'/>
            </layout>
        </encoder>
    </appender>

    <root level='INFO'>
        <appender-ref ref='AZURE'/>
    </root>

</configuration>

6.1.1. Azure Appender

The appender named "AZURE" will publish application log events to the Azure Monitor Logs table you configured earlier.

6.1.2. Emergency Appender

Since the Azure appender queues log messages and then writes them remotely, there are situations which might result in log events not getting remoted correctly. To address such scenarios you can configure the emergency appender to preserve those messages. To do this, uncomment the "STDOUT" appender element, and the appender-ref element in the "AZURE" appender element.

In the example the emergency appender writes to the console, but any valid Logback appender can be used.

6.1.3. Blacklisting Loggers

The AzureAppender supports blacklisting loggers by specifying the logger name(s) to exclude in blackListLoggerName elements.

6.1.4. Additional configuration options

You can customize your JsonLayout with additional parameters that are described in the Logback JsonLayout documentation.

6.2. Set Application Configuration Properties

Update application.properties as follows:

src/main/resources/application.properties
micronaut.application.name=default
(1)
azure.logging.data-collection-endpoint=<endpoint>
(2)
azure.logging.rule-id=<ruleid>
(3)
azure.logging.stream-name=Custom-MicronautTable
(4)
azure.logging.enabled=true
1 Replace <endpoint> with the logs ingestion endpoint URL that you saved earlier
2 Replace <ruleid> with the rule ID that you saved earlier
3 Set the value of the azure.logging.stream-name property with the name of the workspace table you created, "Custom-MicronautTable"
4 Change to false to disable the appender, e.g., per-environment

Having configured the appender and the application configuration, you can proceed to run the application, publishing the logs.

7. Running the application

To run the application, use the ./mvnw mn:run command, which starts the application on port 8080.

Send some cURL requests to test logging:

curl -id '{"message":"your message here"}' \
     -H "Content-Type: application/json" \
     -X POST http://localhost:8080/log

Run the az monitor log-analytics query command to retrieve log events that were pushed while running your application:

az monitor log-analytics query \
   --workspace <workspace-guid> \
   --analytics-query "MicronautTable_CL | where TimeGenerated > ago(1h)"

Replace <workspace-guid> with the workspace GUID that you saved earlier.

Note that it can take a few minutes for log entries be available.

8. Cleaning Up

After you’ve finished this guide, you can clean up the resources you created on Azure so you won’t take the risk of being billed because of them in the future.

Delete the resource group and all of its resources with:

az group delete --name micronautguides

or run these commands to delete resources individually:

az monitor data-collection rule delete --name micronautCollectionRule --resource-group micronautguides

az monitor log-analytics workspace table delete --name MicronautTable_CL --workspace-name micronautworkspace --resource-group micronautguides

az monitor data-collection endpoint delete --name micronautCollectionEndpoint --resource-group micronautguides

az monitor log-analytics workspace delete --workspace-name micronautworkspace --resource-group micronautguides --force

az group delete --name micronautguides

9. Next Steps

Explore more features with Micronaut Guides.

Read more about the Micronaut Azure Logging integration.

10. License

All guides are released with an Apache license 2.0 license for the code and a Creative Commons Attribution 4.0 license for the writing and media (images…​).