Table of Contents

Use the Micronaut Object Storage API to store files in Google Cloud Storage

Learn how to upload and retrieve files from Google Cloud Storage using the Micronaut Object Storage API

Authors: Álvaro Sánchez-Mariscal

Micronaut Version: 4.4.0

1. Getting Started

In this guide, we will create a Micronaut application written in Java.

It will be a microservice to store, retrieve and delete profile pictures for users.

2. What you will need

To complete this guide, you will need the following:

Some of the following commands use jq

jq is a lightweight and flexible command-line JSON processor

3. Google Cloud Platform

Signup for the Google Cloud Platform

3.1. Cloud SDK

Install the Cloud SDK CLI for your operating system.

Cloud SDK includes the gcloud command-line tool. Run the init command in your terminal:

gcloud init

Log in to your Google Cloud Platform:

gcloud auth login

3.2. Google Cloud Platform Project

Create a new project with a unique name (replace xxxxxx with alphanumeric characters of your choice):

gcloud projects create micronaut-guides-xxxxxx
In GCP, project ids are globally unique, so the id you used above is the one you should use in the rest of this guide.

Change your project:

gcloud config set project micronaut-guides-xxxxxx

If you forget the project id, you can list all projects:

gcloud projects list

4. Solution

We recommend that you follow the instructions in the next sections and create the application step by step. However, you can go right to the completed example.

5. Writing the Application

Create an application using the Micronaut Command Line Interface or with Micronaut Launch.

mn create-app example.micronaut.micronautguide \
    --features=yaml,object-storage-gcp,graalvm \
    --build=maven \
    --lang=java \
    --test=junit
If you don’t specify the --build argument, Gradle is used as the build tool.
If you don’t specify the --lang argument, Java is used as the language.
If you don’t specify the --test argument, JUnit is used for Java and Kotlin, and Spock is used for Groovy.

The previous command creates a Micronaut application with the default package example.micronaut in a directory named micronautguide.

If you use Micronaut Launch, select Micronaut Application as application type and add yaml, object-storage-gcp, and graalvm features.

If you have an existing Micronaut application and want to add the functionality described here, you can view the dependency and configuration changes from the specified features and apply those changes to your application.

Setup IntelliJ IDEA to develop Micronaut Applications.

6. Create a bucket

Use the Google Cloud CLI to create a bucket:

gcloud storage buckets create gs://micronaut-guide-object-storage

Then, configure the bucket name in application.yml:

Since Micronaut Framework 4.0, to use YAML configuration, you have to add the YAML dependency.
src/main/resources/application.yml
micronaut:
  object-storage:
    gcp:
      default:
        bucket: micronaut-guide-object-storage

7. Controller API

Let’s define an interface with the endpoints of the profile pictures microservice:

src/main/java/example/micronaut/ProfilePicturesApi.java
public interface ProfilePicturesApi {

    @Post(uri = "/{userId}", consumes = MediaType.MULTIPART_FORM_DATA) (1)
    HttpResponse upload(CompletedFileUpload fileUpload, String userId, HttpRequest<?> request);

    @Get("/{userId}") (2)
    Optional<HttpResponse<StreamedFile>> download(String userId);

    @Status(HttpStatus.NO_CONTENT) (3)
    @Delete("/{userId}") (4)
    void delete(String userId);
}
1 The @Post annotation maps the method to an HTTP POST request.
2 The @Get annotation maps the method to an HTTP GET request.
3 You can return void in your controller’s method and specify the HTTP status code via the @Status annotation.
4 The @Delete annotation maps the delete method to an HTTP Delete request on /{userId}.

And then, an implementation with the required dependencies:

src/main/java/example/micronaut/ProfilePicturesController.java
@Controller(ProfilePicturesController.PREFIX) (1)
@ExecuteOn(TaskExecutors.BLOCKING) (2)
public class ProfilePicturesController implements ProfilePicturesApi {

    static final String PREFIX = "/pictures";

    private final GoogleCloudStorageOperations objectStorage; (3)
    private final HttpHostResolver httpHostResolver; (4)

    public ProfilePicturesController(GoogleCloudStorageOperations objectStorage, HttpHostResolver httpHostResolver) {
        this.objectStorage = objectStorage;
        this.httpHostResolver = httpHostResolver;
    }

}
1 The class is defined as a controller with the @Controller annotation mapped to the path /pictures.
2 It is critical that any blocking I/O operations (such as fetching the data from the database) are offloaded to a separate thread pool that does not block the Event loop.
3 GoogleCloudStorageOperations is the cloud-specific interface for using object storage.
4 HttpHostResolver allows you to resolve the host for an HTTP

7.1. Upload endpoint

Implement the upload endpoint by receiving the file from the HTTP client via CompletedFileUpload, and the userId path parameter. Upload it to Google Cloud using GoogleCloudStorageOperations, and then return its ETag in an HTTP response header to the client:

src/main/java/example/micronaut/ProfilePicturesController.java
@Override
public HttpResponse<?> upload(CompletedFileUpload fileUpload, String userId, HttpRequest<?> request) {
    String key = buildKey(userId); (1)
    UploadRequest objectStorageUpload = UploadRequest.fromCompletedFileUpload(fileUpload, key); (2)
    UploadResponse<Blob> response = objectStorage.upload(objectStorageUpload);  (3)

    return HttpResponse
            .created(location(request, userId)) (4)
            .header(HttpHeaders.ETAG, response.getETag()); (5)
}

private static String buildKey(String userId) {
    return userId + ".jpg";
}

private URI location(HttpRequest<?> request, String userId) {
    return UriBuilder.of(httpHostResolver.resolve(request))
            .path(PREFIX)
            .path(userId)
            .build();
}
1 The key represents the path under which the file will be stored.
2 You can use any of the UploadRequest static methods to build an upload request.
3 The upload operation returns an UploadResponse, which wraps the cloud-specific SDK response
4 We return the absolute URL of the resource in the Location header.
5 The response object contains some common properties for all cloud vendors, such as the ETag, that is sent in a header to the client.

7.2. Download endpoint

For the download endpoint, simply retrieve the entry from the expected key, and transform it into an StreamedFile:

src/main/java/example/micronaut/ProfilePicturesController.java
@Override
public Optional<HttpResponse<StreamedFile>> download(String userId) {
    String key = buildKey(userId);
    return objectStorage.retrieve(key) (1)
            .map(ProfilePicturesController::buildStreamedFile); (2)
}

private static HttpResponse<StreamedFile> buildStreamedFile(GoogleCloudStorageEntry entry) {
    Blob nativeEntry = entry.getNativeEntry();
    MediaType mediaType = MediaType.of(nativeEntry.getContentType());
    StreamedFile file = new StreamedFile(entry.getInputStream(), mediaType).attach(entry.getKey());
    MutableHttpResponse<Object> httpResponse = HttpResponse.ok()
            .header(HttpHeaders.ETAG, nativeEntry.getEtag()); (3)
    file.process(httpResponse);
    return httpResponse.body(file);
}
1 The retrieve operation returns an ObjectStorageEntry, in this case an GoogleCloudStorageEntry, which allows accessing the Google Cloud-specific Blob
2 We transform the GoogleCloudStorageEntry into an HttpResponse<StreamedFile>.
3 The response contains not only the file, but also an ETag header.
The HTTP client could have used the ETag from the upload operation and send it in a If-None-Match header in the download request to implement caching, which then would have been to be implemented in the download endpoint. But this is beyond the scope of this guide.

7.3. Delete endpoint

For the delete endpoint, all we have to do is invoke the delete method with the expected key:

src/main/java/example/micronaut/ProfilePicturesController.java
@Override
public void delete(String userId) {
    String key = buildKey(userId);
    objectStorage.delete(key);
}

8. Running the Application

To run the application, use the ./mvnw mn:run command, which starts the application on port 8080.

9. Generate a Micronaut Application Native Executable with GraalVM

We will use GraalVM, the polyglot embeddable virtual machine, to generate a native executable of our Micronaut application.

Compiling native executables ahead of time with GraalVM improves startup time and reduces the memory footprint of JVM-based applications.

Only Java and Kotlin projects support using GraalVM’s native-image tool. Groovy relies heavily on reflection, which is only partially supported by GraalVM.

9.1. GraalVM installation

The easiest way to install GraalVM on Linux or Mac is to use SDKMan.io.

Java 17
sdk install java 17.0.8-graal
Java 17
sdk use java 17.0.8-graal

For installation on Windows, or for manual installation on Linux or Mac, see the GraalVM Getting Started documentation.

The previous command installs Oracle GraalVM, which is free to use in production and free to redistribute, at no cost, under the GraalVM Free Terms and Conditions.

Alternatively, you can use the GraalVM Community Edition:

Java 17
sdk install java 17.0.8-graalce
Java 17
sdk use java 17.0.8-graalce

9.2. Native executable generation

To generate a native executable using Maven, run:

./mvnw package -Dpackaging=native-image

The native executable is created in the target directory and can be run with target/micronautguide.

10. Testing

Test the application from the command line.

10.1. Uploading a profile picture

If you want to upload a file larger than 1MB, you need to configure:

Since Micronaut Framework 4.0, to use YAML configuration, you have to add the YAML dependency.
src/main/resources/application.yml
micronaut:
  server:
    multipart:
      max-file-size: 20971520 # 20 * 1024 * 1024 = 20MB

Assuming you have locally a profile picture in a profile.jpg file, you can send it to your application with:

$ curl -i -F 'fileUpload=@profile.jpg' http://localhost:8080/pictures/alvaro

HTTP/1.1 201 Created
location: http://localhost:8080/pictures/alvaro
ETag: "617cb82e296e153c29b34cccf7af0908"
date: Wed, 14 Sep 2022 12:50:30 GMT
connection: keep-alive
transfer-encoding: chunked

Note the Location and ETag headers.

Use the gcloud CLI to verify that the file has been uploaded to an Google Cloud bucket:

gcloud storage ls --recursive gs://micronaut-guide-object-storage

10.2. Download a profile picture

curl http://localhost:8080/pictures/alvaro -O -J

The file will be saved as alvaro.jpg since our download endpoint includes a Content-Disposition: attachment header. Open it to check that it is actually the same image as profile.jpg.

10.3. Delete a profile picture

curl -X DELETE http://localhost:8080/pictures/alvaro

Then, check that the file has actually been deleted:

gcloud storage ls --recursive gs://micronaut-guide-object-storage

11. Cleaning Up

After you’ve finished this guide, you can clean up the resources you created on Google Cloud Platform so you won’t be billed for them in the future. The following sections describe how to delete or turn off these resources.

11.1. Deleting the project

The easiest way to eliminate billing is to delete the project you created for the tutorial.

Deleting a project has the following consequences:

  • If you used an existing project, you’ll also delete any other work you’ve done in the project.

  • You can’t reuse the project ID of a deleted project. If you created a custom project ID that you plan to use in the future, you should delete the resources inside the project instead. This ensures that URLs that use the project ID, such as an appspot.com URL, remain available.

  • If you are exploring multiple tutorials and quickstarts, reusing projects instead of deleting them prevents you from exceeding project quota limits.

11.1.1. Via the CLI

To delete the project using the Cloud SDK, run the following command, replacing YOUR_PROJECT_ID with the project ID:

gcloud projects delete YOUR_PROJECT_ID

11.1.2. Via the Cloud Platform Console

In the Cloud Platform Console, go to the Projects page.

In the project list, select the project you want to delete and click Delete project. After selecting the checkbox next to the project name, click Delete project

In the dialog, type the project ID, and then click Shut down to delete the project.

Deleting or turning off specific resources

You can individually delete or turn off some of the resources that you created during the tutorial.

12. Next Steps

13. License

All guides are released with an Apache license 2.0 license for the code and a Creative Commons Attribution 4.0 license for the writing and media (images…​).