Landlord can reduce the cost of running microservices

When running Spring's PetClinic Spring Boot reference application, my experimental Landlord project appears to save 5 times the memory for the second instance of it. My terse observations show that PetClinic requires about 250MiB of resident memory when running as a standalone Java 8 application. When running two instances of it hosted by Landlord, approximately 342.2 MiB of resident memory is consumed in total i.e. Landlord + PetClinic + PetClinic.

These observations illustrate that sharing JVM machinery has positive impacts on hosting services. While running multiple PetClinic applications on the same machine isn't a particularly realistic scenario, running multiple Spring Boot microservices on the same machine is. Landlord could, therefore, provide a significant cost reduction in the hosting of any JVM based microservices.

Introduction

I've been experimenting with a project I created named "Landlord". The idea of the project is to promote a multi-tenant JVM scenario with the goal of saving memory. Our standard usage of the JVM isn't particularly kind regarding memory usage with a simple Hello World application consuming about 35MiB of memory out of the box. This figure is about 10 times what you get with a native target. For example, the same program built via Scala Native will consume about 4.5MiB of memory. Note that we're talking about resident memory - not the JVM heap (which will be much less than that).

I thought that it'd be fun to run the standard Spring Boot PetClinic application within Landlord in order to get a feel for Landlord's cost savings.

Steps

1. Clone PetClinic and then perform a ./mvnw package in order to get a standalone jar.
2. Let's determine the smallest amount of RAM that PetClinic can get away with comfortably. For this, I kept running the following java command until I stopped seeing Out Of Memory (OOM) exceptions from Java.
   
   java \
     -XX:+UseSerialGC \
     -Xss512k \
     -XX:-TieredCompilation \
     -XX:CICompilerCount=1 \
     -XX:MaxRAM=80m \
     -cp ${pwd}/target/spring-petclinic-1.5.1.jar \
     org.springframework.boot.loader.JarLauncher
   Note that the serial GC and other options are the results of my previous investigations in order to keep JVM resident memory usage down. There are pros/cons, but the above configuration is useful when deploying to small devices such as network gateways. That said, if you can get your application running well with the above options, you're likely to run even better at a larger scale, and potentially save money given a lesser number of machines to host your required load. One more option I'd normally use is -Xss256k, but I observed a stack overflow so it seems that Spring likes lots of stack.
3. When I got to that point, I then profiled the process in order to observe the JVM heap used vs allocated vs the limit. With the above configuration, PetClinic appeared to function and I didn't observe OOM, but observing the JVM heap revealed:
   
   Used: 37MB Alloc.: 38MB Limit: 38MB
   
   That feels a bit too close to comfort and seemed to be causing the GC to kick in each time that I refreshed the page.
   
   So, I ended up specifying -XX:MaxRAM=100m. This then yielded:
   
   Used: 46MB Alloc.: 48MB Limit: 48MB
4. Now, even though I've specified max ram as 100MiB, this turns only to be a starting point for the JVM on how it should size its memory regions. On OS X, if I use the Activity Monitor's inspection for a process (double-click on a process in its memory tab) then the following is reported: Real Memory: 259.3 MB (that's its Resident Set Size - RSS). So, even though we stated that we didn't want more than 100MiB, this is not a limit and the JVM can take more. Apparently, this is a JVM implementation consideration. I assume (hope) that the -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap are much stronger than -XX:MaxRAM, or perhaps this is just some OS X JVM implementation thing... I do wish that the JVM was more predictable in this regard.
5. Time for Landlord. Let's start it up with the same options:
   
   JAVA_OPTS="-XX:+UseSerialGC -Xss512k -XX:-TieredCompilation -XX:CICompilerCount=1 -XX:MaxRAM=100m" \
   daemon/target/universal/stage/bin/landlordd \
     --process-dir-path=/tmp/a
6. Let's observe its memory via the Activity Monitor: Real Memory: 133.8 MB.
7. Now to load PetClinic. For this, we need a script to load it into Landlord:
   
   
   #!/bin/sh
   ( \
     printf "l" && \
     echo "-cp spring-petclinic-1.5.1.jar org.springframework.boot.loader.JarLauncher" && \
     tar -c -C $(pwd)/target spring-petclinic-1.5.1.jar && \
     cat <&0 \
     ) | nc -U /var/run/landlord/landlordd.sock
8. Upon executing the above script: Real Memory: 278.3 MB. That's just 19MiB more than when it was run as a standalone application. Connecting to Landlord via YourKit shows the heap as:
   
   Used: 47MB Alloc.: 48MB Limit: 48MB
   
   ...which is quite similar to before. There doesn't appear to be any GC thrashing either. This shouldn't be any great surprise. Its thread and heap usage is quite minimal.
   
   Now, using Landlord for hosting just one application is not really going to give you any great benefit. Landlord's benefit's kick in when multiple applications are run. Let's run another PetClinic within Landlord.
9. First, so that PetClinic's ports don't clash, declare a random port to bind to within src/main/resources/application.properties:
   server.port=0
10. Package the app via ./mvnw package and then invoke the Landlord script for PetClinic as before.
    
    Unfortunately, this doesn't work... the embedded Tomcat of Spring Boot throws an exception:
    
    
    Caused by: java.lang.Error: factory already defined
    at java.net.URL.setURLStreamHandlerFactory(URL.java:1112) ~[na:1.8.0_131]
    
    While we're at it, there are a couple of places where Spring declares shutdown hooks. Landlord warns of this with the following output:
    
    

    Warning: Shutdown hooks are not applicable within landlord as many applications reside in the same JVM. Declare a `public static void trap(int signal)` for trapping signals and catch `SecurityException` around your shutdown hook code.

    
    Clearly, there are some changes in order for an application to run within a multi-tenant environment. The PetClinic/Spring Boot environment is built to assume that it is running within its own process. Going forward, I believe it would be easy for the Spring Boot project to cater for these multi-tenancy concerns. For now, we change the PetClinic application to use Jetty instead of Tomcat. To do this, we follow the recipe of Spring Boot's documentation.
11. Once the Jetty version is running, I observed the native Java process with a: Real Memory: 245.9 MB. Under Landlord, the same package + Landlord: Real Memory: 290.9 MB. A bit more of a difference to the 278.3 MiB for the Tomcat based package + Landlord, but who knows what the JVM is doing... perhaps we can assume this as being some JVM anomaly.
    
    Now, if we try to run another PetClinic within Landlord then we get an OOM memory error. Clearly, we need more JVM not having very much at hand before. Let's re-run Landlord with -XX:MaxRAM=120m (20MiB more overall).
    
    We now get a problem given a clash of JMX endpoints, so we turn them off (src/main/resources/application.properties: spring.jmx.enabled=false) and try again.
    
    Real Memory: 342.2 MB
    
    That's just 51.3MiB additional RSS to run what would be 245.9 MiB to run an additional PetClinic outside of Landlord. Landlord, at least in this simple observation, is reducing the memory cost by about a factor of 5.

This has been a simple test and I welcome feedback on improving it.