Documenting Spring Boot Microservices with Swagger

It’s a fairly well known fact that many developers don’t like to write documentation, often muttering things like “the code is the documentation” in a half hearted manner suggesting even they don’t believe themselves. Recently I was looking to write a Microservice, so I wanted to also look at ways in which we could make nice easy to use documentation in a consistent manner.

Swagger

This led me to look at Swagger. It’s a way to produce elegant and powerful interactive documentation on your REST API without having to write pages of documentation. And when used with annotations and Spring Boot, truly the code *is* the documentation.

Spring Boot Example

I’m completely taken with Spring Boot already. It takes a lot of the complexity out of getting an initial application up and running, and allows you to add in features easily. In my application, I produced a REST API for getting identities out of LDAP, to support both a lookup on the logged in persons identity, and also other identities.

The POM file

We still use Maven (although it’s getting more and more tempting to switch to Gradle). My pom file contains the following dependencies:

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>    
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-rest</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency> 
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>com.jayway.jsonpath</groupId>
        <artifactId>json-path</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.ldap</groupId>
        <artifactId>spring-ldap</artifactId>
        <version>1.2.1</version>
        <type>jar</type>
    </dependency>
    <dependency>
        <groupId>com.mangofactory</groupId>
        <artifactId>swagger-springmvc</artifactId>
        <version>1.0.2</version>
        <type>jar</type>
    </dependency>
</dependencies>

Most of the dependencies are covering the Spring Boot side of things, REST support, web application and LDAP/JSON support. Note though the bottom dependency, which is including support for Swagger Spring MVC.

The Controller

The controller is fairly simple, it maps two URLs. The key thing of note is the @ApiOperation annotation, where we describe what the method does.

@RestController
public class UserLookupController {
    
    @Autowired
    LdapService ldapService;
    
    @ApiOperation(value="Get the currently logged in users details",notes="Uses the remote user logged in")
    @RequestMapping(value="/my",method=RequestMethod.GET)
    public @ResponseBody Person getMyDetails(HttpServletRequest request) throws ServletException
    {
        if (request.getRemoteUser()==null)
        {
            throw new ServletException("Remote user is null.");
        }
        return ldapService.getPerson(request.getRemoteUser());
    }

    @ApiOperation(value="Get a specific users details",notes="Requires uid of user to look up")
    @RequestMapping(value="/id/{uid}",method=RequestMethod.GET)
    public @ResponseBody Person getUserDetails(@PathVariable("uid") String uid)
    {
        return ldapService.getPerson(uid);
    }
    
}

Swagger Config

We also need a config class to tell Swagger what to do, and also provide high level documentation about what the Service is providing.

@Configuration
@EnableSwagger
@EnableAutoConfiguration
public class SwaggerConfig {
    
    private SpringSwaggerConfig springSwaggerConfig;
 
    @Autowired
    public void setSpringSwaggerConfig(SpringSwaggerConfig springSwaggerConfig) {
        this.springSwaggerConfig = springSwaggerConfig;
    }
    
    @Bean
    public SwaggerSpringMvcPlugin customImplementation() {
        return new SwaggerSpringMvcPlugin(this.springSwaggerConfig)
                //Root level documentation
                .apiInfo(new ApiInfo(
                        "Central Authorisation Service JSON API",
                        "This service provides a JSON representation of the LDAP identity data held in the Central Authorisation Service",
                        null,
                        null,
                        null,
                        null
                ))
                .useDefaultResponseMessages(false)
                //Map the specific URL patterns into Swagger
                .includePatterns("/id/.*","/my");
    }
    
}

The Application Class

The Application class is simple, and just hooks into our LDAP config:

@Configuration
@ComponentScan("uk.ac.ed.ca")
@EnableAutoConfiguration
public class Application {
    
    public static void main(String[] args)
    {
        SpringApplication.run(Application.class, args);
    }
    
    @Bean
    @ConfigurationProperties(prefix="ldap.contextSource")
    public LdapContextSource contextSource() {
        LdapContextSource contextSource = new LdapContextSource();
        return contextSource;
    }

    @Bean
    public LdapTemplate ldapTemplate(ContextSource contextSource) {
        return new LdapTemplate(contextSource);
    }
    
}


HTML UI

Finally, we add in a static set of HTML/CSS/JS to cover the API. You can get the static pages from https://github.com/swagger-api/swagger-ui. Put them in your project under (src/main/)resources/static and they’ll automatically get mapped into the application.

The end result

Swagger UI screenshot

The end result is a service which also provides the following documentation. As the documentation is interactive you can also try to call the id service and see what kind of response it gives. Very neat, very powerful, and a very easy way for us to provide API documentation!

ETag and JSON data

Recently as part of an update project for our university portal MyEd (which runs on uPortal) there was an emphasis on moving our content to more client driven access to data. We wanted to separate out the data and presentation a bit more, and also cut down on the load and traffic which a big single server-side render would produce.

We wanted to use JSON as the data format as it is nice and lightweight, and easy to parse with existing Javascript libraries (like JQuery). We then wrote in static URLs into the uPortal portlets which would allow the currently authenticated user (and them alone) to access their own data.

Our portal is under a reasonably heavy concurrent load at any given time, so we wanted to explore caching of data to make sure we make any client side calls perform well under load.

Cache Headers versus ETag

Cache Headers are used to tell a browser to not re-request an object from the server until a certain time, typically by setting an expiry date. This avoids any traffic going to the server at all, which reduces load but can mean that changes to data are missed because the cache expiry date has not been reached.

ETagging is different, in that an ETag value is set in the header, for example:

ETag: "asb227873hva23456n"

When the browser re-requests data from the url it passes the ETag back to the server in an If-None-Match header, e.g:

If-None-Match: "asb227873hva23456n"

The server then uses the ETag to decide what to do, either to send an HTTP Status Code of 304 not modified (typically with a very short response), or refresh the data and return new information back to the client. This reduces the bandwidth required, but more importently allows the server to decide how and when to respond with fresh data.

In order to get the best performance, you would in most situations use both caching and ETag in order to limit high frequency client traffic to the server but also allow the server to mitigate load using the ETag. We found when using both that behaviour in our uPortal server alongside our load balancer led to unexpected results , so we opted to initially use ETagging only.

(As to why our load balancer was causing unexpected caching behaviour we’ll have to investigate later, and potentially write up another post in and of itself!)

Portlet modifications

So in the portlet itself (which is written in Java), we set the JSON data controller method to add in an ETag.

final String eTag = getETag(data);
final Date expiry = new Date(System.currentTimeMillis() + MAX_AGE_MILLIS);
        
session.setAttribute(SESSION_ATTR_CACHE_ETAG, eTag);
session.setAttribute(SESSION_ATTR_CACHE_EXPIRY, expiry);
response.setStatus(HttpServletResponse.SC_OK);
response.setHeader("Cache-Control", "must-revalidate");
response.setHeader("ETag", eTag);

Finally, we then added a check in the method for the ETag coming from the If-None-Match header:

final String ifNoneMatch = request.getHeader("If-None-Match");
 final String existingETag = (String)session.getAttribute(SESSION_ATTR_CACHE_ETAG);
 final Date existingExpiry = (Date)session.getAttribute(SESSION_ATTR_CACHE_EXPIRY);
 if (null != ifNoneMatch
 && null != existingETag
 && null != existingExpiry
 && ifNoneMatch.equals(existingETag)
 && System.currentTimeMillis() < existingExpiry.getTime())
 {
 response.setStatus(HttpServletResponse.SC_NOT_MODIFIED);
 return null;
 }

The above code checks the passed in ETag, compares it with the one stored in the user session, and additionally compares it with an expiry tag, then responds with a NOT MODIFIED 304 if the tags match and the expiry hasn’t passed. The response is null which means it doesn’t have to query the underlying dataset to respond, and therefore the response time and the bandwidth used are dramatically reduced.

Creating a ‘Bootiful’ new Visitor Registration System

The time has come to replace the venerable Visitor Registration System that has served the University now for quite some time. In June the team established for the COM011 project, destined to fulfil this task, ran user workshops and collected some 480 ‘user stories’ from interested parties around the University who use the incumbent system. User Stories are the cornerstone of the Agile methodology which has been chosen for the project.

Using Agile will allow the team to adapt to changing requirements and produce an end product that reflects the will of the users. What better way to complement this than by also introducing a new light weight, flexible, development tool that encourages rapid development and prototyping into IS apps technology stack? Spring Boot, (or just ‘Boot’), which was released in April, is the culmination of an effort by the huge Java/Spring community to prove the speed and ease with which Java applications can be created. This technology was showcased, to massive excitement, when it was shown that Boot could deliver an entire running web application in a tweet.

Boot has been used as the basis of the new Visitor Registration project; we now have a framework in our code repository that can be reused by anyone who wants to quickly setup a fully functional web application, with responsive front end, security enabled for various user roles, Rest endpoints, Soap endpoints and backend Oracle integration. And all of this functionality is fully unit and integration tested – in keeping with the goal of Agile that software quality should always be paramount. The new Visitor Registration System, using cutting edge technologies, will hopefully stand the test of time as well as its predecessor.

Oracle SOA vs Spring – SOAP Web Service throughput testing

We are soon going to embark on a major project to introduce enterprise notification handling at the University. Part of that will be the ability to handle a large number of messages in an efficient and robust manner. We already use Oracle SOA Suite here at the University, but wanted to test its throughput versus a lighter approach, that of Java and the Spring framework.

The scenarios

We chose four scenarios to test:

  • Basic assign, parameter passed in is passed out as response
  • DB Write , parameter passed in is written to Oracle Database
  • DB Read, parameter passed in is used to read value from Oracle Database
  • DB Read/Write, parameter passed in is written to Oracle Database, then read back out again

Testing constraints

We then applied the same constraints to both Oracle SOA and Java:

  • A connection pool must be used with the same settings (min 1, max 10 connections)
  • The same table structure/setup must be used with both technologies
  • We use the same back-end Oracle database
  • Testing would be done using a SOAP UI load test

For Oracle SOA, we set up a simple composite which tested the various features.

For Java Spring, we used Spring Boot, Spring Web Services, and Spring JPA.

The results

The results were as follows (total timings are rounded up to the nearest second):

Oracle SOA

500 calls 2000 calls 5000 calls
Assign 2 sec | 293 ms avg 6 sec | 504 ms avg 16 sec | 593 ms avg
Write 3 sec | 1284 ms avg 10 sec | 861 ms avg 29 sec | 1094 ms avg
Read 2 sec | 389 ms avg 9 sec | 838 ms avg 21 sec | 803 ms avg
Write Read 3 sec | 1038 ms avg 18 sec | 1644 ms avg 36 sec | 1403 ms avg

Java (Spring framework)

500 calls 2000 calls 5000 calls
Assign 1 sec | 101 ms avg 1 sec | 82 ms avg 2 sec | 72 ms avg
Write 1 sec | 112 ms avg 2 sec | 232 ms avg 5 sec | 203 ms avg
Read 1 sec | 73 ms avg 1 sec | 116 ms avg 3 sec | 116 ms avg
Write Read 1 sec | 271 ms avg 3 sec | 256 ms avg 6 sec | 234 ms avg

Conclusions

It is clear that the Java Spring solution is giving better throughput times,, and that is especially evident when we increase the load. However it would be unfair to use throughput times alone in looking at what Oracle SOA provides. It gives for example an “out of the box” message resilience and  support for automated message retry that would have to be coded in when using Java even with the benefit of Spring frameworks. However, Spring can provide a very useful high throughput entry point into Oracle SOA.

We want to benefit from the strengths of each of the technologies, so we are going to use the following:

  • Java Spring Web Services will be used as the initial entry point for creating/editing/deleting notification messages
  • The Java Spring WS will put a message in a queue for Oracle SOA
  • Oracle SOA will poll the queue for messages, then will apply the necessary business processing and rule logic for pushing notifications out
  • Oracle SOA will handle message retry in the event of processing failures
  • Java Spring Web Services will be used for pulling user notifications out for subscriber systems

As with most of the modern web, building a solution is about choosing the right set of technologies and not choosing a single technology approach. We’re confident now that we can introduce the necessary scale to handle a modern enterprise notifications system.

uPortal release 4.10

uPortal  version 4.10 has been recently released, which has lots of great new features, most notably the move over to a Bootstrap 3 responsive design.

http://www.apereo.org/uportal/download/uportal-4-1-0

We’re also pleased to note that Ross Nicoll in Development Services has contributed to this release. That’s great to see, and hopefully something we can do more of in subsequent releases.

Thanks Ross :)!

Open Source updates

Open Source projects and code are an important aspect of what we do, so it’s good for us to be able to contribute and share with the wider community. Here’s an update of some of the recent work we’ve been doing.

uPortal

On the uPortal front we’ve contributed a bug fix to the core uPortal framework which should be incorporated into the uPortal 4 patch releases.

And the Blackboard  Virtual Classroom portlet which we originally contributed to the Apereo foundation (then JASIG) has been updated by the University of Wisconsin to version 2.

Blackboard Learn

We have submitted a building block we wrote a while ago to do dynamic web forms to project oscelot, which is an open source community site based around eLearning. We chose to release the bb-webform-tool (as we call it) code on Github.

We have also submitted the a Building Block which allows creation and integration with Confluence Wiki spaces to Project Oscelot, and have put the bb-confluence-tool code on Github in preparation.

The portlet and building block code above have been released under the MIT license.

Creating a skeleton Spring MVC Portlet

If you’re needing to create a new JSR-286 portlet, the Apereo Foundation (formerly JASIG), have provided a Maven Archetype for the creation of a skeleton portlet.

Essentially you can run the following command:

mvn archetype:generate -DarchetypeGroupId=org.jasig.portlet.archetype -DarchetypeArtifactId=jsr286-archetype

It will ask you for the group, artefact and package information, and will automatically create a skeleton Spring MVC Portlet for you.

See the using the uMobile Portlet Archetype article for full details.