Following on from previous posts on creating Spring Boot based Microservices, at some point you’re going to have to consider CORS.
Continue reading “Adding CORS support to Spring Boot Microservices”
Software Development Community Blog
University of Edinburgh Software Development Community Blog
Following on from previous posts on creating Spring Boot based Microservices, at some point you’re going to have to consider CORS.
Continue reading “Adding CORS support to Spring Boot Microservices”
A while ago I wrote about how to document Spring Boot Microservices with Swagger. This new post details how to use Swagger v2 documentation in order to document your web services. Using V2 will let you do neat things like use ReDoc, or import your Web Service into Postman (both of which are also awesome!).
Continue reading “Documenting Spring Boot Web Services with Swagger v2”
Following on from my last posts on documenting a Spring Boot micro service, and setting up a Spring Boot OAuth2 server, this post focuses on putting OAuth2 protection on a micro service, and allowing Swagger to use OAuth2.
Continue reading “OAuth2 protecting Spring Boot Microservices with Swagger”
Following on from the previous post about documenting MicroServices with Swagger, we also wanted to have a uniform authorisation/authentication model for access to our services.
Our basic requirements were as follows:
After reviewing our options, OAuth2 was the obvious contender. We already have a web single sign-on solution called EASE which uses Cosign, so we need to use that for any web-based user authentication.
The remainder of this article shows how we went about setting up an OAuth2 service using Spring Boot.
As part of the rollout to the new University Website, a Global Experience Language was developed for Edinburgh University, which was named Edinburgh GEL. The implementation of the GEL is based on Bootstrap.
In order to easily fold this into our Java Web Applications, I wanted to create a WebJar which would allow developers to quickly pull in the Edinburgh GEL and immediately begin to use the resources.
This post is following on from our moves to separate out concerns when it comes to our portlet development.
Following on from my previous post on using JSON and ETag caching, where we focused on provide data for our portlet to render client-side, we then wanted to concentrate on making the UI render side more elegant. Typically to date we’d been using .JSP with Tags to render, and using jQuery to render the JSON to page. That ended up with fairly lengthy <script> and JSP files, which were frankly messy to look at.
Here’s a truncated example of the horror:
$( document ).ready(function() { var jsonurl = "/libraryonline/rest/summary"; $.ajax({ url: jsonurl, dataType: "text", success: function( data ){ var json = $.parseJSON(data); $("#${n}libraryForm").attr("action",json.voyagerUrl); $("#${n}loginId").attr("value",json.barcode); $("#${n}lastName").attr("value",json.surname); $("#${n}loanTotal").html(json.totalLoans); $("#${n}requestTotal").html(json.totalRequests); $("#${n}fineTotal").html(json.totalFines); if(json.loanItems != null && json.loanItems.length > 0){ /* populate loan items table*/ var $itemTable = $("#${n}loanItemsTable"); $.each(json.loanItems, function(key,entry){ /* This is just getting silly */ $itemTable.append("<tr><td>"+ entry.title +"</td><td>"+ entry.statusText +"</td><td>"+ entry.dueDateConverted +"</td></tr>"); }); }
I wanted to introduce some kind of templating for rendering the HTML, and declutter the JavaScript we’re adding into pages to do the render. There are many options here todo this, we settled on Angular.
We currently use Apache with mod_jk in front of our Tomcat application servers. I was exploring how to use an embedded Tomcat while enabling an AJP connector. I wanted all the configuration to be property driven, allow the specification of HTTP/AJP ports, and allow the switching off of AJP for running the app locally.
Here’s how I went about it.
Firstly in the Spring Boot Application class you can tell the application on startup to use custom settings for the embedded Tomcat. Out of the box if you specify server.port as a property it will change the port of the standard http connector. I specified that property and some other values specific to AJP.
server.port=8082 tomcat.ajp.port=9090 tomcat.ajp.remoteauthentication=false tomcat.ajp.enabled=true
They were then wired into the Application class using @Value annotations. The server.port is already handled by Spring Boot so I don’t have to do anything.
@Value("${tomcat.ajp.port}") int ajpPort; @Value("${tomcat.ajp.remoteauthentication}") String remoteAuthentication; @Value("${tomcat.ajp.enabled}") boolean tomcatAjpEnabled;
Then I added in a specific Bean which defines the Tomcat settings, and whether or not to switch on AJP based on a property being set.
@Bean public EmbeddedServletContainerFactory servletContainer() { TomcatEmbeddedServletContainerFactory tomcat = new TomcatEmbeddedServletContainerFactory(); if (tomcatAjpEnabled) { Connector ajpConnector = new Connector("AJP/1.3"); ajpConnector.setProtocol("AJP/1.3"); ajpConnector.setPort(ajpPort); ajpConnector.setSecure(false); ajpConnector.setAllowTrace(false); ajpConnector.setScheme("http"); tomcat.addAdditionalTomcatConnectors(ajpConnector); } return tomcat; }
Then when I start up the application, I end up with an HTTP connector running on a specific port, and also optionally an AJP connector running on a specific port.
2015-06-24 08:40:09.514 INFO 93685 --- [ main] org.apache.coyote.ajp.AjpNioProtocol : Initializing ProtocolHandler ["ajp-nio-9090"] 2015-06-24 08:40:09.516 INFO 93685 --- [ main] org.apache.coyote.ajp.AjpNioProtocol : Starting ProtocolHandler ["ajp-nio-9090"] 2015-06-24 08:40:09.521 INFO 93685 --- [ main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8082 (http) 9090 (http) 2015-06-24 08:40:09.523 INFO 93685 --- [ main] uk.ac.ed.ca.centralauthms.Application : Started Application in 4.178 seconds (JVM running for 4.6)
It’s a fairly well known fact that many developers don’t like to write documentation, often muttering things like “the code is the documentation” in a half hearted manner suggesting even they don’t believe themselves. Recently I was looking to write a Microservice, so I wanted to also look at ways in which we could make nice easy to use documentation in a consistent manner.
This led me to look at Swagger. It’s a way to produce elegant and powerful interactive documentation on your REST API without having to write pages of documentation. And when used with annotations and Spring Boot, truly the code *is* the documentation.
I’m completely taken with Spring Boot already. It takes a lot of the complexity out of getting an initial application up and running, and allows you to add in features easily. In my application, I produced a REST API for getting identities out of LDAP, to support both a lookup on the logged in persons identity, and also other identities.
We still use Maven (although it’s getting more and more tempting to switch to Gradle). My pom file contains the following dependencies:
<dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-rest</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>com.jayway.jsonpath</groupId> <artifactId>json-path</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.springframework.ldap</groupId> <artifactId>spring-ldap</artifactId> <version>1.2.1</version> <type>jar</type> </dependency> <dependency> <groupId>com.mangofactory</groupId> <artifactId>swagger-springmvc</artifactId> <version>1.0.2</version> <type>jar</type> </dependency> </dependencies>
Most of the dependencies are covering the Spring Boot side of things, REST support, web application and LDAP/JSON support. Note though the bottom dependency, which is including support for Swagger Spring MVC.
The controller is fairly simple, it maps two URLs. The key thing of note is the @ApiOperation annotation, where we describe what the method does.
@RestController public class UserLookupController { @Autowired LdapService ldapService; @ApiOperation(value="Get the currently logged in users details",notes="Uses the remote user logged in") @RequestMapping(value="/my",method=RequestMethod.GET) public @ResponseBody Person getMyDetails(HttpServletRequest request) throws ServletException { if (request.getRemoteUser()==null) { throw new ServletException("Remote user is null."); } return ldapService.getPerson(request.getRemoteUser()); } @ApiOperation(value="Get a specific users details",notes="Requires uid of user to look up") @RequestMapping(value="/id/{uid}",method=RequestMethod.GET) public @ResponseBody Person getUserDetails(@PathVariable("uid") String uid) { return ldapService.getPerson(uid); } }
We also need a config class to tell Swagger what to do, and also provide high level documentation about what the Service is providing.
@Configuration @EnableSwagger @EnableAutoConfiguration public class SwaggerConfig { private SpringSwaggerConfig springSwaggerConfig; @Autowired public void setSpringSwaggerConfig(SpringSwaggerConfig springSwaggerConfig) { this.springSwaggerConfig = springSwaggerConfig; } @Bean public SwaggerSpringMvcPlugin customImplementation() { return new SwaggerSpringMvcPlugin(this.springSwaggerConfig) //Root level documentation .apiInfo(new ApiInfo( "Central Authorisation Service JSON API", "This service provides a JSON representation of the LDAP identity data held in the Central Authorisation Service", null, null, null, null )) .useDefaultResponseMessages(false) //Map the specific URL patterns into Swagger .includePatterns("/id/.*","/my"); } }
The Application class is simple, and just hooks into our LDAP config:
@Configuration @ComponentScan("uk.ac.ed.ca") @EnableAutoConfiguration public class Application { public static void main(String[] args) { SpringApplication.run(Application.class, args); } @Bean @ConfigurationProperties(prefix="ldap.contextSource") public LdapContextSource contextSource() { LdapContextSource contextSource = new LdapContextSource(); return contextSource; } @Bean public LdapTemplate ldapTemplate(ContextSource contextSource) { return new LdapTemplate(contextSource); } }
Finally, we add in a static set of HTML/CSS/JS to cover the API. You can get the static pages from https://github.com/swagger-api/swagger-ui. Put them in your project under (src/main/)resources/static and they’ll automatically get mapped into the application.
The end result is a service which also provides the following documentation. As the documentation is interactive you can also try to call the id service and see what kind of response it gives. Very neat, very powerful, and a very easy way for us to provide API documentation!
Recently as part of an update project for our university portal MyEd (which runs on uPortal) there was an emphasis on moving our content to more client driven access to data. We wanted to separate out the data and presentation a bit more, and also cut down on the load and traffic which a big single server-side render would produce.
We wanted to use JSON as the data format as it is nice and lightweight, and easy to parse with existing Javascript libraries (like JQuery). We then wrote in static URLs into the uPortal portlets which would allow the currently authenticated user (and them alone) to access their own data.
Our portal is under a reasonably heavy concurrent load at any given time, so we wanted to explore caching of data to make sure we make any client side calls perform well under load.
Cache Headers are used to tell a browser to not re-request an object from the server until a certain time, typically by setting an expiry date. This avoids any traffic going to the server at all, which reduces load but can mean that changes to data are missed because the cache expiry date has not been reached.
ETagging is different, in that an ETag value is set in the header, for example:
ETag: "asb227873hva23456n"
When the browser re-requests data from the url it passes the ETag back to the server in an If-None-Match header, e.g:
If-None-Match: "asb227873hva23456n"
The server then uses the ETag to decide what to do, either to send an HTTP Status Code of 304 not modified (typically with a very short response), or refresh the data and return new information back to the client. This reduces the bandwidth required, but more importently allows the server to decide how and when to respond with fresh data.
In order to get the best performance, you would in most situations use both caching and ETag in order to limit high frequency client traffic to the server but also allow the server to mitigate load using the ETag. We found when using both that behaviour in our uPortal server alongside our load balancer led to unexpected results , so we opted to initially use ETagging only.
(As to why our load balancer was causing unexpected caching behaviour we’ll have to investigate later, and potentially write up another post in and of itself!)
So in the portlet itself (which is written in Java), we set the JSON data controller method to add in an ETag.
final String eTag = getETag(data); final Date expiry = new Date(System.currentTimeMillis() + MAX_AGE_MILLIS); session.setAttribute(SESSION_ATTR_CACHE_ETAG, eTag); session.setAttribute(SESSION_ATTR_CACHE_EXPIRY, expiry); response.setStatus(HttpServletResponse.SC_OK); response.setHeader("Cache-Control", "must-revalidate"); response.setHeader("ETag", eTag);
Finally, we then added a check in the method for the ETag coming from the If-None-Match header:
final String ifNoneMatch = request.getHeader("If-None-Match"); final String existingETag = (String)session.getAttribute(SESSION_ATTR_CACHE_ETAG); final Date existingExpiry = (Date)session.getAttribute(SESSION_ATTR_CACHE_EXPIRY); if (null != ifNoneMatch && null != existingETag && null != existingExpiry && ifNoneMatch.equals(existingETag) && System.currentTimeMillis() < existingExpiry.getTime()) { response.setStatus(HttpServletResponse.SC_NOT_MODIFIED); return null; }
The above code checks the passed in ETag, compares it with the one stored in the user session, and additionally compares it with an expiry tag, then responds with a NOT MODIFIED 304 if the tags match and the expiry hasn’t passed. The response is null which means it doesn’t have to query the underlying dataset to respond, and therefore the response time and the bandwidth used are dramatically reduced.
We are soon going to embark on a major project to introduce enterprise notification handling at the University. Part of that will be the ability to handle a large number of messages in an efficient and robust manner. We already use Oracle SOA Suite here at the University, but wanted to test its throughput versus a lighter approach, that of Java and the Spring framework.
We chose four scenarios to test:
We then applied the same constraints to both Oracle SOA and Java:
For Oracle SOA, we set up a simple composite which tested the various features.
For Java Spring, we used Spring Boot, Spring Web Services, and Spring JPA.
The results were as follows (total timings are rounded up to the nearest second):
500 calls | 2000 calls | 5000 calls | |
Assign | 2 sec | 293 ms avg | 6 sec | 504 ms avg | 16 sec | 593 ms avg |
Write | 3 sec | 1284 ms avg | 10 sec | 861 ms avg | 29 sec | 1094 ms avg |
Read | 2 sec | 389 ms avg | 9 sec | 838 ms avg | 21 sec | 803 ms avg |
Write Read | 3 sec | 1038 ms avg | 18 sec | 1644 ms avg | 36 sec | 1403 ms avg |
500 calls | 2000 calls | 5000 calls | |
Assign | 1 sec | 101 ms avg | 1 sec | 82 ms avg | 2 sec | 72 ms avg |
Write | 1 sec | 112 ms avg | 2 sec | 232 ms avg | 5 sec | 203 ms avg |
Read | 1 sec | 73 ms avg | 1 sec | 116 ms avg | 3 sec | 116 ms avg |
Write Read | 1 sec | 271 ms avg | 3 sec | 256 ms avg | 6 sec | 234 ms avg |
It is clear that the Java Spring solution is giving better throughput times,, and that is especially evident when we increase the load. However it would be unfair to use throughput times alone in looking at what Oracle SOA provides. It gives for example an “out of the box” message resilience and support for automated message retry that would have to be coded in when using Java even with the benefit of Spring frameworks. However, Spring can provide a very useful high throughput entry point into Oracle SOA.
We want to benefit from the strengths of each of the technologies, so we are going to use the following:
As with most of the modern web, building a solution is about choosing the right set of technologies and not choosing a single technology approach. We’re confident now that we can introduce the necessary scale to handle a modern enterprise notifications system.