Automation within Information Services Applications Team Delivers Efficiency Gains

Recent work within Information Services Applications has realised benefits by significantly reducing the amount of time required to release APIs (code used by developers to innovate and build further tools).

By utilising tools and technologies which partly automate the work involved in rolling out APIs the Development Services teams have managed to reduce the lead time prior to release from around five days to a matter of minutes.  The tools mean that the release process is significantly more straightforward and requires much less setting up and configuration to the point where releasing an API is almost a case of pushing a button.  The automation also ensures that the process is more reliable and less prone to errors.

Furthermore, multiple APIs can be released in parallel meaning that efficiency gains are further increased as more APIs are developed by the University.

The team demonstrated this new capability in a recent deployment of a microservices tool where several existing APIs were redeployed as part of the release.

Further Information on APIs Programme:

There is a need to transform the online experience of students, applicants, alumni, staff and other members of the University community to provide tailored information via personalised digital services.

To deliver these personalised digital services requires a way to plug into information from central systems and use it in new and innovative ways , like new websites, apps for mobile devices and what’s called ‘portlets’ for MyEd that operate like web pages within web pages.

Plugging into the information can be achieved by using Application Programming Interfaces (APIs). Here at the University we use open source tools for APIs – the type of code you can get from online communities where developers collaborate to make even better products and software.

While API technology has been around for a long time, its use beyond the integration of large, complex business systems has grown rapidly in recent years and with the proliferation of devices and mobile technology, API use in getting data from one place to another – from connecting business systems, to mobile devices and apps – has expanded exponentially.  That’s because APIs allow data to be accessed securely and consistently across multiple devices, a tremendously valuable resource for the University.

The University’s Enterprise APIs Programme (part of Digital Transformation) has been set up to deliver APIs to support projects like the User Centred Portal Pilot, the Service Excellence Programme and other digitalisation and transformation Programmes across the University.  In addition to enhancing central services such as MyEd, APIs will provide a consistent way for software developers across the University to build flexible systems at a lower cost, securely, and consistently across multiple systems.

Further Links:

Digital Transformation Website

API Technology Roadmap

 

Combining pdfs and images in ColdFusion using Java and iText

pdfs, images and text combined in real time

We were writing a ColdFusion application that allows users to claim expenses and add PDFs or images of receipts electronically. The client requested that a single PDF for each claim be created which included both the claim details and ALL electronic attachments.

Using <cfpdf> chewed memory and created huge files

ColdFusion has built in PDF functionality but we found that adding BOTH images and merging other pdfs on the fly had performance issues and created documents with bloated file sizes.

Using iText provided a solution

Continue reading “Combining pdfs and images in ColdFusion using Java and iText”

From 0 to OAuth2 with Spring Boot

Following on from the previous post about documenting MicroServices with Swagger, we also wanted to have a uniform authorisation/authentication model for access to our services.

Our basic requirements were as follows:

  • They must support client-side authorisation (e.g. via Javascript calls in browsers)
  • They should have a single authorisation point
  • Session authorisation timeout must be controllable
  • They should be multi-domain ready (e.g. authentication from <user>@ed.ac.uk or <another-user>@another.ac.uk

After reviewing our options, OAuth2 was the obvious contender. We already have a web single sign-on solution called EASE which uses Cosign, so we need to use that for any web-based user authentication.

The remainder of this article shows how we went about setting up an OAuth2 service using Spring Boot.

Continue reading “From 0 to OAuth2 with Spring Boot”

Building a WebJar

As part of the rollout to the new University Website, a Global Experience Language was developed for Edinburgh University, which was named Edinburgh GEL. The implementation of the GEL is based on Bootstrap.

In order to easily fold this into our Java Web Applications, I wanted to create a WebJar which would allow developers to quickly pull in the Edinburgh GEL and immediately begin to use the resources.

Continue reading “Building a WebJar”

Making Portlets Angular

This post is following on from our moves to separate out concerns when it comes to our portlet development.

Following on from my previous post on using JSON and ETag caching, where we focused on provide data for our portlet to render client-side, we then wanted to concentrate on making the UI render side more elegant. Typically to date we’d been using .JSP with Tags to render, and using jQuery to render the JSON to page. That ended up with fairly lengthy <script> and JSP files, which were frankly messy to look at.

Here’s a truncated example of the horror:

$( document ).ready(function() {
	    
	    var jsonurl = "/libraryonline/rest/summary";
	    
	    $.ajax({
	    	url: jsonurl,
	    	dataType: "text",
	    	
	    	success: function( data ){
	    		
	    		var json = $.parseJSON(data);
	    		
	    		$("#${n}libraryForm").attr("action",json.voyagerUrl);
	    		$("#${n}loginId").attr("value",json.barcode);
	    		$("#${n}lastName").attr("value",json.surname);
	    		
	    		$("#${n}loanTotal").html(json.totalLoans);
	    		$("#${n}requestTotal").html(json.totalRequests);
	    		$("#${n}fineTotal").html(json.totalFines);
	    		
	    		if(json.loanItems != null && json.loanItems.length > 0){
	    			/* populate loan items table*/
	    			
	    			var $itemTable = $("#${n}loanItemsTable");
	    			
	    			$.each(json.loanItems, function(key,entry){
	    				/* This is just getting silly */
	    				$itemTable.append("<tr><td>"+ entry.title +"</td><td>"+ entry.statusText +"</td><td>"+ entry.dueDateConverted +"</td></tr>");

	    			});
	    		}

I wanted to introduce some kind of templating for rendering the HTML, and declutter the JavaScript we’re adding into pages to do the render. There are many options here todo this, we settled on Angular.

Continue reading “Making Portlets Angular”

Unit testing AngularJS portlet with Maven and Jasmin

Recently, we have converted a few portlets to use AngularJS for the front end.  We have also used the Jasmine framework for unit testing. Since our portlets are already configured with Maven, we have added Jasmine support with jasmine-maven-plugin to the existing set up. This posts will walk you through the configuration required to run Jasmine tests with Maven.

Portlet directory layout

|—— Portlet
| |——– src
| | | ——– main
| | | | ——— webapp
| | | |————- js
                              app.js
| | | ——– test
| | |———— webapp
| | | |————- js
                              controllerSpec.js

Our angularJs files are contained in src/main/webapp/js directory. Jasmine spec files are placed in a test directory that reflects main.

Plugin configuration 

Based on the directory structure, we can configure jasmine-maven-plugin. we are using version 2.0 which requires Maven 3.1.x. This is an example plugin configuration for our portlet:

<build><plugins>
    …
    <plugin>
       <groupId>com.github.searls</groupId>
       <artifactId>jasmine-maven-plugin</artifactId>
       <version>2.0</version>
       <executions>
          <execution>
             <goals>
               <goal>test</goal>
             </goals>
          </execution>
       </executions>
 <!-- keep the configuration out of the execution so that the bdd goal has access to it -->
       <configuration>
           <jsSrcDir>src/main/webapp/js</jsSrcDir>
           <sourceIncludes>
               <include>app.js</include>
           </sourceIncludes>
           <jsTestSrcDir>src/test/webapp/js</jsTestSrcDir>
           <specIncludes>
               <include>*Spec.js</include>
           </specIncludes>
           <preloadSources>
               <source>webjars/angular.js</source>
               <source>webjars/angular-sanitize.js</source>
               <source>webjars/angular-mocks.js</source>
               <source>webjars/jquery.js</source>
           </preloadSources>
       </configuration>
 </plugin></plugins></build>
<dependencies>
 
<!-- jasmin maven plugin dependencies -->
 <dependency>
      <groupId>org.webjars</groupId>
      <artifactId>jquery</artifactId>
      <scope>test</scope>
 </dependency>
 <dependency>
      <groupId>org.webjars</groupId>
      <artifactId>angularjs</artifactId>
      <scope>test</scope>
 </dependency>
<!-- for headful tests
 <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-firefox-driver</artifactId>
      <scope>test</scope>
 </dependency>
 -->

 </dependencies>

<jsSrcDir>  points to diectory with your JavaScript, this will be your angular controllers, directives, filters, etc.

<sourceIncludes> specifies which files within jsSrcDir should be included and in which order (added sequentially as defined)

<jsTestSrcDir>  points to directory with your Jasmin specs

<specIncludes> specifies which spec files to include and in which order

<preloadSources> JavaScript files to be loaded before the application files and specs, added sequentially as defined; these should be all libraries that you need for your application e.g. angular files, jQuery etc. We can reference webjars files here as done in our example. Since we require them for the tests only, we reference them in the test scope.

Out of the box, jasmine-maven-plugin comes preconfigured with PhantomJS for headless tests. For headful tests, configure the plugin with a different driver, such as:

<webDriverClassName>org.openqa.selenium.firefox.FirefoxDriver</webDriverClassName>

Usage

Jasmine can be used for TDD (Test Driven Development). Run

mvn jasmine:bdd

This will fire off the browser at http://localhost:8234 and run the specs from the test directory whenever you refresh the page. This will give you an immediate feedback as you develop your angular controllers etc.

Running

mvn jasmine:test

will run the headless tests by default, unless configured otherwise.

Running

mvn clean test

will run all your tests in the portlet, including Jasmine headless tests provided there were no test failures before.

With this configuration we are now ready to write Jasmine specs for our portlets.

Adding embedded Tomcat AJP support to a Spring Boot application

We currently use Apache with mod_jk in front of our Tomcat application servers. I was exploring how to use an embedded Tomcat while enabling an AJP connector. I wanted all the configuration to be property driven, allow the specification of HTTP/AJP ports, and allow the switching off of AJP for running the app locally.

Here’s how I went about it.

Application class

Firstly in the Spring Boot Application class you can tell the application on startup to use custom settings for the embedded Tomcat. Out of the box if you specify server.port as a property it will change the port of the standard http connector. I specified that property and some other values specific to AJP.

server.port=8082
tomcat.ajp.port=9090
tomcat.ajp.remoteauthentication=false
tomcat.ajp.enabled=true

They were then wired into the Application class using @Value annotations. The server.port is already handled by Spring Boot so I don’t have to do anything.

@Value("${tomcat.ajp.port}")
int ajpPort;

@Value("${tomcat.ajp.remoteauthentication}")
String remoteAuthentication;

@Value("${tomcat.ajp.enabled}")
boolean tomcatAjpEnabled;

Then I added in a specific Bean which defines the Tomcat settings, and whether or not to switch on AJP based on a property being set.

@Bean
public EmbeddedServletContainerFactory servletContainer() {

    TomcatEmbeddedServletContainerFactory tomcat = new TomcatEmbeddedServletContainerFactory();
    if (tomcatAjpEnabled)
    {
        Connector ajpConnector = new Connector("AJP/1.3");
        ajpConnector.setProtocol("AJP/1.3");
        ajpConnector.setPort(ajpPort);
        ajpConnector.setSecure(false);
        ajpConnector.setAllowTrace(false);
        ajpConnector.setScheme("http");
        tomcat.addAdditionalTomcatConnectors(ajpConnector);
    }

    return tomcat;
}

Then when I start up the application, I end up with an HTTP connector running on a specific port, and also optionally an AJP connector running on a specific port.

2015-06-24 08:40:09.514 INFO 93685 --- [ main] org.apache.coyote.ajp.AjpNioProtocol : Initializing ProtocolHandler ["ajp-nio-9090"]
2015-06-24 08:40:09.516 INFO 93685 --- [ main] org.apache.coyote.ajp.AjpNioProtocol : Starting ProtocolHandler ["ajp-nio-9090"]
2015-06-24 08:40:09.521 INFO 93685 --- [ main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8082 (http) 9090 (http)
2015-06-24 08:40:09.523 INFO 93685 --- [ main] uk.ac.ed.ca.centralauthms.Application : Started Application in 4.178 seconds (JVM running for 4.6)