Home | Topics | Software Engineering

Integrating RevoDeployR from Revolution through RESTful API or XML-RPC with .NET or Drupal

Printer-friendly versionPDF version

Complex statistical computations can be done in R. Closed source applications may very well compute industry standard and financial regulator standard indicators, but when you get stuck on the software's capabilities, shall you invest all hope that the vendor can make the customisations necessary? How much would that cost and how long time would it take?

If you manage to transfer the data into a commercial high performance open source R environment that Revolution Analytics can deliver you will be provided by advanced analytics software at half the cost of existing solutions. By building on open source R—the world’s most powerful statistics software—with innovations in big data analysis, integration and user experience, Revolution Analytics meets the demands and requirements of modern data-driven businesses.

Revolution R Enterprise

RevoDeployR is a web services framework from Revolution Analytics that uses Web 2.0 technology (RevoDeployR is itself based on the Apache Spring framework and various other open source components including R and MySQL) to enable web application developers to bring the power R analyses and graphics to bear on web-based analytics challenges without having to become R experts themselves.

The idea is simple: as an R programmer, you can create a script in R and then publish it to the RevoDeployR server. To make things easy for integration, you'll define the interface to the script in terms of well-specified inputs (akin to  function arguments) and outputs (data, files, graphics). You can use all of the features of R (and extensions like CRAN packages) in the script to implement any calculation you want: grab data from a database (or from one of the supplied inputs), build a statistical model, return forecasts, generate a chart.

Then, it's over to an application developer to integrate your R code into an application. (Of course, if you're a crack API/web services programmer you can do this part, too, but we'll assume different people are doing the R programming and the integration development.) The application developer will use the extension language of the target application: .NET for Microsoft Excel, JavaScript for Jaspersoft, etc. to extend it to call out to RevoDeployR, using the developer libraries provided with RevoDeployR. How the interface looks and behaves is completely up to the developer (within the limits of the target application, anyway): there could be buttons, fields, and file choosers to control the inputs to the R script, and the outputs can be embedded in the application in a data grid, file download, or chart.

From the point of view of the application user, all of this happens invisibly, behind the scenes. There might be a new control, or menu item in the application they've been using for years, and there's some new information (based on some fancy statistical analysis created by the Research folks down the hall) to make decisions with, but other than that everything's as before. The user is most likely unaware that their application is now making RESTful API calls to a remote RevoDeployR server via the web, or even that R is involved at all. And that's a good thing: now, more people can benefit from the power of R, even if they don't know R themselves.

Through the RevoDeployR Application Programming Interface (API) any client program running in a browser or any desktop application connected to the web can employ R to access and transform data, run sophisticated statistical analyses and produce high quality plots and graphs. RevoDeployR implements a client/server model that separates client-side application development from R programming. Web developers can specify the inputs and results required for their applications and hand off the details of the statistical calculations to the R experts. And, for their part, R experts can develop black-box R applications that may be useful for a broad range of web applications. RevoDeployR provides R programmers with a vehicle for publishing R analytical routines in a manner that is immediately useful to web application developers. This document describes RevoDeployR from the R programmer’s point of view and illustrates the simplicity with which R analyses can be made available to web-based applications.

The RevoDeployR Technology Stack

RevoDeployR is a full-featured programming framework for integrating R functionality into web-enabled applications. RevoDeployR consists of four major components:

  1. a RESTful API that provides access to Web services,
  2. a Web Services engine that provides session services, R scripting services, R object management and file management
  3. a stand-alone management console program that provides various administration and user services such as authentication, login and security services etc., and
  4. the Web Services Application Explorer, a stand-alone tool for testing the API.

RevoDeployR also depends on several open source components such as the R language itself, the R package Rserve which coordinates communication between the DeployR server and R, MySQL which is used to maintain a repository for R scripts and objects, Spring which provides the underlying application framework and security model, Apache HTTPd web server, and Apache Tomcat which implements the Java servlet engine that runs the API code. RevoDeployR is a RESTful API in that it adheres to the REST architectural standard for implementing client/server applications. In a REST architecture, clients invoke services by calling a specific HTTP URL on the server. Servers are passive in the sense that they respond to client requests but do not initiate activity.

Any language that supports read/write JavaScript Object Notation (JSON) or Extensible Markup Language (XML) encoding can access the RevoDeployR API.

Contact

Peter Lindmark

Peter Lindmark

CEO, Asymptotix

Tel: +352 691 156 388

Or send him an email!

 

Integrating RevoDeployR with .NET

ASP.NET MVC – Create easy REST API with JSON and XML

ASP.NET MVC Controllers can directly return objects and collections, without rendering a view, which makes it quite appealing for creating REST like API. The nice extensionless Url provided by MVC makes it handy to build REST services, which means you can create APIs with smart Url like “something.com/API/User/GetUserList”

There are some challenges to solve in order to expose REST API:

  • Based on who is calling your API, you need to be able to speak both Json and plain old Xml (POX). If the call comes from an AJAX front-end, you need to return objects serialized as Json. If it’s coming from some other client, say a PHP website, you need to return plain Xml.
  • Similarly you need to be able to understand REST, Json and plain Xml calls. Someone can hit you using REST url, someone can post a Json payload or someone can post Xml payload.

I have created an ObjectResult class which takes an object and generates Xml or Json output automatically looking at the Content-Type header of HttpRequest. AJAX calls send Content-Type=application/json. So, it generates Json as response in that case, but when Content-Type is something else, it does simple Xml Serialzation.

Here’s the ObjectResult that you can use from Controllers to return objects and it takes care of proper serialization method. Above shows the Json serialization, which is quite simple. XmlSerialization is a bit complex though:

Things to note here:

  • You have to force UTF8 encoding. Otherwise it produces UTF16 output.
  • XML Declaration is skipped because that’s not quite necessary. Wastes bandwidth. If you need it, turn it on.
  • I have turned on indenting for better readability. You can turn it off to save bandwidth.

Some of you might be boiling inside looking at my obscure coding style. I love this style! I am spoiled by jQuery. I wish there was a cQuery. I actually started writing one, but it never saw day light just like my hundred other open source attempts.

Now back to Object Serialization, we got the serialization done. Now you can return objects from Controller easily:


You can use the test web project to call these methods and see the result:

So far you have seen simple object and list serialization. A best practice is to return a common result object that has some status, message and then the real payload. It’s handy when you only need to return some error but no object or list. I use a common Result object that has three properties – ErrorCode (0 by default means success), Message (a string data type) and Data which is the real object.

When you want to return only a result with error message, you can do this:

This produces a result like this:

No payload here. So, the return format is always consistent. Those who are consuming service can write a common Xml or Json parsing code to consume both success and failure response. Those who are building API for their website, I humbly request you to return consistent response for both success and failure. It makes our life so easier.

So, far we have only returned objects and lists. Now we need to accept Json and Xml payload, delivered via HTTP POST. Sometimes your client might want to upload a collection of objects in one shot for batch processing. So, they can upload objects using either Json or Xml format. There’s no native support in ASP.NET MVC to automatically parse posted Json or Xml and automatically map to Action parameters. So, I wrote a filter that does it.

This filter intercepts calls going to Action methods and checks whether client has posted Xml or Json. Based on what has been posted, it uses DataContractJsonSerializer or simple XmlSerializer to convert the payload to objects or collections.

You use this attribute on Action methods like this:

The attribute expects a parameter name where it stores the deserialized object/collection. It also expects a root type that it needs to pass to the deserializer. If you are expecting a single object, specify typeof(SingeObject). If you are expecting a list of objects, specify an array of that object like typeof(SingleObject[])

You can test the project live at this URL: http://labs.dropthings.com/MvcWebAPI

The code is also available at: http://code.msdn.microsoft.com/MvcWebAPI

Drupal’s Views as a JSON Web Service with the REST Server

Drupal 6 does not have web services built in – instead they are part of the Services module. The Services module enables your Drupal installation to provide data to clients such as Flash, Javascript AJAX clients, or in this case, an iPhone application.

To use the Services Module, you will have to install additional servers to communicate with clients – for instance, the REST Server and the JSON Server. Using Three20′s GitHub example as a guideline, I can see that JSON parsing is a lot simpler on the iPhone than XML, and I’d prefer to stay away from XML anyway. There is also a native iPhone plist server for Drupal, but at some point I may create an Android client, and there’s no reason to limit myself to iPhone only.

This led me to the JSON Server as the solution, but that turned out to be a dead end. I’ve had much frustration with the JSON Server – from what I can tell, it really doesn’t work. I’ve gotten lots of 404 errors and not much else. From its Issues on Drupal.org, I’m not the only one with JSON Server issues. Luckily the REST Server offers a JSON interface as well, by just appending “.json” to the end of the GET URL.

Getting JSON representations of individual Drupal nodes with the REST Server is very straight forward – make a GET request to http://<yourserver>/services/rest/node/<node-id>.json where <node-id> is 1.

Some gotchas here are that even if you don’t care about securing access to that node (because it’s public content on your Drupal site anyway), you need to enable Key authentication for your services in Drupal and then turn it off in the Services Settings. You also need to remember to enable the Node Service module, and ensure that the “load node data” permission for the node_service module is allowed for anonymous and logged in users (if you want it to be public, of course). Any of these things not being done properly can lead to a 404 error when you make the REST call.

This all works well for loading one node at a time, but I’d like to show information from multiple nodes on one iPhone screen. And I’d like to make that show up as quickly as possible – so that means that rather than making multiple HTTP calls to Drupal over the network, I’d like to just make one call to the server.

Drupal’s views are perfect for this – views are nothing more than a layer on top of SQL, and what I want is basically a few fields from each node reference on my main node. The nice thing here is that I can develop and debug my view from Drupal to make sure it outputs the data that I want first, and then simply load that view into the iPhone app and work with the data there. But to do that I need to have a little chunk of middleware that connects the iPhone app to the Drupal view – this is where the Views Service module comes in. Enable Views Service, and the Services admin settings will show a views.get method. Using the admin interface, you can experiment with the arguments needed to get your view to work as a service. Of course, this is not an endpoint you can actually use as a service, but just an admin tool.

To actually load a view using the REST Server, I had to do some serious research. The best source of information was an issue for the Rest Server where the developer answered – http://drupal.org/node/570778#comment-2547806 another question. Unfortunately his answer included a typo that threw me off for a while, so here’s what you actually need to do -

POST to /services/rest/service_views/get.json

with the arguments ‘view_name’ and ‘args’ – you can test these from the admin interface. Using curl, the request looks like this from the command line:

curl -verbose –data ‘view_name=your_view_name‘ –data ‘args=[your_args_1,your_args_2]‘ http://your_server_name/services/rest/service_views/get.json

RevoDeployR Application Explorer

 Finally we have a peek at the RevoDeployR Application Explorer.

Short URL
Asymptotix on Twitter

Are the key legislative pillars such as Basel II & III, UCITS IV and Solvency II forcing you to re-examine how you identify, measure and manage risk and capital?

Asymptotix work closely with our partners to help clients develop a more proactive, systematic and integrated approach to governance and risk management to deliver proper value.

Asymptotix can offer the support you need to deliver on time. Read more...
 

Is the goal of your website to sell services or products, educate, or collect data?

A positive customer experience is vital to conversion, no matter what your conversion goals may be. Our designers and developers will create a positive experience to maximize your conversions and deliver the optimal return on your investment. We strive to find the perfect balance between the web site’s design and functionality.

Asymptotix implements interactive solutions for European companies. From corporate websites to social communities, our clients will tell you an investment in building a scalable online experience will deliver long-term tangible benefits.

Based in Luxembourg we can help you all over Europe. Our multi-lingual team can work with projects and speak your language! Read more...