Thursday, May 7, 2009

Spring Framework 3.0 M3 and REST enhancements

In my previous blog post, I described how Spring 3.0 Milestone 2 can be applied in real REST web service project, and I enumerated several limitations and problems. M3 is out now. What has changed then?


M3 contains several enhancements over M2, addressing my postulations:



  1. Fixed URL mapping with @RequestMapping annotations: type-level and method-level finally work together in the logical manner, allowing for easy writing of one controller per one REST resource (SPR-5631). I also noticed that at container startup, all the resolved mappings are printed to the log at INFO level - that's nice too.

  2. Trailing slashes insensitive mapping (SPR-5636)

  3. Input data (body) conversion - easy access to request body as controller method param, converter for form encoded data (and other types) (SPR-5409)

  4. Type conversion error for @PathVariable annotated params returns code 400, not 500 (SPR-5622)

  5. Custom processing of object returned from controller method (SPR-5426)

  6. Header-based request matching (SPR-5690)

  7. Flexible exception handling (SPR-5622 and related)


I haven't yet verified in practice the last three of them, but overall it looks pretty good, and the picture is now much more complete than in case of M2.


Small bugs that I've found:


  • Exception when method-level annotation starts with slash (SPR-5631, SPR-5726) - should be already fixed

  • Code 404 returned, when it should be 405 (SPR-4927)


One thing that worries me is the sentence from M3 documentation: Available separately is the JacksonJsonView included as part of the Spring JavaScript project. I thought it was supposed to be moved to core REST packages? As I said earlier, JSON is not a JavaScript. It's a data exchange format you can use to talk Java to .NET, with no JavaScript involved. So having dependency on JavaScript project makes no sense.


I really like the idea of FormHttpMessageConverter, which can bind body data to MultiValueMap:


@RequestMapping(method=PUT, value="{projectName}")
public HttpResponse createOrUpdate(@PathVariable String projectName,
@RequestBody MultiValueMap<String,String> newValues)

It solves (at least partially) problem of servlet getParameter method used with PUT (see my post for REST in M2, and also SPR-5628). Actually, FormHttpMessageConverter calls the getInputStream method on ServletResponse, so after it is called by the framework, the attempt to call getParameter inside controller method will return null not only for PUT requests, but also for POST, because body was already parsed into MultiValueMap (this behavior is valid according to servlet API). But having the map of parameters, there is no need for calling getParameter again.


I have only one problem with MultiValueMap. Previously, I parsed manually the form data from body into PropertyValues object, which was then passed to Spring's DataBinder for updating the model entity class. Now, in M3, the form data is parsed by Spring and I get MultiValueMap<String,String> directly. That's nice, but how to pass it to DataBinder? There is a bind(Map) method in DataBinder, but the problem is that MultiValueMap<String,String> at the very bottom is actually the Map<String,List<String>>. Now if I pass it to bind method directly, I get many binding errors saying "cannot convert List<String> to String", because model class has fields of type String, and the MultiValueMap keeps it as one-element string lists. The simplest solution would be to extend MultiValueMap interface with one method, called e.g. 'asFlatMap()' or so, that would return either copy or the live view of the MultiValueMap<K,V> as Map<K,Object> with all values which are single-element lists replaced by the element itself (so Object is actually either V or List<V>). This way I could pass the map returned by this method to the DataBinder easily. The code below shows the conversion:


Map<String,Object> asFlatMap(MultiValueMap<String,String> map) {
Map<String,Object> convertedMap = new LinkedHashMap<String,Object>();
for (Entry<String,List<String>> entry : map.entrySet()) {
if (entry.getValue().size() == 1) {
convertedMap.put(entry.getKey(), entry.getValue().get(0));
} else {
convertedMap.put(entry.getKey(), entry.getValue());
}
}
return convertedMap;
}

I've created a SPR-5733 issue for this. AFAIK, Spring binding is going to be refactored in RC1, so perhaps it can be aligned with it too.

Saturday, April 25, 2009

NullPointered

As I mentioned in one of the previous posts, I have been outsourced to some project, which uses a home-made, rather naive web framework. This framework has something in common with Struts: it has an abstract Action class, that must be implemented by any action in the system. The main method from this class, that must be implemented by concrete actions, has signature of this shape:


abstract protected Result perform(...) throws ProcessDataException, NullPointerException;

So, how would you implement the "Hello World!" example in this framework? I guess, more or less this way:


public Result perform(...) throws ProcessDataException, NullPointerException {
throw new NullPointerException("Hello World!");
}

Well, this looks like a good candidate for The Daily WTF :)

Wednesday, April 15, 2009

Ext Core - a new weapon in your arsenal

Recently, the Ext-JS team announced introduction of the new product, Ext Core. At this moment it is in the beta stage, but I guess we should see the final version soon. The good news is that Ext Core will have free MIT license for any use, as opposed to Ext-JS, which is not free for commercial use.


Why is this announcement so important? Well, the Ext-JS is excellent JavaScript library, so if you want to build the really big pure-javascript client, Ext-JS is the best choice. However, such clients are still not that popular, and the server-side frameworks remain mainstream. So what most developers do is enhancing the server-side-generated web pages with javascript addons. The low-level DOM manipulation libraries, like Prototype or JQuery are perfect for this purpose. They are lightweight and free, and that's why they got so popular. Obviously, you could use Ext-JS for the same purpose, because it contains the low-level tools capable of doing the same thing. But the whole power of Ext-JS, the component-based UI, wouldn't be used at all. So would you pay for Ext-JS with features you won't use, if the JQuery can do it for free? Obviously, not. This I believe was an important barrier for Ext-JS adoption.


Now, from what I understand, beginning from from version 3, Ext will be split into two distinct layers (and products). Ext Core will contain the low-level code, for dealing with DOM elements and with API concentrated around Ext.Element class, while Ext-JS will contain full Ext distribution: bundled Ext Core plus all the UI widgets, tools and managers, with API concentrated around Ext.Component.


This means that Ext Core becomes a viable alternative for jQuery: it's also lightweight and free, and has fantastic community support on Ext-JS forum (I must say I've never met the second forum where the questions receive so fast responses, in spite of traffic being really high). I haven't done the comparison of Ext Core vs JQuery in terms of API: which library is more powerful and/or simpler. Many concepts are similar, even if they are named or implemented a bit different. In fact, I haven't used much of the low-level features of Ext-JS before, because when you use Ext components, it is usually not needed to bother with DOM-level details. The Ext Core is at beta stage, so probably API can evolve a bit yet. (For example, it's strange for me that there is no simple method in Ext.Element to get attribute value of the element; there is only namespaced-version getAttributeNS(namespace, name), but why there is no getAttribute(name)? How often do you come across namespaced attributes in HTML pages?)


One big advantage of Ext-JS is documentation. I've always found the Ext-JS API documentation more clear and complete than that of jQuery. Ext Core, I hope, will not be worse in this area. At this moment there is an API documentation available, but it also looks like "beta" version, because there is no details section (like in standard Ext-JS docs), and method links point directly to the source, instead of to the details section. That's good idea to link to the source, but I shouldn't be forced to go to the source every time I want to check the full method description. I hope this will be fixed before final release. There is also the Ext Core manual available, which looks very promising, even if it is also not finished yet (there are typos and errors, and some sections are only sketched). One thing is very interesting: authors of the manual teach you the Ext Core with aid of FireBug: there are plenty of examples based on FireBug. That's really cool. On the other hand, the section describing the effects is poor: you see red rectangle and the code which is supposed to do some effects on it (fade, switch off, ...), but you can't execute the code and see it in action. Quite annoying. But I hope it will be also fixed before Ext Core goes final.


Ext Core being free, lightweight and high-quality, should attract many developers. And obviously, in the longer term those people who started to use Ext Core and got to know it well, will be more eager to do one step further and switch to the "full version" of Ext-JS. Things are getting more and more interesting in JavaScript frameworks circle. With the new player, I guess we'll watch some significant changes in JavaScript libraries usage statistics soon.

Thursday, April 9, 2009

Fortunately, XSLT is dead (in web frameworks)

Several years ago there was a trend in Java web frameworks to use XML processing as a foundation for the framework logic: take the data from database and present it as XML document (either convert it from relational data, or use XML capable database), and transform it to HTML using XSLT. There were some books written about it, and some frameworks created (like Cocoon). After some time the hype started to decline, and currently virtually no modern web framework apply this approach, I think.


Actually I've never used it on myself, but I liked the idea at that time. It seemed to be clear and powerful. Funny that only now I have a chance to check how it works in practice. And at least now I know why this approach failed.


For two weeks, I'm outsourced to help some company in their web project. They use their own, home-made web framework for this, to talk to their own, home-made document-repository system. Unfortunately, both seemed to be buggy and unstable, though they say that's not the first project they use them in (I can't believe it's true, really). I guess they would be better off using standard Java Content Repository (JSR-170) for implementing documents repository, and some modern web framework instead of their own home-grown one. If they insisted on XML/XSLT transformations, they could use Cocoon. At least there would be more documentation available, and it would be well-tested and stable. But ok, that's not the first company that suffers from NIH syndrome, or guys are simply too lazy or overloaded to look around for other stuff. The interesting point is: how the XML-based processing works in practice? The short answer is: very poor. And the weakest link in the whole chain is XSLT.


XSLT bloat


I won't dare to say that XSLT is worthless - perhaps in some contexts it can be useful, especially for transforming one document tree into another (valid) document tree, i.e. from DOM to DOM. XSLT gives you guarantee that input and output documents will be valid XML - this is crucial e.g. in SOA applications, transforming one document into other documents, to be processed by machines. (X)HTML is a document tree too, at least formally, but from the point of view of web browser to have perfectly valid XHTML is good, but not crucial, and from the point of view of web designer or developer the DOM behind it doesn't matter at all, and making the template valid XML is of no importance. For dynamic generation of HTML pages in most cases it is much easier if you treat the HTML code as a unformatted text, and make a web page template by embedding some special processing directives in such text. This approach was applied by JSP (first with scriptlets, and than with JSTL), Velocity, FreeMarker, and other technologies. Neither of those technologies use the strict XML as template. On the opposite side we have JSPX (JSP using strict XML) - it never caught on and I guess many Java developers have never met it; and XSLT.


I've used a lot of JSPs with JSTL. It wasn't perfect, but it worked. Now I have to do the same with XSLT and it's a nightmare. Things that took me half an hour to do with JSP take several hours in XSLT. This is a list of things I hate the most in XSLT:



  1. Conditional arguments. For example: how to hide the row in table (using different CSS style), based on some CONDITION, with XSLT? See:

    <tr>
    <xsl:attribute name="style">
    <xsl:choose>
    <xsl:when test="CONDITION">
    <xsl:value-of select="'visibility: visible'">
    </xsl:when>
    <xsl:otherwise>
    <xsl:value-of select="'visibility: collapse'">
    </xsl:otherwise>
    </xsl:choose>
    </xsl:attribute>
    ...
    </tr>

    and now the same with JSP 1.x:

    <tr style='visibility:<%=CONDITION ? "collapse" : "visible"%>'>
    ...
    </tr>

    or with JSP 2.x:

    <tr style='visibility:${CONDITION ? "collapse" : "visible"}'>
    ...
    </tr>


  2. Nested loops. In JSTL the <for-each> tag has the var attribute, which is variable that gets assigned the current element from the collection during looping. In nested loops, you choose different var names, and you have easy access to variable at any level. In similar <for-each> in XSLT there is not var attribute. You must use additional variables as child nodes, or some other workarounds. Its very easy to get lost.

  3. Every XML-like fragment which is not an actual XML must be escaped. Say you have inlined javascript function which appends row to the table:
    onclick="append('<tr><td></td><td></td></tr>')"

    This will work in JSP quite good, but will blow up in XSLT with "could not compile stylesheet" message. You must escape each < character:
    onclick="append('&lt;tr>&lt;td>&lt;/td>&lt;td>&lt;/td>&lt;/tr>')"

    Nobody could understand what is going on here at first look now.

  4. The functional approach applied in XSLT design, instead of the well known (for all programmers) procedural one, makes "thinking in XSLT" very hard. "Normal" approach (JSP, Velocity, etc) takes the HTML template starting from familiar <html><head>...<body>... and looks for special markers, where it puts data from "model". This data can be Java object or XPath-extracted data from other XML. XSLT does it in completely reverse way: it starts with <template match="..."><apply-templates>... so it takes the XML data document first, and tries to manipulate its content to obtain other document. As I said, in SOA processing this is fine. But in HTML generation it looks completely alien. I must say I always had problems with mental visualization of this process.

  5. No formal XML schema for XSLT 1.0 exists. At least I couldn't find it - there is only unofficial DTD somewhere. This ruins IDE code completion abilities. And the XSLT is so complicated, that you can't simply learn it in one day (or even week), so some inline-help would be of real help.


Now take all those points together and multiply by all the places on the web page where dynamic content generation happens. Lengthy, complicated, with all those escapings, plus complicated XPath expressions, plus this "other-way-round" functional approach. That is developer horror. And worse, it is maintainer hell. In the current project, after two weeks I'm not able to understand the sections I have written a few days ago. I cannot imagine looking at it after several months. What's worse, the template code is so different than resultant HTML code, that navigation in the template and finding the actual place I need to edit takes always too much time.


After two weeks I'm fed up with XSLT. It's absolutely unproductive in web frameworks. Now I know why none of XML-based frameworks got big popularity ever. And I know I was completely wrong 3 years ago when I could bet that in the short time JSPX would replace JSP. Fortunately, it didn't.


Any alternatives?


Now that we know XSLT is evil, which view technology should we use in our projects? Stick with JSP? For JSF-based project I wouldn't use JSP for sure, because it simply doesn't play well with JSF, and instead I would go for something like facelets. But actually I wouldn't go for JSF either any longer (that's another story about the thing I used to believe in and predict long life to it, and that I was finally disappointed completely with). So for non-JSF projects there are JSP/JSTL, Velocity, FreeMarker, GSP for Grails projects, SiteMesh for page layouts, perhaps other technologies that I'm not aware of. JSP/JSTL is the most widely used, best known, and has best tool support, even if it is probably the worst one from the group. Take those crazy SQL-based tags in JSTL, or funny standard tags from JSP to deal with request parameters. Whey didn't they just took those tags from JSTL that are used always (if, for-each, format, ...) and make them part of standard JSP? Why I always have to include it as separate library on Tomcat? Besides, I said earlier that the actual input template doesn't have to be valid XML, but I must say I don't like constructs like <input value="<c:out ...>">. Tag inside other tag's attribute - this looks horrible. That is why now I think that template directives should not be constructed with XML tags: so all those custom tags, JSTL tags etc is the wrong direction. Such code is simply too hard to read, because it resembles too much HTML (ok, templates not always are used to generate HTML, but I guess in about 95% of cases they are). The better approach is to use some special characters, like # directives in Velocity, or [# in FreeMarker, or EL syntax ${...} in JSP/JSTL (but EL is too much limited and it is actually not directive; besides assumption that only getters can be called from EL was serious mistake: e.g. you cannot check collection size, because there is no getSize() method, only size()). Compare the if-then-else block created with JSP/JSTL:


<c:choose>
<c:when test="${expr1}">
...
</c:when>
<c:when test="${expr2}">
...
</c:when>
<c:otherwise>
...
</c:otherwise>
</c:choose>

with the same written with Velocity:

#if (expr1)
...
#elseif (expr1)
...
#else
...
#end

Which one is easier to read?


FreeMarker can be also good replacement for existing projects that already use XSLT. From what I see, you can bind the XML data document to some variable, and then access it with XPath queries from FreeMarker template to extract data. Velocity offers similar thing, it's called DVSL, but I doesn't look good to me, because it applies the same functional, other-way-round-alien-looking "apply-templates" approach as XSLT.


Velocity or FreeMarker integrates also well with Spring. Form my point of view, the only serious drawback of those technologies when compared to JSP, is IDE support. In the company where I do my painful job with XSLT, "the only IDE" is NetBeans (I think it is not obligatory, but simply all guys use it and all projects are not build with external Ant script or Maven, but simply by NetBeans, so it is hard to use other IDE anyway). I tried to find some plugin with Velocity or FreeMarker support for Netbeans (at least for syntax highlighting), but looks like there is no one. That's really strange for me - those technologies are on the market for many years, and are quite popular I think. So why there is no support in second most popular Java IDE for them? For Eclipse the situation is better, from what I see.


So if you start new project, think twice (or ten times) before jumping into XSLT. And if you use Eclipse, you can even think twice before using JSP/JSTL. Velocity or FreeMarker might be a better option.


Footnote: this article was selected for the "Developer's Perspective" column at the Javalobby News newsletter from Apr 21, 2009; and also reposted on DZone, where it has started very interesting discussion. If you are interested, look at comments there: you will also find some good points in favor of XSLT, to complete the picture.

Tuesday, April 7, 2009

Relocating resources with RESTful web services

Recently I spent long time on redesigning internals of a RESTful web service. In that service, I try to follow the rules presented in "REST canonical reference" book RESTful Web Services . I decided to use "meaningful" URIs based on resource names. Resource name is editable, so when it is changed, the URI must change too (another solution is to use "non-meaniningful" URIs based on immutable, usually auto-generated IDs of the resources). Say I have the resource named "myProject" addressed by URI http://myservice.org/v1/Projects/myProject. To change its name to "ourProject" one needs to send PUT request to this URI with the new name in request body. Say we use form-encoded representations in request, then we would send PUT request to http://myservice.org/v1/Projects/myProject with request body name=ourProject. Server processes the result, changes the name in the database, and must send the response back to the client. What should be the response? Here the interesting part starts.


First implementation


According to the book, if the PUT request causes the URI to change, the response should be 301 Moved Permanently with the header Location: http://myservice.org/v1/Projects/ourProject in order to let the client know the new location. I've implemented it this way, using Spring 3.0 REST support (with a bunch of my custom classes and annotations):



@RequestMapping(method=PUT, value="/projects/{projectName}")
public HttpResponse createOrUpdate(@PathVariable String projectName, PropertyValues newValues, @RequestUrl String url) {

SaveResult result = projectService.createOrUpdate(projectName, newValues);
if (result.isCreated()) {
return new HttpResponse(SC_CREATED).setHeader("Location", url);
} else if (result.isRenamed()) {
String newUrl = url.substring(0, url.lastIndexOf("/") + 1) + result.getObject().getName();
return new HttpResponse(SC_MOVED_PERMANENTLY).setHeader("Location", newUrl);
} else {
return new HttpResponse(SC_OK);
}
}

So in case when resource was renamed, I construct a new URL and return 301 code with new location. This should work, but it didn't. It looked like this response code triggered immediate GET request to the new location. Initially I thought that it is done internally on server either by Spring or Tomcat, but it turned out to be triggered by client. The same behavior happened for different clients: Firefox, IE, and pure Java client used for testing web services, the Wiztools.org RESTClient. I don't know what HTTP Java library is used internally by the latter tool, but I guess the behavior is implemented in this library, not in the client itself.


I started to search some information about this problem, but Google was of very little help for queries like "PUT with 301 Moved Permanently". Lastly I looked at RFC2616, the HTTP 1.1 specification. This is an excerpt with description of code 301:


If the 301 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.

So it looks like all browsers and other clients are broken in this area: they should not redirect automatically to the new location. So I had to find some workaround for this.


A bit of theory behind it


Suppose that after issuing PUT request which returned 301 code, the browser actually did ask me for permission for redirection. And suppose I gave the permission. What should the browser (or other client) do? Obviously, send the request to the new URL. But what request? GET? or PUT (as initial request was PUT)? What does 301 response actually mean in this case? My first thinking was: I send PUT request to http://myservice.org/v1/Projects/myProject, which changes the data of that resource. If URL is not changed, response is 200. If URL is changed (say to http://myservice.org/v1/Projects/ourProject) as the result of new data submission, server updates the resource, and let me know the new URL with 301 response containing Location header. (At least, this is the logic roughly presented in the aforementioned book, I think). So, is there any point in doing any redirection in such case? The 301 code is only information here, not obligation to do any redirect. I sent PUT request to update the resources, not GET to retrieve it. So sending (either without permission, or with permission) GET to the redirected resource makes no sense, as I didn't wanted to retrieve the resource, only update it.


But then I looked at HTTP spec again, at general description of 3xx codes series:


This class of status code indicates that further action needs to be taken by the user agent in order to fulfill the request. The action required MAY be carried out by the user agent without interaction with the user if and only if the method used in the second request is GET or HEAD.

This clearly indicates that action needs to be taken, not can be taken. Interesting. So I infer that 301 code is actually obligation for redirection. (BTW this quote says that action can be done without user interaction only if second request is GET or HEAD. At the same time, first quote says the same, but it mention first request. Is it a bug in the spec? But logically, both make sense...) So perhaps my thinking was wrong. Imagine different situation: I retrieve myProject representation. After some time, I want to update its description, so I send PUT with new description to the http://myservice.org/v1/Projects/myProject URL. But in the meantime, another user changed this project name to ourProject. If the server is able to remember such changes, when my request to http://myservice.org/v1/Projects/myProject comes, it sees that myProject no longer exists, but remembers it was moved to http://myservice.org/v1/Projects/ourProject, so it sends 301 response with new location, telling me "this is a new location our your resource, send your PUT request there". Now browser can ask me for confirmation "resource has been relocated, should I resend the same PUT request to the new URL?". If I say "Yes", it sends PUT to the new location and updates description of ourProject. So the second request should be PUT, not GET. The note from spec found below 301 description also indirectly points out that original method should be preserved for second request:


Note: When automatically redirecting a POST request after receiving a 301 status code, some existing HTTP/1.0 user agents will erroneously change it into a GET request.

Well, looks like not only some existing HTTP/1.0 user agents, but also modern HTTP/1.1 clients do...


(Note: in the presented scenario the 409 response indicating concurrent modification problem would be probably more accurate; if another user renamed the resource, it could also have changed the description, so actually the current user could be more interested in retrieving the new representation first, and only then updating it.)


But GET is safe, right?


Obviously, the confirmation (in any form, say dialog box) would work better in case of client being a human. If the client is a program (and in case of REST web services it usually will be), for example single-page, rich-client JavaScript application, the confirmation would be a bit more problematic. But that's academic discussion at this moment, because browsers are buggy for 301 codes and don't ask for confirmation anyway. Even if they did, this would be probably beyond control of actual JavaScript application (like in the case of basic authentication dialog pop-up), so it would be of very little practical usage.


So what is the solution? The first idea: in valid REST service, GET requests are safe, so sending such request automatically by the browser after obtaining 301 code would do no harm, apart from increasing a bit the load on the server (without real need). But I see two problems here: the redirection is handled by browser (or other HTTP library) in automatic way, so the actual client don't have access to the response from first request (PUT), only to the one from second request. If response to PUT request contained some special, important headers, they will be lost. But there is also second, more important problem. In case of my service, I use basic HTTP authentication, with some small trick to bypass browser authentication dialog (I'll describe this technique in one of the next posts). After user gets authenticated, I set default authentication header for all Ajax request, using Ext-JS code Ajax.defaultHeaders = {Authentication : Basic HASH}. But it happens that in case of 301 response, the second, after-redirection request is launched without Ext-JS awareness, so the authentication headers are not applied, and the actual response to that request is error 401 or 403. So this seems like a dead-end.


Say goodbye to 301


After thinking about it, I came to the conclusion that probably authors of the book were wrong: in the case of URL change, as a direct result of PUT request, the 301 code is not the proper response. This conclusion is based on the aforementioned phrase from HTTP 1.1 spec, describing 3xx codes as indication that further action needs to be taken by the user agent in order to fulfill the request. So 301 response is not meant to be just an information about resource metadata change. It is rather a kind of directive for the client: actually, most people using HTTP client libraries written in different languages, usually expect the 3xx redirections to be handled automatically be the library itself.


So what response code should be used? The 204 No Content seems to be the best choice - see the spec description for this code:


The server has fulfilled the request but does not need to return an entity-body, and might want to return updated metainformation. The response MAY include new or updated metainformation in the form of entity-headers, which if present SHOULD be associated with the requested variant.

This looks like perfect fit. Server fulfilled the request and updated the resource, but doesn't have to return the resource representation, as it is already known to the client. But as the result of the change, part of object metadata has been changed: namely the URI of the resource. So the response contains Location header with updated metadata. Simple, and works well with all (or most) HTTP clients.

Monday, March 16, 2009

Test drive of Spring 3.0 Milestone 2 REST support

I'm developing a proof-of-concept application based on REST architecture, with pure-JavaScript client written in Ext-JS, and server side based on Spring. Until now, I've used Spring 2.5 to create REST service, but it was a bit problematic: it required me to write too many repetitive code. I've read about JAX-RS and it sound very good to me. There are some implementations already available, but I didn't want to learn and add yet another product to the already challenging technology stack. As it was known at that time that Spring 3.0 is going to include REST support, I decided to wait a bit for it, expecting full JAX-RS support in Spring. Now, after reading Arjen's blog entry, I see JAX-RS will not be implemented: that's a pity, because JAX-RS seems very well designed for me; but ok, if Spring's solution delievers the same value, this doesn't matter much. So today, after reading Arjen's introduction, I switched to 3.0 M2 to see how it fits my needs, and I'd like to share my results.


The most wanted thing, the URI templates are of great help obviously. But this is only the minimum of what I expected. After playing a bit with spring 3.0, I must say I'm a bit disappointed. I simply expected much much more. It seems to me that Spring REST is targeted at server-side web applications, the ones I call "half-REST" applications. This is not what I expected: because Spring REST support is done by Arjen Poutsma, the author of Spring Web Services, I thought REST support was going to be comparable with Spring Web Services. I really like the Spring Web Services approach, focused on real services with data, not some HTML-oriented views. Unfortunately, Spring REST doesn't seem to be like this. It doesn't help much writing real web service which return data, instead of HTML-oriented views. For example, shallow ETag support and HTTP-over-POST method conversion are nice things for "half-REST" application, but they are of little value for fully RESTful applications. In my REST service I plan to rather go for deep ETag, based on database versions, cache states etc, and I use real HTTP methods: PUT, DELETE, instead of PUT-over-POST.


Requirements and assumptions


My application consists of two parts: server side, written in Java, based on Spring, and client-side, written in Ext-JS. Server side exposes the web service based on REST principles, following the guidelines presented in the great RESTful Web Services book. Service returns data in JSON format (I decided JSON better fits my needs than XML, as the client is written in JavaScript, so JSON processing will be faster, and bandwidth usage will be smaller; service can be used also by any other tool, as JSON is widely supported in virtually any language). It accepts POST/PUT request body data mostly as form-encoded pairs, but in some more complicated cases it can accept JSON data too.


Now say we have a resource called "Project". It is addressable by http://mydomain/restservice/v1/projects. GET to this address returns list of projects in JSON format. GET to http://mydomain/restservice/v1/projects/MyProject returns details of MyProject. DELETE to this URL removes this project. PUT to this URL creates or updates MyProject, using form-encoded data from request body (during creation, client knows the URL of project, that is why we use PUT; if we have used database IDs in URL, than creation would be done by POST to http://mydomain/restservice/v1/projects).


This URL may be a base for other related resources, e.g. http://mydomain/restservice/v1/projects/MyProject/tasks will return tasks for MyProject, http://mydomain/restservice/v1/projects/MyProject/tasks/Development will return details about Development task in MyProject, http://mydomain/restservice/v1/projects/MyProject/assignments will return assignment list for MyProject, ... etc.


Note, that those URLs actually represent virtual directories. This means that common rule "trailing slash doesn't matter" (especially for URL representing "list of resources") should be respected: http://mydomain/restservice/v1/projects address is equivalent to http://mydomain/restservice/v1/projects/ address. If you consider that each project is also the container for its tasks or assignments, then also http://mydomain/restservice/v1/projects/MyProject should be equivalent to http://mydomain/restservice/v1/projects/MyProject/, etc.


Each request to the service must return proper response code: 200,201,301 (if data in PUT request changes the resource address), 304 (in conditional GETs), 400, etc. In some cases it must contain proper body (here in JSON format) and proper headers (e.g. Location header in case of 301 response). My services misses only one RESTful requirement: connectivity of representations. In my opinion application of this principle to javascript client, that must be already programmed against known service, doesn't add any value, it only increases bandwidth usage (ok, maybe some very intelligent client could take advantage of it; but for a while, it would be too complicated for my case). JSON representations doesn't fit well to handle links: if I have served linked representations, I would probably have to use XML instead. I also don't use content negotiation: all representations are JSON. Again, this would be overkill for my service, which is primary targeted at predefined JavaScript client. If it was more general, public service, this should be probably refactored.


Implementation with Spring 2.5


Let's implement the Project service with aid of Spring. Annotation-based Spring MVC introduced in version 2.5 seemed to match quite well my requirements, and that is why I decided to use it. So let's start with Spring 2.5, and analyse all the problems I faced; then we'll see which of them are solved by Spring 3.0.M2.


I have resource "Project" with some actions to be performed on it: so logical choice is to put all operations related to Projects into one controller. Base URL of this controller is http://mydomain/restservice/v1/projects or equivalent with trailing slash http://mydomain/restservice/v1/projects/. Some actions (GET project list, POST - if we have used it) are done directly on this URL. Some others (GET project, PUT, DELETE) are done on http://mydomain/restservice/v1/projects/{projectName} URL (or, potentially, equivalent one with trailing slash http://mydomain/restservice/v1/projects/{projectName}/). There can be also other controllers in my service, either completely unrelated to Project resource, like User controller (say http://mydomain/restservice/v1/users) or related to Projects, like tasks controller (http://mydomain/restservice/v1/projects/{projectName}/tasks) or assignments controller (http://mydomain/restservice/v1/projects/{projectName}/assignments). I won't cover those controllers, but we should keep in mind that framework should be able to match the most specific controller properly, that is for request for http://mydomain/restservice/v1/projects/{projectName}/assignments the assignments controller should be picked up, not the projects controller.


So, I would like to write the ProjectController, mapped to http://mydomain/restservice/v1/projects (with or without trailing slash - let's call it "base URL" for now), with 5 methods (method names doesn't matter here): list() mapped to GET with base URL, show() mapped to GET with base+projectName URL, createOrUpdate() mapped to PUT with base+projectName URL, and remove() mapped to DELETE with base+projectName URL. Something like this (assume that URL part http://mydomain/restservice/v1 is already mapped to dispatcher servlet) :



@Controller
@RequestMapping("/projects")
public class ProjectEndpoint {

@RequestMapping(method=GET)
public HttpResponse list() {...}

@RequestMapping(method=GET, value="*")
public HttpResponse show() {...}

@RequestMapping(method=PUT, value="*")
public HttpResponse createOrUpdate() {...}

@RequestMapping(method=DELETE, value="*")
public HttpResponse remove() {...}
}

I would like to take advantage of Spring's @RequestMapping annotation ability to merge class-level path value (/projects) with method-level relative paths. But it doesn't work. To make it work, it seems that class-level path must end with wildcard, and all methods must have some URL suffix. But you can't have methods like list() and show() in this case, one mapped to base URL, and onother to base URL + suffix. This is annoying shortcoming of Spring MVC. There are also many other problems here: The star character (*) matches zero or more characters. That's wrong, because method show() will be matched even for request when no project name was passed. There should be something like "+" character, meaning "one or more characters". Also trailing slashes are real problem here. If I mark controller class with @RequestMapping("/projects") (as in the code above) then only request without trailing slash will be matched. If trailing slash is appended to the actual request, we get 404 response. What's worse, method-level annotation will not be appended to base URL in this case. So we have to change it to @RequestMapping("/projects/*"). Now methods will work, but if you send request to base url to get list of projects, and forget the slash at the end, you will get 404 response again. That's very bad in my opinion. JAX-RS handles it properly: see JAX-RS's @Path Javadoc. Finally, the only solution that works is to omit the "base URL" on class level, and repeat it at each method level (without trailing slashes). This works, but requires mapping repetition, and you can't use trailing slashes in actual requests, unfortunately.


Second problem: in method show() we actually need to extract the project name from URL. Spring 2.5 doesn't offer any help for this, we have to manually parse the URL.


For PUT and POST method I need to get access to request body. For form-encoded data it is not that complicated: I may use @RequestParam annotated params, I may use @InitBinder to initialize command object etc, I may also access directly Request object and call getParameter(). However I faced third problem here: getParameter() works only for POST method, and not for PUT, even if request body is the same. I had not dig into this issue too deeply, but it looks like this has a root in servlet API specification flaw (or Tomcat implementation). However, it would be nice if REST support framework handled it properly, for example by wrapping the request object. In my case, I had to write myself the method to extract request parameters from POST and PUT requests in uniform way:



public static Map<String, Object> decodeFormEncodedParamsFromBody(HttpServletRequest req) {
try {
Map results = new HashMap<String,Object>();
String inputString = IOUtils.toString(req.getReader());
// Parse a x-www-form-urlencoded string
String[] pairs = inputString.split("\\&");
String encoding = req.getCharacterEncoding() != null ? req.getCharacterEncoding() : "UTF-8";
for (String pair : pairs) {
String[] fields = pair.split("=");
String name = URLDecoder.decode(fields[0], encoding);
String value;
if (fields.length > 1) {
value = URLDecoder.decode(fields[1], encoding);
} else {
value = "";
}
if (results.containsKey(name)) {
List<String> values = (List<String>)results.get(name);
values.add(value);
} else {
results.put(name, value);
}
}
return results;
} catch (IOException e) {
throw new RuntimeException(e);
}
}

What if POST or PUT body is in JSON form? I would like to bind it automatically to some "object-oriented" representation: either use some JSON unmarshaller and transform it to Project bean; or bind it to JSONObject from JSON-Lib library, or use other JSON library. For example, in my case I would like to be able to write method like this:



@RequestMapping(method=PUT)
public HttpResponse createOrUpdate(JSONObject projectJson) {...}

But Spring's @InitBinder is not able to bind request body, only request params. After some research I found solution: AnnotationMethodHandlerAdapter has a method setCustomArgumentResolver() taking WebArgumentResolver implementation, which can be used for this purpose. One have to write the resolver and register it in spring context. This is how such resolver could look like:



public class JsonLibJSONObjectArgumentResolver implements WebArgumentResolver {

public Object resolveArgument(MethodParameter methodParameter, NativeWebRequest webRequest) throws IOException {
if (methodParameter.getParameterType().equals(JSONObject.class)) {
ServletRequest req = (ServletRequest)webRequest.getNativeRequest();
String inputString = IOUtils.toString(req.getReader());
JSONObject json = JSONObject.fromObject(inputString);
return json;
}
return UNRESOLVED;
}

}

Last problem relates to returning results from my methods. For GET methods I have to return JSON representations. For PUT, POST, DELETE there is actually no representation: only response code. Each response may (or sometimes must) contain also some headers. I could use JSONView for this, which was around for some time, but in fact I didn't like this approach that much. My general feeling is that Spring MVC is a result of the dominating trend in the past of abstracting away the HTTP nature of web applications: hide HttpRequest and HttpResponse, and deal with models, views, controllers. JSF is the best exmple of this approach. Spring MVC, though closer to HTTP request-response model, still tries to put abstraction layer on top of this, with its views and view resolvers. Perhaps it is good for server-side web applications and frameworks, that use forwards and redirects, templates, includes etc. But for REST service, that accepts and returns representations which are mostly data (like JSON), not "views", this abstraction may be unnecessary. In my case, in my controller methods I simply want to say "return this JSON object" or "return status this, with header this, and with JSON body that". I don't need views, view resolvers etc. The perfect example of this approach is Spring Web Services: you have an endpoint with some methods on it, each method takes some XML (or its part specified with XPath, or its unmarshalled representation by autoconversion) and returns some XML (or some object which is marshalled then). You have adapters for different XML libraries and different XOM libraries. That's great, and I expected something similar from Spring for building REST based web services.


I wasn't able to find any solution how to return simply JSONObject or something similar from controller method. See also this thread on Spring forum for details. Finally I came to the conclusion that I will use Spring views, and created HttpResponse view:



public abstract class HttpResponse implements View {

int statusCode;
Map<String, String> headers;

public HttpResponse(int statusCode) {
this.statusCode = statusCode;
}

public HttpResponse(int statusCode, Map<String, String> headers) {
this.statusCode = statusCode;
this.headers = Collections.unmodifiableMap(headers);
}

public abstract String getContentType();

public void render(Map model, HttpServletRequest request, HttpServletResponse response) throws Exception {
response.setStatus(statusCode);
if (headers != null) {
for (Map.Entry<String, String> header : headers.entrySet()) {
response.setHeader(header.getKey(), header.getValue());
}
}
}
}

I have two implementations: NoBodyResponse and JSONObjectResponse. Names clearly define their meaning. JSONObjectResponse takes JSONObject (from JSONLib) in constructor, and writes it to HTTP response body. Similar implementations could be created for other JSON or XML libraries, or for XOM or JOM (Json-object-marshaller) libraries. This is a kind of basic replacement for ResponseBuilder from JAX-RS.


Implementation with Spring 3.0 M2


Now let's look what we can simplify after migration to Spring 3.0 M2. The @PathVariable introduced in M1 is a great help, as I no longer have to parse the URLs manually. This reduces code duplication and is less error-prone. So now my example of hypothetical ProjectEndpoint could look like this:



@Controller
@RequestMapping("/projects")
public class ProjectEndpoint {

@RequestMapping(method=GET)
public HttpResponse list() {...}

@RequestMapping(method=GET, value="{projectName}")
public HttpResponse show(@PathVariable String projectName) {...}

@RequestMapping(method=PUT, value="{projectName}")
public HttpResponse createOrUpdate(@PathVariable String projectName) {...}

@RequestMapping(method=DELETE, value="{projectName}")
public HttpResponse remove(@PathVariable String projectName) {...}
}

BTW: I've found already a bug in @PathVariable processing: if you have method taking for example id, which is long, marked with @PathVariable, then the id from URL will be automatically converted to long. However if the actual URL contains characters nonconvertible to long, then exception is thrown, and you get 500 response. This is wrong, as this is clearly client error, so the response should be 400 (bad request), not 500. I haven't checked yet if it was already reported.


Unfortunely, the code above still doesn't work. Apart from @PathVariable, nothing has actually changed (at least from what I observed): you cannot map class to /projects nor /projects/, you have to map it to /projects/*. Now you cannot map list() method to do GET for base URL, and another show() method to do GET for sub-URL in the single controller - this will not work, for any reason. So still you have to throw away class-level mapping and copy base URL to each method, to make it work. And trailing slashes are still broken: you can't use it in request URL. So actually we haven't improved the code too much. Looks like @RequstMapping processing was broken in 2.5 and is so in 3.0.


Let's check other points: getParameter() for form-encoded PUT request body doesn't work in Spring 3.0 too, so all Spring autobinding feature and @RequestParam will fail for PUT requests. That's really a pity. Next: automatic binding of request body to some method param, like JSONObject, or JAXB binding object: not much progress here too, but fortunately we can use the setCustomArgumentResolver() that I demonstrated earlier. Next: HttpResponse View or something similar, to easily set response code or response header - nothing at all. Arjen mentions also JacksonJsonView, but it is not in Spring 3.0, but in Spring JavaScript - hello? What JavaScript? I'm building server-side web service here, so why on earth should I use Spring JavaScript module? This makes no sense. JSON is not JavaScript: it's data exchange format, which you can use for example to talk to .NET from Java, if you wish: no JavaScript is involved here.


And one more issue. I said earlier that I don't use content negotation, because I serve only JSON. But suppose that I have need to serve also XML now, and use content negotiation, based on Accept header from client. How to accomplish this? JAX-RS has @Produces annotation for this: simply by marking two methods or classes with this annotation with different value, we are sure that framework will pick up the valid method, according to user request. How to do it in Spring 3.0? From what I understood, I should use the new ContentNegotiatingViewResolver. Is it better solution then JAX-WS? I haven't try it yet, so I can't say much about it. But I can think about different use cases. Let's assume that only one method of my controller is supposed to be content-negotiation aware. For example, GET method may return JSON if Accept: application/json was passed, or the PDF document if application/pdf was passed (as it's less likely that PUT method is going to use PDF document as input here). In such a case JAX-RS solution is a winner: I would simply have two methods for GET, first marked with @Produces("application/json") - it could return my JSONObjectResponse view, and second method marked with @Produces("application/pdf"), returning some PDF view (or model). On the other hand, if my service produces/accepts JSONs and XMLs, it's higly likely that each method will be doubled: one producing/accepting XML, and one for JSON. Then perhaps it would be better to have two controllers instead: e.g. ProductEndpointXML and ProductEndpointJSON. In such a case probably @RequestMapping annotation should be enhanced to take also header values into account (currently it takes path, HTTP method, and request params, but no headers), to be able to route request to proper controller.


As you see, the only real asset of Spring 3.0 for REST in my application is @PathVariable. @PathVariable is really nice, but it is very very small improvement for the major realease of Spring, having said for long time that REST support is one of the key additions in this release. (To be honest I have to mention here small improvements from 3.0 M1: new @RequestHeader annotation and default value in @RequestParam - this is really helpful in RESTful applications too.) Especially if you compare Spring Web Services for building SOAP-oriented web services against Spring 3 REST support for building REST-oriented web services, the latter seems very very poor. If I remember correctly, Arjen said some time ago that he considered building REST support based on Spring Web Services, but decided to go with Spring MVC instead. Looking at Spring 3.0 M2 I'm not really sure if this was the best decision. Also comparison with JAX-RS shows some Spring shortcomings. Look for example on Bill Burke's JAX-RS intro atricle: it describes pretty much interesting stuff, many of which having no counterpart in Spring 3.0. On the other hand, it is still only M2 for Spring 3.0. I hope M3 will significantly improve support for real RESTful web services. If not, I will have to search for different solutions: some JAX-RS implementation with Spring integration, or perhaps Grails?

Summary


So what should be done for REST support in Spring 3.0, to be fully functional and in line with JAX-RS and Spring Web Services? Spring 3.0 should meet in my opinion following requirements:

  • One should be able to write single controller for single logical resource, with base URL mapping declared on class, and sub-URLs mapping declared on some methods (GET,PUT,DELETE); other methods without sub-URL mapping (GET,POST) refer to base URL. Note that controllers can create "nested" structure (e.g. one for project, other for project's tasks). Also trailing slashes in "virtual directory" URLs should be handled properly (trailing slashes should not influence the response).
    Some ideas here: first, all the method-level paths, if they are not absolute, should be treated as relative to class-level path. According to @RequestMapping Javadoc in 2.5, it should behave like this already in 2.5; but from my observations it did only if class-level path ended with wildcard; so in fact fixing it should not break backward compatibility, and should be treated rather as bug fix. @PathVariable fixes already problem of * character matching zero characters, that I mentioned earlier (I assume @PathVariable matches one or more characters only). Problem of trailing slash can be solved this way: first the request URL is matched literally. It there is a match, then continue normally. If there is no match, then look at request URL: if ends with slash, remove it, if it doesn't, add it - and then try the whole matching again.

  • @PathVariable with type conversion should give error 400, not 500, if conversion fails

  • Proper handling of Request.getParameter() for PUT (the same as for POST) - probably by wrapping PUT requests

  • Getting easy access to request body, with automatic conversion/marshalling (e.g. to JSONObject, some XML representations, JAXB binding result, ...), preferably as method param, perhaps with some annotation

  • HttpResponse View implementation out of the box, allowing setting response code and headers, with different implementations for "body-full" responses (JSONObject, XML representations, marshalling, ...)

  • Simplified content-negotiation: for example by extending @RequestMapping annotation to be able to match request headers too (at least Accept header)

  • Move JSON support from Spring JavaScript project to Spring core


This is not exhaustive list obviously. It is based on my observations for specific service I'm building. But I believe in the future there will be more and more such applications and "real" REST services, so it would be really good to have Spring support them fully, not shallowly.