Dragons in the Algorithm
Adventures in Programming
by Michael Chermside

Threadsafe Java Servlets

Web servers are inherently threaded applications: their primary purpose is to serve up a website or web application to a large number of users. Essentially all of the frameworks for creating web applications, such as Java's "servlet" specification and all of the structure built on top of it, provide built-in support for handling queries from different users simultaneously, and they make it possible for these threads to operate "safely" (without data corruption) so long as a few basic rules are followed ("Don't store anything in the servlet instance variables.", and "Don't access anything stored in static variables unless it is threadsafe.").

However, threading issues for web servers are not limited to the fact that there are multiple simultaneous users -- it is also possible to be processing multiple HTTP requests a single user at the same time. In fact, this happens all the time with requests for images. Most browsers today download images 4 at a time (or more) after the basic page has loaded. Fortunately, downloading images tends to be a read-only operation with little opportunity for threading problems. In "the old days" (in our industry that probably means last year) users would mostly view one page, wait while it finished loading, then follow a link or submit a form to a new page. So images were the only thing happening "simultaneously" for a given user, and there was little danger of data corruption.

The advent of AJAXand rich internet applications has changed all that. Modern websites don't consist of a series of pages and forms, they consist of an interactive environment where the user modifies controls which communicate with the server. Think of Yahoo Mail, Google Maps, or NetVibes. And that means that the threading issues which formerly caused very little grief are now poised to become a major issue.

What, you may ask, is so special about multiple requests from the same user? If safely handling multiple simultaneous requests from different users is a solved problem, why would safely handling multiple requests from the same user be any different? Well, two threads can run safely as long as they do not share any objects. Avoiding instance variables in the servlet (or implementing SingleThreadModel, but that's not as good a solution) prevents one possible kind of shared object. Avoiding static instance variables (except possibly those designed for threadsafe use) prevents another. And the specification for servlet says that the container is responsible for ensuring that the other objects available to the servlet are safe to use... except for one: the session.

Tomcat (a leading web application server)The session (usually the HttpSession) is used to carry around data from one call to the next within a user's session. (This is needed because the HTTP protocol is inherently stateless, but rich internet applications are quite stateful -- the state is stored in the session). Some calls (like those downloading an image) won't use the session, and those have no threading issues. But many calls will read some information from the session and write other information. If multiple threads are reading from and writing to the same object and that object is not specially designed to be threadsafe (and HttpSession is not), then it is a recipe for threading disasters.

So what is the solution? I will try to describe the some common "solutions" and why they are not satisfactory for the project I am starting on at the moment. Then (in a future post) I hope to sketch out a more complex but more capable solution which does meet my current needs.

The most common solution, the one used by the vast majority of Java web application programmers is called "pretend it doesn't matter". Whether through ignorance (probably), or a belief that "it won't matter anyway", they simply don't do anything about the threading behavior of session. And that most of the time, they get away with it - after all, data corruption due to threading behavior is rare problem, occurring in an indeterminate fashion usually impossible to replicate in a test case. So the program "nearly always" works, and the few glitches are just ignored. For me, this approach (though tempting) is simply too dangerous: one should never rely on undefined behavior, particularly not if you work for a bank: people tend to get peeved when their bank makes small, random data errors.

Another solution is simply to avoid storing any data in the user's Session. While this would work, it rather undermines the usefulness of the display tier. For most non-trivial web applications we don't want to have to re-fetch data from the database on each and every HTTP request -- we need to cache various pieces of information in the Session.

One long line slows trafficThe next simplest solution is to obtain a lock on the user's Session (or some standard object in it) before reading from or writing to it. The advantage of this approach is that it is straightforward, and that it is guaranteed to avoid concurrent access. There is even support for it in several major frameworks, such as the synchronizeOnSession property on the AbstractController in Spring MVC. The disadvantage is that different HTTP requests are handled one-at-a-time by the server. This means that we don't get to take full advantage of the massive multi-threaded server that is serving our application: sure, all hardware allows us to support hundreds of simultaneous users, but it won't speed up the experience for any one user. It also means that if we issue several requests in quick succession, they will "line up" at the server... I have seen cases where one busy page "locked up" the server with a whole series of requests so that even if the user navigated away to a different page there was still a noticable delay while the server performed the already-queued requests. And finally, it would prevent one thing which I certainly want, which is for certain long-running requests to go on in the background while the user's interactions continue. So this solution, while simple and elegant, won't meet my needs.

The next obvious idea is that instead of locking on the entire session, you can just lock on specific bits of data. So if the session contains variables "loginSuccessful", "userName", "cachedUserData", and "cachedUserPreferences" (the latter two being complex structures in their own right), then instead of obtaining a lock on the entire Session before reading or writing any of the variables, you could instead have a policy of getting a lock on an individual object before accessing it. Unfortunately, this approach is fraught with problems. One is the race condition for object creation: if two threads both check for a "cachedUserData" and find that it is missing, they might both try to load and save it simultaneously. This can be avoided (somewhat awkwardly) by having a separate lock object for every possible variable and creating all of the lock objects at Session initialization.

Unfortunately, there is a more serious problem: that of deadlock. Suppose two different HTTP requests both need to access the "userName" and the "cachedUserData". If one locks the "useName" while the other locks the "cachedUserData" then each tries to obtain the other lock, they will deadlock, and neither can continue (nor can any other thread using these locks!). The only solution that I know of for this problem is to always obtain all locks in a fixed order... but that is particularly difficult: in the general case it requires global knowledge of ALL code in the entire application.

And weighed against these tricky threading difficulties, the benefits of locking on specific bits of data are not very impressive. There will often be a single piece of data ("cachedUserPreferences" perhaps) that is used by nearly every command - in which case it reduces to being essentially a lock on the session.

So how CAN one solve this dilemma? I am hoping to come up with a solution by using immutable (copy-on-write) data structures. But this is long enough already: that will have to be a topic for a future essay.

Posted Tue 23 September 2008 by mcherm in Programming