Dragons in the Algorithm
Adventures in Programming
by Michael Chermside

A Creaky Old Man's Defense of Creaky old Code

I'm beginning to wonder whether I'm turning into an "old fogey" of a computer programmer. You know, the elderly fellow who has an office near the server room; the one who always objects to the use of any new technology and goes on at length about "the good old days" — you know, back when Real Programmers punched their cards out by hand.

Man with White BeardThe signs seem to be there: I am constantly having this same conversation with various people at my current job. It goes something like this: People say "Oh, why don't we use 'FooBaz'!" (some recent piece of fancy technology). Then I respond saying "Hmm... well, we could, but wouldn't that be incompatible with 'BarQuux'?" (some old custom-built thing that we have in our infrastructure). The person replies, "Oh, but BarQuux is an old custom-built piece of junk and it's not standard, and we should just throw it out and replace it with FooBaz." I try to reply "Can you actually compare the features of BarQuux and FooBaz? Does FooBaz do the things that we use BarQuux for?" but by this time they're not listening anymore. My willful refusal to realize that technology marches onward and my untreated Not-Invented-Here syndrome have caused them to set the bozo bit.

It's not that I'm skeptical of new technology just because it's new; rather it's that I'm conservative when it comes to things we have already built and are already working. I am not trying to defend the status quo (there is lots of room for improvement) but that I'm trying to insist that people take into account what is already working and not disturb the status quo unless they can show an absolute benefit to it. A particular example should illustrate what I mean.

Some years back, we built our own database connection pooling. We noticed that the only feature of WebSphere that we were actually using was its database connection pool and if we replaced that we could run on an open source stack instead and save some significant licensing fees. So we built our own connection pool. What we built wasn't particularly fancy — connection pooling isn't really all that difficult a problem so Mark and I just spent a week or two and threw something together that would meet our needs. Over the years it's gotten a couple of features added, but it has never really received much attention.

So I hear people say "Our current connection pooling is just some home-brewed thing: we should throw it out and use the connection pooling provided by JBoss." On the surface, that sounds like a very sensible thing to do — why not go with a standard solution that's provided by somebody else and is well exercised rather than something that we threw together in-house? But it isn't quite that simple. After all, the dinky little database connection pool that we threw together has quite a few features. For instance, it has failover so that if your database goes down it can make its connections to a secondary database instead and continue functioning. It has administrative screens that we built so you can log in and test the DB connection, see the current allocation of connections and a history of pool size over time. It provides simple wrappers around the connections, statements, and result sets that it returns and these do certain logging and also take care to ensure that even if the programmer forgets to properly close their result sets, that Oracle cursors will be released (before we added that one we used to have regular database outages every few months when someone got careless and failed to properly close a cursor and eventually Oracle would run out of cursors and bring the entire system to a halt).

So where are we going to find a database connection pool that has all these features? I'd be perfectly happy to switch to something that's standard, that's not something that we built ourselves, but the people who are proposing switching the pool are doing so without any evaluation of the benefits that are in place now from the system that we are currently using. There are genuine benefits: for instance, ours cannot be monitored by SNM; it's impossible to hire a consultant who is already familiar with our pool.

And I'm not trying to suggest that we have built a database connection system that's more effective than anything out there on the market — that's simply not true! I'm not even claiming that we have one that is better than what's available as open source. If that were true, we should probably just release ours as open source and attract a little bit of free support and goodwill. The database connection pool systems that are out there need to be really robust in a way that ours currently isn't. Ours works with Oracle, with SQL Server, and with an obscure JDBC wrapper for connecting into the Profile banking system but that's really all it's ever been tested with because that's all that we've ever used it for. When we make calls into Oracle it's our standard that we always use stored procedures and always return our results through out parameters not return values. Our wrapper code that closes Oracle cursors wouldn't work properly if you returned a value from a stored procedure or did a direct SQL query instead.

This is commonly the case with software that is built in-house. They have lots of features that are interesting to you and that you use, and they are significantly lacking in certain kinds of flexibility for features that you do not use in your particular company. Whereas libraries and code for general use MUST have the flexibility (or they couldn't survive in the marketplace) but they are often lacking in depth of specific features. Now, I am not saying that it's wise to build everything in-house: there is a an enormous cost to doing so and it is often not the best choice. But people need to be aware that there are advantages to things that have "just been tossed together" in-house. And if you don't even understand what is in place now, then you're simply not qualified to be suggesting that we move to something standard.

Posted Sat 02 February 2008 by mcherm in Programming