Dragons in the Algorithm
Adventures in Programming
by Michael Chermside

Using a Mix of Computers and Humans for Security

Suppose that your bank offers currency conversion as a service: give them a deposit or make a withdrawal in euros and they'll adjust your balance in dollars. They don't do this out of the goodness of their hearts: today's conversion rate is around 1.28 $ / €, so they'd give you 0.75 € for every $ and 1.25 $ for every € so they'd make a good 6.5% margin on the conversions.

By the way, what's the conversion rate on really small amounts? Because of rounding, if you withdraw 1 US cent, converted to euros, they would owe you 3/4 of a euro cent, and rounded to the nearest cent that would be 1 euro cent.

So you walk into the bank and deposit $10. You then tell the teller you would like to make a 1-euro-cent withdrawal. You pocket the euro cent and your balance is $9.99. Ask for another euro cent and your balance is $9.98. Keep going and eventually you'll have 10.00 € (in euro pennies). Now deposit this and they'll give you $12.50. You just made $2.50. Stand there all day repeating this and you could make some real money.

Of course, this would never work. The teller might smile and give you the first euro cent, but by the second or third she'll know that something is up. She may not figure out your currency scheme instantly (although probably she will), but she'll tell you to get out and stop wasting her time. But if you were working with a computer instead of a cashier, then it just *might* work!

The interesting thing to note here is that humans and computers have very different failure modes. There are some kinds of tricks that humans fall for easily, like those that involve distraction or appeal to authority. Whereas computers won't fall for any trick they've been programmed to defend against, but if you find something they are not programmed to expect, then you can exploit it over and over.

That is why robust security mechanisms combine both computers and humans, to get the best of both worlds. Usually this means that the computers do the transaction (it's cheaper to offer online deposits and withdrawals than it is to pay a teller to work in a branch on every street corner), but you don't just rely on the computers to keep things legit. You also have a security team who monitor the transactions and look for anything out of the ordinary. They then investigate -- usually it's just a coincidence or an unusual day, but sometimes it is an attempt at fraud or two friends who made a bet about who could generate the longest bank statement for the month of July. Then the security department can step in.

To put it succinctly and steal a bit from Lincoln: Some of the time you can fool computers all of the time. You can fool all humans but only some of the time. Fooling both is much harder.

Posted Tue 10 January 2012 by mcherm in Security