The End to End Example

16 Jan 2007

So this project I’m on is gonna be all sorts of User-Driven, but we don’t want users posting just anything (like say pictures of 9/11 on a page about a Restaurant). But how do you know a pic is “appropriate”? Computers just ain’t smart enough to do your job for you when it comes to images. And screening all those photos for badness would take an army of dudes.

Well our friends at Amazon are willing to put you in touch with those dudes. For cheap. Their Mechanical Turk service lets you pragmatically post odd jobs to their vast electronic job board. So now you can take a photo on you site, then post a job saying “Hey look at this image and tell me if it’s really Abraham Lincoln or just more boobies and I’ll give you 5 cents.” Then somebody picks up the job and does it. You can even review their work (spot check) to see how they’re doing. Or offer the same job to 5 people and pay 1 cent apiece and see if they all agree. Awesome.

Except that it’s not so easy to use. There’s 2 problems:

One – They wanna do both SOAP and REST. So their REST API looks kinda SOAPY. With REST I should just call a http get on a URL with some params, right? Sorta. The job needs to be formatted in xml. And they want you to use URL escaping but not xml escaping. So when I use Ruby’s built in escapper: CGI.escape it escapes both url and xml reserved characters. Fine. So then I do the escaping manually and… Ruby’s URI parser won’t see it as a good URL (which is kinda good, because without xml escaping it’s what we call “unsafe”). And now I’m stuck. All I get back is a response that tells me the job is invalid. But I don’t know how it’s really supposed to look.

Two – Which leads me to my second problem with the service: Their documentation doesn’t have a good End to End example. Sure there’s some trivial ones. I found an example, in Ruby no less, that showed me how to do a REST call to query my account balance. A good start. But what I really need is a walk through of some hard problem that takes me all the way up to exactly what the URL should look like in said hard problem. Because right now I’m just guessing about the escaping being the issue. Maybe it’s something else crazy easy that I’m missing. More I think about it, that’s probably it.