It all starts with a blog

02 Apr 2012

Almost two months ago I quit my job to work on a project I had an idea for. Ok, mostly I quit because I was hired for something I was really interested in – hadoop / big data / datamining related stuff – but when no projects materialized in that area for my employer I ended up getting sourced out to a project I had no interest working on – namely converting mainframe / cobol based production print applications to a unix / perl / xml configuration based platform. I stuck around for the duration of the project, but when it came to close and it didn’t look like anything related to datamining would be coming along soon, I said my farewell and set out to start up my own project.

Nearly two months later, what do I have to show for myself – not a whole lot. The basic idea was to build a simple mobile application (starting with Android) which would enable volunteers to collect and anonymously publish their geolocational movement data to a public data set that researchers or anybody else could then use as a reference dataset for all sorts of projects. It’s the kind of data I wished I had access too when I was working on datamining, but I thought they only way to ever get access to that type of data would either be get a job at Google, Apple, Navtech (or some other company that collects tons of geolocational data) or create some sort of trojan horse app that offers some minimal benefit but surreptitiously collects users movement data behind the scenes. Then it hit me – I don’t need to trick people into handing that information over – I just don’t have it in me – instead I can just ask them. People volunteer information and services to organizations like wikipedia, openstreetmaps.org and SETI@HOME all the time. If I make it safe, painless and easy enough, maybe people would volunteer their movement data to a similarly minded project.

That was the idea anyway. I’d start out by building a basic android prototype that would just track points in time and write it out to a file on my phone, then I’d add in a bit of anonymization, and have it push the info to a simple webservice. I figured I’d be able to put a basic android prototype app together in a couple weeks. Did I say this was nearly two months ago? I’ll admit, android has turned out to be slightly less trivial to pick up than I expected, but android hasn’t really been the problem. I’ve been the problem. I’ve been procrastinating. When I should be sitting down writing some code in eclipse, I’m either complaining to myself about how the color scheme is too bright and hurts my eyes, and can’t I fix that (and then spending the next two days going off on a tangent in that direction which inevitably doesn’t work nearly as well as just sucking it up and going back to the default color scheme) or I decide it’s a great day to bake some bread (no complaints from my wife there) or I get distracted by the pile of dirty dishes in the kitchen (which resulted from all those days I spend in the kitchen baking bread or cooking something else…)

I’ve also been catching up on video lectures – Coursera is great. So are the lectures at Berkeley as well as Yale. But then I don’t really like to watch or listen to the lectures on my computer, and generally prefer to speed them up at least 50% or so. That means writing scripts to download them enmass (combinations of curl, sed, ruby, cclive, etc) and other ones to convert/speed them up to the appropriate formats that work on my audio and video media devices (mencoder, mplayer, gstreamer, etc). I probably lose as much time looking into the command line options as I save by listening to the tracks sped up. Still, it helps – if I watch or listen from my computer I end up getting distracted and looking things up in the middle of a lecture. I find I pay much better attention to the contents of the lecture itself if I can just relax, sit back and disconnect from the world for a while. The same is true for reading a book which is why I have no interest purchasing an e-reader with wireless access to the internet. It’s for reading a book! I only need two buttons – page forward and page back. I still love my 4 year old Sony prs-505 – it was three years ahead of its time.

So none of this is to say I’ve been “unproductive” over the last two months, just that I haven’t been particularly effective at staying on track… One thing I need to focus on is incremental improvement. I tend to be a perfectionist and focus too much on delivering a production worthy product rather than just getting something to work. I’m not saying the end product doesn’t need to be production worthy, but it’s not where you start. That’s where I’m hoping this blog will help out. It’s not the end product, but it’s a place holder for the site that will one day host the project. The blog is also more for me than for you – I’m hoping it will help me organize my thoughts and keep me on track – although I’ll admit right now that not every blog post will have anything to do with the project. Writing has alwyas been therapeutic for me and I’ll probably use this blog to expel some of the random ideas that distract me from the project just so I can put them to the side and focus back on the task at hand. Finally, although I say the blog is mostly for me, there’s still a good reason for making it public. This is all psychological trickery. If I say something publicly it’s more of a commitment. If I tell the world I’m going to do this, then it will help motivate me to actually do it – else I look like a failure the the whole world (even if nobody’s watching =).

So there it is, my first blog entry for project snail trail. I know I haven’t talked much about the project itself, but in the coming days I will. Eventually I’ll even get around to describing the mission/vision/etc of the project in pages rather than simple blog entries, but like I said, I need to start somewhere if only to get started.