My Own Pirate Radio

One of the big challenges in running a small software startup is service availability and support. This was, perhaps, a little easier in the bad old days when software shipped in boxes and "support" meant picking up a phone between 9-5 on business days, then mailing out a CD or floppy disc (remember those?) with updated bits. Nowadays, much of the software people are building "ships" as a service running 24×7 on the web, or at least has a online service component to it. What’s more, customers increasingly reside around the world, and expect near-realtime responses, especially if their business depends on your software service. You can imagine, then, that for a 1- or 2-person dev shop, making a web service highly available to customers is a particularly daunting challenge.

There are "business" solutions you can use, including limiting your support service level agreement and hiring more support people, perhaps in different time zones around the world. These may be appropriate depending on your customer needs and your budget.

You can also throw technology at it. Here’s what we are trying to do in that vein, on 5 Blocks Out:

(1) Ship quality software. This is motherhood and apple pie, but bears repeating. Lots of software companies still imagine they can somehow cut costs by shipping half-baked code. Sorry, it doesn’t work. If you don’t invest in writing good code in the first place, you will pay orders of magnitude more for it later in support costs and frustrated customers. The only valid exception I can think of is prototype code.

I feel like we have lots more to do here, but for small fry we’re doing well thus far. We try to be thoughtful and minimalist about what we build in the first place. Then, if it gets built, it gets tested. Rails’ built-in testing facilities and various plugins like test/spec help a great deal. Capistrano (for automated deployment and rollback) and Firebug have also proven to be vital. Next on the list is eycap.

(2) Outsource work to a Web Host. For a software startup, the question is not whether to do this, but with whom. Outsourcing the heavy lifting of buying/building and maintaining server farms, managing network bandwidth, and handling some of the basic application-level services is a no-brainer. Pay as you go, and use the time saved to focus on your core competencies. If and when your businesses gets big enough, you can always pull some of the strategic responsibilities in-house.

We currently use Hosting Rails. Generally speaking, their servers run well and their rates are reasonable. I have run into reliability problems lately, though, and despite responsive support it is becoming time to consider other hosters. Slicehost, 3Tera, and the increasingly fashionable Amazon S3 are on the "must review" list. I’ve also taken a brief look at Google App Engine, and concluded its platform is too high-level for our needs.

(3) Automate monitoring. Automated watchdogs can stay up 24×7. You, my friend, cannot. Put down the Red Bull and get yourself a suite of automated watchdogs to monitor your software’s health.

We’re getting going on this now. For starters, the Exception Notification plugin has proven itself mighty useful for after-the-fact diagnosis and debugging of Rails apps. If you’re running a Rails app, I highly recommend this plugin.

Next, you need some 3rd party ping action. Troy tells me Alertra is also supposed to be a good quality monitoring service. We are currently trying out Pingdom. The idea behind these services is to hit your site every minute (or 5 minutes, or 15… you decide) from a series of servers around the world, and let you know whether and how quickly your service is responding to HTTP, pings, and so on. When something goes wrong you get notified by email and/or SMS text message. Pingdom also collects and aggregates the data so you can look at reports over time.

Happily, Pingdom has already paid for itself: immediately after turning it on I found our service was bouncing daily (well, middle-of-nightly), sometimes down for many minutes at a time. HostingRails gave us a month of additional free hosting as compensation, and promised to watch the server more closely. We’ve had zero downtime since then. 

Going a level deeper is also essential: you need realtime monitoring built into the service itself, so that you can detect and resolve problems when their symptoms first surface, or even before then. Monit, God, and Munin are on my "must review" list for this sort of thing.

I’d love to hear back on other tools and tactics startup developers are using to build and run high-availability software services. Without losing sleep, that is.

Happy zzz….

I promised a few weeks back to join Katrin on the Mukodu Blog. This is my first Mukodu post, co-authored with Katrin, and cross-posted on MyOwnPirateRadio. Sorry, no pictures. :-)

People have been asking what we’re up to with Mukodu. While we can’t talk in detail yet on the specifics of what we’re doing, we are ready to share with you our vision, our motivation, and the company culture we intend to build.

Over the past ten years Katrin and I have lived in two countries, three different cities, and five different homes. We’ve also had the good fortune of travelling broadly. In doing so, we’ve recognized a common challenge: more and more people live in and move between urban areas, where they seek new opportunities, new friendships, and increased quality of life. Yet newcomers often don’t know where to begin, or how to get connected in a city, despite being surrounded by people. At the same time, "old-timers" often get so caught up in the routine of daily life — working, commuting, connecting with friends and family — that they fall out of touch with the changes going on around them, and with the people and resources right outside their front doors. It’s a great irony: despite living in populous, thriving, dynamic cities, we are so often disconnected. Thus the inspiration for the Mukodu blog, and for our first product, 5 Blocks Out.

5 Blocks Out is about connecting people with local life and culture. If you’re new to an area and trying to figure out which neighbourhood to call home, 5 Blocks Out will help you. If you want to learn what’s happening in the nooks and crannies of the city around you, 5 Blocks Out will keep you informed and up to date. If you want to hear what real people like you have to say about local places, events, and issues, 5 Blocks Out will help you to tap into urban word of mouth. And if you’re interested in helping others find “nearby good stuff”, thereby supporting local communities (yay!) and reducing environmental footprint (yay!), 5 Blocks Out plans to help you there too. 

Why work on a challenge like this? Because we love the places we’ve lived, and the friendships we’ve made there, and we want more of the same for everyone. We believe cities whose people engage in neighbourhood life, local culture, and civic discourse have unlimited potential as places to live. We believe healthy connections between people within neighbourhoods are essential. We believe the collective voice and wisdom of a city’s inhabitants is more interesting and, indeed, more personally relevant, than the voice of popular media. And so we’re setting out to create a community powered by the people, for the people, with the goal of making cities great places to live.

Mukodu is just getting going: for roughly the past year it’s been Katrin and myself, working together in true startup fashion: 25 hour days, 8 days a week, 1 to 2 minor miracles per day. We’re learning how to bootstrap a business on a shoestring, design and build products together, and leap tall buildings in a single bound. We’ve spent a great deal of time analyzing, debating strategy, and thinking through minute design and implementation details. We’ve learned a lot, and built the beginnings of something we think you’ll love. It’s been thrilling, and terrifying, and the most fun we’ve ever had "working". We count ourselves very fortunate to be doing something we’re passionate about, day in and day out. We owe huge thanks to all of you who have supported us thus far.

Mukodu is also growing, and we’re looking forward to growing further. We’ve recently had a great boost with part-time help from Katy, Aaron, and Troy. We’re plugged in to a wonderful community and office space at the Centre for Social Innovation. And we’re starting to bring on some trusted advisors to kick the tires on our product and provide more guidance as we near public release.

While we aren’t in a position to hire people yet, we soon hope to be. We’ll need help: in particular, more software engineering and operations talent to build and run a kick-ass software service; advisors and board members to guide us; and angels to help prime the pump. We’re looking for these folks, starting now. We want people who share our values, passion, and hunger to change the world. We want people who are smart, scrappy, and get things done. We want people with creativity and a love for data analysis and problem-solving. We want people who understand that "going live on the web" is just the beginning; we must also have the humility to continually seek feedback and iteratively build something that actually works for customers and solves their needs. We want people who can handle the uncertainty and risk inherent in a startup environment, and whose instinctive response to a challenge is, "Can do!". If that sounds like you, or someone you know, let’s talk. 

We’re excited. We are getting great feedback. And with your help, we will make a real change.

Fire it up!

Osh and Katrin