My Own Pirate Radio

I’ve often wondered how well Google’s management structure works, particularly the high engineer-to-manager ratio. I’d always heard it was about 50:1, but this blog post I came across yesterday suggests ratios of 100 to 1 are not uncommon. Now that’s flat. Superflat. If you have the urge to peer into the murky depths of Google or Microsoft, give the post a read. (Yes, it might be a hoax, but personally I think it’s legit. Besides, hoaxes are people too.)

Management structure differences aside, Google really is a great deal like Microsoft was in its earlier days. It’s as if the same parents decided to have another baby 15 years after their first one. Same DNA.

As usual, the comments provide some of the most compelling nuggets. There is a bunch of good meaty stuff once you get past the initial chorus of “why we oughta fire that two-timin- free-breakfast-eatin’- NDA-violatin’- internal-email-leakin’- big-fat-secret-sharin’- son-of-a-!*@# employee!” comments from ’softies. This one particularly resonated with me:

…Many folks at both places seem to harbor a desire to start their own company ‘at some point’, and virtually no one at either place seems to be fully satisfied with the pace of their career growth, but the benefits and continuous train of internal opportunities keep most of those folks happy and entrepreneurially sedated.
…
Both are probably great places to work, especially if you can reconcile yourself to a nice, comfortable, interesting career, and you have the willpower to prioritize family over work and work email. For those who aspire to more, you’ll need to innoculate yourself against the sedating effect of the benefits and ‘industry influence’, get it [sic], build up some experience and a network, and get out.

Amen, brutha!

I subscribe to a discussion group called “TorCamp”, which is a small slice of software people in Toronto. There is an interesting discussion ongoing about Time Magazine’s “Small is Essential” article on 37Signals, the company that makes basecamp and backpack, and one of the chief promoters of Ruby on Rails. The article implies that small companies are The Way to Go, and that we no longer need to big build organizations in order to get big things done.

My take (reposted from the discussion thread):

There is a sort of “magic” about very small groups:
- you know everyone, what they’re doing, and how your work relates to everyone else’s
- very low communication overhead
- only one leader/manager required to handle tiebreaker decisions
- incompetent people and jerks have nowhere to hide :-)
- you stay focused on the core, because there is generally not enough money, time, or skill to take on on low priority objectives

I’ve found you can keep these qualities in a team up to about 60 people in size. But that was really 60 people subdivided into several smaller groups of, say, 5 to 12 people, each with a group leader, rather than one big 60-person team with a completely flat management structure. So at the core it was still a bunch of small groups, banded together into a bigger tribe. (Anomaly: Google apparently has a 50:1 engineer-to-manager ratio, and I’m curious as to how well that works.)

At some point — a 100 person tribe, let’s say — the team members don’t all know each other, or what everyone is doing, or how it all fits together. So you resort to more communication, more managers, more reliance on formal “deliverable agreements” and architecture and tools (like basecamp) guiding the collaboration.

There’s also a correlation with the nature of the product that’s impossible to disentangle. Small groups tend to take on smaller goals, and build smaller products. Big goals and big products require more brains, i.e. tribes. You can’t argue that taking a spaceship to the moon, for instance, could have been done by a lean, mean, 8-person version of NASA. The problem space was simply too big, no matter how you sliced it. So should we not have gone to the moon? Heck no! We need to do big things too. So small teams are not a cure-all. The world needs big tribes too.

That said, if you can chunk tribe-sized work into an architecture that allows a lot of small teams work on it in parallel, with a minimal number of interdependencies, do it. And I agree 100% about the earlier comment on passion: I’d rather work with 5 people who are passionate about what they’re doing than 10 who feel it’s just a job. Life is too short for the latter.

Related: The Dunbar Number as a Limit to Group Sizes

I have a large and growing collection of photos, videos, music, reference files, and project files on my home network. I’ve long passed the point where it all fits on any one laptop or desktop PC… it’s probably upwards of 400GB. And now with a home office I have a lot of work-related files that I really really can’t afford to lose. Frankly, all this data has become a gigantic headache. So I’ve recently invested some time organizing everything to be Just So. I thought I would share my nerdy little organizing scheme, in the hope that others might find it useful.

1) Organize your folders. All important files live in folders named as follows:

\Music = music currently in play
\MusicArchive = music archive (not currently in play)
\Photos = archive of photographs (subfolders 2004 ,2005, 2006, etc.)
\Projects = active projects and archived project files (subfolders for each project)
\Reference = miscellaneous folders, reference/archival use only

2) Synch “active” folders using Foldershare. All files within folders that I access and update on a regular basis are synchronized (mirrored) across all home computers using Microsoft Foldershare. This is done with a single Foldershare account, just to keep things simple. Foldershare automatically synchronizes each machine whenever it happens to be on the network, which is most of the time in our house… we have a wired and wireless network. Its pretty good at letting multiple people work independently on files without stepping on each others’ toes. I’m told Groove is also good at this, but I haven’t tried it.

3) Revision-Control Special Files. Our source code and production web site files need to be revision controlled, so that we can roll back to older versions if need be. For this we use a Subversion repository provided by our web host, HostingRails.com. Subversion is nicely integrated into Windows Explorer with TortoiseSVN (so you can manage any file that Explorer can see), and into Eclipse with the Subclipse plugin (so you can manage source code easily). There’s also a command line interface, of course. I’d recommend subversion to anybody working with source code or other versioned files. The server setup is a bit of a pain; I imagine there are hosts that do it all for you, though.

4) Get a Big Backup Box. As I mentioned above, this data doesn’t all fit on one box. So we purchased a 2TB Buffalo Terastation network-attached storage box, which several friends have recommended as a reliable backup and file server box. 2 TB! Sheesh.

The terastation has three jobs: (i) provide a comprehensive mirror/archive of all our files; (ii) act as standalone media server for music and photos, and (iii) act as a standalone print server (you can hook up printers to it directly through USB). So all the folders we care about, regardless of whether they are actively being updated or not, get synchronized regularly to the Terastation. I also map network drives to make it easy to access things… M: for Music, P: for Photos, W: for Work, and so on.

The Terastation is configured by default to use RAID-5, so with the 2 TB model you get about 1.4 TB of usable storage. If one of the four internal hard drives fails you can swap it out without losing any data.

Terastation comes bundled with Memeo Backup software, but from I’ve seen so far I don’t think I can use it as a file synch solution. Memeo doesn’t appear to let you configure any thing but the absolute basics, which doesn’t work for me. It also doesn’t support true mirroring of files, which is what we really need for most of our data. (”Rolling back” to a previous revision doesn’t really make sense for photos or music libraries… we need mirroring, not backup.) So my current solution is to run Robocopy periodically from one of the PCs, mirroring files to the Terastaton box. This can be done lazily, e.g. copy once an hour or once a day, so that you have a “safe window” of time during which you can fetch old copies of files from the NAS box in case you screw up. Or you can make it copy within one minute of any change.

In an ideal world Foldershare would run on the Terastation too, or handle network drives, so that we wouldn’t have to deal with another piece of software like Robocopy or Memeo. It looks like Foldershare does (or did) run on at least one NAS box out there, but it doesn’t on Terastation. Anyone know of a solution? I’d guess Microsoft wants to make money from licensing Foldershare for use on file servers, so I’ll be surprised if it’s easy to make this work.

5) Playback. I plan to hook up a music streaming device to the Terastation. This should be super easy… the Terastation exposes SMB file shares, which devices like Sonos, Audiotron etc. all work with. And I’d love to have a digital photo frame that pulls in photos from the Photos folder on the Terastation, over ethernet or wireless. This could sit in the living room or kitchen or wherever. Anyone know of one?

6) Bonus Points. File encryption and search on the NAS box. I’ll post again if/when I get that figured out.

Notes:
Cost: Two things in this solution cost money: the Terastation ($800+ USD, $1100+ CAD) and my hosting account at HostingRails.com (~ $100 USD/yr). Everything else is free.

Keep it Simple (and cheap): Don’t bother with network attached storage devices like the Terastation if you don’t need to. Just use Foldershare to mirror files (or particular important folders) between your computers.

Archival versus Active Use: Some folders, like MusicArchive and photos from earlier years, will end up living on the Terastation only for archival/reference purposes. This makes sense because they are rarely used, and anyway would not fit in entirety on any one of our PCs. We can move files into one of the “active” folders by simply moving or copying, e.g. moving a music album from MusicArchive\ to Music\. Or we can just reference them directly on the Terastation box, using it like a big remote hard drive.

Compatibility: in theory this setup should work nicely with both PCs and Macs, since Foldershare and the Terastation work well with Macs.

Foldershare quirks: Foldershare does have a limit on filename length, so you can’t have a really deep directory structure. The free version of Foldershare is also limited to 10,000 files per shared library. I get around this limit by keeping folders small, and only using Foldershare to synch folders that are being actively updated. For instance, in January 2007 I remove my Photos\2006\ folder from Foldershare’s control, since I no longer have any need to add files to that folder, and I add Photos\2007\ to Foldershare so that new photos stay up-to-date during the year.

Versioning everything: I recently came across a post by Joey Hess, who uses Subversion to version ALL of his files and keep them in synch across multiple machines. This is, I think, a little extreme. And scary… personally I wouldn’t really want a copy of every file I’ve ever created to exist. But it’s pretty damned cool too, so if you want to turn the volume up to level 11, check out Joey’s solution.