Why Silos work

I think we need to have a conversation about Silos, specifically about why they work. There is another level of depth to the silo analogy that Doc Searls uses, which I present here.

Contemporary farming uses a process known as ensilage preserve crops for feed during the off season. The process allows for the slowing of the otherwise rapid decay of plant material by limiting the intrusion of oxygen, and controlling unfavorable reactions. It requires fixed infrastructure (silos) and a set of skilled workers to prevent unfavorable results.

The current version of the Internet relies on a similar process. Companies such as Google, Yahoo, Twitter, Flickr and others provide server farms to store content for distribution of users. They also provide the set of skilled workers to prevent unfavorable results.

At first glance, the analogy seems to be a fairly simple one... a place where things are stored, and kept out of the weather. However the analogy has a lot more depth than even Doc Searls might have imagined. The mechanism of preserving something and transforming it by adding value also works to give the metaphor more depth.

Taking something as ephemeral as a grass crop and storing it for 6 months is a remarkable achievement if you can do it on a sustained basis. The same can be said for taking the daily diaries of the general public, and keeping them online for years. It takes a persistent effort by a skilled set of workers in both cases to keep conditions optimal.

The farm worker is trying to prevent decomposition, control pests, and maintain sweet silage. The internet worker is trying to thwart hackers, spam, zombies, and any number of other pests, while working with fundamentally unreliable hardware and internet connectivity.

Flickr is a typical internet silo. The input is millions of photos (and now videos) from users throughout the world, along with some of their time and attention and a dash of identity. The value added is that of hosting the photos, transforming them automatically into thumbnails and a number of other sizes. Simply storing everyone's snapshots is nice, but there is far more value added than is immediately obvious.

When you link to a Flickr picture, everyone knows it's safe because it is really a photo, and not some malware waiting to trick your jpeg processing library in your browser. They also know it's not likely to be offensive, because of the filtering done on the images to conform with social standards. A further sense of safety is implied because the identity of the photographer is coupled to the photos, which also allows safe conversations back through the built in mail system.

Flickr works mostly because it was there first, has good network scale, and provides a great deal of safety. It's hard to replicate these things by accident, so it's important to have done your homework and looked deep into the real value proposition of the existing silos.

Twitter works because it makes IM safe and easy (when it works). A distributed twitter system has to replicate not only the message passing and filtering part of twitter, but the core value in terms of social value... the ability to ban users and moderate content.

There's a lot more to a good silo than servers and bandwidth. It takes a skilled team to keep the technology working, and a different skilled team to keep the social networking going as well.

It will be interesting to see what kinds of distributed silage systems get built in the next 10 years. I'm willing to help anyone who wants to start one.

--Mike--