Spring cleaning

It's spring cleaning time... I've trimmed up the tags used on this blog, and I'm planning to get everything in the archives into proper categories. This should make it easier to discover stuff one is interested in reading.

I also plan to put a blogroll back in place, with a small twist... I'll provide a post about each entry, why I link to them, and the value they provide. I'll add a link to that post as well in the blogroll.

Indeterminant Intermediaries Imminent

Doc Searls recently wrote a blog post - The Revolution Will Not Be Intermediated in which he states some hopeful things:

We still seem to think that progress on the Net is the work of “brands” creating and disrupting and doing other cool stuff. Those may help, but what matters most is what each of us does better than anybody or anything else. The term “content” insults the nature of that work. And of its sources.

The revolution that matters — the one that will not be intermediated — is the one that puts each of us in the driver’s seat, rather than in the back of the bus. Or on a bus at all.

As much as I like Doc, and wish he were right, I can't help but be cynical for a number of reasons. I'm hoping to be proven wrong, and/or convinced that there is room for hope. Please find some flaws with my logic...

The future of the live web is in doubt, for good reasons.

The long tail - theory vs practice

The Long tail theory holds that everyone should have a blog, and they will have a voice. This is widely interpreted to mean that the tools are sufficient to get your views into public. This theory is valid, but as with any description of reality, it fails at the edges, and a better model of understanding should eventually replace it.

The blogging tools now have succeeded in making it possible for anyone to write an opinion, and have it accessible in an instant from any Internet connected PC in the world. There are limitations when Governments, or ISPS get in the way, but for the most case those limits are exceptions _post _facto__ and do not impose prior constraints.

However, the freedom of speech enshrined in our US Constitution is worthless without the freedom of assembly, which is the freedom to hear someone else speech, and to have conversation with them. It is here that the intermediation sets in.

Historically, the cost and logistics publishing provided a natural damper on the quantity of material, and people generally focused on getting quality of content up to the point where it seemed acceptable to expend the effort to publish and distribute it. The nature of the Internet and the web has done away with this limitation, and made a more basic limitation which has always been there, apparent to all... the limits of human attention.

Now that we all have fast and effectively zero incremental cost to say what we want, the problem now becomes connecting with an audience. Out here in the long tail, where this blog will reside for the foreseeable future, the loyal audience is very small, mostly family and a few friends. In this blogs history, the rare moments in the spotlight are for things unrelated to the normal subject matter which I tend to be interested in, for example... this post "Songs about teamwork" is the source of more than 50% of all hits here... ever.

This means that while the tools make it possible to speak, there is very low probability of being heard when you take the time and effort to set up a blog, especially if your areas of interest are many and varied. An intelligent response is to consider alternate ways to route around the attention problem, and to write where the readers are.... thus putting content on social networking sites, into emails and other non-live-web related channels, and into comments, twitter, and other aggregated sources of attention.

The long tail in practice works out far from theory, we now all have a voice, but we have to find an audience... everyone has a soapbox, but the public square is full of people with ipods, tuned into their own narrow circle of interest. The key is to find a topic already in progress, and to attempt to join into the conversation. We all have our own bus (or car), now we need passengers who are going to our destination.

I'm a commuter, I regularly share my trip with a circle of friends on the South Shore railroad daily to Chicago. We all give up a bit of our privacy to have a shared experience that is more cost effective and efficient for each of is... this is the same logic behind using social media sites, twitter, FaceBook, etc.

If the train fails to arrive, gets delayed, etc... we route around it, share rides, etc... just like when twitter fails, etc. The metaphor can be extended quite a way if you like... it almost writes itself.

It's a silo, sure.. but that is easy to ignore if it stays out of view, and your friends are there. It's only when you have a dispute or disagreement with management that the situation becomes unacceptable.

With this in mind, it's not hard to disagree when Charles Arthur wrote in the Guardian that The long tail of blogging is dying. I'm spending upwards of 4 hours working on this post, if not more... and I might have 30 people read it, or maybe a few thousand if I'm VERY lucky and it goes viral. The residual effects will be very, VERY limited, in my estimation. It's only faith and the need to speak my piece which keep me going.

With all of this rambling in place, let's go back to the quotes from Doc again...

We still seem to think that progress on the Net is the work of “brands” creating and disrupting and doing other cool stuff. Those may help, but what matters most is what each of us does better than anybody or anything else. The term “content” insults the nature of that work. And of its sources.

I think it's better to say that we're used to having reliable labels on our content, enforced by law (Branding, Trade Mark, Copy Right). These labels made it possible to aggregate many different pieces of stuff under one reputation umbrella.

The main challenge in the live web is to figure out how to aggregate enough reputation to be easily quantified by anonymous third parties for purposes of deciding where to spend their limited resources of time, attention, energy, and possibly even money.

Google tries to use PageRank as a proxy for quality, on a per site basis... which isn't a fine enough grain to measure quality on each piece that someone like Demand Media might produce, so the quality is going to be spotty. The idea of tying PageRank to a person is interesting, but that really won't work out in the long run, because we all have different areas of expertise. For example, you might decide to trust my opinions on computers and security, but you probably disagree with my opinions on political matters. These areas are orthogonal to each other, so there need to be many degrees of freedom in ranking and labeling content.

Google is slipping, but all engines slip from time to time. They can only keep up a certain pace, as their infrastructure gets more complex and entrenched in a certain paradigm. They have made excellent use of the resources they have available, especially the 20% time they give to feed (and harvest) what would otherwise be the seeds of their destruction.

I'm a geek, and I tend to think in terms of technical solutions to problems... I see this all as a matter of lack of metadata and the infrastructure to support it. I think over time it'll all pull together, it's just a question of timeframes. It took 500 years for print to get to where it was when the Internet took off... it may take a few lifetimes for the Internet to get figure out to the same extent.

I'd of course like to skip some of the learning curve and get more benefits now, while I can appreciate them, and pass them along to my daughter.

Learning more about tools

With the latest addition to the personal computing empire, and the consequent lost of context that having an empty slate entails, I've decided it's time to learn to use Delicious and/or sync all of my bookmarks.

I've added a tag cloud off to the right of this blog, which should get more useful as I plow through my vast hoard of bookmarks with an eye towards present and future value, and sharing.

Tags - an oops

I started to get rid of tags from entries on my blog because I thought a shorter tag list would be more useful, but I've figured out that is not the case. There is an expressive force in tags that is lost when you cut out the free association aspects of it. You need to be able to make those serendipitious connections, this is where ideas come from.

I'm going to review all of my previous posts, and put tags as appropriate on them, as time permits. This will allow eaiser discovery of the good stuff, and make the barriers to it lower for someone who happens in.

Small improvements in the right places

I'm cleaning up the tags on this blog, getting rid of the most single instance items which are noise. I'm also clearly tagging all of the rants, just because I think I need to. These small changes should at a bit of value to this blog.

Blogging forces some unique constraints upon the author, these are mostly societal and expectations, not technical. The blogger is expected to write a short essay (french for "to try") on a regular basis. This stream of text is also expected to stay on a single topic, the best to feed the point and click sensibilities of the larged audience possible.

No more!

Blogging is broken. Here's why.

The short form doesn't allow for the necessary amount of exposition necessary to lay the groundwork for common understanding that is necessary to overcome the limitations of text and the vagaries of human language. There are always differences between the intended meanings of the author, and those inferred upon reading. It's necessary to supply more than one set of explanation to help the reader decide which of their multiple guesses at the intent of the author is closest. 

Conversation involves listening to the other partner, and giving feedback on the concepts discussed, to make sure your minds are as close to synchronized as possible. This helps then ensure clear communication of the intented subjects. The better the communication, the further the reach from the everyday is possible.

In order to be able to discuss deep subjects, specific vocabularies must be developed... specific metaphors, and examples. This is the reason doctors have their own medical speak, and programmers know what loops are.

The short form doesn't allow this exposition at length. It forces an artificial division of text... which reduces the amount of signal, and increases the proportion of text used to frame each piece... thus it's not best for very deep subjects.

The limits of what you can express and explain are the very limits of your ability to change the world. If nobody was able to explain the chain reaction, the atom bomb would have never made it out of the theory of one physicic and into reality.

Language is power.

Blogging can be tweeked, ever so slightly, to be more powerful. The limitations are not technical, they are societal, so we don't even need new tools... just new expectations.  Fortunately, expectations can be adjusted as a matter of intent.

Blogging tools allow keywords on pieces of exposition, to link things together. Blogging tools allow the linking of a longer explaination seemlessly into your text, to help shorted the text for the reader already in sync, while providing additional support for the reader who is not familiar with the topic, or has additional doubt about the intent of the author.

The need to stay on a single topic is an artifact of the pre-tagging days of blogging, and is hereby declared obsolete!  (HA)

Actually, if you give up the idea of having a stream of regular readers, and rather chance meetings of other minds, your point of view is shifted... and your emphasis should then be on allowing the best (easiest) discovery of the content you already have. Linking back to previous work related is one of the most powerful tools to accomplish this. The calendar based archive is appropriate for a single-topic blog, but the entire creative output of a person will necessitate new and better approaches.

----

Suggestions for improving blogging in general.

Don't waste time and emotional energy limiting yourself to a single topic, issue, etc...  go ahead and enjoy the freedom of free association, but please be sure to tag things, and nuture your own links, and links to others.

Revisit your older work, update, revise, correct it... much as if you were to produce a new edition of a book... if you expect people to read it, you should be willing to take the effort to fix mistakes that you've found, and to clarify it.   Mark your changes to help avoid confusion if they are significant.

Give up the idea of having to do something every day... just spend time on it when you like.  If you need to peek at your Google analytics, go ahead... but it's like dieting, there are ups and downs, and you have to keep focused on the big picture and not the numbers immediately in front of you.

Tag everything, prune the tags (get rid of tags only used once that you doubt you'll use again) to make your index smaller.

Ask others for feedback... give others feedback.  It's lonely here out on the small end of the long tail. Your time and attention is the most valuable thing you have to give. A little bit goes a long way.

----

In summary... blogging daily on a single topic is broken... blogging at length with good links back to relevant topics and a good index is better.

It's the same tool set, you just use it a bit better. You extend the reach of topics you can discusss, and thus extend the extent to which you can change the world.

Which is what we all want in the end... to leave it better than we found it.

Right?

--Mike--

Somewhere between twitter, rss, and tag clouds lies a better path forward

Dave Winer correctly points out some serious flaws in twitter, and while he might not get brownie points or whuffie for it, he's right. He also shows where he thinks things could go. It's all about the metadata, and this subtle point seems to fly right past most people who read him.

I think it's time to explode twitter into its components making each of them public, implementable as a service, or as a hosted app, or something run on the end user's hardware.

The success of Twitter is because it allows for the rapid spread of messages from a controlled user base... there is a central control that can (but doesn't always) ban a user, etc. This means that every message is authenticated a bit... and all messages are tied to an identity. This makes filtering possible.

There is NOT any really good rating metadata. The messages are too short. This sets expectations, but really does cripple it for important stuff.

Blogging is seen as too slow, but RSS is a slow version of twitter. I think that metadata richness is the fix to this whole thing. Trade a bit of speed (twitter is too fast anyway) for expressive power.

We need to be able to aggregate our own stuff, which is one of the strengths of RSS. The ability to follow (and unfollow) an authenticated message channel is a great plus.

Collective ratings are ok, but the scales of services such as digg (up or down), and slashdot (funny is the same as insightful), are very limiting. It would be more useful for longer term (slower) conversations to be able to add more expressive metadata. Funny, Insightful, Biased, SelfPromoting, etc... could all be but a few of the plethora of possible bits of critical review that could be added.

It would be nice to be able to actually markup hypertext... but that still appears to be outside the range of feasibility.

Maybe it could be called TLM - Text Language Markup?

Signal and noise - Separation of concerns

I keep seeing the same meta-problems over and over again. Some new tool comes along to make it easier to communicate, and it takes off. Then people complain that it makes it too easy to communicate... and it loses steam.

We've been through this cycle so many times, and history always rhymes. This particular portion of the rhyme of time involves Doc Searls, Twitter and Facebook, because he mentioned all of them in one place.

History - Email

Email made asynchronous messaging possible for most of us. It became a de-facto component of any "internet access" package. There were some improvements along the way, but the basic protocol hasn't really changed since the switch from ! to @ email addressing.

Because the environment has changed, the lack of authentication of the sender of a message has turned it into the spam pit we all learn to accept as the status quo. If you take spam out of the picture, email is a great way to get data from sender to reciever. It's a deliberate 1:1 transmission means, which means the sender implicitly signals the intent for someone to read the message.

Mailing lists preserve the intent because the are opt-in and dedicated to a certain topic. Social conventions arose to keep the clutter down, and keep things on-message.

History - Usenet

Usenet is a distributed protocol for hosting discussions on an internet-wide basis. Before the spam got out of control, it was a useful way to get information on a wide variety of topics. Once again, the nature of the environment changed, and the lack of authentication made it into an even bigger spam pit than email is today.

History - IRC / Instant Messaging

IRC - Internet Relay Chat made it possible for real-time group discussions across the internet. The spam factor drove some innovation amoung closed systems, which requried some form of authentication, spawning a wide range in so called Instant Message (IM) offerings amoung the various walled garden services such as AOL and Prodigy.

The VHS/Betamax style battles to capture customers lead to a lack of a single service arising to unify things, though with clients like Trillian it was possible. The need to install a specialized client was a big hindrance to adoption as well.

History - Twitter

While there have been other web-based messaging platforms in the past, the rise of twitter as a way to broadcast messages has been quite solid, in spite of some growing pains. The ability to see all public "tweets" and the additional innovation of third party tools such as search has kept it growing.

The fact that there is a central authority to register users and enforce some rules does help keep the signal to noise ratio in check. Like instant messaging, you chose who you listen to, but in addition your audience choose you as well.

The problem faced by twitter is one of the loss of Intent. When you send a message to twitter, it goes out to the entire audience, there is no way to segment your audience by intent, which forces you to artificially limit yourself to the one common interest of your audience if you don't want to lose them. Otherwise you have to play a very delicate (and unneccessary from my point of view) balancing act to try to keep everyone happy.

History - Facebook

Facebook gives us an easy to use place for putting sharing our stuff with others. Because of the lack of intent, there is becoming a friction between work and home, and people really like the tool because of ease of use, so they don't want to lose it to the lessor choice. The tension of multiple audiences is becoming easier to see with the rise in user population, and the increasing adoption of social networking platforms by businesses who have gotten a clue.

History - Internet Access and the soda straw

I have at least 320 Gigabytes of stuff I'd like to share. I'm sure that anyone who has had a digital means of capture of images or sound could easily generate a gigabyte or two per month without breathing hard. The entrenched providers have built their networks on an assymetric model that prohibits running servers, which is the only reasonable strategy for making this much content available on a discretionary basis.

Prohibition of servers at home actually then forces us to chose carefully the content we wish to share, and to send it to a silo where we begin to lose control and rights over our own stuff. We're forced to live our online life breathing digitally through a soda straw.

The Present

There are many tools to allow you to share content, but there is something lacking in each of them. As we see the ones that allow specifying audience don't seem to have the necessary authentication or regulation of the senders. The ones that work well for social networking don't allow for the separation of concerns. There is an additional factor of the difficulties of moving content to, from, and between these platforms and silos. The fact that this all has to be done through a limited bandwidth connection certainly doesn't help much.

Defining The Future - Social Aspects

If we are to have the future we desire, we must articulate the vision clearly and build consensus to push towards it as a goal. There are no obvious technical limitations that prohibit a future where we can share all the content we care to create. We do need to take care to make sure the signal to noise ratio remains as high as possible for those we share with. I believe that the simple act of being polite and mindful of our audience is going to carry the highest social value in the future, as it currently does with our limited tools.

Defining The Future - Technology

The future is all about metadata. We don't have enough of it right now. Flickr helps by allowing tags, and this is a very powerful tool. We need to build tool sets that allow tagging to become a socially shared tool as well. It would be valuable to allow someone else to review my photos, and decide which are keepers, or to tag them for specific keywords, possible venues, etc.

HTML needs to be expanded to allow for the markup and tagging of existing content by other parties. Of course, there needs to be some authetication built into it, to prevent grafitti and other forms of spamming from taking over. This would allow someone to highlight the "geek speak" parts of this very blog post, for example. It would also allow someone to highlight the part they found insightful, or insulting, or whatever.

Offline forms of sharing should be sought out to allow familes and others to route around the poorly designed "broadband" we've all been brought up to think was fast.

Twitter, Facebook, and others should allow for some form of tagging, like flickr, to make it possible to subscribe to only certain content from other users.

In Summary

The limitations of Twitter and Facebook that Doc Searls complains about are those of metadata and intent. They are not unsolvable, and they can be addressed. It remains to be seen if that happens via inclusion of new features, or migration to even newer networks that offer the required features.

Old posts - adding value to the archives

I'm wondering what the rest of the world things about old posts... so I came up with this Google query to find out

http://www.google.com/search?q="old+posts+are+like"

What I found was a total of 7 hits... which isn't much... so I must have worded it wrong... but the hits I did get seem to indicate that the main reason for leaving them around is to show up more often in search engines... which could very loosely be considered a proxy for value.

I'm interested in makeing this pile of posts into something other than more wasted disk space... that's why I'm adding labels (there's a list at the bottom of this blog) and trying to revisit things and add value.

Archives are important... and need to be nurtured. You don't prune them back, but you can tend them.

Why I blog, and what to do afterwards.

Andrew Sullivan has written a great piece over in the Atlantic about why he blogs. He goes through the history of blogging, and weighs it against other forms of writing. I agree with it in the most part, but I disagree with his characterization of blogging at the start: (italics are mine)

This form of instant and global self-publishing, made possible by technology widely available only for the past decade or so, allows for no retroactive editing (apart from fixing minor typos or small glitches) and removes from the act of writing any considered or lengthy review. It is the spontaneous expression of instant thought—impermanent beyond even the ephemera of daily journalism. It is accountable in immediate and unavoidable ways to readers and other bloggers, and linked via hypertext to continuously multiplying references and sources. Unlike any single piece of print journalism, its borders are extremely porous and its truth inherently transitory. The consequences of this for the act of writing are still sinking in.

I agree that social convention around blogging is to not go back and do heavy editing of what you've written in the past. This is a crucial cornerstone on which we build our arguments with each other and ourselves over time. If the past is allowed to be edited, then anything can be forced to be true with sufficient effort. So there is much value in keeping the archives safe.

However, there is also something to be said for adding to the archives, which I don't think is currently done on anywhere near the scale that it could be done. It might be useful for me at some future point to add references that point back to this article, add corrections, etc. I think there is quite a bit of value to be added in this way.

I personally flit around a bit too much for even my own sensibilities... tending towards a tangential life at times, but I do manage to get back to the basics, and get things done sufficiently well to allow society to consider me a valued member. (I hope)

I'll try to go back, see what value I can add, and make this blog a bit less ephemeral, and a bit better value for everyone. There are lots of good and bad arguments that have been made, decisions informed, and lessons learned. It's a shame to lose the value in them because they simply are too hard to find.

I believe we need to put a bit more effort into this, collectively as well. We need to slow down our pace, be a bit more considered, and we'll all be better for it. This requires no new tools, just a bit of a tweak to the social contract we bloggers share.  We can all make our existing work more valuable, and a proper gift to our children, instead of just a random pile of rants.

Thanks for your time and attention.

Blogging at the end of the long tail

If you're reading this, thank you for your time and attention.

Doc Searls is one of the bloggers I read on a daily basis. He's consistently promoted blogging as a way for us to express ourselves and help each other out. I recently posted comments about the nature of blogging out here in what's known as the "long tail"... so described because if you graph the amount of readers of traffic amongst all blogs, I'd be out in the long tail with most other blogs, having only a few readers.

Here's the basic dynamic, based on an illustration I grabbed from WikiPedia:



The environment that Doc is used to is radically different than the one the rest of us live in. Because he's got a stream of followers, he gets constant feedback on how he's doing. Out here in the tail, comments are a rare occurrence. The torrent of attention becomes a trickle out here. Thus the dynamics that work for him don't play out in the long tail.

It's more likely that someone will leave a comment for Doc, because more people read his work. It's also true that the person leaving the comment is more likely to get other people to read their comment as well, because comments are usually public. Private comments like emails don't enter this picture, but I strongly suspect they have correlation with popularity as well.

The positive feedback loop that helps push up the top bloggers works for the other end as well, the top get pushed higher, and the bottom gets pushed lower. Someone on the long tail might have a few interested followers, but they will likely not bother to go through the hassle of signing up to put a single comment on a web site. An email, or offline comment is the more likely route.

Then there is the male culture factor...

As men, a blogger starts with a disadvantage. Guys like to solve problems, we’re taught not to comment on things unless we can solve a problem, or have our 2 cents to throw in to a discussion. This is why we make crappy bloggers, we’re not good at the blog relationship thing, because we don't give lots of feedback.

We’re also impatient… it takes YEARS to find an audience, we’re used to getting new skills by working hard, the harder we work, the faster we get better…. blogging isn’t like that.

Last but not least, there's the problem of having a wide field of interests...

When you’ve got no traffic, it also doesn’t make sense to put things in separate blogs… so the audience you do have gets a lot of stuff they don’t care about… which discourages them as well. In my own case I’ve realized this and am in the process of separating out my areas of interest into different blogs. Most of them get NO hits on a given day… and one or two every once in a while thanks to random web searches. Is it really worth it?

Summary...

Do we have a voice in this bold new world or not? From out here on the long tail, it’s VERY hard to tell.

I think the #1 thing we can all do is to make it a point to at least leave 1 comment per day on someone’s person blog. Like complements, they only have value if they say something positive, and are true.

In other words, The love we share, is the love we receive.

What comes after blogs?

I post here on a bunch of different themes related to my many roles in life... Person, Husband, Father, IT Guy, Scientist, Photographer, Citizen, etc... and they all end up here. So how do I keep posting, yet not bore everyone with the stuff they don't care about?

Doc Searls addresses it on the reader side by using Google to find blogs, instead of using a set of favorites like I do.

How do you find things to read? Am I typical, with my set of blogs, or not?

What comes next?

Tuning loops, redux

Doc says that blogging is provisional. I take it that this is because it's part of a conversation. I can understand how he's saddened by the loss of the one good paper in his town, and the overall trend of the demise of local news. He's searching hard for technology, the net, or even smart mobs to replace it... he's searching in vain. The way we consume "news", is, in fact, the problem.

Cool hunting continous partial attention just isn't going to replace local news, no matter how you aggregate it. I'm never going to replace the local reporter with a beat to cover. Using technorati, digg, metafilter, or friends who forward me amusing links isn't going to do it either.

No matter how you slice or dice it... local news requires someone dedicated to the local beat. A real, live human being, who can always find other, more amusing things to do. So, we need to encourage all of the good local reporting we find. We also need to actively seek it out, and patronize it.

It turns out, the way this works in the blog-o-sphere is one already overlooked as old news.... hyperlinks and blogrolls. The fact is that what we pay attention to determines what we find. If we take the time to keep our reading lists reasonable, and up to date, we'll find a lot of value that we'd otherwise miss trolling digg for the next new thing.

I started this post last night, and as typical for Doc... he somehow does a zen reply to a post before I've even written it... today he writes about finding value via Ronni Bennet's favorites list... and behold, the circle is complete.

The act of blogging, and keeping your blogroll and web site up to date are creative. Sucking down RSS feeds and cool hunting are consumptive. It's better to create.

Doc, the replacement for local newspapers is blogs and blogrolls... the tools still need work, and we still need to find social incentives to appreciate quality, and reward effort... but you've already found the answer... you just don't know it yet.

--Mike--

Blogging Meta - revised

There is a subtle yet profound difference between blogs and previous methods of communicating on the net. Blogs are anchored to a location, which requires ownership. This difference is the major factor which lead to it's current popularity. Unlike email, you can't forge entries from a blog. This allows trusted relationships.

When you read a blog for the first time, you're deciding if you like the author or not. This uses our natural instincts for deciding who we want to build relationships with. It grows over time. You'll develop a set of favorites that you read over time.

When someone makes their list of favorites available, it's a blogroll. When you put someone on your blogroll, it's more than just a person reference, it becomes an endorsement. This means that people think twice before adding someone to a blogroll... which helps keep the quality up.

Blogrolls and easier tools made blogging popular. This lead to the current rush of traffic. All because of the ownership and identity and relationships inherent in the simple fact of a URL anchor.

The tools make it dead easy to link to other blogs, and build the live web. This means that the sheer amount of stuff to discover will probably continue to grow for quite some time. However, this presents the problem of data overload.

The solution that Doc Searls uses is to "aggressively subscribe to RSS searches". I see a danger in that because it breaks the mold of relationships that got us here.

When we were strictly manually building blogrolls, any spam blogs (splogs) that arose would quickly be discounted, and just not added to the growing web of links. This is the only form of spam defense which isn't likely to be overwhelmed in the near future. It's pretty much 100% effective.

When you rely on a search engine to find things for you, you're making a tradeoff. You're taking input from a far larger range of sources, but you loose the inherent filtering present from relationships. This is where the opening comes for spam blogs.

We can either shun search engines, or possibly find ways to make the relationships more explicit so then can be machine readable. If the search engine could factor in your web of trusted relationships, it could then help you manage a much larger group of "friends", or more appropriately non-spammmers.

It remains to be seen if this is a good idea or not. I've revised this since it's original post, I hope this is a lot clearer.

--Mike--

The Blogging Meta

This post rambles wayyy too much, as Dave points out in the comments. Skip it, and read the next one instead. --Mike--


The history of the internet is rich in details and many examples of technologies that have come and gone. The overall pattern is one of new systems solving the problems of older technologies, then winning popularity. Sometimes the advantages of the new system are readily apparent, and at other times the differences weren't so easy to spot.

I'm writing this in an attempt to highlight what I think are the reasons for the popularity of blogging up to this point, and to help provide a wider foundation for discussion about future directions it might take.

Blogging is a relatively new form of social networking to appear on the internet. The owner of a blog makes a series of posts about topics they find interesting, on a regular basis. The entries are usually titled and dated, and presented in reverse chronological order. The entries are usually in the form of commentary on other blogs, web content, news of the day, person observations, or centered on some theme.

There is a subtle yet profound difference between blogs and previous methods of communicating on the net. Blogs are anchored to a location, which requires ownership. This difference is the major factor which lead to it's current popularity.

Now... you might just think I'm full of shit, but consider this:

When you read a blog for the first time, you're deciding if you like the author or not. This is a personal relationship based on ownership and content of the blog. It grows over time. You'll develop a set of favorites that you read over time.

When someone makes their list of favorites available, it's a blogroll. When you put someone on your blogroll, it's more than just a person reference, it becomes an endorsement. This means that people think twice before adding someone to a blogroll... which helps keep the quality up.

Blogrolls and easier tools made blogging popular. This lead to the current rush of traffic. All because of the ownership and identity and relationships inherent in the simple fact of a URL anchor.

The tools make it dead easy to link to other blogs, and build the live web. This means that the sheer amount of stuff to discover will probably continue to grow for quite some time.

When we were strictly manually building blogrolls, any spam blogs (splogs) that arose would quickly be discounted, and just not added to the growing web of links. This is the only form of spam defense which isn't likely to be overwhelmed in the near future. It's pretty much 100% effective.

Now that the search engine tools are arriving, we're starting to hear about splogs. This is due to the fact that the seach engines lack the reputation metadata that is inherent in blogrolls and the personal discernment of content quality. We need to find a way to supply this metadata, or create a suitable replacement.

That is the challenge we face, to build search engines that can use reputation to moderate out spam before it gets a chance to grow. It has to be based on relationships, and identity. If there's any way that it can get gamed, it will, so we need to be very careful.

At this point, I bow to the collective wisdom of you, gentle reader, and the rest of the community. I'm open for any and all suggestings that don't spam me. 8)

I thank you for your time, and I hope you found some new thoughts to consider.

--Mike--

Thinking about gatekeepers, spam, and metadata

This is a long one... please forgive the length... I'm thinking it out loud as I go.

Doc Searls isn't a gatekeeper... but he does a great service for me, he helps filter out the massive volumes of stuff that is Web 2.0 (or blog-o-sphere... I don't have a good term for it).

I count on Doc to share interesting things he's found related to Web 2.0, Blogging, RSS, Podcasting, and the societal impact thereof. If you want to know how to build a new hierarchy of your own, Doc's the man.

Doc recently pointed out that he subscribes to several keyword searches to keep abreast of the topics he's interested in. In this case, he doesn't use any human gatekeepers, but is relying on the technology as a tool. This allows Doc to get input from a much larger span of sources than any other human could deal with. When you operate this way, you too can get a much broader range of sources, and be much better connected.

Unfortunately there is a big downside to this.... SPAM.

I tried out Doc's approach, because I'm interested in Computer Security. I'm interested in a very special aspect of it though, I'm very pragmatic in that I want to be able to run untrusted code... which makes finding the right keywords almost impossible. Once you do find the right keywords, guess what happens.... you hit the wall-o-spam again....

For example, the search for keykos security at technorati returns 2 results in the past 33 days. The first post is spam, and there's my post right below it. This is a very narrow search, yet spam comes up first. Unless of course I'm wrong and a new blog with no real content named "search engine submission" really isn't spam... ;-)

Points so far:

• We need other people's help to filter Web 2.0 down to size
• Spam is a persistent problem
• Keywords only go so far
• Tagging helps

The tools and social constructs are getting better, which is encouraging. What I think we need now is better tools. We need better ways of sharing metadata.

For example, this very document before your eyes could be generating community metadata as you read it. How much of a page you read, how much time you spent reading it, and when, are all useful bits of knowledge if you have efficient ways to store and index it. I know that I certainly would like to know which parts of this blog are interesting, so that I can improve my writing skills to help get my point across effectively.

I'd like to know exactly what I've read. I'd like to be able to pull that data up, and add some metadata and comments, and rate it. This would be useful for my own reference, and possibly for others as well.

Annotation + Rating
A lot of what Doc Searls does manually by blogging could be interpreted as 3rd party annotation. He points us towards things he finds interesting, along with clues about his opinion of it (agree/disagree/funny/insightful, etc). I believe it would be very useful for the rest of us to do the same in a more automated, tool leveraged way.

The broken promise of Web 1.0 is that HTML doesn't actually allow Markup of Hypertext. We've had to evolve blogs, and commenting about posts, and technorati, and all sorts of other infrastructure just to get to the point where we kinda make up for a basic design error.

In my ideal world, you'd be able to pull up a page, highlight a section of it, and add notes, ratings, links to supporting material or ideas. The correct term for this is Markup, but that's been diluted by the term HTML as to become useless as a search term, so we're forced to use Annotation instead.

Just imagine if you could right-click on any web page and then click "flag as spam". The backside of this would be that you could then know before opening a link, what was spam. It would get really powerful if you could then have this data factored in to your search engine results before you see them.

Just imagine if you could right-click on the same page, and add technorati tags to it... including SPAM. ;-)

We need to be able to do this... the last search engine to do it is a wrotten egg!

Well.. that's a lot of wandering around the topic... more of a brain dump actually....

Thanks for sticking with it, hope you found it interesting.

--Mike--

Feeding the long tail

Today's bit of wisdom comes from Scott Karp at "Publishing 2.0":

Consumer-created media takes a lot of time and energy — unless we develop economic models to meaningfully compensate the long tail, the ego payoff for most people won’t be enough to justify the effort. The cost of entry to create content is low in terms of dollars, but the cost of sustainable content creation is very high in terms of time, which in this short life is our most valuable commodity.

Amen! It doesn't take much at all, a link here or there, a mention or two, and we in the long tail stay happy, and keep plugging away at it. If you want to think of it as building a market, or building a community, or better yet, building a society, please do so.

I'm of the solid opinion that the larger the blog-o-sphere, the better we all are. It helps us face down one of the three biggest issues facing Democracy, which is a good thing.

--Mike--

Expressive Linking

In my previous gut reaction post, I noticed that we need a richer set of expressions that can be encoded along with a link.

For example, if I wish to have a blogroll, I might want to include ratings of one type or another as part of the link. Here are some examples that would be inside the A HREF= link

• agreement="100%"
• agreement="0%"
• trust="10%"
• nofollow agreement="0%"
• flamebait="100%"
• humor="100%"

Or something like this. Obviously those are somewhat silly suggestions. I'm trying to point out that since HTML is a consensus, and any new elements within a link can be added transparently without killing browsers or breaking anything in a bad way, we should experiment.

I've looked at VoteLinks, and they seem to be fairly limited in their scope, for example.

Let's talk about some of the flags that might get used to do more than tag a link, but also help to place its credibility, trust, likelyhood of being spam, authority (cringe), etc. The more data we can include with a link, the more useful they can be.

Of course, nothing is as good as honest markup (aka annotation)... but we'll probably never have that because of spam issues.

It wouldn't be too hard to get this stuff included in new releases of blogging and html editors if it took off. We can make it human editable fairly easy as well.

I've had this bug up my butt for years... as I said 2 years ago in a comment on Aaron's blog:

How about slashdot as model for discovering the truth? It only takes one meta-tweak to make it so. Allow voting on a number of dimensions, including Humor, Truth, Accuracy, Flame/Constructive, and anything else someone dreams up (including CowboyNealNess), on a -5 to 5 scale.

Or… get a real MARKUP language, and allow people to markup existing debates with their own layers of commentary and gloss on top of it, along with TML (Truth Markup Language), and things of that ilk. The same text could host thousands of discussions all selectively visible.

Even more radical would be the possibility of selectively filterng your worldview to exclude things you don’t believe, automatically, or actively finding things which you wish to ponder because they are near the edge of your beliefs.

Food for thought, and metathought.

What does everyone else think?

--Mike--

What blogging means to me

Dave asks how blogging changes beliefs.

My own blogging I've learned a lot of things because I wanted to join in the conversation. Reading Doc, one gets a sense that climbing the tail is the only way to go... so I signed on to that mission.

One of the things that rubbed off on me from my day job is that one should have clear goals, and measureable results. My goals for this blog at this point are:

• find my voice - a vague 60's sounding thing reminiscent of "finding myself"
• express myself - vague as well
• improve the human condition - nebulous and vague as well
• measure my rank - this is ego, pure and simple
  
• not waste my time doing the above - well... that will always be up for discussion

So, I have no rock solid goals. I defintely do not expect to make a dime off of this. I charge $100/hour to fix things for people if compelled to do so, and that's dirt cheap... blogging is definitely not a money motivated thing for me.

I share Dave's dislike of bullshit, marketing or otherwise. Nobody likes being manipulated, and I'm especially sensitive to it, or I like to think that I am. Update: Noran tells me I'm a piece of moldable clay... ;-)

I do like to experiment... and this blog is one huge experiment, just to see what the heck happens. I've learned a lot because of it.

#1 - Blog Mechanics and Physics (Rank)

I've learned that my blog has about 10 visitors per day if I'm careful and exclude myself. This number seems to have absolutely no correlation to Technorati's ranking. I had expected them to be somewhat inversely proportional from the ClueTrain and LongTail theories.

Using Sitemeter, I'm now learning that people hit this blog from 2 sources:

• When I happen to say something that results in a link from an "A lister"
• When someone does a Google search and matches something I've written about

#2 - Attention

Occasionally someone will read something on one of my pages that actually causes them to want to read more... at this point I've actually gained some attention which is the hot commodity in the blog-o-sphere.

I'd estimate that perhaps 1 to as many as 10 page reads per day are due to actual interesting material. This is sufficient interest to me to warrant further writing and posting, and perhaps even finding a better blogging software to support categories, which would make it easier to discover my other "valuable" (ha!) "content" (aka stuff or miscellaneous scribblings)

Now, I'm fairly certain that at this point I have only two regular readers... myself and my lovely wife, Noran. I doubt that Doc, Dave, or anyone else actually reads this blog on a regular basis. I'm too sporadic in posting to meet the implied social contract of the blog-o-sphere... one post per day, minimum, no exceptions.

#3 - Editing - as process

My actual flow in building a post is:

• Somebody says something I want to comment about
• Start up a blog comment
• Start putting together content, with links, etc...
• reconsider what I'm saying
• extract redundant stuff, crap, and unneccessary flamebait
• read it out loud (if circumstances permit)
• edit some more
• post it
• ping the original sources (put comments on their sites)

I worry I don't spend enough time editing, but the limited feedback from Doc tells me I'm ok blatering a bit... could it be Doc is wrong?

#4 What Doc Searls means to me

Sidenote: Like Dave, I really like Doc. One day I hope to meet him. It was weird hearing his voice on a podcast though... I expected him to be someone who listens, thinks a minute, then gives a slowly paced, very wise answer... he talks wayyyy faster than that.

Doc's got a lot of oars in the water (or so it seems to me)... and our agendas aren't always in sync. When Doc links to me, I get a big spike, usually only to the page directly linked, which I've learned to expect.

Doc always picks something I wouldn't ever expect out of what I write. I spend too much time trying to be clever at times... or is that just self doubt?

Anyway, Doc is an editor by trade, and serves as my editor on occasion, which I think is a valuable service to me and the community as a whole.

#5 - oh... back to beliefs, and how they change.

Dave and others offer a refreshing view of the world. I wouldn't have found Dave except as a result of my interaction with the blog-o-sphere... and I'm glad I found him.

I now spend a ton less time trying to be in the first 100 posts on slashdot, and spend about the same amount of time, overall, here on this blog. I'm finding it to be a worthwhile trade for me.

I find myself open to more views, and also more careful about how and what I say. I think this will have an overall positive effect on me in the long run.

I believe that while bloggers are good at smoking out lies, they can't do the job alone. I know there are a lot of good people doing a lot of work, and getting even a small group going consistently in the same general direction can be a very potent force.

back to blogging and beliefs... Again!

Companies need to fear us, the Cluetrain is dead on accurate about that. Bloggers are reactionary, and like to pile on to a story, it's human nature. I'm not sure that there is much real power in the blog-o-sphere... but we all like to think there is... the jury (for me) is still out on this one.

It takes a work and especially intention to get results that matter. Finding our voices, and learning to direct them for good is a long term challenge. The cluetrain is a visionary statement, that many of us want to be true. If we work hard, some of it will come true, but of course reality is always different that what we expect.

I've written a lot about this, but no hard evidence, only my gut feelings on it... I hope that's enough Dave.

--Mike--

Finding your voice in Web 2.0

I've gotten hits from Doc and others, which pushed my my Techorati rank, for a while... but it's becoming quite clear to me that I'm not a compulsive poster... doing something day in and day out isn't my strong suit. I do better with spur of the moment commentary and analysis.

I use sitemeter to track visits to this blog... it's obvious if you look at the bottom of the page. I've noticed that a lot of my referrers are from google searches. It appears that I've written on subjects people are interested in... and that's where my base (less than 10 visits/day average) traffic is coming from. It's random, but persistent.

I think this is actually where my voice is going to be found. I'm not going to be very good at climbing the blogging tail, but I may be more successful commenting on things I have some good ideas on, worrying far less about frequency of postings, and more about quality.

I read Doc every day... I'm sure that at least 1000 other people do as well. The rock star economics of it mean that I'm never going to be as popular in those terms unless I post often, and very consistently, and keep feeding the beast that is rank. My opinion is that Doc is a writer by trade, and he does treat it as essentially an email with CC to everyone...

I don't have that many friends to do that with... but here I am... I'm learning, and trying to find my niche. It's all that any of us can be expected to do.

Oh... I uploaded some photos of Grandma's visit this past Sunday.

Jessica and Valerie, I'm sorry I took so long to get them uploaded.

--Mike--