We actually may have taken a relatively large step this week: the new Obama adminstration has declared a very open information policy, and their IT structure shows promise in moving this same direction.  While not running Apache and Red Hat yet, they’ve certainly adopted some open social interaction structures.  It would be safe to say that the Obama administration is the best so far at “getting it“.

The missing element here, and the biggest specific step the US Government could make next would be to demand that all documentation be saved in an open format.  Want proof that Microsoft still has a monopoly?  Try sending someone your resume, proposal, or memo in something other than .DOC format.  If the government simply declared that all archives and transactional documents must be saved in .  The ODF is a deeply-flawed, but acceptable good start.  Ultimately, I am not sure the document format will matter.  Within the next 3 years, I bet that words and numbers and tables and figures and images are simply kept in the cloud.  I know I am not the first one to say this– but I can hope that the government would take an active role in pushing documentation into the common external places.  Cloud computing is not a technological hurdle, it’s a social acceptance problem.  I still encounter many people that resist putting things “out there”.  When pressed, there is no specific reason, other than people think the hard drive on their laptop is somehow safer than the huge servers tucked into concrete bunkers somewhere along the Columbia River.

I would hope that the transparency issue continues to gather steam.  I would hope to see the day when the government simply insists that all documentation: project bids, meeting minutes, deliberations, and especially lobbyist efforts, get published in a format that is easily remixed, chewed up, and boiled around in ways that slightly scare the powers-that-be.  We’ve seen a steady march forward with DARPA since the 60s and 70s, early gopher scientific info in the 80s, and then Thomas in the 90s, but the government certainly has dropped the ball in the last 5 years.  If the government can resume its Jeffersonian role in pushing new things for the republic, and allowing the market to fill in where advantageous, then maybe someone’s next interaction with the government will force them into putting things out there for scrutiny (policy or code), and they’ll realize that privacy ultimately depends on open scrutiny, not secrecy.

No home wifi firewall will protect us from an opaque government.

Every successful project, especially in the Open Source world, has a charismatic leader bean at the center.  There’s no money in there, so developers must be drawn in by the concept, but also by the visionary personality of the creator.  Some types have this in spades: Steve Jobs, Dean Kamen, Sergey Brin.  Some have the management chops to a certain level, but ultimately let some self-ascribed non-negotiable prinicple screw up an otherwise solid run: Jerry Yang, Jerry McGuire Sun Microsystems, and a whole zoo of half-assed half-built open source projects.

Someone once told me, “It’s not the strength of the idea, it’s the strength of your ability to convince other people of the strength of that idea.”  The most successful projects that eventually flourished into a viable open source application/platform all made that transition from mad-scientist-in-his-lab to wide corporate/commuinity acceptance because either one of two things happened:

1. the founder has enough charisma and business acumen to hire and manage around them (and let someone else do the day-to-day)
2. The founder happens to have a best friend early on in the process to do the business, and all the grovelling, compromising, yelling, coaching, and convincing that is needed to build up and run the circus.

So, If you’ve got that genius scheme out there, and it’s just not getting enough love on Freshmeat, it may be because you’re not sexy enough.  That, or you’re not writing enough polemic diatribes and visionary screeds on your project website.

Now that I am on the business side, and not held directly responsible for the tactical stuff, I can see why the business suits always simply stare at IT people in agogged wonder– these tight decision trees have nothing to do with the real world. Developers and engineers live their whole day in an artificial construct of reference hash tables, primary keys, routing diagrammes, and copies of 1s and 0s that need to be shephered from here to there and back safely. The engineer’s job consists of either a) building more of that artificial construct, b) cleaning up someone’s poor interpretation of that same construct, or c) defending their version against someone’s proposed revision.

Business people think in analog– sales are way up, slightly up, break even, almost to goal, a little off, under plan, or ‘in need of budget revision’. Notice the complete lack of any diacritical statements next time you talk to marketing– it’s all shades of orange (the new gray), very little black and white. Notice how the IT guys will pepper endless questions trying to make some logical tree out of statements like “make it cool”.

So what? Well, I don’t know yet. I’ve spent several years trying to come up with the magick formula, the correct set of questions to ask, the right analogy to frame things for the business owners. I hve learned the following points (in no logical order):

• never start with a stark choice– it scares the business types, and makes them inherently defensive
• when picking your analogy, try something close to the listeners’ heart: cars seem to be popular here in the midwest, while history worked well on the east coast. Japanese like to use organic metaphors: seeds, vines, roots (nemawashi), etc. Be careful with chess– your listened either has no clue about the game, or is a grandmaster– either way, they’ll start asking questions about your analogy for which you don’t have answers (you didn’t think that many moves ahead– oh the irony!)
• When laying the groundwork for your eventual “this vs. that” question commital point, make sure you attribute all the incoming data to someone else: bonus points for attibuting the background research/material to the same person you are about to ask for a decision– they feel smarter already.
• multi-variate choices are much better in terms of quick understanding, but they usually require a whiteboard to lay out the different factors involved (i.e. cost and complexity, time and ROI). At this point, the best you’re gonna get is to have your decider play pin-the-tail-on-the-donkey somewhere on the whiteboard. This is a false answer– it is still analog-y and relative to their opinion. You still don’t have a hard decision (maybe that’s enough?). Warning: do not attempt multi-variate using only verbal communication. Double Warning: don’t try this with a metaphor, as your listener will forget the question and only answer their opinion about dogs/cars/football teams/naval battles.
• Whenever possible, practice your Fractal Management.

In that context, I’ve had at least 6 people come up to me at work asking for my take on the Microsoft bid for Yahoo. In short: meh. Do I think Open Source is doomed at Yahoo? No. Do I think that Ballmer will make sweeping changes at Yahoo to have them tear out their BSD machines for IIS? No. Will Yahoo see talent drain out the backdoor? No more than is currently happening.

If anything, I see Yahoo infecting Microsoft much more than Microsoft dictating terms to Yahoo, much like Hong Kong is affecting the PRC after Beijing retook the colony. Just because MS is the one forking over the moolah, that doesn’t mean they want to dictate terms, it means they want to get something for their money: ad revenue, dynamic online channels, and perhaps (ZOMGWTFBBQ) some street cred with the online and open source market in that they might be able to show that even MS can own some datacenters with BSD and perl and php and apache and not muck things up. I’m willing to give them a shot.

Let’s remember what we all have been praying for: the end of the Windows Hegemony. It may actually be closer than we anticipated, and it may be a lot more peaceful than we thought (or ”hoped” in our bloodlust). At this point, MS is a big fat company that realizes they cannot develop the way they had for the last 15 years. Vista is the last hurrah. MS execs have said as much, and told us where they want to go. For once, I actually believe them. MS is a big fat wallet that needs to get the best ROI for investors– that’s what the stock market is all about. Yahoo may bring some new juice to the bar, and it will be up to MS to either do something good with it, leave it alone, or fuck it up. Either way, the knowledge transfer will be going from San Mateo to Redmond, not the other way.

In short, lighten up, Francis.

BONUS: The Economist thinks the deal is a steal for investors, and they use the Ballmer dance-monkey-dance video as their evidence.

This isn’t such a big revelation, but just something to think about next time people write off Open Source Software– it’ll be around for a while. Sure, the skins will change, and the emphasis will move around, but the kernels will always be there: Linux, Postgresql, mysql, perl, and yes– vi. People continue to riff off of these core pieces, just like musicians remix the masters

People remix the core bits of open source, and invariably the great riffs make it back into The Canon, only to be recycled and spun off in some new direction. I realize that many of the readers here know this already, but I offer up the analogy to those who have a hard time ‘getting’ open source: it’s rock-n-roll, man. It’s cool, and adapts to whatever the kids are into these days.

Backcountry.com has a webfarm at a hosting center, and then a rootdb back at the warehouse– pretty standard setup. For the sake of speed, however, the company wanted to hold as much information (i.e. dynamic content) as close to the web servers as possible, and therefore wanted to replicate a good chunk of rootdb out at the ISP. If they were using Oracle, this would be a relatively straightforward configuration. However, because the system was built around Postgresql, we needed to configure something out. Enter the geniuses as End Point Communications — an entire system of flags sit around data tables, and these flags are synced back and forth, to determine which data then needs to be pushed to the other side. The trick here is to have the 2-way asynchronous flows of information while maintaining basic ground rules of data integrity.

The system has had several months of getting the shit beat out of it, literally. Early on, syncs would hang on a missing flag, or an unsuually large amount of data, and things would back up, only to then try and flow through like a traffic jammed freeway once the flags fell back into place. Well, redundancies have since been built into the messaging system, and the logic has been wrung to optimised levels where flags that don’t need to be in place are simply bypassed afte x minutes. It’s all rather elegant.

Props to all those that were involved with this, and special props to those at End Point who suggested the path to enlightenment lie in giving back to the community.

1. Novell is already perceived as ‘tainted’ by their previous licencing agreements, whereas Red Hat follows a much quieter path
2. Red Hat has better pipeline into the technical arguments that may be needed to defend this (think old-skool UNIX interfaces)
3. Red Hat has better cashflow

In the end, I wouldn’t be surprised if Microsoft ends up ‘coming to the rescue’ to side with Novell (and grudgingly Red Hat). Afterall, one of the major (the only one I’ve seen?) eye-candy bits to Vista is their multiple desktop window 3-d thingamajig (which is crude and stupid ompared to the beauty of Beryl).

Overall, I’ve been happy with OpenOffice.org, but I can certainly see how the power users of MS Excel are very very wedded to their pivot tables and other mojo packed into the newer versions. Mircosoft’s best bet may be to flaunt this extended feature set– Excel is much more powerful than OpenOffice.org’s spreadsheet (there, I said it) for those who know what they’re doing. Granted, 95% of the people will never be at this level, but that other 5% are usually the thought-leaders on this kind of stuff in an office environment.

On a lighter note– who knew that the wiki for Star Trek is bigger than the wikipedia in over half of the European languages? Those trekkers are nothing if not well-versed.