Huzzah for backcountry.com, on taking a big step toward full open source citizenship. My former employer announced last week that they would release some postgresql code back to the community. “Finally stop mooching” is how their usual informal press release put it, and props for that. The code allows for master< ->master replication across postgresql databases. It’s called ‘bucardo’ after an extinct goat that lived in the Pyrennes mountains that might be cloned back into existance (replicated, get it?)

Backcountry.com has a webfarm at a hosting center, and then a rootdb back at the warehouse– pretty standard setup. For the sake of speed, however, the company wanted to hold as much information (i.e. dynamic content) as close to the web servers as possible, and therefore wanted to replicate a good chunk of rootdb out at the ISP. If they were using Oracle, this would be a relatively straightforward configuration. However, because the system was built around Postgresql, we needed to configure something out. Enter the geniuses as End Point Communications — an entire system of flags sit around data tables, and these flags are synced back and forth, to determine which data then needs to be pushed to the other side. The trick here is to have the 2-way asynchronous flows of information while maintaining basic ground rules of data integrity.

The system has had several months of getting the shit beat out of it, literally. Early on, syncs would hang on a missing flag, or an unsuually large amount of data, and things would back up, only to then try and flow through like a traffic jammed freeway once the flags fell back into place. Well, redundancies have since been built into the messaging system, and the logic has been wrung to optimised levels where flags that don’t need to be in place are simply bypassed afte x minutes. It’s all rather elegant.

Props to all those that were involved with this, and special props to those at End Point who suggested the path to enlightenment lie in giving back to the community.