“Lost Cat” by Caroline Paul

After all the fun reading “Meanwhile in San Francisco”, I looked to see if this duo had co-written any other books. Sure enough, they had.

“Lost Cat” tells the true story of how an urban cat owner (one of the authors) loses her cat, then has the cat casually walk back in the door weeks later healthy and well. The book details various experiments the authors did using GPS trackers, and tiny “CatCam” cameras to figure out where her cat actually went. Overlaying that data onto google maps surprised them both – they never knew their cats roamed so far and wide across the city. The detective work they did to track down and then meeting with “Cat StealerA” and “Cat Stealer B” made for a fun read… Just like “Meanwhile in San Francisco”, the illustrations are all paintings. Literally. My all-time favorite painting of any cat ever is on page7.

A fun read… and a great gift to any urban cat owners you know.

Calling all Release Engineers, Hortonworks is hiring

The Hortonworks Release Engineering team is growing, so we’re hiring!

We’re passionate about open source, and ensure that all 100% of code in a Hortonworks HDP release is open sourced in the Apache Software Foundation Hadoop project. We work with other large organizations to help them upstream their contributions to the Apache project, which helps accelerate the general Hadoop community. Its so important to us, it is part of the Hortonworks Manifesto.

We’re proud of our HDP releases. Our clients rely on HDP in production environments where phrases like “petabytes per day” and “zettabytes” are common. We sim-ship on centos5, centos6, ubuntu, debian, suse and windows – all from the same changeset. Building and testing at this scale has its own special forms of challenges, and is exciting. In the rare case where customers hit production issues, we are able to deliver supported fixes super-quickly.

The Hortonworks Release Engineering team works hard behind the scenes to design, build and maintain the infrastructure-at-scale needed to make this possible. For more details, and to apply, click here.

Note: The current team is spread across 3 cities, so remoties are welcome, even encouraged! Hardly a surprise if you read the other remoties posts on my blog, but worth stating explicitly!

Hortonworks HDP 2.1 shipped!

HDP2.1 shipped on 22apr2014.

This was the first significant feature release shipped since I joined Hortonworks at the start of the year. There’s lots of interesting new features, and functionality in this HDP2.1 release – already well covered by others in great detail here. Oh, and of course, you can
download it from here.

In this post, I’ll instead focus on some of the behind-the-scenes mechanics. There were lots of major accomplishments in this release, but the ones that really stood out to me were:

1) sim-ship windows and linux.
This was the first HDP release where all OS were built from the same changeset and shipped at the same time. Making this happen was a hectic first priority in January. As well as the plumbing/mechanics within RelEng, it also took lots of coordination changes across different groups within Hortonworks to make this happen. The payoff on this was great. We sim-shipped, which is great and massively important for HWX as a company. Even more importantly, we set things up so we could sim-ship for every HDP2.1-and-above release going forward… and we proved it by sim-shipping the quick followup HDP2.1.2.0 release on 02may2014.

2) adding 5 new components.
HDP2.1 contained 17 components, compared to HDP 2.0 (with 12 components) and HDP 1.3 (with 10 components), making HDP2.1 the largest growth of components ever?!? Oh, and in addition to the new components, every one of the 12 pre-existing components were also significantly updated to newer versions. That meant that each required significant new integration work, new installers on all supported OS (…remember the “sim-ship” goal?). Oh, and we were to ship all this new functionality at the fastest cadence yet.

3) improving support for other trains.
In January, we were learning how to support 3 active trains of code: supporting 1.3 and 2.0 maintenance work, while also building out infrastructure for 2.1 new-product-development-work… even while the 2.1 development work was in progress, which obviously complicated things for developers. Today, we’re supporting 4 active trains: maintenance work for 1.3, 2.0 and 2.1, as well as the 2.2 new-product-development-work. This time, the 2.2 infrastructure was built out and live before developers finished working on 2.1… enabling the developers! Things are not perfect yet, by any means, but today (with 4 trains) feels calmer and more organized then earlier this year (with “only” 3 trains).

All great improvements to see up close, and all important to us as we scale. Big thanks to everyone for their help… and do stay tuned for even more improvements already underway.


“Meanwhile in San Francisco” by Wendy MacNaughton

I stumbled across this book by accident recently, and really enjoyed it. One of the reasons I love to travel is because of the different cultural norms… what is “normal” in one location would be considered downright “odd/strange/unusual” in another location. Since I first moved to San Francisco, the different types of people, from different backgrounds, who each call this town “home” continue to fascinate me… and all in a small 7mile x 7mile area.

This book is painted (yes really!) by a San Francisco resident, and does an excellent job of describing the heart of many different aspects of this unique town: Mah Jong in Chinatown, the SF City Library’s fulltime employee who is a social worker for homeless people, Frank Chu, Critical Mass, dogwalkers, Mission Hipsters, Muni drivers … and of course, everything you need to know about a Mission burrito!

A fun read… and a great gift to anyone who has patiently listened while you’ve tried to explain what makes San Francisco so special.

BBC’s Panorama coverage of The Spaghetti Harvest

(“Panorama” is the very-serious-current-affairs program of the British Broadcasting Corporation, and has been running continuously since 1953, making it the longest running current affairs program in the world.)

On 1st April, 1957, Panorama ended its show with a brief ~3minute segment on the early harvest of the Spaghetti trees along the Swiss-Italian border.

It is believed to be one of the first times an April’s Fool joke was played on television viewers, and caused quite the stir at the time. Excellently put together, with great attention to detail, and a script echoing an earlier segment about the French wine harvest, I found it a great fun 3minute watch.


More details on Wikipedia and The BBC.

RelEngCon 2014 registration is now open!

In case you missed the announcements, RelEngConf 2014 is officially now open for registrations. This follows the inaugural and wildly successful Release Engineering conference , held in San Francisco on 20may2013, as part of ICSE 2013. More background here.

Last year’s event was great. The mixture of attendees and speakers, from academia and battle-hardened industry, made for some riveting topics. So I already had high expectations for this year… no pressure on the organizers! Then I heard this years will be held in Google HQ MountainView, and feature opening keynotes from Chuck Rossi (RelEng, Facebook, click for linkedin profile), and Dinah McNutt (RelEng, Google, click for linkedin profile). Looks like RelEngConf 2014 is already lining up to be special also.

If you build software delivery pipelines for your company, or if you work in a software company that has software delivery needs, I recommend you follow @relengcon, block off April 11th, 2014 on your calendar and book now. It will be well worth your time.

See you there!