Friday, May 7, 2010

Adventures with chef-server and Gentoo, part 6

Continued from Part 5

Lots of updates:

  1. I further refined the portage_conf definition. You can pass :force_regen to force a regeneration of make.conf. This is necessary in the case of mirrorselect if you need to install a package.

  2. I added a bunch of recipes. Here is an index of them so far:


    Base portage


    Installs veszig's chef-overlay


    Enable building binary packages


    Fetch binary packages from binhost


    Do not build docs, man pages, and info pages


    Adds emerge --sync to cron for nightly-syncs


    Exclude certain categories from sync, such as nethack ...


    Point to a site binhost


    Point to a site rsync mirror


    Selects the three fastest Gentoo Mirror.


    Installs gentoolkit


    Sets up a binhost server with Nginx


    Sets up a local rsync mirror

  3. I talked to jtimberman and veszig on and got some feedback:

    • jtimberman suggested Resource/Provider to replace portage_conf. That was the direction I've been heading towards, though what I have now works. (Unless you want to notify/subscribe to it, then it doesn't)

    • veszig showed me what he has in gentoo-cookbooks. He is using eix for package queries, and actually has a LWRP for emerges. I think it is called gentoo-package. It will let you set USE flags and masks, unmasks, and keywords. I've been using definitions, but again, I can't send notifications to them. I think the way he has set up the emerge is better than what I was going to do -- monkeypatch the Portage provider

    • jtimberman mentioned that knife can do Rackspace deploys, though it is Debian/Ubuntu centric. But the are accepting patches :-) He also mentioned more things about the data-driven database and application recipes, which I can't wait to get started working on that.

All in all, good times. I'm going to attempt a merge of my stuff into a fork of gentoo-cookbooks, and move anything generic back into the mainline cookbooks repo. I may still end up submitting an improved Portage provider, just in case you want to still remain distro-agnostic.


It has been two months since I wrote "Intangible Assets and Sunk Cost Fallacy", and this conversation with jtimberman and veszig illustrates the most noticeable payoff of intangible assets, at least in the Rails world. The fact that I can look at veszig's code and veszig can look at my code meant that we can exchange best practices faster. I didn't know about eix, and while more complicated, portage_conf is pretty useful. This would be much more difficult to do if we were exchanging words. Neither of us can guarantee that we are following the exact steps as specified (something that, I'm sure, eats up the vast majority of systems troubleshooting). On the other hand, we each can find out what the other did, simply by looking at the code. "Best Practice" becomes self-documenting, executable code, code that can be conserved, exchanged, and source-controlled.

Ruby, Python, Perl, and Java each have a vast, vibrant community because of the libraries of gems, eggs, CPAN, and Java code; I'm glad to be a part of something new here -- a library of infrastructure code that is still largely invisible to the mainstream. It's my "secret" competitive advantage.


  1. Don't know if this is any good for you, here's what I'm using for my gentoo based rackspace cloud chef nodes instead of `knife rackspace server ...`:

  2. Thanks, I saw those gists this morning. They'll probably be useful.