Writing ebooks with ERB

For me, the upper limit of acceptable commute time is 10 minutes, but a longer commute time does give me time to think up of things while navigating a familiar route. One such idea came from a sudden desire to write a book. Having gotten Stanza for my iPod, and having a mind that cross-links ideas together like an encyclopedia, I'd want to publish it via epub and on a MediaWiki platform. I'm getting too old for copy-and-paste editing, so having a source document written in ERB would be ideal. In fact, I even got as far as a vague notion of using Nokogiri and/or XML as some sort of intermediate representation.

Turns out, while (as usual) searching for something completely unrelated, I found that someone has already implemented this tool: http://snk.tuxfamily.org/lib/erbook/.

Stealing Let from Rspec

Since Rspec 1.3, you could define variables in example groups that cascaded down, thus more or less eliminating the need for a before block. You do this by calling let(). By defining them with the right words, the spec code gets simplified and reads more naturally. I found it weird that it would be named after something I kept seeing around in Lisp, so I looked it up and found this. I did not understand it at all. When I cracked open the Rspec, I found an absurdly simple implementation: the Let module defines a memoized method when you call let. Since Rspec 2 breaks up examples into their own class, you get the cascading effect simply from Ruby inheritance and scoping. Dead simple. Most of the text in that file is actually documentation.

This memoized patterns happens all over the place. One example is in inherited_resources:

class ProjectsController < InheritedResources::Base
  protected
  def collection
    @projects ||= end_of_association_chain.paginate(:page => params[:page])
  end
end

I have some similar code, but since I am working on a pure web service project, I'm not actually using any assigns. But the technique is similar. inherited_resource calls out methods that are sensible defaults most of the time. To customize it, you would override one of several of the hooks, such as object, collection, end_of_association_chain. But this looks a lot nicer using the let() syntax:

class ProjectsController < InheritedResources::Base
  include Let
  let(:collection) { @projects ||= end_of_association_chain.paginate(:page => params[:page]) }
end

And then I dropped this into app/concerns. This was actually extracted from my (working) Rails 3 web services project:

# http://gist.github.com/453389
module Let
  extend ActiveSupport::Concern

  included do
    extend Let::ClassMethods
  end

  private

  def __memoized # :nodoc:
    @__memoized ||= {}
  end

  module ClassMethods
    def let(name, &block)
      define_method(name) do
        __memoized[name] ||= instance_eval(&block)
      end
      protected(name)
    end
  end
end

I stole this from Rspec 2, then ripped out the documentation and refactored it using ActiveSupport::Concern. I added a line to make the let() bindings protected so it won't show up as an action for the controller. This is probably the behavior you'd want in non-controller anyways.

The neatest thing I had found about let() in Rspec was easily making an implicit DSL inside my spec. The fact that I can carry this over to my controller is simply too cool not to share.

Untitled Haiku #1

irb(main):001:0> lisp(:code_generator).to_ruby
=> :metaprogramming

The knife is out for ruby ... ORLY?

Warning: Link Bait

From Hacker News, comes "The Knife is out for Ruby". "Dynamic languages are coming to the end of their honeymoon period. This is bad for Python and maybe PHP, but it is nothing short of a disaster for Ruby."

... leaving aside the religious war of Static vs. Typed language, how about viewing this from the point of view of Pragmatic Extropy?

"Type safe static languages like C#, Java and Managed COBOL are much more challenging to work with, or at least they were."

... X is better because it is more challenging to work with. Does that mean that the programmer has to be smarter?

"However, the gap is closing or may have completely closed. Type inference, generics and much better IDEs have made static languages really easy to work with. The permissive nature of dynamic languages is only really treating the symptom that programmers do not want to have to remember and key-in everything's type."

... X is better because it forces you to think through about issues Y?

Scrolling down to the comments, the author reveals a bit about himself:

"My day job is in a compiler writing team. I spend a lot of time in a lot of different languages. My view is that dynamic languages make sense for code bases which do no justify heavy tooling and which are not going to be around for a very long time. I used to think that dynamic languages were a lot more useful than that because of productivity - but recent advances in IDEs and type systems have convinced me that even if it is not the case now, over the next few years the difference in productivity will diminish a lot."

I was talking to folks on #pragmatic_extropy about my experience geocaching with a friend. I watched my friend get so dependent on the GPS, missing out obvious environmental cues and clues. I promised myself that I wouldn't do that if I held the GPS in my hand. My friend got a call and handed me to the GPS, and within five minutes, he was pointing out cues I missed because I was navigating with the GPS.

The GPS had easily become an extension. The neural net of the brain adapts to the external interface, in which the external interface becomes part of the complete circuit. There's a paper written about the phenomena of two neural nets forming an internal language idiosyncratic to those two particular neural nets. You can see it in two intimately-close people who have been around each other for a long time. Likewise, I remember an article about how family groups would socially (and unconsciously) assign specialty knowledge to specific family members. The same is true for any tool we use -- tab completion on the command line for example. I don't feel right without being able to hit the tab key since I remember filenames more on the first few letters and the number of tabs I have to hit. Other tools include code snippets, VIM cursor controls, distributed version control ... and IDE-based type inference and whatnot.

Some languages, however, such as Lisp, has the rare ability to transform how we think because the language is powerfully expressive enough. Its power derives from having very little syntax and does not require external tools to grok. Someone programming in .NET, for example, require an IDE to be productive because the IDE forms the complete neural circuit. Something like Lisp (within certain cases) has the complete neural circuit held inside the head. There is more transformative value for a Lisp-user than a .NET-user, even with a better IDE. While it can be argued that the IDE closes the gap in terms of productivity, it does not close the gap in terms of arete, though it may go with the grain of extropy.

Improving macro registration for Remarkable 4

Since one of the emerging design principle for Remarkable 4 is to shadow Rails 3, and Rails 3's biggest strength is its modularity, I've decided to split Remarkable into components following the Rails 3 modules. For now, we have remarkable (core), remarkable-activemodel, and remarkable-activerecord. I plan for remarkable-rails to be split into remarkable-rack and maybe remarkable-actioncontroller.

However, with all of this, we now introduced the idea of macro collections depending on other macro collections. However, the way these macros are loaded into Rspec does not easily support this proliferation of modules.

Currently, macro developer has to do this:

Remarkable.include_matchers!(Remarkable::ActiveModel, Rspec::Core::ExampleGroup)

But seriously, do the developers need to know they are targeting Rspec::Core::ExampleGroup? This should be a default, with it being overridden by people who know what they are doing.

The other thing is that due to the modularity of Rails 3, it makes sense to have a set of macros built on top of other macros. So for example, ActiveRecord macros are built on top of ActiveModel macros. Not everyone will want to include the entire Rails stack just to use the macros, and we certainly don't want to expose a target (at least, not without having to dig).

Here is a mock API of what I'm thinking of:

The developer adds this:

    Remarkable.register_macros(:activemodel, Remarkable::ActiveModel)
    Remarkable.register_macros(:activemodel, Remarkable::ActiveModel,  
      :target => Rspec::Core::ExampleGroup)
    Remarkable.register_macros(:activerecord, Remarkable::ActiveRecord,
      :depends_on => :activemodel

This makes it easy for plugin developers:

    Remarkable.register_macros(:foreigner, Remarkable::Foreigner,
      :depends_on => :activerecord)
    Remarkable.register_macros(:paperclip, Remarkable::Paperclip,
      :depends_on => :activerecord)

If you are using only pieces, then you can active the macros:

    Remarkable.activate!(:activerecord)
    Remarkable.activate!(:paperclip)

Which will idempotently include the macros and its dependencies into Rspec2.

Remarkable::Rails, being the gregarious gem it is, will call:

    Remarkable.activate!

... which will load up all of the macros. This would be in keeping with the expected convention for Rails while still providing modularity.

I've opened up this issue on the github to discuss this further.

Remarkable 4.0.0.alpha1

I finally got a Remarkable 4 alpha1 one out. Here is how to use it in your project:

gem install remarkable remarkable_activemodel remarkable_activerecord --pre

There is no 4.0.0.alpha1 release for remarkable_rails. Please upvote and comment this github issue if this is important to you.

To install:

# In Gemfiles
group :test do
   gem 'rspec'
   gem 'rspec-rails'
   gem 'remarkable', '>=4.0.0.alpha1'
   gem 'remarkable_activemodel', '>=4.0.0.alpha1'
   gem 'remarkable_activerecord', '>=4.0.0.alpha1'
end

and add this in your spec/spec_helper.rb

  Remarkable.include_matchers!(Remarkable::ActiveModel, Rspec::Core::ExampleGroup) 
  Remarkable.include_matchers!(Remarkable::ActiveRecord, Rspec::Core::ExampleGroup) 

Please note, you will need to have rspec 2.0.0.alpha7 to run this, either installed in system gems or in the bundle.

Caveats

While I ported as many of the features of Remarkable 3.3 over, there were some things that were dropped:

• support_attributes and passing attributes into a describe or context block no longer works. Rspec2 provides a metadata functionality allowing you to pass metadata in examples and example groups, and this conflicts with the semantics for support_attributes. Quite frankly, I never even knew this feature existed until I attempted this port; it isn't something I ever needed since I do not write models in a way where its behavior changes drastically based on state. When it does change, I typically write out the descriptions long-hand.
  
  If there are enough upvotes for this issue, or someone who cares enough decide to submit a patch, we can add this back in. Any implementation will use the Rspec2 metadata though.



• pending example groups are baked into Rspec2, so I took it out of Remarkable. The specs for the macros show they work.


• I took out the validations that were implemented in ActiveModel and moved them over to a brand-new, shiny remarkable_activemodel gem. You will need both remarkable_activemodel and remarkable_activerecord to use the validations. I plan on making the packaging better so you can easily just activate ActiveModel macros whenever you load in ActiveRecord models. This will allow people to use the standard validation macros against any ActiveModel::Validations-compliant models, which I expect the vast majority of the NoSQL and ActiveResource models to implement.

Please post any issues to the Remarkable issue tracker

Taking over the Remarkable gem

Remarkable is an awesome gem. If you use Rspec for a Rails project, this macro framework is certainly something to consider.

The classic way of using Rspec to test your ActiveRecord model looks like this:

class Person < ActiveRecord::Base
  belongs_to :organization

  validates_presence_of :name
  validates_presence of :email
  validates_uniqueness_of :email
end

# In spec/models/person_spec.rb
describe Person do
  before(:each) do
    @valid_attributes = {
      :name => "Name"
      :email => "email@example.com"                                                                                                                                                 
    }
  end
                                                                                                                                                                       
  it "should create a new instance given valid attributes" do
    Person.create!(@valid_attributes)
  end

  it "should validate presence of :name" do
    @person = Person.new(@valid_attributes.except(:name)
    @person.should_not be_valid
    @person.errors.on(:name).should_not be_nil
  end
                                                                                                                                                                   
  it "should validate presence of :email" do
    @person = Person.new(@valid_attributes.except(:email)
    @person.should_not be_valid
    @person.errors.on(:email).should_not be_nil
  end

   it "should validate uniqueness of :email" do
    @original_person = Person.create!(@valid_attributes)
    @clone = Person.create!(@valid_attributes)
    @person.should_not be_valid
    @person.errors.on(:email).should_not be_nil
   end
end

[ See Gist ]

What this does is create a template model that has all the required attributes. We have an example that makes sure it is valid. Then for every validation, we create a modified attribute set so we can test each attribute in isolation.

This gets old very fast. Trust me, I've written many tests like this since Rspec got popular in 2008.

So how can we make this easier?

We can try using fixtures. The problem with fixtures is that (1) we have to manage our own primary keys and argue with the database and (2) you would have to load the fixture in, create a clone, and attempt to test each of the validation.

We can try mocking. This would give us the isolation when we stub out the mock model, but that's essentially another way of writing a valid_attributes. Besides, now we're digging deep into the internals of ActiveRecord.

We can use factories. I like factories, in particular, I currently use the Machinist in combination with Faker gem and Forgery. This gives me the advantage of having a blueprints.rb file I can include in db/seeds.rb to generate data I want to drop into my views. However, we are still essentially testing the ActiveRecord framework.

This is where Remarkable comes in. The above Rspec condenses down to:

describe Person do
  it { should validate_presence_of :name }
  it { should validate_presence_of :email }
  it { should validate_uniqueness_of :email }
end

Done.

[ See Gist]

Notice that these macros shadow the ActiveRecord declarations. This is true for all the other macros, including association macros such as should have_many, should belong_to. Instead of focusing on boilerplate test examples, you focus on writing examples for custom code and really weird business logic.

Remarkable has come a long way from its beginning as a "Shoulda for Rspec". Since version 3.0, Remarkable has been refactored into a generic framework for creating Rspec macros. Remarkable::ActiveRecord and Remarkable::Rails were extensions of this macro. As a result, we saw some other macros: a remarkable for paperclip, mongo, and datamapper. There is also an internationalization package that lets you internationalize your rspec.

However, with big refactoring of Rails 3, followed by the big refactoring of Rspec 2, Remarkable needs some work. Carlos Brando, the founding author of the Remarkable gem has been looking for a new maintainer, and has passed the torch to me. As I stated in the Remarkable mailing list and on Rails-talk, my plan is to release a Remarkable 4.0 that is compatible with Rails 3 and Rspec 2, however, it will not be backwards compatible with Rails 2 and Rspec 1. Remarkable 3.3 will be around for that.

If you use Remarkable and plan on using it for your Rails 3 project, please speak up now. We're going to take advantage of the mass refactoring to do some refactoring of our own.