A drop of Coffee

by Aurélien Thieriot

At Box UK, we recently switched our internal messaging system from IRC to Hipchat. For me, it was the opportunity to re-open a recurrent idea of mine: Writing a convenient deploying Bot. More generally, I’m quite fond of small command line tools that helps with the development process which is often unique to each team. Without a second thought, I was already reading the Hipchat API documentation.

But why did I choose Node.js? Well. As a primary argument, it’s not the first side project where I experiment Coffeescript. I start being reasonably fluent with it and as I wanted to quickly bosh a prototype, it was not the good playground for a completely new piece of tech. In addition, this Bot would heavily use HTTP APIs (We use Github quite a lot) and even not in a server context, Node.js is particularly efficient and adapted at consuming web services. Finally, hey, why not?

Another question would be: Why not Hubot? First of all, the Hubot adapter for Hipchat use the XMPP compatibility wchich ultimately involve more cost for the company so the free API was mandatory. Also, Hubot is really great but full of a lot of general scripts and in this case, having a tailored workflow seemed to be better.

Anyway, after two days I had a prototype and after one week of feedbacks from the team, a satisfying usable version. It was good enough to quickly bootstrap something and solid to support maintaining. In this blog post, I wanted to show my personal undergoing of the state of the Node.js and Coffeescript ecosystem outside the current global passion for MVC frameworks.

Because every story has a beginning

Does everyone knows npm? If not, it’s the awesome package manager embedded in Node.

A good entry point for any new project:

  $ npm init

After a couple a questions, you end up with a complete package.json, all set for a minimal Node module.

But the real first step, the one I got in mind even before starting, was commander.js. It’s one of the many option parser available to Node.js developers (You have a billion of those for every languages). Written by the fantastic TJ Holowaychuk it is particularly adapted for albot because you can not only parse options but build commands as well. Like in Git for example, you can easily handle lines like

  $ albot do something --verbose

If you want more details, maybe that will help you: aurelien.thier.io/wing-commander-js

To install it:

  $ npm install commander --save

Using --save add the dependency to your package.json as well as downloading it. --save-dev will add the module to your devs dependencies.

Armed with these clever tools. Let’s see how to write an executable script:

  #!/usr/bin/env coffee

  Program = require 'commander'
  Fs = require 'fs'

  Program.version JSON.parse(Fs.readFileSync('./package.json', 'utf8')).version

  Program
    .command(‘pulls’)
    .description(‘Display the list of Pull Requests of your organisation’)
    .action(pulls)

  pulls = (arg) ->
    console.log ‘Requesting: #{arg}’

  Program.parse process.argv

Usage:

  $ chmod +x albot.coffee
  $ ./albot.coffee pulls things

Almost as quick as a bash script but ready to use modules and many more delicacies.

Bonus. Npm allows you to create globally installable executables. Add these lines in your package.json:

 "bin": {
   "albot": "albot.coffee"
 }

Then run:

  $ npm adduser # Only the first time
  $ npm publish

To use your awesome tool, on the other side of the world, a user can just do:

  $ npm -g install albot
  $ albot --help

The precise process is very well explain in this documentation: https://npmjs.org/doc/developers.html

Can we code now?

Yes. Let’s start the serious business. As a first command, as it's better to go back and forth in order to validate the Hipchat system, I built a command that retrieved all the pending Pull Requests of every repository in our organization (filtered by a term). When you work on a multi-modules applications in a multi-projects company, it’s not always easy to sort out the Pull Requests page on Github.

  $ albot pulls with urgent

Is one of the requests you can type.

No Coffeescript code will be covered here. The website http://coffeescript.org/ is far better than me to explain the difference between -> and =>) but there is some observations:

There was a library for everything. One for the Github API, one for the Hipchat API and so on. Very easy to use, very easy to include in the project (The main issue was with the Hipchat API itself). In fact, it’s quite impressive to see how the Javascript community work around libraries instead of frameworks. There are frameworks of course but, for a majority of things, you will find a small library that do one job and do it well. You are not tied to one provider. Even if sometimes, you will figure out that you download often the work of the same people. TJ Holowaychu and the Nodejitsu team are some of those talented guys.

One of the first concern I got was to find a nice way to retrieve the user configurations. API Tokens, filters, etc... A popular tool for such need is Nconf. Built by Nodejitsu as a part of the Flatiron project, it let you read a simple json file, organise overriding, default values, etc...

Finally, even if Coffeescript delayed the need for Underscore.js (Or equivalent functional oriented sugar for Javascript), I quickly felt the necessity for something to help me handle the nested callbacks. Requesting loads of details from the Github API is easy but require more than one call where the result of the previous one is used in the next. Async is perfect for this and if it is well integrated with the Node.js way of doing stuff, I could probably have been more meticulous with it. An example of nested call:

     Async.waterfall [
       (callback) ->
          github.pullRequests.get {user: org, repo: reponame, number: number}, callback
       ,
       (pr, callback) ->
          github.statuses.get { user: org, repo: reponame, sha: pr.head.sha }, callback
     ], (error, result) ->
        if (not error?)
          console.log result

At the end, I reckon that I chose the most popular challenger in many cases. But, after all, that shows that the ecosystem can be mature and offer a default set of libraries for standard development scenarios.

Wait a minute. You should add some tests

I believe that the real challenge of a language is the tooling. Having enough glue for your code and your community. A classic stack would be: Testing, Continuous Integration, scripting, mock, code linting, debugging... Feel free to ask in the comment if I forgot something.

Nothing fancy in term of testing: Mocha with the BDD flavour of Chai. Solid, feature complete for synchronous and asynchronous tests, watcher included and loads of fun reporters (You can display a Nyan cat while your tests are passing... Isn’t it awesome?)

It appears that it’s extremely easy to make your test suite pass on Travis CI. Some package.json tweaking:

  #Note: All your dependencies can be included as a local binary
  "scripts": {
     "test": "./node_modules/.bin/mocha"
  }

And the standard Travis config:

  language: node_js
  node_js:
     - "0.10"
     - "0.11"

As the code was strongly using HTTP calls, it needed integration tests with mocks. Nodejitsu helped me once again with Nock. You describe the URL you want to match and give a mocked result. No need to tell you that JSON is a first class citizen in this process.

  Nock('https://api.github.com')
    .persist()
    .get('/orgs/testorg/repos?per_page=100&access_token=testtoken')
    .reply(200, [
        {
          "name": "test-repo",
        }
      ])

I didn’t mentioned that before, but until then, everything was handling Coffeescript naturally. It’s another story when comes the linting and code coverage. You can do it, don’t get me wrong, but it require additional tools.

Coffeelint to validate your code and coffee-coverage for instrumentation. I have to admit, though, that it was a pain to configure a nice code coverage report without too much code changes. By default, with JsCoverage, Blanket and Istanbul you have to make heavy changes in your tests to require the instrumented code. I finally opted for covershot, a small project which offer a nice way to get around this issue. It also offer a good looking HTML reporting and a straightforward integration with coveralls: https://coveralls.io

From this point, you may want something more. You may want to be able to customize your testing steps. Probably add more steps or configurations. Provide an entry point to run only on Travis or you may simply want to organise your code. A little while ago, the Node.js people were using a Makefile for this, or maybe a Rakefile. Not anymore. Now we have Grunt. Fruit of a difficult battle, Grunt is the best you could hope. You write your steps in Javascript/Coffeescript, it feels like home (really). It comes with a wide range of plugins for all usage like launching Mocha, linting code and much more. (If you are into web development, look at Yeoman.io, it’s a nice showcase for Grunt)

One of the most useful plugin for a command line tool was grunt-release. It add a couple of steps that automatically upgrade the version of your package, create a tag, push to Github and publish the result in Npm:

  $ grunt release:minor

Last point on the tooling: Debugging. Unfortunately, there is not a lot of alternatives here. One of the most know: node-inspector is no more maintained. It worth for Coffeescript and Javascript. Apparently, the WebStorm IDE is good for it but I was unable to find a stable solution to use in the command line (Or at least as a standalone tool). It’s a shame because everything is perfect until you have to add some console.log in order to find a problem.

Conclusion

The scope of this project is quite limited compared to a web application. However, I was delighted to be able to build a complete command line tool using only Coffeescript. It is a perfectly valid alternative to Ruby, Python or even Bash/Zsh for automation scripts.

The ecosystem is still dynamic and mature libraries have emerged. You can find all the comfort of any other platform.

About

As a follower of the software craftsmanship movement, an open source enthusiast and an Agile advocate, I have worked with computers for 10 years. While starting my career as a technician, I quickly became an experienced Java developer and a good team worker. I loves the web, small automation tools and I am always looking for new technologies to play with. More about me: http://aurelien.thier.io/

Box UK is an award-winning software development consultancy, combining strategic insight and technical development expertise with proven Agile and Lean delivery techniques. We embrace flexibility and adaptability to quickly respond to changing conditions while maintaining the highest standards of quality.

Gun.io is how top entrepreneurs and executives launch incredible products.

Hire Freelancers

Or
Apply to Freelance








Ready to make your product launch a success?








gun.io, 2017. Built with in Berkeley, Boston, Nashville, New York, and Philadelphia. | email: team@gun.io | tel: +1 (234)-56GUNIO