Testing your Hadoop jobs with MRUnit

Last Tuesday I gave a short presentation at the new Boulder Hadoopers Group about testing Hadoop jobs with MRUnit. You will have to know what Hadoop is and how to read Groovy code to fully understand it. I am including the important notes on the slides as well.

If your browser doesn’t support flash, check out the slides at slideshare

Why use MRUnit?

Testing a Hadoop job requires a lot of effort not related to the job. You must configure it to run locally, create a sample input file, run the job on your sample input, and then compare to an expected output file. This not only takes time, but makes your tests run very slow due to all the file I/O.

A CSS-only speech bubble

I generally try to avoid using images or Javascript when I can accomplish good presentation with CSS. In this case, I wanted to apply CSS shapes to make a clever speech bubble.

The problem with obtuse triangles and CSS

Obtuse triangles are slightly more complicated, since you can only create acute and right triangles with the CSS shapes method linked above. Therefore I created two triangles: a positive (black) right triangle, and then a negative (white) triangle to emulate an obtuse triangle.

Interview with Andres Almiray

Griffon logoAndres Almiray is the lead developer of Griffon, a Grails like application framework for developing desktop applications in Groovy, and a committer on the Groovy programming language. I consider him to be one of the most influential programmers in the world. I strongly recommend that you subscribe to his blog and follow him on Twitter.

Andres was kind enough to answer questions that I think are relevant to all of you readers. I hope you learn from them as much as I have. Without further ado…