It’s with a sad heart that we have to announce that the Kenai.com domain will be shutdown as part of the consolidation of project hosting sites now that Sun is a wholly owned subsidiary of Oracle.

This is sad because I thought Kenai had some really killer features like excellent JIRA and NetBeans integration. Nevertheless, it’s not up to me to decide.

Software is only as good as it’s community

A great project cannot thrive without people to improve and maintain it. The reason I am choosing GitHub is the number of people (especially friends) already on it. The is it’s main advantage over something like bitbucket. Git, in my opinion, has great momentum in the OSS community and is roughly equivalent to mercurial in functionality (with a few differences, obviously). Both DVCS are far superior to their non-distributed counterparts. To sum up the biggest benefit in a phrase: “Local commits FTW!”

Moving to GitHub has already paid off because my recent javascript-stacktrace project already has over 50 watchers and a couple forks.

Follow me

You should follow me on GitHub if you are interested in any of my projects, or if you think I might be interested in yours. Only good can come from reaching out. I look forward to seeing your OSS projects.

New links for my other projects (update your bookmarks or whatever):

• Cheqlist – JavaFX desktop application for Remember The Milk
• GroovyRTM – Groovy API for Remember The Milk

Agree about GitHub? Chime in on the “preferred version control system” poll and tell me if I’m full of crap in the comments.

Related posts:

1. Introducing GroovyRTM: A Groovier way to Remember The Milk
2. Follow-up: Why programmers should twitter
3. Interview with Andres Almiray

With that in mind, I am introducing an application that I think will help you with that: Cheqlist. A desktop application powered by Remember The Milk that does 2 things very well: make managing tasks very efficient and look sweet on your desktop.

What makes Cheqlist awesome

I wanted to build an application that I, myself, would use everyday. There are a few key features that would make that happen:

• Adding tasks quickly using RTM’s new Smart Add feature
• Visual appeal and lots of room for customization. Special thanks to Rakesh Menon for allowing me to use his JavaFX color picker
• Easily search tasks and use RTM’s custom search keywords
• Must work on all the OSes I use: Windows, Mac OS X, and Linux

On top of that, I think there are some things you’ll come to appreciate:

• Easy-update: Cheqlist checks for updates on startup, and if you want to upgrade, it takes just seconds!
• Free and open-source
• Lots of nice “easter egg” features for those of you who take time to find them ;)

A quick look

I’ve created a short (3 minutes) introductory video to show you the main features. UPDATE: Apparently screenr videos don’t show up in RSS properly. View it here.

Here are a few screenshots:

You can find more on the Cheqlist Wiki.

Try it out!

What you need:

• A marginally recent (6u11 or newer) version of Java
• Remember The Milk account
• Launch Cheqlist

Other resources

Something missing? Request a feature at the issue tracker.

Having trouble? Let me know directly.

For more information, you can hit the project page and Wiki on GitHub.

Enjoy!

Related posts:

1. Introducing GroovyRTM: A Groovier way to Remember The Milk
2. What and why: the tech behind Cheqlist
3. Why I’m moving my projects to GitHub

Restrain yourself from making fun of my geeky voice and focus instead on the awesomeness of Selenium, a fantastic web-application testing tool. I talk about what Selenium is, some features and uses, and speculate on possible future extensions.

The interview starts at the 12th minute or so. That said, I have learned a lot from these guys and think you’d benefit from giving their podcast a listen.

Check it out!

Related posts:

1. A quick way to backup a Firefox extension or profile
2. Firefox extension to watch: Firecookie
3. Cutting Edge: Exclusive Firefox 3 about:config hacks

In it, I explain a bit about how browser caching works, how setting headers can affect caching and what you can use in your .htaccess file (or Apache httpd.conf) to maximize it.

Go check it out!

Related posts:

1. Guest Post: Save Text Size Preference Using MooTools and PHP
2. Guest Post: change text size onclick with JavaScript
3. 8 steps to my personal Firefox setup for productivity

Why would a personal wiki make you productive?

In a phrase: to keep a your web of knowledge accessible from one place. You need a place to put your meeting agenda, important project/server links, and even a to-do or waiting-for list. Not only can you store a lot, you can tag it, search it and otherwise customize the crap out of Tiddlywiki.

You are not going to be able to remember everything you need, keeping links and information in email folders just doesn’t cut it. You can easily keep track of your meeting notes, DB schema diagrams, bug lists, blah blah blah…

In other news, great programmers Venkat Subramaniam and Mark Richards called Tiddlywiki their favorite technical tool. Favorite. As in more valuable than their IDE.

Roll your own… now!

It’s really stinkin’ easy! Just grab my favorite flavor of Tiddlywiki, d-cubed, and save it somewhere you can access easily. The first time you use it your browser will ask you to confirm that you can edit your HTML file through it (standard security check).

You will probably want to theme it so it fits you, and I found a bunch of nice themes at TiddlyThemes. It’s just CSS and JavaScript anyway, dowhatyouwant! whatyouwant!!

Once you have customized your wiki’s look-and-feel etc., it’s time to grab some slick plugins. Here are a few I use and recommend:

• Checkbox plugin – So you can manage your tasks with checkbox style
• Calendar plugin and Date plugin – Manage your calendar in your wiki!
• Your Search plugin – Really, really sweet search upgrade. A must-grab!

To install a plugin, just make a new Tiddler (wiki page) with the same title, content, and tags. So easy!

Next steps

I recommend starting by creating tiddlers for all your projects and just putting links to other resources in there. After awhile, copy bits of emails you need to remember to relevant places so you can search it anytime. Baby steps, but stick to it. The setup cost will be returned in cases (as in the beer others will buy you for rocking)!

You’re now well on your way to wiki-fying yourself. Share your experiences and have fun!

Related posts:

1. The unsung key to programmer success
2. What I wanted to know before I left college: A programmer reflects
3. 7 things you should do with Ubiquity instead

When it comes to productivity I can’t think of any application that has saved more time and frustration than Vim. Vim is the ultimate productivity tool for programmers. Most users of Vim that I know that have spent the time to learn it absolutely love it and couldn’t imagine a world without it. My only regret with Vim is not learning it sooner.

So what is Vim?

Vim is a modal text editor that is highly configurable, efficient, and lightweight. Some people refer to it as the programmer’s text editor because it has a plethora of features that make programming easier, although it could be used for other things like writing email and editing config files.

What makes Vim especially great is the shortcuts, commands, and the blistering speed at which you can edit text files since Vim has all the common editing tasks mapped to single keys on or near the home row on your keyboard.

Sounds great… but there is a catch:

Vim isn’t an editor designed to hold its users’ hands. It is a tool, the use of which must be learned.

Easing the learning curve

Vim is known for having a steep learning curve. I think Vim has this reputation because people tend to want to learn all of the features right away and quickly become overwhelmed.

Vim is a lot like Adobe Photoshop, you can learn just a few simple techniques in Photoshop that can make a world of a difference in the images you can create and ways you can manipulate them. Yet it may take you a lifetime to master all the functions Photoshop has to offer. Vim is the same way. You can learn only a few shortcuts and commands that will completely change the way you do text editing but may take years to learn them all (not that you would need to).

The great thing about Vim is everyone uses it differently. I have my favorite shortcuts that I use every day. Someone else might have a different set they use depending on what types of files they generally edit. Once you learn a few basics and get comfortable with the shortcuts and commands that you use every day you can learn another to increase your productivity.

Getting Started

I suggest going through vimtutor once right now and again after you have played around with Vim for a while. You can access vimtutor by typing in the command, ‘vimtutor’ for Linux users and by running vimtutor.bat in your Vim folder for Windows users.

Vimtutor has all the commands and how to use them as well as practice sections for all your basic editing needs. Don’t worry if you don’t remember all the commands when going through the vimtutor for the first time; however, it is important to do the exercises at the end of each section so you can get used to the way Vim works. It should only take about a half hour to make it through your first run through the tutorial.

You might consider bookmarking this article for your reference and checking out vimtutor now.

Making Vim Work for You

After using Vim for a while now I have a good idea which commands I use most often when doing web development, programming, and configuration editing. We will use this experience to look at different ways Vim can improve your programming and make it faster while trying not to overwhelm you with too many new commands. Grab a few of these, use them, and add more when you feel comfortable.

Vim modes

In Vim there are three major modes:

• Insert mode is like the Windows ‘notepad’. Simply type and text will output to the screen.
• Command mode is where Vim really shines. This mode is usually called "normal mode" since you will be using this mode a lot. To get into Command mode just hit ‘Esc’. When in command mode almost all letters numbers and keys on the keyboard are now shortcuts. The keys, ‘hjkl’ in command mode are now your "arrow keys".
• Visual mode is where you can highlight text.

If you ever made a mistake and hit the wrong key you can use ‘Esc‘ to get back to normal mode.

Commands by function

Note: As with other *nix tools, these commands are case sensitive.
You can use these commands in conjunction with another command or repeated x number of times by adding a digit to the beginning. For example, ‘4dd‘ would delete 4 lines. ‘dw‘ would delete a word.

Here are some of the most common vim commands and their meanings:

Open and save:

‘:e path/to/filename‘ – With Vim running, this will open the file you specify. This command can also be used to open remote files over SSH.
‘:w‘ – Will save the current file
‘:q‘ – Exits Vim

You can also use these commands together:

‘:wq‘ – Writes to the file and exits Vim

If you want to quit without saving your changes you can do

‘:q!‘ – Quit without saving

Moving Around:

To move your cursor around with vim you will use:

h – left
j – down
k – up
l – right

These movement keys are all located on the home row. It will take a little getting used to but you will thank yourself for not using arrows or the mouse when you realize much time is wasted moving your hands. I even find myself trying to use these keys in other programs and get frustrated when they don’t work. Sometimes I am pleasantly surprised when programs can be navigated with these keys.

Undo and Redo:

Of course you can’t have an editor without undo and redo.

u – undo the last change
U – undo all changes on that line. This is very useful when programming.
Ctrl + r – redo the last change

Copy and Paste:

You will probably find yourself wanting to copy and paste at some point. I will first explain how to copy and paste lines of text in the way that you are used to and then describe a few alternative ways.

To highlight the text you want to copy you will need to put Vim into visual mode. You can do this different ways.

v – go into visual mode and highlight with the Vim movement keys.
V – go into visual mode and highlight lines of text

Once you have the text highlighted you will then "yank" the text with the ‘y’ key.

y – yank (copy) text.

Once you have the text yanked (copied) you can "put" (paste) it with the ‘p’ key.

p – put/paste text.

So far Vim has failed to show any real improvements over the way you are probably used to copying and pasting text. y/p probably isn’t much faster than ctrl+c ctrl+v other than the fact that you don’t have to stretch your fingers to do it. Here are some improvements.

yy – copy current line. This is incredibly useful and fast when doing programming or any type of editing. When using this command in conjunction with, ‘p’ it will automatically add a line to make room. For example you can go to the desired line, hit ‘yy’ then ‘p’ and it will copy the line paste it below and move your cursor to the newly created line.

My most commonly used commands

Like I have said earlier Vim has more shortcuts than you would ever even want to know about but here are the golden ones. These are the commands that I use daily.

• ‘h,j,k,l‘ – The movement keys are essential
• ‘w,b‘ – move forward ‘w’ and backward ‘b’ one whole word.
• ‘dw‘ – delete a word
• ‘u,ctrl+r‘ – Undo and Redo are a must
• ‘dd‘ – This deletes the current line. I am constantly using this. It also puts the text into a buffer so you can put/paste it.
• ‘0,$‘ – The ‘0′ will put you at the beginning of the line and ‘$’ will put you at the end.
• ‘i,a‘ – The ‘i’ will put you in insert mode at the cursor. ‘a’ will put you in insert mode after the cursor. Think of it like append.
• ‘:52‘ – This will put you at line 52. When you have errors in your program on line X use this command.
• ‘o,O‘ – The ‘o’ will create a line below and put you in insert. ‘O’ does the opposite. It creates a line above. This doesn’t really seem all that useful because you can always hit the enter key if you are at the end of the line but you are not always in insert mode or at the end of the line when you decide to make another line.
• ‘f,F‘ – This is one of my favorite commands. ‘f’ will search forward in the line for the next character you type and ‘F’ will search backwards.

For example, lets say you have the following line of code:

print LOG "StateProvinceCode: $StateProvinceCode\n";

You could hit the ‘f‘ key then the ‘\‘ key and the cursor would be positioned right near the ‘n’ so I could change this to a tab character instead of a newline character.

This doesn’t mean it will keep on searching, there is a different key for that.

‘/‘ – Search. This is the key to initiate a search. You can use regular expressions or do a simple word search.

Special commands for programmers

Syntax highlighting — If for some reason Vim isn’t highlighting code by default you can turn it on by typing, ‘:syntax on‘ or add it to your .vimrc config file.
‘=%‘ – Will auto tab nested ifs/loops/functions. Although Vim automatically tabs everything out for you, sometimes you will run into old code or poorly tabbed code where you want to fix it up real quick. Just go to the first ‘{‘ in the code and hit this command and everything will be auto tabbed.
‘gd‘ – Jump to variable definition. The g stands for go. There are multiple ways you can use ‘g‘, ‘gg‘ will go to the beginning of the file and ‘G’ will go to the end. In this case we are using ‘gd’ to go to the variable definition. For example, lets say you are about a mile down your code and you have a function that calls the variable, $someOldVariable. To find out what is inside this variable or where it was first created you would move your cursor to that variable and hit, ‘gd’ this would auto-magically move you to where that variable was first seen, AKA the variable definition.
‘%‘ – Go to matching () [] {}. Sometimes you will have a very long loop and a bunch of nested functions. If your tabbing isn’t the best it is sometimes tricky to find out which bracket belongs to which function. The ‘%‘ will move your cursor to the start or end of the matching bracket.
‘>> and >‘ – The double arrow, ‘>>‘ will shift the current line over one tab. If you highlight a region of text you can shift the entire text over with ‘>‘.

Conclusion

Although there are literally hundreds of shortcuts and commands for Vim you will be far ahead of the regular text editor with just these few. If you simply learned the commands in this article alone you would program/edit much faster and be more productive.

Remember: learn just the basics first. Open a text file or project source file and move around and learn how to save it. Learn how to undo-redo and basic editing in insert modes. To really get the hang of Vim you will need to spend about 5 hours messing around in the editor.

Related posts:

1. awk is a beautiful tool
2. grep is a beautiful tool
3. Find is a beautiful tool

Tutorial

To best introduce awk I’d like to start with a practical example. Most of the applications for awk that I’ve dealt with involve formatting some output or data into something cleaner and more usable. This is certainly not the limit of awk, it is a full fledged language with all the power and responsibility to go with it.

Awk operates on one "record" at a time, which is each line by default. Each "field" in a record is separated by a space (by default) or another defined separator (using the -F option).

We’re going to print the file names, line-numbers, and function names of all duplicate functions so they’re super easy to find and remove. Suppose we have some output from grep with file names and line numbers using a command like this (pulled from my grep tutorial):

# Prints javascript functions like this - <file>:<line-num>:<line-content>
grep -EnH "^\s*function \w+" *.js | sort

That’s all good and well, but this doesn’t quite give us what we want in a clear manner, it prints all functions and information in a kinda cludgy fashion. We can clean this up with a bit of awk. First let’s learn a basic awk command:

awk '/'^$'/ { print "blank line" }' myfile

This snippet will print "blank line" for line that matches the regex: ^$ (a blank line). The pattern (between the two slashes, inclusive) is optional and we’ll see in the next example:

awk 'BEGIN { print "Hello, awk!" } { print $2, $1 } END { print "Goodbye, awk!"}' myfile

How this reads is: Before processing (BEGIN), print "Hello, awk!" and then print the second field ($2) and then the first ($1) for each line of myfile, then after processing (END) print "Goodbye, awk!". The BEGIN and END clauses are optional.

As explained above, a field is a sequence of non-whitespace characters. So if a line contained "foo bar other stuff", awk would print "bar foo".

One more thing before continuing: awk scripts can get ugly so it is useful to know you can read a file for awk commands:

#My awk file: foo.awk
BEGIN { print "Hello, awk!" }
{
    print $2, $1
    # This is an awk comment
}
END { print "Goodbye, awk!" }

#Invoke foo.awk
awk -f foo.awk myfile

OK, let’s add some power to that old grep command. We can use the -F option to specify our delimiter so let’s add to our grep command like so:

grep -Eoni "^\s*function \w+" *.js | awk -F ':' '{print $3," ",$1,$2}' | sort

Now we have a sorted list of function names followed by their filename and line number by spaces. We needed to print $3 first so that we could easily sort by the function name and look for duplicates, but we’re not going to do that manually… oh noooooo way.

Let’s make it a bit prettier and uniq-ify our list of functions by function:

# New part of our command
awk '{print $3,"line",$4,$2}' | uniq -f 3 -D

# Full command
grep -Eoni "^\s*function \w+" *.js | awk -F ':' '{print $3," ",$1,$2}' | sort | awk '{print $3,"line",$4,$2}' | uniq -f 3 -D

Here we pipe our last command back into awk printing <filename> line <line-num> <function-name> and then use uniq on the 3rd field (-f 3) showing only the duplicates (-D).

Other awk examples

#Backup all JavaScript files with a .bak extension -- replace 'bash' with your shell
 ls *.js | awk '{print "cp "$0" "$0".bak"}' | bash
#Print the number of lines that contain "function"
awk '/'function'/ {i = i + 1} END {print i}' myfile.js

Conclusion

AWK is admittedly a ten ton gorilla of a tool, so there is no way I could cover everything that it can do in one post. If there is enough demand I can write about some of the more advanced features like conditionals, variables, and formatting. Stay tuned!

Further Reading

To see how far this rabbit hole goes, you should check out a couple of my favorite references:

• awk, nawk, and gawk cheat sheet
• AWK – Wikipedia
• Hartigan’s AWK reference

Hope you found this informative and want to know more. I know you all probably have some awk gems to share, let’s see them!

Related posts:

1. grep is a beautiful tool
2. Vim is a beautiful tool
3. Find is a beautiful tool

Soon we’ll be looking at awk and other tools to continue the quest for command-line fluency. If you haven’t already, install Cygwin and check out part 1 and part 2.

Tutorial

One of the best ways to crank out code quickly is by using templates. Using the Stream EDitor, you can streamline the use of templates.

Suppose we have a template HTML file that we want to reuse often. Maybe it looks like this:

<html>
    <head>
        <title>template.html</title>
     </head>
    <body>
        <div id="nav">Navigation here</div>
        <div id="content">
%%CONTENT%%
        </div>
    </body>
</html>

Now we want to replace "%%CONTENT%%" with the contents of an HTML Fragment file. The syntax is simple: ‘/<pattern>/r’:

sed '/^%%CONTENT%%/r fragment.htmlf' template.html

The above script will append the contents of fragment.htmlf immediately after "%%CONTENT%%". So we can use the delete command to fix that:

sed -e '/^%%CONTENT%%/r fragment.htmlf' -e '/^%%CONTENT%%/d' template.html > whole.html

This might seem slightly useless, but the power here is in the simplicity. Many times I’m generating bits of Wiki code or HTML, and this has been invaluable.

OK, now for one more command: write (w). Suppose we have a CSV file that we want to split into several files depending on the value in the last cell. We could do this with grep, or awk (coming soon), but with sed we can do it with more efficiently:

#sedscript file
/,[0-9]+$/w numbers.csv
/,[A-Za-z]+/w letters.csv
/,[^A-Za-z0-9]+/w symbols.csv

sed -r -f sedscript original.csv

Now numbers.csv will contain all rows that the last cell containing numbers, and so on for letters.csv and symbols.csv. A neat application for this might be sorting your giant contacts list into different files based on some criteria. This is a simple example, but you can probably think of a more useful scenario where you’d want to filter and split a file.

Other Examples

#Print everything outside the <html> tag (check DOCTYPES)
sed '/<html>/,/</html>/d' myfile.html

#Convert rn (DOS) to UNIX n
sed 's/$//' myfile        #Windows
sed 's/.$//' myfile       #Linux/UNIX

Conclusion

You have now learned several sed commands and patterns you can use to make editing files and some searching tasks much more efficient, and even better, scriptable so they can be automated in a process. One cool application would be to get comments from a set of files and post them to a wiki. It would sure make collaboration slick, right?

Obviously, you can bookmark this stuff but you’ll really get good at it only if you just try it out. Keep sharing your experiences and command lists, they’re great!

Related posts:

1. Get sed savvy – part 2
2. Get sed savvy – part 1
3. awk is a beautiful tool