grep is a beautiful tool

Global Regular Expression Print is a staple of every command-line user’s toolbox. As with find, it derives a lot of power from being combined with other tools and can increase your productivity significantly.

Following is a simple tutorial that will help you realize the power of this simple and most useful command. If you are on Windows and haven’t already, download and install Cygwin. If you are also new to regular expressions (regex), here is a great regular expressions reference to get you started.

Tutorial

Suppose we want to search for duplicate functions in all of our JavaScript files. Let’s start basic and work up to it. This technique can be used to search for a TON of duplicate items like:

  • Duplicate HTML IDs
  • Check how many times a CSS class is used
  • Duplicate java classes
  • many, many more…
# Search JS files in this directory for "function"
grep function *.js

The above command will print the lines containing "function" in all JavaScript files in the current directory (NOT subdirectories). Printing out line contents would be much more helpful if we knew what files they come from and their line numbers:

# Print filenames, line #s, and lines that start with "(white space)function"
grep -EHn "^\s*(function \w+|\w+ \= function)" *.js

Depending on how you format your JavaScript files, something like this will omit comments, anonymous functions, and also words like "functionality" giving you better results.

# Print a list of: function <function-name> and sort it
grep -Eho "^\s*function \w+" *.js | sort

-o prints only the part that matches the regular expression. -E options gives me extended regex and -h suppresses printing of the file name. I am then piping to sort which just sorts the output so it a list of function <function-name>. If you don’t have a lot of files/functions to go through, you can just scan the list and then note the duplicate function names you see. Let’s go a step further for those that DO have a big list:

# Print only duplicate function names
grep -hEo "^\s*function \w+" *.js | sort | uniq -d

There we go! That will list only the duplcated functions. I know that we can expand this with awk or other stuff and get the file names and line numbers of the duplicates, but I don’t want to explaining the details of awk ;). I actually had it in this article and then removed it so leave a comment or contact me if you want the code for that.

Other Examples

# Count the number of functions in all JS files
grep -c function *.js

# Print lines that DO NOT have "function"
grep -v function *.js

# List processes that match "pidgin" (non-Windows)
ps -ef | grep pidgin

Conclusion

grep is one of the most used command-line tools, often piped to for filtering output. Understanding it is essential to increasing productivity on the command-line. There is so much more to grep than what I’ve shown here, and it would be cool to see your best uses in the comments!

Find is a beautiful tool

I have blogged before that knowledge of command-line tools is essential to take the next step in programming productivity. I think it would be useful to provide simple tutorials for these powerful tools, starting with find. I hope you agree, and would appreciate your feedback via the contact page or in the comments.

Tutorial

If you’re on Windows, I would recommend installing Cygwin to bring the power of a real shell to your OS. Let us start with a simple example and build upon it:

find . -name "*.css"

This will list all CSS files (and directories ending with ".css") under the current directory (represented by "."). We only want to match files so we’ll go ahead and change it to this:

find . -type f -name "*.css"

Now we will only match CSS files. Nothing special? Fine, I see how it is. Let’s find all CSS files that do something with your HTML ID #content next:

find . -name "*.css" -exec grep -l "#content" {} \;

Here we combine find with grep (covered in detail later) using the
We’re starting to get productive now, so let’s keep going. Suppose now we want to change every reference to the color #FF0000 (red) to #00FF00 (green). Normally you would have to have your editor search and replace them, if it even has that capability. Even then it’s slow, this statement is fast:

find . -name "*.css" -exec sed -i -r 's/#(FF0000|F00)\b/#0F0/' {} \;

Gasp! Wait a minute, I just searched for both ways to specify red and replaced it with green in my CSS!! How long would that have taken otherwise? Do you see now how you can code faster by automating it and combining powerful tools? Let’s look at some other cool search options find has to offer:

Other Examples

# find files changed in the last 1 day
find . -ctime -1 -type f 

#find files larger than 1 Mb in /tmp
find /tmp -size 1M -type f

#find files newer than main.css in ~/src
find ~/src -newer main.css

Conclusion

By itself, find is only as good as say… Google Desktop. The real power, as with other shell tools, is the ability to combine with other tools seamlessly. Effective use of tools like find very often make the difference between an average programmer and one that is 10x more effective (actual multiples up for debate).

These are just some of the basic features of find. Take advice from Chris Coyier and use your new power responsibly. Find is a beautiful tool.

8 steps to my personal Firefox setup for productivity

I need to give you a bit a background so you can see what my priorities are when setting up my favorite browser. I am a computer programmer that deals with web stuff as well as back-end stuff. I am NOT a designer in general, mostly a coder. I am running Firefox 3 beta 5 (not RC1 yet) since it is compatible with almost all of my extensions.

Firefox 3 top bar

Step 1: My Theme

NASA Night Launch - This has little effect on my productivity but I get asked about this more than anything else. I like the cool blackness and the fact that this theme is Firefox 3 compatible.

Step 2: Get Extensions

Firefox extension list

  • Adblock Filterset.G Updater - Update ad killer
  • Adblock Plus - Kill ads
  • All-in-One Sidebar - Really, really slick sidebar
  • Delicious Bookmarks - I use del.icio.us for bookmarking everything so having this is a must. I basically switched to FF3 once this extension was compatible.
  • DOM Inspector - For all those tricky userChrome.css questions you give me ;)
  • Firebug 1.2 Beta - In case you haven’t heard there is a new beta with some more cool features!
  • Firecookie 0.5beta - Firebug extension that is now essential for cookie debugging
  • FireGestures - Simple mouse gestures extension. Compatible with FF3 unlike many others
  • Fission - Not a productivity enhancer, just cool to put my progress bar behind my address bar like Safari
  • Keyconfig - Configure everything to be a simple key command away! OK, you got me, the latest keyconfig is not compatible with FF3b5. Wait, what’s this I made for you?
  • Live HTTP Headers - Analyze traffic. Useful to see exactly what is being POSTed and where.
  • Web Developer - Another staple for anyone doing web development work.
  • XUL/Migemo - Enhances search by allowing regex search with nice highlighting. Way better than I can describe in this one-liner.
  • YSlow - Web performance analytics tool that works through Firebug. Another must-have.

I also plan on using this extension when they are compatible with Firefox 3:

  • Google Browser Sync - Sync history, bookmarks, cookies, etc. across my many Firefox instances.

Step 3: Menu & Interface Customization

I right click and customize my top toolbar by dragging the back, forward, etc. buttons to the top, as well as the address bar and the search bar. I remove pretty much everything else and am left with only the Menu Bar and the Web Developer Toolbar on top.

I first configure All-in-One Sidebar by making basically everything except "Page Source" open in the sidebar. I leave the hotkeys at their default values and enable the thin sidebar switch. I put a few buttons for things on my sidebar but really I use hotkeys to access everything on the sidebar. Ctrl-B gets me my searchable del.icio.us bookmarks, Ctrl-H for searchable history etc. More on that in Step 7

Last but not least, my status bar with Firebug, YSlow and Greasemonkey sits on the bottom. I also use the XUL/Migemo extension to freshen up my search, allowing it to use regular expressions and highlighting the text just like Safari. A lot of you have specific questions about this, so keep ‘em coming by contacting me.

Step 4: userChrome.css

You have probably already seen most of my userChrome.css file posted in a couple earlier posts, you can just grab the file I use here (Right-click to Save Link As…). Here are instructions just in case ;)

Step 5: userContent.css

Firefox 3 about:blank

I am not going to post the thousands of lines of code in my userContent.css file, but you can download my personal userContent.css here (Right-click to Save Link As…). Allow me to give you the important bits - the titles are pretty self-explanatory:

Just like editing userChrome.css, and instructions are here.

Step 6: about:config

You might be surprised to learn that I don’t use all of the about:config entries that I have posted. Let’s go through the ones I DO use:

browser.cache.disk.capacity » 150000
browser.cache.offline.capacity » 20480
browser.download.manager.closeWhenDone » true
browser.search.openintab » true
browser.tabs.loadDivertedInBackground » true
browser.tabs.opentabfor.middleclick » false
browser.urlbar.hideGoButton » true
browser.urlbar.maxRichResults » 5
network.http.pipelining » true
network.http.proxy.pipelining » true
zoom.maxPercent » 400

I seem to remember there being more that I used with Firefox 2 but Firefox 3s interface just suits my needs more so I don’t have to do so much :)

Step 7: Special bookmarks

Lifehacker provided me with some really cool chrome bookmarks that I use to quickly get to pretty much any Firefox setting quickly. Let me give you some examples: Ctrl-L goes to the URL bar, then I type "opt" to access Firefox options. Similarly, I can type "gmonkey" to access my greasemonkey scripts. You can also set this up yourself to access Live HTTP Headers and friends. Cool, huh?

Step 8: Hotkeys etc.

The point of many of these tweaks is to make everything available with few keystrokes. Of course, I set hotkeys for pretty much everything. Another really important part of my setup is bookmark keywords. I use this alongside del.icio.us to make my browsing super-efficient. When you hit Ctrl-D to enter a bookmark, make sure you have a short "keyword" so that you only have to type "g" and hit Enter to go to Google.

Use all these things and I think you will be a master web browser. What do you do to optimize your browsing? Which extensions? Let’s hear it!