Since yesterday I've been looking for a simple and effective way to search the Netbeans source code as the need arises every now and then as I need to check for an example of how a particular API (like the HyperlinkProviderExt or the GSF API) is implemented in some existing project.

My first go at it was the nasty Windows search, which turned out to be as expected, nasty in its results. It doesn't search very well within files (I don't have any idea why it doesn't now, but it used to, maybe because I installed and removed that Windows Desktop Search thing). Anyway, it just sucked. Explorer hangs for a while anyway when you open the Netbeans source code folder and search, which takes ages (especially in this particular case where I was searching for text contained inside files).

There was ofcourse the "Find" feature of Netbeans. But I had even tried this before, opening all the Netbeans projects in NB sources just sort of crashes your system, as Netbeans takes hours to finish all the background scanning and compilation of newly opened projects (214520 items in 754 projects, totalling 1.6 GB! -- its a killer for Windows to handle).

I looked over at hg.netbeans.org/main, but the search feature there just searches the revision history and not really the actual code files. Then after a bit of googling I found out that there's even an OpenGrok instance up there for Netbeans at deadlock.netbeans.org/opengrok. I got all excited about it, but to my dismay even its search feature was non operational (reported).

I also tried out Google Code Search. It's really cool, it's fast, allows you to search through code all over the net, and even browse through it. As usual they have provided an API for it to integrate it into your own IDE, etc. Alas, it doesn't search CDDL code and there's no Netbeans.org results.. You might be interested to read a blog post by Geertjan I stumbled upon called "3 Lines of Code to Integrate Google Code Search in NetBeans IDE" which creates a plugin to search for highlighted code at Google Code Search.

There were other options available at this wiki article - FishEye, a CVSup mirror, etc, but most of those services have either not been updated since ages or are just not accessible.

I then got the bright idea to install OpenGrok on my system to get the work done. OpenGrok is a wicked fast source browser by and for the OpenSolaris.org community. It works pretty well and the installation was pretty straightforward (download, set config, run JAR), but (alas, there's always a but..), once installed, I ran opengrok to index my Netbeans sources, It ran fine for about 20 minutes (and made me so happy to see the HD activity LED flickering continuously, and the index data folder created by it getting populated with files, showing that the indexing is going on smooth), and then it stalled with a java exception (with the usual stack trace) telling me that it encountered an "Unexpected end of file". Yep, you guessed it right, I had no clue of finding out which file. So the OpenGrok story got over there and then and I deleted it.

I had emailed Jesse Glick about the deadlock.netbeans.org/opengrok search problem and he promptly replied that the installation was broken and wasn't being maintained. He took down the link from the wiki and also gave a suggestion to use the "hg loc" (hg = Mercurial for the Kalahari desert folks) command along with "xargs" to find any code that I want in a jiffy! Thanks Jesse!

I immediately switched over to Solaris (goodbye windows, yet again. I've been moving between them alot lately, due to some or the other lack in either) to try out the command. I've created a script which uses the hg loc as well as the find command (2 alternatives) for doing the same thing. hg loc is much faster if you have a built source tree since it will not waste time traversing build products. Also, for rarer filename patterns (e.g. '**Handler*.java'), hg loc will be much faster because it looks for the filename itself directly in the Hg manifest, not by doing a tree traversal. Anyway just to be sure I've got both in the same script:

#!/bin/sh
cd main
echo Searching for $2
if test $1 -eq 1
then
        echo using "find"
	find . -name "*.java" | xargs grep $2
else
	if test $1 -eq 2
	then
	echo using "hg loc"
	hg loc -r tip '*.java' | xargs grep $2
	fi
fi

Please let me know if you know and use some other wicked cool way of searching your code!

Update: Just got a suggestion over at nbdev, by Jaromir Uhrik on how to use the "Find" feature of Netbeans without crashing your system (not literally crashing, but yes it becomes unstable due to all the processing involved in opening 754 projects): Press Ctrl+3 to bring up the Favorites window, right-click to add to favorites the "main" folder of the repository, and then use the find tool on this folder. Cool, but way slower than hg loc.

Update 2: Tip by antonio: "locate" is faster than find, as it uses an internal database that is updated periodically.". So use locate instead of find like so: locate *.java | xargs grep 'code to search'

Technorati Tags: source-code, opensource, netbeans, opengrok, google, mercurial, shell-scripting, solaris, java