chandanlog(3C): OpenGrok

15 Nov 2005 OpenGrok

We are releasing OpenGrok, which is a fast and usable source code search and cross reference engine, written in Java.

It helps you search, cross-reference and navigate your source tree. It can understand various program file formats and version control histories like SCCS, RCS and CVS.

OpenGrok is the tool that powers the OpenSolaris Source Browser. It is now open sourced and available for download and use.

Features include:

It can search program symbols, definitions, full text, history log comments (like SCCS, CVS, RCS log), file path; and the ability to AND or OR or NOT any of the search clauses. It has a simple query syntax familiar to users of most web search engines.
It can show hyper-texted and color coded source listing, with symbols cross-referenced to their definitions
For the above searches and cross-references, it can understand various program formats, like C/C++,Java, Shell Scripts like ksh, bash, Perl, Makefiles, XML/HTML/SGML files, ELF files, Java class files, Java Jar files, archive files like TAR, GZip, BZip2, Zip files, man page formats like troff and more. You can easily write a plugin to analyze your own program format.
It can understand version control systems like SCCS, RCS and CVS and provides online user interface to see the history log (either at file or directory level), differences between any two revisions.
Look and feel is customizable using CSS

It aims to be an universal source code search engine, with online browsing for any version control system, which is both usable and fast.
More about the features and usage is in my entry describing the OpenSolaris source browser, John Stearns' article in developer.sun.net and in OpenGrok website (source + downloads).

Link | Comments [16]

Comments:

Great!

However... I think I found a bug :-)

(Is there a better place to report this? I couldn't find any contact information...)

Posted by Kate on November 16, 2005 at 10:00 AM PST #

Ah., I must watch for \\ ! Send bugs, feedback or ideas to opengrok (at] sun.com

Posted by Chandan on November 16, 2005 at 10:12 AM PST #

I can't get it to work! I just tried with a jdk 1.6.0 build, all I got was this:

% java -jar opengrok.jar -c ..\ctags554\ctags -s ..\mustang\j2se ..\mustang_data
Processing j2se
Optimizing the index ... done
Generating spelling suggestion index ... done

Now there's plently of java files in there (and native too) but I get nothing when i try to list the index, or search.

Posted by Sam M. on November 20, 2005 at 12:39 AM PST #

Never mind, i found out SRC_ROOT needs to be the complete path, not relative.

Posted by Sam M. on November 20, 2005 at 03:46 AM PST #

It ignores symbolic links. (if your top level directory happens to be a symbolic link, then it does nothing). The SRC_ROOT in webapp should be complete path, for tomcat to understand it, but relative path to -s should have worked.
You have forward slash in your path (windows?) I had never tested it on windows.

Posted by Chandan on November 20, 2005 at 09:10 AM PST #

This is exactly the type of tool I am intersted in for a project at work. However, the primary language we will be doing a lot of our development in is in MATLAB (it's a research group), which is not natively supported by CTAGS. I noticed that you can write 'plugins' for your own program formats...which I am assuming opens the door to languages that are not supported by default. I briefly looked on the CTAGS site, and did not find instructions on how to do this...could you let me know where to go for instructions/examples of how to customize OpenGrok to languages that do not fall under CTAGS' standard set (i.e.-MATLAB)?

Posted by Tom on November 26, 2005 at 07:04 AM PST #

You can add the following to your ~/.ctags file:

--langdef=matlab
--langmap=matlab:.m
--regex-matlab=/^function[ \t]*[a-zA-Z0-9_]+[ \t]*=[ \t]*([a-zA-Z0-9_]+)/\1/f,function/
--regex-matlab=/^function[ \t]*([a-zA-Z0-9_]+)[^=]*$/\1/f,function/
--regex-matlab=/^function[ \t]*\[.*\][ \t]*=[ \t]*([a-zA-Z0-9_]+)/\1/f,function/

This will enable searching for definitions within matlab files. To hypertext-cross reference the matlab files, a new analyzer needs to be written which extends org/opensolaris/opengrok/analysis/plain/PlainAnalyzer and handles % comments. Will add this to todo list.

Posted by Chandan on November 29, 2005 at 02:25 PM PST #

Hi, I installed opengrok on Windows, the indexing works as designed. I deployed the webapp on Tomcat 5.5 on /xref and I'm able to browse the source code too. However, if I click on an identifier, the link goes to http://localhost:8080/source/s?defs=handle_t I get a Tomcat 404 error. The webapp itself is deployed in /xref, but the link goes back to / obviously. And there is no sign of a "source" file within the webapp xml config file. What could I be missing here?

Posted by Fabian R. on November 30, 2005 at 04:21 AM PST #

Followup: I deployed source.war to /source and the link above works now flawlessly. However, all path related stuff is not working properly (navigation, search for files) which I blame to the code not written with windows backward slashes in mind. Thanks anyway.

Posted by Fabian R. on November 30, 2005 at 06:50 AM PST #

I had never tried to run it on windows, (are there developers who use windows ;-)) but figured out that making it work on windows is easy. Windows support is coming soon in next release along with few bug fixes (I am planning to release in couple of days)
The webapp context was hard coded to /source, though I have eliminated most of those usages, a few places still remain and might require a new option to opengrok.jar application.

Posted by Chandan on November 30, 2005 at 07:31 AM PST #

Thanks for your reply. Yeah don't tell me about Windows... For a quick test of opengrok, a Windows local admin has all you need to get Tomcat up and running within minutes... I guess I need to talk to the admins of our unix/linux boxes :) I'll watch the opengrok page for a new release in a week or so.

Posted by Fabian R. on November 30, 2005 at 08:28 AM PST #

tomcat does not require root acess to run, Running it is as easy as untar, and run bin/startup.sh and you get tomcat on localhost:8080

Posted by Chandan on November 30, 2005 at 08:49 AM PST #

This is fantastic - thanks! Navigation to definitions in C# code could be better but I love it!

Posted by Graeme Foster on December 09, 2005 at 04:11 AM PST #

Any idea/suggestions that you can give regarding deploying opengrok w.r.t website hosting. Being a webmaster I want to install Opengrok in my website : * Deploying a Opengrok's war file is easy & fine * But how do I populate the data as there will be limited or less access to users to run scripts/applications in most of the hosting websites. If possible can you provide some pointers w.r.t hosted websites using Opengrok. I think it helps lots of community websites to share the code with their members. Looking forward for your help. Thanks, nanda kishor www.c4swimmers.net

Posted by nandak on April 08, 2007 at 11:09 PM PDT #

OpenGrok now supports the concept of projects. Users could have the ability to just upload/manage thier project directories. The opengrok indexer is run by the webmaster in a cronjob, say hourly, so users do not need to run any scripts.

Posted by Chandan on April 09, 2007 at 01:25 PM PDT #

I'm using OpenGrok to index C++ code. I notice that if I search for a definition, OpenGrok seems to be simply listing all lines that contain the identifier . Clearly, this includes the definition but also a lot of other lines. Is there a way to just have the definition listed in the results?

Posted by pksh on March 31, 2008 at 10:56 AM PDT #