Friday May 25, 2007

One of the things I did this week was to rewrite the code which extracts documentation for all the core Ruby APIs (which are defined in the C Ruby implementation). This fixes some bugs where a method here and there were missing its documentation.

In doing so, I started looking at the actual IDE presentation of the documentation as well, and thought I could make it a lot more appealing. Without further ado, here's how the documentation looks now:

The most interesting part here is that the IDE now figures out whether portions of the documentation corresponds to Ruby code, and then presents it with syntax highlighting right there in the documentation popup. (I've also improved the way the method signatures are displayed so it should use less space and be more readable now, although that's harder to spot than if you compare before-and-after shots.)

Heres another example. This shows a migration in a Rails application, where you're getting code completion on the create_table method.

One interesting aspect of this highlighting is that it's easier to spot mistakes in your comments. As I was testing this feature, I pulled up the documentation on the Rails send_file method - and the following code fragment looked wrong:

Sure enough, it looks like it's missing a single-quote right after 404.html. This is the kind of thing which will be a lot easier to spot and fix with the new enhanced highlighting!

(P.S.2: If you're trying to download this, see my other blog entry on the current temporary download location.)

Updated Monday May 28: The machine is back up and running (and much faster than before). You can grab up to date builds from the old location again (and the wik installationi page points back to it again). The latest builds have both a fix for the RHTML problem as well as support for encoding properties on projects, so the old file.encoding=UTF-8 system property workaround should no longer be necessary.

The deadlock.nbextras.org machine, which I've been pointing to regularly as the best place to get up to date builds of the Ruby support, is off-line. It is being replaced by a brand new machine, much faster and more reliable, and should be back up on-line on Monday (May 28th).

Apologies to those of you relying on deadlock.nbextras.org for regular updates. The service should be getting a lot better from now on. The reason it's taking several days to perform the migration is that this is a volunteer-driven effort, so there's a bit of cooperation necessary to get the machine picked up at the ISP, the disks mirrored over to the new machine and the machine taken back in to the ISP and get back up and running.

In the meantime, I've set up a temporary location where you can get up to date bits. The instructions and URL will be kept up to date on this wiki page.

Friday May 18, 2007

First the screenshot - click for full resolution:

The NetBeans Ruby support now lets you debug Rails applications. As the screenshot shows, you can even step right through RHTML files! All the usual debugging features are there - breakpoints, stack view, local variables view - and balloon evaluation, where you hover the mouse over a variable and it displays the current value as a tooltip.

I can't take any credit for this; the debugger has been written by Martin Krauskopf, and uses the debug-commons library. Martin was my co-speaker for the Ruby talk (TS-9972) at JavaOne. If you missed it, Brian Leonard will be doing the same talk at RailsConf this weekend. One thing I want to clarify is that not everything I talked about in the type inference section is actually implemented yet in NetBeans. The talk represents everything I plan to do, not everything I've already done.

Note that the RHTML debugging feature is not present in the recent Milestone 9 bits. We've made a lot of improvements since then. Just this morning I integrated various fixes to make code completion work much better in Rails. You now properly get code completion in your model and migration files etc - as the following screenshot shows:

Another feature we added after Milestone 9 was some preliminary support for Auto Test - see the testing page for some screenshots and more info.

For now, I recommend that you use NetBeans Milestone 9 as your base IDE, and then use the trunk/daily Ruby bits with them. The easiest way to do that is to install Milestone 9, then follow the instructions in the RubyInstallation wiki page to blow away the ruby1 cluster in your M9 installation and replace it with a .zip download from deadlock.nbextras.org. Also be sure to nuke your userdir for any custom installed modules there (usually ~/.netbeans/dev). In the future, staying up to date with the Tools | Plugins manager will be easy, but with Milestone 9, it only displays branched Milestone 9 modules, and we aren't merging and rebuilding the Ruby stuff into the M9 branch at this point.

If all that sounded complicated, just go and download the Ruby-only IDE zip here. Get the zip containing rubyide in the name, unzip somewhere, and run nbrubyide/bin/nbrubyide. This will be daily-everything as opposed to a stable IDE + trunk Ruby, but - it will probably be okay.

Finally, if you're going to be debugging, make sure you first visit the RubyDebugging wiki page which will tell you some really important configuration information - like how to enable fast debugging.

Saturday April 21, 2007

I've heard from a number of people that the ability to jump to a file by type (Open Type), or by filename (Open File) are really important to them. To the point of being adoption blockers.

Both are in the works.

NetBeans has always had a Go To Type for Java (Alt/Ctrl-Shift-O), which we're now retrofitting to become a generic Goto Type facility that other languages can plug into. That means introducing a new API etc. which requires some more effort - so this isn't there for Milestone 9. However, in the meantime, I've taken the early work on that and hacked it into a Ruby-only module which will let you jump to Ruby types. It's there in the editor context menu (under Go To), bound to a different shortcut. Unlike the general Open Type which will work from any file (or even when you're in a non-source editor window), this will only work from Ruby files. You can get the module here. That's an NBM file - a netbeans module bundle - which you can install via Tools | Plugins - just switch to the Downloaded tab and point to your NBM. Click Add Plugins. (Sorry, the modules are not signed so you'll get a warning.) It will require a restart. Also, the revamped Plugin Manager is brand new and is stabilizing right now - you may run into some issues. One thing to try is to install one module at a time. You need brand new Ruby bits for this too - go to nbextras.org.

Here's a screenshot of what it looks like (although this includes Java type matches, which the provided hacked module won't do) :

The ability to jump to any file based only on the filename (not the path) is tracked in issue 44586, and there's been some work happening in a branch. Even though it's not done, you may find it useful in your day to day work. To get it, you can download the module here. Once done, you'll have a Goto File In Project item in your Navigate menu. The operation is faster after the first time.

Finally, I've updated the TextMate snippet importer I recently discussed with some bug fixes - get the latest version here.

Friday April 20, 2007

SapphireSteel, makers of the Ruby In Steel plugin for Microsoft Visual Studio, has been touting their own code completion feature for a while. One thing which has irked me a bit is that they tend to dismiss "code completion" as inferior to "true IntelliSense" - as if code completion is the name for simple identifier matching. Just to be clear - IntelliSense is a trademark of Microsoft, so nobody else calls the same feature that. Borland has referred to it as CodeInsight, but most IDEs refer to this feature as code completion.

In this article, SapphireSteel is throwing down the gauntlet by encouraging their users to see if anyone else can produce results like theirs:

To the best of our knowledge, Ruby In Steel’s IntelliSense for Ruby is unrivalled by any other IDE. {...} We’ve also supplied a few files from our IntelliSense test suite which we invite you to use with Ruby In Steel and any other editor or IDE of your choice.

So, I thought I would show what code completion in NetBeans is capable of. Let's start with test 1, which calls for code completion on an array literal in the source:

As you can see, we show the type of methods right in the code completion dialog. It doesn't matter much for an Array, but if you had invoked code completion on the numbers themselves, you'd see that some methods are coming from Integer, and some are coming from its parent class Numeric - and sometimes it's interesting to know that. Next, let's look at test 2:

The above example shows the type analyzer tracking types of variables. More interestingly, notice that while we show the methods on the class "above the line", there are other methods there too. Those are methods inherited into MyClass. In the SapphireSteel screenshots, these methods were turned off, presumably because in Visual Studio they would be interleaved with the regular methods. The evaluation document describes how you can turn Object inheritance on and off in the options. I think our solution is nicer because you get the best of both worlds: the methods you are likely most interested in appear first, but when you ned the "standard" methods (they are there for a reason, after all), they also appear. Thus, if you have a String object and you're completing on the letter "e", you see "empty?" (a String method), but also "eval" (an Object method).

Next, test 3 which tests module mixins:

Looks fine - and so does test 4 which is a more complicated scope test:

Anyway, I don't want to imply that our code completion is perfect! I still have a lot of things I want to do in our type inference engine. In particular, I want to make sure that code completion works well where it really matters: In code you're writing. Here's a final screenshot from a Rails application where you're writing a controller. Code completion works on important methods like render, or a method I found myself calling the other day - send_file:

Rails does a number of things that are very tricky to handle for the IDE, so I'll be spending more time in this area to improve code completion for Rails apps.

P.S. To try it yourself, get bits from nbextras.org. As of yesterday we're in Milestone 9 stabilization, so a promoted build should be available within the next couple of weeks.

Tuesday April 10, 2007

Code completion now inserts the parameters for the method you select. They are inserted as a live code template such that you can jump from one parameter to the next using Tab or Return. More importantly, the parameters are also displayed as a tooltip over the method call. You can invoke this whenever you're typing in code - not just using code completion. Hit Alt-P (or on the Mac, Ctrl-P) and if the IDE can figure out the method signature of the method call surrounding the caret, it will display the parameter list, and show the current parameter in bold. As you move the caret around, the tooltip will be updated to show which parameter you're currently editing.

Messing with the user's editing workflow is always risky, so please let me know how you feel the new code completion behavior works. The parameter tooltip depends on being able to figure out the signature of the surrounding method call, which depends on type inference, so it may not always work - but it will also improve as I tweak and improve type analysis across code completion, goto declaration, etc.

Also, there is also a Go To Test action now for jumping quickly between a test class and its unit test. Should work in Rails, RSpec, ZenTest, and also between any classes named Foo, FooTest or TestFoo regardless of which files they're in. This action is in the editor context menu, and is bound to Ctrl-Shift-E. To get this stuff, get at least version 0.65 of the Ruby Feature module (see these installation instructions).

P.S. You can find some more information on the Ruby support in an interview I did with Geertjan Wielenga a couple of weeks ago. There are some other interviews in the same series talking about the new language support in NetBeans. The Rails support in NetBeans will soon have improved support for JavaScript, YAML, and RHTML - stay tuned!

Thursday April 05, 2007

I've received several requests for support for TextMate-style snippets. TextMate is a very popular editor among Ruby developers, and a lot of its power comes from its code snippet functionality. Snippets are basically identical to what NetBeans calls "Live Code Templates". The code templates are live in the sense that when you insert the template, you're in a mode where you can jump from one logical section of the template to another, and edits in one section can automatically be updated in other sections.

I've just implemented an import mechanism for TextMate bundles. Point at your own bundles (or even the ones distributed with TextMate, although I don't know what the legal issues are with that), and choose import. This will convert all* the snippets into NetBeans Live Code Templates (*=for some values of all).

There are two features in TextMate snippets that are not supported:

• Command execution. A snippet can contain a `shell command` (in backquotes), and at template evaluation time, the command will be executed and the result inserted. In the snippet bundles I looked at, this was used only for a single thing: inserting parentheses on a conditional basis. So I just made the import handle parentheses directly instead.
• Regular expression replacement. A template can perform regular expression replacement on different instances of the parameters. I can't even tell exactly how this is supposed to work, but for now, I skip snippets that do this.

TextMate is essentially a text editor; it doesn't (as far as I can tell, having never used it but having read their snippets and macros in detail) have any real semantic information about the code. Therefore, doing complicated text processing with regular expressions is the Way of TextMate to achieve logical code operations. Live Code Templates on the other hand can take advantage of all the information the IDE has about the code. I've added a bunch of logical code template variables you can use in templates. Rather than try to reproduce the regular expression operations of TextMate's snippets engine, I would like to create logical Ruby operations for the code templates and use those instead.

Here are some examples of what is possible with this mechanism. None of the imported snippets use these variables, but the imported snippets are useful in their own right - especially for those of you with muscle memory.

${unusedlocal defaults="i,j,k,x,y,foo,bar"}

First, we create the template, naming it "dob":

Then, in some method I want a do-block. I just type "dob" and hit Tab, and voila:

It inserts a do-end block. When it comes to the block variable, it has looked at the context and discovered that the first two preferred alternatives, i and j, are both taken - so it chooses k. This shows the advantage of using semantically aware code templates - you don't accidentally alias an existing variable. This wouldn't be a problem in Java, where it would complain that your new variable already exists in scope, but in Ruby, this is valid (and sometimes, but not usually, what you want.)

As another example, here's a silly template which spits out a hash that defines a bunch of local context:

Let's say I have the following silly file, and type "ctx":

If I now hit tab, I get this:

I will now be able to walk through (with the tab key) each of the variable sections and edit them. For an example like this, it's likely that we don't want these things to be edited by the user - so just add editable=false on each such variable (e.g. ${class editable=false}) and they will be processed but not edited by the user.

One of the things that makes this really powerful, is that it uses a lot of the context information the IDE has about your program. In this example, computing the super class of the current class isn't that hard - and a fancy Ruby shell script executed might be able to pull it off. But if I was simply redefining an existing class, let's say Integer, and there is no local reference to Integer's parent class, the live code template will spit out Numeric. Yes, it looks in the index.

Call for Help

As I've shown, the basic machinery is there now, but what we really need is a set of code templates to be bundled with the IDE. Some will be simple substitutions or linked variables, but we should be able to come up with some much stronger ones. Please let me know what kinds of logical parameters you can take advantage of. For example, perhaps a parameter which evaluates to the require statements necessary for the classes referenced in the file? Or a parameter which looks for database table names in the database? Perhaps some simple transformations which produce a constant name or a method name (upcasing and downcasing) from some other symbol? There are a lot of possibilities here, but since I'm not a real Ruby programmer I'm not the best person to dream this stuff up. If you can look at snippets you have written, import them into NetBeans, optionally edit them to add more semantic analysis if appropriate, and contribute them, I'd be very thankful!

Also, I have never used TextMate, so I would appreciate feedback on whether the templates work the way they do in TextMate, or if I misunderstood something from just reading the snippets themselves. Bug reports (preferably via the issue tracker) are much appreciated.

So, where can you get this stuff? I just integrated everything, but the Live Code Template management is not in stock NetBeans; it's in an optional download called "CodeTemplateTools" (in contrib/CodeTemplateTools), written by Sandip Chitale. More info about this module is here. I have built an NBM you can install directly - just download the module bundle file, then install it via the IDE via the Update Center (in the first panel choose manually downloaded NBM). Note however that you must have the other IDE fixes as well; they should be live on deadlock.nbextras.org soon but probably won't hit the Update Center for a couple of days. (I'll try to make the slim IDE bundle the tools). As always, see the wiki for installation instructions. Once you have installed the Code template tools, look for "Show Templates..." in the Edit menu. The import UI is still a bit rough - no progress bar etc - I'll worry about that later. There's one more rough spot: the "|" character must be escaped (by typing it twice) because historically, NetBeans used "|" to indicate the cursor position. There is the logical variable ${cursor} for that now, but there's still some issues we need to resolve in this area.

Wednesday March 28, 2007

Since I cheated last week, I'll make up for it by posting several screenshots this week.

First: The continuous builds on deadlock.nbextras.org are back up and running. One of the things you'll find there is a Ruby-only slimmed down version of NetBeans. This might be attractive for those of you who are used to editing Ruby and think of IDEs as overkill. The download is not tiny, but that's because it includes a full installation of JRuby as well as Ruby On Rails and all its supporting libraries. Perhaps we can create a configuration which doesn't JRuby and Rails, since the tool is also perfectly happy to use any existing installations of Ruby and Rails you might point it at.

The configuration is experimental, so menu items etc. might not always be ordered correctly. But if you're a Ruby-only programmer, you'll find it less crowded than the full-featured IDE, and a faster download.

I've tried to improve the Rails support lately, since I've gotten lots of feedback from developers using NetBeans+Ruby with Rails.

Rails projects now have a "Rake Target" action, which is a pull right menu. The pull right shows, hierarchically, all the rake targets available for the project. The target descriptions, if any, are shown as tooltips. Recently executed targets are listed before the hierarchical list of available targets. (Determining the set of targets for the UI is expensive, so it is not kept up to date automatically. There is therefore an explicit user action for refreshing the list when you have edited your Rakefiles or have installed software which adds available targets.)

Database migrations are supported in a similar way. There is a "Migrate Database" context menu (as you can see above). It's a pull right menu which lets you choose either to migrate to the current version, or a specific version (the IDE lists versions that it finds in the migrations directory). You can also "run" any of your migrations files (Shift-F6), and this will run the corresponding migrations target with the version number of the migrations file.

Rails supports lots of additional code generators, performing tasks from generating login pages to localizations and sparklines. The Generator action is now fully aware of generator plugins. They show up in the generator drop down, and usage information is displayed for all generators right there, inlined in the generator dialog. In addition, there is an action for quickly installing more code generators in the Gem Manager.

I just integrated support for running the Mongrel web server, which is popular with many Rails developers. All you need to do now is install Mongrel via the Gem Manager and run your application.

Finally, it's been a few weeks since I added it, but it's useful enough to point out in case you missed it: There is a command for jumping from actions to views and back (Ctrl-Shift-View). It's in the Go To context menu as well.

Saturday March 24, 2007

Several people have pinged me about this so I thought I would post a blog entry on it:

The machine which was running nbextras.org is down because of hardware failure. It's at home with Trung now who's fixing it. Not sure when it will go back to the ISP and be back online.

The Ruby IDE Installation Instructions refer to kits from deadlock.nbextras.org. You'll have to use the other option for now - the Daily Update Center. It's lagging a bit behind the most recently integrated changes, which unfortunately means it doesn't have the latest and greatest.

I've uploaded a custom build of version 0.45 here (9.3 Mb). It's the contents within the ruby1 cluster. This means that you unzip this content in your user directory (if you've been installing the Ruby support from the update center in the past), or inside the ruby1 cluster in your IDE installation (if you've been downloading full kits from nbextras.org), or inside the extra/ cluster (if you're using an older version of the IDE where ruby1 isn't in the included clusters list, or if you were installing into a separate cluster directory referenced from your netbeans.conf). (By the time you reach this, the Daily Update center may have caught up, so check that option first. The version number on the update center is displayed in the dialog when you select the Ruby feature module.)

One of the new features you'll find (added yesterday) is an enhancement to the occurrences highlighting. Now, if you place the cursor on the declaration line for a method, the exit points for the method are highlighted: the last statement, any return statements, any yield statements, and any raise or fail calls.

Wednesday March 21, 2007

I just realized last night that the session slides for my Java One talk are due today! So I did a marathon session, starting at 6am this morning, putting it all together. I just submitted the slides, with a whole 4 minutes to spare before the midnight deadline. That's 18 hours straight with just a short dinner break. Phew.

So I'm going to cheat for this week's Ruby screenshot. Take a look at the following picture; it shows two new NetBeans 6 features applied to Ruby: Local editing file history (with version diffing), and the brand new diff view. I've long thought that the Mac OSX XCode filemerging window reigned supreme, but with the new smooth spline curves and even character diffing within lines, I have a new favorite! This file is not under version control - the local file history feature tracks local edits and lets you diff auto-saved versions.

Full size

I know this is cheating since all I did was bring up a Ruby editor on non-Ruby related functionality. But there have actually been some great improvements in the Ruby support in the last week. First, the Ruby debugger support has landed! Second, native Ruby execution should now be working finally on Unix, including with Rails. There are some other changes too (plus some pending one I'm about to check in). Check the wiki for installation instructions - to get fast debugging you'll need to perform some manual steps.

Zzzz

Wednesday March 14, 2007

I have added a number of Ruby Documents to the NetBeans wiki. You can now find a feature list, as well as instructions for both how to download and install the Ruby support as well as how to build it from scratch. And of course, the todo list!

Perhaps most important to those of you using the Ruby builds is the Recent Changes document, where I try to summarize important changes that are checked in. As you can see, in the last week, the long startup indexing delays are gone, and code completion is smarter - it now tracks types of variables and fields in local scope - plus it better handles singleton classes.

If you haven't tried the Ruby support yet, the Installation document might be helpful.

Monday March 05, 2007

Roman has created two excellent demos of the new NetBeans + Ruby support. The first one shows Rails support, the second gets into editing. Let me me point out that until now, my focus has been on editing - so the Rails support is very preliminary. The interesting features are in the editing area, but as the Rails demo shows, if you're building a Rails application you should be able to get your work done. (There is an open bug pertaining to process management - launched Ruby processes aren't always killed by the IDE, so keep an eye on your process list for older processes. Yes, it will be fixed!)

Here's a quick hint, since it's not really obvious from the IDE: Use Shift F6 early and often. It "runs" the current file. For a unit test, that will run the tests. In a Rails app, it will open the browser on the "relevant" URL. For example, if you're editing a controller, or a view, or a helper, it will show the associated view.

I recently added a new feature to the editor you can play with: Smart Selection. Press Ctrl-Shift-Up/Down (on the Mac, Command-Shift up/down). It will select progressively larger code blocks based on the parse tree. Within a comment, it first selects the line, then the whole comment block.

Finally, I just integrated a Ruby Gem Manager (available from the Tools menu). This lets you easily see which Ruby gems you have installed, and more importantly upgrade them or install new ones. This assumes there is a gem command next to your Ruby interpreter, which is the case for the bundled JRuby, but maybe not if you're configured NetBeans to run with whatever you have installed in /usr/.

Here's a screenshot of the initial screen - my locally installed gems:

Here's what you get when you click on Install New:

Actually, if you just press Install New you see a much larger list - all available gems. You can filter down the list by typing in the Filter field - regular expressions are okay. Here I've typed rails to see any gems that have "rails" in the name or description.

I'm writing this from San Francisco international airport - I'm waiting for my flight out to Denver for the Posse Roundup 2007. It's going to be a fantastic week - technical talks, skiing, and of course I'm excited to see the unveiling of Josh Marinacci and Robert Cooper's project. With all the attention around the Ruby work this week it's bad timing to go off the grid, but at least I can leave by pointing out that this is an open source project, all the code is available and if there's a fatal bug in there, somebody can find it and fix it! Have a great week - I know I will.

P.S. I've seen lots of blog entries around the NetBeans Ruby support lately - thank you all for trying it out and writing about it. I found this Ruby IDE comparison particularly interesting.

Tuesday February 27, 2007

Tim Bray just posted an entry on his first experiments with the Ruby and NetBeans support and in particular, talks about the syntax highlighting. I was already planning on spending this week's screenshot on the highlighting features, so I'll use Tim's ape.rb source code and show you a few more details:

Notice the following:

• Notice how unused local variables are highlighted (see dummy which I inserted to make a point - that's not Tim's fault!).
• Notice how the symbol under the caret gets highlighted along with all other uses of the same symbol! This is pretty useful when you're dealing with blocks for example - it will show you that a local variable may be used unintentionally as a block variable and will get modified as a side effect.
• Notice how methods calls are highlighted in bold. This is handy when you have a typo, and what you thought was a local variable read is actually a call to an unbound method.
• Notice how strings are lexed too. Here you can see that it's telling you (in red italics) "\n" is not expected as an escape code - this is a single quoted String after all.
• Embedded Ruby code within Strings - #{ } - is properly shown as Ruby code. Occurrences highlighting and code completion works in these sections too.

In doing this screenshot I noticed a couple of bugs I'm off to fix now!

Tuesday February 20, 2007

If you try code completion on any complicated arbitrary expression, it will show you a number of possible method completions along with the corresponding class. This gives you much the same information as ri:

However, in many contexts, we know the exact type. For example, if you type File. and invoke code completion, NetBeans will show you only the singleton methods of class File (as well as inherited methods and module mixins):

In some cases, we know more than just the class - we know that we're dealing with an instance of that object. In that case we can show all the instance methods too - not just the class methods:

Here's we're applying code completion on a regular expression literal.

This works for all kinds of literals in your code - Strings, regular expressions (shown above), arrays (try [1,2,3].e to see the each method and its documentation for example), hashes, symbols, numbers, ... you can even try it on the "nil" keyword!

Friday February 02, 2007

Yes, the code is now in CVS. Please note that this is under active development. There are known bugs and instability. This is for early adopters only! As things get more stable we'll be making more official builds for a broader audience. I'll post a feature list here soon so you know what you can play with.

Did I mention that this may be unstable? Okay, good. I've posted screenshots here before, but remember that screenshots will show a particular case known to work, not a feature working in all conditions... Don't rush into this with high expectations. I will however be doing my best to fix bugs and add features. Please file bugs with Issuezilla in the scripting category with ruby as a sub category. Join the mailing lists to provide feedback and participate.

Did I mention that this is for early-adopter types only at this stage? Okay, good.

If you felt cheated by the above screenshot, here's another one. Can't remember the escapes in a Regular Expression? No problem!

This is not particular to regular expressions. Try it on dollar variables, or in literal strings. Here's another example: