Thursday April 05, 2007
Ruby Screenshot of the Week #8: Live Code Templates and TextMate Snippets I've received several requests for support for TextMate-style snippets. TextMate is a very popular editor among Ruby developers, and a lot of its power comes from its code snippet functionality. Snippets are basically identical to what NetBeans calls "Live Code Templates". The code templates are live in the sense that when you insert the template, you're in a mode where you can jump from one logical section of the template to another, and edits in one section can automatically be updated in other sections.
I've just implemented an import mechanism for TextMate bundles. Point at your own bundles (or even the ones distributed with TextMate, although I don't know what the legal issues are with that), and choose import. This will convert all* the snippets into NetBeans Live Code Templates (*=for some values of all).
There are two features in TextMate snippets that are not supported:
- Command execution. A snippet can contain a `shell command` (in backquotes), and at template evaluation time, the command will be executed and the result inserted. In the snippet bundles I looked at, this was used only for a single thing: inserting parentheses on a conditional basis. So I just made the import handle parentheses directly instead.
- Regular expression replacement. A template can perform regular expression replacement on different instances of the parameters. I can't even tell exactly how this is supposed to work, but for now, I skip snippets that do this.
TextMate is essentially a text editor; it doesn't (as far as I can tell, having never used it but having read their snippets and macros in detail) have any real semantic information about the code. Therefore, doing complicated text processing with regular expressions is the Way of TextMate to achieve logical code operations. Live Code Templates on the other hand can take advantage of all the information the IDE has about the code. I've added a bunch of logical code template variables you can use in templates. Rather than try to reproduce the regular expression operations of TextMate's snippets engine, I would like to create logical Ruby operations for the code templates and use those instead.
Here are some examples of what is possible with this mechanism. None of the imported snippets use these variables, but the imported snippets are useful in their own right - especially for those of you with muscle memory.
${unusedlocal defaults="i,j,k,x,y,foo,bar"}
This will evaluate to an unused local variable in the current scope. It will try to use the provided suggested names, in order. It checks whether each one is already in scope, and if so, tries the next candidate. (If none succeed, it starts appending numbers to find a unique name. If there are more Rubyish ways of generating unique variables, please let me know.)
First, we create the template, naming it "dob":
Then, in some method I want a do-block. I just type "dob" and hit Tab, and voila:
It inserts a do-end block. When it comes to the block variable, it has looked at the context and discovered that the first two preferred alternatives, i and j, are both taken - so it chooses k. This shows the advantage of using semantically aware code templates - you don't accidentally alias an existing variable. This wouldn't be a problem in Java, where it would complain that your new variable already exists in scope, but in Ruby, this is valid (and sometimes, but not usually, what you want.)
As another example, here's a silly template which spits out a hash that defines a bunch of local context:
Let's say I have the following silly file, and type "ctx":
If I now hit tab, I get this:
I will now be able to walk through (with the tab key) each of the variable sections and edit them. For an example like this, it's likely that we don't want these things to be edited by the user - so just add editable=false on each such variable (e.g. ${class editable=false}) and they will be processed but not edited by the user.
One of the things that makes this really powerful, is that it uses a lot of the context information the IDE has about your program. In this example, computing the super class of the current class isn't that hard - and a fancy Ruby shell script executed might be able to pull it off. But if I was simply redefining an existing class, let's say Integer, and there is no local reference to Integer's parent class, the live code template will spit out Numeric. Yes, it looks in the index.
Call for Help
As I've shown, the basic machinery is there now, but what we really need is a set of code templates to be bundled with the IDE. Some will be simple substitutions or linked variables, but we should be able to come up with some much stronger ones. Please let me know what kinds of logical parameters you can take advantage of. For example, perhaps a parameter which evaluates to the require statements necessary for the classes referenced in the file? Or a parameter which looks for database table names in the database? Perhaps some simple transformations which produce a constant name or a method name (upcasing and downcasing) from some other symbol? There are a lot of possibilities here, but since I'm not a real Ruby programmer I'm not the best person to dream this stuff up. If you can look at snippets you have written, import them into NetBeans, optionally edit them to add more semantic analysis if appropriate, and contribute them, I'd be very thankful!
Also, I have never used TextMate, so I would appreciate feedback on whether the templates work the way they do in TextMate, or if I misunderstood something from just reading the snippets themselves. Bug reports (preferably via the issue tracker) are much appreciated.
So, where can you get this stuff? I just integrated everything, but the Live Code Template management is not in stock NetBeans; it's in an optional download called "CodeTemplateTools" (in contrib/CodeTemplateTools), written by Sandip Chitale. More info about this module is
here. I have built an NBM you can install directly - just download the module bundle file, then install it via the IDE via the Update Center (in the first panel choose manually downloaded NBM). Note however that you must have the other IDE fixes as well; they should be live on deadlock.nbextras.org soon but probably won't hit the Update Center for a couple of days. (I'll try to make the slim IDE bundle the tools). As always, see ${cursor} for that now, but there's still some issues we need to resolve in this area.
Comments are closed for this entry.
Posted by Si on April 06, 2007 at 12:26 AM PDT #
Looking forward to trying this out! Can you confirm where you would like us to put comments noting there is your blog, netbeans ruby wiki, netbeans mailing list, netbeans bug tracker (I've not use this before).
Which of these areas would you prefer to receive:
a) feeback on your questions from the blog
b) bugs we find in the latest release drops
c) ideas for improvements re ruby/rails support
Cheers Greg
Posted by GregH on April 06, 2007 at 05:11 AM PDT #
Posted by Brian Ehmann on April 06, 2007 at 06:15 AM PDT #
On top of what it says, for the TextMates functionality in particular I'm really curious how the editing part works compared to TextMate - having never tried TextMate I don't know, but I want to make sure that the muscle memory (and productivity) is the same.
Posted by Tor Norbye on April 06, 2007 at 07:28 AM PDT #
Posted by Hemant Kumar on April 07, 2007 at 01:16 AM PDT #
for example, in emacs: If i place cursor at =>, after :partial and press C-Enter, then it would take me to the partial.
Posted by Hemant Kumar on April 07, 2007 at 01:34 AM PDT #
I go Edit/Show Templates/Import, but then Netbeans is looking for a "Abbreviations" file, for which I can't find a file in the checked out Textmate bundle files that match?
Can anyone see where I'm going wrong?
Cheers Greg
Posted by GregH on April 07, 2007 at 04:03 AM PDT #
Amazing feature list, Tor!
Could you please give some hints on the basics, though: when is it advisable to run JRuby and when ordinary ruby binary is better?
At my first attempt, I configured Netbeans to use c:\InstantRails\ruby\bin\ruby, and then it started to reindex not just my app, but also all installed gems and Rails and it probably took 2 or 3 hours on my Centrino 1.7 laptop. It finished at last, but when I restarted Netbeans, the indexing seemed to begin all over the same gem directories again.
With JRuby, reindexing behaves better, but the difficulty is having to install second copy of all gems and plugins that already sit in InstantRails in order to work with the existing application. Another disadvantage would be the script execution performance. What is the current performance loss with JRuby at running, say, unit and functional tests?
One unrelated question - with some *.rb files, the "Run" menu item is grayed out. Why is that? The same files run perfectly fine from console. For example, I cannot run script\about within Netbeans, and neither my custom-made maintenance scripts. On the contrary, the test cases have that "Run" menu item, and the process running progress bar appears for a while, but the output remains empty and the logs are not written to, so probably something does not go right.
Finally, one feature that I miss from Eclipse is being able to see what files in the file tree are covered by version control and what files are not - it helps a lot when committing a subset of modified files.
Regards
--
Yar
Posted by Yar Dmitriev on April 07, 2007 at 08:06 AM PDT #
Greg, have you tried selecting the -directory- named ".tmbundle" (e.g. Ruby.tmbundle) and importing that? I know, I should make a separate filter for it and use icons in the file chooser on these directories to make it more obvious.
Yar, perfect timing! You have version 0.58; in version 0.59 indxing should be -immediate- for native ruby! (I actually integrated this into version 0.58 and it worked on my mac, but I just noticed there was a problem on Linux that I made a quick fix for.). There was another problem related to symlinks on Linux which made it index all of Rails twice; thats also gone now.
In general, I'd say JRuby is for when you want to write Ruby applications that access Java classes, or for very easy out of the box configuration (for example, the system supplied Ruby on Mac's doesnt work well and it's nontrivial to build your own - most systems don't have cc installed etc). In the future, JRuby may also be very fast; they're making a lot of strides in this area.
I'm not sure what the deal is with grayed out Run files. If you could mail one to me (along with a description of where in the project the file sits) I can take a look (tor.norbye@sun.com).
I'll pass the version control comment on to the version control group. But in NetBeans you get a lot of help in this area at checkin time (try it - it will also include new files not included by version control as checkin suggestions - within reason (E.g. it skips files named *~, .backup, .class, etc.). I'm using the NetBeans CVS support daily and have never needed the icon badging that I believe we had in the old days for NetBeans; the new CVS support changed the way this works. On the other hand, if you really like a different workflow we should try to support it.
Posted by Tor Norbye on April 07, 2007 at 09:54 AM PDT #
Posted by Tor Norbye on April 07, 2007 at 10:15 AM PDT #
http://blogs.sun.com/scblog/entry/define_your_own_ code_template
Posted by Sandip on April 07, 2007 at 11:03 AM PDT #
Great, 0.60 improved the indexing speed in my setup (with native ruby) - it was finished within minutes. However, after restarting the application some of the indexing starts again. For most gem dirs it happens quickly and says "scanning". But my project dirs, such as "helpers", "vendors" are indexed again and again. Maybe I should drop the config line:
netbeans_extraclusters="c:/Program Files/netbeans/my_clusters/ruby1"
and just put ruby1 into C:\Documents and Settings\yar\.netbeans\ (is that correct?).
BTW, for clarity, I delete C:\Documents and Settings\yar\.netbeans\ and C:\Documents and Settings\yar\.netbeans-derby\ every time I replace ruby1 with new downloaded copy.
As for not-runnable files, here is a screenshot.Posted by Yar Dmitriev on April 08, 2007 at 12:27 AM PDT #
Regarding code indexing: It's true that it will index the -user- directories over again, at every startup (well, it should be looking at file timestamps and comparing to the index time stamps). It doesn't do this for the libraries and gems. The assumption is that user-files may have been edited by other tools or editors in between editor sessions so it does a check. I was assuming this check would be pretty quick, and that IDE startup is not that frequent so avoiding an out-of-date index was more important. Perhaps I should introduce a flag to suppress this behavior? Note that this was much worse until two days ago, because it turned out that on some Linux systems, the Rails command created symlinks in the user directory (in rails' vendor/ directory) back into the Rails libraries, so the indexer was happily rescanning all of Rails every startup. Ouch. That should be fixed now.
Regarding some files being disabled: From the screenshot it looks like the issue is that these are files which do not correspond to a "source root" (the directories in the logical/projects view show the source roots). I should still be able to execute these things, so I'll look into that.
Posted by Tor Norbye on April 08, 2007 at 08:19 PM PDT #
Posted by Aaron on April 08, 2007 at 09:10 PM PDT #
Posted by Tor Norbye on April 09, 2007 at 07:49 AM PDT #
Posted by Ralph on April 09, 2007 at 01:57 PM PDT #
- How do i update netbeans itself, when i have done a binary install in Linux. I am currently using ML-8.
- Just updating the cluster would update ruby/rails only features, right?
- I am getting a really nasty exception, when using Control-Space, both with Jruby and native Ruby. Where shall i file the bug report.
- There is no support for manual edit while replaying a macro in NBM. In emacs, i can pause a macro while replaying/recording and do some recursive/manual edit and resume the macro. Its very powerful feature, but most IDEs I have came across ignore it.
Thanks for the work on toggle.el.Posted by Hemant Kumar on April 09, 2007 at 02:41 PM PDT #
Posted by Hemant Kumar on April 09, 2007 at 02:43 PM PDT #
Posted by Aaron on April 09, 2007 at 06:48 PM PDT #
http://blogs.sun.com/scblog/entry/netbeans_module_update_center
The Code Template Tools module also has online help that can be accessed using Help:Content.
Posted by Sandip on April 09, 2007 at 11:18 PM PDT #
Posted by Si on April 10, 2007 at 03:25 AM PDT #
It's probably a good idea to grab the updated bits; I've fixed a bunch of issues. First, I believe I have fixed the "Run File" command being disabled that Yar reported. Second, there were some corrupted data in the native ruby code indices, which caused some ugly problems with code completion (this is probably Hemant's issue), third, I've made the Go To Test action work a bit better (in addition to fixing a problem with ZenTest, I noticed that Ruby tests often use "Test" as a prefix rather than a suffix), and a few other things.
Aaron, I have an open task to add a encoding property to the Ruby project types the way the Java project types recently did.
There's some information on installation and updating here: http://wiki.netbeans.org/wiki/view/RubyInstallation.
Posted by Tor Norbye on April 10, 2007 at 06:38 AM PDT #
WRT "ugly problems" you mention in your post, the bug that worst affects me by far happens during rhtml editing. Would this fall under scripting/ruby, or is it handled elsewhere currently? (scripting/www?).
It seems to be document structure related. For example swapping repeated p tags to li tags, for a better semantic match, I get Assertion errors popping up every keystroke. Portions of rhtml start dropping from the editor view, needing the file to be reloaded in the editor.
Posted by Si on April 10, 2007 at 09:28 AM PDT #
Posted by Tor Norbye on April 10, 2007 at 10:11 AM PDT #