Friday August 31, 2007
Friday August 31, 2007
Groovy Palette Item Wizard
Still working on Groovy support, one new feature lets you type a code snippet in a wizard, like this:
As you can see, my Schliemann syntax coloring also works in the JEditorPane. And then when you finish the wizard, you have a new module that will install your new snippet in the Groovy palette:
Plus, you can then distribute the snippet, since its now provided by a module.
Aug 31 2007, 11:34:05 AM PDT Permalink
Add a Comment
Monday August 27, 2007
How to Write a Groovy Editor (Part 4)
This is going to be one of those 'thinking out loud' blog entries, written in notepad as the thoughts occur (not unlike James Joyce, but then different) and then pasted into my blog. (Only that way does it make sense, in the sense of it being helpful, it won't be watered down or summarized, but really as things happen.) Basically, fundamentally, what I want to get done, finally, absolutely, and completely, is the Groovy tokens. Without that, no grammar rules and therefore no reliable Groovy editor. An ad-hoc on-the-fly editor won't cut it. And I do, despite earlier comments on this, think that the more religiously one sticks to the original ANTLR, the better. So, the time's come to pull out that bottle of merlot and really figure all this out for real. The template that I'm starting with, after considering the javascript.nbs, is the java.nbs. Not sure why, just instinctively. Maybe the javascript.nbs is closer to what the groovy.nbs should be than the java.nbs, but it just seems an additional step removed from the original language (i.e., Java), so let's take the existing Java tokens as our starting point and then compare to whatever comes up in the GroovyLexer.html, which defines Groovy tokens. So, take the GroovyLexer.html, look at each of the tokens, and map each to the tokens in the java.nbs. And then see what comes of it. Here's what comes of it, in the order that it comes:
The GroovyLexer.html begins with a long list of individual tokens (i.e., not tokens defining groups of characters, but just single characters). Sixty of them, in fact. For example, two of them are as follows:
mSEMI : ';' ; mDOLLAR : '$' ;
Jim Clarke's conversion (i.e., draft conversion, since we didn't, and still don't yet, know the 100% correct, if there is such a thing, mapping between ANTLR and NBS) of the above is as follows:
TOKEN:SEMI: ( ';' ) TOKEN:DOLLAR: ( '$' )
That definitely resembles what I expect NBS to be, and, in fact, there are no errors when the above is pasted into an NBS file. Plus, as can be seen in yesterday's blog entry, I was able to assign a color to the 'DOLLAR' token (otherwise the dollar character would not have been red in yesterday's blog entry).
So, how do tokens such as the above appear in java.nbs, assuming (which they logically must) that they appear there at all? Like so:
TOKEN:operator: (
"==" | "!=" | "<=" | ">=" | "?" | ":" | "<" | ">" | "/" | "*" | "-" |
"+" | "." | "," | "=" | "(" | ")" | "[" | "]" | "!" | "@" | "#" | "$" |
"%" | "^" | "&" | "~" | "|" | "\\"
)
And, for good measure, here's the javascript.nbs version:
TOKEN:js_operator: (
"==" | "!=" | "<<" | ">>" | ">>>" | ">=" | "<=" | "++" | "--" |
"+=" | "-=" | "*=" | "/=" | "%=" | "<<=" | ">>=" | ">>>=" | "&=" |
"^=" | "|=" | "&&" | "||" | "===" | "!==" |
"?" | ":" | "<" | ">" | "*" | "-" | "+" | "." | "," | "=" |
"(" | ")" | "[" | "]" | "!" | "@" | "#" | "%" | "^" | "&" |
"~" | "|" | "\\"
)
Let's be rigorous. Let's look at each individual token in GroovyLexer.html. Let's look at each individual token and see if each individual token is covered in the above grouped TOKENs (for want of a better word). Question mark - yes. Left parentheses - yes. Right parentheses - yes. Left bracket - yes. Right bracket - yes. Left curly - no. Right curly - no. Colon - yes. Comma - yes. Dot - yes. Assign - yes. Compare to (<=>) - no. (Cool! This is known as the spaceship operator.) Equal - yes. Exclamation mark - yes. Tilde - yes. Not equal - yes. Forward slash (i.e., DIV) - yes for java.nbs, but no for javascript.nbs. Forward slash + assign (DIV_ASSIGN) - no for java.nbs, yes for javascript.nbs. Plus - yes. Plus assign - no for java.nbs, yes for javascript.nbs. Plus plus - no for java.nbs, yes for javascript.nbs. Minus - yes. Minus assign - no for java.nbs, yes for javascript.nbs. Double minus - no for java.nbs, yes for javascript.nbs. Star - yes. Star assign - no for java.nbs, yes for javascript.nbs. Modulus - yes. Modulus assign - no for java.nbs, yes for javascript.nbs. Two right arrows - no for java.nbs, yes for javascript.nbs. Two right arrows assign - no, yes. Three right arrows - no, yes. Three right arrows assign - no, yes. One right arrow assign - yes. Greater than - yes. Two left arrows - no, yes. Two left arrows assign - no, yes. One left arrow assign - yes. Less than - yes. Hat - yes. Hat assign - no, yes. Or - yes. Or assign - no, yes. Double or - no, yes. Ampersand - yes. Ampersand assign - no, yes. Double ampersand - no, yes. Semicolon - no. Dollar - yes, no. Inclusive range - no. Exclusive range - no. Triple dot - no. Spread dot - no. Optional dot - no. Member point - no. Member find - no. Regex match - no. Star star - no. Star star assign - no. Closure op - no. At -yes.
Right. Time for the next glass of merlot before drawing any conclusions here. OK. So, the first thing to say is that there seem to be a bunch of operators unique to Groovy (at least, when compared to Java and JavaScript):
<=> .. ..< ... *. ?. .& =~ ==~ ** **= ->
Here's some info on some of the above. And here's some more. In addition, the Groovy definition of tokens includes the semi-colon, the left curly brace, and the right curly brace. That's kind of interesting.
And what tokens, from the java.nbs and javascript.nbs, are NOT included in the individual characters that make up the tokens of the GroovyLexer.html?
\\ # !== ===
So if we were to create a single TOKEN for all the single characters that are assigned to tokens in the GroovyLexer.html, it would be as follows:
TOKEN:OPERATOR: (
"==" | "!=" | "<<" | ">>" | ">>>" | ">=" | "<=" | "++" | "--" |
"+=" | "-=" | "*=" | "/=" | "%=" | "<<=" | ">>=" | ">>>=" | "&=" |
"^=" | "|=" | "&&" | "||"| "?" | ":" | "<" | ">" | "*" | "-" | "+" |
"." | "," | "=" | "(" | ")" | "[" | "]" | "!" | "@" | "%" | "^" |
"&" | "~" | "|" | "<=>" | ".." | "..<" | "..." | "*." | "?." | ".&" |
"=~" | "==~" | "**" | "**=" | "->" | "{" | "}"| ";"
)
I'm not sure if we'd want to make the above token, because possibly we want some of those characters to have unique colors (maybe the "->", for example, should have its own distinct color). However, the above is the token for Groovy that most closely matches the 'operator' token for Java and JavaScript. Also interesting to see that the result is closer to JavaScript than Java. However, that could simply mean that the java.nbs isn't very complete, which is even more true when you look at this document, which seems to imply that there are more 1:1 operator mappings between Groovy and Java than one would assume from the above.
Next, the first multi-character token in GoovyLexer.html is this one:
mWS : ( ' ' | '\t' | '\f' | '\\' mONE_NL )+ ;
Jim Clarke converted that to this for NBS:
TOKEN:WS: ( ( ' '
| '\t'
| '\f'
| '\\' ONE_NL )+
)
Here, we're dealing with white space, hence the capitalized WS, i.e., standing for "White Space". In the javascript.nbs, we find the following:
TOKEN:js_whitespace: ([" " "\t" "\n" "\r"]+)
The same is true for the java.nbs:
TOKEN:whitespace: ([" " "\t" "\n" "\r"]+)
Let's adopt this for our groovy.nbs, because even sticking in the converted WS produces errors. So now we have this:
TOKEN:WS: ([" " "\t" "\n" "\r"]+)
Look, white space is white space. Let's not get hung up about it. I could be very wrong, but I doubt Groovy is all that different when it comes to white space. In addition to white space, there's a new line token in the GroovyLexer.html. In the conversion, it is like this:
TOKEN:ONE_NL: ( ( "\r\n" | '\r'
| '\n' )
)
So, we'll turn it into this:
TOKEN:ONE_NL: ("\r\n" | "\r" | "\n")
Next, as explained in this wonderfully commented document, there is a token called NLS, which "groups any number of newlines (with comments and whitespace) into a single token". The conversion gives me this:
TOKEN:NLS: ( ONE_NL
( ( ONE_NL
| WS
| SL_COMMENT
| ML_COMMENT )+ )
)
Above, the arguments refer to 'one new line', 'white space', 'single line comment', and 'multiline comment', respectively. Let's not get too excited about any of these. Simply:
TOKEN:ML_COMMENT: ("/*" - "*/")
TOKEN:SL_COMMENT: ("//"[^"\n""\r""\uffff"]*)
So, based on the converted token (i.e., in case you forgot at this point, the above is how the NLS token looks after conversion by Jim Clarke, from the original GroovyLexer.html) we now should have this:
TOKEN:NLS: ( ONE_NL ( ( ONE_NL | WS | SL_COMMENT | ML_COMMENT )+ ) )
It seems, though, that tokens in NBS cannot refer to other tokens. However, if we turn it into a grammar rule, instead of a token, we might end up doing ourselves a favor. To do so, we need to write the statement like this:
nls=(<ONE_NL> ( ( <ONE_NL> | <WS> | <SL_COMMENT> | <ML_COMMENT> )+ ) );
Notice that the grammar rule ends with a semicolon, that each of the references to tokens are surrounded by pointy brackets, that there is no 'TOKEN' text in front of it, and that an equal sign is used instead of a colon.
The absence of square brackets is a bit disconerting at this point. Somehow, no square brackets, which imply optional statements, are in the original GroovyLexer.html. Hmmm. Ah, well. Problem for later.
Looking back, I find the clunkiness of this interesting:
TOKEN:ML_COMMENT: ( "/*"
( '*'
| ONE_NL
| ( '*'
| '\n'
| '\r'
| '\uffff' ) )*
"*/" )
Why couldn't that have been expressed like this:
TOKEN:ML_COMMENT: ("/*" - "*/")
That's how Schliemann does it. Everything from the "/*" to the "*/" is a comment, and because it can skip whitespaces, I guess because of that reason, it swallows up everything between the beginning point and the end. No need for all those line breaks and ufffs... :-) OK, so what we have right now has been hard won, that's for sure. No time like the present, then, to try it out, a little bit, by adding a declaration of a code fold and a declaration of a color to our NBS file:
FOLD:ML_COMMENT:"comment"
COLOR:SL_COMMENT: {
foreground_color: "orange";
font_type:"bold";
}
And now, you should see this when you run your editor, assuming you type the same text as me:
That's encouraging. When you collapse one of the folds, you should see the word 'comment' as the fold's label. If you want a different label there, change the fold's definition above. Also, hovering the mouse over a collapsed fold presents you with a popup showing the content of the fold, just like any other code fold in the IDE.
Let's move on now... Next up, there's a TOKEN for the script header, which is like this in the original GroovyLexer.html:
// Script-header comments
SH_COMMENT
options {
paraphrase="a script header";
}
: {getLine() == 1 && getColumn() == 1}? "#!"
(
options { greedy = true; }:
// '\uffff' means the EOF character.
// This will fix the issue GROOVY-766 (infinite loop).
~('\n'|'\r'|'\uffff')
)*
{ if (!whitespaceIncluded) $setType(Token.SKIP); }
//ONE_NL //Never a significant newline, but might as well separate it.
;
From the above, I'm guessing the tilde symbol is the equivalent of the hat symbol in regular expressions, for indicating the characters that should be excluded, so that we should end up like this:
TOKEN:SH_COMMENT: ("/#!"[^"\n""\r""\uffff"]*)
For the rest, there's a bunch of string tokens and regular expression tokens, as well as an identifier token. Instead of those, I'll simply use these two, until everything blows up in my face:
TOKEN:IDENTIFIER: (
["a"-"z" "A"-"Z"]
[^" " "\t" "\n" "\r" "?" ":" "<" ">" "/" "*" "-" "+" "." "," "=" "{" "}"
"(" ")" "[" "]" "!" "@" "#" "$" "%" "^" "&" "~" "|" "\\" ";"
]*
)
TOKEN:STRING: (
"\""
[^ "\"" "\n" "\r"] *
"\""
)
So the first is simply for texts that do not start with quotation marks, while the second must begin with quotation marks. Let's test that, as follows:
COLOR:STRING: {
foreground_color: "magenta";
font_type:"bold";
}
COLOR:IDENTIFIER: {
foreground_color: "red";
font_type:"bold";
}
And that gives us this color scheme:
Since I didn't actually put the Groovy operator into the NBS file, the exclamation mark is unrecognized and therefore it is underlined in squiggly red, plus obviously it doesn't have a color (because, not only didn't I assign a color, I also didn't recognize it because I didn't declare a token for it, as stated in this sentence, i.e., prior to this parenthetical part).
This time, we'll include the Groovy operator and assign it a color:
COLOR:OPERATOR: {
foreground_color: "green";
font_type:"bold";
}
Plus, we'll include numbers, loosely interpreted from the GroovyLexer.html:
TOKEN:NUMBER: (["0"-"9"] ["0"-"9" "."]*)
And that's it. We'll leave it there. Clearly, we've abandoned all claims to being religious now. We have a rather truncated interpretation of the original GroovyLexer.html, but that's okay. So long as we're aware of what we've done, we'll know that when things are going wrong, we need to go back to our GroovyLexer.html and see if the problem might be because of our flexible interpretation. Just need to remember that when the grammar rules are added, a variety of tokens will need to be replaced by the more relaxed ones discussed here. Also, it's probably better to keep it simple the first time round anyway. Now, we'll assign colors to our operators and our numbers:
COLOR:OPERATOR: {
foreground_color: "green";
font_type:"bold";
}
COLOR:NUMBER: {
foreground_color: "black";
font_type:"bold";
}
And now, when we run the editor again, we see that our operators are green and our numbers are black:
So, now we've pretty much negotiated our way to a set of tokens to use for Groovy. This is all of them:
TOKEN:OPERATOR: (
"==" | "!=" | "<<" | ">>" | ">>>" | ">=" | "<=" | "++" | "--" |
"+=" | "-=" | "*=" | "/=" | "%=" | "<<=" | ">>=" | ">>>=" | "&=" |
"^=" | "|=" | "&&" | "||"|
"?" | ":" | "<" | ">" | "*" | "-" | "+" | "." | "," | "=" |
"(" | ")" | "[" | "]" | "!" | "@" | "%" | "^" | "&" |
"~" | "|" | "<=>" | ".." | "..<" | "..." | "*." | "?." | ".&" |
"=~" | "==~" | "**" | "**=" | "->" | "{" | "}"| ";"
)
TOKEN:WS: (" " | "\t" | "\\f" | "\\"+)
TOKEN:ONE_NL: ("\r\n" | "\r" | "\n")
TOKEN:ML_COMMENT: ("/*" - "*/")
TOKEN:SL_COMMENT: ("//"[^"\n""\r""\uffff"]*)
TOKEN:SH_COMMENT: ("/#!"[^"\n""\r""\uffff"]*)
TOKEN:STRING: (
"\""
[^ "\"" "\n" "\r"] *
"\""
)
TOKEN:IDENTIFIER: (
["a"-"z" "A"-"Z"]
[^" " "\t" "\n" "\r" "?" ":" "<" ">" "/" "*" "-" "+" "." "," "=" "{" "}"
"(" ")" "[" "]" "!" "@" "#" "$" "%" "^" "&" "~" "|" "\\" ";"
]*
)
TOKEN:NUMBER: (["0"-"9"] ["0"-"9" "."]*)
Admittedly, not all of it is crystal clear to me. Parts of it come from the original GroovyLexer.html, other bits come from the java.nbs and/or the javascript.nbs, exactly as discussed above. What's especially cool is that we've been really rigorous with the operations, so at least the nice shiny Groovy operations will be included from the start. As for our strings, and other tokens, they're not brilliant, but they'll do for the first cut (and potentially be sufficient right the way through). So, that's tokens. (Above, they're all capitalized, but that's not a requirement at all, by the way.) Don't know if any of this is useful to anyone, but I imagine that the considerations I went through above are going to be common to anyone creating an editor on the Schliemann framework. Time for a celebratory glass of merlot. And next time... grammar rules!
Aug 27 2007, 05:39:23 PM PDT Permalink
Sunday August 26, 2007
How to Write a Groovy Editor (Part 3)
First, a thousand words:
So, here, apart from some obvious syntax coloring, there are two other interesting things going on. First, there's a code fold for classes:
And also for the body of classes:
I found that it is easier to create code folds before creating syntax colors. Syntax coloring can be extremely tricky, because the order in which you define TOKEN declarations is important. When the IDE parses the NBS file, it reads from top to bottom and so if the text in the editor applies to more than one TOKEN declaration, one TOKEN declaration's color may end up overwriting another TOKEN declaration's color. So, to take that factor out of the equation, it makes sense to first make sure that the grammar rules make sense. If you have a grammar rule for a class definition, there's no better way of assessing whether it is defined correctly than to assign a code fold to it, as below:
FOLD:ClassDeclaration: {
expand_type_action_name:"LBL_ExpandComments";
collapse_type_action_name:"LBL_CollapseComments";
fold_display_name:"Groovy class";
}
(Note that the 'expand_type_action_name' and the 'collapse_type_action_name' resolve to labels that appear on two menu items that magically appear in the editor's popup menu, for expanding and collapsing the editor's code folds.)
Another issue that's problematic is coming to a stable set of grammar rules in the first place. The "hit and miss" approach I used in the first part of this series cannot be the ultimate solution. Better to have the GroovyRecognizer.html and GroovyLexer.html close by. The absolute purist approach, which would be to stick religiously to these two HTML documents, is not likely to work either. For one thing, it's hard to figure out exactly how to translate these definitions to Schliemann. I had a very good start, thanks to some work that Jim Clarke did some months ago, but even that (i.e., generated draft NBS from the above HTML documents) isn't the final answer. Some common sense, I guess, is the bridge between the "hit and miss" (i.e., starting with another NBS and tuning it to your own needs) and the purist (i.e., be religious and do not deviate from the official ANTLR definition) approach.
Also note that I have the Navigator working, for the first time for Groovy. Not perfect yet, but a clear start. As an example, this is how the class ends up in the Navigator:
NAVIGATOR:ClassDeclaration: {
display_name: org.netbeans.modules.mygroovyeditor.Groovy.className;
icon: "/org/netbeans/modules/languages/resources/class.gif";
isLeaf: "false";
}
The above should be put in the Groovy.nbs file. (By the way, the 'isLeaf' key doesn't seem to make a difference, currently, whether you put true or false, but I think it does need to be there anyway.) And then here, in my Grovvy.java class, I have the className method:
public static String className(SyntaxContext context) {
ASTPath path = context.getASTPath();
ASTItem functionClause = (ASTNode) path.getLeaf();
ASTItem name = null;
for (ASTItem item : functionClause.getChildren()) {
name = item;
}
if (name != null) {
String nameStr = ((ASTNode) name).getAsText();
return "Class: " + nameStr.substring(0, nameStr.indexOf("{"));
}
return "?";
}
The string manipulation is a bit messy, I'm pretty sure it should be a cleaner solution than that, but it gets the job done.
But the coloring is a long way from done. I'm only just beginning to understand how the grammar rules work. In between, I'm working on the hyperlinks in the Output window, which appear there when Groovy returns error messages when a script is run. Some of the error messages are convenient in that they include the line number where the error occurred. It is then very easy to create the hyperlink such that the user can click it to jump to the erroneous line. Other errors, especially those coming from javax.script.ScriptException, give no indication of where the error took place. This is a pretty big problem, one that may not be possible to solve. Adding annotations to the other error lines, i.e., those that can be identified, will be easy. Just the same old annotation approach as done for all editors, just attach the annotation to the line and you're done, like so:
It's pretty cool that Schliemann gives lots of room for incorporating the standard NetBeans API classes. (By the way, the 'Groovy error' message in the screenshot above, which pops up when the mouse hovers over the red error mark, is a bit truncated, that's not how it is in real life, just how my screenshot ended up.)
Well, that's where things are right now. Way too soon to release all this in any form, but if things continue in this way, maybe by the end of the week. The grammar rules are going to continue being a problem, though. To get an absolutely reliable editor may take quite some time in tweaking and so on. And then there's also code completion to look at, of course...
A final happy picture, of some SwingBuilder code:
Aug 26 2007, 08:26:51 AM PDT Permalink
Friday August 24, 2007
How to Write a Groovy Editor (Part 2)
I've come to believe that by far the most important feature of an editor is its syntax coloring. Forget all the rest, even code completion—the ability to distinguish one piece of syntax from another is the fundamental demand that an editor should fulfil. Yesterday we made some progress in providing this for Groovy. At least, I discussed the main principles and applied them to the import statement and the Groovy keywords. We started with the Java NBS file and began whittling it down to the more relaxed Groovy standard. However, this is not an approach that will ultimately be satisfactory. Really, we need to take the official token and grammar definitions from Groovy (in ANTLR) and work with that. Anything else, such as my "lets start with Java NBS and somehow fumble our way to Groovy NBS" approach, is just too haphazard and random. I have several resources that will make for a better end result, based on ANTLR definitions, and I will look at those in one of the next parts of this series.
For now, even more important at this stage than complete and absolute syntax coloring, is the ability to run Groovy scripts inside the IDE. Not just that, though. We also need to write the Groovy output to the NetBeans Output window. Kind of exactly like this:
So, you should be able to right-click in the Groovy editor, choose 'Run Groovy Script' and then the Output window contains the results, as shown above. (Just for laughs, I started adding a drag-and-drop code snippet palette to the editor, as can be seen above, which takes about 3 surprisingly simple steps to create, as you will see later in this blog entry.)
However, when things go wrong with the Groovy script, we want the error output to appear as hyperlinks in the Output window, via our trusty old friend OutputListener, the tireless Tonto to any lone NetBeans editor. As a result, we should be able to see a hyperlink for each error, in the Output window, as below:
Now we will, step by step, create all the above functionality...
- First we need to create the "Run Groovy Script" menu item in the Groovy editor. "Ah," you are now probably thinking (if you are familiar with the NetBeans Platform), "that means, we should use the New Action wizard to create a CookieAction implementation. The CookieAction.enable method could then be used as a filter, if needed." Well, nice try, but wrong. Schliemann lets you declare ACTIONs, which you implement in Java code. So, to see how this works, add this to your Groovy.nbs:
ACTION:run_script: { name:"LBL_Run"; performer:org.netbeans.modules.mygroovyeditor.Groovy.performRun; enabled:org.netbeans.modules.mygroovyeditor.Groovy.enabledRun; explorer:"false"; }Then, in the bundle file add this:
LBL_Run=Run Groovy Script
At this point, one thing you might be wondering is: "How will the NBS file find the value of the 'LBL_Run' key?" Well, in step 4 of yesterday's blog entry, we created this BUNDLE declaration in the Groovy.nbs file:
BUNDLE "org.netbeans.modules.mygroovyeditor.Bundle"
So, via that BUNDLE declaration, we will get our label on the menu item.
- The most important parts of the ACTION declaration are the 'performer' and 'enabled' keys. The latter determines when the menu item will be enabled. The former determines what happens when the menu item is clicked. So they're like the methods enable and actionPerformed. We already have a class called Groovy.java, because that's where we put the code completion code yesterday. Let's now add the code that will determine whether the menu item is enabled:
public static boolean enabledRun(ASTNode node, JTextComponent comp) { try { ClassLoader cl = Groovy.class.getClassLoader(); Class managerClass = cl.loadClass("javax.script.ScriptEngineManager"); return managerClass != null; } catch (ClassNotFoundException ex) { return false; } } - Now, in the performRun method, before we do anything else, let's first get to a point where we have the Groovy scripting engine available to us. At the end of this step, we will print the available script engines to the Output window, and we must be able to see that Groovy is included, because by default only the other two are available:
First, our performRun method:
public static void performRun(final ASTNode node, final JTextComponent comp) { RequestProcessor.getDefault().post(new Runnable() { public void run() { Groovy gr = new Groovy(); ScriptEngineManager factory = new ScriptEngineManager(); Listse = factory.getEngineFactories(); gr.printCollection(se); } }); } And here is the helper method:
private void printCollection(List c) { @SuppressWarnings(value = "unchecked") Iteratori = c.iterator(); while (i.hasNext()) { ScriptEngineFactory item = i.next(); System.out.println("Engine name: " + item.getEngineName() + ",\nEngine version: " + item.getEngineVersion() + ",\nLanguage name: " + item.getLanguageName() + ",\nLanguage version: " + item.getLanguageVersion() + "\n"); } } I got the above code from this helpful blog. (Make sure that you set the enabledRun method to simply return true, at this stage, because if you do not have the Groovy scripting engine available, the menu item itself will not be enabled and so you will not be able to click it to produce the results in the Output window!) When you now install the editor, open a Groovy file, and click the 'Run Groovy Script' menu item, the Output window should show you that you have two script engines available, neither of them being Groovy. You need to create a library wrapper module for groovy-engine.jar, with a dependency on another library wrapper module that contains groovy-all-1.0.jar. These library wrapper modules then need to be declared as dependencies of the MyGroovyEditor module. Once you've done all that, you've put the Groovy scripting engine functionality on the editor's classpath and then, when you click the menu item again, you will see that the information about the Groovy scripting engine will be printed to the Output window. Do not continue with these instructions until you are able to see the info about the Groovy scripting engine in the Output window, because as long as it isn't there, you do not have the Groovy scripting engine available to your editor.
- Next we will rewrite the performRun method to run the current Groovy script, and print hyperlinks to the
Output window on failure. The cool thing about JSR 223 is that, because of the script engine standardization, we can simply reuse code from the same method in the JavaScript.java file, which seeks to do the same thing. So, without further ado, here it is, with comments interspersed, which will hopefully explain things here and there:
public static void performRun(final ASTNode node, final JTextComponent comp) { RequestProcessor.getDefault().post(new Runnable() { InputOutput io; ClassLoader cl; FileObject fo; public void run() { try { //Get our Groovy script engine: ScriptEngineManager factory = new ScriptEngineManager(); ScriptEngine engine = factory.getEngineByExtension("groovy"); //Get our document, from the component //thst we received from the ACTION declaration, //and the name, for display in the Output window tab: Document doc = comp.getDocument(); DataObject dob = NbEditorUtilities.getDataObject(doc); String name = dob.getPrimaryFile().getNameExt(); //Load the script engine class: cl = Groovy.class.getClassLoader(); Class engineClass = cl.loadClass("javax.script.ScriptEngine"); //Open the Output window, //and set the label in the tab //to Run + the name of the file: io = IOProvider.getDefault().getIO("Run " + name, false); //Get the script context class //and use it to write to //the Output window: Class contextClass = cl.loadClass("javax.script.ScriptContext"); Method setWriter = contextClass.getMethod("setWriter", new Class[]{Writer.class}); Method setErrorWriter = contextClass.getMethod("setErrorWriter", new Class[]{Writer.class}); Method setReader = contextClass.getMethod("setReader", new Class[]{Reader.class}); Method getContext = engineClass.getMethod("getContext", new Class[]{}); Object context = getContext.invoke(engine, new Object[]{}); setWriter.invoke(context, new Object[]{io.getOut()}); setErrorWriter.invoke(context, new Object[]{io.getErr()}); setReader.invoke(context, new Object[]{io.getIn()}); //Select the Output window: io.select(); //Get the 'eval' method //and then invoke it, using the text //from the received component: Method eval = engineClass.getMethod("eval", new Class[]{String.class}); Object o = eval.invoke(engine, new Object[]{doc.getText (0, doc.getLength ())}); //If the Groovy script fails... } catch (InvocationTargetException ex) { try { Class scriptExceptionClass = cl.loadClass("javax.script.ScriptException"); if (ex.getCause() != null && scriptExceptionClass.isAssignableFrom (ex.getCause().getClass())) { if (io != null) { String msg = ex.getCause().getMessage(); int line = 0; if (msg.startsWith("sun.org.mozilla")) { msg = msg.substring(msg.indexOf(':') + 1); msg = msg.substring(0, msg.lastIndexOf('(')).trim() + " " + msg.substring(msg.lastIndexOf(')') + 1).trim(); try { line = Integer.valueOf(msg.substring(msg. lastIndexOf("number") + 7)); //NOI18N } catch (NumberFormatException nfe) { } } //Implement OutputListener, for hyperlinks: io.getOut().println(msg, new MyOutputListener(fo, line)); } else { Exceptions.printStackTrace(ex); } } } catch (Exception ex2) { Exceptions.printStackTrace(ex2); } } catch (Exception ex) { Exceptions.printStackTrace(ex); } } }); } - Next, create a class called MyOutputListener, let it implement OutputListener and, for now, just let the IDE generate three method stubs that we can fill out some other time. So, for now, when you click a link in the Output window, you'll receive an error message because you haven't implemented the method yet.
- And what about the palette? Just download it here, add it to the module suite, and then tweak a few things. For example, the palette was written for text/x-java, so change that to text/x-groovy. You might also want to do some refactoring, to make the class names, and so on, Groovy-oriented rather than Java-oriented. Then change the sample snippet to generate the code that you want to have generated. Now your module suite should be as follows:
It really is cool that one can now (from 6.0 onwards) create a palette in one module, and (via MIME-type registration in the layer) make it available to an editor defined in a different module. Potentially, your users do not want to have a palette in their Groovy editor. Now they don't have to have it. They simply would not install it. Plus, one engineer could 'own' the palette module, while another engineer could 'own' the editor module, thus allowing for a very clean separation of work areas. Ah, but anyone reading this blog, and who has got this far in this particular blog entry, should be aware of the benefits of module-based development already... :-)
When you now install the editor again, you should see the same things as shown in the two screenshots at the start of this blog entry. Hurray! Thank you, JSR 223 and thank you Schliemann. Now there's scripting support for Groovy in NetBeans IDE.
In other news. My colleague Rohan, from Bangalore, wrote me with this cool news: "We, at Sun India, have launched a unique contest called the CODE FOR FREEDOM CONTEST to encourage students in India to adopt open source technologies and contribute to them. You can find more details here. NetBeans is part of the open source technologies covered by this contest." Hurray!
Aug 24 2007, 01:57:58 AM PDT Permalink
Thursday August 23, 2007
How to Write a Groovy Editor (Part 1)
I thought I'd put down on 'paper' the things I've been looking at in relation to Groovy and providing an editor for it. By 'editor' I mean primarily 'syntax coloring', 'code folding', 'code completion', and 'navigator support'. Secondarily I mean 'run' functionality and 'console integration'. The key to everything in the primary section in relation to NetBeans is Schliemann. Fortunately, a Schliemann file for Java already exists (here), so since Groovy is closely related to Java, kind of like its scruffy nephew, let's not reinvent the wheel. Let's just take the Java NBS file, adapt it to our purposes, and add whatever functionality we'd like to provide. That should work, all things being equal (though they seldom are or seem to be), one would think.
- Use the Module wizard to create a module project called 'MyGroovyEditor'. Make the code name base 'org.netbeans.modules.mygroovyeditor'.
- Right-click the project node, choose New and then Other, and then choose 'Language Support' from the Module Development category in the New Project wizard. Click Next. Type 'Groovy' in File Name. Click Next. Type 'text/x-groovy' in Mime Type and 'groovy' in Extensions. Click Finish.
The cheerful image that now greets your gaze is as follows:
Note: The Groovy.nbs file that you see above is a template for a Schliemann file. It already contains a few declarations, but we will end up replacing all of them. However, it is useful in giving you an immediate feeling for what Schliemann is and does.
- Next, let's create some templates for Groovy, so that we have a few use cases of the language for trying out our language support features. After a few seconds of googling, I've come up with the following template content:
class Person { firstName lastName address1 city province postcode } p = new Person(firstName:'Ian', lastName:'Darwin', province:'Ontario') println "Hello ${p.firstName} ${p.lastName}"The above bit comes from here, while Wikipedia gave me this:
import groovy.xml.MarkupBuilder def myXMLDoc = new MarkupBuilder() myXMLDoc.workbook { worksheet(caption:"Employees") { row(fname:"John", lname:"McDoe") row(fname:"Nancy", lname:"Davolio") } worksheet(caption:"Products") { row(name:"Veeblefeetzer", id:"sku34510") row(name:"Prune Unit Zappa", id:"sku3a550") } } println myXMLDoc..while elsewhere I find this:
import java.awt.BorderLayout swing = new groovy.swing.SwingBuilder() myFrame = swing.frame(title: 'Sample Frame', location:[100,100], size:[300,200]) { menuBar { menu(text: 'File') { menuItem(text: 'Exit', actionPerformed:{System.exit(0)}) } } panel(layout: new BorderLayout()) { label(text: 'Input:', constraints: BorderLayout.WEST, toolTipText: 'Tool Tip') textField(constraints: BorderLayout.CENTER) button(text: 'Hello', constraints: BorderLayout.SOUTH, actionPerformed:{println("World")}) } } myFrame.visible = trueLet's take these as our three templates. First, before even creating them, let's register them in the layer file:
<folder name="Templates"> <folder name="Groovy"> <file name="Groovy1.groovy" url="Groovy1.groovy"> <attr name="template" boolvalue="true"/> </file> <file name="Groovy2.groovy" url="Groovy2.groovy"> <attr name="template" boolvalue="true"/> </file> <file name="Groovy3.groovy" url="Groovy3.groovy"> <attr name="template" boolvalue="true"/> </file> </folder> </folder>And now use the Empty File template in the Other category, in the New File wizard, to create 'Groovy1.groovy', 'Groovy2.groovy', and 'Groovy3.groovy'. Put them in the main package, where the other files are already. Paste the first snippet above into the first file, the next into the second, and the third in the last.
- Our next step can be summed up by the word 'plagiarism'. Click here and copy the entire content into your 'Groovy.nbs' file. When I did this, I found several red error marks in the right side of the editor. I found that it wasn't necessary to fix everything but, at the time of writing, the content I got had some flaws in the Schliemann definitions, in the COLOR definitions which, at the time of this writing, begins in line 458. There, for each COLOR definition, add a colon after each COLOR name, so that there is a colon right before and right after each name, such as here for 'separator':
COLOR:separator: { color_name: "operator"; }Do this for each COLOR definition. Then also for the Navigator definitions. Then delete the ERROR definitions; let's not worry about those yet, they're just going to get in our way later. The remaining error markings are harmless. Fix the BUNDLE declaration to the following:
BUNDLE "org.netbeans.modules.mygroovyeditor.Bundle"
- Now add 'def' to the long list of keyword TOKENs, as shown below:
For now, we'll just stick the 'def' keyword in with the standard Java keywords, but later we'll take this handy PDF containing Groovy keywords and give the Groovy-specific keywords special treatment.
- Right, looks like we're good to go. Right-click the project node and choose 'Install/Reload in Target Platform'. Our editor is built, a new instance of the IDE starts up, and the editor is installed within it. Once the IDE is started, create a new Java application project (or, in fact, any project at all). For the project you created, open the New File wizard and see that you have three templates in a new Groovy folder:
Also notice the icon that accompanies these three files. That's an icon that all Schliemannized file types get by default. So, from the fact that our templates have this icon, we can ascertain that the 'text/x-groovy' MIME type has been successfully registered and that it has been mapped to the 'groovy' file extension. Let's take the second template and then click through the remainder of the wizard. Once the wizard is complete, you should see the following in the editor:
Take special note of the fact that 'def' is blue, above. We've simply assigned it to the 'keyword' TOKEN, hence it is treated like any other keyword.
Note: The TOKEN named 'keyword' and the TOKEN named 'string' each have a default color, as you can see above. That's true for all NBS files: if your NBS file has a TOKEN called 'keyword', then anything assigned to that TOKEN has the default color blue, as shown above, so that (unless you want a different color) you don't need to specifically define a color for 'keyword'. Same situation for 'string', except that here the default color is brown, as can be seen above.
- Let's now have a look at the grammar section of the file. To begin with, let's look at the way that the classes are defined and how import statements are handled. Right after the TOKEN definitions, you should (at the time of this writing) find the following declarations:
S = [PackageDeclaration] (ImportDeclaration)* (TypeDeclaration)*; PackageDeclaration = "package" Name ";"; ImportDeclaration = "import" Name ["." "*"] ";"; TypeDeclaration = ClassDeclaration | InterfaceDeclaration | ";"; ClassDeclaration = ClassDeclarationModifiers UnmodifiedClassDeclaration; ClassDeclarationModifiers = ("abstract" | "final" | "public")*; UnmodifiedClassDeclaration = "class" <identifier> ["extends" Name] ["implements" NameList] "{" ClassBody "}"; NestedClassDeclaration = NestedClassDeclarationModifiers UnmodifiedClassDeclaration; NestedClassDeclarationModifiers = ("static" | "abstract" | "final" | "public" | "protected" | "private")*; ClassBody = ( Initializer | NestedClassDeclaration | NestedInterfaceDeclaration | ConstructorDeclaration | MethodDeclaration | FieldDeclaration )*; InterfaceDeclaration = InterfaceDeclarationModifiers UnmodifiedInterfaceDeclaration; InterfaceDeclarationModifiers = ("abstract" | "public")*; NestedInterfaceDeclaration = NestedInterfaceDeclarationModifiers UnmodifiedInterfaceDeclaration; NestedInterfaceDeclarationModifiers = ( "static" | "abstract" | "final" | "public" | "protected" | "private" )*; UnmodifiedInterfaceDeclaration = "interface" <identifier> ["extends" NameList] "{" InterfaceMemberDeclaration "}"; InterfaceMemberDeclaration = ( NestedClassDeclaration | NestedInterfaceDeclaration | MethodDeclaration | FieldDeclaration )*;Well, that's our strict ol' uncle Java. Let's Groovify it, by making the modifiers and semi-colons optional. Below, the lines in bold are those that I changed:
S = [PackageDeclaration] (ImportDeclaration)* (TypeDeclaration)*; PackageDeclaration = "package" Name [";"]; ImportDeclaration = "import" Name ["." "*"] [";"]; TypeDeclaration = ClassDeclaration | InterfaceDeclaration | [";"]; ClassDeclaration = [ClassDeclarationModifiers] UnmodifiedClassDeclaration; ClassDeclarationModifiers = ("abstract" | "final" | "public")*; UnmodifiedClassDeclaration = "class" <identifier> ["extends" Name] ["implements" NameList] "{" ClassBody "}"; NestedClassDeclaration = [NestedClassDeclarationModifiers] UnmodifiedClassDeclaration; NestedClassDeclarationModifiers = ("static" | "abstract" | "final" | "public" | "protected" | "private")*; ClassBody = ( Initializer | NestedClassDeclaration | NestedInterfaceDeclaration | ConstructorDeclaration | MethodDeclaration | FieldDeclaration )*; InterfaceDeclaration = [InterfaceDeclarationModifiers] UnmodifiedInterfaceDeclaration; InterfaceDeclarationModifiers = ("abstract" | "public")*; NestedInterfaceDeclaration = [NestedInterfaceDeclarationModifiers] UnmodifiedInterfaceDeclaration; NestedInterfaceDeclarationModifiers = ( "static" | "abstract" | "final" | "public" | "protected" | "private" )*; UnmodifiedInterfaceDeclaration = "interface" <identifier> ["extends" NameList] "{" InterfaceMemberDeclaration "}"; InterfaceMemberDeclaration = ( NestedClassDeclaration | NestedInterfaceDeclaration | MethodDeclaration | FieldDeclaration )*;So I used square brackets to make the modifiers optional (since public is assumed) and I also made the semi-colons optional.
- Now, let's immediately add code folding functionality, for all 'types', which, based on the above Schliemannesque declarations, are classes and interfaces. So add this line in the 'code folding' section, which is near the end of the file:
FOLD:TypeDeclaration:"{...}"Excellent, let's see where things are and install the editor as before. This time, use the first template and then notice that when you complete the wizard, your class definition has a code fold:
Hurray! Believe me, creating a code fold in Java, on the NetBeans Platform, is as much fun as a slap in the face. The above... well, it couldn't be any simpler.
Also notice that there seems to be some kind of error marking for the single-quotes around the strings, in the screenshot above. Seems Groovy is flexible enough to cater for single quotes, while grumpy Java has problems with it. So redefine the TOKEN named 'string' to the following:
TOKEN:string: ( ["\""]|["'"] ( [^ "\"" "\n" "\r"] | ("\\" ["r" "n" "t" "\\" "\'" "\""]) )* ["\""]|["'"] )I changed the first line and the last line of the declaration above, so that either a double-quote or a single-quote is optional. Now, when I reinstall the editor, the small error markings under the single-quotes have vanished:
- Let's now look at the import statements, so that we can end up with the same result as shown in the previous blog entry. First, we'll color the import statements. From the earlier snippet, we know that the name of the grammar rule defining import statements is 'ImportDeclaration'. Hence, we end up with this COLOR definition, to make our import statements a festive green:
COLOR:ImportDeclaration: { foreground_color: "green"; font_type: "bold+italic"; }Install the editor again and then open the third template, which should now look as follows:
However, as in the previous blog entry, we want to be able to add the Groovy keyword 'as', together with an alias, in such a way that the 'as foo' ends up looking different to the rest of the line. That way, the user will be able to read and maintain the import aliases much better, because they'll be easy to identify. Currently, if we use the 'as' keyword, the import statement looks as follows:
...which is a bit dull. Let's make it red instead. Find this line, back in the grammar secion, which defines your import statements:
ImportDeclaration = "import" Name ["." "*"] [";"];
Replace the above line with these three:
ImportDeclaration = ImportBasisDeclaration [ImportWithAliasDeclaration] [";"]; ImportBasisDeclaration = "import" Name ["." "*"]; ImportWithAliasDeclaration = "as" <identifier>;
And now replace the COLOR definition for the ImportDeclaration rule, with these two COLOR definitions:
COLOR:ImportBasisDeclaration: { foreground_color: "green"; } COLOR:ImportWithAliasDeclaration: { foreground_color: "red"; font_type: "bold+italic"; }Install the editor again and now you should see this result, when you add some alias to your import statements:
- Notice something else... when you put the cursor on an open bracket, or a close bracket, its matching bracket is highlighted in green, together with the selected bracket, so that you can see which bracket matches with which other bracket:
Above, you see one example, and below another:
That's just default behavior. (I'm sure that, some months ago, when I looked at all this for the first time, the above brace matching had to be explicitly declared. However, makes sense if that isn't necessary anymore, because brace matching such as the above is always going to have this behavior, so if it is taken care of under the hood, then that makes perfect sense to me.) Both curly braces and standard brackets are handled in this way.
- Now we'll give all Groovy-specific keywords a distinct color. Let's make it red, so that our import alias, which is also Groovy-specific, matches with our other Groovy-specific syntax. From our Groovy reference card, we can ascertain that the following keywords are distinctly Groovy:
- as
- def
- in
- property
Fine. Remove 'def' from the TOKEN called 'keyword'. Create a new TOKEN, called 'keywordGroovy', and add the above keywords, so that you have this result:
TOKEN:keywordGroovy: ( "as" | "def" | "in" | "property" )Then add a new COLOR definition, amongst the other existing COLOR definitions:
COLOR:keywordGroovy: { foreground_color: "red"; font_type: "bold+italic"; }After reinstalling the editor, you should see things similar to this, when working with import statements, Groovy import aliases, and the Groovy keywords listed above:
- In a similar way to how the class and interface declarations were adapted for Groovy, you could now adapt methods, variables, fields, and so on, just changing the definitions from Java-oriented to Groovy-oriented.
- Next, let's think about code completion. We'll turn here in the NetBeans sources, to see how Schliemann provides language support for JavaScript. There are quite a few things to learn in that directory, such as in the JavaScript.nbs file itself. Here, among other things, we find the following:
COMPLETION:js_identifier, js_keyword, js_whitespace, js_operator, js_separator, js_comment: { text1:org.netbeans.modules.languages.javascript.JavaScript.completionItems; confirmChars:";(,."; }So, here various grammar rules have their code completion story defined in a class called 'JavaScript', in the package 'org.netbeans.modules.languages.javascript', which is this one. - First, create a class called Groovy, in your main package, and then copy the following code, which is the code completion section from the file above, together with some supporting methods and slightly tweaked declarations, into the Groovy class:
public class Groovy { private static final String DOC = "org/netbeans/modules/mygroovyeditor/Documentation.xml"; private static final String DOM0 = "org/netbeans/modules/mygroovyeditor/DOM0.xml"; private static final String DOM1 = "org/netbeans/modules/mygroovyeditor/DOM1.xml"; private static final String DOM2 = "org/netbeans/modules/mygroovyeditor/DOM2.xml"; private static LibrarySupport library; private static LibrarySupport getLibrary () { if (library == null) library = LibrarySupport.create ( Arrays.asList (new String[] {DOC, DOM0, DOM1, DOM2}) ); return library; } // code completion ......................................................... public static ListcompletionItems (Context context) { if (context instanceof SyntaxContext) { List result = new ArrayList (); return merge (result); } List result = new ArrayList (); TokenSequence ts = context.getTokenSequence (); Token token = previousToken (ts); String tokenText = token.text ().toString (); String libraryContext = null; if (tokenText.equals ("new")) { result.addAll (getLibrary ().getCompletionItems ("constructor")); return merge (result); } if (tokenText.equals (".")) { token = previousToken (ts); if (token.id ().name ().endsWith ("identifier")) libraryContext = token.text ().toString (); } else if (token.id ().name ().endsWith ("identifier") ) { token = previousToken (ts); if (token.text ().toString ().equals (".")) { token = previousToken (ts); if (token.id ().name ().endsWith ("identifier")) libraryContext = token.text ().toString (); } else if (token.text ().toString ().equals ("new")) { result.addAll (getLibrary ().getCompletionItems ("constructor")); return merge (result); } } if (libraryContext != null) { result.addAll (getLibrary ().getCompletionItems (libraryContext)); result.addAll (getLibrary ().getCompletionItems ("member")); } else result.addAll (getLibrary ().getCompletionItems ("root")); return merge (result); } private static List merge (List items) { Map it = items.iterator (); while (it.hasNext ()) { CompletionItem completionItem = it.next (); CompletionItem current = map.get (completionItem.getText ()); if (current != null) { String library = current.getLibrary (); if (library == null) library = ""; if (completionItem.getLibrary () != null && library.indexOf (completionItem.getLibrary ()) < 0 ) library += ',' + completionItem.getLibrary (); completionItem = CompletionItem.create ( current.getText (), current.getDescription (), library, current.getType (), current.getPriority () ); } map.put (completionItem.getText (), completionItem); } return new ArrayList (map.values ()); } private static Token previousToken (TokenSequence ts) { do { if (!ts.movePrevious ()) return ts.token (); } while ( ts.token ().id ().name ().endsWith ("whitespace") || ts.token ().id ().name ().endsWith ("comment") ); return ts.token (); } } - Set a dependency on Lexer, which should make all the scary red lines go away, and copy the content of Documentation.xml, Dom0.xml, Dom1.xml, and Dom2.xml into files of the same name in your main package. Another place to access them all from is here.
- Next, in our Groovy.nbs file, rewrite the JavaScript code completion declaration to the following:
COMPLETION:identifier, keyword, whitespace, operator, separator, comment: { text1:org.netbeans.modules.mygroovyeditor.Groovy.completionItems; confirmChars:";(,."; } - Install the editor again and now notice the following:
- So, the code completion is working. Now we need to edit the XML files we created above, to provide just the values that we need. Understandably, this could be a lot of work. There are values that are 'built in' to a language, and then there are those that are generated from libraries. Built-in values are, for example, the keywords. Those coming from libraries are, for example, Swing components, methods, and properties. For the latter, there is some kind of complex (or so it seems) approach relating to the Common Scripting Language Platform Support, by Tor, which was used for Ruby and which Caoyuan Deng used for Erlang. However, I don't understand it yet, and it doesn't seem stable, so I'm not making use of it yet. It provides an indexing mechanism for parsing and retrieving items from a Lucene engine, if I understand it correctly. However, the JavaScript language support relies purely on the XML approach outlined above, so doing it that way does seem to be possible.
- Let's generate an XML file containing Swing components and Swing methods. The following is a complete Generator class, received from Jim Clarke, which in this case generates the XML content of a new file, called "DOM3.XML":
public class Main { private static MapThe Output window now contains a very long list of XML tags. Create DOM3.xml and paste the content into that file. Again, this should be done in a different way, not via a documentation XML file, but the other way doesn't make sense to me yet and doesn't seem stable (requiring hacks in various places).
- Tweaking the above generator slightly, you can use it to generate the following:
TOKEN:swingComponents:( "table" | "checkBoxMenuItem" | "dialog" | "layeredPane" | "tree" | "hstrut" | "button" | "gridBagLayout" | "gridLayout" | "textField" | "vglue" | "viewport" | "menuItem" | "menu" | "window" | "fileChooser" | "toggleButton" | "menuBar" | "ridgerea" | "checkBox" | "tabbedPane" | "springLayout" | "splitPane" | "boxLayout" | "textPane" | "cardLayout" | "overlayLayout" | "textArea" | "popupMenu" | "flowLayout" | "hglue" | "scrollBar" | "spinner" | "comboBox" | "buttonGroup" | "hbox" | "gridBagConstraints" | "desktopPane" | "scrollPane" | "separator" | "vbox" | "passwordField" | "toolBar" | "radioButton" | "optionPane" | "radioButtonMenuItem" | "list" | "editorPane" | "vstruct" | "borderLayout" | "frame" | "internalFrame" | "label" | "progressBar" | "colorChooser" | "slider" | "glue" | "formattedTextField" | "panel" ) - So paste the above into your Groovy.nbs file and declare your new DOM3.xml file:
private static final String DOM3 = "org/netbeans/modules/mygroovyeditor/DOM3.xml";
Then add DOM3 to your getLibrary() method:
private static LibrarySupport getLibrary () { if (library == null) library = LibrarySupport.create ( Arrays.asList (new String[] {DOC, DOM0, DOM1, DOM2, DOM3}) ); return library; }And also declare a color for your Swing components:
COLOR:swingComponents: { foreground_color: "magenta"; font_type: "bold"; }And add the swingComponents keyword to the COMPLETION declaration:
COMPLETION:identifier, keyword, keywordGroovy, whitespace, operator, separator, comment, swingComponents: { text1:org.netbeans.modules.mygroovyeditor.Groovy.completionItems; confirmChars:";(,."; } - Install the editor again and now notice the following in the third template:
Notice how the Swing components have a distinct color and that you can call up code completion on them, in this case, in the screenshot above, on the 'panel' component.
In the editor, many others things can be added, such as actions, hyperlinks, and navigator support. Let's leave all that for another day. Clearly the basis of a Groovy editor is laid out above. Who will pick it up and run with it?
Aug 23 2007, 12:35:09 AM PDT Permalink
Monday August 20, 2007
Groovy Import Aliasing in the IDE
Aug 20 2007, 07:58:55 PM PDT Permalink
Friday August 17, 2007
Identifying Words in HTML Documents
If you study Tim's Wicket tag parser, which underlies the navigator mentioned in yesterday's blog entry, then you'll find it's not very hard to adapt it to your own purposes. For example, here's the same parser in action, but slightly modified. Here, instead of looking for Wicket tags, it looks for all words in the document and then prints them to the navigator:
Why might this be useful? Well, it's now a small step to a spell checker. The user would specify a file containing words, the words would be compared to the words in the document, and all the words that are not found in the file would be printed to the navigator. And those words, because they don't match the words in the file, would be the ones that are incorrect in one way or another. And that's all a spell checker should tell you, i.e., which words are incorrect. I've made a spell checker before, using annotations in the editor, but I haven't been able to find the code. Plus, I prefer this navigator approach to adding still more annotations to the editor. So, watch this space for HTML spell checker developments.
Aug 17 2007, 10:02:57 AM PDT Permalink
Thursday August 16, 2007
Wicket Tag Navigator
The Wicket suite for NetBeans IDE 6.0, discussed yesterday, contains (the start of) a Wicket tag navigator, as shown below:
Right now, nothing happens when you click a tag in the navigator. However, ultimately, the related HTML document will open and the cursor will land on the selected Wicket tag. From there, you can hold down the Ctrl key, move the mouse over the tag in the editor, and then the tag becomes a hyperlink, which you can click to open the corresponding Java document.
Aug 16 2007, 11:29:02 PM PDT Permalink
Wednesday August 15, 2007
When Boudreau Met Wicket...
The latest contributor to the NetBeans Wicket Support Project is Tim Boudreau himself. As a direct result, most of the 5.5 functionality is already available for 6.0, even though 6.0 doesn't officially exist yet. To check things out, go here and check out branch nb_60. Then open 'suite', which will give you this:
Run the suite. A new instance of the IDE starts up, installing the above two modules. Since you also have the source code of the modules, feel free to extend them with your own Wicket-related functionality. Here's what you'll find in the 6.0 version of Wicket support, thanks to Tim:
- There is now a suite.
- There is now a module which installs the following libraries:
- Wicket (with extensions).
- Wicket Security (some more extensions for using HiveMind).
- Jasypt - Very handy simple strong encryption.
- Refactorings are working again. Added a copy-class refactoring plugin, so if you copy a Java file with markup somewhere, the markup gets copied too.
- Hyperlinks work again, now using the Lexer API. Just hold down the Ctrl key while moving the mouse over a Wicket ID in an HTML file. You'll see a link that you can click to open the corresponding Java document.
- Deleted the hints, verification, etc. stuff for now. Planning to merge it back in from the trunk and get it working piece by piece.
- Removed a bunch of duplicate code and created a small API/SPI for a few pretty straightforward things:
- MarkupForJavaQuery - you can guess what these do - right now they just look for side-by-side html and Java files, but Wicket does let you do other layouts, so perhaps we can detect that and provide an implementation.
- JavaForMarkupQuery
- WicketProjectQuery - figures out if a project is using Wicket so any other functionality should be available.
- The Pizza example now lives in templates/pizza, and is rebuilt when you build the module.
Most interestingly, Tim added a much more extensive self-made sample project, in addition to the Pizza application. You'll find it in Samples | Web, together with the Pizza application:
What it is is a basic full blown web application with:
- User authentication - login, register and change password support
- SSL support for pages that need it
- Access control for pages based on whether the user is logged in or not
- Shows use of repeaters and panels
- ...and a bunch of other stuff
When you run it, you'll see the following in your browser:
These two samples, the Pizza application and the Login application, should set you up for working with Wicket, together with all the various project and file templates for which there is continuing support:
Hurray for Wicket users in NetBeans IDE. There's now more than enough functionality already available for 6.0 to get you up and started with Wicket! I doubt there is anything in the 5.5 Wicket tutorial that you cannot now already apply to 6.0, so go here for the tutorial. However, Tim's Login sample is the most interesting new piece in the current 6.0 story for Wicket, so have a look at that and see how everything fits together simply and coherently, which is the typical Wicket way.
Aug 15 2007, 05:59:17 PM PDT Permalink
Tuesday August 14, 2007
When Schliemann Met JFugue...
After the sudden and unexpected acceleration of developments in relation to the JFugue Music NotePad, I thought it might be time to... introduce the application to 6.0. In other words, we've been basing it on 5.5 thus far. Moving it to 6.0 would take a lot of work, no? In fact, no, it was a lot less work than I had thought. Then again, the application doesn't make use of much of the APIs that have undergone change. Specifically, if your application makes use of the editor-related APIs, there's a pretty good chance that there's some work ahead to convert the application to 6.0. The one and only area where the editor-related APIs are touched by the JFugue Music NotePad is... the syntax highlighting of the JFugue notation. That made use of the old NetBeans Syntax class, while 6.0 has Lexer... and... Schliemann...
Okay, so which one to choose? (In fact, someone asked me this very question yesterday.) Everything you ever wanted to know about 'Lexer' (what it is and what's so special about the NetBeans approach to it) is discussed here, in an interview with Mila Metelka, the author of the NetBeans Lexer API:
10 Questions about the new NetBeans Lexer API
Some of the content in the above interview is now out of date, but the principle statements there are still accurate. However, there's also 'Schliemann', the declarative approach to language programming. The core concept to understand in relation to Schliemann is—you don't need to be a brain surgeon to use this approach, whereas Lexer is still pretty complex, although much simpler than the old Syntax class. In fact, you don't even need to know Java. Read that again and let it sink in. That's why it's there, principally for scripting language authors, who might not even know Java themselves. These people need a rough and ready way to introduce their scripting language to the IDE, thus providing tooling support for their scripting language, which in turn will serve to popularize it. So, what happens when the world's simplest syntax coloring approach (i.e., Schliemann) meets the world's simplest language notation (i.e., JFugue)? Well, here's everything I needed to do to recreate my coloring in the Schliemann world:
# definition of tokens
TOKEN:instrument:( "I" "[" ["A"-"Z" "_"]* "]"* )
TOKEN:noteStart:( ["A"-"G"] )
TOKEN:noteEnd:( ["0"-"9"] "q" | "i" | "w" | "h" | "s" )
TOKEN:tempo:( "T" ["0"-"9"]* )
COLOR:instrument: {
foreground_color: "orange";
}
COLOR:noteStart: {
foreground_color: "blue";
}
COLOR:noteEnd: {
foreground_color: "red";
}
COLOR:tempo: {
foreground_color: "magenta";
}
By the way, the above is not a code snippet. It is the entire file. When I open the file, I see exactly what you see above, nothing else, not one additional character, is in the actual file. And the result? Well, look here:
Now, what are the benefits of this approach? Well, when I check in my changes, guess what, all of the Java classes, and one settings file, shown below, can be deleted:
That's great news—when debugging, I will not need to dig through 10 different classes to find my problem. Instead I will have one simple declarative file, which I can scan with the naked eye, without needing to translate from the Java world to my own non-Java brain. And why did I choose Schliemann instead of Lexer? Because Schliemann is faster to implement (as evidenced above, since no syntax coloring implementation could possibly be simpler than that). I simply need a MIME resolver and the above file, which I register in the XML layer, and I am done. However, although I can add a variety of other language features, they'll always be coarse grained. My understanding of the distinction between Lexer and Schliemann is that you can achieve a lot more with Lexer. For example, all the semantic analysis done in the new Java editor, as well as in the Ruby editor, isn't possible when you're using Schliemann. That's when you need Lexer. However, I imagine that Schliemann will get more and more powerful, in fact, it is already possible to refer to Java snippets from the above Schliemann file, so that you can create a lot more finetuned code for analyzing a syntax. However, for notations such as JFugue which have simplicity at its heart, Schliemann is a gift from heaven. No, not from heaven. From Sun.
In other news. I haven't committed my Schliemann implementation to CVS yet, so the application still runs on 5.5. Waiting to add some more Schliemann features, and doing some testing, prior to committing them, probably to a 6.0 branch of the JFugue Music NotePad.
Aug 14 2007, 10:50:45 PM PDT Permalink
Monday August 13, 2007
Get Started with NetBeans Platform 6.0
In Tim's Selection Tutorials Ported to 6.0, I announced that the 4-part selection series, by Tim Boudreau, is available for 6.0. However, before you follow those tutorials, you probably need to take some orientating steps through the NetBeans Platform tools and some of the main APIs. Now, the main NetBeans Platform tutorials are also available for 6.0:
- NetBeans Platform 6.0 Quick Start Tutorial
- NetBeans Platform 6.0 Paint Application Tutorial
- NetBeans Platform 6.0 Feed Reader Tutorial
The samples on which the above tutorials are based were written by some of the undisputed heros of the NetBeans Platform—Jaroslav Tulach wrote the HTML Editor, Tim Boudreau the Paint Application, and Rich Unger the Feed Reader. So, you are in good hands! Now these samples can also be built in 6.0, if you use the tutorials above. Best to follow them in the above sequence, because they increase in complexity. The differences between 5.5 and 6.0, in relation to these tutorials, is minor. However, the screenshots have been fixed (many places where there are small u.i. tweaks in 6.0) and, especially in the case of the Feed Reader tutorial, a lot of explanatory text has been added and some restructuring has been done.
All three of the above are in some kind of 'draft' state, although they should definitely provide the promised result (i.e., the code should be solid). The titles might change by the time 6.0 is released, but the URL to the tutorials should remain the same. Further explanatory texts need to be added, especially in the Feed Reader tutorial. Any comments are welcome, use the Feedback links at the end of each of the tutorials.
Aug 13 2007, 11:47:14 PM PDT Permalink
Sunday August 12, 2007
Significant Steps Forward for JFugue Music NotePad (Part 2)
I've added the print functionality described here and here. As a result, when you have this in your music sheet...
...and you then choose 'Print' under the File menu, you will be shown this print preview:
And from there, you should be able to print it, by choosing 'Print' in the print preview dialog above. However, because of the second rule for a paperless office (as described here) I am unable to test out the print functionality... :-)
In other news. I forgot to mention yesterday that Michal added 'Delete' funtionality, so that a selected note/s can be deleted, simply by pressing the Delete key on your keyboard. And I'm looking at adding functionality that will increase/decrease the duration of selected note/s, after the user presses the left/right arrow keys on the keyboard. For example, if the selected note is a quarter note, and the user presses the right arrow key, the note will become a half note. It's basically working, but not very elegant code yet.
Aug 12 2007, 03:58:04 AM PDT Permalink
Saturday August 11, 2007
Significant Steps Forward for JFugue Music NotePad (Part 1)
One of the biggest issues in the free and open source JFugue Music NotePad has always been the fact that one wasn't able to move notes after dropping them. That, all on its own, made the application unusable in real world scenarios. But... guess what? Someone called Michal Fapso, studying Graphical User Interfaces in Java at the Faculty of Information Technology in Brno in the Czech Republic, came across the project. And he had chosen to work on a music score editor for a term project. Instead of starting from scratch, he proposed to start with the existing JFugue Music NotePad and... enhance it with editing capabilities. His project instructor agreed and now...
...you can select a note (or multiple notes, by using the Ctrl key), which results in the selected notes turning red. Then you can press the up and down arrow keys to move the selected notes up and down in the score! How cool is that? Here you see three red notes, which means I selected them for moving up and down with the arrow keys:
In addition, you can see above that Michal added flats and sharps (which weren't supported before) and I added a Save button (so that you can save your compositions). The Save functionality needs to be enhanced, but basically does the job for now.
I asked Michal to explain how the editing capabilities work and he summarized the story as follows:
- StaveListener.java. Created this class for selecting notes with mouse (holding CTRL key allows selecting more than 1 note) and moving notes with up/down arrow keys.
- Stave.java. Added code for setting the StaveListener.
- SingleStave.java. Added method getNoteAtPos(...) and in method paint(...), added code for drawing the key signature (sharps, flats), and filtering the selected notes with the "red filter", added method selectNotePrefixSymbol(...) for drawing sharps, flats and restores beside notes.
- Score.java. Added some hash tables and arrays for working with notes, added method setKeySignature(...) for setting the score's basic scale and for filling the arrays: nativeNoteValues, alteredNoteValues and restoreNoteValues (these arrays are used in SingleStave::paint method).
- SelectNoteImageFilter.java. Created the "red filter" used for selected notes.
And here's a pic of Michal, the newest contributor to the JFugue Music NotePad project on dev.java.net (click here):
Finally... guess what happened when he handed his project in to his instructor? He got full marks. 20 out of 20. Michal graduated this year and will continue with his PhD. Well done Michal, that sounds like music to my ears! About the JFugue Music NotePad, Michal writes: "It was really nice to work on this project. It is well designed and I had to do only some minor changes to the existing code for implementing the editing capabilities." That's great to hear Michal, and mainly due to the Belgian developer Pierre Matthijs, who added a lot of insight and structure (and lots of code) to the project after I open sourced a rather humble beginning. It is extremely cool to hear that this project has proved beneficial to someone in their studies, who has now also learned about the NetBeans Platform. Plus, very cool that the enhancements have been contributed back to the project. Hurray for everyone!
Aug 11 2007, 05:20:20 AM PDT Permalink
Friday August 10, 2007
Anonymous Class to Inner Class
One underhighlighted improvement in the upcoming NetBeans IDE 6.0 is that it will support keyboard users far more than before. When coding, I really hate having to break the flow of my activity by needing to call up dialog boxes and then having to click various things within them. Of course, there are ways of calling up those dialog boxes without using a mouse, but those keyboard combinations are a bit tricky and my life would be simpler if I could simply continue typing instead of needing to resort to dialog boxes. So, 6.0 comes to the rescue with its heavy focus on keyboard-based programming. Here's an example. You're adding an action listener to a JButton:
As one has been able to do in the past, one can let the IDE generate the required abstract methods, by simply clicking the hint that is shown above:
However, now I'm cleaning up my code and I don't want that class to be anonymous anymore. Instead, I want it to be an inner class. In the past, I'd have to call up a dialog box, click various things, and then the inner class would be created. In 6.0, I'll be able to simply select something in the anonymous class and then press Alt-Enter:
Then the small popup shown above appears. I simply press Enter again. And then... I have a new inner class:
And the anonymous class is transformed into a call to my inner class:
And if I don't like the name of my new inner class, I can simply click Ctrl-R over the name, and then I will be able to change the name, as well as references to it, in-line immediately, without any dialog boxes. Hurray!
Aug 10 2007, 05:45:16 AM PDT Permalink
Thursday August 09, 2007
Instant JChem: Platform for Chemical Applications
This week, an interesting article was published (here) about Instant JChem (IJC), which is an IDE for scientists. There's a lot of reason for celebration—IJC 2.0 has just been released. IJC is a complete out-of-box application for working easily and efficiently with chemical structures. The article quotes IJC lead developer Tim Dudgeon as saying: "Scientists want a powerful yet easy-to-use application that lets them sketch structures and run queries efficiently while insulating them from the complexities of how.” (Many animations of working with IJC can be found here.)
In other words, IJC is to scientists what NetBeans IDE is, in at least some respects, to developers... Especially when you look at the NetBeans Visual Web Pack (formerly known as Creator), the 'hiding of complexities' factor is very comparable to what IJC does for scientists. But, in fact, IJC is a lot more than that. Just like NetBeans IDE is built on top of an infrastructure that can be reused by other applications, IJC itself is also built on a reusable infrastructure. Now, before you think: "Ah, okay, this is another NetBeans Platform story," let's back up a bit, because this is not another NetBeans Platform story.
In short: IJC is, strictly speaking, not built on the NetBeans Platform. It is built on top of the IJC Platform. The IJC Platform consists of several different components, one of which is the NetBeans Platform. Here's a picture that sums it all up:
Interestingly, just like the NetBeans Platform, the IJC Platform can be extended too! Why would you want to extend it? For the same reasons as one would want to extend the NetBeans Platform. Either you are enhancing an application that is based on the IJC Platform (such as IJC itself, for example) or you are creating a completely new application on top of the IJC Platform. Both are possible. You can disassemble IJC because, just like NetBeans IDE, it is composed of modules. Then you can rebuild it, together with your own additions, thus forming your own new application.
And the IJC Platform provides three modules with APIs that you can reuse:
- Discovery Informatics Framework: Provides the data model and persistence tiers for applications that need to use chemical and biological data.
- Instant JChem Core: Provides the basic user interface for Instant JChem and through its API it provides the ability to extend the IJC user interface.
- UI Widget Renderers: Provides the basic set of visual components, known as widgets, which can be used in the form designer. All widgets implement APIs provided by the Instant JChem Core module and are plugged into the system using declarative XML layers. Some widgets share the rendering of data values and so make use of the UI Widget Renderers module, which contains shared renderer implementations.
And how do you get started using these three modules? Well, there's even a set of IJC API tutorials, to help you get started:
Aug 09 2007, 04:25:29 AM PDT Permalink