Jennifer's Weblog

« Communicating Combob... | Main

Friday May 16, 2008

Server-side image processing with JRuby-on-Rails and the Java 2D API

One of the many advantages of creating Rails applications with JRuby is the access that JRuby gives you to the rich set of Java libraries available in the Java platform.

This blog entry provides step-by-step instructions for creating a simple Rails application that uses the Java 2D^TM API to perform server-side image processing. It also shows you how to install and use the GlassFish^TM v3 Gem, which contains only the GlassFish v3 kernel, Grizzly, and other utilities, thereby giving you an application server with a smaller size and a faster start-up time.

Although Rails is intended for developing database-backed web applications, this application does not use a database. Many think that it is better to use the file system rather than a database to store binaries. I'll leave it up to you whether you use a database or not for that. In any case, this blog entry focuses on taking advantage of JRuby to access Java platform libraries from a Rails application.

Installing the Software

To get started, go through these steps to install JRuby, Rails, and the GlassFish v3 Gem if you haven't already.

Download jRuby-bin-1.1.zip from jruby.codehaus.org and unpack the zip file.
Add the path to your jRuby installation to your system path.
Install Rails on your JRuby VM by running this command:
```
jruby -S gem install rails
```
Install the GlassFish v3 gem on your JRuby VM by running this command:
```
jruby -S gem install glassfish
```

Creating the Rails Application

Now, let's set up the application like you would with any JRuby-on-Rails application.

Go to <JRUBY_INSTALL>/samples.
Generate the application's directory structure:
```
jruby -S rails photo
```
Go to the photo directory you just created.

Generate a controller and a default view:

jruby script/generate controller home index

Tell Rails that you are not using a database for this application by doing the following:
1. Open the <JRUBY_INSTALL>/samples/photo/config/environment.rb file in a text editor.
2. Remove the hash mark (#) from line 21 so that it reads:
```
config.frameworks -= [ :active_record, :active_resource, :action_mailer ]
```

Setting Up the User Interface

This application has a simple UI that displays an image, a combobox, and a button. The user selects an image-filtering effect from the combobox and clicks the button. When the user clicks the button, the application performs an image processing operation on the displayed image to produce the effect and opens a new page that displays the processed image. Here's a screenshot of the first page:

To set up the UI, do the following:

Copy a JPEG image to <JRUBY_INSTALL>/samples/photo/public/images.
Open <JRUBY_INSTALL>/samples/photo/app/views/home/home.html.erb in a text editor.
Replace its contents with the following:
```
<html>
<body>
<img src="../../images/kids.jpg"/><p>
<% form_tag :action => 'seeimage' do -%>
<%= select_tag "operation",
        "<option selected='selected'>Grayscale</option>
        <option>Negative</option>
        <option>Brighten</option>
        <option>Sharpen</option>
%>
<<% end -%>
</body>
</html>
```
Here you're using the form tag and select_tag helpers that Rails provides. It's a good idea to use the helpers rather than to write this HTML by hand because the helpers take care of a lot of extra stuff you'd have to add if you wrote the HTML yourself. When you run the application, just view the source of the page and you'll see what HTML the helpers generate for you. Check out Ruby on Rails Manual ActionView::Helpers for more information on the various helpers for views.
From the combobox on this page, the user can select from four different image effects: Grayscale, Negative, Brighten, and Sharpen. The selected name of the effect is saved into the operation variable and is passed as a request parameter to the seeimage action of the controller. The action will use this request parameter to execute the appropriate image-processing code.

Adding the Image Processing Code to the Controller

This application gives you a good overview of how to access Java libraries from a Rails application using Ruby code. It also includes a nice sampling of the image-processing operations available in the Java 2D API.

While adding the Ruby code that performs the image processing to the controller, you'll learn the following concepts involved in using Java libraries in a Rails application:

Giving your controller access to Java libraries
Referring to Java classes
Performing file input and output using the java.io and javax.imageio packages
Assigning Java objects to Ruby objects
Calling Java methods and using variables
Converting arrays from Java language arrays to Ruby arrays
Streaming files to the client

You'll also learn the following concepts associated with performing image processing with Java 2D:

Reading an image file into a buffered image so that you can process it.
Obtaining a Graphics2D object from the buffered image so that you can draw the processed image.
Using the different image operations offered by Java 2D
Filtering a buffered image through one of the image operations
Writing the processed image to a byte output stream so that you can stream it to the client.

For your convenience, I've included a copy of the controller online here: home_controller.rb.

Giving the Controller Access to Java Libraries

To allow your controller to use Java libraries, perform these steps:

Open <JRUBY_INSTALL>/samples/photo/app/controllers/hello_controller.rb in a text editor
Add the following line right inside the HomeController class declaration:
```
include Java
```

This one line is all you need to access Java libraries from your controller.

Referencing Java Classes

The photo example uses constants to reference Java classes it uses frequently.

To add the constants you need for this application, do the following:

After the include Java statement, add the following constant declarations in your controller:
```
  BI = java.awt.image.BufferedImage
  CS = java.awt.color.ColorSpace
  IO = javax.imageio.ImageIO
```
Now you can use the constant to reference the class later, as shown by this line:
```
bi2 = BI.new(w, h, BI::TYPE_INT_RGB)
```

In addition to using constants, you have three ways to reference a Java class from Ruby code:

Use the familiar import statement:

import java.awt.image.BufferedImage
...
bi2 = BufferedImage.new(w, h, BufferedImage::TYPE_INT_RGB)

Include the class using the include_class statement:
```
include_class 'java.awt.image.BufferedImage'
```

Reference the fully-qualified name of the class when invoking its methods:

filename = "#{RAILS_ROOT}/public/images/kids.jpg"
file = java.io.File.new(filename)

Creating the Actions Needed in the Controller

index.html.erb

index

seeimage

seeimage.html.erb

index

seeimage

Inside the HomeController class declaration and after the constants you added in the previous section, add a seeimage action:
```
def seeimage
end
```
After the seeimage action, add an index action:
```
def index
end
```

Getting Request Parameters

param

The operation request parameter has the value the user selected from the menu on index.html.erb. To get the value of operation, do the following:

Inside the seeimage action, read the value of the operation request parameter into a variable called @data:
```
   @data = params[:operation]
```

Reading the Image File Into a Buffered Image

BufferedImage

Inside the seeimage action, right after the assignment of the operation request parameter into the @data variable, add the following code:

filename = "#{RAILS_ROOT}/public/images/kids.jpg"
imagefile = java.io.File.new(filename)
bi = IO.read(imagefile)
w = bi.getWidth
h = bi.getHeight
bi2 = BI.new(w, h, BI::TYPE_INT_RGB)
big = bi2.getGraphics
big.drawImage(bi, 0, 0, nil)
bi = bi2

Reads the image file into a File object.
Uses the ImageIO class to store the image file into memory as a BufferedImage object so that you can perform operations on it.
Creates a new BufferedImage with the preferred size and bit-depth to facilitate image processing.
Creates a Graphics2D object from the new BufferedImage object so that the graphics context, or drawing surface, has the proper size.
Uses the Graphics2D object to draw the original buffered image to the graphics context.
Saves the new buffered image into the original one.

As you can see, referencing Java classes and methods from Ruby code is not much different from doing it from Java code. Notable differences are the following:

You do not need to declare any types when using Ruby code. Ruby can infer the type based on the return value of the method call or the method's argument list. For example, Ruby can tell that bi is a BufferedImage object because that's what the read method of ImageIO returns.
You don't need to add parentheses to a method call when it takes no arguments.
You don't add semicolons to the ends of method calls.
You use nil instead of null to represent a null value.
You use a double colon in between the class name and the field name when referencing static values, such as when referencing the TYPE_INT_RGB field of BufferedImage:
```
BI::TYPE_INT_RGB
```

Creating a Filter That Can Produce the User's Chosen Effect

@data

case

@data

case

BufferedImageOp

Using Java 2D's Image Processing Model

This section goes into some detail about Java 2D image processing. If you're more interested in using Java libraries with Ruby code rather than the Java 2D API, just look for the Ruby_Info tag delimeters.

After the code to save the image into a buffered image, create a variable to hold the image filter:
```
op = nil
```

Add the following case statement:

case @data
   when "GrayScale"
      colorSpace = CS.getInstance(CS::CS_GRAY)
      op = java.awt.image.ColorConvertOp.new(colorSpace, nil)
   when "Negative"
      lut = Array.new
      for j in 0..255
         lut[j] = 256-j
      end
      jlut = lut.to_java :byte
      blut = java.awt.image.ByteLookupTable.new(0, jlut)
      op = java.awt.image.LookupOp.new(blut, nil)
   when "Brighten"
      op = java.awt.image.RescaleOp.new(1.4, -25, nil)
   when "Sharpen"
      data = [-1, 0, -1, 0, 5, 0, -1, 0, -1]
      dataFloat = data.to_java :float
      sharpen = java.awt.image.Kernel.new(3, 3, dataFloat)
      op = java.awt.image.ConvolveOp.new(sharpen)
end

<Ruby_Info>

case

switch

</Ruby_Info>

This case statement has a lot going on. Let's take it one piece at a time.

Converting the Image to GrayScale

case

ColorConvertOp

The Java 2D API provides a set of color spaces, such as CS_GRAY and CS_CMYK. You just need to create a new ColorConvertOp instance and give it your chosen color space.

Creating a Negative of the Image

LookupOp

The LookupOp class uses a lookup table to filter the color values of pixels from a source image to a destination image. A pixel's color is made up of three components: red, green, and blue, each of which is represented by a value within the 8-bit range, 0-255.

To produce the negative of an image, you need to create a lookup table that has the values 0-255 in the reverse order so that each pixel's color will be set to the color's complement, as the following for loop does:

lut = Array.new
for j in 0..255
  lut[j] = 256-j
end

<Ruby_Info>

The preceding code uses a Ruby array and a for loop. As with other variables in Ruby, you don't need to declare the type of the array, nor do you need to initialize it to a certain length. Same thing with the for loop: you don't need to initialize the iteration variable. And you don't need to explicitly increment it either. Finally, to indicate the range for the for loop, you just give the starting value and ending value of the iteration variable, separated by two dots.

After the for loop exits, you have a Ruby array. What you need to do is convert it into a Java array so that you can use the array with Java libraries. To convert the array, you use the to_java function and indicate the type that you want to assign to the array:

jlut = lut.to_java :byte

</Ruby_Info>

Now that you have converted the Ruby array to a Java array, you can use it to create a lookup table and pass the lookup table to an instance of LookupOp:

blut = java.awt.image.ByteLookupTable.new(0, jlut)
op = java.awt.image.LookupOp.new(blut, nil)

Brightening the Image

RescaleOp

op = java.awt.image.RescaleOp.new(1.4 -25, nil)

Sharpening and Edge-Detection

The way you perform convolution with the Java 2D API is by creating a Kernel object and then using it to construct a ConvolveOp object. The photo example uses a kernel that causes a sharpening effect:

To create the filter that will perform this sharpening effect, you would use the following code:

data = [-1, 0, -1, 0, 5, 0, -1, 0, -1]
dataFloat = data.to_java :float
sharpen = java.awt.image.Kernel.new(3, 3, dataFloat)
op = java.awt.image.ConvolveOp.new(sharpen)

Kernel

The following matrix would give you the edge-detection effect:

[1, 0, 1, 
 0, -4, 0, 
 1, 0, 1]

Filtering the Image

dest = op.filter(bi, nil)
big.drawImage(dest, 0, 0, nil);

op

dest

Streaming the Image File to the Client

read

ImageIO

write

Add the following lines to finish up the example by streaming the filtered image to the client:

os = java.io.ByteArrayOutputStream.new
IO.write(dest, "jpeg", os)
string = String.from_java_bytes(os.toByteArray)
send_data string, :type => "image/jpeg", :disposition => "inline", :filename => "newkids.jpg"

send_data

If you prefer to save the image to a file rather than a stream and save the file to disk, you can use send_file instead of send_data:

send_file writefilename, :type => 'image/jpg', :disposition => 'inline'

Running the Application

Deploy the application on the GlassFish v3 GEM:
```
jruby -S glassfish_rails photo
```
Run the application by entering the following URL into your browser:
```
http://localhost:3000/home/index
```
Select an image filtering operation from the combobox and click Submit.
After the filtered image is displayed in the browser, click the browser's Back button to return to the previous page if you want to filter the image again.

That's all there is to it. For more information on JRuby, Ruby-on-Rails, and the Java 2D API, visit the following links:

Posted at 01:15PM May 16, 2008 by jenniferb in Sun | Comments[20]

Comments:

[Trackback] One of the main advantages of using JRuby-on-Rails instead of Ruby-on-Rails is to harness the power of Java libraries available in the Java platform. The Java 2D API is a set of classes for advanced 2D graphics and imaging,...

Posted by Arun Gupta's Blog on May 19, 2008 at 06:31 AM PDT #

This entry inspired me to add some methods to image_voodoo (jruby-extras) imaging processing library. I also added a blog entry showing how it could do something similiar to what you did in this entry:

http://www.bloglines.com/blog/ThomasEEnebo?id=50

-Tom

Posted by Thomas E Enebo on May 20, 2008 at 08:14 AM PDT #

Hi Thomas,

Thanks for your feedback. We writers always like it when we can help improve the quality of software.

Jennifer

Posted by Jennifer on May 21, 2008 at 05:18 PM PDT #

Hi Jennifer, great article!

I'm coming from the Ruby side but it's very informative to see what you think needs to be explained to people coming to this from Java.

Regarding Rubyisms .. here's a simple way to create the lut table -- I like how it reads better than the for loop.

lut = (0..255).to_a.reverse

creates this Array:

[255, 254, 253, ... 0]

You probably know this but I thought some of your readers might enjoy the idea.

Explanation:

0..255 is a range object

The Range class mixes in all the methods defined in the Enumerable Module including the method to_a (convert to an array).

Parens need to be put around the range object in order for the parser to properly recognize the intended receiver for the 'to_a' message. Without parens the parser will assume the receiver is the number object 255.

Then the message 'reverse' is sent to the new Array object which reverses the order originally generated when the range object was converted to an Array.

Posted by Stephen Bannasch on May 21, 2008 at 08:22 PM PDT #

Thanks, Stephen. No, I didn't know about that easier way to create the lut table. I'll have to mention that in the entry when I get a chance. I'm actually very new to Ruby. I owe a great debt to folks on the #jruby IRC channel for helping me out.

Posted by Jennifer on May 22, 2008 at 02:51 AM PDT #

<a href="http://rizo.codiego.com/">リゾートバイト</a> <a href="http://elbruz.kind7.biz/">自動車保険</a> <a href="http://pastel.aodiego.net/">ヒルズコレクション</a> <a href="http://scalp-d.aodiego.net/">スカルプＤシャンプー</a> <a href="http://hillz.aodiego.net/">ヒルズダイエット</a> <a href="http://xn--n-pfuj3b1czif2l.seesaa.net/">パステルゼリー</a> <a href="http://xn--pckhnj3jpfxak0e.269g.net/">ヒルズコレクション</a>
<a href="http://xn--pckhnj3jpfxak.seesaa.net/">ヒルズダイエット</a>

Posted by ken on June 02, 2008 at 08:29 PM PDT #

assas

Posted by 59.90.19.42 on August 23, 2008 at 12:01 AM PDT #

Good collection and submission!

Regards

SBL - BPO Services|www.saibposervices.com

Posted by BPO Services on October 28, 2008 at 04:19 AM PDT #

[http://www.ireneblea.com/kinnsi/ 錦糸眼科の評判・口コミ・うわさ]
[http://www.sostips.com/kinnsi/ 錦糸眼科]
[http://www.wolfpackden.com/ 基礎代謝]
[http://www.2007senkyo.jp/pc/re-sikku/ レーシック失敗]
[http://www.2007senkyo.jp/pc/fx/ FX比較]
[http://www.2007senkyo.jp/pc/gold/ ゴールドカード]
[http://dental-job.jp/re-sikku/ レーシック]
[http://dental-job.jp/goldcard/ ゴールドカード]
[http://dental-job.jp/fx/ FX初心者]
[http://dental-job.jp/kinnsi/ 錦糸眼科]
[http://dental-job.jp/kanagawa/ 神奈川クリニック眼科]
[http://www.2007senkyo.jp/pc/kinnsi/ 錦糸眼科]
[http://www.2007senkyo.jp/pc/kanagawa/ 神奈川クリニック眼科レーシック]
[http://www.chinagate.jp/kinnsi/ 錦糸眼科]

Posted by mikkyo on November 13, 2008 at 10:23 PM PST #

SEO (or come grinding engine optimization) is to rewrite the web page to appear higher in search results for a particular search engine. Its technology. Also known as search engine optimization. The English "Search Engine Optimization" to take initial measures which are known as the SEO. Search engine optimization has become a target, Google often. The foreign (especially American) by the high market share in Google. In Japan, Yahoo! For many users of the search, Yahoo! Is focused search measures.

Posted by SEO on January 28, 2009 at 06:24 PM PST #

http://www.gunow.net
http://www.vangsof.com
http://www.voview.org
http://www.wampipirik.com
http://www.waroom.org
http://www.totown.org
http://www.abduet.com
http://www.pustudy.com
http://www.webnot.org
http://www.cufire.com
http://www.wfolk.net
http://www.fufire.com
http://www.ceratin.net
http://www.srisen.com
http://www.nabody.com
http://www.kenmind.com
http://www.byobizz.org
http://www.strykertrauma.biz
http://www.bchour.com
http://www.vofolk.com
http://www.vostaff.com
http://www.chartbc.com
http://www.hupoint.com
http://www.om-shiv.com
http://www.simgle.org
http://www.ikyo9.com
http://www.fisz.info
http://www.pudepot.com
http://www.hippc.net
http://www.gprekyba.com
http://www.exdale.com
http://www.joicity.com
http://www.ignightseattle.com
http://www.enmage.com
http://www.tjzhuyou.com
http://www.futsubay.com
http://www.consfry.com
http://www.wupoint.com
http://www.XJXUELIAN.COM
http://www.wuquick.com
http://www.xiday.org
http://www.kocable.net
http://www.lifairy.com
http://www.enjoywd.com
http://www.bctroop.com
http://www.popeo.net
http://www.kennkoou.com
http://www.kodisk.org