Melvin Koh's Weblog
I'm just a contractor
I'm just a contractor
Thursday May 10, 2007
Sun Grid Engine 6.1 - Name changes
Sun Grid Engine (note the N1 has been dropped) has a new version. The latest 6.1 version contain several new features, but the most anticipated feature for me is the new resource quota capability. We had customer asking for this features long time ago, and the only way to achieve it was to use plenty of scripting. Now, SGE 6.1 supports defining finegrain resource limits at user, queue and host level. For this feature, a new command "qquota" has been introduced. For more details about resource quota, see here.
Posted by melvin ( May 10 2007, 02:26:44 PM SGT ) Permalink Comments [0]
Friday March 02, 2007
Clustered JVM - Terracotta
Terracotta is an open source distributed shared object facility for Java, which allows multithreaded applications to run on clusters with minimal changes. It works with existing application servers and other web platforms, which makes distributing application loads across multiple nodes (JVMs) straightforward. It performs thread synchronization and even thread migration transparently for the user.
In addition to the runtime facilities, Terracotta provides a declarative approach to clustered software. That is, the programmer merely annotates which data members are shared. Likewise, the user may specify which methods contain critical sections, thereby creating a monitor.
The system architecture relies on a central server that stores the state of shared objects. Client nodes (JVMs) receive updates for objects currently in memory; thus, any data transfers occur only at the object level. For fault tolerance, the server itself may be clustered with one live and others in standby.
Best of all, it is open source. While I still don't think that Java is suitable for building HPC applications, the solution may be great for developing Grid and cluster management tools.
Thursday February 22, 2007
Java is not slow?
There is an article that debunks the rumor that Java is slow. This article suggests that the "myth" of C/C++ applications, as native binary, will always be faster than Java, is not true. To a certain extend I agree with the author - poorly written C/C++ app can perform much worse than a Java app. Now, Java is my favorite programming language, but as someone working in HPC, all I can say is that I'm not at all convinced.
Posted by melvin ( Feb 22 2007, 04:53:26 PM SGT ) Permalink Comments [4]
Tuesday January 23, 2007
Solaris 10 Applications List
Want to know if certain applications run on Solaris 10? Bigadmin now has a list of over 4000 applications that runs on Solaris 10 x64 or SPARC. You can even make a request to Sun if you can't find the applications you want in the list.
http://www.sun.com/bigadmin/apps/
Tuesday November 28, 2006
Interoperability - Open Standards
This morning I had my breakfast at macdonald. The macs in spore offers free wifi access, so I connected my Nokia E61 PDA phone to the free wireless network and read my mail as i sipped my coffee. A mail contain a link to a podcast, so I downloaded it to my phone. Then after breakfast, on my way in the MRT train to my office, i listened to the podcast i downloaded earlier and watched the mp4 video of an interview of andy bectolshiem, which I downloaded yesterday. After I reached office, I was able to sync my phone with my laptop thru bluetooth.
Its amazing how technology has improved over the years, but all these mean little if they cannot interoperate. Now I'm a strong advocate of open standards and software, which is why i was a little annoyed that, after typing this entry on my phone, i realized that it only suport msword format. Guess there's still some way to go before reaching full interoperability...
Posted by melvin ( Nov 28 2006, 11:13:37 AM SGT ) Permalink Comments [1]
Thursday November 23, 2006
Supercomputing 2006, Tampa
The 5days SC'06 conference last week was my first time, and I had a great time there. The exhibition was huge, and you can find many HPC vendors and research institutes demoing their products and projects. So obviously, I've learnt many new technology and met a lot of friends, old and new. Also not forgetting to mention that I collected many free gifts from the booths as well. ;-)
As I'm very much a software guy, I won't comment on the new and cool hardware stuffs. Sun made several announcements during the event, one of which is the opensourcing of 3 Grid Engine modules in Dec - ARCo, Windows Execution and the new Service Domain Management modules. The Service Domain Management module should be very useful, especially to those who run multiple Grid Engine clusters in their environment. Goto here for the announcement.
There are also many other interesting projects. Nortel had a demo that perform automated migration of VMs across different datacenters based on sensors. For example, you can have temperature sensors that monitor the heat of the servers. The workload on the servers can be migrated to other datacenters on the event that servers are overheating. Slides on the demo are available here. The Korean reseach lab, KISTI, also has two interesting projects: Grid ASP and Grid-based online Game Service. There are too many interesting stuffs for me to comment on, so if you do not want to miss out, be sure to attend next year SC in Reno.
HPC Consortium 2006, Tampa
I attending Sun HPC consortium and the Supercomputing 2006, both held in Tampa, last week. This time, the HPC consortium was held in a very wonderful golf resort, but too bad I don't play golf. As usual, we had many great presentations by our Sun folks, customers and partners. Josh Simons gave a very nice summary of the 3 days on his blog, but I just want to highlight something that I find very interesting, which is the Secure Grid Portal by HPCVL.
The Secure Grid Portal (SGP) is based on the integration of Sun Java Portal Server, Sun Secure Global Desktop (previously known as Tarantella) and the certificate authority system by Entrust. The neat thing about SGP is that being integrated with Secure Global Desktop, it is able to remotely launch GUI applications with the need to install any additional software other than a modern web browser. There is a public demo of the Secure Global Desktop for everyone to test. Perhaps the Sun Grid Compute Utility can do something like this for launching interactive applications.
Wednesday October 25, 2006
Project Blackbox
I just got back from my 2 weeks of annual army reservist training. I belong to the Signal vocation, you know, the people who carry radio sets with long antenna sticking out and aways the first sniper's target :p. Anyway, I realized that today, technology plays a vital role in modern warfare. I believe many modern army are using technology to speed up information update, e.g. monitor the battle progress in real time by the command center; command your troops using point and click like playing Command & Conquer (well, not quite there yet, but not totally impossible).
Now that I'm back, I came across something new in several of the blogs. I find it such a cool idea that I have to put it here: Introducing the "world's first virtualized data center", Project Blackbox. I say put it on a truck with a satellite connection and you got a moving command center! Well, probably not for the military, but what about a disaster recovery coordinating center? Think Katrina and Tsunami...
Posted by melvin ( Oct 25 2006, 02:19:05 PM SGT ) Permalink Comments [3]
Thursday October 05, 2006
IdM Standards
I've been looking at Identity Management (IdM) and Federation recently, which is why I created a new blog category. APSTC has joined a consortium for the EU SORMA Project and part of our task is to investigate and apply IdM technology to the architecture.
When it comes to IdM standards, first comes to mind will probably be SAML (Security Assertion Markup Language) by OASIS. Then there's also the Liberty Alliance organization that is also working on Id Federation specifications like ID-WSF, ID-SIS, ID-FF etc. Now, newbies to these standards (like me) will probably get very confused as Liberty and SAML has combined some of these specifications into the new SAML 2.0. Worse still, there are also other competing standards like WS-Federation. Not too long ago, Sun also opensourced the part of the code for Access Manager and named it OpenSSO. Then there are many IdM software, Shibboleth for example.
So with so many different standards and tools, how does one make sense of everything? And how does one ensure interoperability?
Posted by melvin ( Oct 05 2006, 02:22:40 PM SGT ) Permalink Comments [0]
Monday October 02, 2006
TACC Supercomputer
A great news to share with everyone, but seeing that this news has appeared in several other blogs in here, I guess this is old news by now. TACC has been awarded $59M by NSF for a huge Sun supercomputer that will consist of over 13,000 AMD opteron processors and 1.7 petabyte of storage. For more details, stay tune to Marc's Blog.
After reading all the news and blogs about this win, I was kind of a little disappointed. Not that I'm disappointed with the news, but rather on the fact that most of the info and questions people asked are mainly on the hardware. What is the processor? What network interconnect? How much memory/storage space? What about power and cooling? etc. Of course I won't deny the fact that the hardware architecture is the most important factor, but I just don't think that Sun is chosen just because of that, and also because Sun is able to offer much more in terms of HPC Solution -- software included. We have great management tools like N1 Grid Engine and N1 System Manager. Our development tools, Sun Studio compilers and performance analyzer etc, are excellent for HPC. Also not to mentioned that Sun has many partners, CoEs, collaborators who are well-known experts in their own HPC fields, experts who can help Sun to value-add to our customers.
Posted by melvin ( Oct 02 2006, 02:09:53 PM SGT ) Permalink Comments [0]
Tuesday September 19, 2006
N1GE Scheduler
Stephan Grell from the N1GE team has left Sun. Although I've never met him, but I've read a lot of his postings in the mailing list, always very helpful and knowledgeble especially in the N1GE scheduling. Stephan posted plenty of N1GE tips in his blog, so before his blog account gets deleted, I'll post of his past entries here.
From Stephen Grell's profiling blog entry:
N1GE 6 - Profiling
The Grid Engine software provides a profiling facility to determain where the qmaster and the scheduler spend their time. This has been introduced long before the N1GE 6 software. With the development of N1GE 6 it was greatly improved and its improvement continued over the the different updates we had for the N1GE 6 software. It was used very extensivly to analyse bottlenecks and find missconfigurations in existing installations. Until now, the source code was the only documentation for the output format, which might change with every new udpate and release. Lately a document was added to the source repository to give a brief overview of the output format and the different switches. The document is not complete, though it is a good start.
Profiling document
Posted by melvin ( Sep 19 2006, 11:29:25 AM SGT ) Permalink Comments [0]
Wednesday August 30, 2006
ZFS Get Started Guide
Thomas Nau from University of Ulm did a very good presentation of ZFS during the last HPC Consortium held in Singapore. For those who are looking for simple docs to get started quickly on ZFS can check out his presentation slides.
Posted by melvin ( Aug 30 2006, 03:58:40 PM SGT ) Permalink Comments [0]
Solaris 10 Containers and Resource Management
Some neat video demos of configuration and administration of Solaris Containers with resource management.
http://www.posix.brte.com.br/blog/index.php?page_id=30
Posted by melvin ( Aug 30 2006, 03:49:10 PM SGT ) Permalink Comments [0]
Sun Grid: The next evolution?
In one of his blog entry, Marc wrote about the next version of the Sun Grid with many new and cool technology to be added to the infrastructure. In APSTC, we are working on this project call "Grid Market Framework" which is our vision of the what the future Sun Grid will look like. To explain what is this vision about, let me give an analogy:
In Singapore, we have eating places which we call hawker centers. These hawker centers are food centers which have different food stalls offering many variety of food in a single location. The typical business model is that each food stall owners lease the space from the hawker center owner for a fix price, they are largely free to choose their own menu and set their own prices (with certain guidelines). These hawker centers are hugely popular among the locals because there are many food to choose from and it's cheap because of competition.
So how does this relate to our vision? Imagine that the hawker centers are the compute resources, owned by Sun, the food stall owners are ISVs and the food the application. Our vision is a Grid Market where ISVs can sell their applications as services and leasing the compute resources from the Sun Grid. Customers of the Sun Grid can select the services they want to buy, and using our cool Sun technology, the purchased service will be provision to the compute resource with the required OS, patches, packages etc. and ready to use almost immediately. Customer pays ISV for the service, ISV pays Sun for the compute resource.
Too far-fetched you say? To a certain extend I do agree, there are still alot of gaps to fill, but as we've seen from saleforce.com and google, I don't think it will be too long until we see something like this. So if there is anyone who can do it, surely Sun can.
Posted by melvin ( Aug 30 2006, 03:36:29 PM SGT ) Permalink Comments [0]
Wednesday August 02, 2006
Manual display of table contents in Java Studio Creator 2
I'm playing around with the new Java Studio Creator 2 recently and find that it is able to perform alot of powerful stuff very easily. For example, to display the content of a database table, just create a JSF table component, from the datasources drag the database table to the table component and you're done! The JSF table component should now list the content of the database table with the columns representing each fields. To customize the table, just click on the column and modify the properties or edit the JSP codes if you're programming inclined.
All these is well and good but what happen if the content is not stored in a proper database? I wanted to display the content of a textfile in the JSF table and it took me a while to figure how to do that. The trick is to use either a Object List or Object Array Data provider. Firstmost, I drag-n-drop an ObjectListDataProvider to my page. Then I need to create a javabean wrapper class for my data in the textfile. In the page init(), I do whatever I need to read the first line of datafile, transfer to the bean object, throw it into an ArrayList, and repeat for each line until EOF. Next assign the ArrayList to the ObjectListDataProvider by doing:
objectListDataProvider.setList(arrayList);
Finally I need to bind this data provider to the JSF table. I switched to the JSP view, locate the table code and change the sourceData="#{....}" to:
<ui:tableRowGroup binding="#{main.tableRowGroup1}"
id="tableRowGroup1" rows="10"
sourceData="#{page1.objectListTableDataProvider}"
sourceVar="currentRow">
For each column, I also have to change the text="#{currentRow.value['xxx']}" by replace xxx with the variable name of the javabean class, then I'm done!
This is a personal weblog, I do not speak for my employer.
