Sanjiv Jivan's Blog

SmartGWT is arriving soon!

I'd like to post an update on the progress of SmartGWT. Things are coming along great and users can expect a rather full featured alpha release soon. I've been very pleased working with the SmartClient API's. SmartClient has a very complete API and during the development of SmartGWT things just worked! No corner cases, quirks or missing API's. A good percentage of SmartGWT is being generated from metadata and uses GWT 1.5 features like Overlays where appropriate so the alpha release is going to be quite stable and perform well.

To pique your interest, I'm posting a screenshot of the SmartGWT Showcase which is still being worked on.

There are a whole lot of cool features that I'm anxious to share once the alpha is out but the one feature I can't resist mention is SmartClient's Developer Console. It is an inbuilt souped-up "Firebug" like console that users can fire up and examine the SmartClient widget hierarchy (as opposed to the browser DOM), select methods on SmartClient classes to profile or observe, examine render times, change log levels dynamically, DOM inspection and much more. Click on the image below for a more detailed view.

This really kicks ass and works in all browsers, which means it can be used for troubleshooting even in GWT's hosted mode.

One last think I'd like to mention is the SmartClient Look & Feel. A few of the older skins looked a little dated however the SmartClient folks have created a few slick looking new skins and have more coming as well. What do you think of them? Feel free to post your comments / suggestions.

Unfortunately I will not be making the Ajax Experience in Boston next week, but the Isomorphic / SmartClient folks will be demoing SmartGWT there so be sure to check it out.

SmartGWT : GWT API's for SmartClient

I have been keeping a close eye on SmartClient for a while and have been very impressed by their library. A few years ago SmartClient had a commercial-only offering but have now adopted an OSS model offering their library under standard LGPL 3 terms with no convoluted or ambiguous usage terms.

It has a very comprehensive and complete API with a consistent component model. It handles full-cycle databinding, cache management and automatic cache updates, and adaptive use of client-side filtering and client-side sorting that kicks in once a dataset has been filtered down so that it fits in cache. All of this works with any data provider. Some other cool features include WebService bindings, TreeGrid and OLAP / Cube Grid.

I've mentioned before that I feel that events of the past are good indications of the future. SmartClient has clean and credible track record.

You will learn a fair bit about SmartClient by browsing through their forums.

After the Ext foul play of switching from LGPL to the much more restrictive GPL license, the folks at SmartClient contacted me saying that SmartClient is available under standard LGPL terms and asked if I would like to get involved in bootstrapping an OSS project that provides GWT API's for SmartClient. They even provided a signed document stating

I agreed! I have started work on SmartGWT and things are coming along quite well. The project will be hosted on Google Code and be open to other contributors.

My involvement with gwt-ext will not change and I will continue to support it and be involved as I currently am. SmartGWT just presents users with yet another viable open source commercial-friendly option.

SmartClient has adopted the OSS model of providing their library under the commercial friendly LGPL 3 terms and has other services and optional productivity tools they charge for. This is the model that has been adopted by other popular OSS projects like Spring and Hibernate. By making an official statement that they are not going to switch licenses one can rest assured that this company is going to operate with integrity in supporting the users of its LGPL library.

Remember The Milk : One of my fav Ajax applications

I was just reading the Ajaxian conference schedule that is being held in Boston (Sept 29th, 2008) and was pleasantly surprised to see the session Building a Highly Scalable Ajax Application: Lessons from Remember The Milk (RTM) by Omar Kilani.

For those not familiar with Remember the Milk, it is a to-do task management app. However it's not just another todo app - it is an extremely well written app that truly encapsulates the essence of Web 2.0 and has been one of my favorite Ajax apps from the time they first launched in 2005. They were well ahead of the Web 2.0 game and had usability concepts and technical stuff that became mainstream "patterns" only much later and even today they introduce ideas and incorporate technologies in a way that is very intuitive (think gmail).

So even if you aren't interested in a to-do management app, I'd recommend that you register and use their app for a few days. I'm confident you that they'll be a few things that will make you go - "wow, this is really cool" and you can think of ways that your application can benefit from it.

Here are some of the things that are really neat about RTM. I'm sure many of you are familiar with some of the features / patterns listed here, but RTM really makes a good use of them in one application.

Registration Screen : I had previously blogged about the importance of an intuitive and easy-to-use registration screen and RTM has one of the best ones I've come across. Email triggered actions. On registration, you are sent a private RTM specific email eg : sanjiv.jivan+455014@rmilk.com. So sending an email to this rmilk email address will automatically be added to your tasks. This is a quick and easy way to add a TODO without having to log into RTM. Tags. Noting new here but it makes a good use of tags.
Undo pattern. Use of Undo instead of warning prompt prior to action. Similar to GMails delete email etc.
Smart Dates like "next Friday or in 2 weeks". Mad Cool Date is a library that supports such date specifications but this feature has been part of TM for a while. FireBug detection. Gmail now does this, but RTM has had FireBug detection warning users of potential slowdown prior to Gmail (AFAIK) A very innovative way of integrating the RTM tasks with Gmail, Google Calendar. and Google Gadgets. Offline support via Gears. RTM was quick to have a really nice support of offline use and synchronization via Gears - the first nice implementation of Gears that I know of after Google Reader. Smart Search. A simple text box that allows the user to to a full text search across all entities (like tasks, inbox, etc). This is another pattern that is gaining popularity over having an elaborate search dialog. One can add special prefixes to the searches to limit search (like gmail's search textbox). Tabbed Interface: Tabbed Interfaces are getting increasingly popular with Ajax apps as it allows easy access to multiple views with minimal page-to-page navigation. Message Bus. Tibco's Pagebus does just his but RTM has had an implementation of publish-subscribe of "messages" in their web app for inter-component communication from the very beginning. I learnt a lot by understanding how they have implemented and used this technique very effectively. Map support: Using Google Map or a Yahoo Map in one's application has become relatively straightforward these days but RTM users this very effectively in a way that integrates the RTM functionality very naturally with the use of Maps. Check it out yourself. Use of bookmarklets to add tasks to your todo list from any website. Excellent integration with Instant Messengers (for reminders), RSS / Atom feeds, iPhone UI An API to interact with RTM functionality

The above is not a comprehensive list of RTM's features and you're likely to find other cool features as you work with the app. RTM is a good example of hiding complexity with the end goal of making things easy and intuitive to the end user much like what several google apps accomplish. I haven't had the pleasure of meeting Omar Kilani but I tip my hat to him. If you are attending Ajaxian's conference don't miss out on his session.

New GWT-Ext commiters

The GWT-Ext team will comprise of project owners and project members. Project owners will be part of the coding, prioritization and decision making process and project members will have commit access. Project members can grow into a project owner role.

I am pleased to announce that Mario Lim and Abhijeet Maharana are now part of the GWT-Ext project owners. Andrew Gillett is a project member and is welcome to join the owners group at any time. Rob Smith joins Matthew Lieder as a project member.

All the above users have made significant contributions (gwt-ext-ux, tutorials, helping users), demonstrated knowledge of GWT-Ext and most importantly willingness to learn and help others. Please join me congratulate these users!

Every user can contribute by helping other users, submitting bug fixes and patches. Users new to the project can start by contributing enhancement / bug fix patches (after sending in a CLA). If you're interested in taking up a bigger role, please send me an email.

Update on future direction of GWT-Ext

In light of ExtJS going GPL, I'd like to provide an update to GWT-Ext users on the future direction of the project. While many users would like to see GWT-Ext continue to provide an LGPL stack with Ext 2.0.2, a few others felt that GWT-Ext should just go ahead and support the GPL versions of ExtJS.

One of the early adopters of GWT-Ext sent me an email yesterday voicing his concerns saying that since GWT-Ext fully depends on ExtJS, moving to support Ext GPL would be the right thing to do. He goes on to say that "..licensing is a big deal, but to most people who are using the ext product, GPL is not a bad change. The biggest hits will be the big companies that are using it, not the little people. GPL for these bigger companies is perfectly fine."

I value his opinion and I'm sure some others feel the same way, but I have a different view on this matter. There are two aspects to ExtJS going GPL :

There are users that are going to be disappointed no matter what direction GWT-Ext takes. Many of them feel that GWT-Ext should move along with Ext GPL, and others who have equally invested time in GWT-Ext only because they knew it was available under LGPL terms.

The bigger picture is that libraries come and go and the shelf life of libraries and tools are limited. Take for example Struts, EJB 1.x, Symantec Visual Cafe and many others. Software evolves and technology choices change by the day and whats best today might not be even relevant in a year or two. For all we know a company like Google will make GWT-Ext / MyGWT / Ext GWT / library X irrelevant sooner than we think.

At the end of the day to me its about doing the right thing rather than the myopic view of what is the most effective strategy for the next 6-12 months.

Software is all about choices and I realize that each person has to make a decision based on their schedule / requirements and so forth. GWT-Ext today is there to present users with an option to work with an LGPL stack and only supports Ext 2.0.2. However I also recognize that many other users who have invested a lot in GWT-Ext want to see GWT-Ext proceed to support other Ext versions. I have had discussions with several key members of the GWT-Ext community and got their input and feedback.

After giving the matter a fair amount of thought, I have decided to step down as the GWT-Ext project lead. This was not what I had signed up for when I first started the project. I fully expect the transition to the new project leads to take several months and during this period I'll continue to support GWT-Ext w/ Ext 2.0.2 so there's no need to feel unsettled. Once the transition is complete it will be entirely up to the new team to proceed with whatever direction they feel is in the best interest of the project and its users. Updates on this will be posted in the GWT-Ext forum. I'll continue supporting users of GWT-Plus and once the transition is complete, I'll pass this over to the new team as well. They can use it to cover operational costs or make it available to the community. I'm working with some of the experienced GWT-Ext users but feel free to email me if you're interested in becoming a project member.

Thanks for reading and your support!

Update on GWT-Ext

My response to Jack Slocum's post on the GWT-Ext forum

Most of you probably know that ExtJS has suddenly been changed to GPL and now Jack Slocum, Ext author, has decided to start spreading FUD and misinformation on my forum. Here's my response to his post. I have decided to respond in my blog as I feel the community deserves to know the unethical path that Ext has chosen to take.

Jack,
Welcome to the GWT-Ext forum. Its unfortunate to see you join the forum only to see you stoop to the level you have. I really don't know where to start as what I'm seeing of you is like a bad dream.

Lets start from the beginning and for the benefit of the community lets do a recap first.

I have had nothing but good words for you in the past and have been a great evangelist for Ext. This should be evident all over my blog. For example :

http://www.jroller.com/sjivan/entry/ext_vs_dojo

I have never tried to misrepresent what GWT-Ext does, or even take credit for any of your work.

http://www.infoq.com/news/2008/02/gwt-ext-2

I started this project knowing the Ext JS would was available to the community under a commercial friendly LGPL terms. I started GWT-Ext as a pet project but decided to make it available to the users after the great feedback and interest I received.

http://www.jroller.com/sjivan/entry/gwt_ext_integration

This should all seem familiar to you as the inception of Ext was similar where you initially started out extending YUI's capabilities out of passion. Now that I had made GWT-Ext an LGPL based open source project, I had a commitment to the community and it was no longer a pet project. I worked tirelessly during nights and weekends working on a project that I truly believed in.

During this period I received encouraging words from you showing your full support. Ext 2.0 came out and I worked overtime redesigning and supporting the new API's. The userbase of GWT-Ext grew very rapidly and you reaped the benefits by increased sales of ExtJS commercial license / support. (I have always encouraged GWT-Ext users to support Ext). I know of alteast what amounts to $30-40K of sales of Ext licenses that can be attributed to sole users of GWT-Ext. What do I get in return? Nothing monetarily, but working on an OSS project helping users, working on cool stuff and so on was rewarding to me. Edit Yes, I do have a GWT-Plus library for facilitating GWT-RPC to Store integration that is not free but this is completely optional, used by a very small percentage of users, and I made it very clear at the very beginning that its not open source. It was never advertised as LGPL and then turned to GPL / commercial ware.

Seeing the popularity of GWT-Ext grow, you send me a note asking if I'd like to be a sub-project of ExtJS. I respectfully turn it down saying that I'd rather build my own reputation and identity in the OSS community. In hindsight, this was probably the wisest move that I've made in my professional career.

In the meantime another GWT based project named MyGWT starts which uses Java API's but borrows Ext's CSS and Stylesheets without attributing those to ExtJS. You catch wind of it and work out some deal with him to allow use of what you view as Ext based assets.

Ext grows from strength to strength as the community make contributions to their favorite open source Javascript library - ExtJS. It is clearly stated in the Ext 2.0.2 (and lower) license that

Ext is also licensed under the terms of the Open Source LGPL 3.0 license. You may use  our 
open source license if you:
 * Want to use Ext in an open source project that precludes using non-open source software
 * Plan to use Ext in a personal, educational or non-profit manner
 * Are using Ext in a commercial application that is not a software development library
or toolkit, you will meet LGPL requirements and you do not wish to support the project

http://www.gnu.org/licenses/lgpl.html

Users start contributing to wiki, spending inordinate time on contributing user extensions, and per LGPL contributing back their modifications to Ext by posting on the Ext forums.

Things start looking suspicious as the Ext 2.0 final license is full of ambiguities designed to trick users into buying a commercial license, yet users were told Ext is LGPL thereby drawing more users in.

At this point you're trying to support MyGWT (LGPL at the time) as well as support my project to maximize your revenue.

What triggered you recently was this post http://mjg59.livejournal.com/84586.html where some user looked at your Ext 2.0.2 license and said, "wait a minute.. this license is dubious and to debunk it I'll basically remove all ExtJS's additional license restriction clauses and redistribute it purely as LGPL, because per-LGPL, I'm well within my rights to do so."

This upset you because you realized that the FUD license you created that was intended to prevent redistribution under pure LGPL and / or forking was flawed and you panicked. Within a few weeks of that blog, you suddenly, without any notice to the community, change the Ext license to GPL in a minor release. At the same time you quickly adopt the not-as-complete GWT solution of MyGWT and also make that library GPL too. Your greed for money came before your moral ethics and you decided to screw all the users who you lead to believe that Ext was LGPL all along.

What percentage of your community do you think you would have had, if users were told upfront that Ext would turn GPL? Do you think Ext would have been where it is today without the active participation and voluntary time put in by extremely dedicated and smart users. I have high regards for several voluntary "Ext Support Staff" like Animal, Saki, mystix, hendricd and a few others - but don't you think you used them to meet your own selfish goals? Do you think that you would have been able to man the support for your 35,000+ users without their help?

Back to GWT-Ext. With your bait-and-switch to a GPL license, you figured - "hey, I'll join with the MyGWT project and make MyGWT licensed under GPL too. Actually lets call it "Ext GWT" so that it adds confusion. Now since GWT-Ext requires users to download and use Ext, when I move to a GPL license, users of GWT-Ext will also need to buy Ext JS licenses." You suported my project all along yet adopted MyGWT just so that you could get revenue from both sources. You also removed the Ext 2.0.2 license and download from extjs.com in an attempt to force users to use the GPL 2.1 version of Ext.

Unfortunately for you, this plan backfired bigtime. I refuse to screw my users by making the GWT-Ext project GPL too. I state on my forum that the Ext 2.1+ roadmap for the next year or so doesn't really have anything useful to add since most of the functionality on Ext's roadmap 1) Is already provided by GWT itself, 2) functionality is already available today in GWT-Ext from user contributed plugins or 3) Not relevant to GWT Java development.

Seeing this, you send me a nasty email trying to strong-arm me into changing my license to GPL. However I am not willing to change the GWT-Ext license to GPL just to please you.

And now you have the audacity to come on my board, scaring users off saying the use of GWT-Ext with Ext 2.0.2 was not permitted by your license. How dare you!

Well that's enough of a background, so lets talk business now.

Now before you continue spreading more FUD on my forum telling users what they can and can't do with Ext 2.0.2, please have your IP council clearly state every single detail of where the violation has occurred (if any and unbenounced to me) and I'll make sure that they are rectified promptly. The bottom line is I am not going to change the license of GWT-Ext to GPL only because you "feel" I should. You cannot undo the Ext 2.0.2 license.

There's nothing real exciting in the Ext 2.1 roadmap over the next year+ for GWT-Ext users. GWT-Ext on the other hand will continue to grow and expand with other library capabilities like Tibco Message Bus, Fusion Charts, Pipes, Extensions and more and will do so with commitment and integrity to its userbase.

Yes, hard work deserves to be rewarded but not at the cost of being unethical and taking advantage of the trust of the community. I'm sure no one would have had any issues had ExtJS been GPL / commercial to start with. Users should think twice before dealing with such practices. One never knows when the commercial license fees might double or even change to a per-CPU model.

Sanjiv

Update 4/27/2008 2pm : Jack had responded to my post but when I recently checked this afternoon, it was edited out citing that it has been removed as it was like a soap opera. I feel these are important issues and his response deserves to be heard by the community as well. For this reason I have put up a copy of his original response which I had captured earlier today. It can be found here. It's worth noting that yet again he changes his stance on use of GWT-Ext with Ext 2.0.2 saying : "That’s fine, you are welcome to stay on Ext 2.0.2. I wonder what happens though when FF3 or IE8 is released?". Make up your mind Jack and give us an answer based on legal merit, and not on your mood.

Am I trying to stir the pot? Absolutely not. Licensing is a very serious issue and discussions on it cannot be considered soap opera like. My response can be found in the comments there. Now is the right time to get the facts straight and licensing inconsistencies sorted out once and for all.

searchable javadocs

Javadocs are good but not great as they miss a key feature of being able to do a full text search across not only the method names but also the actual doc content.

This morning a read about a really neat tool called Documancer from Abhijeet's blog. This tool essentially allows you to point to the index.html Javadoc file of a given library and one can then run full text searches through the Javadocs.

To test drive this, I decided to try the tool against the GWT-Ext Javadocs. Here are the steps involved

Download and install Documancer from here Now launch Documancer and go to "Settings --> Manage Books". Then Click the "New" button and enter the name of the book. A "book" is nothing but a name for the library as you can have javadocs for multiple libraries added as "books" to Documancer. Select HTML as the content provider and point to the Javadocs index.html for the title page. Make sure you check the "Enable full text search for this book" option.

Documancer now goes through the javadocs and indexes it. It uses Apache Lucene under the hoods for indexing and the Gecko engine to display the javadocs. Once this gets complete, you can view the Javadocs as you normally do in a browser. However you can now also carry out full text searches and add bookmarks

Pretty cool!! It would be great if this tool supported reading Javadocs from jars too, and building a library of "books" by pointing it to a maven repository. I'm a big IntelliJ IDEA fan and hopefully this feature makes it natively into IntelliJ.

Posted by sjivan ( Apr 15 2008, 02:25:50 PM EDT ) Permalink Comments [2]

GWT-Ext 2.0.3 released with charting, maps, portal and other goodies

GWT-Ext 2.0.3 has been released. This version is compatible with Ext 2.0.2 and GWT 1.5. The new features in this release are :

Charts
Support for Charting via YUI charts. Line, Bar, Pie, Column charts are all available as a standard GWT-Ext Panel subclass. Charts use the standard Store as thier underlying data source.

Mapping API
Support for Mapping API via Mapstraction. The supported Map providers include OpenStreetMap, Yahoo, Google, Microsoft, Map24, MultiMap, MapQuest, FreeEarth and OpenLayers.

Various Maps are now available as a standard GWT-Ext Panel subclass. The motivation for the GWT-Ext Mapping API is not to provide the same level of features as other GWT Google Maps API's which are far more feature rich and complete. Instead, this API is over mapstraction which provides several commonly used features such as markers, polylines, GeoRSS and such using the same API and this may be sufficient for many applications and gives end users some flexibility to pick a map provider that meets their needs and license requirements.

The desired map provider can be instantiated by a factory, and the rest of the code remains the same and works across various map providers (some free, and others commercial). Thanks to VideoProtein for sponsoring this effort.

Portal
Initial Portal support.

The GWT-Ext User Extensions project is also coming along nicely with a lot of contributions coming from the community. Thanks everyone!

Feel free to add your feedback, comments, suggestions or what you'd like to see most in the next release. Thanks.

Thoughts on Java web frameworks and RIA

This morning I read a brief coverage of TheServerSide's Java Symposium Web Frameworks smackdown (via Matt Raible's blog). This prompted me to share my thoughts on the various Java web development frameworks.

Web applications fall in two very distinct categories :

SEO, bookmarkability, Restful URL's are all requirements of such applications. The appropriate frameworks for such applications are traditional request-response based frameworks. The prominent applicable frameworks in this category are Spring MVC, Struts, Tapestry, Wicket, and Seam / JSF. Any of these frameworks can be decorated with Ajax goodness for select components on the page using Javascript libraries like JQuery, ExtJS , Dojo, Scriptaculous, mootools etc.

GWT and Flex are really not suitable for such applications even if there are some hacks possible to support Search Engine Optimization (SEO) functionality. I'd even say that JSF is not really suitable for such apps and is more geared towards category 2 above ( Rich data centric apps with little or no static content).

I've said this before that I feel that JSF is an incarnation of EJB 1.x which is overengineered and too focused on "enterprise patterns" making the simple things not so simple to implement. I looked at Tapestry a while ago and it introduced some really nice concepts but didn't pass the 15 minute test and felt it was quite over engineered.

I haven't looked at Tapestry 5 but I just have this feeling of complexity associated with it having a not-so-easy time with it on initial evaluation. So I'm reluctant to go back to re-evaluate it... Kinda like the lingering stigma of Struts 1.x that is sometimes associated with Struts 2. I have used SpringMVC for a long time now and it has served me well but it really doesn't make developing larger web applications enjoyable or easy - it just allows you to get the job done.

Wicket passed my 15 minutes test with flying colors. On first glance itself I was able to "get it". Its templating mechanism with ability to associate widgets from the template to a corresponding Java class where the controller logic can be added makes it very easy to separate the work of designers and programmers, and gives you the ability to add as little or as much dynamic content to a page with static content. Infact the new proposal for declarative UI capabilities to GWT follows a model quite similar to the Wicket approach. Then again the Wicket model also resembles the Flex code-behind model of being able to specify the layout of components using mxml and the have the controller logic in a corresponding Actionscript class.

These applications are what we call RIA - Rich Internet applications. The prominent applicable frameworks in this category are GWT, Flex, JSF and Javascript widget libraries like ExtJS or Dojo. JQuery is more suitable to decorate pages that fall in the "Internet based apps with static and dynamic content" as opposed to building the entire application as it is more of a Ajax library than a complete Ajax framework.

As mentioned above, I'm not a big fan of JSF and dont feel its the right technology to build RIA's. In the smackdown article Ed Burns is quoted saying

These impressive lines might help sell JSF to management, but when it comes down to actually developing a rich application, JSF doesn't event come close to providing the richness and interactivity that other frameworks in this category can deliver. Additionally the productivity level using, say , GWT is far more than using JSF. Seam which is based on JSF does have some GWT integration but one has to question the need for JSF + GWT instead of just going with GWT alone. Maybe someone can provide the motivation behind this.

If Javascript is your thing using ExtJS or Dojo w/ DWR to build the entire UI is certainly a viable option. It mostly depends on the skillset in your team.

I feel that Flex and GWT are the key players for web development with Java backends in the area of RIA. Before discussing GWT and Flex I'd like to mention that while RIA stands for Rich Internet Applications, the suitability of GWT or Flex for a given application depends on the kind of application - whether its a Rich Internet Application or Rich Intranet Application.

There are a lot of similarities in what GWT and Flex deliver from a functionality stand point however the right technology depends on the application being build. There is a lot to discuss regarding the differences and sweetspots of GWT over Flex and vice-versa, and so I'll write a separate blog on this.

For now I'll just mention that GWT really does a good job optimizing the size of the application JS and making sure that the web application is loaded in the most efficient way over the wire with smart caching etc. This is great for Internet based applications but pretty much a non-issue for Intranet applications where size doesn't really matter to a large degree due to the lower traffic and higher network speeds. Flex on the other hand allows you to do things that are not possible to accomplish with Ajax technologies like large tables ( 20K rows can be rendered under a second ), interactive charts, collaboration and multimedia however Flex apps tend to be larger in size and "feel" different from DHTML / Ajax pages. Flex is based on Actionscript which in many regards is very Java-like and supports very tight integration with Java backends similar to GWT's RPC mechanism or DWR. Stay tuned for a detail comparison on the two..

These are just my thoughts based on my experience. Take it for what its worth. As always, feedback is welcome.

GWT-Ext 2.0 out with improved Showcase demo

GWT-Ext 2.0 has been released with an improved Showcase demo. This version is based on Ext 2.0 and has several cool new features and performance enhancements. You can learn more about it by reading the release notes.

It's been a while since my last blog post but I hope to pick things up. I've been working on many cool things that I would like to blog about but just haven't have much time to do so as I was busy working on getting GWT-Ext 2.0 out and also setting up a project website http://www.gwt-ext.com.

Odd PostgreSQL transaction behavior

I'm not sure if this falls under "its a feature, not a bug" category, but I discovered a somewhat odd PostgreSQL transaction behavior. The usecase is that I need to carry out a bunch of operations against a database, and be able to programatically deal with conditions such as duplicate keys or foreign key violations. Consider requiring to import a bunch of data from, say, a flat file to a table where it is expected that the external data is new data i.e SQL inserts. There is a possibility that there is some bad data that results in duplicate key violations of foreign key violations. I would like to be able to log these errors but not necessarily abort the transaction. However there may be certain rules where I need to rollback the transaction as well. One way to deal with this is to query the database and check to make sure that the new data is valid before inserting. Another option is to execute the insert query, catch any violation exception raised by the DB, log it, and then continue processing the rest of the data. The latter is going to be more efficient as they'll be a lot fewer queries run against the database.

I get the following exception at commit time when trying the second approach with PostgreSQL.

25P02, current transaction is aborted, commands ignored until end of transaction block

Turns out that PostgreSQL aborts the entire transaction if any database constraint violation is encountered during transaction. After looking into this issue, it appears to be a PostgreSQL feature and their way to deal with this is to create a savepoint before any expected potential errors occur, and if errors are encountered the rollback to the previous savepoint.

  String savepointName = "mySavePoint";
  Savepoint savePoint = conn.setSavepoint(savepointName);
  try {
     stmt.execute();
  } catch (SQLException sqle) {
      conn.rollback(savePoint);
  }

In my case this means that I would have to create a savepoint after every record insert which is certainly going to impact performance significantly. Additionally I would prefer to not use exotic database features since 1) they might not work against other vendors and 2) they might not stable 3) there are some issues with this approach when using Hibernate. I tried flushing the Hibernate session and manually rolling back to a savepoint using the Session's underlying connection but this results in the system getting in a screwy state.

Is there a database standard that talks about the correct behavior under such circumstances? I'd be interest to hear if you use savepoints and if you have any knowledge of MySQL and Oracle's support of savepoints.

My "Ajax Experience"

Dojo : Alex Russell is very sharp guy and an articulate presenter. All his presentations / keynotes were very engaging and thought provoking. I attended his presentation on Dojo and its great to see that Dojo 0.9 has has several improvements over version 0.4. One of the areas they've improved on is performance. They have dropped support for namespaced widget declaration, which previously was a way to support valid XHTML, and now only support widget declaration using custom attributes. Dojo has earned the reputation to be the library that truly innovates and pushes the boundaries of what can be done and they deserve a lot of credit for that. It was good to see that Dojo is going to now have a rich Grid widget and they are looking to add model bound widget support. This is not something new to users of Ext, but was a serious shortcoming in Dojo which I'm glad to see being incorporated. Alex is very active in the community and influential in the Javascript and CSS world and I think this goes a long way with the adoption and popularity of Dojo as people feel comfortable working with a library where the lead developer is an expert in the area. My personal opinion is that Ext is a step ahead of Dojo in certain areas like 1) more polished widgets and 2) better MVC based design. Dojo on the other hand beats Ext when it comes to accessibility and innovation like charting capabilities and Dojo Offline. Although I think Dojo should just be using Flash charting instead of fancy VML / SVG stuff :)

Ext : Although I'm "reviewing" Ext second in this list, I really feel that Ext was sold short at this event. There was a pretty good turnout for Rich's talk on Ext and many of them showed a lot of interest after the talk discussing potential use of Ext in their products with Rey Bango and Rich. Rey was fantastic, and I was fortunate to talk to him a fair bit. The problem was that while the users who showed up for Rich's talk were wowed by the demos, there was not one mention of Ext in any of the expert discussions and Keynotes. Joe Walker, Alex Russell, Stuart Halloway (prototype) and John Resig were part of many talks on frameworks, javascript and future direction and they repeated "Dojo, JQuery, Prototpye, Scriptaculous and YUI" time and again but there was not one mention of Ext. Could this be a coincidence? Or a case of "He Who Cannot Be Named". All the previously mentioned speakers are hard core developers and highly respected in the community. I think Jack needs to be part of such a panel so that Ext is not left out in such a mainstream conference, especially when Ext is clearly one on the best open source JS frameworks. By not having an open discussion where, say, the Dojo folks talk about the advantages of Dojo over Ext or vice-versa, users of this conference will not be able to make the most informed decision when choosing a library for their projects.

GWT : I feel very much the same way about GWT as I do about Ext. GWT was also poorly represented. The core developers from each project need to be present so that there is an open discussion of the pro's an con's of each framework. John Resig, Joe Walker and Alex Russel were taking questions and when one attendee asked about GWT, John responded saying "Why would I take an inferior server language to the client to build a UI" and laughed about it. What?? That's a pretty weak response. I don't think such mockery of GWT would be made had Bruce (GWT architect) been in the room. Atleast Dion laughingly responded "while you guys are busy doing cool stuff, I'll just write my app in GWT and Ext and go home to spend time with my family". It's very likely that attendees not aware of GWT before coming to this conference would go back with the impression GWT sucks, and possibly miss out on the most appropriate technology for their requirements. And in talking with a bunch of people, I got the impression that a large percentage of the attendees were using Java as their primary development environment

JQuery : JQuery is really slick. Its lightweight, powerful and has a very intuitive API. Unlike Dojo or Ext, JQuery is a library and not a framework and typically targets different use cases. Dojo or Ext are probably not the best choices if you want to sprinkle some Ajax magic on some of your internet web pages. Their sweet spots are building enterprise / intranet applications while JQuery is perfect if you have an internet application like a social site where you need to abide by the rules of having a degradable site that functions without javascript and need REST'ful URL's with separate pages for SEO reasons. JQuery is great at taking existing markup like divs and traditional forms and converting them into highly functional Ajax components.

Prototype and Scriptaculous : These libraries target the same use cases as JQuery. They are extremely popular being one of the very first Ajax libraries. I personally think JQuery superceeds Prototype / Scriptaculous but there will continue to be a huge userbase just like Struts 1 is still one of the most widely used Java web frameworks.

Offline apps using Google Gears : The Google Gears talk was disappointing. I really wish the Gears team had sent a core developer to talk instead of an evangelist who was unable to answer simple questions regarding the security of the SQLite database Gears uses. The question was how is security handed if the local data is based on the end point URL. The speaker did not have an answer and tried to talk his way out of this :) For those at the presentation, check out Dojo's Offline API which sits on top of Gears and provides API's to encrypt data stored in the local DB.

Offline or Desktop apps using Adobe AIR : This presentation was excellent. Kevin Hoyte did a great job explaining how things worked, showed some really cool demo's and threw out some interesting usecases and scenarios where AIR could be used. An important point he raised was that unlike Google Gears, you do not necessarily have to use AIR for an offline web applications. AIR can be used to build rich desktop applications that interact with the local machine as well as the internet. For example an application where you have , say, Google Maps embedded but are able to save screenshot's of various maps with application specific data locally or drag and drop between your desktop to your application.

There were a few other gems: "Advanced Web Security" by Joe Walker. Try getting a copy of the slides from this presentation as it has a lot a good, eye opening, and rather scary stuff. The talk on Comet by Michael Carter was also excellent. I wasn't expecting too much out of this one it turned out to be one of the best technical talks at this conference.

Tomorrow is the last day of this conference and I'm looking forward to it. Overall I'd say that the conference was pretty hard core with little fluff. My biggest criticism is that I don't feel that there was enough open discussion comparing the obvious contenders and the pro's and con's of each. And that not all toolkits were well represented in the little open discussion that they had. The presentations were given in isolation without the hard questions being answered like why Dojo vs. GWT. vs Flex vs Ext vs JQuery vs. Prototype and when each library is the most appropriate to use. This is where I think someone like Matt Raible can add tremendous value as he views each library / toolkit as an independent user and can give an unbiased opinion on what he thinks is the right stack based on the users existing environment and requirements.

Ajax conference in Boston

It's going to be a very busy three days at the conference, but its going to be informative and fun as well. If you see a name tag with my name please do say "Hi" :)

Does the "unread" spam folder in gmail annoy you?

I really like gmail but one of the things that really annoys me is the unread spam folder. Lately I've been getting a fair amount of spam and gmail correctly places them in the spam folder but I can't help but empty it each time I see unread / new spam. I've tried to ignore it on numerous occasions but just cant. It bothers me too much for some silly reason. Unlike Yahoo mail where you can empty the spam folder without getting into it, gmail allows you to empty the spam folder only from within the folder.

Fortunately I don't have to deal with this anymore. A friend of mine pointed me to Customize Google, a must have Firefox plugin. The amount of customization is supports is fantastic and the feature list too long to list here but I will mention a few of my favorite features.

It allows you to hide the Spam folder count and now I can read my email without getting aggravated.

It allows you to hide Google Ad's in gmail, google search and other google services. Stream Google search result pages. It converts the paged Google Search results into a single page where vertical scrollbar automatically fetches the next / previous page results on demand. Kinda like the dzone.com site.

There are plenty of other goodies so check it out yourself. I was surprised I hadn't found this plugin sooner. Do you know of any other undiscovered Firefox plugin gems?

GWT-Ext 0.9.2 Released with cool Tree features

I am happy to announce the release of GWT-Ext 0.9.2. It has been over a month since the previous release but a lot has gone into this release. Several important pieces of functionality are included in this release along with API enhancements and bug fixes. GWT-Ext 0.9.2 supports Ext 1.1.1 and GWT 1.4 final. A vast majority of the kinks have been ironed out and you'll find this version to be pretty solid. You can check out the GWT-Ext 0.9.2 Showcase demo here.

Core Drag and Drop functionality has been included in this release. Tree functionality is the other major piece that has been worked on and is now very feature complete. The new Tree features include :

There are a lot of other enhancements which you can read about in the Release Notes and learn about as you work with the API.

I would like to thank the GWT-Ext users who have suggested enhancements / improvements and reported bugs. The primary focus of the next release will be Javadocs and user extensions like FileUploader / Accordion. Please let me know if there's a specific example that you'd like to see in the Showcase Demo.

If you've found GWT-Ext useful, you can show your support by making a Donation here. Thanks!

API differences between library versions

Have you wondered what API changes have been made across Spring versions? Or between JDK 1.4 and JDK 5 for the java.lang or java.util package? Release notes typically capture the high level enhancements and changes but its often useful to see what API's have been added in newer versions of a library.

I first came across such a diff report of JDK 6 accidently when I was searching for some information on JDBC's ResultSetMetaData class. When I viewed the page I immediately knew that this was something that I would like to see of other libraries as well. After searching a bit I found out that the tool that was used to generate the JDK diff reports was TopBlend, a tool developed by AT&T Labs. However this tool is not publicly available yet.

Fortunately I did find an open source alternative - JDiff. JDiff is really easy to use and produces very useful API diff reports. I used this tool to generate diff reports between GWT-Ext 0.9 and 0.9.1 and this allows users to quickly identify the API's that were added / changed or removed.

I'm surprised that such diff reports are not more widely used. I think the community can benefit when popular libraries like Spring and Hibernate ship with such reports. While these libraries are pretty stable with most changes being backward compatible (and hence not many API modifications or removals), such a report will help users identify and use newly introduced API's / functionality.

Posted by sjivan ( Aug 28 2007, 01:44:26 PM EDT ) Permalink Comments [3]

GWT-Ext 0.9.1 Released with improved Showcase demo

I'm happy to announce that GWT-Ext 0.9.1 has been released. This release is compatible with Ext 1.1 and has several functional and usability enhancements along with bug fixes. The API coverage of Ext continues to increase with only a small percentage of Ext API's remaining.

The complete list of API changes from GWT-Ext 0.9 to 0.9.1 can be found here and a high level summary of the changes can be viewed here. As you can see, around 40 classes and 300 new methods have been added since the 0.9 release.

I've enhanced the Showcase example to include a few more samples but more importantly you can now view the source code and associated data along with each demo. This should make it a lot easier for users get a feel of GWT-Ext without downloading it to view the sources. Ofcourse I'm confident that after you view the demos you will download GWT-Ext :) I plan on adding a lot more samples so check for updates. Note that the samples are hosted on SVN and the download rates are pretty slow so you might notice a little lag the first time you view the demo.

What's next? One of the critical pieces that is not yet available is the integration of data returned from GWT-RPC calls with Ext's Store API. The Store API allows you to bind data to UI components like Grid and ComboBoxes. Consider making a GWT-RPC call that returns data beans like User objects which seamlessly integrates with the Ext Store API. The Store API allows you to render the data in a variety of widgets like a Grid or a ComboBox. And since the source data is "bound" to the widget, you have access to any modifications the user makes to the data from a widget such as an EditableGrid. You can then pass back these modifications to the server using a GWT-RPC call. I think that would be pretty slick. I've already make good progress on this and will post updates on the GWT-Ext Developer forum.

Other things that I'm planning on working on in order of priority are - A lot more samples, completing the rest of the Ext API's and finally Javadocs. If you're a GWT-Ext user and would like see things done in a different priority please speak up. I have a lot more ideas but I'll save that for later.

Posted by sjivan ( Jul 31 2007, 04:25:19 PM EDT ) Permalink Comments [15]

Ext vs. Dojo

Respect is something that is earned and not bought. This is not a technical comparison between Ext and Dojo but rather the difference in the way the two libraries have grown and built a user base.

I was prompted to write this blog entry after reading Alex Russell's (Dojo creator) article Why Dojo? where he talks about why Dojo should be chosen over other Ajax libraries highlighting Dojo's Breadth & Depth, Quality, Performance and Community. He also mentions that

I have used Dojo a fair amount back when they were at 0.3.x and 0.4.x when there weren't many choices available and Dojo was ahead of the pack. My experience with Dojo was quite the opposite of what Alex says today. Sure it had breath and depth, but almost everyone who used it said that they found it bloated and there was a legitimate reason for this. Dojo applications quickly got really sluggish the moment you started building anything larger than a simple app. There were regular browser freezes due to the synchronous nature of the Dojo template loads and it was openly admitted by several commiters that there were leaks in the Dojo widget framework especially with tabs and dialogs and they really didn't have a good idea of what was causing it. The community was large and growing either because Dojo was ahead of the pack at the time or because they had already invested too much into it, however it was anything but a rosy picture. A large percentage of users were disgruntled and frustrated by the poor performance and lack of proper documentation.

Fast forward a year and the Dojo 0.9 gets released on 07/03/2007 with large portions of Dojo rewritten and is now supposedly "blazing fast". I appreciate the hard work they've put into releasing a new and improved Dojo but what irks me is that given the history of Dojo, less than a week after Dojo 0.9 beta is released the author starts talking about how fast Dojo is, how its being used in several high profile sites and why users should choose it over other frameworks. I'm sure he has some valid reasons, but cant we have the community use it first and validate the claims before declaring success? Can we please hear more about how great Dojo is from its users rather than from the commiters?

Now lets talk about Ext - Ext originally started out as an extension of YUI and quickly caught the attention of users with the amazing work Jack Slocum started putting out. The wordpress style comments on the authors original site was an instant success with users posting rave remarks. Jack just continued doing his thing and producing top quality work in incredibly short periods of time like the Grid Widget and the Tree Widget, both of which are far ahead in terms of quality and functionality than any other library that I know of. Jack's a super smart guy but still got the community involved with the design of major components like the grid and incorporated changes based on user feedback. YUI-Ext started growing at a pace that was too fast for YUI to keep up with and that's when it changed from an extension of YUI to a comprehensive library with a clean and powerful API.

The difference between Ext and Dojo is evident by browsing through the Ext forums where users are extremely excited about using Ext and are very happy by the results they are producing. I have on countless number of occasions seen users make feature requests and Jack responds saying that the functionality is already available pointing them to the appropriate section in the docs. I've not seen this of any other library that I've worked on and its quite incredible how Jack always seems one step ahead of the users when it comes to adding features.

There weren't any big news flashes from Jack about how great Ext was. Instead its the users who keep saying how great Ext is. There are numerous links to sites built with Ext and not some phantom list of high profile, high traffic sites.

Jack has earned the respect of the community not by talking about his work but rather by working hard and the producing a top quality library that users simply love.

Edit 09/07 : Here's a link to Dojo's forum. You can find examples and applications developed by users using Dojo in the 'Dojo Showcase' forum.

Posted by sjivan ( Jul 09 2007, 03:29:21 PM EDT ) Permalink Comments [13]

GWT-Ext 0.9 released

A quick update : I've released GWT-Ext 0.9. The project is hosted on Google Code : http://code.google.com/p/gwt-ext/. There's a link to the GWT-Ext Feed Viewer demo on the project home page. The GWT-Ext Developers Forum is located here : http://groups.google.com/group/gwt-ext.

A word of caution : There's presently not much documentation besides a Getting Started doc and skeletal Class Reference API docs. I have bundled the source of the GWT-Ext Feed Viewer and a few other examples so many of you should not have much trouble getting started. I'll start adding Javadocs shortly.

May the force be with you :)

Posted by sjivan ( Jul 09 2007, 01:40:15 AM EDT ) Permalink Comments [5]

GWT-Ext status update and Feed Viewer on AIR

I've been working tirelessly on getting 100% API coverage of the Ext 1.1 Beta 2 release and I'm *very* close. Things are shaping up really well and I'm pretty excited about this. Having said that, there's still a lot more work to be done especially with respect to adding Javadocs and testing and it will be a couple of weeks before I have anything for public consumption. Edit : Please check back later today for further info. Several users have expressed interest in test driving the library and i'm working on having it available for download. I'll have details up later today. <end edit>

I have setup a few GWT JUnit tests for some non-UI Ext's classes like DomQuery and Store. In addition, I have also added a few test cases for Ext UI widgets starting with the Grid. Contrary to what some say, it is absolutely possible to test UI elements using GWTTestCase. Infact its a great way to test UI aspects of your GWT app. By default the GWTTestCase JUnit tests run in headless mode with the hosted browser "hidden" but you can pass an (undocumented) JVM arg and have the hosted browser show up and display the UI as you add Widgets and asserts in your JUnit test. This combined with asserts using the powerful query capablities of Ext's DomQuery makes it a breeze to write UI test cases. This is a good substitute for Selenium RC when working with GWT. Selenium IDE is still pretty handy when it comes to generating tests by recording user actions. Fortunately Selenium IDE supports generating test code based on templates and so I've written a template that writes out GWTTestCase's.

In other news, Didier Girard, director of onGWT, is going to talk about and demo Ext / GwtExt in his GWT Conference on the 4th of July. So look out for this session if you're planning on attending. Hopefully Didier will invite me to his next conference in Paris :)

I've made the GwtExt Feed Viewer app available for download here. Rename the file to fv.war and drop in it tomcat. You can then access the application by going to http://localhost:8080/fv/

I've also ported the Gwt Ext FeedViewer to Adobe AIR. You can download the distributable here. Rename file to GwtExtFeedViewer.air and install using Adobe AIR.

From the Ext forums it appears that a very small percentage of the users are using Java as their primary dev platform. I think Ext and Gwt have a symbiotic relationship and my hope is that this library makes Ext more accessible to Java developers.

Posted by sjivan ( Jul 02 2007, 11:19:10 AM EDT ) Permalink Comments [14]

GWT - Ext Integration

Ext to me is by far the best Javascript Ajax library around and this is evident by the amazing UI's that can built with it. GWT on the other hand is a blessing for Java developers who often struggle with Javascript. I am very impressed by the job these guys have done. Developing Ajax applications in Java using Swing like code just makes it so much easier, especially since we can step though code using a Java debugger, hot compile, typesafety, refactoring etc. Also Java has the effect of promoting well designed code. GWT also seamlessly facilitates client-server RPC for Java development thereby eliminating the need for libraries like DWR to bridge Java services with Javascript clients. However GWT seriously lacks polished widgets out of the box. I could go on about how great Ext and GWT are but I'll save that for later.

Since I really like working with Ext and GWT, I have built a fairly complete GWT wrapper for Ext 1.1. It's around 80 - 90% complete with all the major components and classes covered. To test drive it I decided to rewrite Jack's Feed Viewer 3 using Gwt-Ext. Feed Viewer 3 is written using unreleased Ext 2.0 code, while my version uses Ext 1.1 so you'll notice a few minor cosmetic differences. Unfortunately jroller doesn't allow uploading of html files so I cannot host a live demo and put up a flash demo instead.

Flash DEMO

Here's a quick look at how some of the code looks like :

Store store = new Store( new HttpProxy("/feedproxy"), new XmlReader("item", new RecordDef(new Field[]{ new StringField("title"), new StringField("author"), new DateField("pubDate", "m/d/Y"), new StringField("content") })) ); //when the store is loaded, select the first row store.addStoreListener(new StoreListenerAdapter() { public void onLoad(Store store, Record[] records) { if (records != null && records.length > 0) { grid.getSelectionModel().selectFirstRow(); } } }); ... ... Toolbar tbar = new Toolbar(Ext.generateId()); tbar.addButton(new ToolbarButton("AddFeed", new ButtonConfig() { { setAutoCreate(true); setIconCls("add-feed"); setButtonListener(new ButtonListenerAdapter() { public void onClick(Button button, EventObject e) { showWindow(button); } }); } })); ... ...

It's worth mentioning that it took me around a day to build the GWT-Ext version of the Feed Viewer. I modeled it closely to the original version and aslo used it's CSS which saved me some time.

You can have a look at the entire GWT-Ext 'Feed Viewer' code by downloading this file (rename to zip). Feedback appreciated.

Posted by sjivan ( Jun 20 2007, 12:20:24 AM EDT ) Permalink Comments [30]

Hibernate Search - cool, but is it the right approach?

Sounds pretty cool, but what is the problem they're tying to solve? Their site states that it "brings the power of full text search engines to the persistence domain model". But what is the use case they're trying to address? It seems that it is basically to provide full text search over the underlying data which resides in a database. So the question is what does using the Hibernate Query API on the domain model buy you over simply using full text searches directly over the database tables?

Let walk through a common use case where you have a web application and you want to provide full text search capabilities. A few required steps to get this working with Hibernate are :

I see several problems with this approach :

So I feel the so called mismatch between (Lucene) indexing and being able to run full text search queries against objects is a problem looking for a solution.

Having said all this, there's one advantage in using Hibernate Search and it may still be the right technology choice for your application. If you're working with small amounts of data in your application and are not planning on building a "Web 2.0 social site" or an amazon.com, Hibernate Search is very appealing because it brings seach capabilities to your application without much effort or requiring to learn a new API. This is true for many intranet applications.

I dont think it is suitable for high volume internet sites for the reasons mentioned above. I feel that indexing and managing the various search parameters directly against the tables is the right way to go. However working directly with the Lucene API's seems like more work than using 'Hibernate Search'. I've looked into the various tools that are available for Lucene and found DBSight. It is a comprehensive solution that brings full text search capabilities to your applicaiton pretty much without requiring any coding. It allows you to create and manage indexes through a feature rich admin UI, out of the box templates and highlighting of search results, ability to return results as HTML, XML or Json which opens up possiblities of using cool Ajax Grid widgets to display the data, categorized search results, RSS feeds, a company focused on high performance, advanced features like scheduling and remote index replication and the list goes on.

When I last looked at DBSight there was a limit on the index size for the free version of thier software but this seems to have changed and they no longer have any index size limitation for their free version. The free version doesn't support Scheduler and remote index replication but even for the whole shebang, a price of ~$3000 seems like a bargain for companies needing a powerful and scalable full text search solution and definitely a case of buy versus build. I feel that full text search capabilites is a separate concern and should not be mixed with the applications domain model or business logic.

Check out thier "Creating a Lucence Database Search in 3 minutes" presentation. Its quite impressive.

If you've worked with DBSight or Hibernate Search please share your thoughts and experiences.

PS: Obligatory disclaimer that I do not work for DBSight :)

Posted by sjivan ( Jun 14 2007, 03:04:14 PM EDT ) Permalink Comments [6]

Hibernate debugging tip

The actions most likely of interest to me (like check for dirty objects, translation to sql etc) occur only during the flush invocation so stepping through the Hibernate code when, say, I add or remove and object to a collection doesn't really help much.

The first step to troubleshooting it to turn on sql logging to see the SQL statements issued by Hibernate. This can be done either by setting the log4j setting log4j.logger.org.hibernate.SQL=debug or setting the Hibernate session factory setting hibernate.show_sql=true. Note that enabling these settings only displays the prepared statement query strings. ie sql with "?" placeholders. In order to have the actual parameters to the queries be logged, you need to set the log4j setting log4j.logger.org.hibernate.type=debug. If that doesn't help resolve the problem I set a breakpoint in the private void log(String sql) method of the Hibernate class org.hibernate.jdbc.AbstractBatcher.

The log() method is called for all SQL commands that Hibernate executes. Since the app might be issuing many other select or insert/update/delete queries that are not related to the problem you're tying to troubleshoot, we need to set a conditional breakpoint in the log() method above or else the breakpoint will be hit a gazillion times before getting to the SQL of interest to you. You probably have a rough idea of the SQL (from the Hibernate SQL logs) or table in question that is being affected for the problem you're trying to troubleshoot. So setting a conditional breakpoint such as sql.contains("delete") or sql.startsWith("select * from ( select users0"); will help you filter out the irrelevant sqls and hit the breakpoint only when the query of interest in being executed.

Once the breakpoint is hit you'll want to examine the method call trace that is resulting in the query being executed. This is done by selecting the desired frame in your debugger and you can then examine the variable values or evaluate expressions in the context of the selected frame. I find this extremely useful in quickly zoning in on the problem.

Posted by sjivan ( Jun 06 2007, 11:52:49 AM EDT ) Permalink Comments [2]

Rearranging taskbar buttons

Posted by sjivan ( Jun 04 2007, 08:09:27 PM EDT ) Permalink Comments [1]

Always run your application with debugging enabled

I always run and deploy my applications with remote debugging enabled, even in production. Wouldn't that slow things down in production? Prior to JDK 1.4, when debugging was enabled, JIT got disabled and the program executed using the interpreter which causes a considerable slowdown that is not acceptable for production environments. However JDK 1.4.2 introduced a really cool feature that many aren't aware of. JDK 1.4.2 and greater supports "full-speed debugging" whereby applications execute at "full speed" even when started with debugging. The execution of the method containing the breakpoint switches to interpreted mode only when a breakpoint is hit.

The advantages of running an app with debugging in development are more obvious and is something I always do. It allows me to examine my code along with frame variables as it executes making it easy to eyeball any bugs or flaws in the logic. I also use JDK's ability to hotswap classes dynamically without having to restart the whole application to pick up the changes. Presently JDK only supports certain class changes to be dynamically reloadable (basically code changes in existing methods are permissible), but it still is a huge timesaver. There is an open issue for JDK to support reloading of classes with structural changes.

Running an application with debugging enabled in production has many of the same advantages as running in development. A lot of hard to track bugs often do not show up immediately when the application is run. Certain bugs show up only after days of execution and others show up under unexpected conditions where a certain user gets into a state that was not anticipated during development or covered by the test cases. If you've worked on larger projects with several customers, I'm sure you know exactly what I'm talking about. There are also times that you'd like to have had an extra log statement or a minor bug fix without bringing down the application in production. Having run the application in debug mode in production, this is now possible. Infact one of the reasons that Sun introduced full speed debugugging was because "organizations deploying long running servers wish to be able to fix bugs without taking down the server."

During development you have an IDE like IDEA or Eclipse which makes it really easy to connect to a remote JVM to debug and reload classes. You might not have this luxury if your application is running in a hosted environment or at the customers site. In such cases one can use jdb, JDK's command line java debugger, for setting breakpoints and examining the frame. jdb does not allow you to reload classes though..

To support HotSwap of classes without an IDE, I wrote a standalone command line utility class that reloads classes using the Java Debug Interface.

import com.sun.jdi.*; import com.sun.jdi.connect.*; import java.io.*; import java.util.*; public class HotSwap { private VirtualMachine jvm; private static final String HOST = "hostname"; private static final String PORT = "port"; private static final String NAME = "name"; private static final String DT_SOCKET = "dt_socket"; private static final String DT_SHMEM = "dt_shmem"; public HotSwap() { } public HotSwap(String host, int port) throws IOException, IllegalConnectorArgumentsException { connect(host, port); } /** * Connect to the JVM using socket transport. Allows you to connect to a JVM running on a remote machine. * * @param host remote jvm host name * @param port remote jvm JDPA port */ public void connect(String host, int port) throws IOException, IllegalConnectorArgumentsException { VirtualMachineManager vmManager = Bootstrap.