Saturday, April 17, 2010

Fluent Interface Inheritance

Fluent Interface is a convenient way of writing DSLish code in the general-purpose programming language. Some of the examples are quite impressive. But there comes a problem when you try to reuse this pattern via inheritance, because the basic method chaining implementations do not actually support it very well. I will use a JPA example found in Wikipedia to demonstrate the problem:

public Collection<Student> findByNameAgeGender(String name, int age, Gender gender) {
return em.createNamedQuery("Student.findByNameAgeGender")
.setParameter("name", name)
.setParameter("age", age)
.setParameter("gender", gender)
.setFirstResult(1)
.setMaxResults(30)
.setHint("hintName", "hintValue")
.getResultList();
}

Looking at this code we can now create a naive Query implementation:

public interface Query {
Query setParameter (String name, Object value);
Collection getResultList() throws SqlException;
}

public class PreparedSqlQuery implements Query {
protected Map params = new HashMap();
public Query setParameter (String name, Object value) {
params.add(name, value);
return this;
}
public Collection getResultList() throws SqlException {
// Execute query and return results
}
}

public class SimpleQueryUser {
public static void main(String[] args) throws Exception {
new PreparedSqlQuery().setParameter("name", "value").getResultList();
}
}

No problems at all, our Fluent Interface is clean and elegant. But what happens when you create a subclass for executing prepared statements? You don't want to implement all that multiple setXxx methods once again, and instead decide to inherit from PreparedSqlQuery:

public class StoredProcedureCall extends PreparedSqlQuery {
protected String outputParameter = null;
// Stored procedures may have both input and output parameters
public StoredProcedureCall setOutputParameter(String param) {
outputParameter = param;
}
public Collection getResultList() throws SqlException {
// Execute stored procedure and return results
}
}

public class SimpleQueryUser {
public static void main(String[] args) throws Exception {
// This works fine
new StoredProcedureCall().setOutputParameter("OUT").setParameter("name", "value").getResultList();
// This produces compilation errors
new StoredProcedureCall().setParameter("name", "value").setOutputParameter("OUT").getResultList();
}
}

The second oneliner produces a compilation error because Query object returned by setParameter() does not actually have setOutputParameter() method. To address this problem we can use return type covariance, which is a language feature added to Java 5. Here goes a fixed version of our StoredProcedureCall:

public class StoredProcedureCall extends PreparedSqlQuery {
protected String outputParameter = null;
public StoredProcedureCall setOutputParameter(String param) {
outputParameter = param;
}
// Compiler allows us to use StoredProcedureCall instead of Query as the return type here because StoredProcedureCall implements Query
public StoredProcedureCall setParameter (String name, Object value) {
super.setParameter(name, value);
return this;
}
public Collection getResultList() throws SqlException {
// Execute stored procedure and return results
}
}

Actually you can see exactly the same approach used in the real-world Query implementations, like OpenJPAQuery. Now what if you don't want to override all that numerous setXxx methods in all subclasses just to change their return types? Here is my solution utilizing another Java 5 language feature - Generics:

public interface Query <T extends Query >{
T setParameter (String name, Object value);
Collection getResultList() throws SqlException;
}

public class PreparedSqlQuery<S extends PreparedSqlQuery> implements Query<S> {
protected Map params = new HashMap();
public S setParameter (String name, Object value) {
params.add(name, value);
return (S) this;
}
public Collection getResultList() throws SqlException {
// Execute query and return results
}
// Hide constructor, use factory to ensure users won't try to do something like: new PreparedSqlQuery<StoredProcedureCall>().setOutputParameter
}

// We make it generic too to enable further extension
public class StoredProcedureCall<R extends StoredProcedureCall> extends PreparedSqlQuery<StoredProcedureCall> {
protected String outputParameter = null;
public R setOutputParameter(String param) {
outputParameter = param;
return (R) this;
}
// Note we don't have to override setParameter here
public Collection getResultList() throws SqlException {
// Execute stored procedure and return results
}
}

public class EntityManager {
public static StoredProcedureCall<StoredProcedureCall> createStoredProcCall() {
return new StoredProcedureCall<StoredProcedureCall>();
}
public static PreparedSqlQuery<PreparedSqlQuery> createNamedQuery() {
return new PreparedSqlQuery<PreparedSqlQuery>();
}
}

public class SimpleQueryUser {
public static void main(String[] args) throws Exception {
EntityManager.createStoredProcCall().setParameter("name", "value").setOutputParameter("OUT").getResultList();
}
}

All that trickery with generics is done to fool the compiler. Good thing about it is that no additional methods are created, that means there are no super calls, which can potentially improve runtime performance. Bad things are obvious too:
  1. It is not safe unless you are using factories to instantiate that builders;
  2. It is definitely not obvious and looks ugly;


Disclaimer: this is a hack, never use it in your code :)

Friday, January 29, 2010

WebSphere + GWT + Comet

IBM supports Reverse AJAX (AKA Comet) in a form of WebSphere Feature Pack for Web 2.0 for WebSphere Application Server 6.0+. It is shipped by default with the latest fix packs for WAS 6.1 and in all 7.0 versions. Actually this Feature Pack has a lot of nice Web 2.0 features, but at the moment we are interested in Web Messaging Service only.

To install and enable FP you need to follow the documentation, it won't take long. As the result you will get the ability to push data to your client's web browser by publishing it to the special JMS topics. As an alternative, you may also use a more specialized API for doing this, but the Publisher object is stored in the Servlet context, therefore it can be somewhat tricky to access it from an arbitrary part of your application, which is not true with JMS.

I assume that you have successfully installed and tested your Web Messaging, and now want to upgrade your QuoteStreamer sample application to GWT. For example, you have a widget and want its methods to be called when some data arrives from the backend.

It is possible to use JSNI to achieve exactly this goal. The idea is simple: when you need to push some object to the client (for example, out of your Message Driven Bean), you first serialize it using GWT RPC mechanism, then put the resulting string to the appropriate JMS Topic. This string is wrapped to JSON envelope, sent to the client via WFP and unwrapped there via Dojo. Your JavaScript listener should be triggered at this moment. But as we use GWT for all our JavaScript programming, we should implement that listener as JSNI snippet:
private native void initCallbackAndSubscribe (String someParam) /*-{
if ($wnd.dojox) {
$wnd.dojox.cometd.subscribe("/test/" + someParam, this,
function (comet) {
module.@com.test.MyCoolWidget::onReverseAjax(Ljava/lang/String;)(comet.data)
});
}
}-*/;

// This is a normal Java code
public void
onReverseAjax(String msg) throws SerializationException {
SerializationStreamReader r = ((SerializationStreamFactory) svc).createStreamReader(msg);
Object data = r.readObject();
// Do something with that data arrived
}
As you see, the data is deserialized later in your normal Java code called by this listener. That's it. If you have initialized everything right (in your index.jsp or some similar place), this scheme should work fine. Now I will show you how to send data using GWT RPC. It should be simple via RPC.encodeResponseForSuccess method, but there is one problem - it requires SerializationPolicy object for the security reasons. This mechanism restricts serialized objects to the limited set described in *.gwt.rpc files, generated during compilation. Another problem is that there can be several such files. I haven't found a good way to get the SerializationPolicy other than by concatenating these files altogether:
public SerializationPolicy getSerializationPolicy() throws Exception {
String result = "";
for (File file : new File(PATH_TO_YOUR_GENERATED_STUFF).listFiles(new FilenameFilter() {
public boolean accept(File dir, String name) {
return name.endsWith(".rpc");
}
})) {
result += (FileUtils.readFileToString(file));
}
return SerializationPolicyLoader
.loadFromStream(new ByteArrayInputStream(result
.getBytes("UTF-8")), null);
}

public void publishObject(String topic, Object obj, Method method) throws Exception {
String r = RPC.encodeResponseForSuccess(method, obj, getSerializationPolicy());
publishStringViaJms(topic, r.toString());
}
I forgot to mention that RPC.encodeResponseForSuccess requires a Method instance, but it is used only to extract the return value type, so any method returning the object of the obj.getClass() type will do.

In general, there are some good things about such WebSphere + GWT + Comet integration, the major one is that it uses WFP, which is a pretty powerful Comet server, compared to some custom-built solutions. For example (at least on WAS 7.0), it uses connection multiplexing, nio and all that cool stuff you expect from the modern web server. Also, it naturally integrates into your ESB, so you can use it in your Process Server applications with ease. Some limitations are obvious too, for example it won't work in GWT Hosted Mode with Tomcat. Also it seems to me that this should be implemented somehow simpler, but I was unable to find it. So if you have any other ideas, please give me a clue.

P.S.: Actually, there are some alternatives to this approach. For example you can use gwt-comet-streamhub to enable Reverse AJAX for your GWT applications. But it provides its own Comet server called StreamHub which is definitely not a part of the IBM WebSphere product line :)

P.P.S.: The described method should work with other Comet-enabled J2EE servers like Jetty with minor modifications.

P.P.P.S.: There is left one more issue with security. As for now, it is not really clear for me how to implement role-based security for Comet... I can think of using several update servlets at the same time, and it should allow me to use webapp security. But this issue definitely needs further investigation.

Tuesday, January 26, 2010

WADL toolbox

WADL stands for Web Application Description Language and currently is in a state of W3C Member Submission, that means at the early stage of W3C standardization process. The purpose of the "language" is similar to WSDL, with the orientation towards RESTful APIs in the first place. What is essential for me is that this language perfectly suits the task of Pomodoro Server API description. There are surprisingly few tools utilizing this format, I guess this is due to the immaturity of the language. Though I can name some of the most useful ones:
  • Jersey, a reference implementation of JAX-RS. It is able to generate WADL at run time based on the metadata found in your Java classes.
  • soapUI, a great tool for testing web services. As one of its numerous new features in version 2.5 it can utilize WADL web service description and generate sample test cases automagically. It can even do the trick in the opposite direction, i. e. infer both WADL and XSD from the existing test case structure! Unfortunately I've found some aspects of this generation somewhat buggy not really obvious. For example, it does not inherit template parameters for nested resources. And despite the ability to specify application-wide settings for HTTP Basic Auth, it does not get applied somehow, so it should be specified for each test case separately. Anyway, it is a great tool and I'm looking forward to see it in a bit more stable state.
  • WADL to HTML is an advanced XSLT stylesheet to generate documentation from your WADLs. It seems that either my WADL is all wrong (then why it conforms to the WADL Schema?), or my XSLT processors are broken (both Saxon and Xalan running on Windows and Linux), because I was unable to get any output. But they say it works for them, so I tend to think it is actually my own problem. Anyway, I've already wrote my own simple XSLTs and going to put it here in a matter of few days :)
  • wadl2java generates a web service client based on WADL. It uses JAXB to create the necessary representations based on XSD referred in grammars section.
  • REST Describe & Compile is yet another client generator. It can infer WADL from the set of supplied request URLs, and then generate client code in several languages, namely Java and Python. The tool looks promising, but as for me, the UI is not intuitive enough. Also, you have no control over code generation, so good chances are that you will have to modify your clients manually after generation. Here you can find a great deal of documentation for the tool and underlying concepts.
P. S.: You can find WADL for Pomodoro Server here. Also you can see some generated documentation in the Google Code wiki.

Wednesday, December 23, 2009

Some useful JSR-168 resources

While trying to figure out the essentials of portlet development, I found few useful resources on this topic:
Here goes a mindmapped extract of those two:

Thursday, December 17, 2009

Using Spring in server side of GWT

Thanks to this post, I was able to use Spring in my GWT-RPC services. Though, I had to fix it a bit - use autowireBeanProperties instead of autowireBean (otherwise my services didn't get populated).

Saturday, November 7, 2009

Relational storage using JAXB, JPA and HyperJAXB

Let us imagine a situation when you are writing an application sitting on the ESB, or providing a web service enabled interface. In both cases you will probably end up having some XML Schema describing your data transfer format. Also, good chances are that you will have to have a RDBMS schema, too. Usually both are designed, implemented and documented separately. But when you write your application from scratch, you have a unique possibility to seriously reduce amount of work need to be done. Here I want to share some of my ideas how to achieve this.

Few variants are possible. You can start with DDL, generate JPA-annotated beans using Hibernate Tools for example, and then annotate it with some JAXB stuff (or use something not requiring annotations at all, like XStream). You will be able to get an XSD using some JavaBeans-to-XSD tool (like you have in IBM Rational tools when generating WSDLs for given beans), or just generate a bunch of sample XML files by marshaling some test data and use any modern XML editor to generate XSD by example. But I must warn you that the XSD will be rather ugly, because usually you don't have too much control over the whole process, that's why the proper tooling is vital for success. Nevertheless, this approach is quite common and supported by many powerful tools, such as my favorite IBM Rational Software Architect 7.5.

From the other hand, you can start with some UML and generate Java Beans, then put all the necessary annotations (both JAXB and JPA) inside. But you will have to do a double amount of work fixing the beans structure and hardcoding annotations for both technologies. Once again, you will need to fix the resulting XSD manually, that's for sure.

What I prefer to do is design the data layer entirely in XSD (which maps to Java classes in a more natural way than DDL), then generate Java beans using JAXB or something similar, enrich it with JPA annotations and generate DDL as the result. Sounds pretty nice, but there are some problems, particularly at the JPA annotations stage. The major one is that you lose flexibility in XSD changing, because after each change your beans will be generated again and you will lose all your precious annotations (yes, you can keep JPA configuration in XML, but this approach has its own drawbacks, and it's beyond the scope of this post anyway).

Fortunately there exists a tool which reduces (and in some cases can even eliminate it at all) the amount of the manual work at the JPA stage. It is implemented by Aleksei Valikov and called HyperJAXB 3 (version 1 is outdated and version 2 supports Hibernate instead of pure JPA). It is more-or-less a plugin set for XJC, adding JPA annotations to JAXB-generated beans. It is distributed as an easy to use bundle containing everything (including build.xml) to start converting your XSDs to JAXB + JPA enabled Java Beans.

Here comes the concepts of relationships and keys. The problem is, when not specified explicitly, HyperJAXB creates an autogenerated integer ID for every entity, which is not always what you really want to get. Let me explain. I want to automate not only the data schema creation, but also the API to access data. For example, when there comes an XML from the client, I don't want to check if the same piece of data is already present in my DB (in fact it can be a hell lot of work to do, especially when some complex, deeply nested XML structures are considered). I want the intelligent middleware to merge it automatically with the existing data. But it won't happen by default, because of the autogenerated IDs, which are always issued new. So, if you will try to persist the same XML twice, it will put two copies in your DB, which is probably not what you expect to see.

To fix this behavior, you need to explicitly specify the meaningful IDs for your entities. With HyperJAXB you can specify it directly in your XSD by marking some of the entity fields with the special XSD annotation in the same way you document your schemas (For details please consult HyperJAXB documentation. Yes, you still can't use the XSD uniqueness constraints support because it's a bit too hard to parse by XJC-like tool). Same is true for the relationship options – for example, you can define one-to-many with all the possible JPA attributes, so it's quite flexible in case you need it.

So far, so good. Things start being complicated when you have compound IDs (and I think it happens really soon in the real life applications). Fortunately HyperJAXB gained support for IdClass few weeks ago. But here comes another problem of “cascading” IDs – when you have a complex primary key (in entity A) used as a foreign key in another entity (B), which at the same time is a part of its (B's) primary key. We had a brief discussion on this topic with Aleksei (BTW, I want to say THANKS! for his great instant support for HyperJAXB and related stuff), and it seems to be too hard to implement in the foreseeable future due to some limitations of XJC extension mechanisms. So, the only way out is to tweak generated beans manually. Well, I think it is something like 20% of the job, and another 80% is done automatically.

So, in the end the results are pretty sweet. You design your data structure as XML Schema (which itself can be generated based on XML samples). Then you put some metadata (mostly ID information) inside to make sure it is complete. Now you run HyperJAXB against your XSDs and fix the generated Java Beans in case you need to handle some “complex” situations. Then, using one of the tools (either Hibernate Tools or OpenJPA Mapping Tool) you get DDL in a matter of seconds. After that you are able to (un)marshal your entities in a single line of code using JAXB, and load / merge them from / to any RDBMS in a single LOC using JPA. That totally eliminates all the serialization and storage-related code, reducing complexity, improving reliability, bla-bla-bla :)

P. S.: I must warn you that there are some issues when using this approach with different JPA providers. I've tried both Hibernate and OpenJPA and they seem to be not really compatible with each other for anything but the most trivial models. It worth a separate post and I'm not going to explain it here, just to make it short: I prefer OpenJPA over Hibernate nowadays, because I find it less restricting in defining relationships and complex IDs. Also, from the tooling point of view, those are pretty close.

P. P. S.: HyperJAXB functionality is growing rapidly, so I won't be surprised to see the majority of the issues mentioned in this post addressed in the upcoming releases. I wish this project good luck and can't wait to see it gaining popularity in the JEE development community.

P. P. P. S.: Code samples and / or detailed tutorial will come later, because it is a considerable amount of work and I'm pretty busy nowadays. Though, if you have any questions regarding the aforementioned approach, please feel free to ask.

Friday, August 28, 2009