Optimize the client for the server’s sake

The Story

Once upon a time there was an application that was running on some server and the client functionality was implemented in HTML/CSS and JavaScript. The application was serving trillion (not literally) of users all hanging on the end of some phone lines talking to customers who were usually impatient and needed fast resolution to their problems. Typical call center application where speed is key.

Users were dissatisfied by the speed of the service.

No surprise. They usually are.

The application was delivering static resources for the client and JSON encoded data via REST interface. The underlying data structure was using relational database managed from Java using JOOQ. All good technologies were applied to make the service as fast as possible, still the performance was not accepted by the users. Users claimed that the system was slow, unusable, annoying, dead as fish frozen in a lake (yes, actually that was one of the expression we got in the ticketing system). We were aware that “unusable” was some exaggeration: after all there were thousands of queries running through the system daily. But “slow” and “annoying” are not measurable terms not to mention “dead fish”. First thing first: we had to measure!

Measure

To address the issue we injected some JavaScript that was measuring the actual performance and it was also reporting the client measured response times to a separate server via some very simple and very fast REST service. We paid attention not to put extra load on the original servers not to make the situation even worse. The result showed that some of the results arrived to the client within 1sec, most of them in 2sec but there was actually a significant tail of the Poisson distribution with some responses as long as 15sec. We also had the measurement on the server side and the results were similar. On the server side we measured approximately 10% more transactions that were lost for the measurement on the client and the Poisson tail on the server contained responses up to 90sec. We did not pay attention to these differences until a bit later.

Meeting the requirements may not be enough.

The actual measurements showed that the response times were in-line with the requirement so we created a report showing all good and shiny hoping that this will settle the story. We presented the results to the management and we almost got fired. They were not interested in measurements and response time milisecs. All they cared was user satisfaction. (Btw: At this point I understood why the name “user acceptance test” is not “customer acceptance test”.) We were blatantly directed not to mess with some useless measurements but go and stand by some of the users and experience direct eyes how slow the system was. It was a kind of shock. Standing by a user and “feeling” the system speed was not considered to be an engineering approach. But having nothing else in hand we did. And it worked!

Assess

We could see that some of the users were impatient. They clicked on a button and after a second when nothing happened they clicked on it again. It meant that the browser was sending a request to the server but before the response arrived the communication was cancelled on the client side and the request was sent again. Processing started from zero by the second button press but the wait time for the user accumulated.

Fix

To help the patience of the users we introduced some hour glass effect on the JavaScript level that signalled to the user that they have pressed a button and that the button press was handled by the application. Also the hour glass was moving “entertaining” the users and we also hid the button (and the whole filled in form) behind a semitransparent DIV layer actually preventing double submit. We did not have high expectations. Afterall it did not make the system faster. The users loved the new feature. First of all they felt that we care. They had been complaining and now we were doing something for them. Interestingly they also felt the system faster because of the rotating hour glass on the screen. End of story? Almost.

Learn

After a week or so we executed the measurement again. It was not a big effort since all the tooling was already there. What we experienced was that the 10% difference between the number of transactions measured on the client and on the server practically vanished. Probably these were the transactions when the user pressed the button second time. It was a full processing run on the server side, but was not reported by the client since the transaction as well as the measurement on the client side was cancelled. These got eliminated with the improved user interface that also decreased the load on the server by 10%. Which finally resulted slightly faster response times.

Usual disclaimers apply.

Advertisement

The Little Architect

Uncle Bob published recently an article titled “A Little Architecture“. The article is a conversation between a young developer and a senior (Uncle Bob himself presumably) about being software architect. The article starts with these sentences:

• I want to become a Software Architect.
• That’s a fine goal for a young software developer.
• I want to lead a team and make all the important decisions about databases and frameworks and web-servers and all that stuff.

The next part asks the young developer to list what the important things are. However that is not the only thing that may be interesting in this last sentence. There is another thing, perhaps less technical, that hit me. The young developer says: “make … decisions”.

That may be a mistake. You can interpret it differently what “making decision” means, but let me here tell you my thoughts about that. Some thoughts that were triggered by those two words. First of all here is a story, when I was making some decisions.

Story

Not really many years ago when I was much younger I was acting as system architect and I made a decision on how to store some content. Mainly text and not too large pictures. The obvious choice could be to use database and implement the CRUD operations. A database is always a good solution just as a scarf is always a good gift for Christmas. You love getting a new scarf for every Christmas, don’t you?

On second thought, however the real power of database is when the content is to be searched, indexed and when transactions are executed. They are not really requirements for a media store. On the other hand versioning and user level access control was. I have previously implemented something like that in the past and that time we used SVN for content storage. And that worked fine. So I decided that we should go and use SVN this time also. The project was a success story. A little bit more story than success though. Halfway thriving towards the solution the back-end storage was replaced by a DB layer.

Why didn’t SVN work?

The reason is simple. The developers did not like and understood the decision. They were not familiar with the technology. They used SVN for source code storage but they never used the programming API of it. Instead of using the Java client they forked external svn processes and they were checking out files individually. Displaying a directory containing 20 files was starting 20 processes one after the other. On that system that was approx. 20 seconds.

Okay. It could have been mended in different ways: there was not enough control on the use of the technology and there was a lack of professional code review as well as performance testing due time and so on. The root of the problems though was that I made the decision. I was acting like an omnipotent god, who knows it much better. I was not and I did not.

So what?

I could do it better discussing the solution more with the developers until we all agree on what the solution could have been. I could understand that the DB solution was better or they could understand how the SVN could have been used that way. We could make a decision together. I could make it so that they could make the decision.

A real architect never makes a decision.

A real architect works with the team developing the software asking the right questions making sure that the team make the right decision.

Good architects approve the decision of the team and bear the responsibility. Bad architects make the decisions and blame the team.

Part of it is psychology. If the team makes the decision they are more likely to love the ideas than if they were force fed. They may come up with some ideas that you missed. Good architects recognize that and improve him/herself. Really good architect can even admit at this stage being wrong. On the contrary of what young developers think this increases the esteem. (Unless the architect is wrong more times than not, in which case he/she is not really a good architect.)

Asking the questions also reveal if the team is not prepared for some of the technologies. If they have to learn something new. It may turn out that education is in place or some more familiar technology is to be used. This may also be a smell that you wanted to use some niche technology that may require expensive developers in the coming years to maintain the product. You better don’t!

This does not mean that you should open the floodgates. You still should approve the decision and you should not approve a decision you can not live with. If the team makes a decision on some technology that you feel not good enough it means you have not asked the right questions. You should ask more. The responsibility is your.

I recommend that if you want to be a good architect let the team make the decision and help them forging a good one. Approve it and never blame them. That way they will not leave you in cold water. If you even bring free pizza now and then they may even love you.

Creating proxy object using djcproxy

During the last weeks I have shown how to create a proxy object using Java reflection API and cglib. In this article I will show you how this can be done using djcproxy.

Oh, not again, another proxy implementation!

What is the point to write about this in addition to the selfish fact that I created this proxy? The point is that this is a proxy that is written in Java, it creates Java code that can be examined. It also compiles and loads the created Java classes on the fly so it is also usable but the main advantage is that you can easily get a good insight how a dynamic proxy works. At least a bit easier than digging around the code of cglib, which is creating byte codes directly.

How to use it

You can get the source from github or you can just add the dependency to you project maven pom.

<dependency>
	<groupId>com.javax0</groupId>
	<artifactId>djcproxy</artifactId>
	<version>2.0.3</version>
</dependency>

After that you can use the following code:

class A {
  public int method() {
  return 1;
  }
}
class Interceptor implements MethodInterceptor {

  @Override
  public Object intercept(Object obj, Method method, Object[] args,
    MethodProxy mproxy) throws Exception {
      if (method.getName().equals("toString")) {
        return "interceptedToString";
      }
      return 0;
  }
}

 ...

    A a = new A();
    ProxyFactory<A> factory = new ProxyFactory<>();
    A s = factory.create(a, new Interceptor());

_{This code can be found in the tests of the project in GitHub. This is an edited abbreviated version prone to editing errors.}

The class ‘A’ is the original class and when we want to create a new proxy object we create a proxy to an already existing object. This is different from reflection or cglib. In case of cglib you create a proxy object and it “contains” the original object. It is not really a containment in OO terms, because the proxy class extends the original class. However because of this extending the proxy object is also an instance of the original class. Cglib does not really care which class instance (object) you want to intercept. You can inject a reference to any object instance to your interceptor if you want. Djcproxy uses a different approach and it does that for you and in your interceptor you will get this object passed as argument. This is why you have to instantiate the object in line 20.

The Interceptor implements the interface MethodInterceptor also provided in the library. It has only one method: intercept, which is invoked when the proxy object method is called. The arguments are

• obj – the original object
• method – the method that was invoked in the proxy object
• args – the arguments that were passed to the method call on the proxy object. Note that primitive arguments will be boxed.
• mproxy – the method proxy that can be used to call the method on the original object or on just any other object of the same type

This is all about how to use this library. The next thing is to have a look at what is generated so that you can get a better understanding how a proxy works. Insight never hurts, even if you use a different proxy. Many times debugging or just generating better code is easier when you know the principles of a library you use.

While cglib gives you a static factory method to create new objects djcproxy requires that you create a proxy factory. This is on line numbered above 21. If you want to use it the same way as you used cglib you can declare a static ProxyFactory field in the class where you want to use the factory from. On the other hand it is possible to have different factories in different parts of the code. Although the advantage of it is rare, still I believe it is a cleaner approach than providing static factory method.

How does the proxy work?

The extra thing in this package is that it lets you get access to the generated source. You can insert the lines

    String generatedSource = factory.getGeneratedSource();
    System.out.println(generatedSource);

to print out the generated proxy class which is after some formatting is this:

package com.javax0.djcproxy;

class PROXY$CLASS$A extends com.javax0.djcproxy.ProxyFactoryTest.A implements com.javax0.djcproxy.ProxySetter {
    com.javax0.djcproxy.ProxyFactoryTest.A PROXY$OBJECT = null;
    com.javax0.djcproxy.MethodInterceptor PROXY$INTERCEPTOR = null;

    public void setPROXY$OBJECT(java.lang.Object PROXY$OBJECT) {
        this.PROXY$OBJECT = (com.javax0.djcproxy.ProxyFactoryTest.A) PROXY$OBJECT;

    }

    public void setPROXY$INTERCEPTOR(com.javax0.djcproxy.MethodInterceptor PROXY$INTERCEPTOR) {
        this.PROXY$INTERCEPTOR = PROXY$INTERCEPTOR;

    }

    PROXY$CLASS$A() {
        super();

    }

    private com.javax0.djcproxy.MethodProxy method_MethodProxyInstance = null;

    @Override
    public int method() {

        try {
            if (null == method_MethodProxyInstance) {
                method_MethodProxyInstance = new com.javax0.djcproxy.MethodProxy() {
                    public java.lang.Object invoke(java.lang.Object obj, java.lang.Object[] args) throws Throwable {
                        return ((com.javax0.djcproxy.ProxyFactoryTest.A) obj).method();

                    }
                };
            }
            return (int) PROXY$INTERCEPTOR.intercept(
                    PROXY$OBJECT, PROXY$OBJECT.getClass().getMethod("method", new Class[]{}),
                    new Object[]{}, method_MethodProxyInstance);
        } catch (Throwable e) {
            throw new RuntimeException(e);
        }

    }


... other overridden methods deleted ...

}

Note that the class A is a static nested class of ProxyFactoryTest for this generated code.

The interesting code is the overriding of the method method(). (Sorry for the name. I have no fantasy to have a better name for a method that does nothing.) Let’s skip the part where the method checks if there is already a MethodProxy instance and if is missing it creates one. The method method() actually calls the interceptor object that we defined, passing the proxied object, the reflective method object, the arguments and also the method proxy.

What is the method proxy

The name may be confusing first because we already have an “object” proxy. There is a separate method proxy for each method of the original class. These can be used to invoke the original method without reflective call. This speeds up the usage of the proxies. You can also find this call and a similar mechanism in cglib.

Notes

The implementation has some flows, for example the late method proxy instantiations have no advantage really but the same time may hurt in case of multi-thread execution of the proxies. It could also be possible to create a proxy object that not only extends a class but also implement arbitrary interfaces (perhaps some that is not even implemented by the extended class). The implementation is used in some other hobby opensource project also available on github about which I may write in the future. They are more demonstrative, educational and proof of concept projects than production code. If you have anything to say on the implementation, the ideas, or just any comments, please reward me with your comments.