Monthly Archives: February 2014

Synthetic and bridge methods

If you have ever played with reflection and executed getDeclaredMethods() you may have been surprised. You may get methods that are not present in the source code. Or, perhaps, you had a look at the modifiers of some of the methods and saw that some of these special methods are volatile. Btw: this is nasty question for Java interviews “What does it mean, when a method is volatile?” The proper answer is that a method can not be volatile. At the same time there can be some method among those returned by getDeclaredMethods() or evengetMethods() for which Modifier.isVolatile(method.getModifiers()) is true.

This has happened to one of the users of the project immutator. He realized that immutator (which itself digs quite deep into the dark details of Java) generated Java source that was not compilable using the keyword volatile as modifier for a method. As a consequence it did not work either.

What has happened there? What are the bridge and syntethic methods?

Visibility

When you create a nested or embedded class the private variables and methods of the nested class are reachable from the top level class. This used by the immutable embedded builder pattern. This is a well defined behavior of Java, defined in the language specification.

JLS7, 6.6.1 Determining Accessibility

… if the member or constructor is declared private, then access is
permitted if and only if it occurs within the body of the top level class (§7.6)
that encloses the declaration of the member or constructor…

package synthetic;

public class SyntheticMethodTest1 {
    private A aObj = new A();

    public class A {
        private int i;
    }

    private class B {
        private int i = aObj.i;
    }

    public static void main(String[] args) {
        SyntheticMethodTest1 me = new SyntheticMethodTest1();
        me.aObj.i = 1;
        B bObj = me.new B();
        System.out.println(bObj.i);
    }
}

How is it handled by the JVM? The JVM does not know inner or nested classes. For the JVM all classes are top level outer classes. All classes are compiled to be a top level class, and this is the way how those nice ...$. .class files are created.

 $ ls -Fart
../                         SyntheticMethodTest2$A.class  MyClass.java  SyntheticMethodTest4.java  SyntheticMethodTest2.java
SyntheticMethodTest2.class  SyntheticMethodTest3.java     ./            MyClassSon.java            SyntheticMethodTest1.java

If you create an nested or inner class it will be compiled to be a full blown top level class.

How will the private fields be available from the outer class? If those get into a top level class and are private, as they really are, then how will they be reachable from the outer class?

The way javac solves this issue that for any field, method or constructor being private but used from the top level class it generates a synthetic method. These synthetic methods are used to reach the original private filed/method/constructor. The generation of these methods are done in a clever way: only those are generated that are really needed and used from outside.

package synthetic;

import java.lang.reflect.Constructor;
import java.lang.reflect.Method;

public class SyntheticMethodTest2 {

    public static class A {
        private A(){}
        private int x;
        private void x(){};
    }

    public static void main(String[] args) {
        A a = new A();
        a.x = 2;
        a.x();
        System.out.println(a.x);
        for (Method m : A.class.getDeclaredMethods()) {
            System.out.println(String.format("%08X", m.getModifiers()) + " " + m.getName());
        }
        System.out.println("--------------------------");
        for (Method m : A.class.getMethods()) {
            System.out.println(String.format("%08X", m.getModifiers()) + " " + m.getReturnType().getSimpleName() + " " + m.getName());
        }
        System.out.println("--------------------------");
        for( Constructor<?> c : A.class.getDeclaredConstructors() ){
            System.out.println(String.format("%08X", c.getModifiers()) + " " + c.getName());
        }
    }
}

Since the name of the generated methods depend on the implementation and is not guaranteed the most I can say for the output of the above program is that on the specific platform where I executed it produced the following output:

2
00001008 access$1
00001008 access$2
00001008 access$3
00000002 x
--------------------------
00000111 void wait
00000011 void wait
00000011 void wait
00000001 boolean equals
00000001 String toString
00000101 int hashCode
00000111 Class getClass
00000111 void notify
00000111 void notifyAll
--------------------------
00000002 synthetic.SyntheticMethodTest2$A
00001000 synthetic.SyntheticMethodTest2$A

In the program above we assign value to the field x and we also call the method of the same name. These are needed to trigger the compiler to generate the synthetic methods. You can see that it generated three methods, presumably the setter and the getter for the fieldx and a synthetic method to the method x(). These synthetic methods, however, are not listed in the next list returned by getMethods() since these are synthetic methods and as such are not available for generic invocation. They are, in this sense, as private methods.

The hexa numbers can be interpreter looking at the constants defined in the class java.lang.reflect.Modifier:

00001008 SYNTHETIC|STATIC
00000002 PRIVATE
00000111 NATIVE|FINAL|PUBLIC
00000011 FINAL|PUBLIC
00000001 PUBLIC
00001000 SYNTHETIC

There are two constructors in the list. There is a private one and a synthetic one. The private exists, since we defined it. The synthetic on the other hand exists because we invoked the private one from outside. Bridge methods, however, do not had any so far.

Generics and inheritance

So good, so far, but we still did not see any “volatile” methods.

Looking at the source code of java.lang.reflec.Modifier you can see that the constant 0x00000040 is defined twice. Once as VOLATILE and once as BRIDGE (this latter is package private and is not for general use).

To have such a method a very simple program will do:

package synthetic;

import java.lang.reflect.Method;
import java.util.LinkedList;

public class SyntheticMethodTest3 {

    public static class MyLink extends LinkedList<String> {
        @Override
        public String get(int i) {
            return "";
        }
    }

    public static void main(String[] args) {

        for (Method m : MyLink.class.getDeclaredMethods()) {
            System.out.println(String.format("%08X", m.getModifiers()) + " " + m.getReturnType().getSimpleName() + " " + m.getName());
        }
    }
}

We have a linked list that has a method get(int) returning String. Let’s not discuss the clean code issues. This is a sample code to demonstrate the topic. The same issues come up in clean code as well, though more complex and harder to get to the point when it causes a problem.

The output says

00000001 String get
00001041 Object get

we have two get() methods. One that appears in the source code and another one, which is synthetic and bridge. The decompiler javap says that the generated code is:

public java.lang.String get(int);
  Code:
   Stack=1, Locals=2, Args_size=2
   0:   ldc     #2; //String
   2:   areturn
  LineNumberTable:
   line 12: 0


public java.lang.Object get(int);
  Code:
   Stack=2, Locals=2, Args_size=2
   0:   aload_0
   1:   iload_1
   2:   invokevirtual   #3; //Method get:(I)Ljava/lang/String;
   5:   areturn

The interesting this is that the signature of the two methods is the same and only the return types are different. This is allowed in the JVM even though this is not possible in the Java language. The bridge method does not do anything else, but calls the original one.

Why do we need this synthetic method? Who will use it. For example the code that wants to invoke the method get(int) using a variable that is no of the type MyLink:

        List<?> a = new MyLink();
        Object z = a.get(0);

It can not call the method returning String because there is no such in List. To make it more demonstrative lets override the method add() instead of get():

package synthetic;

import java.util.LinkedList;
import java.util.List;

public class SyntheticMethodTest4 {

    public static class MyLink extends LinkedList<String> {
        @Override
        public boolean add(String s) {
            return true;
        }
    }

    public static void main(String[] args) {
        List a = new MyLink();
        a.add("");
        a.add(13);
    }
}

We can see that the bridge method

public boolean add(java.lang.Object);
  Code:
   Stack=2, Locals=2, Args_size=2
   0:   aload_0
   1:   aload_1
   2:   checkcast       #2; //class java/lang/String
   5:   invokevirtual   #3; //Method add:(Ljava/lang/String;)Z
   8:   ireturn

not only calls the original one. It also checks that the type conversion is OK. This is done during run-time not done by the JVM itself. As you expect it does throw up in the line 18:

Exception in thread "main" java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
	at synthetic.SyntheticMethodTest4$MyLink.add(SyntheticMethodTest4.java:1)
	at synthetic.SyntheticMethodTest4.main(SyntheticMethodTest4.java:18)

When you get the question about volatile methods at an interview next time, you may know even more than the interviewer.

Blog Service Message about syndication with DZONE and JCG

I have contracted DZONE and equally Java Code Geeks as syndication partners. They started to select my articles for republishing on their sites.

The cooperation between me and the syndication partners is non-profit and I truly believe that this is mutually benefiting all parties including the syndication partners as well as you, the reader. Cooperation with syndication partners will hopefully increase the number of people using the blog and belonging to a larger technical community is an advantage.

Feel free to visit these sites and use the services, read articles, express your point of view in forms of comments, whatever you feel appropriate.

Never Test Logging

Technical logging is usually not tested. As commentator write on stack overflow:

… I practice TDD/BDD pretty religiously and I almost never test logging. With some exceptions logging is either a developer convenience or a usability factor, not part of the method’s core specification.

There is also a technical side why developers are reluctant, as Jon writes on the same page:

It’s a pain, either making the production code messy (due to injecting the logger) or the test smelly (replacing the static logger with a mock).

After those two statements we have to stop and think for a while. (After all, thinking never hurts, does it?) When we are talking about logging, do we mean the logging as a function or the tools that we use? Many times there is no difference: we use logging tools for logging. Absolutely logical. On the other hand when somebody asks a question about how to test logging there is a good chance that s/he is using the logging tool for something else than logging.

Using logging tools and logging functionality are sometimes not the same.

When testing logging comes into picture you should feel code smell.

Testing Logging Functionality

The first question that we have to answer is : what is logging as a functionality? What is it for? (And this time this is not about deforestation.)

When you write statements, like log.debug("accountIsDisabled() returned true");, is there any functional specification that you fulfill? I bet there is none. Technical logging is not a functional requirement. Logging is used to help the developer and the support people to better understand the behavior of the program, when something non expected happens in the program. This is not something that is inherent to the core functionality of the code.

The important fraction of the above sentences is “when something non expected happens”. I hear the roar of junior and semi senior developers: “We also log when something expected but exception occurs, like database connection dropped.” Well, my friend, let me tell you that you only think you log. You actually do not log. You alert. You presumably use some logging tool to perform alerting and this is what makes you think that you do logging. In reality, however, you are not. And this is very important.

I do not say that you should not use a logging tool for anything else other that logging. You can send alerts to a file, send SMS, tweet, whatever using a special log4j appender. No problem. However make sure that this is the best choice from the available tools. If you think you are logging, if you are not aware that you are actually alerting you prevent yourself realizing that you perhaps use a sub optimal tool for the purpose. When you send anything through your log tool’s drain to a log file that describes something, which is the description and the details of a well expected behavior then you should ask yourself the question: am I logging, or am I doing something else?

(Note: that something non-expected may happen outside of the program as well, in which case we also need logging. However that is not technical logging. Typically this is legal audit logging. You should test such logging.)

After we defined what I really mean when I talk about logging, my next statement is the following:

You should not test technical logging!

The statement may be shocking the first time. Why did not I write “you need not test”? Simply because there is nothing in programming that you “may but need not do” if you are a professional. You and your team have a goal. It includes product, time, budget, quality and all other “such” things. You get there on a way paved with effort. You have to minimize this effort. Not for your own good, or because you are lazy, but for the shareholders sake. Effort is cost. They provide the budget not for your enjoyment, but rather for achieving a business goal. That is the way businesses work, and professional programmers operate in business. That is one of the mandatory requirements to be professional. If you need not do something to achieve the goals, then you should not. Otherwise you waste the money that is not yours.

If you still feel that there is a real business need to test logging then start to sniff. This is code smell again. You probably are not logging, only using logging tools.

Testing Logging Tools Functionality

When you use a logging tool for something other than logging then you may well want to do some testing. Assume you decided after careful and professional assessment of all the possible technical solutions that you will use a logging framework for something, which is not logging. Alerting for example. In that case you want to test that your code uses the logging appropriately. Then comes the issue with the private static final loggers that you can not overwrite even using reflection. (You may succeed if you try, but that is against the JLS and JVM standards and may not always work.)

But again: this is not logging, this is using only the logging tool for some function, say alerting. Alerting functionality should be coded testable. In that case put aside the static loggers and focus on functionality. Separate the technical logging from alerting and properly inject dependency and mock the objects as usual during testing. Wrap alerting into a separate class, package and mock that while testing and test the wrap separately.

Whenever you program something to be tested you have to code it testable. Which is obvious since you develop your code using TDD.

Use Java annotation deprecated the right way

There is hardly anything more infuriating that seeing a method @Deprecated without a proper documentation. I feel lost. Should I still use the method? Probably that is not the intention of the developer, that is why he/she added the deprecation annotation. Should I use something else? So….

What are the rules using @Deprecated?

Rule #1: do Javadoc how not to

Whenever you deprecate a method create a JavaDoc that tells the programmer how not to use this method any more. Do not only say that “this method is deprecated, don’t use it”. That is exactly what the deprecation annotation and the JavaDoc @deprecated word says. There is no point to repeat it in English. The targeted audience is Java programmer, supposed to know what deprecation means.

Name the new methods, that replace the old one. (Use @link!) This may or may not be enough. The new method will have some documentation that explains how to use it. Do not repeat that (text or meaning) in the JavaDoc. Just do not repeat yourself, documentation should also be DRY. On the other hand you may want to describe how to replace the old call with the new one. You may give hint for the refactoring.

Rule #2: do not Javadoc how to

Remove the old JavaDoc documentation. Some may argue that users who maintain the legacy code may still need that. The fact is that they use an old version of the method in an old version of a library. The documentation in the old version is still there, frozen carved into stone (or rather carved into a release in the repository). The actual version that deprecates the method should not contain the outdated documentation. That would encourage the programmers to keep on using the method. There is one single way to use a deprecated method: not using it. JavaDoc should be current describing only that, as described above in rule#1.

Rule #3: no apology in JavaDoc

Do not explain in the JavaDoc why the method was deprecated. You are a responsible developer. This is your decision. You made your choice. Others have to live with it. Write a blog about the background of the architectural decision if you wish. It may be helpful, but JavaDoc is not the place for it.

Deprecated API JavaDoc is exclusively to explain how not to use.

The emphasis is on how. Not “why not to use it”.

Rule #4: do deprecate

If you feel like needing to deprecate a method: do it! If you are afraid of your users and you do not want to make their life miserable deprecating some method this decision will make your life miserable instead. Do all measures to have an API that will not need deprecation so long as long possible. But if ever something needs to be deprecated: throw it right away. Do not feel guilty why you did not see the future when the api was designed. None of us sees the future perfect. After all, life would be boring knowing the future.

If you are interested on opinions about this topic on stakcoverflow visit the linked page. Or just start a flame war, if you wish, here. I am totally trollerant.