Monthly Archives: January 2013

No loop, do break

axReading the standard of the Java language I found something interesting. Something that implies that break can be used without a loop, and not only inside a loop but inside any block. And it does:

package wierdo;

public class Wierdo {
	public static void main(String[] args) {
		label: {
			if (args.length == 0)
				break label;
			System.out.println("We happy, we have arguments!");
		}
		System.out.println("Hello Wierldo!");
	}
}

Weird, is it? You can read some more on this.

Practical consequences? If you are an architect and you work with subordinates: keep an axe by your side to chop off hands writing those in prod code. If you are coders: mind the architect approaching with the axe. (Just kidding…)

Advertisements

Saving Bytes

In the good old day’s we were hacking Z80 machine code and we were writing assembly on paper. I had a laminated card with all the opcodes and I had to look up the codes from the card and write on the paper along the assembly. The calculation of relative jumps was also a good practice, which is nowadays extinct. At third grade I wrote a small booklet on Z80 programming with some sample programs and explanation that was replicated in the official printer factory of the Uni. PDF, eMail? It was late ’80. I used Tasword. Can any of you recall?

Those days we were competing on who can write the shorter program for a given problem. There were many tricks that we could use to save up bytes. We created code that used the byte offset of the jump instruction as an op code depending on where the program was coming from and the decimal BCD correction instructions in totally different way they were supposed to be used. Good old days.

These days we do not need to save up bytes and I am happy about that. We can focus on problems that make more sense and higher level. These low level issues are to be solved by compilers, loaders and after all we have plenty of memory these days. My mac has 8GB memory, which is 175000 times more than the memory than the ZX Spectrum had. If we would line up that many Spectrums side by side on the road it would make 61km, that is 38 miles. And the compilers and optimizers also know this and they are not keen on spending much CPU time to find some clever way to compress the program. So good so far. Usually.

Some days ago on a forum I was discussing with some guy about clean code practice and I said that any method argument variable in Java should have been final. And it is a good practice to write the final keyword in front of each argument variable and this is a language design flaw that these variables are not final by definition. The reply was that some of the variables are altered and in this case a new local variable is to be used. Waste of memory. Well, yes, and no. The JIT compiler in HotSpot JVM will optimize that. “Too late” the answer came. By the time the JIT optimizes the JVM code was already loaded into the limited memory phone. And javac does not optimize too much: should not, that would be premature optimization.

So we are saving bytes again? Unfortunately. The compiler and the other tools are designed for the major use. Niche use should use niche tools if exist. Some do. But then phones also tend to have huge enough memory to accommodate our code. Then the vacuum cleaners will come, and the light switches. But at some time all memory shortage will disappear, no? Well, I have to say no. Memory, perhaps. Then comes the bandwidth shortage. You will want to download some program to your personal space-ship that is 2h light speed away close to Jupiter. Hand shaking, error corrections, lag… bytes count again. Then this is solved later and something new comes, perhaps some swarm effect in hyperspace that we can not even imagine today.

What is the morale? No matter what happens old technologies and tricks come back from time to time. They will perhaps have slightly different format, but the root problem is always the same: do not waste resources. Save the trees, eat … whatever.

How to mock the System.class

In this article I will show you a way to test code that uses static methods from a final class and how to mock that class. The example to mock is the System class. (We are not playing in the sand, we are real warriors.) We will use mockito, powermock, maven, eclipse and lots of brain of yours to follow. (You are also a java warrior after all!)

The Problem

When you unit test your application you usually use some mocking framework. Some of the most known are EasyMock and Mockito. There are others as well but for now I am not going to talk about these. They are light weight tools when you are really heavily into testing. Especially when you are creating test code that was created without caring about testability. Consider for example the following code fragment:

    public String getConfigValue(final String key) {
        String configValue = null;
        final String envKey = "sb4j." + key;
        if (configProperties != null && configProperties.containsKey(key)) {
            configValue = configProperties.getProperty(key);
        }
        String sysValue = null;
        if ((sysValue = System.getenv(envKey)) != null) {
            configValue = sysValue;
        }
        if ((sysValue = System.getProperty(envKey)) != null) {
            configValue = sysValue;
        }
        return configValue;
    }

A very simple code that tries to look up some configuration key in a Properties type variable (not defined here in the fragment, it is defined as a field in the class) but the configuration value for a certain key can be redefined in the environment variables and in the system properties (those defined with the java option -D on the command line). The strongest is the system property. If that is defined everything else is irrelevant. The second strongest is the environment variable, and the final choice is the configuration properties variable that are read from a .properties file (reading also not listed here to save space).

You may notice that the system property and the environment variable names are prefixed using the string “sb4j.” You can guess that this code fragment is from ScriptBasic for Java while it was under development.

The code is simple and seems to be OK, but trust me (not because I am engineer [as a matter of fact I am], but because of my experience): no code can be so simple that it can not contain a bug. I have learnt it modifying a method once, simpler than this above and since I had ten more minutes before heading towards home not to miss my movie for the evening I wrote a unit test. The movie was long time over when I finished with the fifth unit test that I created that night: every new bug you find deserves its own unit test.

We have to write unit test for the code. We need to mock the external classes that are used by the code. Some of them at least. We have the classes java.util.Properties, java.lang.String and java.lang.System. Obviously we need not mock String. Even though this is a class, it is almost like a primitive type. Similarly we need not mock Properties. Whatever the mock could provide us a stub instance of the Properties class can provide. We will not be able to check that the properties were really read using containsKey and getProperty method calls but if we get back the value we inserted into the stub properties variable we should be ok.

What we need to mock however is System and we have to mock the static methods getenv and getProperty.

The Solution

To do that we have to use Powermock. This is an extension to EasyMock and to Mockito (my fav is the second over the first one) and gives methods that let us mock static methods. To do so it needs to craft some hefty things into the Java byte code that I would not ever like to have in a production code except for testing and mocking. I am not knowledgeable how power mock really works, but I have the feeling that they are poking some areas that are beyond the official Java contract. I never mind this at this moment. Lets go and prepare the test.

To do so we need a unit test and we have to tell the test framework that we use Powermock so that a modified runner provided by Powermock is used instead of the standard JUnit runner. This is very simple to tell it actually, all we have to do is to annotate the class using the annotations:

@RunWith(PowerMockRunner.class)
@PrepareForTest({ System.class })
public class TestBasicConfiguration {

RunWith is org.junit.runner.RunWith and it is processed by the JUnit framework. PrepareForTest is org.powermock.core.classloader.annotations.PrepareForTest and informs PowerMock that it has to prepare the list of classes (in our case a single class, called System) to be mocked.

For your reference I include here the full source code of the test with all test methods as they were in the test class:

package com.scriptbasic.configuration;

import java.util.Properties;

import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mockito;
import org.powermock.api.mockito.PowerMockito;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.PowerMockRunner;

import com.bdd.Business;
import com.scriptbasic.interfaces.Configuration;

@RunWith(PowerMockRunner.class)
@PrepareForTest({ System.class })
public class TestBasicConfiguration {

    private static final Configuration config = new BasicConfiguration();
    private static final String key = "key";
    private static final String value = "value";
    private static final String badValue = "badValue";

    @Test
    public void testGetConfigValueWhenConfigKeyExistsShallReturnValue() {
        new Business() {
            private String actual;
            private Properties props;

            @Override
            public void given() {
                props = Mockito.mock(Properties.class);
                config.setConfigProperties(props);
                Mockito.when(props.containsKey(key)).thenReturn(true);
                Mockito.when(props.getProperty(key)).thenReturn(value);
            }

            @Override
            public void when() {
                actual = config.getConfigValue(key);
            }

            @Override
            public void then() {
                Mockito.verify(props).containsKey(key);
                Mockito.verify(props).getProperty(key);
                Assert.assertEquals(value, actual);
            }

        }.execute();
    }

    @Test
    public void testGetConfigValueWhenEnvironmentKeyExist() {
        String actual;
        Properties props;
        // GIVEN
        props = PowerMockito.mock(Properties.class);
        PowerMockito.mockStatic(System.class);
        config.setConfigProperties(props);
        Mockito.when(props.containsKey(key)).thenReturn(true);
        Mockito.when(props.getProperty(key)).thenReturn(badValue);
        Mockito.when(System.getenv("sb4j." + key)).thenReturn(value);
        // WHEN
        actual = config.getConfigValue(key);
        // THEN
        Mockito.verify(props).containsKey(key);
        Mockito.verify(props).getProperty(key);
        PowerMockito.verifyStatic();
        System.getenv("sb4j." + key);
        Assert.assertEquals(value, actual);
    }

    @Test
    public void testGetConfigValueWhenEnvironmentKeyExists() {
        new Business() {
            private String actual;
            private Properties props;

            @Override
            public void given() {
                props = PowerMockito.mock(Properties.class);
                PowerMockito.mockStatic(System.class);
                config.setConfigProperties(props);
                Mockito.when(props.containsKey(key)).thenReturn(true);
                Mockito.when(props.getProperty(key)).thenReturn(badValue);
                Mockito.when(System.getenv("sb4j." + key)).thenReturn(value);
            }

            @Override
            public void when() {
                actual = config.getConfigValue(key);
            }

            @Override
            public void then() {
                Mockito.verify(props).containsKey(key);
                Mockito.verify(props).getProperty(key);
                PowerMockito.verifyStatic();
                System.getenv("sb4j." + key);
                Assert.assertEquals(value, actual);
            }

        }.execute();
    }

    @Test
    public void testGetConfigValueWhenSystemPropertyExists(){
        new Business() {
            private String actual;
            private Properties props;
            
            @Override
            public void given() {
                props = PowerMockito.mock(Properties.class);
                PowerMockito.mockStatic(System.class);
                config.setConfigProperties(props);
                Mockito.when(props.containsKey(key)).thenReturn(true);
                Mockito.when(props.getProperty(key)).thenReturn(badValue);
                Mockito.when(System.getenv("sb4j." + key)).thenReturn(badValue);
                Mockito.when(System.getProperty("sb4j." + key)).thenReturn(value);
            }
            
            @Override
            public void when() {
                actual = config.getConfigValue(key);
            }
            
            @Override
            public void then() {
                Mockito.verify(props).containsKey(key);
                Mockito.verify(props).getProperty(key);
                PowerMockito.verifyStatic();
                System.getenv("sb4j." + key);
                PowerMockito.verifyStatic();
                System.getProperty("sb4j." + key);
                Assert.assertEquals(value, actual);
            }
        }.execute();
    }
}

For demonstration purposes some comments were deleted, but other than those, the code above is complete.
PowerMock is powerful and makes life easy when testing static method, but not that powerful. If we execute maven, we get the following error:

Results :

Failed tests: 
  testGetConfigValueWhenEnvironmentKeyExist(com.scriptbasic.configuration.TestBasicConfiguration): 

Tests in error: 
  testGetConfigValueWhenEnvironmentKeyExists(com.scriptbasic.configuration.TestBasicConfiguration): 
  testGetConfigValueWhenSystemPropertyExists(com.scriptbasic.configuration.TestBasicConfiguration): 

Tests run: 56, Failures: 1, Errors: 2, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 20.666s

What is the problem? Let us have a look at the surefire reports file:

$ cat target/surefire-reports/com.scriptbasic.configuration.TestBasicConfiguration.txt 
-------------------------------------------------------------------------------
Test set: com.scriptbasic.configuration.TestBasicConfiguration
-------------------------------------------------------------------------------
Tests run: 4, Failures: 1, Errors: 2, Skipped: 0, Time elapsed: 3.455 sec <<< FAILURE!
testGetConfigValueWhenEnvironmentKeyExist(com.scriptbasic.configuration.TestBasicConfiguration)  Time elapsed: 0.83 sec  <<< FAILURE!
Wanted but not invoked java.lang.System.getenv("sb4j.key");
Actually, there were zero interactions with this mock.
	at org.powermock.api.mockito.internal.invocationcontrol.MockitoMethodInvocationControl.performIntercept(MockitoMethodInvocationControl.java:292)
	at org.powermock.api.mockito.internal.invocationcontrol.MockitoMethodInvocationControl.invoke(MockitoMethodInvocationControl.java:194)
	...

testGetConfigValueWhenEnvironmentKeyExists(com.scriptbasic.configuration.TestBasicConfiguration)  Time elapsed: 0.083 sec  <<< ERROR!
org.mockito.exceptions.misusing.MissingMethodInvocationException: 
when() requires an argument which has to be 'a method call on a mock'.
For example:
    when(mock.getArticles()).thenReturn(articles);

Also, this error might show up because:
1. you stub either of: final/private/equals()/hashCode() methods.
   Those methods *cannot* be stubbed/verified.
2. inside when() you don't call method on mock but on some other object.

	at com.scriptbasic.configuration.TestBasicConfiguration$2.given(TestBasicConfiguration.java:109)
	at com.bdd.Business.execute(Business.java:17)

You can see on the highlighted code the relevant error messages. System was NOT mocked, and not because it is a system class but rather because this is final. Looking at the code in RT you can see that it really is, just as you can also have a look at the documentation: http://docs.oracle.com/javase/7/docs/api/java/lang/System.html

We are doomed? We can not test this code?

The good approach, generally is, to move all these external dependencies to a utility class, having static methods proxying the call to the System class. In that case we can mock the utility class and life is beautiful again. To be honest, I could do that in the example case. The developed code of sb4j was in my hands and I could modify it any way I wanted. But the idea to overcome this issue just did not leave my mind. There is a problem that I can have a workaround for and not a solution. The solution is to test the code as it is now!

If I only had a System class that was not final. Hey!! Wait!! I can have a class named System that is not final:

public class System {
    public static String getenv(String key) {
        return java.lang.System.getenv(key);
    }

    public static String getProperty(String key) {
        return java.lang.System.getProperty(key);
    }
}

If I place this class in the same package as the tested code then the compiler will compile the tested class again this System stub instead of the java.lang.System and this class can be mocked. But this alter the code base. Does not change the code itself but the class file at the end calls this stub, which calls system and this affects performance. After all this is all about reading configuration, so performance should not be a big issue, but even though: I just do not like it. And if the production code contains the stub System then it also needs testing even if it as simple as cold water. And to test it I have to mock java.lang.System and the circle of hell just closed.

What if this code would only be compiled against the code for the testing and not for the compilation of the production code? In that case I test the original code without modification and the production .class is not influenced by the test needs. Even more: if I use my stub System class only for testing, it does not need to proxy the methods getenv and getProperty to java.lang.System since it will never been used in production code. And we do not need testing it since this is not production code.

Interesting is it? Let’s give it a try. Let me create a class under src/test/java in the same class as the above code containing a simpler version of the above class:

public class System {
    public static String getenv(String key) {
        return null;
    }

    public static String getProperty(String key) {
        return null;
    }
}

And executing the code in Eclipse: tadam! It works. Problem solved.

Almost. Running mvn clean install from the command line I get exactly the same error as the above. The version of the class used for testing is still compiled against the java.lang.System. On second thought this is fine. When we compile using maven we get one set of classes from our java sources. If you look at the maven log on the screen, you see:

[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ jscriptbasic ---
[INFO] Compiling 243 source files to ... target/classes
[INFO] 
[INFO] --- maven-resources-plugin:2.4.3:testResources (default-testResources) @ jscriptbasic ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 31 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ jscriptbasic ---
[INFO] Compiling 38 source files to ... target/test-classes

that the test compilation compiles only the test classes, which are few in this case compared to the production classes.
Note that I deleted the full path from the printout.
When the tests run the directories target/test-classes and targed/classes are on the classpath in this order. It means that test classes are found first and production classes are found the second. If I want to have a version of the tested class for testing then I have to compile it twice: once for the production code and once for the test. There will be two versions of the class on the classpath: one in the target/test-classes directory and one in the targed/classes directory, and the test framework will perform the tests on the first one, “linked” against the stub System.

The only issue that remained is how to tell maven to compile the source code again for testing. There is no easy solution as maven, by default is handling sources from a single directory. I also do not want to have the test classes in the production JAR file and even more I do not want to have the production code compiled against my stub.

The help comes with the maven helper plugin that can be used to configure extra directories for the test compilation. Actually this is a bit of hack: we tell the compiler that there are “generated” sources (ha ha 😀 ) for the test:

<plugin>
	<groupId>org.codehaus.mojo</groupId>
	<artifactId>build-helper-maven-plugin</artifactId>
	<version>1.4</version>
	<executions>
		<execution>
			<id>add-test-source</id>
			<phase>generate-sources</phase>
			<goals>
				<goal>add-test-source</goal>
			</goals>
			<configuration>
				<sources>
					<source>${basedir}/src/main</source>
				</sources>
			</configuration>
		</execution>
	</executions>
</plugin>

And finally we are there. What we gained is that

  • we tested the source code
  • we did mock the System.class
  • there is no production code left untested

The drawbacks are

  • we tested the code, but a different compilation, not the same .class
  • all production classes are compiled twice

The Conclusion

There was a code that used System static methods directly and needed testing. Usually those dependencies should be “moved far away” from the code and the code should be crafted bearing in mind the testability. I did not discuss how to make the code testable. I only focused on the technical issue: how to test the given class without modifying the source code.

What we test actually is the source code. Usually we do not make any distinction between testing the source code or testing the compiled class file. In this case there is difference and the approach found does test the source code compiled to a test class file. Does it matter?

When we execute unit tests we use mocks. The test class uses a to-be-mocked stub implementation of the class that we need to mock instead of the original. If we could mock the original the test would execute exactly the same way. For this reason I see no real risk in this approach. We never measure what we are interested in. In this case, how the code behaves in operation. You can not test that. Not only in practice, but also theory supports that: you can not tell if Schrödinger’s cat is dead or alive. Instead we measure something that is close to the thing we really are interested in. In this case we test the compiled code in a test environment. Now using this approach we test the source code compiled to a test class in a test environment. We moved a bit from the usual measuring point, however I believe that the move was tangential, and the distance did not change.

My final note below this article: if you need to mock static methods, and especially methods from the system class: start to consider refactoring the code.

Does javac do optimization? Does not seem…

We usually say that java programmers have to write code that looks good and all other issues are solved by the compiler. For example having a complex boolean expression is better moved to a separate method with a good name and with a single return statement containing the expression. The original if or while will be much easier to understand. The java compiler is clever enough to see that the code is only called from a single place and will move the code inline.

Is it really true? I have heard that the JIT compiler does the optimization and the javac compiler does not. Let us have a look at some simple class:

public class OptimizeThis {
	private int a(int x, int y) {
		return x + y;
	}

	public int add(int x, int y, int z) {
		return a(a(x, y), z);
	}
}

There is a lot of space for optimization. The method a() could be left out from all the fun. The code could be included in the method add() and the code would be much faster.
Something like this:

public class Optimized {
	public int add(int x, int y, int z) {
		return x + y + z;
	}
}

Let us compile the class OptimizeThis and disassemble using javap:

verhasp:java verhasp$ javac OptimizeThis.java
$ javap -v -p OptimizeThis.class
Classfile /Users/verhasp/.../src/main/java/OptimizeThis.class
  Last modified 2012.07.08.; size 327 bytes
  MD5 checksum 9ba33fe0979ff0948a683fab2dc32d12
  Compiled from "OptimizeThis.java"
public class OptimizeThis
  SourceFile: "OptimizeThis.java"
  minor version: 0
  major version: 51
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
   #1 = Methodref          #4.#15         //  java/lang/Object."<init>":()V
   #2 = Methodref          #3.#16         //  OptimizeThis.a:(II)I
   #3 = Class              #17            //  OptimizeThis
   #4 = Class              #18            //  java/lang/Object
   #5 = Utf8               <init>
   #6 = Utf8               ()V
   #7 = Utf8               Code
   #8 = Utf8               LineNumberTable
   #9 = Utf8               a
  #10 = Utf8               (II)I
  #11 = Utf8               add
  #12 = Utf8               (III)I
  #13 = Utf8               SourceFile
  #14 = Utf8               OptimizeThis.java
  #15 = NameAndType        #5:#6          //  "<init>":()V
  #16 = NameAndType        #9:#10         //  a:(II)I
  #17 = Utf8               OptimizeThis
  #18 = Utf8               java/lang/Object
{
  public OptimizeThis();
    flags: ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #1                  // Method java/lang/Object."<init>":()V
         4: return
      LineNumberTable:
        line 1: 0

  private int a(int, int);
    flags: ACC_PRIVATE
    Code:
      stack=2, locals=3, args_size=3
         0: iload_1
         1: iload_2
         2: iadd
         3: ireturn
      LineNumberTable:
        line 3: 0

  public int add(int, int, int);
    flags: ACC_PUBLIC
    Code:
      stack=4, locals=4, args_size=4
         0: aload_0
         1: aload_0
         2: iload_1
         3: iload_2
         4: invokespecial #2                  // Method a:(II)I
         7: iload_3
         8: invokespecial #2                  // Method a:(II)I
        11: ireturn
      LineNumberTable:
        line 7: 0
}
verhasp:java verhasp$

You can see we have both of the methods. The code

  private int a(int, int);
    flags: ACC_PRIVATE
    Code:
      stack=2, locals=3, args_size=3
         0: iload_1
         1: iload_2
         2: iadd
         3: ireturn

is the private method a() and the code

  public int add(int, int, int);
    flags: ACC_PUBLIC
    Code:
      stack=4, locals=4, args_size=4
         0: aload_0
         1: aload_0
         2: iload_1
         3: iload_2
         4: invokespecial #2                  // Method a:(II)I
         7: iload_3
         8: invokespecial #2                  // Method a:(II)I
        11: ireturn

is the public method add(). The code itself is simple. The method a() loads on the operand stack the first local variable (iload_1), then it does the same with the second (iload_2), and then adds the two (iadd). What is left on the operand stack is used to return (ireturn).

  1. the local variable number zero is this in case of non-static methods
  2. the arguments are also treated as local variables
  3. for the first few local variables there are shorthand java byte codes, because the generated code accesses these the most and this saves some space and speed
  4. we are using int only and thus we need not care about more complex issues, like a double occupying two slots.

Them method add() is almost as simple. Loads the value of this on the operand stack two times. It is needed to call the non-static method a(). After that it loads the first and the second local variable on the stack (the first two method arguments) and in the command #4 (line 61.) calls the method a(). After this it loads the third local variable on the stack. This time the stack contains the variable this, the result of the previous call to method a() and then the third local variable, which is the third argument to the method add(). Then it calls the method a().

Let us have a look at the code generated from the class Optimized:

$ javap -v -p Optimized.class
Classfile /Users/verhasp/.../src/main/java/Optimized.class
  Last modified 2012.07.08.; size 251 bytes
  MD5 checksum 2765acd1d55048184e9632c1a14a8e21
  Compiled from "Optimized.java"
public class Optimized
  SourceFile: "Optimized.java"
  minor version: 0
  major version: 51
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
   #1 = Methodref          #3.#12         //  java/lang/Object."<init>":()V
   #2 = Class              #13            //  Optimized
   #3 = Class              #14            //  java/lang/Object
   #4 = Utf8               <init>
   #5 = Utf8               ()V
   #6 = Utf8               Code
   #7 = Utf8               LineNumberTable
   #8 = Utf8               add
   #9 = Utf8               (III)I
  #10 = Utf8               SourceFile
  #11 = Utf8               Optimized.java
  #12 = NameAndType        #4:#5          //  "<init>":()V
  #13 = Utf8               Optimized
  #14 = Utf8               java/lang/Object
{
  public Optimized();
    flags: ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #1                  // Method java/lang/Object."<init>":()V
         4: return
      LineNumberTable:
        line 1: 0

  public int add(int, int, int);
    flags: ACC_PUBLIC
    Code:
      stack=2, locals=4, args_size=4
         0: iload_1
         1: iload_2
         2: iadd
         3: iload_3
         4: iadd
         5: ireturn
      LineNumberTable:
        line 3: 0
}

Much simpler. Is it also faster? The proof of the pudding is in the eating. If it is not tasty then the dog will eat it. However…

Here we have again the two classes extended with some main methods (one for each).

public class OptimizeThis {
	private int a(int x, int y) {
		return x + y;
	}

	public int add(int x, int y, int z) {
		return a(a(x, y), z);
	}

	public static void main(String[] args) {
		OptimizeThis adder = new OptimizeThis();
		final int outer = 100_0000_000;
		final int loop = 100_0000_000;
		Long tStart = System.currentTimeMillis();
		for (int j = 0; j < outer; j++) {
			for (int i = 0; i < loop; i++) {
				int x = 1;
				int y = 2;
				int z = 3;
				adder.add(x, y, z);
			}
		}
		Long tEnd = System.currentTimeMillis();
		System.out.println(tEnd - tStart);
	}
}

and

public class Optimized {
	public int add(int x, int y, int z) {
		return x + y + z;
	}

	public static void main(String[] args) {
		Optimized adder = new Optimized();
		final int outer = 100_0000_000;
		final int loop = 100_0000_000;
		Long tStart = System.currentTimeMillis();
		for (int j = 0; j < outer; j++) {
			for (int i = 0; i < loop; i++) {
				int x = 1;
				int y = 2;
				int z = 3;
				adder.add(x, y, z);
			}
		}
		Long tEnd = System.currentTimeMillis();
		System.out.println(tEnd - tStart);
	}
}

In addition to this we also created an empty class, named Empty that returns constant zero.

public class Empty {
	public int add(int x, int y, int z) {
		return 0;
	}

	public static void main(String[] args) {
		Empty adder = new Empty();
		final int outer = 100_0000_000;
		final int loop = 100_0000_000;
		Long tStart = System.currentTimeMillis();
		for (int j = 0; j < outer; j++) {
			for (int i = 0; i < loop; i++) {
				int x = 1;
				int y = 2;
				int z = 3;
				adder.add(x, y, z);
			}
		}
		Long tEnd = System.currentTimeMillis();
		System.out.println(tEnd - tStart);
	}
}

Here we have an executing script that can be called after executing the command javac *.java:

#! /bin/sh
echo "Empty"
java Empty
echo "Optimized"
java Optimized
echo "OptimizeThis"
java OptimizeThis

And the result:
STOP!!!! Before you open it try to estimate the ration between the optimized and non-optimized version and also how much faster the class Empty is. If you have your estimation you can open the result:

verhasp:java verhasp$ ./testrun.sh
Empty
1970
Optimized
1987
OptimizeThis
1970
verhasp:java verhasp$ ./testrun.sh
Empty
1986
Optimized
2026
OptimizeThis
2001
verhasp:java verhasp$ ./testrun.sh
Empty
1917
Optimized
1892
OptimizeThis
1899
verhasp:java verhasp$ ./testrun.sh
Empty
1908
Optimized
1903
OptimizeThis
1899
verhasp:java verhasp$ ./testrun.sh
Empty
1898
Optimized
1891
OptimizeThis
1892
verhasp:java verhasp$ ./testrun.sh
Empty
1896
Optimized
1896
OptimizeThis
1897
verhasp:java verhasp$ ./testrun.sh
Empty
1897
Optimized
1903
OptimizeThis
1897
verhasp:java verhasp$ ./testrun.sh
Empty
1908
Optimized
1892
OptimizeThis
1900
verhasp:java verhasp$ ./testrun.sh
Empty
1899
Optimized
1905
OptimizeThis
1904
verhasp:java verhasp$ ./testrun.sh
Empty
1891
Optimized
1896
OptimizeThis
1896
verhasp:java verhasp$ ./testrun.sh
Empty
1895
Optimized
1891
OptimizeThis
1904
verhasp:java verhasp$ ./testrun.sh
Empty
1898
Optimized
1889
OptimizeThis
1894
verhasp:java verhasp$ ./testrun.sh
Empty
1917
Optimized
1894
OptimizeThis
1898
verhasp:java verhasp$

Conclusion? Before you vote on the first choice read all the possible answers!


The tests were executed on a 8GB memory MacBook Pro7,1 with OS X 10.7.4, 7-es Java (you could notice it that it was already java7) Still here you can have the output of ‘java -version’:

verhasp:java verhasp$ java -version
java version "1.7.0_04"
Java(TM) SE Runtime Environment (build 1.7.0_04-b21)
Java HotSpot(TM) 64-Bit Server VM (build 23.0-b21, mixed mode)


Next time something more interesting.

A little bit of testing

Behaviour Driven Development usually says that the test structure consists of

  • GIVEN
  • WHEN
  • THEN

Even Mockito has an DBBMockito façade implementation that fits to this habit. If you want a big shot you can have JBehave. However for now let us starts with some simple unit test:

    private static final Configuration config = new BasicConfiguration();
    private static final String key = "key";
    private static final String value = "value";
    private static final String badValue = "badValue";
...
    public void testGetConfigValueWhenEnvironmentKeyExists() {
        String actual;
        Properties props;
        // GIVEN
        props = PowerMockito.mock(Properties.class);
        PowerMockito.mockStatic(System.class);
        config.setConfigProperties(props);
        Mockito.when(props.containsKey(key)).thenReturn(true);
        Mockito.when(props.getProperty(key)).thenReturn(badValue);
        Mockito.when(System.getenv("sb4j." + key)).thenReturn(value);
        // WHEN
        actual = config.getConfigValue(key);
        // THEN
        Mockito.verify(props).containsKey(key);
        Mockito.verify(props).getProperty(key);
        PowerMockito.verifyStatic();
        System.getenv("sb4j." + key);
        Assert.assertEquals(value, actual);
    }

Do you feel something under you skin that this is not the real one? What are those comments? Whenever I feel the need to write comments into a method I start to think about better variable and method names and refactoring. Why is this method so complex that this is hard to understand without the comments? Yes, sure: I can remove those three comment lines, but even then it is not really an improvement.

What if we could create a simple Business class and we could book our ticked from the economy class to business class? We are not even forced to finish the class, it is OK half way ready:

public abstract class Business {

    abstract public void given();
    abstract public void when();
    abstract public void then();
    public void execute(){
        given();
        when();
        then();
    }

}

It is so small that it just fits any project. Whenever we have it the test code looks much better:

    @Test
    public void testGetConfigValueWhenEnvironmentKeyExists() {
        new Business() {
            private String actual;
            private Properties props;

            @Override
            public void given() {
                props = PowerMockito.mock(Properties.class);
                PowerMockito.mockStatic(System.class);
                config.setConfigProperties(props);
                Mockito.when(props.containsKey(key)).thenReturn(true);
                Mockito.when(props.getProperty(key)).thenReturn(badValue);
                Mockito.when(System.getenv("sb4j." + key)).thenReturn(value);
            }

            @Override
            public void when() {
                actual = config.getConfigValue(key);
            }

            @Override
            public void then() {
                Mockito.verify(props).containsKey(key);
                Mockito.verify(props).getProperty(key);
                PowerMockito.verifyStatic();
                System.getenv("sb4j." + key);
                Assert.assertEquals(value, actual);
            }

        }.execute();
    }

Much better, is it. The only question remaining: who in the hell could we mock the System class? Believe me: the code above is copy/paste without modification from a real live test and it works. Stay tuned for some coming posts where I detail how it was made.

Problem with Java Security


This article is about my thoughts about the currently emerged security issue related to Java. Not too technical and only general thoughts.

Recently a new vulnerability of the Java runtime emerged. You can have a look at http://nakedsecurity.sophos.com/2013/01/10/protect-yourself-against-latest-java-zero-day-vulnerability-now-maljavajar-b/ and http://nakedsecurity.sophos.com/2013/01/11/apple-and-mozilla-just-say-no-to-java/ and also http://www.pcworld.com/article/2025160/its-time-to-rewrite-java-from-scratch-security-expert-says.html

The latest article suggest not less than rewriting the security part of the Java system. The title exaggerates to rewriting Java as a whole. And this is one of the major, though not the largest problem with Java security that I will discuss in this article: journalism. But we will see that later. The first question I want to talk about is why this issue came up now and not earlier. Similar vulnerabilities were discovered in Java last year and that time I could not see articles stating that Java generally is insecure.

You can think of this process of awakening like the two stages of incident management and problem management that may sound familiar if you know ITIL. When the problem first appeared it was fixed. That is the most important thing because many businesses depend on Java and if Java turns to be unusable as a whole that costs a lot of money. Really a lot. Fortunately there is marginal probability to find Java insecure without reasonable way to fix the issue. Even if the issue is fixed the vulnerability costs us money. There can be security breaches during the period between the vulnerability discovery and the time the fix is applied. Applying the fix is also cost for many businesses and institutions, and those are not happy spending money for something that does not lead to higher income. Thus we raise the question: “this happened this time, but can we trust your technology that it will not happen in the future?”. To find the answer is problem management.

Institutions like men working in support are lazy. After all we are programmers down in our heart and programmers are lazy. Actually programmers have to be lazy to be good in their job. Because of that support people say “this was a one time problem, we fixed it, but it is not likely that there are more such bugs in the system”. Wrong. Totally wrong this time and, my personal opinion is that such answer and attitude is totally wrong at any time. If you find a cockroach under your bed, it is a vain hope that it was a lonely warrior. There will be more and you better call bug hunters. As for Java it became obvious this year on January 11, 2013. A similar bug to the one appeared a year ago emerged again. Dear ORACLE: what are you doing with it now? What is your answer to the “problem management” question. What will you do so that it does not happen any more time? We wait for the answer not being ready from the vendor side. In the meantime we may speculate what the answer can be. There are technical aspects to anticipate the answer as well as industry answers.

In this article I would like to express my thoughts on the topic. Before getting into the details I want to state that I am not a security expert. Even not being one you can not avoid dealing with security issues if you work for the IT industry for more than 20 years, which I do.

problem #1

Nothing is prohibited…

The first and most technical issue with Java considering security that the language and the runtime are rather permissive. When you start a Java application with some security context (discussed a bit later) you can do anything unless the security context forbids that. This is not a secure approach. The secure approach is to forbid everything that is not allowed. Java is just the other way. Why?

This is because secure systems are hard to use. Their use costs a lot because of the hardness of the use. To set the permissions properly so that our business functions smoothly you have to discover your business processes and set the permissions to that everybody can do what it needs to do to operate the business. Having a more exhaustive map of the business processes costs way more than the permissive approach. If you trust the players at your company you can use the permissive approach. There are some things that are definitely forbidden: for example your secretary can not give herself raise. On the other hand she can occasionally handle parcels arriving to the company when you are away. If the company controls are permissible there is no need to say that: it is common sense. When the players can not be trusted you need to be restrictive. You can replace a secretary you do not trust but you can not replace the Internet users. They, or rather we are the people of the earth with our human nature. Crime, fighting, war, on-line felony are inherent part of human nature. Live with it!

This control issue goes down to every aspect of your company from key distribution (I mean keys made of metal to doors) down to software, even Java. Java is permissive.

problem #2

Way too complex

When you start a Java application the security context that is meant to control the access to various resources is null. This is not the case for Java Applets, but I will talk about that later. It is possible to set the security context programmatically and most application servers do that. Opps… sorry. They actually don’t. They can be configured to do that. By default they do not. Why? Because it would be so difficult to use them if they forced the administrators to configure the security that the majority of their user base would be nonexistent. Security is complex. This is the unfortunate fact but it is not because of Java. It is a general fact. I have never seen any production installation of Tomcat configured with non null security context.

For applets this is not the case. Applets run in a sand box of the Internet browser and the Java programs are allowed to mangle the sand only in the box. The sand is a metaphor an in this case means files to be read, written, executed, ports to listen on, memory objects out of the process space of the applet thread and other operating system resources. The Java system, more specifically the JVM that executes the code uses these resources on behalf of the Java programs and it does not check security (I am not absolutely sure about that, but the depth I dig into showed that). Java code is supposed to be blocked by the JDK routines to reach that level. And there are many functions that have to be controlled by security. It is like a house that has many entrances. One for the house master, one for the kids, one for the maid and there is even some hole on the wall for the mice. Each one except the forgotten ones has guards. Some are good, some have some faults as we see recently. One is fixed does not guarantee there isn’t another somewhere else. Even these doors are connected with a complex maze of tunnels. Way too complex.

If you want security controlled you need something as simple as possible. The more complex the more bugs there can be. Have only one door and simple rules.

Forget Applets

The easiest solution to this problem on the problem management level is what Apple and Firefox did. Switch off Java for the browsers. Applets are Java programs that intended to give client side functionality to browsers. The idea was perhaps too early, or just became the victim of the Sun vs. MS war. It alone would be a long post to discuss the reasons. Whatever: Java applets are rarely used and are generally considered legacy. Most of the functionality that was meant to be solved using applets are solved now by JavaScript that developed enormously during the recent years. Even though applets are still around and the browsers support applets. There are a very few applications that need them. I personally know two. One is a home banking application. Terminate your account and contract a different bank! The other one I know and recently faces is Atlassian JIRA that uses applet to ease the upload of screen shots. You can upload the JPG file instead. No problem.

Thus my suggestion to solve the issue on the problem management level is to forget applets once and for all. At least that is the way we are heading.

problem #3

Scripting languages on the road

Would forgetting applets solve the issue? For the current issue yes, but again: this is incident management. From a different angle the issue remains. Java is not secured inside. If we want to use Java as language and runtime environment in which some of the players are not trusted then the architecture is not appropriate. I do not see such use in addition to applets at the moment but with the dawn of scripting languages and their viral spread on JVM raises similar issue again:

Soon we will want to run scripted programs inside the JVM. These programs will source from untrusted players who extend the functionality of the program. This is as blog engine’s HTML pages were extended by the HTML text of the blogger and the commenter. Does WordPress trust me that much? I doubt. Thus they make counter measures so that I can not inject XSS, click jacking and other naughty things (or can I?). We will create/use applications that allow the users to write simple scripts to perform operations that are tailor made and that are not readily available from top of the head of the application designers.

This situation is similar to the applet. In case of the applet the outside world has to be protected from untrusted code written in Java. In case of scripts the outside world including the JVM and Java code running in the application has to be protected from the scripting solution. Many of the scripting implementation do not go that way. Programs written in JRuby, Scala, Jython and so on are compiled to JVM byte code and run just as they were written in Java. The Java architecture however is far from being ready to protect one piece of Java code from the other running in the same JVM. There is no internal access control in the JVM.

We have a long way to go.

problem #4

Journalism

The aim of journalists is to gather as many readers as possible. Even I, writing this article, do it to have many readers. Not for money, though. To have readers the article has to be interesting and appealing. If the truth is not appealing enough a bit of color, slipping a bit the facts does not harm. Or does it?

Yes, it does, especially in the arena of security. People like to feel threatened. That is why horror movies and thrillers exist. That is the reason most of the people read security related articles. It helps the soul reassuring ourselves. The side effect is that some of the people not only shivers reading the articles but partially understands some of the statements. Not all, only some of them. And most of the people are not knowledgeable enough to judge the statements. If there are some bend in the facts, some statements slip then the perception may diverge from reality 180 degree. Just going the other way around.

The articles say that Apple and Firefox “say no to Java”. Actually they say no to Java applets and Java application started right from the web. Apple actually does that for all applications anyway unless you configure security setting to differ from default.

The title of the other article says: “It’s time to rewrite Java from scratch, security expert says”. If you read the article it talks about the security part of Java.

The casual reader from these will deduct that Java is wrong. That is not the case and if you believe something that is different from the fact you get poorer. You make decisions whether you want to use Java in your company or become a Microsoft shop based on your knowledge. Thus those exaggerations are good to the competitors only. Not for the users of the technology and at the end of the line the customers of the users of the technology.

Summary

Consider these as random thoughts only. I wanted to be as precise as I could, however security and Java security is not my major. These are just some spots on the whole issue and many areas remain darkened. Feel free to discuss what you agree or see my approach wrong.

My personal area that is a bit related to this area is ScriptBasic for Java, which is a scripting language NOT compiled to Java byte code, but rather interpreted by Java code (thus it actually is slower) and can be fully controlled by the embedding application what scripts can do. It was designed to provide a mean to application programmers to provide a tool to users to extend applications embedding the interpreter in a way that can not harm the application.

Enough of me. Now: what are your thoughts?