Java Deep

Pure Java, what else

Comparing Golang with Java

First of all I would like to make a disclaimer. I am not an expert in Go. I started to study it a few weeks ago, thus the statements here are kind of first impressions. I may be wrong in some of the subjective areas of this article. Perhaps I will write some time a review of this one later. But until then, here it is and if you are a Java programmer, you are welcome to see my feelings and experiences and the same time you are more than welcome to comment and correct me if I am wrong in some statements.

Golang is impressive

As opposed to Java, Go is compiled to machine code and is executed directly. Much like C. Because this is not a VM machine it is very much different from Java. It is object oriented and the same time functional to some extent thus it is not only a new C with some automated garbage collection. It is somewhere between C and C++ if we think the world of programming languages is on a single line, which it is not. Using a Java programmer’s eyes some things are so much different that learning them is challenging and may give a deeper understanding on programming language structures and how objects, classes and all these things are … even in Java.

I mean that if you understand how OO is implemented in Go, you may understand also some of the reasons why Java has them different.

In short if you are impatient: do not let yourself freak out by the seemingly weird structure of the language. Learn it and it will add to your knowledge and understanding even if you do not have a project to be developed in Go.

GC and not GC

Memory management is a crucial point in programming languages. Assembly lets you do all things. ​Or rather it requires you to do it all. In case of C there are some support functions in the standard library but it is still up to you to free all memory that you have allocated before calling malloc. Automated memory management starts somewhere along C++, Python, Swift and Java. Golang is also in this category.

Python and Swift use reference counting. When there is a reference to an object the object itself holds a counter that counts the number of references that point to it. There are no pointers or references backward, but when a new reference gets the value and starts to reference an object the counter increases and when a reference becomes null/nil whatever or references another object the counter goes down. So it is known when the counter is zero, there are no references to the object and it can be discarded. The problem with this approach is that an object still may be unreachable while the counter is positive. There can be object circles referencing each other and when the last object in this circle is released from the static, local and otherwise reachable references then the circle starts to float in the memory like bubbles in water: the counters are all positive but the objects are unreachable. The Swift tutorial has a very good explanation of this behavior and how to avoid it. But the point is still there: you have to care about memory management somewhat.

In case of Java, other JVM languages (including the JVM implementation of Python) the memory is managed by the JVM. There is a full blown garbage collection that runs from time to time in one or more threads, parallel with the working threads or sometimes stopping those (a.k.a. stop the world) marking the unreachable objects, sweeping them and compacting the presumably scattered memory. All you have to worry about is performance if at all.

Golang is also in this category with a small little, tiny exception. It does not have references. It has pointers. The difference is crucial. It can be integrated with external C code and for performance reasons there is nothing like a reference registry in the run-time. The actual pointers are not known to the execution system. The memory allocated can still be analyzed to gather reachability information and the unused “objects” can still be marked and swept off, but memory can not be moved around to do the compacting. This was not obvious for me some time from the documentation and as I understood the pointer handling I was seeking for the magic that Golang wizards implemented to do the compacting. I was sorry to learn, they simply did not. There is no magic.

Golang has a garbage collection but this is not a full GC as in Java, there is no memory compaction. It is not necessarily bad. It can go a long way running servers very long time and still not having the memory fragmented. Some of the JVM garbage collectors also skip the compacting steps to decrease GC pause when cleaning old generations and do the compacting only as a last resort. This last resort step in Go is missing and it may cause some problem is rare cases. You are not likely to face the problem while learning the language.

Local variables

Local variables (and sometimes objects in newer versions) are stored on the stack in Java language. So are in C, C++ and in other languages where call-stack as such is implemented. Golang related to local variables is no exception, except…

Except that you can simply return a pointer to a local variable from a function. That was a fatal mistake in C. In case of Go the compiler recognizes that the allocated “object” (I will explain later why I use quotes) is escaping the method and allocates it accordingly so that survives the return of the function and the pointer will not point to an already abandoned memory location where there is no reliable data.

So this is absolutely legal to write:

package main

import (

type Record struct {
	i int

func returnLocalVariableAddress() *Record {
	return &Record{1}

func main() {
	r := returnLocalVariableAddress()
	fmt.Printf("%d", r.i)


What is more you can write functions inside functions and you can return functions just like in a functional language (Go is a kind of functional language) and the local variables around it serve as variables in a closure.

package main

import (

func CounterFactory(j int) func() int {
	i := j
	return func() int {
		return i

func main() {
	r := CounterFactory(13)
	fmt.Printf("%d\n", r())
	fmt.Printf("%d\n", r())
	fmt.Printf("%d\n", r())

Function return values

Functions can return not only one single value, but multiple values. This seems to be a bad practice if not used properly. Python does that. Perl does that. It can be of good use. It is mainly used to return a value and a ‘nil’ or error code. This way the old habit of encoding the error into the returned type (usually returning -1 as error code and some non-negative value in case there is some meaningful return value as in C std library calls) is replaced with something much more readable.

Multiple values on the sides of an assignment is not only to functions. To swap two values you can write:

  a,b = b,a

Object Orientation

With closures and functions being first class citizens Go is at least object oriented as JavaScript. But it is actually more than that. Go lang has interfaces and structs. But they are not really classes. They are value types. They are passed by value and wherever they are stored in memory the data there is only the pure data and no object header or anything like that. structs in Go are very much like they are in C. They can contain fields, but they can not extend each other and they can not contain methods. Object orientation is approached a bit different.

Instead of stuffing the methods into the class definition you can specify when you define the method itself which struct it applies to. Structs can also contain other structs and in case there is no name for the field you can reference it by the type of it, which becomes its name implicitly. Or you can just reference a field or method as they belonged to the top struct.

For example:

package main

import (

type A struct {
	a int

func (a *A) Printa() {
	fmt.Printf("%d\n", a.a)

type B struct {
	n string

func main() {
	b := B{}
	b.A.a = 5
	fmt.Printf("%d\n", b.a)

This is almost or a kind of inheritance.

When you specify the struct on which the method can be invoked you can specify the struct itself or a pointer to it. If the method is applied to the struct then the method will access a copy of the caller struct (this struct is passed by value). If the method is applied to a pointer to the struct then the pointer will be passed (passed by reference kind of). In the latter case the method can also modify the struct (in this sense the structs are not value types, since value types are immutable). Either can be used to fulfill the requirement of an interface. In case of the example above Printa is applied to a pointer to the struct A. Go says that A is the receiver of the method.

Go syntax is also a bit lenient about structs and pointers to it. In C you can have a struct and you can write b.a to access the field of the struct. In case of a pointer to the structure in C you have to write b->a to access the same field. In case of a pointer b.a is a syntax error. Go says that writing b->a is pointless (you can interpret this literally). Why litter the code with -> operators when the dot operator can be overloaded. Field access in case of struct and, well field access through pointers. Very logical.

Because the pointer is as good as the struct itself (to some extent) you can write:

package main

import (

type A struct {
	a int

func (a *A) Printa() {
	if a == nil {
		fmt.Println("a is nil")
	} else {
		fmt.Printf("%d\n", a.a)

func main() {
	var a *A = nil

Yes, this is the point as a true hearted Java programmer you should not freak out. We did call a method on a nil pointer! How can that happen?

Type in in the variable and not the object

This is why I was using quotes writing “object”. When Go stores a struct it is a piece of memory. It does not have an object header (though it may, since it is a matter of implementation and not the language definition, but it reasonably does not). It is the variable that holds the type of the value. If the variable type is a struct then it is known already at compile time. If this is an interface then the variable will point to the value and the same time it will also reference the actual type it is having the value for.

If the variable a is an interface and not a pointer to a struct you can not do the same: you get runtime error.

Implementing interfaces

Interfaces are very simple in Go, and the same time very complex, or at least different from what they are in Java. Interfaces declare a bunch of functions that structs should implement if they want to be compliant with the interface. The inheritance is done the same way as in case of structs. The strange thing is that you need not specify in case of a struct that it implements an interface if it does. After all it is really not the struct that implements the interface, but rather the set of functions that use the struct or a pointer to the struct as receiver. If all the functions are implemented then the struct does implement the interface. If some of them are missing then the implementation is not complete.

Why do we need the ‘implements’ keyword in Java and not in Go? Go does not need it because it is fully compiled and there is nothing like a classloader that loads separately compiled code during run-time. If a struct is supposed to implement an interface but it does not then this will be discovered during compile time without explicitly classifying that the struct does implement the interface. You can overcome this and cause a run-time error if you use reflection (that Go has) but the ‘implements’ declaration would not help that anyway.

Go is compact

Go code is compact and not forgiving. In other languages there are characters that are simply useless. We got used to them during the last 40 years since C was invented and all other languages followed the syntax, but it does not necessarily mean that it is the best way to follow. After all we all know since C that the ‘trailing else’ problem is best addressed using the { and } around the code branches in the ‘if’ statement. (May be Perl was the first mainstream C-like syntax language that requested that.) However if we must have the braces there is no point to enclose the condition between parentheses. As you could see in the code above:

	if a == nil {
		fmt.Println("a is nil")
	} else {
		fmt.Printf("%d\n", a.a)

there is no need and Go does not even allow it. You may also notice that there are no semicolons. You can use them, but you need not. Inserting them is a preprocessing step on the source code and it is very effective. Most of the time they are clutter anyway.

You can use ‘:=’ to declare a new variable and assign a value of it. On the right hand side the expression defines the type usually, so there is no need to write ‘var x typeOfX = expression‘. On the other hand if you import a package, assign a variable that you do not use afterwards: it is a bug. Since it can be detected during compile time it is a code error, compilation fails. Very smart. (Sometimes annoying when I import a package that I intend to use, and before referencing it I save the code and IntelliJ intelligently removes the import, just to help me.)

Threads and queues

Threads and queues are built into the language. They are called goroutines and channels. To start a goroutine you only have to write go functioncall() and the function will be started in a different thread. Although there are methods/functions in the standard Go library to lock “objects” the native multi-thread programming is using channels. Channel is a built-in type in Go that is a fixed size FIFO channel of any other type. You can push a value into a channel and a goroutine can pull it off. If the channel is full pushing blocks and in case the channel is empty the pull is blocking.

There are errors, no exceptions. Panic!

Go does have exception handling but this is not supposed to be used like in Java. Exception is called ‘panic’ and this is really to be used when there is some real panic in the code. In Java term it is similar to some throwable that ends with ‘…Error’. When there is some exceptional case, some error that can be handled this state is returned by the system call and the application functions are expected to follow similar pattern. For example

package main

import (

func main() {
	f, err := os.Open("filename.ext")
	if err != nil {
	defer f.Close()

the function ‘Open’ returns the file handler and nil, or nil and the error code. If you execute it on the Go Playground (click on the link above) you get the error displayed.

This is not really fitting the practice we got used to when programming in Java. It is easy to miss some error condition and write

package main

import (

func main() {
	f := os.Open("filename.ext")
	defer f.Close()

that just ignores the error. It is also cumbersome to check the possibility of error at each and every system or application call that may return error when we are interested in a longer chain of commands if any of those produced error and we do not really care which one.

No finally, defer instead

Closely coupled with the exception handling is the feature that Java implements with the try/catch/finally feature. In Java you can have code that is executed in a finally code no matter what. Go provides the keyword ‘defer’ that lets you specify a function call that will be invoked before the method returns even if there is/was a panic. This is a solution to the problem that gives you less options to abuse. You can not write arbitrary code to be executed deferred only a function call. In Java you can even have a return statement in the finally block or see a mess trying to handle the situation when the code to be executed in the finally block may also throw exception. Go is prone to that. I like that.

Other things…

that also may seem weird at first are like

  • public functions and variables are capitalized, there are no keywords like ‘public’, ‘private’
  • source code of libraries are to be imported into the source of the project (I am not sure I understood that properly)
  • lack of generics
  • code generation support built into the language in forms of comment directives (this is really a wtf)

In general Go is an interesting language. It is not a replacement for Java even on a language level. They are not supposed to serve the same type of tasks. Java is enterprise development language, Go is a system programming language. Go, just as well as Java, is continuously developing so we may see some change in that in the future.


I was working five years in telecom industry where service availability was God. We had to reach five nines, that is 99.999% percent of the time the service had to work.

Just a simple calculation 99% availability means that the service may be out of service 3 and half day per year. 99.9% means an outage period close to 9 hours. 99.99% means barely an hour and 99.999% five minutes. Is it okay if the phone is out of service at most 5 minutes a year? Certainly. How about your online banking interface? How about a pace maker?

How many 9s do we need?

The point is that you should not aim five nines (or any other availability value) if you do not know why you need that. The bottom line is, as always money. To get to a level of availability has a cost label attached. The more nines you need in your uptime number the more cost you have. The latter nines costs exponentially more. On the other side you have the income, the reputation and all other things that count as money value and are adversely affected by downtime.

Telecom invests a lot of money to get to five nine. Does it pay off? If we only count the lost revenue during the downtime (people can not call and thus do not pay for the talk) then it does not. If we count the reputation, lost customers churning to competition then maybe. If we consider emergency calls: definitely.

Telecom has its values established long time ago and the five nines came from common knowledge distilled during a long time. When you design a service for the enterprise or for the public you do not have that. You have to assess what is worth doing.

Ten years ago I was studying economy. A friend was also studying there and at one of the micmac (mixed micro and macro economy subject) exam he barely passed, though he was usually performing excellent. Being shocked by the unexpected low performance the professor asked him about it. He said: “I was not only studying but also following in practice what you taught us. Invest no more than what is needed to reach the goals. Any more invenstment is wasted.”

Assume for now that we assessed the costs and benefits and we decided we need X nines.

Is more always better?

But how about accidental reaching more uptime than needed? Operation wasted money, invested too much into availability. This is certainly not good. It may also happen that by time small investments (e.g.: continous improvement of personnel, new technologies) lead to better uptime. If it costs nothing the higher uptime is extra gift. But if there is a significant difference between the aimed availability and the actual, the investment was definitely oversized. What will prevent operation to aim higher than what financially feasible is?

It is a difficult organizational question. This is not simply put money in on one end of the tube and get availability on the other. You can invest in many different things that lead to higher availability. You can educate your support people. You can buy higher quality, larger MTBF harware. You can invest in software quality. Some of these have lower cost others cost a lot. If operation reaches in practice higher availability than needed then they wasted money. But it is hard if not impossible to tell which money.

Measure the people performance on system uptime?

Should you measure your operation people on uptime? Certainly. That is a core competency they have: provide the aimed uptime. If the operation does not get the required uptime there will be consequences. You have to assess the reasons, design changes in the system, in the policies, in management hierarchy, but certainly something is to be changed. If there is no change the availability remains low.

If there are no consequences of underperformance on people certainly there will be on the company.

Should you incentive the higher than aimed uptime? Obviously no. May be not that obviously, but certainly: no. If you incentive then they will be motivated reach the goal. If they are motivated to reach something the company does not need they will spend company money to reach that. If you incentive anyone for getting X they will get X. Simple feedback loop. Simple. Ok. But if you do not incentive higher than needed availability then …

should you punish higher availability?

If people get punished for higher availability, operation will still be motivated to spend extra money to get higher availability and ruin it to the required level by means that they can control. (Simply switch off service “illegally” or claiming maintenance.) Even if you do not punish them, they are still motivated to spend extra money just to be safe. We get to the mix of incentives and punishments controlling quality and budget and companies sometime end up with extremely complex motivational schemes.

This dilemma is nothing special. This is a very general management topic. You can not measure what you really want to achieve in your organization. In case of operation you can not measure that oepration is spending the money the most possible wise way and deliver services reliably the level business need. You can not measure it simply because it can not be defined what most wise way is.

What we can do is measure something that is somehow related to what we really need. We start to measure X. The management problem is that if you measure X people will deliver X. So what is the solution? Measurement and culture. We know what the measurements are, we aim the direct goals we are measured on but perform the strategic, tactic and everyday tasks keeping an eye on company goals. That is the culture part. Without culture we are only robots, and robots don’t do good work.


Random Ideas about Code Style

Some of the sentences of this article are ironic. Others are to be taken serious. It is up to the reader to separate them. Start with these sentences.

How long should a method be in Java?

This is a question I ask many times during interviews. There is no one best answer. There are programming styles and different styles are just different and many can be ok. I absolutely accept somebody saying that a method should be as short as possible, but I can also accept 20 to 30 lines methods. Above 30 lines I would be a bit reluctant.

“When I wrote this code only God and I understood it. Now only God does.”
Quote from unknown programmer. Last quarter of the XX. century.

The most important thing is that the code is readable. When you write the code you understand it. At least you think you understand what you wanted to code. What you actually coded may be a different story. And here comes the importance of readability as opposed to writeablity.

When you refactor a code containing some long method and you split up the method into many small methods you actually create a tree structure from a linear code. Instead of having one line after the other you create small methods and move the actual commands into those. After that the small methods are invoked from a higher level. Why does it make the code more readable?

First of all, because each method will have a name. That is what methods have and in Java we love camel cased talking names.

private void pureFactoryServiceImplementationIncomnigDtoInvoker(IncomingDto incomingDto){

But why is it any better than inlining the code and using comments?

// pure factory service implementation incoming dto invoker

Probably that is because you have to type pureFactoryServiceImplementationIncomnigDtoInvoker twice? I know you will not type it twice. You will copy paste it or use some IDE auto-complete feature and for that reason the type replacing ‘Incoming’ to ‘Incomnig’ does not really matter.

When you split up the code into small methods the names are a form of comment.

Very much like what we do in unit tests using JUnit 4.0 or later. Old versions had to start the test methods with the literal test... but that was not a good idea. It was discovered long time ago. (I just wonder when Go will get there.) These days Groovy (and especially spock) lets us use whole sentences with spaces and new lines as method names in unit tests. But those method names fortunately should not be typed twice. They are listed and invoked by Junit via reflection and thus they really are what they really are: documentation.

So then the question still is: Why is tree structure better than linear?

Probably that is how our brains work. We look at a method and see that there are two-three method calls in that. There can be a simple branch or loop structure in it, perhaps one nested to the other but not much deeper than that. It is simple and if method names are selected well (I mean really in a good, meaningful and talking way), they are easy to understand, easy to read.

The we can, using the navigational aid of the IDE go to the methods and we can concentrate on the limited context of the method we are looking at. There is a rough rule:

You should be able to understand what a method does in 15 seconds.

If you stare at the method longer and you still have no idea what the method does it means it is too complex. Some people are better apprehending the structure of the code, others are challanged in that. I am in the latter group, so when I review code I many times prefer smaller and simpler methods. I refuse the code to be merged or I refactor it myself depending on the role, the actual task I perform. Juniors I work with think that I am strict and picky. The truth is I am slow. The complexity of the code should be compatible with the weakest chain: any one of the team (including imaginable future maintainers of the next coming 20 years till the code is finally deleted from production) should understand and maintain the code easily.

Many times looking at git history I see refactoring ping-pong. For example the method

Result getFrom(SomeInput someInput){
  Result result = null;
  if( someInput != null ){
    result = someInput.get();
  return result;

is refactored to

Result getFrom(SomeInput someInput){
  final Result result;
  if( someInput == null ){
    result = null;
    result = someInput.get();
  return result;

and later the other way around.

One is shorter, while the other one is more declarative. Is the repetitive refactoring back and forth a problem? Most probably is, but not for sure. If it happens only a few times and by different people then this is not something to worry about too much. When the code gets refactored the developer feels the code more attached to him/herself. A more “it is my code” feeling, which is important. Even though a good developer is not afraid to touch and modify any code. (what could happen? test fail? so what? test? what test?) Note that not all developers are good developers. But what is a good developer after all? It is relative. There are better developers and there are not so good. If you see only good developers who are better than you, then probably you are lucky. Or not.

Implementing an annotation interface

Using annotation is every day task for a Java developer. If nothing else simple @Override annotation should ring the bell. Creating annotations is a bit more complex. Using the “home made” annotations during run-time via reflection or creating a compile time invoked annotation processor is again one level of complexity. But we rarely “implement” an annotation interface. Somebody secretly, behind the scenes certainly does for us.

When we have an annotation

public @interface AnnoWithDefMethod {
    String value() default "default value string";

then a class annotated with this annotation

@AnnoWithDefMethod("my default value")
public class AnnotatedClass {

and finally we when get the annotation during runtime executing

AnnoWithDefMethod awdm = AnnotatedClass.class.getAnnotation(AnnoWithDefMethod.class);

then what do we get into the variable awdm? It is an object. Objects are instances of classes, not interfaces. Which means that somebody under the hood of the Java runtime has “implemented” the annotation interface. We can even print out features of the object:

        for (Method m : awdm.getClass().getDeclaredMethods()) {

to get a result something like

my default value
class com.sun.proxy.$Proxy1
interface AnnoWithDefMethod

So we do not need to implement an annotation interface but we can if we wanted. But why would we want that? So far I have met one situation where that was the solution: configuring guice dependency injection.

Guice is the DI container of Google. The configuration of the binding is given as Java code in a declarative manner as it is described on the documentation page. You can bind a type to an implementation simply declaring


so that all TransactionLog instance injected will be of DatabaseTransactionLog. If you want to have different imlpementation injected to different fields in your code you should some way signal it to Guice, for example creating an annotation, putting the annotation on the field, or on the constructor argument and declare the


This requires PayPal to be an annotation interface and you are required to write an new annotation interface acompaniing each CreditCardProcessor implementation or even more so that you can signal and separate the implementation type in the binding configuration. This may be an overkill, just having too many annotation classes.

Instead of that you can also use names. You can annotate the injection target with the annotation @Named("CheckoutPorcessing") and configure the binding


This is a tehnique that is well known and widely used in DI containers. You specify the type (interface), you create the implementations and finally you define the binding type using names. There is no problem with this, except that it is hard to notice when you type porcessing instead of processing. Such a mistake remains hidden until the binding (run-time) fails. You can not simply use a final static String to hold the actual value because it can not be used as the annotation parameter. You could use such a constant field in the binding definition but it is still duplication.

The idea is to use something else instead of String. Something that is checked by the compiler. The obvious choice is to use a class. To implement that the code can be created learning from the code of NamedImpl, which is a class implementing the annotation interface. The code is something like this (Note: Klass is the annotation interface not listed here.):

class KlassImpl implements Klass {
    Class<? extends Annotation> annotationType() {
        return Klass.class
    static Klass klass(Class value){
        return new KlassImpl(value: value)
    public boolean equals(Object o) {
        if(!(o instanceof Klass)) {
            return false;
        Klass other = (Klass)o;
        return this.value.equals(other.value());
    public int hashCode() {
        return 127 * "value".hashCode() ^ value.hashCode();
     Class value
    Class value() {
        return value

The actual binding will look something like

  public RealBillingService(@Klass(CheckoutProcessing.class) CreditCardProcessor processor,
      TransactionLog transactionLog) {

In this case any typo is likely to be discovered by the compiler. What happens actually behind the scenes, and why were we requested to implement the annotation interface?

When the binding is configured we provide an object. Calling Klass.klass(CheckoutProcessing.class) will create an instance of KlassImpl and when Guice tries to decide if the actual binding configuration is valid to bind CheckoutCreditCardProcessor to the CreditCardProcessor argument in the constructor of RealBillingService it simply calls the method equals() on the annotation object. If the instance created by the Java runtime (remember that Java runtime creates an instance that had a name like class com.sun.proxy.$Proxy1) and the instance we provided are equal then the binding configuration is used otherwise some other binding has to match.

There is another catch. It is not enough to implement equals(). You may (and if you are a Java programmer (and you are why else you read this article (you are certainly not a lisp programmer)) you also should) remember that if you override equals() you have to override also hashCode(). And actually you should provide an implementation that does the same calculation as the class created by the Java runtime. The reason for this is that the comparison may not directly be performed by the application. It may (and it does) happen that Guice is looking up the annotation objects from a Map. In that case the hash code is used to identify the bucket in which the comparing object has to be and the method equals() is used afterwards to check the identity. If the method hashCode() returns different number in case of the Java runtime created and out objects they will not even match up. equals() would return true, but it is never invoked for them because the object is not found in the map.

The actual algorithm for the method hashCode is described on the documentation of the interface java.lang.annotation. I have seen this documentation before but understood the reason why the algorithm is defined when I first used Guice and implemented a similar annotation interface implementing class.

The final thing is that the class also has to implement annotationType(). Why? If I ever figure that out I will write about that.

Java compile in Java

In a previous post I wrote about how to generate a proxy during run-time and we got as far as having Java source code generated. However to use the class it has to be compiled and the generated byte code to be loaded into memory. That is “compile” time. Luckily since Java 1.6 we have access the Java compiler during run time and we can, thus mix up compile time into run time. Though that may lead a plethora of awful things generally resulting unmaintainable self modifying code in this very special case it may be useful: we can compile our run-time generated proxy.

Java compiler API

The Java compiler reads source files and generates class files. (Assembling them to JAR, WAR, EAR and other packages is the responsibility of a different tool.) The source files and class files do not necessarily need to be real operating system files residing in a magnetic disk, SSD or memory drive. After all Java is usually good about abstraction when it comes to the run-time API and this is the case now. These files are some “abstract” files you have to provide access to via an API that can be disk files but the same time they can be almost anything else. It would generally be a waste of resources to save the source code to disk just to let the compiler running in the same process to read it back and to do the same with the class files when they are ready.

The Java compiler as an API available in the run-time requires that you provide some simple API (or SPI of you like the term) to access the source code and also to send the generated byte code. In case we have the code in memory we can have the following code (from this file):

public Class<?> compile(String sourceCode, String canonicalClassName)
			throws Exception {
		JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
		List<JavaSourceFromString> sources = new LinkedList<>();
		String className = calculateSimpleClassName(canonicalClassName);
		sources.add(new JavaSourceFromString(className, sourceCode));

		StringWriter sw = new StringWriter();
		MemoryJavaFileManager fm = new MemoryJavaFileManager(
				compiler.getStandardFileManager(null, null, null));
		JavaCompiler.CompilationTask task = compiler.getTask(sw, fm, null,
				null, null, sources);

		Boolean compilationWasSuccessful =;
		if (compilationWasSuccessful) {
			ByteClassLoader byteClassLoader = new ByteClassLoader(new URL[0],
					classLoader, classesByteArraysMap(fm));

			Class<?> klass = byteClassLoader.loadClass(canonicalClassName);
			return klass;
		} else {
			compilerErrorOutput = sw.toString();
			return null;

This code is part of the opensource project Java Source Code Compiler (jscc) and it is in the file

The compiler instance is available through the ToolProvider and to create a compilation task we have to invoke getTask(). The code write the errors into a string via a string writer. The file manager (fm) is implemented in the same package and it simply stored the files as byte arrays in a map, where the keys are the “file names”. This is where the class loader will get the bytes later when the class(es) are loaded. The code does not provide any diagnistic listener (see the documentation of the java compiler in the RT), compiler options or classes to be processed by annotation processors. These are all nulls. The last argument is the list of source codes to compile. We compile only one single class in this tool, but since the compiler API is general and expects an iterable source we provide a list. Since there is another level of abstraction this list contains JavaSourceFromStrings.

To start the compilation the created task has to be “call”ed and if the compilation was successful the class is loaded from the generated byte array or arrays. Note that in case there is a nested or inner class inside the top level class we compile then the compiler will create several classes. This is the reason we have to maintain a whole map for the classes and not a single byte array even though we compile only one source class. If the compilation was not successful then the error output is stored in a field and can be queried.

The use of the class is very simple and you can find samples in the unit tests:

	private String loadJavaSource(String name) throws IOException {
		InputStream is = this.getClass().getResourceAsStream(name);
		byte[] buf = new byte[3000];
		int len =;
		return new String(buf, 0, len, "utf-8");
	public void given_PerfectSourceCodeWithSubClasses_when_CallingCompiler_then_ProperClassIsReturned()
			throws Exception {
		final String source = loadJavaSource("");
		Compiler compiler = new Compiler();
		Class<?> newClass = compiler.compile(source, "com.javax0.jscc.Test3");
		Object object = newClass.newInstance();
		Method f = newClass.getMethod("method");
		int i = (int) f.invoke(object, null);
		Assert.assertEquals(1, i);

Note that the classes you create this way are only available to your code during run-time. You can create immutable versions of your objects for example. If you want to have classes that are available during compile time you should use annotation processor like scriapt.

Optimize the client for the server’s sake

The Story

Once upon a time there was an application that was running on some server and the client functionality was implemented in HTML/CSS and JavaScript. The application was serving trillion (not literally) of users all hanging on the end of some phone lines talking to customers who were usually impatient and needed fast resolution to their problems. Typical call center application where speed is key.

Users were dissatisfied by the speed of the service.

No surprise. They usually are.

The application was delivering static resources for the client and JSON encoded data via REST interface. The underlying data structure was using relational database managed from Java using JOOQ. All good technologies were applied to make the service as fast as possible, still the performance was not accepted by the users. Users claimed that the system was slow, unusable, annoying, dead as fish frozen in a lake (yes, actually that was one of the expression we got in the ticketing system). We were aware that “unusable” was some exaggeration: after all there were thousands of queries running through the system daily. But “slow” and “annoying” are not measurable terms not to mention “dead fish”. First thing first: we had to measure!


To address the issue we injected some JavaScript that was measuring the actual performance and it was also reporting the client measured response times to a separate server via some very simple and very fast REST service. We paid attention not to put extra load on the original servers not to make the situation even worse. The result showed that some of the results arrived to the client within 1sec, most of them in 2sec but there was actually a significant tail of the Poisson distribution with some responses as long as 15sec. We also had the measurement on the server side and the results were similar. On the server side we measured approximately 10% more transactions that were lost for the measurement on the client and the Poisson tail on the server contained responses up to 90sec. We did not pay attention to these differences until a bit later.

Meeting the requirements may not be enough.

The actual measurements showed that the response times were in-line with the requirement so we created a report showing all good and shiny hoping that this will settle the story. We presented the results to the management and we almost got fired. They were not interested in measurements and response time milisecs. All they cared was user satisfaction. (Btw: At this point I understood why the name “user acceptance test” is not “customer acceptance test”.) We were blatantly directed not to mess with some useless measurements but go and stand by some of the users and experience direct eyes how slow the system was. It was a kind of shock. Standing by a user and “feeling” the system speed was not considered to be an engineering approach. But having nothing else in hand we did. And it worked!


We could see that some of the users were impatient. They clicked on a button and after a second when nothing happened they clicked on it again. It meant that the browser was sending a request to the server but before the response arrived the communication was cancelled on the client side and the request was sent again. Processing started from zero by the second button press but the wait time for the user accumulated.


To help the patience of the users we introduced some hour glass effect on the JavaScript level that signalled to the user that they have pressed a button and that the button press was handled by the application. Also the hour glass was moving “entertaining” the users and we also hid the button (and the whole filled in form) behind a semitransparent DIV layer actually preventing double submit. We did not have high expectations. Afterall it did not make the system faster. The users loved the new feature. First of all they felt that we care. They had been complaining and now we were doing something for them. Interestingly they also felt the system faster because of the rotating hour glass on the screen. End of story? Almost.


After a week or so we executed the measurement again. It was not a big effort since all the tooling was already there. What we experienced was that the 10% difference between the number of transactions measured on the client and on the server practically vanished. Probably these were the transactions when the user pressed the button second time. It was a full processing run on the server side, but was not reported by the client since the transaction as well as the measurement on the client side was cancelled. These got eliminated with the improved user interface that also decreased the load on the server by 10%. Which finally resulted slightly faster response times.

Usual disclaimers apply.

The Little Architect

Uncle Bob published recently an article titled “A Little Architecture“. The article is a conversation between a young developer and a senior (Uncle Bob himself presumably) about being software architect. The article starts with these sentences:

  • I want to become a Software Architect.
  • That’s a fine goal for a young software developer.
  • I want to lead a team and make all the important decisions about databases and frameworks and web-servers and all that stuff.

The next part asks the young developer to list what the important things are. However that is not the only thing that may be interesting in this last sentence. There is another thing, perhaps less technical, that hit me. The young developer says: “make … decisions”.

That may be a mistake. You can interpret it differently what “making decision” means, but let me here tell you my thoughts about that. Some thoughts that were triggered by those two words. First of all here is a story, when I was making some decisions.


Not really many years ago when I was much younger I was acting as system architect and I made a decision on how to store some content. Mainly text and not too large pictures. The obvious choice could be to use database and implement the CRUD operations. A database is always a good solution just as a scarf is always a good gift for Christmas. You love getting a new scarf for every Christmas, don’t you?

On second thought, however the real power of database is when the content is to be searched, indexed and when transactions are executed. They are not really requirements for a media store. On the other hand versioning and user level access control was. I have previously implemented something like that in the past and that time we used SVN for content storage. And that worked fine. So I decided that we should go and use SVN this time also. The project was a success story. A little bit more story than success though. Halfway thriving towards the solution the back-end storage was replaced by a DB layer.

Why didn’t SVN work?

The reason is simple. The developers did not like and understood the decision. They were not familiar with the technology. They used SVN for source code storage but they never used the programming API of it. Instead of using the Java client they forked external svn processes and they were checking out files individually. Displaying a directory containing 20 files was starting 20 processes one after the other. On that system that was approx. 20 seconds.

Okay. It could have been mended in different ways: there was not enough control on the use of the technology and there was a lack of professional code review as well as performance testing due time and so on. The root of the problems though was that I made the decision. I was acting like an omnipotent god, who knows it much better. I was not and I did not.

So what?

I could do it better discussing the solution more with the developers until we all agree on what the solution could have been. I could understand that the DB solution was better or they could understand how the SVN could have been used that way. We could make a decision together. I could make it so that they could make the decision.

A real architect never makes a decision.

A real architect works with the team developing the software asking the right questions making sure that the team make the right decision.

Good architects approve the decision of the team and bear the responsibility. Bad architects make the decisions and blame the team.

Part of it is psychology. If the team makes the decision they are more likely to love the ideas than if they were force fed. They may come up with some ideas that you missed. Good architects recognize that and improve him/herself. Really good architect can even admit at this stage being wrong. On the contrary of what young developers think this increases the esteem. (Unless the architect is wrong more times than not, in which case he/she is not really a good architect.)

Asking the questions also reveal if the team is not prepared for some of the technologies. If they have to learn something new. It may turn out that education is in place or some more familiar technology is to be used. This may also be a smell that you wanted to use some niche technology that may require expensive developers in the coming years to maintain the product. You better don’t!

This does not mean that you should open the floodgates. You still should approve the decision and you should not approve a decision you can not live with. If the team makes a decision on some technology that you feel not good enough it means you have not asked the right questions. You should ask more. The responsibility is your.

I recommend that if you want to be a good architect let the team make the decision and help them forging a good one. Approve it and never blame them. That way they will not leave you in cold water. If you even bring free pizza now and then they may even love you.

Creating proxy object using djcproxy

During the last weeks I have shown how to create a proxy object using Java reflection API and cglib. In this article I will show you how this can be done using djcproxy.

Oh, not again, another proxy implementation!

What is the point to write about this in addition to the selfish fact that I created this proxy? The point is that this is a proxy that is written in Java, it creates Java code that can be examined. It also compiles and loads the created Java classes on the fly so it is also usable but the main advantage is that you can easily get a good insight how a dynamic proxy works. At least a bit easier than digging around the code of cglib, which is creating byte codes directly.

How to use it

You can get the source from github or you can just add the dependency to you project maven pom.


After that you can use the following code:

class A {
  public int method() {
  return 1;
class Interceptor implements MethodInterceptor {

  public Object intercept(Object obj, Method method, Object[] args,
    MethodProxy mproxy) throws Exception {
      if (method.getName().equals("toString")) {
        return "interceptedToString";
      return 0;


    A a = new A();
    ProxyFactory<A> factory = new ProxyFactory<>();
    A s = factory.create(a, new Interceptor());

This code can be found in the tests of the project in GitHub. This is an edited abbreviated version prone to editing errors.

The class ‘A’ is the original class and when we want to create a new proxy object we create a proxy to an already existing object. This is different from reflection or cglib. In case of cglib you create a proxy object and it “contains” the original object. It is not really a containment in OO terms, because the proxy class extends the original class. However because of this extending the proxy object is also an instance of the original class. Cglib does not really care which class instance (object) you want to intercept. You can inject a reference to any object instance to your interceptor if you want. Djcproxy uses a different approach and it does that for you and in your interceptor you will get this object passed as argument. This is why you have to instantiate the object in line 20.

The Interceptor implements the interface MethodInterceptor also provided in the library. It has only one method: intercept, which is invoked when the proxy object method is called. The arguments are

  • obj – the original object
  • method – the method that was invoked in the proxy object
  • args – the arguments that were passed to the method call on the proxy object. Note that primitive arguments will be boxed.
  • mproxy – the method proxy that can be used to call the method on the original object or on just any other object of the same type

This is all about how to use this library. The next thing is to have a look at what is generated so that you can get a better understanding how a proxy works. Insight never hurts, even if you use a different proxy. Many times debugging or just generating better code is easier when you know the principles of a library you use.

While cglib gives you a static factory method to create new objects djcproxy requires that you create a proxy factory. This is on line numbered above 21. If you want to use it the same way as you used cglib you can declare a static ProxyFactory field in the class where you want to use the factory from. On the other hand it is possible to have different factories in different parts of the code. Although the advantage of it is rare, still I believe it is a cleaner approach than providing static factory method.

How does the proxy work?

The extra thing in this package is that it lets you get access to the generated source. You can insert the lines

    String generatedSource = factory.getGeneratedSource();

to print out the generated proxy class which is after some formatting is this:

package com.javax0.djcproxy;

class PROXY$CLASS$A extends com.javax0.djcproxy.ProxyFactoryTest.A implements com.javax0.djcproxy.ProxySetter {
    com.javax0.djcproxy.ProxyFactoryTest.A PROXY$OBJECT = null;
    com.javax0.djcproxy.MethodInterceptor PROXY$INTERCEPTOR = null;

    public void setPROXY$OBJECT(java.lang.Object PROXY$OBJECT) {
        this.PROXY$OBJECT = (com.javax0.djcproxy.ProxyFactoryTest.A) PROXY$OBJECT;


    public void setPROXY$INTERCEPTOR(com.javax0.djcproxy.MethodInterceptor PROXY$INTERCEPTOR) {




    private com.javax0.djcproxy.MethodProxy method_MethodProxyInstance = null;

    public int method() {

        try {
            if (null == method_MethodProxyInstance) {
                method_MethodProxyInstance = new com.javax0.djcproxy.MethodProxy() {
                    public java.lang.Object invoke(java.lang.Object obj, java.lang.Object[] args) throws Throwable {
                        return ((com.javax0.djcproxy.ProxyFactoryTest.A) obj).method();

            return (int) PROXY$INTERCEPTOR.intercept(
                    PROXY$OBJECT, PROXY$OBJECT.getClass().getMethod("method", new Class[]{}),
                    new Object[]{}, method_MethodProxyInstance);
        } catch (Throwable e) {
            throw new RuntimeException(e);


... other overridden methods deleted ...


Note that the class A is a static nested class of ProxyFactoryTest for this generated code.

The interesting code is the overriding of the method method(). (Sorry for the name. I have no fantasy to have a better name for a method that does nothing.) Let’s skip the part where the method checks if there is already a MethodProxy instance and if is missing it creates one. The method method() actually calls the interceptor object that we defined, passing the proxied object, the reflective method object, the arguments and also the method proxy.

What is the method proxy

The name may be confusing first because we already have an “object” proxy. There is a separate method proxy for each method of the original class. These can be used to invoke the original method without reflective call. This speeds up the usage of the proxies. You can also find this call and a similar mechanism in cglib.


The implementation has some flows, for example the late method proxy instantiations have no advantage really but the same time may hurt in case of multi-thread execution of the proxies. It could also be possible to create a proxy object that not only extends a class but also implement arbitrary interfaces (perhaps some that is not even implemented by the extended class). The implementation is used in some other hobby opensource project also available on github about which I may write in the future. They are more demonstrative, educational and proof of concept projects than production code. If you have anything to say on the implementation, the ideas, or just any comments, please reward me with your comments.

Creating a proxy object using cglib

In the previous post I was talking about the standard Java based proxy objects. These can be used when you want to have a method invocation handler on an object that implements an interface. The Java reflection proxy creation demands that you have an object that implements the interface. The object we want to proxy is out of our hand, it does not implement the interface that we want to invoke from our handler and still we want to have a proxy.

When do we need proxy to objects w/o interface?

This is a very common case. If ever we have a JPA implementation e.g. Hibernate that implements lazy loading of the records. For example the audit log records are stored in a table and each record, except the first one has a reference to the previous item. Something like

class LinkedAuditLogRecord {
  LinkedAuditLogRecord previous;
  AuditLogRecord actualRecord;

Loading a record via JPA will return an object LinkedAuditLogRecord which contains the previous record as an object and so on until the first one that probably has null in the field named previos. (This is not an actual code.) Any JPA implementation grabbing and loading the whole table from the start to the record of our interest would be an extremely poor implementation. Instead the persistence layer loads the actual record only and creates a proxy object extending LinkedAuditLogRecord and that is what the field previous is going to be. The actual fields are usually private fields and if ever our code tries to access the previous record the proxy object will load it that time. This is lazy loading in short.

But how do the JPA implementations create proxies to objects of classes that do not implement interfaces? Java reflection proxy implementation can not do that and thus JPA implementation uses something different. What they usually use is cglib.

What is cglib

Cglib is an open source library that capable creating and loading class files in memory during Java run time. To do that it uses Java byte-code generation library ‘asm’, which is a very low level byte code creation tool. I will not dig that deep in this article.

How to use cglib

To create a proxy object using cglib is almost as simple as using the JDK reflection proxy API. I created the same code as the last week article, this time using cglib:

package proxy;

import net.sf.cglib.proxy.Enhancer;
import net.sf.cglib.proxy.MethodInterceptor;
import net.sf.cglib.proxy.MethodProxy;

import java.lang.reflect.Method;

public class CglibProxyDemo {

    static class Original {
        public void originalMethod(String s) {

    static class Handler implements MethodInterceptor {
        private final Original original;

        public Handler(Original original) {
            this.original = original;

        public Object intercept(Object o, Method method, Object[] args, MethodProxy methodProxy) throws Throwable {
            method.invoke(original, args);
            return null;

    public static void main(String[] args){
        Original original = new Original();
        MethodInterceptor handler = new Handler(original);
        Original f = (Original) Enhancer.create(Original.class,handler);

The difference is that name of the classes are a bit different and we do not have an interface.

It is also important that the proxy class extends the original class and thus when the proxy object is created it invokes the constructor of the original class. In case this is resource hungry we may have some issue with that. However this is something that we can not circumvent. If we want to have a proxy object to an already existing class then we should have either an interface or we have to extend the original class, otherwise we just could not use the proxy object in place of the original one.

Java Dynamic Proxy

Proxy is a design pattern. We create and use proxy objects when we want to add or modify some functionality of an already existing class. The proxy object is used instead of the original one. Usually the proxy objects have the same methods as the original one and in Java proxy classes usually extend the original class. The proxy has a handle to the original object and can call the method on that.

This way proxy classes can implement many things in a convenient way:

  • logging when a method starts and stops
  • perform extra checks on arguments
  • mocking the behavior of the original class
  • implement lazy access to costly resources

without modifying the original code of the class. (The above list is not extensive, only examples.)

In practical applications the proxy class does not directly implement the functionality. Following the single responsibility principle the proxy class does only proxying and the actual behavior modification is implemented in handlers. When the proxy object is invoked instead of the original object the proxy decides if it has to invoke the original method or some handler. The handler may do its task and may also call the original method.

Even though the proxy pattern does not only apply into situation when the proxy object and proxy class is created during run-time, this is an especially interesting topic in Java. In this article I will focus on these proxies.

This is an advanced topic because it requires the use of the reflection class, or byte code manipulation or compiling Java code generated dynamically. Or all of these. To have a new class not available as a byte code yet during run-time will need the generation of the byte code, and a class loader that loads the byte code. To create the byte code you can use cglib or bytebuddy or the built in Java compiler.

When we think about the proxy classes and the handlers they invoke we can understand why the separation of responsibilities in this case is important. The proxy class is generated during run-time, but the handler invoked by the proxy class can be coded in the normal source code and compiled along the code of the whole program (compile time).

The easiest way to do this is to use the java.lang.reflect.Proxy class, which is part of the JDK. That class can create a proxy class or directly an instance of it. The use of the Java built-in proxy is easy. All you need to do is implement a java.lang.InvocationHandler so that the proxy object can invoke that. InvocationHandler interface is extremely simple. It contains only one method: invoke(). When invoke() is invoked the arguments contain the original object, which is proxied, the method that was invoked (as a reflection Method object) and the object array of the original arguments. A sample code demonstrate the use:

package proxy;

import java.lang.reflect.InvocationHandler;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;

public class JdkProxyDemo {

    interface If {
        void originalMethod(String s);

    static class Original implements If {
        public void originalMethod(String s) {

    static class Handler implements InvocationHandler {
        private final If original;

        public Handler(If original) {
            this.original = original;

        public Object invoke(Object proxy, Method method, Object[] args)
                throws IllegalAccessException, IllegalArgumentException,
                InvocationTargetException {
            method.invoke(original, args);
            return null;

    public static void main(String[] args){
        Original original = new Original();
        Handler handler = new Handler(original);
        If f = (If) Proxy.newProxyInstance(If.class.getClassLoader(),
                new Class[] { If.class },


If the handler wants to invoke the original method on the original object it has to have access it. This is not provided by the Java proxy implementation. You have to pass this argument to the handler instance yourself in your code. (Note that there is an object usually named proxy passed as an argument to the invocation handler. This is the proxy object that the Java reflection dynamically generate and not the object we want to proxy.) This way you are absolutely free to use a separate handler object for each original class or use some shared object that happens to know some way which original object to invoke if there is any method to invoke at all.

As a special case you can create an invocation handler and a proxy of an interface that does not have any original object. Even more it is not needed to have any class to implement the interface in the source code. The dynamically created proxy class will implement the interface.

What should you do if the class you want to proxy does not implement an interface? In that case you have to use some other proxy implementation. We will look at about that next week.


Get every new post delivered to your Inbox.

Join 1,116 other followers