Start a Java app without installing it

This article describes how you can use JShell to download and execute a Java application. It will eliminate the need for the installation of the application.

Do not install, just run!

The first obstacle that you have to overcome to make people use your app is the installation. You want people to use the app, try it out. To do that, they first have to install it. At least they have to download it and type in a command line to start it up. If your program is excellent and valuable for the users, they will use it after this step. They will see that it works and can do what it is supposed to do. Before the installation, however, this is a bit different. Users will install the program if they really, really need it. Installation is undoubtedly an entry threshold.

Jamal as Example

My example is Jamal that you can download at https://github.com/verhas/jamal. I wrote the first version twenty years ago, and I named it Jamal. The name stands for Just Another Macro language, which I intended to be sarcastic. I thought it was sarcastic because it was so much more different than any other text macro applications. It seems the name was not interpreted as sarcastic but rather literally. Users saw it really as “just another” and it did not become widespread. A few people bothered to install it.
Now I have a Java version, which is even more versatile and powerful than the previous version. However, if you wanted to use it, you had to install it and start it up with a relatively complex java -cp ... command line. My first attempt to overcome this was to create a Maven plugin. A maven plugin executes without installing it. If you have installed Maven, all you need to run a plugin is a Maven command line. A kind of complex one, though. Or it would help if you had a pom.xml.

I also created the Maven plugin version because I used Jamal to maintain the pom.xml files with Jamal preprocessed. That way, as you can see in an earlier article, I can write

{dependencyManagement|{dependencies|
    {@for MODULE in (testsupport,api,core,tools,engine,extensions)={dependency :com.javax0.jamal:jamal-MODULE:{VERSION}}}
    {@for MODULE in (api,engine,params)={dependency :org.junit.jupiter:junit-jupiter-MODULE:5.2.0:test}}
    }}

instead of a much longer and redundant XML fragment. This source, pom.xml.jam is then converted to pom.xml, and Maven runs fine.

The solution can still be better because many people do not use Maven. Jamal is not a Maven dependent tool.

I also use a different project to *.md.jam files to edit my next book. A book, as a project, does not require Maven. This book is not a Java book. I happen to have Maven on my machine, but the project does not need that.

There is no reason to require installed Maven as a precondition.

There is one precondition that I have to require, and that is an installed Java JDK. I cannot skip that because Jamal is written in Java.

You can also miss this precondition if you have docker, but then you need docker.

However, if you have the JDK installed (at least Java 9), you can quickly start a JShell. JShell executes the Java code from some input file that you specify on the command line. If you want to start Jamal, then the command is:

jshell https://git.io/jamal

The command file is on GitHub, and JShell can download it from there. This command file downloads the JAR files needed to run Jamal, and then it starts Jamal in a separate process.

The actual script splits into separate parts, and the jamal.jsh content is

/open scripts/version.jsh
/open scripts/jarfetcher.jsh
/open scripts/executor.jsh
/open scripts/optionloader.jsh
/open scripts/defaultoptions.jsh

download("01engine/jamal-engine")
download("02api/jamal-api")
download("03tools/jamal-tools")
download("04core/jamal-core")
download("08cmd/jamal-cmd")

loadOptions()

for(String jarUrl:extraJars){
    LOCAL_CACHE.mkdirs();
    downloadUrl(jarUrl,LOCAL_CACHE);
    }

execute()

/exit

As you can see, the JShell commands and the Java snippets are mixed. The script loads other scripts using the JShell /open command. These snippets define the method download(), loadOption() and downloadUrl().

The script version.jsh defines the global variable VERSION:

String VERSION="1.2.0";

Downloading and Caching the Program

The next script, jarfetcher.jsh is a bit more complicated. As of now, it is 100 lines. If you want to look at the whole code, it is available on GitHub. You can calculate the URL from the argument of the /open statement and from the URL above used to start Jamal.

The core functionality implemented in this script is the one that downloads the JAR files. This is the following:

void downloadUrl(String urlString,File cacheRootDirectory) throws IOException {
    final URL url = new URL(urlString);
    File jar = new File(cacheRootDirectory.getAbsolutePath() + "/" + getFile(url));
    classPath.add(jar.getAbsolutePath());
    if (jar.exists()) {
        return;
    }
    System.out.println("downloading " + url);
    System.out.println("saving to file " + jar.getAbsolutePath());
    HttpURLConnection con = (HttpURLConnection) url.openConnection();
    con.setRequestMethod("GET");
    con.setConnectTimeout(CONNECT_TIMEOUT);
    con.setReadTimeout(READ_TIMEOUT);
    con.setInstanceFollowRedirects(true);
    final int status = con.getResponseCode();
    if (status != 200) {
        throw new IOException("GET url '" + url.toString() + "' returned " + status);
    }
    InputStream is = con.getInputStream();
    try (OutputStream outStream = new FileOutputStream(jar)) {
        byte[] buffer = new byte[8 * 1024];
        int bytesRead;
        while ((bytesRead = is.read(buffer)) != -1) {
            outStream.write(buffer, 0, bytesRead);
        }
    }
}

The method caches the downloaded files into a directory. Environment variables can configure the directory. The default location is ~/.jamal/cache/.jar/.

If the file exists, then it does not download it again. The code assumes that the files we are using are released JAR files that do not ever change. If this file was never downloaded before, it downloads the file and stores it in the cache directory.

Executing the macro processor

When all the files are there, then the script executed Jamal. It is coded in the executor.jsh. The method execute.jsh contains the following method:

void execute() throws IOException, InterruptedException {
    ProcessBuilder builder = new ProcessBuilder();
    String sep = System.getProperty("path.separator");
    String cp = String.join(sep,classPath);
    List<String> arguments = new ArrayList<>();
    arguments.addAll(List.of("java", "-cp", cp, "javax0.jamal.cmd.JamalMain"));
    arguments.addAll(commandLineOptions.entrySet().stream().map(e -> "" + e.getKey() + "=" + e.getValue()).collect( Collectors.toSet()));
    System.out.println("EXECUTING");
    for( String a : arguments){
        System.out.println(a);
    }
    builder.command(arguments.toArray(String[]::new))
        .directory(new File("."));
    Process process = builder.start();
    process.getInputStream().transferTo(System.out);
    int exitCode = process.waitFor();
}

As you can see, this script is using the standard Java ProcessBuilder to create a new process and then executes Jamal in it.

Extra details

The actual operation is a bit more complex. Many options can control Jamal. In the Maven plugin version, these options are in the pom.xml file. The command-line version uses, eventually, command-line options. JShell does not handle command-line options that would pass to the executing JShell engine. There are some tricks, like using system properties or environment variables. I find those cumbersome and tricky to use. You usually execute Jamal using the same configuration in a single project. The best way is to have the options in a file. The Jamal startup JShell script reads the file ./jamal.options. The file has a simple key value format. It can contain values for the command line options as keys and extra jar and cp keys. Jamal is extensible. Extra classes on the classpath may contain macro implementations in Java, and they are used from the text files. Every jar defines a URL from where a JAR file downloads. The cp key defines local files to be added to the classpath.

These files are project-specific; therefore, these will cache in the current working directory. The cache directory will be ./.jamal/cache/.jar/.

If the jamal.options file does not exist, then the script’s first execution will create. The auto-created file will contain the default values and also some documentation.

Summary

A Java application can start without downloading it first using JShell. The startup JShell script can be located on the net and downloaded on the fly. It can also fetch other scripts, and you can mix the Java snippets with JShell commands. I recommend having some caching strategy for the JAR files to avoid repetitive downloads. The JShell script can start your application in a new process. You cannot pass command line parameters to a JShell script, but you can use an options file or something else.

Happy scripting.

All you wanted to know about Throwable

This article is a tutorial about exceptions. But not the usual one. There are many of those that tell you what exceptions are for, how you can throw one, catch one, the difference between checked and runtime exceptions, and so on. There is no need for another. It would also be boring for you. If not, then go and read one of those and come back when you have learned what they teach. This article starts where those tutorials end. We dive a bit deeper into Java exceptions, what you can do with them, what you should do with them, and what features they have that you may not have heard about. If setStackTrace(), getCause() and getSuppressed() are the methods you eat for breakfast then you can skip this article. But if not, and you want to know a bit about these, then go on. This article is long. It took a long time to write, and it will take a long time to read. It is needed.

Introduction

In this article, we will talk about exceptions and what we can and should do with Java exceptions. The simplest case is to throw one and then catch it, but there are more complex situations, like setting a cause or suppressed exceptions. We will look at these possibilities, and a bit more. To discover the possibilities we will develop a simple application and step-by-step we will create four versions developing the application further and further using more and more exception handling possibilities. The source code is available in the repository:

https://github.com/verhas/BLOG/tree/master/exception_no_stack

The different versions are in different Java packages. Some classes that did not change in the different versions are one package higher, and they are not versioned.

  • The first version v1 simply throws en exception, and it is not handled by the application. The test code expects the test setup to throw the exception. This version is the baseline to demonstrate why we need more complex solutions. We will experience that there is not enough information in the exception to see where the actual issue has happened.
  • The second version v2 catches the exception at higher levels and throws a new exception with more information about the exceptional case, and the new exception has the original one embedded as cause. This approach gives enough information to track the location of the issue, but it can even be enhanced so that it is easier to read and recognize the actual problem.

  • The third version v3 will demonstrate how we can modify the creation of the new exceptions so that the stack trace of the higher level exceptions will not point to the location where the original exception was caught, but rather where the original exception was thrown.

  • Finally, the fourth version v4 will demonstrate how we can suppress expressions when it is possible to go on with the processing in case of en exceptional case even if the operation cannot be finished successfully. This “going further” makes it possible to have an exception at the end that collects the information about all discovered exceptional cases and not only the first occurrence.

If you look at the code, you will also find there the original text of this article, and the setup that helps to maintain the code snippets copying them into the article from the source keeping all of them up-to-date. The tool that does it for us is Java::Geci.

Sample Application

We use exceptions to handle something that is outside of the normal flow of the program. When an exception is thrown the normal flow of the program is interrupted, and the execution stops dumping the exception to some output. These exceptions can also be caught using the try and catch command pair built into the language.

    try {
        ... some code ...
        ... even calling methods
                      several level deep     ...
        ...    where exception may be thrown ...
      }catch(SomeException e){
        ... code having access to the exception object 'e'
            and doing someting with it (handling) ....
      }

The exception itself is an object in Java and can contain a lot of information. When we catch an exception in our code, we have access to the exception object, and the code can act upon the exceptional situation also having access to the parameters that the exception object is carrying. It is possible to implement our own exceptions extending the Java
java.lang.Throwable class or some of the classes that directly, or transitively extend Throwable. (Usually, we extend the class Exception.) Our own implementation can hold many parameters that describe the nature of the exceptional situation. We use object fields for the purpose.

Although there is no limit for the data an exception can carry, it usually does not contain more than a message and the stack trace. There is room – as defined in the class Throwable – for other parameters, like the exception that was causing the current one (getCause()), or an array of suppressed exceptions (getSuppressed()). They are rarely used, presumably because developers are not aware of these features and because most cases are simple and do not need these possibilities. We will have a look at these possibilities in this article so that you will not belong to the group of ignorant developers who do not use these methods only because they are not aware of them.

We have a sample application. It is a bit more than just throwing, catching, and handling an exception in the catch branch that lets the code to continue. That is simple and is explained in the tutorial you have read when learning to program in Java the first time.

Our sample application will be a bit more complex. We will list the files in a directory, read the lines, and count the number of wtf strings. This way we automate the code review process quality measurement (joking). It is said that the code quality is reverse proportional to the number of the WTFs during the code review.

The solution contains

  • a FileLister that can list the files,
  • a FileReader that can read a file,
  • a LineWtfCounter that will count the wtfs in a single line,
  • a FileWtfCounter that will use the previous class to count all the wtfs in the whole file listing the lines, and finally,
  • a ProjectWtfCounter that counts the wtfs in the whole project using the file level counter, listing all the files.

Version 1, throw and catch

The application functionality is fairly simple and because we focus on the exception handling the implementation is also trivial. For example, the file listing class is as simple as the following:

package javax0.blog.demo.throwable;

import java.util.List;

public class FileLister {

    public FileLister() {
    }

    public List<String> list() {
        return List.of("a.txt", "b.txt", "c.txt");
    }
}

We have three files in the file system, a.txt, b.txt, and c.txt. This is a mock, of course, but in this case, we do not need anything more complex to demonstrate the exception handling. Similarly, the FileReader is also a kind of mock implementation that serves demonstration purposes only:

package javax0.blog.demo.throwable.v1;

import java.util.List;

public class FileReader {
    final String fileName;

    public FileReader(String fileName) {
        this.fileName = fileName;
    }

    public List<String> list() {
        if (fileName.equals("a.txt")) {
            return List.of("wtf wtf", "wtf something", "nothing");
        }
        if (fileName.equals("b.txt")) {
            return List.of("wtf wtf wtf", "wtf something wtf", "nothing wtf");
        }
        if (fileName.equals("c.txt")) {
            return List.of("wtf wtf wtf", "wtf something wtf", "nothing wtf", "");
        }
        throw new RuntimeException("File is not found: "+ fileName);
    }

}

The counter, which counts the number of wtf occurrences in a line is

package javax0.blog.demo.throwable.v1;

public class LineWtfCounter {
    private final String line;

    public LineWtfCounter(String line) {
        this.line = line;
    }

    public static final String WTF = "wtf";
    public static final int WTF_LEN = WTF.length();

    public int count() {
        if (line.length() == 0) {
            throw new LineEmpty();
        }
        // the actual lines are removed from the documentation snippet
    }

}

To save space and focus on our topic the snippet does not display the actual logic (was automatically removed by Java::Geci). The reader can create a code that actually counts the number of wtf substrings in a string, or else simply “wtf”. Even if the reader cannot write such a code it is available from the repository mentioned at the start of the article.


The logic in our application says that this is an exceptional situation if one of the lines in the file has zero length. In that case, we throw an exception.


Usually, such a situation does not verify to be an exception, and I acknowledge that this is a bit contrived example, but we needed something simple. If the length of the line is zero then we throw a LineEmpty exception. (We do not list the code of LineEmpty exception. It is in the code repo, and it is simple, nothing special. It extends RuntimeException, no need to declare where we throw it.) If you look at the mock implementation of FileReader then you can see that we planted an empty line in the file c.txt.

The counter on the file level using the line level counter is the following:

package javax0.blog.demo.throwable.v1;

public class FileWtfCounter {
    // fileReader injection is omitted for brevity
    public int count() {
        final var lines = fileReader.list();
        int sum = 0;
        for (final var line : lines) {
            sum += new LineWtfCounter(line).count();
        }
        return sum;
    }

}

(Again, some trivial lines are skipped from the printout.)

This is the first version of the application. It does not have any special exception handling. It just sums up the values that the line counters return and in case there is an exception on the lower level, in the line wtf counter then this will automatically propagate up. We do not handle that exception in any way on this level.

The project level counter is very similar. It uses the file counter and sums up the results.

package javax0.blog.demo.throwable.v1;

import javax0.blog.demo.throwable.FileLister;

public class ProjectWftCounter {
    // fileLister injection is omitted for brevity
    public int count() {
        final var fileNames = fileLister.list();
        int sum = 0;
        for (final var fileName : fileNames) {
            sum += new FileWtfCounter(new FileReader(fileName)).count();
        }
        return sum;
    }
}

We test it using the simple test code:

package javax0.blog.demo.throwable.v1;

import javax0.blog.demo.throwable.FileLister;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.catchThrowable;

public class TestWtfCounter {

    @Test
    @DisplayName("Throws up for a zero length line")
    void testThrowing() {
        Throwable thrown = catchThrowable(() ->
                new ProjectWftCounter(new FileLister())
                        .count());
        assertThat(thrown).isInstanceOf(LineEmpty.class);
        thrown.printStackTrace();
    }

}

A unit test usually should not have a stack trace print. In this case we have it to demonstrate what is thrown. The stack trace in the error will show us the error as the following:

javax0.blog.demo.throwable.v1.LineEmpty: There is a zero length line
    at javax0.blog.demo.throwable.v1.LineWtfCounter.count(LineWtfCounter.java:18)
    at javax0.blog.demo.throwable.v1.FileWtfCounter.count(FileWtfCounter.java:19)
    at javax0.blog.demo.throwable.v1.ProjectWftCounter.count(ProjectWftCounter.java:22)
    at javax0.blog.demo.throwable.v1.TestWtfCounter.lambda$testThrowing$0(TestWtfCounter.java:18)
    at org.assertj.core.api.ThrowableAssert.catchThrowable(ThrowableAssert.java:62)
    ...
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)

There is a little problem with this exception. When we use this code it does not tell us anything about the actual file and line that is problematic. We have to examine all the files and all the lines if there is an empty one. It is not too difficult to write an application for that, but we do not want to work instead of the programmer who created the application. When there is an exception we expect the exception to give us enough information to successfully tackle the situation. The application has to tell me which file and which line is faulty.

Version 2, setting cause

To provide the information in the exception we have to gather it and insert it into the exception. This is what we do in the second version of the application.

The exception in the first version does not contain the name of the file, or the line number because the code does not put it there. The code has a good reason to do that. The code at the location of the exception throwing does not have the information and thus it cannot insert into the exception what it does not have.

A lucrative approach could be to pass this information along with the other parameters so that when an exception happens the code can insert this information into the exception. I do not recommend that approach. If you look at the source codes I published on GitHub you may find examples of this practice. I am not proud of them, and I am sorry.
Generally, I recommend that the exception handling should not interfere with the main data flow of the application. It has to be separated as it is a separate concern.

The solution is to handle the exception on several levels, on each level adding the information, which is available at the actual level. To do that we modify the classes FileWtfCounter and ProjectWftCounter.

The code of ProjectWftCounter becomes the following:

package javax0.blog.demo.throwable.v2;

public class FileWtfCounter {
    // some lines deleted ...
    public int count() {
        final var lines = fileReader.list();
        int sum = 0;
        int lineNr = 1;
        for (final var line : lines) {
            try {
                sum += new LineWtfCounter(line).count();
            }catch(LineEmpty le){
                throw new NumberedLineEmpty(lineNr,le);
            }
            lineNr ++;
        }
        return sum;
    }

}

The code catches the exception that signals the empty line and throws a new one, which already has a parameter: the serial number of the line.

The code for this exception is not so trivial as in the case of LineEmpty, thus it is listed here:

package javax0.blog.demo.throwable.v2;

public class NumberedLineEmpty extends LineEmpty {
    final protected int lineNr;

    public NumberedLineEmpty(int lineNr, LineEmpty cause) {
        super(cause);
        this.lineNr = lineNr;
    }

    @Override
    public String getMessage() {
        return "line " + lineNr + ". has zero length";
    }
}

We store the line number in an int field, which is final. We do it because

  • use final variables if possible
  • use primitives over objects if possible
  • store the information in its original form as long as possible so that the use of it is not limited

The first two criteria are general. The last one is special in this case, although it is not specific to exception handling. When we are handling exceptions, however, it is very lucrative to just generate a message that contains the line number instead of complicating the structure of the exception class. After all, the reasoning that we will never
use the exception for anything else than printing it to the screen is valid. Or not? It depends. First of all, never say never. Second thought: if we encode the line number into the message then it is certain that we will not ever use it for anything else than printing it to the user. That is because we cannot use it for anything else. We limit ourselves. The today programmer limits the future programmer to do something meaningful with the data.

You may argue that this is YAGNI. We should care about storing the line number as an integer when we want to use it and caring about it at the very moment is too early and is just a waste of time. You are right! At the same time, the person who is creating the extra field and the getMessage() method that calculates the text version of the exception information is also right. Sometimes there is a very thin line between YAGNI and careful and good style programming. YAGNI is to avoid complex code that later you will not need (except that when you create it, you think that you will need). In this example, I have the opinion that the above exception with that one extra int field is not “complex”.

We have a similar code on the “project” level, where we handle all the files. The code of ProjectWftCounter will be

package javax0.blog.demo.throwable.v2;

import javax0.blog.demo.throwable.FileLister;

public class ProjectWftCounter {
    // some lines deleted ...
    public int count() {
        final var fileNames = fileLister.list();
        int sum = 0;
        for (final var fileName : fileNames) {
            try {
                sum += new FileWtfCounter(new FileReader(fileName)).count();
            } catch (NumberedLineEmpty nle) {
                throw new FileNumberedLineEmpty(fileName, nle);
            }
        }
        return sum;
    }
}

Here we know the name of the file and thus we can extend the information adding it to the exception.

The exception FileNumberedLineEmpty is also similar to the code of NumberedLineEmpty. Here is the code of FileNumberedLineEmpty:

package javax0.blog.demo.throwable.v2;

public class FileNumberedLineEmpty extends NumberedLineEmpty {
    final protected String fileName;

    public FileNumberedLineEmpty(String fileName, NumberedLineEmpty cause) {
        super(cause.lineNr, cause);
        this.fileName = fileName;
    }

    @Override
    public String getMessage() {
        return fileName + ":" + lineNr + " is empty";
    }
}

At this moment I would draw your focus to the fact that the exceptions that we created are also in inheritance hierarchy. They extend the other as the information we gather and store is extended, thus:

FileNumberedLineEmpty - extends -> NumberedLineEmpty - extends -> LineEmpty

If the code using these methods expects and tries to handle a LineEmpty exception then it can do even if we throw a more detailed and specialized exception. If a code wants to use the extra information then it, eventually, has to know that the actual instance is not LineEmpty rather something more specialized as NumberedLineEmpty or FileNumberedLineEmpty. However, if it only wants to print it out, get the message then it is absolutely fine to handle the exception as an instance of LineEmpty. Even doing so the message will contain the extra information in human-readable form thanks to OO programming polymorphism.

The proof of the pudding is in the eating. We can run our code with the simple test. The test code is the same as it was in the previous version with the only exception that the expected exception type is FileNumberedLineEmpty instead of LineEmpty. The printout, however, is interesting:

javax0.blog.demo.throwable.v2.FileNumberedLineEmpty: c.txt:4 is empty
    at javax0.blog.demo.throwable.v2.ProjectWftCounter.count(ProjectWftCounter.java:22)
    at javax0.blog.demo.throwable.v2.TestWtfCounter.lambda$testThrowing$0(TestWtfCounter.java:17)
    at org.assertj.core.api.ThrowableAssert.catchThrowable(ThrowableAssert.java:62)
...
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
Caused by: javax0.blog.demo.throwable.v2.NumberedLineEmpty: line 4. has zero length
    at javax0.blog.demo.throwable.v2.FileWtfCounter.count(FileWtfCounter.java:21)
    at javax0.blog.demo.throwable.v2.ProjectWftCounter.count(ProjectWftCounter.java:20)
    ... 68 more
Caused by: javax0.blog.demo.throwable.v2.LineEmpty: There is a zero length line
    at javax0.blog.demo.throwable.v2.LineWtfCounter.count(LineWtfCounter.java:15)
    at javax0.blog.demo.throwable.v2.FileWtfCounter.count(FileWtfCounter.java:19)
    ... 69 more

We can be happy with this result as we immediately see that the file, which is causing the problem is c.txt and the fourth line is the one, which is the culprit. On the other hand, we cannot be happy when we want to have a look at the code that was throwing the exception. Sometime in the future, we may not remember why a line must not have zero length. In that case, we want to look at the code. There we will only see that an exception is caught and rethrown. Luckily there is the cause, but it is actually three steps till we get to the code that is the real problem at LineWtfCounter.java:15.

Will anyone ever be interested in the code that is catching and rethrowing an exception? Maybe yes. Maybe no. In our case, we decide that there will not be anyone interested in that code and instead of handling a long chain of exception listing the causation of the guilty we change the stack trace of the exception that we throw to that of the causing
exception.

Version 3, setting the stack trace

In this version, we only change the code of the two exceptions: NumberedLineEmpty and FileNumberedLineEmpty. Now they not only extend one the other and the other one LineEmpty but they also set their own stack trace to the value that the causing exception was holding.

Here is the new version of NumberedLineEmpty:

package javax0.blog.demo.throwable.v3;

public class NumberedLineEmpty extends LineEmpty {
    final protected int lineNr;

    public NumberedLineEmpty(int lineNr, LineEmpty cause) {
        super(cause);
        this.setStackTrace(cause.getStackTrace());
        this.lineNr = lineNr;
    }

    // getMessage() same as in v2

    @Override
    public Throwable fillInStackTrace() {
        return this;
    }
}

Here is the new version of FileNumberedLineEmpty:

package javax0.blog.demo.throwable.v3;

public class FileNumberedLineEmpty extends NumberedLineEmpty {
    final protected String fileName;

    public FileNumberedLineEmpty(String fileName, NumberedLineEmpty cause) {
        super(cause.lineNr, cause);
        this.setStackTrace(cause.getStackTrace());
        this.fileName = fileName;
    }

    // getMessage(), same as in v2

    @Override
    public Throwable fillInStackTrace() {
        return this;
    }
}

There is a public setStackTrace() method that can be used to set the stack trace of an exception. The interesting thing is that this method is really public and not protected. The fact that this method is public means that the stack trace of any exception can be set from outside. Doing that is (probably) against encapsulation rules.
Nevertheless, it is there and if it is there then we can use it to set the stack trace of the exception to be the same as it is that of the causing exception.

There is another interesting piece of code in these exception classes. This is the public fillInStackTrace() method. If we implement this, like the above then we can save the time the exception spends during the object construction collecting its own original stack trace that we replace and throw away anyway.

When we create a new exception the constructor calls a native method to fill in the stack trace. If you look at the default constructor of the class java.lang.Throwable you can see that actually this is all it does (Java 14 OpenJDK):

public Throwable() {
    fillInStackTrace();
}

The method fillInStackTrace() is not native but this is the method that actually invokes the native fillInStackTrace(int) method that does the work. Here is how it is done:

public synchronized Throwable fillInStackTrace() {
    if (stackTrace != null ||
        backtrace != null /* Out of protocol state */ ) {
        fillInStackTrace(0);
        stackTrace = UNASSIGNED_STACK;
    }
    return this;
}

There is some “magic” in it, how it sets the field stackTrace but that is not really important as for now. It is important, however, to note that the method fillInStackTrace() is public. This means that it can be overridden. (For that, protected would have been enough, but public is even more permitting.)

We also set the causing exception, which, in this case will have the same stack trace. Running the test (similar to the previous tests that we listed only one of), we get the stack print out:

javax0.blog.demo.throwable.v3.FileNumberedLineEmpty: c.txt:4 is empty
    at javax0.blog.demo.throwable.v3.LineWtfCounter.count(LineWtfCounter.java:15)
    at javax0.blog.demo.throwable.v3.FileWtfCounter.count(FileWtfCounter.java:16)
    at javax0.blog.demo.throwable.v3.ProjectWftCounter.count(ProjectWftCounter.java:19)
    at javax0.blog.demo.throwable.v3.TestWtfCounter.lambda$testThrowing$0(TestWtfCounter.java:17)
    at org.assertj.core.api.ThrowableAssert.catchThrowable(ThrowableAssert.java:62)
...
    at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
Caused by: javax0.blog.demo.throwable.v3.NumberedLineEmpty: line 4. has zero length
    ... 71 more
Caused by: javax0.blog.demo.throwable.v3.LineEmpty: There is a zero length line
    ... 71 more

It should be no surprise that we have a FileNumberedLineEmpty with a stack trace that starts on a code line LineWtfCounter.java:15 that does not throw that exception. When we see this there can be some debate about:

  • Why do we need the causing exceptions attached to the original when we overwrite the stack trace? (We do not.)
  • Is this a clean solution? It may be confusing that the stack trace originates from a line that does not throw that exception.

Let’s answer these concerns with, yes, they are needed for the demonstration purpose, and in a real application every programmer may decide if they want to use a solution like that.

Is this the best solution we can get? Probably no, because, as I promised, we have a fourth version of the application.

Version 4, suppressing exceptions

When we created the mock FileReader we were optimistic a lot. We assumed that there is only one line that has zero length. What if there are more than one lines like that? In that case, the application stops at the first one. The user fixes the error either adding some characters to the line, so that this is not an empty one, or deleting it altogether so that this is not a line anymore. Then the user runs the application again to get the second location in the exception. If there are many such lines to correct then this process can be cumbersome. You can also imagine that the code in a real application may run for long minutes let alone for hours. To execute the application just to get the next location of the problem is a waste of human time, waste of CPU clock, energy, and thus clean oxygen generating CO2 unnecessarily.

What we can do is, alter the application so that it goes on processing when there is an empty line, and it throws an exception listing all the lines that were empty and discovered during the process only after all the files and all the lines were processed. There are two ways. One is to create some data structure and store the information in there and at the end of the processing, the application can have a look at that and throw an exception if there is any information about some empty lines there. The other one is to use the structures provided by the exception classes to store the information.

The advantage is to use the structures provided by the exception classes are

  • the structure is already there and there is no need to reinvent the wheel,
  • it is well-designed by many seasoned developers and used for decades, probably is the right structure,

  • the structure is general enough to accommodate other types of exceptions, not only those that we have currently, and the data structure does not need any change.

  • Let’s discuss the last bullet point a bit. It may happen that later we decide that lines that contain WTF all capital are also exceptional and should throw an exception. In that case, we may need to modify our data structures that store these error cases if we decided to craft these structures by hand. If we use the suppressed exceptions of the Throwable class then there is nothing extra to do. There is an exception, we catch it (as you will see in the example soon), store it, and then attach it at the end of the summary exception as a suppressed exception. Is it YAGNI that we think about this future possibility when it is extremely unlikely that this demo application will ever be extended? Yes, and no, and generally it does not matter. YAGNI is usually a problem when you devote time and effort to develop something too early. It is an extra cost in the development and later in the maintenance. When we are just using something simpler that is already there then it is not YAGNI to use it. It is simply clever and knowledgable about the tool we use.

    Let’s have a look at the modified FileReader that this time already returns many empty lines in many files:

    package javax0.blog.demo.throwable.v4;
    
    import java.io.FileNotFoundException;
    import java.util.List;
    
    public class FileReader {
        final String fileName;
    
        public FileReader(String fileName) {
            this.fileName = fileName;
        }
    
        public List<String> list() {
            if (fileName.equals("a.txt")) {
                return List.of("wtf wtf", "wtf something", "", "nothing");
            }
            if (fileName.equals("b.txt")) {
                return List.of("wtf wtf wtf", "", "wtf something wtf", "nothing wtf", "");
            }
            if (fileName.equals("c.txt")) {
                return List.of("wtf wtf wtf", "", "wtf something wtf", "nothing wtf", "");
            }
            throw new RuntimeException("File is not found: "+ fileName);
        }
    
    }
    

    Now all three files contain lines that are empty. We do not need to modify the LineWtfCounter counter. When there is an empty line, we throw an exception. On this level, there is no way to suppress this exception. We cannot collect here any exception list. We focus on one single line that may be empty.

    The case is different in FileWtfCounter:

    package javax0.blog.demo.throwable.v4;
    
    public class FileWtfCounter {
        private final FileReader fileReader;
    
        public FileWtfCounter(FileReader fileReader) {
            this.fileReader = fileReader;
        }
    
        public int count() {
            final var lines = fileReader.list();
            NumberedLinesAreEmpty exceptionCollector = null;
            int sum = 0;
            int lineNr = 1;
            for (final var line : lines) {
                try {
                    sum += new LineWtfCounter(line).count();
                }catch(LineEmpty le){
                    final var nle = new NumberedLineEmpty(lineNr,le);
                    if( exceptionCollector == null ){
                        exceptionCollector = new NumberedLinesAreEmpty();
                    }
                    exceptionCollector.addSuppressed(nle);
                }
                lineNr ++;
            }
            if( exceptionCollector != null ){
                throw exceptionCollector;
            }
            return sum;
        }
    
    }
    

    When we catch a LineEmpty exception we store it in an aggregate exception referenced by the local variable exceptionCollector. If there is not exceptionCollector then we create one before adding the caught exception to it to avoid NPE. At the end of the processing when we processed all the lines we may have many exceptions added to the summary exception exceptionCollector. If it exists then we throw this one.

    Similarly, the ProjectWftCounter collects all the exceptions that are thrown by the different FileWtfCounter instances and at the end of the processing it throws the summary exception as you can see in the following code lines:

    package javax0.blog.demo.throwable.v4;
    
    import javax0.blog.demo.throwable.FileLister;
    
    public class ProjectWftCounter {
    
        private final FileLister fileLister;
    
        public ProjectWftCounter(FileLister fileLister) {
            this.fileLister = fileLister;
        }
    
    
        public int count() {
            final var fileNames = fileLister.list();
            FileNumberedLinesAreEmpty exceptionCollector = null;
            int sum = 0;
            for (final var fileName : fileNames) {
                try {
                    sum += new FileWtfCounter(new FileReader(fileName)).count();
                } catch (NumberedLinesAreEmpty nle) {
                    if( exceptionCollector == null ){
                        exceptionCollector = new FileNumberedLinesAreEmpty();
                    }
                    exceptionCollector.addSuppressed(nle);
                }
            }
            if( exceptionCollector != null ){
                throw exceptionCollector;
            }
            return sum;
        }
    }
    

    Now that we have collected all the problematic lines into a huge exception structure we get a stack trace that we deserve:

    javax0.blog.demo.throwable.v4.FileNumberedLinesAreEmpty: There are empty lines
        at javax0.blog.demo.throwable.v4.ProjectWftCounter.count(ProjectWftCounter.java:24)
        at javax0.blog.demo.throwable.v4.TestWtfCounter.lambda$testThrowing$0(TestWtfCounter.java:17)
        at org.assertj.core.api.ThrowableAssert.catchThrowable(ThrowableAssert.java:62)
        at org.assertj.core.api.AssertionsForClassTypes.catchThrowable(AssertionsForClassTypes.java:750)
        at org.assertj.core.api.Assertions.catchThrowable(Assertions.java:1179)
        at javax0.blog.demo.throwable.v4.TestWtfCounter.testThrowing(TestWtfCounter.java:15)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
        at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
        at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
        at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
        at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
        at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
        at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
        at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
        at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:205)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:201)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1510)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1510)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
        at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
        at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
        at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
        at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:132)
        at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:69)
        at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
        at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
        at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
        Suppressed: javax0.blog.demo.throwable.v4.NumberedLinesAreEmpty
            at javax0.blog.demo.throwable.v4.FileWtfCounter.count(FileWtfCounter.java:22)
            at javax0.blog.demo.throwable.v4.ProjectWftCounter.count(ProjectWftCounter.java:21)
            ... 68 more
            Suppressed: javax0.blog.demo.throwable.v4.NumberedLineEmpty: line 3.
                at javax0.blog.demo.throwable.v4.LineWtfCounter.count(LineWtfCounter.java:15)
                at javax0.blog.demo.throwable.v4.FileWtfCounter.count(FileWtfCounter.java:18)
                ... 69 more
            Caused by: javax0.blog.demo.throwable.v4.LineEmpty: There is a zero length line
        Suppressed: javax0.blog.demo.throwable.v4.NumberedLinesAreEmpty
            at javax0.blog.demo.throwable.v4.FileWtfCounter.count(FileWtfCounter.java:22)
            at javax0.blog.demo.throwable.v4.ProjectWftCounter.count(ProjectWftCounter.java:21)
            ... 68 more
            Suppressed: javax0.blog.demo.throwable.v4.NumberedLineEmpty: line 2.
                at javax0.blog.demo.throwable.v4.LineWtfCounter.count(LineWtfCounter.java:15)
                at javax0.blog.demo.throwable.v4.FileWtfCounter.count(FileWtfCounter.java:18)
                ... 69 more
            Caused by: javax0.blog.demo.throwable.v4.LineEmpty: There is a zero length line
            Suppressed: javax0.blog.demo.throwable.v4.NumberedLineEmpty: line 5.
                at javax0.blog.demo.throwable.v4.LineWtfCounter.count(LineWtfCounter.java:15)
                at javax0.blog.demo.throwable.v4.FileWtfCounter.count(FileWtfCounter.java:18)
                ... 69 more
            Caused by: javax0.blog.demo.throwable.v4.LineEmpty: There is a zero length line
        Suppressed: javax0.blog.demo.throwable.v4.NumberedLinesAreEmpty
            at javax0.blog.demo.throwable.v4.FileWtfCounter.count(FileWtfCounter.java:22)
            at javax0.blog.demo.throwable.v4.ProjectWftCounter.count(ProjectWftCounter.java:21)
            ... 68 more
            Suppressed: javax0.blog.demo.throwable.v4.NumberedLineEmpty: line 2.
                at javax0.blog.demo.throwable.v4.LineWtfCounter.count(LineWtfCounter.java:15)
                at javax0.blog.demo.throwable.v4.FileWtfCounter.count(FileWtfCounter.java:18)
                ... 69 more
            Caused by: javax0.blog.demo.throwable.v4.LineEmpty: There is a zero length line
            Suppressed: javax0.blog.demo.throwable.v4.NumberedLineEmpty: line 5.
                at javax0.blog.demo.throwable.v4.LineWtfCounter.count(LineWtfCounter.java:15)
                at javax0.blog.demo.throwable.v4.FileWtfCounter.count(FileWtfCounter.java:18)
                ... 69 more
            Caused by: javax0.blog.demo.throwable.v4.LineEmpty: There is a zero length line
    

    This time I did not delete any line to make you feel the weight of it on your shoulder. Now you may start to think if it was really worth using the exception structure instead of some neat, slim special-purpose data structure that contains only the very information that we need. If you start to think that, then stop it. Don’t do it. The problem, if any, is not that we have too much information. The problem is the way we represent it. To overcome it the solution is not to throw out the baby with the bathwater… the excess information but rather to represent it in a more readable way. If the application rarely meets many empty lines, then reading through the stack trace may not be an unbearable burden for the user. If it is a frequent problem, and you want to be nice to your users (customers, who pay your bills) then, perhaps, a nice exception structure printer is a nice solution.

    We actually have one for you in the project

    javax0.blog.demo.throwable.v4.ExceptionStructurePrettyPrinter

    that you can use and even modify at your will. With this the printout of the previous “horrendous” stack trace will print out as:

    FileNumberedLinesAreEmpty("There are empty lines")
        Suppressed: NumberedLineEmpty("line 3.")
          Caused by:LineEmpty("There is a zero length line")
        Suppressed: NumberedLineEmpty("line 2.")
          Caused by:LineEmpty("There is a zero length line")
        Suppressed: NumberedLineEmpty("line 5.")
          Caused by:LineEmpty("There is a zero length line")
        Suppressed: NumberedLineEmpty("line 2.")
          Caused by:LineEmpty("There is a zero length line")
        Suppressed: NumberedLineEmpty("line 5.")
          Caused by:LineEmpty("There is a zero length line")
    

    With this, we got to the end of the exercise. We stepped through the steps from v1 simply throwing and catching and exception, v2 setting causing exceptions matryoshka style, v3 altering the stack trace of the embedding exception, and finally v4 storing all the suppressed exceptions that we collected during our process. What you can do now is download the project, play around with it, examine the stack traces, modify the code, and so on. Or read on, we have some extra info about exceptions that are rarely discussed by basic level tutorials, and it is also worth reading the final takeaway section.

    Other things to know about exceptions

    In this section, we will tell you some information that is not well known and is usually missing from the basic Java tutorials that talk about exceptions.

    There is no such thing as checked exception in the JVM

    Checked exceptions cannot be thrown from a Java method unless the method declaration explicitly says that this may happen. The interesting thing is that the notion of checked exceptions is not known for the JVM. This is something handled by the Java compiler, but when the code gets into the JVM there is no check about that.

    Throwable (checked) <-- Exception (checked) <-- RuntimeException (unchecked)
                                                <-- Other Exceptions (checked)
                        <-- Error (unchecked)
    

    The structure of the exception classes is as described above. The root class for the exceptions is the Throwable. Any object that is an instance of a class, which extends directly or indirectly the Throwable class can be thrown. The root class Throwable is checked, thus if an instance of it is thrown from a method, then it has to be declared.
    If any class extends this class directly and is thrown from a method then, again it has to be declared. Except if the object is also an instance of RuntimeException or Error. In that case the exception or error is not checked and can be thrown without declaring on the throwing method.

    The idea of checked exception is controversial. There are advantages of its use but there are many languages that do not have the notion of it. This is the reason why the JVM does not enforce the declaration of checked exceptions. If it did it would not be possible reasonably to generate JVM code from languages that do not require exceptions declared and want to interoperate with the Java exceptions. Checked exceptions also cause a lot of headaches when we are using streams in Java.

    It is possible to overcome of checked exceptions. A method created with some hack, or simply in a JVM language other than Java can throw a checked exception even if the method does not declare the exception to be thrown. The hacky way uses a simple static utility method, as listed in the following code snippet:

    package javax0.blog.demo.throwable.sneaky;
    
    public class SneakyThrower {
        public static <E extends Throwable> E throwSneaky(Throwable e) throws E {
            throw (E) e;
        }
    }
    

    When a code throws a checked exception, for example Exception then passing it to throwSneaky() will fool the compiler. The compiler will look at the declaration of the static method and cannot decide if the Throwable it throws is checked or not. That way it will not require the declaration of the exception in the throwing method.

    The use of this method is very simple and is demonstrated with the following unit test code:

    package javax0.blog.demo.throwable.sneaky;
    
    import org.junit.jupiter.api.DisplayName;
    import org.junit.jupiter.api.Test;
    
    import static javax0.blog.demo.throwable.sneaky.SneakyThrower.throwSneaky;
    import static org.assertj.core.api.Assertions.assertThat;
    import static org.assertj.core.api.Assertions.catchThrowable;
    
    public class TestSneaky {
    
        @DisplayName("Can throw checked exception without declaring it")
        @Test
        void canThrowChecked() {
            class FlameThrower {
                void throwExceptionDeclared() throws Exception {
                    throw new Exception();
                }
    
                void throwExceptionSecretly() {
                    throwSneaky(new Exception());
                }
            }
            final var sut = new FlameThrower();
            assertThat(catchThrowable(() -> sut.throwExceptionDeclared())).isInstanceOf(Exception.class);
            assertThat(catchThrowable(() -> sut.throwExceptionSecretly())).isInstanceOf(Exception.class);
        }
    
        int doesNotReturn(){
            throw throwSneaky(new Exception());
            // no need for a return command
        }
    
    }
    

    The two methods throwExceptionDeclared() and throwExceptionSecretly() demonstrate the difference between normal and sneaky throwing.

    The method throwSneaky() never returns, and it still has a declared return value. The reason for that is to allow the pattern that can be seen in the method doesNotReturn() towards the end of the text code. We know that the method throwSneaky() never returns, but the compiler does not know. If we simply call it then the compiler will still require some return statement in our method. In more complex code flow it may complain about uninitialized variables. On the other hand if we “throw” the return value in the code then it gives the compiler a hint about the execution flow. The actual throwing on this level will never happen actually, but it does not matter.

    Never catch Throwable, ...Error or COVID

    When we catch an exception we can catch checked exception, RuntimeException or just anything that is Throwable. However, there are other things that are Throwable but are not exceptions and are also not checked. These are errors.

    Story:

    I do a lot of technical interviews where candidates come and answer my questions. I have a lot of reservations and bad feelings about this. I do not like to play “God”. On the other hand, I enjoy a lot when I meet clever people, even if they are not fit for a given work position. I usually try to conduct the interviews that the value from it is not only the evaluation of the candidate but also something that the candidate can learn about Java, the profession, or just about themselves. There is a coding task that can be solved using a loop, but it lures inexperienced developers to have a solution that is recursive. Many of the developers who create the recursive solution realize that there is no exit condition in their code for some type of the input parameters. (Unless there is because they do it in the clever way. However, when they are experienced enough, they do not go for the recursive solution instead of a simple loop. So when it is a recursive solution they almost never have an exit condition.) What will happen if we run that code with an input parameter that never ends the recursive loop? We get a StackOverflowException. Under the pressure and stress of the interview, many of them craft some code that catches this exception. This is problematic. This is a trap!

    Why is it a trap? Because the code will not ever throw a StackOverflowException. There is no such thing in the JDK as StackOverflowException. It is StackOverflowError. It is not an exception, and the rule is that

    YOUR CODE MUST NEVER CATCH AN ERROR

    The StackOverflowError (not exception) extends the class VirtualMachineError which says in the JavaDoc:

    Thrown to indicate that the Java Virtual Machine is broken

    When something is broken you can glue it together, mend, fix, but you can never make it unbroken. If you catch a Throwable which is also an instance of Error then the code executing in the catch part is run in a broken VM. What can happen there? Anything and the continuation of the execution may not be reliable.

    Never catch an Error!

    Summary and Takeaway

    In this article we discussed exceptions, specifically:

    • how to throw meaningful exceptions by adding information when it becomes available,
  • how to replace the stack trace of an exception with setTrackTrace() when it makes sense,

  • how to collect exceptions with addSuppressed() when your application can throw exceptions multiple times We also discussed some interesting bits about how the JVM does not know about checked exceptions and why you should never catch an Error.

  • Don’t just (re)throw exceptions when they happen. Think about why and how they happen and handle them appropriately.

    Use the information in this article to make your code exceptional 😉

    (Code and article were reviewed and proofread by Mihaly Verhas. He also wrote the takeaway section including the last
    sentence.)

    JDK14 instanceof EA issue

    Tagir Valeev recently had a tweet about the preview feature of the soon coming JDK14 release of Java:

    https://platform.twitter.com/widgets.js

    The issue is that there is a planned and in the EA release already available new feature of Java that introduces pattern variables and the current version of the proposed new standard leaves room for some really spooky coding issue.

    Following the tweet, the details were discussed in detail enough to understand the actual problem. In this article, however, I will summarize what all this is about so that you do not need to dig yourself through the tweets and the standards.

    What is a pattern variable

    Before getting into the deep detail of the issue outlines in the tweet above, let’s discuss a bit, what a pattern variable is. (Maybe a bit sloppy, more explanatory than precise and complete, but here it comes.)

    Programming many times we need to check the type of some objects. The operator instanceof does that for us. A typical sample code can be something like this:

    // HOW THIS IS TODAY, JAVA < 14
    
    Object z = "alma";
    if (!(z instanceof String)){
        throw new IllegalArgumentException();
    }
    System.out.println(((String)z).length());
    

    In real life, the variable z may come from somewhere else, in which case it is not so obvious that this is a string. When we want to print out the length of the string using println we already know that the object referenced by z is a string. The compiler, on the other hand, does not.We have to cast the variable to a String and then we can use the length() method. Other languages do it better. Ideally, I could write:

    // HOW IT WOULD BE THE SIMPLEST
    
    Object z = "alma";
    if (!(z instanceof String)){
        throw new IllegalArgumentException();
    }
    System.out.println(z.length());
    

    That is not the Java way and also that is not the way JDK14 simplifies this programming pattern. Instead, the proposed feature introduces a new syntax for the instanceof operator that introduces a new variable: a pattern variable.

    To make a long story short, the above example will look the following:

    // HOW IT IS IN JDK14-EA / OpenJDK (build 14-ea+28-1366)
    
    Object z = "alma";
    if (!(z instanceof String s)){
        throw new IllegalArgumentException();
    }
    System.out.println(s.length());
    

    It introduces a new variable s that is in scope only when the referenced object is a String. A simpler version of the code without the exception throwing part would be

    Object z = "alma";
    if (z instanceof String s){
        // we have here 's' and it is a String
        System.out.println(s.length());
    }
    
    // we do not have 's' here
    

    When the condition is true, the object is a string thus we have ‘s’. If the condition is false then we jump over the then_statement, and there we do not have ‘s’ as we do not have a string. ‘s’ is available in the code which only runs when the object is a string. That way the variable scope of a pattern variable is determined and constrained not only by the syntactical scope of the variable but also by the possible control flow. Only the control flow that can be analyzed with certainty is taken into account.

    Such control-flow analysis is not unparalleled in the Java compiler. A Java program will not compile, for example, if there is an unreachable code that the compiler can detect.

    So far it seems to be simple and we are all happy to get the new feature in Java 14.

    The JSL14 standard

    The precise scope calculation is defined in the JLS14 (Java Language Specification 14) standard. At the time of this article, the spec is only available as a preview.

    http://cr.openjdk.java.net/~gbierman/jep305/jep305-20191021/specs/patterns-instanceof-jls.html#jls-6.3.2.2

    As the execution flow of a Java program can be controlled by many different language-constructs the scope of a pattern variable is defined for each of these structures. There are separate sections for the different logical operators that evaluate short-circuit, ‘if’ statement, ‘while’ statement and so on. I do not want to discuss the different cases extensively. I will focus here only on the case of the ‘if’ statement without the ‘else’ part. The standard cited above says:

    The following rules apply to a statement `if (e) S` (14.9.1):

    * A pattern variable introduced by e when true is definitely matched at `S`.

    It is a compile-time error if any pattern variable introduced by `e` when true is already in scope at `S`.

    * `V` is introduced by `if (e) S` if and only if `V` is introduced by `e` when `false` and `S` cannot complete normally.

    It is a compile-time error if any pattern variable introduced by the `if` statement is already in scope.

    The interesting part is the “cannot complete normally”. A good example of this is our example above: we create a so-called guarding if statement. When the variable z is not a String then we throw an exception, return or do something else that will always prevent the execution to reach the code after the if statement when the variable is not a String.

    In the case of a throw or return statement, it is usually very straightforward and easy to see that the code “cannot complete normally”. In case of an infinite loop, this is not always so evident.

    The Problem

    Let’s have a look at the following code fragment:

    private static boolean FLAG = true;
    static String variable = "Hello from field";
    
    public static void main() {
        Object z = "Hello from pattern matching";
        if (!(z instanceof String variable)){
            while (FLAG) {
                System.out.println("We are in an endless loop");
            }
        }
        System.out.println(variable);
    }
    

    In this case, we have a loop, which is infinite or not. It depends on the other part of the code that may alter the value of the class field FLAG from true to false. This part of the code “can complete normally”.

    If we modify the above code just a little making the field FLAG to be final, as

    private static final boolean FLAG = true;
    static String variable = "Hello from field";
    
    public static void main() {
        Object z = "Hello from pattern matching";
        if (!(z instanceof String variable)){
            while (FLAG) {
                System.out.println("We are in an endless loop");
            }
        }
        System.out.println(variable);
    }
    

    then the compiler will see that the loop is infinite and cannot complete normally. The program will print out Hello from field in the first case, and it will print Hello from pattern matching. The pattern variable in the second case hides the field variable because of the scope of the pattern variable is extended to the commands following the if statement because the then-part cannot complete normally.

    This is really a problem with this preview feature as it is. The readability of the code, in this case, is very questionable. The scope of the pattern variable and if it is hiding a field or not depends on the final modifier of the field, which is not there. When we look at some code the actual execution and the result of the code should be simple and should not really depend on some code that is far away and may skip our attention reading the code locally.

    This is not the only situation in Java that has this anomaly. You can have a class named String for example in your codebase. The code of the classes, which are in the same package will use that class when they refer to the type String. If we delete the String class from the user code then the meaning of the String type becomes java.lang.String. The actual meaning of the code depends on some other code that is “far”.

    This second example, however, is a hack and it is not likely that a Java programmer who has not lost their mind names a class String (seriously https://github.com/verhas/jScriptBasic/blob/master/src/main/java/com/scriptbasic/classification/String.java?) or some other name that also exists in the JDK in the java.lang package. Maybe it is pure luck, maybe it was well considered during the decision making to avoid the mandatory import of the classes from the java.lang package. This is history.

    The variable name shadowing and the situation above is, on the other hand, does not seem to be so weird and something that surely will not accidentally happen in some Java code.

    Fortunately, this is only a preview feature. It will be in the JDK14 as it is, but as a preview feature it is only available when the javac compiler and the java execution uses the --enable-preview flag and the preview feature may change in the future in an incompatible way.

    Solution

    I cannot tell how it will change. I cannot even tell that it will change at all. It is only my personal opinion that it would be very sad if it remained like that. With this feature, Java would be a better language so long as long we count how brilliantly and readable a seasoned Java programmer can program. But it will worse if we look at how a non-seasoned, fresh junior can fuck the code up. In my humble opinion, this second is the more important and Java has a very strong point in this. Java is not a hacker language, and you should be very desperate to write a very unreadable code. I would not like it changing.

    After having said that we can look at the technical possibilities. One is to abandon the feature, which would not really be a good solution. It would not actually be a solution.

    Another possibility is to limit the scope of the pattern variables to the then statement or to the else statement.

    That way we do not rely on the “cannot complete normally” feature of the code. The else guarantees that the else branch is executed only when the condition of the if statement is false. This will make the solution less elegant.

    Again another possibility is to forbid for the pattern variables to shadow any field variable. It would solve the problem outlined above but would introduce a different one. With this restriction, it could happen that an existing class with methods and pattern variable V stops compiling when we introduce a new field variable named V. At least this issue is compile-time and not some code that is buggy during run-time.

    I rather have 100 compile time error than one run-time error.

    Still another possibility is to abandon the pattern variable and just to use the original variable with extended type information where the current preview solution uses the pattern variable. Kotlin fans would love this solution. This would also elegantly eliminate the shadowing issue because the local variable already shadows (or does not) the field variable. The drawback of this solution is that the variable type re-scoped would have different types in different places in the code. Let’s have a look at the following code:

    package javax0.jdk14.instanceof0;
    
    public class Sample2 {
        public static class A {
            public static void m(){
                System.out.println("A");
            }
        }
        public static class B extends A {
            public static void m(){
                System.out.println("B");
            }
        }
        public static void main(String[] args) {
            A a = new B();
            if( a instanceof B b){
                b.m();
            }
            a.m();
        }
    }
    

    This code will print out B then A because the call to b.m() is the same as B.m() based on the declared type of the variable b and the same way a.m() is the same as A.m() based on the declared type of the variable a. Omitting the pattern variable and using the original variable could make confusion:

    // NOT ACTUAL CODE
        public static void main(String[] args) {
            A a = new B();
            if( a instanceof B){
                a.m();
            }
            a.m();
        }
    

    Would a.m() call different methods on different lines?

    As you can see there is no known good or best solution to this issue… except one. Call your representative in the JDK “parliament” and tell them that it is not good that way. (Psst: they already know it from the original tweet.)

    Takeaway

    This is a special article because this is not about some well established Java feature or some good programming tool or style, pattern, methodology. We discussed a preview feature. A preview feature that, perhaps, proves why we need preview features in Java.

    Use the latest LTS version for long-running commercial projects that will need long term support from you.

    Use the latest released Java version for your experiments and opensource projects and be prepared to support older Java versions if the users need it.

    Do not use the preview features in your projects or be prepared to have a new release from your code in case they change in the next Java releases when they become non-preview but normal features.

    Experiment with the pre-view features to embrace them and to have a kind of muscle memory when they become real features. And also to give feedback to the Java community in case you feel they are not really perfect.

    Be part of the community!

    Repeated code

    Introduction

    It is usually not good to have copy/paste code in our Java application but sometimes it is unavoidable. For example the project License3j provides a method isXXX in the Feature class for each XXX type it supports. In that case, we can do no better than write

        public boolean isBinary() {
            return type == Type.BINARY;
        }
    
        public boolean isString() {
            return type == Type.STRING;
        }
    
        public boolean isByte() {
            return type == Type.BYTE;
        }
    
        public boolean isShort() {
            return type == Type.SHORT;
        }
    
    and so on
    
    

    for each and every feature type the application supports. And there are some types there: Binary, String, Byte, Short, Int, Long, Float, Double, BigInteger, BigDecimal, Date, UUID. It is not only a boring task to type all the very similar methods, but it is also error-prone. A very few humans are good at doing such a repetitive task. To avoid that we can use the Java::Geci framework and as the simplest solution we can use the generator Iterate.

    POM dependency

    To use the generator we have to add the dependency

    <dependency>
        <groupId>com.javax0.geci</groupId>
        <artifactId>javageci-core</artifactId>
        <scope>test</scope>
        <version>1.4.0</version>
    </dependency>
    

    The library is executed only during when the tests run, therefore the use of it does not imply any extra dependency. Whoever wants to use the library License3j does not need to use Java::Geci. This is only a development tool used in test scope.

    Unit Test running it

    The dependency will not run by itself. After all the dependency is not a program. It is a bunch of class files packaged into a JAR to be available on the classpath. We have to execute the generator and it has to be done through the framework creating a unit test:

        @Test
        @DisplayName("run Iterate on the sources")
        void runIterate() throws IOException {
            Geci geci = new Geci();
            Assertions.assertFalse(
                geci.register(Iterate.builder()
                                  .define(ctx -> ctx.segment().param("TYPE", ctx.segment().getParam("Type").orElse("").toUpperCase()))
                                  .build())
                    .generate()
                , geci.failed()
            );
        }
    

    It creates a Geci object, instantiates the generator using a builder and then invokes generate() on the configured framework Geci object. The define() call seems a bit cryptic as for now. We will shed light on that later.

    Source Code Preparation

    The final step before executing the build is to define a template and the values to insert into the template. Instead of writing all the methods all we have to do is to write a template and an editor fold segment:

        /* TEMPLATE
        LOOP Type=Binary|String|Byte|Short|Int|Long|Float|Double|BigInteger|BigDecimal|Date|UUID
        public boolean is{{Type}}() {
            return type == Type.{{TYPE}};
        }
    
         */
        //<editor-fold id="iterate">
        //</editor-fold>
    

    When we execute the generator through the framework it will evaluate the template for each value of the placeholder Type and it will replace each {{Type}} with the actual value. The resulting code will be inserted into the editor-fold segment with the id “iterate”.

    Looking at the template you can see that there is a placeholder {{TYPE}}, which is not defined in the list. This is where the unite test define() comes into the picture. It defines a consumer that consumes a context and using that context it reads the actual value of Type, creates the uppercased version of the value and assigns it to the segment parameter named TYPE.

    Generally, that is it. There are other functionalities using the generator, like defining multiple values per iteration assigned to different placeholders, escaping or skipping lines and so on. About those here is an excerpt from the documentation that you can read up-to-date and full az https://github.com/verhas/javageci/blob/master/ITERATE.adoc

    Documentation Excerpt

    In the Java source files where you want to use the generator you have to annotate the class with the annotation @Geci("iterate"). You can also use the @Iterate annotation instead, which is defined in the javageci-core-annotations module. This will instruct the Geci framework that you want to use the iterate generator in the given class.

    TEMPLATE

    A template starts after a line that is /\*TEMPLATE or TEMPLATE. There can be spaces before and after and between the /* and the word TEMPLATE but there should not be anything else on the line. When the generator sees such a line it starts to collect the following lines as the content of the template.

    The end of the template is signaled by a line that has */ on it and nothing else (except spaces).

    The content of the template can contain parameters between {{ and }} characters similarly as it is used by the mustache template program. (The generator is not using mustache, template handling is simpler.)

    LOOP

    While collecting the lines of the template some of the lines are recognized as parameter definitions for the template. These lines do not get into the trunk of the template. (Command names on these lines are always capital.)

    As you could see in the introduction the line

        LOOP type=int|long|short
    

    is not part of the template text. It instructs the generator to iterate through the types and set the parameter {{type}} in the text to int first, long the second and short the last. That way you can iterate over multiple values of a single parameter.

    A more complex template may need more than one parameter. In that case, you can list them in the LOOP line as

        LOOP type,var=int,aInt|long,aLong|short,aShort
    

    This will tell the generator to set the parameter {{type}} the same way as before for the three iterations but the same time also set the parameter {{var}} to aInt in the first loop, to aLong in the second loop and aShort in the last loop.

    If the list of the values is too long it is possible to split the list into multiple LOOP lines. In this case, however, the variables have to be repeated in the second, third and so on LOOP lines. Their order may be different, but if there is a variable undefined in some of the LOOP lines then the placeholder referring to it will be resolved and remains in the {{placeholder}} form.

    The above example can also be written

        LOOP type,var=int,aInt
        LOOP var,type=aLong,long
        LOOP type,var=short,aShort
    

    and it will result in the same values as the above LOOP repeated here:

        LOOP type,var=int,aInt|long,aLong|short,aShort
    

    Default editor-fold

    The templates are processed from the start of the file towards the end and the code generated is also prepared in this order. The content of the generated code will be inserted into the editor-fold segment that follows the template directly. Although this way the id of the
    editor-fold segment is not really interesting you must specify a unique id for each segment. This is a restriction of the the Java::Geci framework.

    Advanced Use

    EDITOR-FOLD-ID

    It may happen that you have multiple templates looping over different values and you want the result to go into the same editor-fold segment. It is possible using the EDITOR_FOLD_ID.

    In the following example

    package javax0.geci.iterate.sutclasses;
    
    public class IterateOverMultipleValues {
        /* TEMPLATE
        {{type}} get_{{type}}Value(){
          {{type}} {{variable}} = 0;
          return {{variable}};
        }
    
        LOOP type,variable=int,i|long,l|short,s
        EDITOR-FOLD-ID getters
         */
        //
                // nothing gets here
        //
    
        //
        int get_intValue(){
          int i = 0;
          return i;
        }
    
        long get_longValue(){
          long l = 0;
          return l;
        }
    
        short get_shortValue(){
          short s = 0;
          return s;
        }
    
        //
    }
    

    the generated code gets into the editor-fold that has the id name getters even though this is not the one that follows the template definition.

    Use this feature to send the generated code into a single segment from multiple iterating templates. Usually, it is a good practice to keep the template and the segment together.

    ESCAPE and SKIP

    The end of the template is signaled by a line that is */. This is essentially the end of a comment. What happens if you want to include a comment, like a JavaDoc into the template. You can write the */ characters at the end of the comment lines that still have some characters in it. This solution is not elegant and it essentially is a workaround.

    To have a line that is exactly a comment closing or just any line that would be interpreted by the template processing, like a LOOP line you should have a line containing nothing else but an ESCAPE on the previous line. This will tell the template processing to include the next line into the template text and continue the normal processing on the line after.

    Similarly, you can have a line SKIP to ignore the following line altogether. Using these two commands you can include anything into a template.

    An example shows how you can include a JavaDoc comment into the template:

    package javax0.geci.iterate.sutclasses;
    
    public class SkippedLines {
        /* TEMPLATE
        /**
         * A simple zero getter serving as a test example
         * @return zero in the type {{type}}
        ESCAPE
         */
        // SKIP
        /*
        {{type}} get_{{type}}Value(){
          {{type}} {{variable}} = 0;
          return {{variable}};
        }
        LOOP type,variable=int,i|long,l|short,s
        EDITOR-FOLD-ID getters
         */
        //
        /**
         * A simple zero getter serving as a test example
         * @return zero in the type int
         */
        int get_intValue(){
          int i = 0;
          return i;
        }
        /**
         * A simple zero getter serving as a test example
         * @return zero in the type long
         */
        long get_longValue(){
          long l = 0;
          return l;
        }
        /**
         * A simple zero getter serving as a test example
         * @return zero in the type short
         */
        short get_shortValue(){
          short s = 0;
          return s;
        }
        //
    }
    

    The template starts with the comment and a comment can actually contain any other comment starting. Java comments are not nested. The end of the template is, however the line that contains the */ string. We want this line to be part of the template thus we precede it with the line ESCAPE so it will not be interpreted as the end of the template. On the other hand, for Java, this ends the comment. To continue the template we have to get “back” into comment mode since we do not want the Java compiler to process the template as code. (Last but not least because the template using placeholders is probably not a syntactically correct Java code fragment.) We need a new /* line, which we do not want to get into the template. This line is, therefore, preceded with a line containing // SKIP. (Skip lines can have optional // before the command.)

    The result you can see in the generated code. All methods have the proper JavaDoc documentation.

    SEP1 and SEP2

    Looping over the values you have to separate the names of the placeholders with , and | the list of the values. For example, the sample above contains

        LOOP type,variable=int,i|long,l|short,s
    

    two placeholder names type and variable and three values for each. Placeholders do not need to contain special characters and it is the best if they are standard identifiers. The values, however, may contain a comma or a vertical bar. In that case, you can redefine the string (not only a single character) that the template LOOP command can use instead of the single character strings , and |.

    For example the line

        SEP1 /
    

    says that the names and the values should be separated by / instead of only one and

        SEP2 &
    

    the list of the values should be separated by one character &amp; string. The SEP1 and SEP2 will have effect only if they precede the LOOP command and they are effective only for the template they are used in. Following the above commands, the LOOP example would look like

        LOOP type/variable=int/i&long/l&short/s
    

    That way there is nothing to prevent us to add another value list

        LOOP type/variable=int/i&long/l&short/s&byte,int/z
    

    which eventually will result in a syntax error with the example template, but demonstrates the point redefining the name and the value list separators.

    Configuration

    The generator is implemented the configuration tools supported by the Geci framework and all the parameters are configurable. You can redefine the regular expressions that match the template start, end, skip and so on lines in the unit test where the generator object is created, in the annotation of the class or in the editor-fold parameters.

    Takeaway

    The iterate generator is an extremely easy to use generator to create code that is repetitive. This is also the major danger: you should be strong enough to find a better solution and use it only when it is the best solution.

    Supporting Java 8

    Although Java has version 13 released as for now, there are a lot of production installations running with Java 8. As a professional, I develop Java 8 code many times even these days and I have to be happy that this is not Java 6. On the other hand as an open-source developer, I have my liberty to develop my Java code using Java 11, 12 or even 13 if that pleases me. And it does.

    On the other hand, though, I want my code to be used. Developing a tool like License3j or Java::Geci, which are kind of libraries releasing Java 11 compatible byte code cuts off all the Java 8 based applications that may use these libraries.

    I want the libraries to be available from Java 8.

    One solution is to keep two branches parallel in the Git repo and have a Java 11+ and a Java 8 version of the code. This is what I have done for Java::Geci 1.2.0 release. It is cumbersome, error-prone and it is a lot of work. I had this code only because my son, who is also a Java developer starting his career volunteered.

    (No, I did not pressure him. He speaks and writes better English than I do, and he regularly reviews these articles fixing my broken languages. If he has different opinion about the pressure, he is free to insert here any note till the closing parenthesis, I will not delete or modify that. NOTE: )

    Anything above between the NOTE: and ) is his opinion.

    The other possibility is to use Jabel.

    In this article, I will write about how I used Jabel in the project Java::Geci. The documentation of Jabel is short but still complete and it really works like that for simpler projects. For example I really only had to add a few lines to the pom.xml in case of the Licenese3j project. For more complex projects that were developed over a year without thinking about any compromise for Java 8 compatibility, it is a bit more complex.

    About Jabel

    Jabel is an open-source project available from https://github.com/bsideup/jabel. If you have a Java 9+ project source you can configure Jabel to be part of the compilation process. It is an annotation processor that hooks into the compilation process and kind of tricks the compiler to accept the Java 9+ features as they were available for Java 8. The compiler will work and generate Java 8, Jabel does not interfere with the byte code generation, so this is as genuine as it can be out of the Java compiler fresh and warm. It only instructs the compiler not to freak out on Java 9+ features when compiling the code.

    The way it works and why it can work is well written on the project’s GitHub page. What I wrote above may not even be precise.

    Backport issues

    When creating Java code using Java 9+ features targeting a Java 8 JVM it is not only the byte code version that we should care about. The code executed using the Java 8 JVM will use the Java 8 version of the JDK and in case we happen to use some classes or methods that are not available there then the code will not run. Therefore we have two tasks:

    • Configure the build to use Jabel to produce Java 8 byte-code
    • eliminate the JDK calls that are not available in Java 8.

    Configure Build

    I will not describe here how to configure Jabel to be part of the build using Maven. It is documented on the site and is straightforward.

    In the case of Java::Geci I wanted something different. I wanted a Maven project that can be used to create Java 8 as well as Java 11 targets. I wanted this because I wanted Java::Geci to support JPMS just as before and also to create state-of-the-art byte code (class nesting instead of bridge methods for example) for those projects that run on Java 11 or later.

    As the first step, I created a profile named JVM8. Jabel is only configured to run only when this profile is active.

    This profile also sets the release as

    <release>8</release>
    

    so the very first time the compiler was freaking out when it saw the module-info.java files. Fortunately, I can exclude files in the POM file in the JVM8 profile. I also excluded javax0/geci/log/LoggerJDK9.java and I will talk about that later.

    I also tried to use Maven to automatically configure the version number to have the -JVM8 postfix if it runs with the JVM8 profile but it was not possible. Maven is a versatile tool and can do many things and in case of a simpler project doing that should be the approach. In the case of Java::Geci I could not do that because Java:Geci is a multi-module project.

    Multi-module projects refer to each other. At least the child module reference the parent module. The version of the child module may be different from the version of the parent module. It is kind of logical since their evolution and development are not necessarily tied together. However, usually, it is. In projects, like Java::Geci that has seven child modules and each child module has the very same version number as the parent the child modules can inherit all the parameters, dependencies, compiler options and so on, from the parent but the version. It cannot inherit the version because it does not know which parent version to inherit it from. It is a catch 22.

    The Java::Geci development goes around this problem using the Jamal preprocessor maintaining the eight pom.xml files. Whenever there is a change in the build configuration it has to be edited in one of the pom.xml.jam files or in one of the included *.jim files and then the command line mvn -f genpom.xml clean will regenerate all the new pom.xml files. This also saves some repetitive code as the preprocessed Jamal files are not so verbose as the corresponding XML files. The price for this is that the macros used have to be maintained.

    Java::Geci has a version.jim file that contains the version of the project as a macro. When targeting a Java 8 release then the version in this file has to be changed to x.y.z-JVM8 and the command mvn -f genpom.xml clean has to be executed. Unfortunately, this is a manual step that I may forget. I may also forget to remove the -JVM8 postfix after the Java 8 target was created.

    To mitigate the risk of this human error I developed a unit test that checks the version number is coherent with the compilation profile. It identified the compilation profile reading the /javax0/geci/compilation.properties file. This is a resource file in the project filtered by Maven and contains

    projectVersion=${project.version}
    profile=${profile}
    

    When the test runs the properties are replaced by the actual values as defined in the project. project.version is the project version. The property profile is defined in the two profiles (default and JVM8) to be the name of the profile.

    If the version and the profile do not match the test fails. Following the philosophy of Java::Geci, the test does not just order the programmer to fix the “bug” when the test itself can also fix the bug. It modifies the version.jim file so that it contains the correct version. It does not, however, run the pom file generating Jamal macros.

    As a result of this I will get release files with version x.y.z and also x.y.z-JVM8 after the second build with some manual editing work.

    Eliminate Java 8+ JDK calls

    Simple calls

    This is a simple task at first sight. You must not use methods that are not in Java 8 JDK. We could do anything using Java 8 so it is a task that certainly possible.

    For example every

    " ".repeat(tab)
    

    has to be eliminated. To do that I created a class JVM8Tools that contain static methods. For example:

    public static String space(int n){
        final StringBuilder sb = new StringBuilder(/*20 spaces*/"                    ");
        while( sb.length() < n){
            sb.append(sb);
        }
        return sb.substring(0,n).toString();
    }
    

    is defined there and using this method I can write

    space(tab)
    

    instead of the invocation of String::repeat method. This part was easy.

    Mimicking getNestHost

    What was a bit more difficult is to implement the getNestHost() method. There is no such thing in Java 8, but the selector expressions included in the Tools module of Java::Geci lets you use expressions, like

    Selector.compile("nestHost -> (!null & simpleName ~ /^Map/)").match(Map.Entry.class)
    

    to check that the class Entry is declared inside Map, which it trivially is. It makes sense to use this expression even in Java 8 environment someone chooses to do so and I did not want to perform amputation dropping this feature from Java::Geci. It had to be implemented.

    The implementation checks the actual run-time and in case the method is there in the JDK then it calls that via reflection. In other cases, it mimics the functionality using the name of the class and trying to find the $ character that separates the inner and the enclosing class name. This may lead to false results in the extremely rare case when there are multiple instances of the same class structures loaded using different class loaders. I think that a tool, like Java::Geci can live with it, it barely happens while executing unit tests.

    There is also a speed drawback calling the method Class#getNestHost reflectively. I decide to fix it if there will be real demand.

    Logging support

    The last issue was logging. Java 9 introduced a logging facade that is highly recommended to be used by the libraries. Logging is a long-standing problem in the Java environment. The problem is not that there is not any. Quite the opposite. There are too many. There is Apache Commons Logging, Log4j, Logback, the JDK built-in java util logging. A standalone application can select the logging framework it uses, but in case a library uses a different one then it is difficult if not impossible to funnel the different log messages into the same stream.

    Java 9 thus introduced a new facade that a library can use to send out its logs and the applications can channel the output through the facade to whatever logging framework they want. Java::Geci uses this facade and provides logging API for the generators through it. In case the JVM8 environment this is not possible. In that case Java::Geci channels the log messages into the standard Java logger. To do that there is a new interface LoggerJDK implemented by two classes LoggerJVM8 and LoggerJDK9. The source code for the latter is excluded from the compilation in case the target is Java 8.

    The actual logger tries to get the javax0.geci.log.LoggerJDK9#factory via reflection. If it is there, then it is possible to use the Java 9 logging. If it is not there then the logger falls back to with the factory to javax0.geci.log.LoggerJVM8#factory. That way only the logger factory is called via reflection, which happens only once for every logger. Logging itself is streamlined and uses the target logging without any reflection, thus without speed impediment.

    Takeaway

    It is possible to support Java 8 in most of the library project without unacceptable compromise. We can create two different binaries from the same source that support the two different versions in a way that the version supporting Java 9 and later does not “suffer” from the old byte code. There are certain compromises. You must avoid calling Java 9+ API and in case there is an absolute need, you have top provide a fall-back and you can provide a reflection-based run-time detection solution.

    Creating a Video Tutorial

    I usually write technical articles here. This article is an exception. I do not know if this is a checked exception or not though. I do not even know if this really is an exception or rather an error or just something throwable. (I am just fooling around with the different Java exception types only because I am such a fun guy and also because this is a Java blog, so it SHOULD, as defined in rfc2119, have some words about Java.)

    This article is about how I create video tutorials. I have created a few. Not many. The implication of the amount is that what I tell you here is not the ultima ratio. I am almost sure that in many things I am wrong and I am open to criticism. Just be polite: a few people actually read this blog, including the comments.

    I created screen video recording as product documentation when I was running my own company ten years ago. I also created some as a training for my current employer, EPAM, and also for this blog and for PACKT. (Yes, this part of the article is a commercial, please go and subscribe and learn Java 9 new features from me listening to Java 9 New Features Deep Dive [Video].)

    Length

    The length of a video should be 5 to 10 minutes. The shorter the better. I was worried at first about not being able to fill these time frames. But it is easy. I usually struggle with the opposite. Sometimes I can not make the video as short as I would like to.

    Presentation

    Many times I create a presentation to highlight what I will talk about during the demonstration. This is important. These visuals help the audience get the content and understand what they can expect in the coming five or ten minutes. In other cases, the presentation itself is the main attraction. I usually use Microsoft PowerPoint simply because that is what I have the most experience with and it is available both on Windows and OSX.

    Screen Recording

    I use OSX and iShowU Instant. I record video in HD format these days and I also use an external monitor attached to my mbp. The recording control is on the built-in display of the mbp, which is a bit higher resolution than HD and the recorded scene is running on the external screen.

    I record applications maximized and if possible set to full screen. There is no reason to show the little “minimize, maximize, close” icons or the application frame. This is equally true on OSX, Linux or Windows.

    When you do something on the screen do not explain it while doing it. Explain it before and then do it. The reason for that is that this way the keyboard and mouse noise is separated from the talk and can be muted. Also, when you type silently you have the option later while editing the video to speed up the typing. The audience gets bored seeing how the typed letters come up one after the other. You can simply speed it up for a longer typing ten times even. They will see that this is sped up, but that is not a problem unless you want to demonstrate the speed of something.

    Voice Recording

    I live in a little, peaceful Swiss dorf (village). The road is near and the airplanes landing to Kloten (ZRH) just fly above the house, so the voice recording environment is not ideal, but around 10pm it is acceptable. In my former (Budapest) location, I could not record without noise. So the first thing is that you need a very quiet environment. Perhaps this is the most costly investment, but it also serves other purposes: it boosts your sleep, irons your nerves. Peace is invaluable, world peace… you know.

    When you consider the noise, do not only rely on your ears. I have a neighbor who is a professional drum player. Switzerland has strict noise regulation and these guys living here mean it: he is using some special drum set that suppresses the sound a lot. I am 52 and it means my hearing started to slowly decay. I would not have noticed that he is playing the drum sometimes till 11pm (which is strictly illegal, you can do the noisy activity until something like 8pm) unless I started recording. The microphone was recording it and I could hear it in the headset attached to the mic.

    I also realized through the headset that the table and the chair is a huge source of the noise. PACKT supports content creators (at least they supported me) with some PDFs that give some very practical technical advice and the chair was mentioned there. Table was not. Do not lean on the table when recording. Better yet, do not even touch it.

    The second important thing is the microphone. I tried to use the built-in mic of my MacBook Pro, which is exceptionally good for things like Skype, ad-hoc recording, recording a meeting, but not sufficient for tutorial recording. I bought an external microphone for 28CHF but it was not good enough. It was noisy. The one that I finally found is sufficient is a Zoom H2n recorder that also works as a USB microphone.

    It stands on my desk on a tripod. I usually put a pillow between the mic and the notebook, so the noise of the vent is dumped and I also moved the external HDD under the table. The pillow thing was coming from one of the PACKT materials and it is a great idea: it works and it is simple. The HDD now stands on the floor on a cork wood base (originally it was some IKEA cooking thing) which is put on top of a thick, folded (multiple times) cloth. Even though the noise of it is almost inaudible I disconnect it when I do the recording. That also prevents a backup firing off while recording eating the CPU off from the screen recorder, which itself is not a CPU hog to my surprise, but that is a different story. Here we talk about the noise (sic) recording. Btw: while recording also disable the network, unless you want to demonstrate something that needs it. You do not want to record notification popups.

    While I talk I attach a BOSE Q25 headset directly to the mic and through that I hear my own voice amplified. Because you hear your own voice from inside through your bones when you talk it sounds totally different when you listen to the recording. With the headset, the voice leaving my mouth is amplified and with active noise cancellation, I hear myself more from outside only through the microphone. It helps me to articulate better and also to recognize when my tongue twists.

    Talking

    I had to realize that I have to talk slow. I mean really slow. And as far as I know, most people who record voice run in the same shoe. When you record something, slow down your talk and when you feel that this is ridiculously slow then it probably is just okay.

    When your tongue twists or you just realize that you made a mistake in a sentence: do not correct the part like you would do in a live presentation. Stop. Take a breath. Think. Wait 5 seconds or more. Take your time and restart from the start of the last, erroneous sentence. The 5 seconds helps you to think about where to restart from, but this is also something easy to notice on the waveform when the recorded video is edited. If there is a pause in the voice it probably is something to cut off. I also hit the table with my palm, which makes a noise overloading the microphone and is a clearly visible peak on the waveform. You can also clap your hands or use a whistle. May seem ridiculous first.

    Recording face

    You may want to record your face while you talk about some slides. This is good for the audience, it makes your presentation more personal. I use an external webcam for this purpose. Although iShowU Instant can put the video input on the recorded screen as a picture in the picture, I decided to record the video input separately. On OSX I can record simultaneous screen using iShowU and the video input using PhotoBooth. That way both inputs will have the same audio recorded from the same microphone. This helps to put the two videos in sync when editing and then one of the audio (presumably the one from the presentation, as it is the one less sensitive to slipped audio) can be deleted.

    This way it is also possible to put the PIP face at different locations although I do not recommend that you move it a lot around. But it can many times be removed from the screen. For example, if you record slides as well as demo code then you can show your talking head on the slides and hide it when showing code demo on the screen.

    When you talk you have to face the camera. It is difficult because you want to talk about a slide that is not in the camera. It is on a screen that is just at the side of the camera. The bad news is that the audience will see that you are not looking into their eyes (which is the camera). You HAVE TO look into the camera.

    I was told to look at the slide, read it and then look into the camera and say the text again and then cut it off during the editing phase. It did not work. What worked was that I created a teller machine from a cardboard box, picture glass, and black paint. I also bought for something like 5$ a teller application that runs on my iPad and is reflected from the glass, which is set 45 degrees in front of the webcam. It all stands on a tripod on the table.

    Video Background

    I use a curtain behind my chair to have an ambient background. There is nothing wrong with a room in the background, but it may cause some problems.

    A clock on the wall will show that you recorded the video in several steps. It will jump back and forth and it is distracting for the audience. It is also bad when some background items, chairs, tables, etc. jump between different cuts of the video.

    Video Editing

    To edit the video I use iMovie. This comes free with the operating system on a mac and has enough functionalities to edit a technical video. Sometimes I feel I lack some features, which are available in professional video editing software products but later I realize that I do not need them.

    I value the Kern Burns cropping functionality very much. This was originally invented to show still pictures in a dynamic, moving way in a movie. When doing screen capture I can use this functionality to move the focus to the area of the screen, (usually showing the IDE when programming Java) that is important from the demo point of view.

    Takeaway

    There are many ways of doing tutorial videos, and I cannot tell what will fit your personality, topic, and audience. I wrote down my experience and I hope you can find something useful in it for you.

    A New Era for Determining Equivalence in Java?

    A few month ago I read a blog post of the title “A New Era for Determining Equivalence in Java?” and it was somehow very much in line with what I developed that time in my current liebling side project Java::Geci. I recommend that you pause reding here and read the original article and then return here, even knowing that telling that a sizable percentage of the readers will not come back. The article is about how to implement equals() and hashCode() properly in Java and some food for thoughts about how it should be or rather how it should have been. In this article, I will details these for those who do not read the original articles and I also add my thoughts. Partly how using Java::Geci addresses the problems and towards the end of my article how recursive data structures should be handled in equals() and in hashCode(). (Note that the very day I was reading the article I was also polishing the mapper generator to handle recursive data structures. It was very much resonating with the problems I was actually fixing.)

    If you came back or even did not go away reading the original article and even the referenced JDK letter of Liam Miller-Cushon titled “Equivalence” here you can have a short summary from my point of view of the most important statements / learning from those articles:

    • Generating equals() and hashCode() is cumbersome manually.
    • There is support in the JDK since Java 7, but still the code for the methods is there and has to be maintained.
    • IDEs can generate code for these methods, but regenerating them is still not an automated process and executing the regeneration manually is a human-error prone maintenance process. (a.k.a. you forget to run the generator)

    The JDK letter from Liam Miller-Cushon titled “Equivalence” lists the tipical errors in the implementation of equals() and hashCode(). It is worth reiterating these in a bit more details. (Some text is quoted verbatim.)

    • “Overriding Object.equals(), but not hashCode(). (The contract for Object.hashCode states that if two objects are equal, then calling the hashCode() method on each of the two objects must produce the same result. Implementing equals() but not hashCode() makes that unlikely to be the case.)” This is a rookie mistake and you may say that you will never commit that. Yes, if you are a senior as a programmer but not yet a senior in your mental capabilities e.g.: forgetting where your dental prostheses are then you will never forget to create hashCode() whenever you create equals(). Note, however, that this is a very short and temporal period in life. Numerous juniors also form the codebase and the lacking hashCode() may always lurk in the deep dark corners of the haystack of the Java code and we have to use all economically viable measures to avoid the non-existence of them.
    • “Equals implementations that unconditionally recurse.” This is a common mistake and even seniors many times ignore this possible error. This is hardly ever a problem because the data structures we use are usually not recursive. When they are recursive the careless recursive implementation of the equals() or hashCode() methods may result in an infinite loop, stack overflow, and other inconvenient things. I will talk about this topic towards the end of the article.
    • “Comparing mismatched pairs of fields or getters, e.g. a == that.a && b == that.a. This is a topical typing error and it remains unnoticed very easily like topical -> typical.
    • Equals implementations that throw a NullPointerException when given a null argument. (They should return false instead.)
    • Equals implementations that throw a ClassCastException when given an argument of the wrong type. (They should return false instead.)
    • Implementing equals() by delegating to hashCode(). (Hashes collide frequently, so this will lead to false positives.)
    • Considering state in hashCode() that is not tested in the corresponding equals() method. (Objects that are equal must have the same hashCode().)
    • equals() and hashCode() implementations that use reference equality or hashCode() for array members. (They likely intended value equality and hashCode().)
    • Other bugs (which are out of scope for the proposal): usage errors like comparing two statically different types, or non-local errors with definitions (e.g. overriding equals and changing semantics, breaking substitutability)

    What can we do to avoid these errors? One possibility is to enhance the language, as the mentioned proposal suggests so that the methods hashCode() and equals() can be described in a declarative way and the actual implementation, which is routine and cumbersome is done by the compiler. This is a bright future, but we have to wait for it. Java is not famous for incorporating ideas promptly. When something is implemented it is maintained for eternity in a backward-compatible manner. Therefore the choice is to implement it fast, possibly in the wrong way and live with it forever. Or wait till the industry is absolutely sure how it has to be implemented in the language and then and only that time implement it. Java is following the second way of development.

    This is a shortage in the language that comes from language evolution as I described in the article Your Code is Redundant…. A temporal shortage that will be fixed later but as for now, we have to handle this shortage.

    One answer to such shortage is code generation and that is where Java::Geci comes into the picture.

    Java::Geci is a code generation framework that is very well fitted to create code generators that help reduce code redundancy for domain-specific problems. The code generators run during unit test execution time, which may seem a bit later, as the code was already compiled. This is, however, fixed with the working that the code generating “test” fails if it generated any code and executing the compilation and the tests the second time will not fail anymore.

    Side note: This way of working may also be very familiar to any software developer: let’s run it again, it may work!

    In the case of programming language evolution shortages Java::Geci is just as good, from the technical point of view. There is no technical difference between code generation for domain-specific reasons and code generation for language evolution shortage reason. In the case of language evolution issues, however, it is likely that you will find other code generation tools that also solve the issue. To generate equals() and hashCode() you can use the integrated development environment. There can be nothing simpler than selecting a menu from the IDE and click: “generate equals and hashCode”.

    This solves all but one of the above problems, assuming that the generated code is well-behaving. That only one problem is that whenever the code is updated it will not run the code generator again to update the generated code. This is something that IDEs can hardly compete with Java::Geci. It is more steps to set up the Java::Geci framework than just clicking a few menu items. You need the test dependency, you have to create a unit test method and you have to annotate the class that needs the generator, or as an alternative, you have to insert an editor-fold block into the code that will contain the generated code. However, after that, you can forget the generator and you do not need to worry about any of the developers in your team forgetting to regenerate the equals() or hashCode() method.

    Takeaway

    • Having the proper equals() and hashCode() methods for a class is not as simple as it seems. Writing them manually is hardly ever the best approach.

    • Use come tool that generates them and ensure that the generated code and the code generation does not exhibit any of the above common mistakes.

    • If you just need it Q&D then use the IDE menu and generate the methods. On the other hand, if you have a larger codebase, with many developers working on it and it is possible that the code generation may need re-execution then use a tool that automates the execution of the code generation. Example: Java::Geci.

    • Use the newest possibe version of the tools, like Java so that you do not lag behind available technology.

    Java Record

    The https://openjdk.java.net/jeps/359 outlines a new Java feature that may/will be implemented in some future versions of Java. The JEP suggests having a new type of “class”: record. The sample in the JEP reads as follows:

    record Range(int lo, int hi) {
      public Range {
        if (lo > hi)  /* referring here to the implicit constructor parameters */
          throw new IllegalArgumentException(String.format("(%d,%d)", lo, hi));
      }
    }
    

    Essentially a record will be a class that intends to have only final fields that are set in the constructor. The JEP as of today also allows any other members that a class has, but essentially a record is a record, pure data and perhaps no functionality at its core. The description of a record is short and to the point and eliminates a lot of boilerplate that we would need to encode such a class in Java 13 or less or whichever version record will be implemented. The above code using conventional Java will look like the following:

    public class Range {
    
        final int lo;
        final int hi;
    
        public Range(int lo, int hi) {
            if (lo > hi)  /* referring here to the implicit constructor parameters */
                throw new IllegalArgumentException(String.format("(%d,%d)", lo, hi));
            this.lo = lo;
            this.hi = hi;
        }
    }
    

    Considering my Java::Geci code generation project this was something that was screaming for a code generator to bridge the gap between today and the day when the new feature will be available on all production platforms.

    Thus I started to think about how to develop this generator and I faced a few issues. The Java::Geci framework can only convert a compilable project to another compilable project. It cannot work like some other code generators that convert an incomplete source code, which cannot be compiled without the modifications of the code generator, to a complete version. This is because Java::Geci works during the test phase. To get to the test phase the code has to compile first. This is a well-known trade-off and was a design decision. In most of the cases when Java::Geci is useful this is something easy to cope with. On the other hand, we gain the advantage that the generators do not need configuration management like reading and interpreting property or XML files. They only provide an API and the code invoking them from a test configure the generators through it. The most advantage is that you can even provide call-backs in forms of method references, lambdas or object instances that are invoked by the generators so that these generators can have a totally open structure in some aspects of their working.

    Why is this important in this case? The record generation is fairly simple and does not need any complex configuration, as a matter of fact, it does not need any configuration at all. On the other hand, the compilable -&gt; compilable restrictions are affecting it. If you start to create a record using, say Java 8 and Java::Geci then your manual code will look something like this:

    @Geci("record")
    public class Range {
    
        final int lo;
        final int hi;
    }
    

    This does not compile, because by the time of the first compilation before the code generation starts the default constructor does not initialize the fields. Therefore the fields cannot be final:

    @Geci("record")
    public class Range {
    
        int lo;
        int hi;
    }
    

    Running the generator we will get

    package javax0.geci.tests.record;
    
    import javax0.geci.annotations.Geci;
    
    @Geci("record")
    public final class Range {
        final  int  lo;
        final  int  hi;
    
        //<editor-fold id="record">
        public Range(final int lo, final int hi) {
            this.lo = lo;
            this.hi = hi;
        }
    
        public int getLo() {
            return lo;
        }
    
        public int getHi() {
            return hi;
        }
    
        @Override
        public int hashCode() {
            return java.util.Objects.hash(lo, hi);
        }
    
        @Override
        public boolean equals(Object o) {
            if (this == o) return true;
            if (o == null || getClass() != o.getClass()) return false;
            Range that = (Range) o;
            return java.util.Objects.equals(that.lo, lo) && java.util.Objects.equals(that.hi, hi);
        }
        //</editor-fold>
    }
    

    what this generator actually does is that

    • it generates the constructor
    • converts the class and the fields to final as it is a requirement by the JEP
    • generates the getters for the fields
    • generates the equals() and hashCode() methods for the class

    If the class has a void method that has the same (though case insensitive) name as the class, for example:

        public void Range(double hi, long lo) {
            if (lo > hi)  /* referring here to the implicit constructor parameters */
                throw new IllegalArgumentException(String.format("(%d,%d)", lo, hi));
        }
    

    then the generator will

    • invoke that method from the generated constructor,
    • modify the argument list of the method to match the current list of fields.
        public void Range(final int lo, final int hi) {
            if (lo > hi)  /* referring here to the implicit constructor parameters */
                throw new IllegalArgumentException(String.format("(%d,%d)", lo, hi));
        }
    
        //<editor-fold id="record">
        public Range(final int lo, final int hi) {
            Range(lo, hi);
            this.lo = lo;
            this.hi = hi;
        }
    

    Note that this generation approach tries to behave the possible closest to the actual record as proposed in the JEP and generates code that can be converted to the new syntax as soon as it is available. This is the reason why the validator method has to have the same name as the class. When converting to a real record all that has to be done is to remove the void keyword converting the method to be a constructor, remove the argument list as it will be implicit as defined in the JEP and remove all the generated code between the editor folds (also automatically generated when the generator was executed first).

    The modification of the manually entered code is a new feature of Java::Geci that was triggered by the need of the Record generator and was developed to overcome the shortcomings of the compilable -&gt; compilable restriction. How a generator can use this feature that will be available in the next 1.3.0 release of Java::Geci will be detailed in a subsequent article.

    Takeaway

    The takeaway of this article is that you can use Java records with Java 8, 9, … even before it becomes available.

    Handling repeated code automatically

    In this article I will describe how you can use Java::Geci generator Repeated to overcome the Java language shortage that generics cannot be primitive. The example is a suggested extension of the Apache Commons Lang library.

    Introduction

    When you copy-paste code you do something wrong. At least that is the perception. You have to create your code structure more generalized so that you can use different parameters instead of similar code many times.

    This is not always the case. Sometimes you have to repeat some code because the language you use does not (yet) support the functionality that would be required for the problem.

    This is too abstract. Let’s have a look at a specific example and how we can manage it using the Repeated source generator, which runs inside the Java::Geci framework.

    The problem

    The class org.apache.commons.lang3.Functions in the Apache Commons Lang library defines an inner interface FailableFunction. This is a generic interface defined as

        @FunctionalInterface
        public interface FailableFunction<I, O, T extends Throwable> {
            /**
             * Apply the function.
             * @param pInput the input for the function
             * @return the result of the function
             * @throws T if the function fails
             */
            O apply(I pInput) throws T;
        }
    

    This is essentially the same as Function<I,O>, which converts an I to an O but since the interface is failable, it can also throw an exception of type T.

    The new need is to have

        public interface Failable<I>Function<O, T extends Throwable> 
    

    itnerfaces for each <I> primitive values. The problem is that the generics cannot be primitive (yet) in Java, and thus we should separate interfaces for each primitive types, as

        @FunctionalInterface
        public interface FailableCharFunction<O, T extends Throwable> {
            O apply(char pInput) throws T;
        }
        @FunctionalInterface
        public interface FailableByteFunction<O, T extends Throwable> {
            O apply(byte pInput) throws T;
        }
        @FunctionalInterface
        public interface FailableShortFunction<O, T extends Throwable> {
            O apply(short pInput) throws T;
        }
        @FunctionalInterface
        public interface FailableIntFunction<O, T extends Throwable> {
            O apply(int pInput) throws T;
        }
    ... and so on ...
    

    This is a lot of very similar methods that could easily be described by a template and then been generated by some code generation tool.

    Template handling using Java::Geci

    The Java::Geci framework comes with many off-the-shelf generators. One of them is the powerful Repeated generator, which is exactly for this purpose. If there is a code that has to be repeated with possible parameters then you can define a template, the values and Repeated will generate the code resolving the template parameters.

    Adding dependency to the POM

    The first thing we have to do is to add the Java::Geci dependencies to the pom.xml file. Since Apache Commons Language is still Java 8 based we have to use the Java 8 backport of Java::Geci 1.2.0:

        <dependency>
          <groupId>com.javax1.geci</groupId>
          <artifactId>javageci-core</artifactId>
          <version>1.2.0</version>
          <scope>test</scope>
        </dependency>
    

    Note that the scope of the dependency is test. The generator Repeated can conveniently be used without any Geci annotations that remain in the byte code and thus are compile-time dependencies. As a matter of fact, all of the generators can be used without annotations thus without any compile dependencies that would be an extra dependency for the production. In the case of Repeated this is even easy to do.

    Unit test to run the generator

    The second thing we have to do is to create a unit test that will execute the generator. Java::Geci generators run during the unit test phase, so they can access the already compiled code using reflection as well as the actual source code. In case there is any code generated that is different from what was already there in the source file the test will fail and the build process should be executed again. Since generators are (should be) idempotent the test should not fail the second time.

    As I experience, this workflow has an effect on the developer behavior, unfortunately. Run the test/ fails, run again! It is a bad cycle. Sometimes I happen to catch myself re-executing the unit tests when it was not a code generator that failed. However, this is how Java::Geci works.

    There are articles about the Java::Geci workflow

    so I will not repeat here the overall architecture and how its workflow goes.

    The unit tests will be the following:

        @Test
        void generatePrimitiveFailables() throws Exception {
            final Geci geci = new Geci();
            Assertions.assertFalse(geci.source(Source.maven().mainSource())
                    .only("Functions")
                    .register(Repeated.builder()
                        .values("char,byte,short,int,long,float,double,boolean")
                        .selector("repeated")
                        .define((ctx, s) -> ctx.segment().param("Value", CaseTools.ucase(s)))
                        .build())
                    .generate(),
                geci.failed());
        }
    

    The calls source(), register() and only() configure the framework. This configuration tells the framework to use the source files that are in the main Java src directory of the project and to use only the file names "Functions". The call to register() registers the Repeated generator instance right before we call generate() that starts the code generation.

    The generator instance itself is created using the built-in builder that lets us configure the generator. In this case, the call to values() defines the comma-separated list of values with which we want to repeat the template (defined later in the code in a comment). The call to selector() defines the identifier for this code repeated code. A single source file may contain several templates. Each template can be processed with a different list of values and the result will be inserted into different output segments into the source file. In this case there is only one such code generation template, still, it has to be identified with a name and this name has also to be used in the editor-fold section which is the placeholder for the generated code.

    The actual use of the name of the generator has two effects. One is that it identifies the editor fold segment and the template. The other one is that the framework will see the editor-fold segment with this identifier and it will recognize that this source file needs the attention of this generator. The other possibility would be to add the @Repeated or @Geci("repeated") annotation to the class.

    If the identifier were something else and not repeated then the source code would not be touched by the generator Repeated or we would need another segment identified as repeated, which would not actually be used other than trigger the generator.

    The call to define() defines a BiConsumer that gets a context reference and an actual value. In this case, the BiConsumer calculates the capitalized value and puts it into the actual segment parameter set associated with the name Value. The actual value is associated with the name value by default and the BiConsumer passed to the method define() can define and register other parameters. In this case, it will add new values as

    value       Value
    
    char    --> Char
    byte    --> Byte
    short   --> Short
    int     --> Int
    long    --> Long
    float   --> Float
    double  --> Double
    boolean --> Boolean
    

    Source Code

    The third thing is to prepare the template and the output segment in the source file.

    The output segment preparation is extremely simple. It is only an editor fold:

        //<editor-fold id="repeated">
        //</editor-fold>
    

    The generated code will automatically be inserted between the two lines and the editors (Eclipse, IntelliJ or NetBeans) will allow you to close the fold. You do not want to edit this code: it is generated.

    The template will look like the following:

        /* TEMPLATE repeated
        @FunctionalInterface
        public interface Failable{{Value}}Function<O, T extends Throwable> {
            O apply({{value}} pInput) throws T;
        }
        */
    

    The code generator finds the start of the template looking for lines that match the /* TEMPLATE name format and collect the consecutive lines till the end of the comment.

    The template uses the mustache template placeholder format, namely the name of the values enclosed between double braces. Double braces are rare in Java.

    When we run the unit test it will generate the code that I already listed at the start of the article. (And after that it will fail of course: source code was modified, compile it again.)

    Summary and Takeaway

    The most important takeaway and WARNING: source code generation is a tool that aims to amend shortages of the programming language. Do not use code generations to amend a shortage that is not of the language but rather your experience, skill or knowledge about the language. The easy way to code generation is not an excuse to generate unnecessarily redundant code.

    Another takeaway is that it is extremely easy to use this generator in Java. The functionality is comparable to the C preprocessor that Java does not have and for good. Use it when it is needed. Even though the setup of the dependencies and the unit test may be a small overhead later the maintainability usually pays this cost back.

    Your code is redundant, live with it!

    This article is about necessary and unavoidable code redundancy and discusses a model of code redundancy that helps to understand why source code generators do what they do, why they are needed at all.

    Intro

    The code you write in Java, or for that matter in any other language, is redundant. Not by the definition that says (per Wikipedia page https://en.wikipedia.org/wiki/Redundant_code):

    In computer programming, redundant code is source code or compiled code in a computer program that is unnecessary, such as…

    Your code may also be redundant this way, but that is a different kind of story than I want to talk here and now. If it is, then fix it, and improve your coding skills. But this probably is not the case because you are a good programmer. The redundancy that is certainly in your code is not necessarily unnecessary. There are different sources of redundancy and some redundancies are necessary, others are unnecessary but unavoidable.

    The actual definition of redundancy we need, in this case, is more like the information theory definition of redundancy (per the Wikipedia page https://en.wikipedia.org/wiki/Redundancy_(information_theory))

    In Information theory, redundancy measures the fractional difference between the entropy H(X) of an ensemble X, and its maximum possible value log(|A_X|)

    UPPPS… DO NOT STOP READING!!!

    This is a very precise, but highly unusable definition for us. Luckily the page continues and says:

    Informally, it is the amount of wasted “space” used to transmit certain data. Data compression is a way to reduce or eliminate unwanted redundancy.

    In other words, some information encoded in some form is redundant if it can be compressed.

    For example, downloading and zipping the text of the classical English novel Moby Dick will shrink its size down to 40% of the original text. Doing the same with the source code of Apache Commons Lang we get 20%. It is definitely NOT because of this “code in a computer program that is unnecessary”. This is some other “necessary” redundancy. English and other languages are redundant, programming languages are redundant and this is the way it is.

    If we analyze this kind of redundancy we can see that there are six levels of redundancy. What I will write here about the six layers is not well-known or well-established theory. Feel free to challenge it.

    This model and categorization are useful to establish a way of thinking about code generation when to generate code, why to generate code. After all, I came up with this model when I was thinking about the Java::Geci framework and I was thinking about why I invested a year of hobby time into this when there are so many other code generation tools. This redundancy model kind of gives the correct reason that I was only feeling before.

    Levels of Redundancy

    Then the next question is if these (English and programming language) are the only reasons for redundancy. The answer is that we can identify six different levels of redundancy including those already mentioned.

    0 Natural

    This is the redundancy of the English language or just any other natural language. This redundancy is natural and we got used to it. The redundancy evolved with the language and it was needed to help the understanding a noisy environment. We do not want to eliminate this redundancy, because if we did we may end up reading some binary code. For most of us, this is not really appealing. This is how human and programmer brain works.

    1 Language

    The programming language is also redundant. It is even more redundant than the natural language it is built on. The extra redundancy is because the number of keywords is very limited. That makes the compression ration from 60% percent up to 80% in the case of Java. Other languages, like Perl, are denser and alas they are less readable. However, this is also a redundancy that we do not want to fight. Decreasing the redundancy coming from the programming language redundancy certainly would decrease readability and thus maintainability.

    2 Structural

    There is another source of redundancy that is already independent of the language. This is the code structure redundancy. For example when we have a method that has one argument then the code fragments that call this method should also use one argument. If the method changes for more arguments then all the places that call the method also have to change. This is a redundancy that comes from the program structure and this is not only something that we do not want to avoid, but it is also not possible to avoid without losing information and that way code structure.

    3 Domain induced

    We talk about domain induced redundancy when the business domain can be described in a clear and concise manner but the programming language does not support such a description. A good example can be a compiler. This example is in a technical domain that most programmers are familiar with. A context-free syntax grammar can be written in a clear and nice form using BNF format. If we create the parser in Java it certainly will be longer. Since the BNF form and the Java code mean the same and the Java code is significantly longer we can be sure that the Java code is redundant from the information theory point of view. That is the reason why we have tools for this example domain, like ANTLR, Yacc and Lex and a few other tools.

    Another example is the Fluent API. A fluent API can be programmed implementing several interfaces that guide the programmer through the possible sequences of chained method calls. It is a long and hard to maintain way to code a fluent API. The same time a fluent API grammar can be neatly described with a regular expression because fluent APIs are described by finite-state grammars. The regular expression listing the methods describing alternatives, sequences, optional calls, and repetitions is more readable and shorter and less redundant than the Java implementation of the same. That is the reason why we have tools like Java::Geci Fluent API generators that convert a regular expression of method calls to fluent API implementation.

    This is an area where decreasing the redundancy can be desirable and may result in easier to maintain and more readable code.

    4 Language evolution

    Language evolution redundancy is similar to the domain induced redundancy but it is independent of the actual programming domain. The source of this redundancy is a weakness of the programming language. For example, Java does not automatically provide getters and setters for fields. If you look at C# or Swift, they do. If we need them in Java, we have to write the code for it. It is boilerplate code and it is a weakness in the language. Also, in Java, there is no declarative way to define equals() and hashCode() methods. There may be a later version of Java that will provide something for that issue. Looking at past versions of Java it was certainly more redundant to create an anonymous class than writing a lambda expression. Java evolved and this was introduced into the language.

    Language evolution is always a sensitive issue. Some languages run fast and introduce new features. Other languages, like Java, are more relaxed or, we can say conservative. As Brian Goetz wrote in response to a tweet that was urging new features:

    “It depends. Would you rather get the wrong feature sooner, but have to live with it forever?”

    @BrianGoetz Replying to @joonaslehtinen and @java 10:43 PM · Sep 16, 2019

    The major difference between domain induced redundancy and language evolution caused redundancy is that while it is impossible to address all programming domains in a general-purpose programming language, the language evolution will certainly eliminate the redundancy enforced by language shortages. While the language evolves we have code generators in the IDEs and in programs like Lombok that address these issues.

    5 Programmer induced

    This kind of redundancy correlates with the classical meaning of code redundancy. This is when the programmer cannot generate good enough code and there are unnecessary and excessive code structures or even copy-paste code in the program. The typical example is the before mentioned “Legend of the sub-par developer”. In this case, code generation can be a compromise but it is usually a bad choice. On a high level, from the project manager point of view, it may be okay. They care about the cost of the developers and they may decide to hire only cheaper developers. On the programmer level, on the other hand, this is not acceptable. If you have the choice to generate code or write better code you have to choose the latter. You must learn and develop yourself so that you can develop better code.

    Outro

    … or takeaway.

    When I first started to write about the Java::Geci framework, somebody commented “why another code generation tool”? And the question is certainly valid. There are many tools like that as mentioned in the article.

    However, if we look at the code redundancy categorization then what we can see is that Java::Geci can be used to manage the Domain Induced redundancy and perhaps the Language Evolution caused redundancy. In the case of the latter, there are many concurrent programs, and Java::Geci cannot compete, for example with the ease of use of the IDE built-in code generation.

    There are many generators that address some specific domains and manage the extra redundancy using code generation. Java::Geci is the only one to my knowledge that provides a general framework that makes the domain-specific code generator creation simple.

    To recognize that the real use case is for domain-specific generators the above redundancy model helps a lot.