Yupiik Batch Introduction

Long running applications: applications which start and are not supposed to stop. It is the case of web application where the server is supposed to serve any request any time or daemon applications which are running all the time (for example the network manager on your computer).

Task oriented applications: it concerns applications starting and stopping once their task is done. It concerns the simplest command line interface (date command for example which just prints the current date) to the longest batch which can take hours but once it completed its task, it exits.

Be injectable in the application (i.e. be configurable from the environment and/or the system properties of the java command),

Be documentable since the first users of the configuration will be the ops guys,

(optionally) Be easy to integrate. One example of that is to easily generate a form from your configuration to re-run a batch for example if needed.

define your configuation using simple-configuration which consists in marking a Java class fields with @Param:

public class MyConf {
    @Param(description = "Driver to use", required = true)
    private String driver;

    @Param(description = "JDBC URL to use", required = true)
    private String url;

    @Param(description = "Database username.")
    private String username;

    @Param(description = "Database password.")
    private String password;
}

define a batch implementing Batch<YourConf>

public class MyBatch implements Batch<MyConf> {
    @Override
    public void accept(final MyConf configuration) {
        // ...
    }
}

implement the batch in accept method - where the configuration instance is considered initialized and injected. Here it is recommended to use the Batch DSL:

from()
    .map("step1", new MyStep1())
    .map("step2", new MyStep2())
    .run(runConfiguration);

launch the batch using Batch.run which will automatically populate a configuration instance from the environment and call accept method of your batch:

public static void main(final String... args) {
    Batch.run(MyBatch.class, args);
}

A simple deployment friendly configuration solution well integrated with the batch runtime,

A simple to deploy and wire UI to monitor your batches,

A simple and very flexible programming model Java friendly to write any batch application.

It integrates very well in Kubernetes through Job or CronJob and the configuration make it very smooth to use with a dedicated ConfigMap (highly recommended).

It is very simple to test since it is a plain standalone application - even if you can use Spring, CDI or Guice if you need an IoC, so JUnit 5 tests are trivial to write:

@Test
void test() {
    new MyBatch().accept(new MyConfig());
    assertItDidWhatWasNeeded();
}

For a complete E2E deployment you can integrate it very easily with Yupiik Logging which will enable you to have a JSON logging and therefore make the Kubernetes log aggregation way more efficient. The tip there is to extend the default JSON formatter to inject a batch identifier which will enable you to filter logs per execution too (and potentially wire it in the comment automatically).

It is easy to integrate withYupiik Minisite which enables to generate and deploy a static documentation on Github/Gitlab/FTP pages very easily.