Hop Maven PluginDocs

Menu

Usage

This page walks through a real Apache Hop project from scratch and shows the concrete goals you apply at each step: render diagrams, generate documentation, build a distribution and run it. The authoritative, per-goal reference (every parameter, its type, its default and its Maven property) is generated from plugin.xml and available in the xref:mojo/hop-maven-plugin.adoc[goal reference index].

Getting started: a minimal Hop project

Apache Hop pipelines and workflows are XML files, normally authored in the Hop GUI. A Hop project is just a folder - by convention pipelines (.hpl) and workflows (.hwf) live at the project root, the metadata in metadata/, and the project variables in project-config.json:

demo-hop/
  metadata/                <- database connections, run configurations
  project-config.json      <- config.variables
  my-pipeline.hpl          <- one pipeline (XML)

This layout matches the plugin defaults, so render works out of the box on any such project. The plugin only reads the XML files; it does not need the Hop GUI to render or run them.

TIP

the distribution/CLI goals (run, gui, encrypt, server) default their repository to ${project.basedir}/src/main/repositories and merge it onto the exploded distribution when it exists. If your Hop project home lives elsewhere, set <repository> explicitly; the merge is simply skipped when the folder does not exist.

Use case 1: render the diagrams in the build

Render every .hpl/.hwf to SVG (and optionally Mermaid) headlessly, keeping the source-relative path. This gives a reproducible, always-current picture of your flows on every commit or CI run.

From the shell, on a project with the standard layout: 

mvn io.yupiik.hop:hop-maven-plugin:<version>:render

Parameters are overridable with the -Dhop.render.<name> properties declared in the render reference . For example, to render only SVG into a dedicated folder:

mvn io.yupiik.hop:hop-maven-plugin:<version>:render \
  -Dhop.render.targetFolder=target/hop-diagrams \
  -Dhop.render.formats=SVG

To get it on every build, bind the goal in a phase - process-resources so the diagrams are available to later steps (for example to be copied into the generated documentation):

<build>
  <plugins>
    <plugin>
      <groupId>io.yupiik.hop</groupId>
      <artifactId>hop-maven-plugin</artifactId>
      <executions>
        <execution>
          <id>render-diagrams</id>
          <phase>process-resources</phase>
          <goals>
            <goal>render</goal>
          </goals>
          <configuration>
            <sourceFolder>${project.basedir}</sourceFolder>
            <metadataFolder>${project.basedir}/metadata</metadataFolder>
            <configFile>${project.basedir}/project-config.json</configFile>
            <targetFolder>${project.build.directory}/hop-diagrams</targetFolder>
            <formats>
              <format>SVG</format>
            </formats>
          </configuration>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

When your flows use transforms not shipped by the base plugin classpath, enable autoResolve and the goal fetches the matching hop-transform-*/hop-action-* modules with the Maven resolver:

<configuration>
  <autoResolve>true</autoResolve>
</configuration>

Use case 2: generate documentation for every flow

generate-doc reuses the same discovery as render and writes one page per flow plus an index (and optionally the diagram images and an llms.txt mirror), derived directly from the Hop files:

<execution>
  <id>generate-doc</id>
  <phase>process-resources</phase>
  <goals>
    <goal>generate-doc</goal>
  </goals>
  <configuration>
    <sourceFolder>${project.basedir}</sourceFolder>
    <outputDirectory>${project.build.directory}/generated-site/content/hop</outputDirectory>
    <imageFolder>${project.build.directory}/generated-site/images/diagrams</imageFolder>
    <includeImage>true</includeImage>
  </configuration>
</execution>
TIP

point outputDirectory at a src/main/minisite/content/... folder to feed the generated pages straight into a yupiik minisite, exactly like the reference pages you are reading now.

Enriching the generated pages from src/main/hop/documentation

The pages are derived from the Hop files, but the human prose (role/objective, launch snippet, notes) is yours. By convention generate-doc reads a per-artefact AsciiDoc file from enrichmentLocation (default src/main/hop/documentation/<flowName>.adoc); inline <enrichments> entries fall back or override it. The index metadata is carried by document attributes:

= T_SYNC_LOAD
:hop-index-role: Phase 2 : extraction partitionnee source.
:hop-index-order: 7

T_SYNC_LOAD est la phase 2 de l'archivage (la copie des donnees de la partition).

== Script de lancement

[source,bash]
.....
hop-run.sh -f utilities/oracle_sync/T_SYNC_LOAD.hpl -r local
.....

== Voir aussi

* link:../design.html[Design]

The body keeps the narrative order of the page: the prose before the first == becomes the overview, each == section and the launch script are rendered in the order they were written, prose that follows a heading stays under it, and nested === headings are preserved (nothing is re-sorted to the end). To place one of the generated sections (configuration, steps, flow, diagram) exactly where the thread needs it, write a reserved heading at that spot:

== :configuration

The reserved headings are :configuration, :steps, :flow and :diagram. Any generated section that is not anchored is appended after the doc thread in that canonical order. With an inline <enrichments> entry (no ordered file body) the fixed order Overview, Configuration, Steps, Flow, Diagram, Script, then the extra sections is used instead.

Use case 3: bundle a self-contained distribution

build takes a stock Hop client zip, enriches it and packages a zip you can ship. It resolves the version from versionUrl (defaults to the Apache downloads) and, from the baseDirectory (default src/main/hop/distribution), copies your files on top so they win over the upstream content:

<execution>
  <id>build-hop</id>
  <goals>
    <goal>build</goal>
  </goals>
  <configuration>
    <hopVersion>2.20.0</hopVersion>
    <baseDirectory>${project.basedir}/src/main/hop/distribution</baseDirectory>
    <drivers>
      <driver>com.oracle.database.jdbc:ojdbc11:23.5.0.24.07</driver>
    </drivers>
    <libs>
      <lib>org.jooq:jooq:3.19.0</lib>
    </libs>
    <transforms>
      <transform>com.acme:acme-custom-transform:1.0.0</transform>
    </transforms>
    <customizations>
      <deletions>
        <!-- strip stock content you do not want to ship (ant-style globs) -->
        <deletion>samples/**</deletion>
        <deletion>lib/core/hop-transform-textfile*.jar</deletion>
      </deletions>
      <additions>
        <!-- add a dependency at a relative path; a trailing '/' means a folder and appends the file name, otherwise it is the exact destination file path -->
        <addition>
          <gav>com.acme:acme-driver:jar:1.0.0</gav>
          <target>lib/jdbc/</target>
        </addition>
        <addition>
          <gav>com.acme:acme-config:2.0.0</gav>
          <target>config/acme.json</target>
        </addition>
      </additions>
    </customizations>
  </configuration>
</execution>

The zip is written to target/<finalName>.zip and attached to the build by default; override the classifier with classifier and disable packaging or attachment with packageDistribution/attach. A later goal in the same reactor can reference it as a distribution.

The optional customizations block runs last, after drivers/libs/javaagents/transforms/plugins and the baseDirectory merge: deletions remove files matching ant-style relative globs (a matched directory is removed recursively, a pattern matching nothing is ignored) and additions copy a resolved Maven dependency (as groupId:artifactId:type[:classifier]:version) at a relative target. Deletions run before additions, so an addition can replace a just-deleted path. Both are guarded: a pattern or target resolving outside the distribution root fails the build.

Use case 4: run a pipeline or workflow headlessly

run resolves a distribution (the one attached by build, or a groupId:artifactId:version[:type[:classifier]] coordinate), unpacks it, merges a repository folder on top and executes a pipeline or workflow via hop-run.sh/hop-run.bat:

<execution>
  <id>run-demo</id>
  <goals>
    <goal>run</goal>
  </goals>
  <configuration>
    <!-- the distribution built by the same reactor -->
    <distribution>${project.groupId}:${project.artifactId}:${project.version}:zip</distribution>
    <repository>${project.basedir}</repository>
    <pipeline>my-pipeline.hpl</pipeline>
    <runConfig>local</runConfig>
    <parameters>
      <pc.env>LOCAL</pc.env>
    </parameters>
  </configuration>
</execution>

A headless hop-run needs a run configuration Hop can find. With the default project home it is read from the project metadata, i.e. in the source repository metadata/pipeline-run-configuration/local.json (landing in config/projects/<name>/metadata/... after the merge); with an explicit repository merged at the distribution root, put it under config/metadata/pipeline-run-configuration/local.json (Hop reads config/metadata, not metadata/). Reference it with runConfig=local.

The repository defaults to ${project.basedir}/src/main/repositories (skipped when absent), so on the conventional layout mvn hop:run -Dhop.run.pipeline=my-pipeline.hpl just works. With the default home the merged content lands in config/projects/<name> of the exploded distribution (the canonical Hop project layout, where <name> is the name field of the source project-config.json or the repository folder name) and the project is registered in config/hop-config.json and activated with --project <name>. When the repository is the whole project root (like ${project.basedir} above), VCS folders (.git/.hg/.svn) and the Maven build directory are excluded from the merge.

If -f/--file (or -r/--runconfig, or --project) is passed in the args, it takes precedence: -f switches run to manual mode (no automatic -f, no automatic --project), -r wins over runConfig, and --project wins over the automatic project activation.

Use case 4b: open the project in the Hop GUI

gui resolves a distribution, merges the same project home and starts hop-gui.sh/hop-gui.bat in the background (waitFor=false by default, so Maven exits and you close the window to stop the GUI). On the conventional layout:

mvn hop:gui -Dhop.version=2.20.0

When the merged project home has a project-config.json, the project is registered in the exploded distribution config/hop-config.json (name from project-config.json name, else the repository folder name), made the defaultProject and activated with -j <name> (the GUI's project option; run/server use --project <name> instead), so your pipelines/actions are visible immediately. Everything Hop writes (config, audit, logs, UI state) stays in the workdir (target/hop/gui/); nothing is ever written back into the source project.

Use case 5: run the Hop server and stop it

server starts the Hop server in the background for interactive or long-running workloads; server-stop sends the authenticated /shutdown request:

<execution>
  <id>start-server</id>
  <goals>
    <goal>server</goal>
  </goals>
  <configuration>
    <distribution>${project.groupId}:${project.artifactId}:${project.version}:zip</distribution>
    <repository>${project.basedir}</repository>
  </configuration>
</execution>
<execution>
  <id>stop-server</id>
  <goals>
    <goal>server-stop</goal>
  </goals>
  <configuration>
    <stopPassword>your-server-password</stopPassword>
  </configuration>
</execution>

Prerequisites and pitfalls

These operational details matter when using the distribution/run/CLI goals; each goal's parameters are documented in its goal reference .

Passing arguments to the wrapped CLIs (encrypt, run, gui, server)

Extra launcher arguments must be a list of separate tokens via <args>. A single string -D...args="-hop supersecret" becomes one argv element and will not be split:

<args>
  <arg>-hop</arg>
  <arg>supersecret</arg>
</args>

For encrypt (a typical need when externalizing a configuration), the underlying hop-encrypt.sh expects hop-encrypt.sh <-hop|-server> <password> and prints Encrypted <hash> on success. Keep waitFor=true (the default) so the exit code and output are captured.

Resolving extra transforms/plugins (the "marketplace")

build resolves transform/plugin coordinates through the Maven resolver, so the project must declare the repository that hosts them. Hop artifacts live on the ASF releases repository:

<repositories>
  <repository>
    <id>apache-releases</id>
    <url>https://repository.apache.org/content/repositories/releases</url>
  </repository>
</repositories>

Each installed plugin is written under hop/plugins/<category>/<artifactId>/ (jar + lib/ deps and a META-INF/jandex.idx). Custom transforms usable by name in a pipeline are then registered by Hop at run time.

Javaagent injection

Javaagent coordinates in javaagents are downloaded into hop/lib and injected into the JVM launch of every hop-*.sh/.bat, in definition order. For example the OpenTelemetry agent:

<javaagents>
  <javaagent>io.opentelemetry.javaagent:opentelemetry-javaagent:2.10.0</javaagent>
</javaagents>

You can verify the agent is active from its startup banner in the launcher output, e.g. opentelemetry-javaagent - version: 2.10.0.

Next steps

See the Goals page for the full goal catalog, the xref:mojo/hop-maven-plugin.adoc[goal reference index] for every parameter, and Configuration for the render-specific variable, metadata and format wiring.

Navigation