Java plugins to catch up with Embulk v0.10 from v0.9

The latest guide to catch up is at: For Embulk plugin developers: Get ready for Embulk v0.11 and v1.0!

Apply the Gradle plugin org.embulk.embulk-plugins

Follow the guidance at: https://github.com/embulk/gradle-embulk-plugins

It needs Gradle 6.

Choose your own Maven groupId

Java-based Embulk plugins are to be released as Maven artifacts, in addition to Ruby gems. Set your own Maven groupId to make it available!

Note that org.embulk is only for plugins in https://github.com/embulk/. Use your own groupId for your own plugins, not org.embulk.

If you are not familiar with Maven’s naming convention

Nice to read the Guide to naming conventions on groupId, artifactId, and version at first.

If you have your own domain name

A typical manner is to use the domain name of yours in a reverse order. For example:

group = "com.example.any_sub_domain_if_wanted)"  // NOTE: "com.example" is just an example. Do not use "com.example" actually!
If you don’t have your own domain name

We observe that people often use io.github.???. See io.github in Maven Central. It should be safe because the sub domain name <your-github-username>.github.io is allocated to your GitHub account unless you remove the GitHub account of yours.

group = "io.github.<your-github-username>.any_sub_domain_if_wanted"

If your GitHub username includes - (hyphen) or _ (underscore), please make sure what sub domain name you own under github.io. Domain names must not include _ (underscore) in general. So, GitHub may perform some conversion in your sub domain name under github.io, and the converted name may conflict with others.

Configure to release to your preferred Maven repository

We don’t limit to specific Maven repositories to release Embulk plugins. You can choose your preferred one: Maven Central, Bintray, JCenter, GitHub Packages, or else… But anyway, you will need to configure your build.gradle for your destination.

For example, if you are releasing to Maven Central, the configuration would be like below. See their requirements, jfyi.

publishing {
    publications {
        // ...
    }

    repositories {
        maven {  // publishMavenPublicationToMavenCentralRepository
            name = "mavenCentral"
            if (project.version.endsWith("-SNAPSHOT")) {
                url "https://oss.sonatype.org/content/repositories/snapshots"
            } else {
                url "https://oss.sonatype.org/service/local/staging/deploy/maven2"
            }

            credentials {
                username = project.hasProperty("ossrhUsername") ? ossrhUsername : ""
                password = project.hasProperty("ossrhPassword") ? ossrhPassword : ""
            }
        }
    }
}

signing {  // Maven Central needs signing.
    sign publishing.publications.maven
}

If your plugin uses Jackson, Guava, Apache commons-lang3, or javax.validation

Include those libraries explicitly in dependencies. Use the 100% same versions with org.embulk:embulk-core:0.10.29.

"com.fasterxml.jackson.core:jackson-annotations:2.6.7"
"com.fasterxml.jackson.core:jackson-core:2.6.7"
"com.fasterxml.jackson.core:jackson-databind:2.6.7"
"com.fasterxml.jackson.datatype:jackson-datatype-guava:2.6.7"
"com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.6.7"
"com.fasterxml.jackson.datatype:jackson-datatype-joda:2.6.7"
"com.google.inject:guice:4.0"
"javax.validation:validation-api:1.1.0.Final"
"org.apache.commons:commons-lang3:3.4"

embulk-core would stop depending on those libraries at some point during v0.10. Including the 100% same versions of them keeps your plugin work before and after the removal in embulk-core.

If your plugin uses Joda-Time in it

Use Java 8’s standard java.time classes instead.

If your dependency uses Joda-Time transitively, include Joda-Time explicitly in your dependencies, exact 2.9.2. But we basically recommend to replace Joda-Time with java.time.

"joda-time:joda-time:2.9.2"

If your plugin uses Guice, JRuby (org.jruby classes), or Logback (ch.qos.logback classes)

Stop using them, and find alternatives.

A typical use-case of JRuby has been to use Ruby’s date-time formatter. It can be replaced with embulk-util-timestamp or embulk-util-rubytime now.

If your plugin uses the standard Java EE classes, typically javax.xml.*

Java 9 and 10 need a command line option to use those classes. Java 11+ no longer includes those classes. See JEP 320.

To make the plugin work even with Java 9+, include them explicitly in your dependencies.

Embulk will never provide those classes by itself for plugins. It will be of plugin’s responsibility.

If your plugin uses Embulk’s TimestampFormatter or TimestampParser

Those are deprecated. Use embulk-util-timestamp instead.

(Almost all the plugins) If your plugin uses Embulk’s ConfigSource#loadConfig or TaskSource#loadTask

They will be gradually deprecated. Try using embulk-util-config instead.

If your plugin uses the followings :

Old New
Exec.getLogger org.slf4j.LoggerFactory.getLogger
org.embulk.spi.Timestamp java.time.Instant as far as possible
Guava Optional java.util.Optional
Guava Throwables Read Guava’s document
org.joda.time.* java.time.*
org.jruby.* Stop using it – embulk-util-rubytime may work for parsing date-time
javax.* See JEP 320, and check if it’s removed in Java 11+ (Issue)
SuppressFBWarnings Remove it – FindBugs is no longer supported
ModelManager Stop using it, and build your own Jackson ObjectMapper
@Config @org.embulk.util.config.Config in embulk-util-config
@ConfigDefault @org.embulk.util.config.ConfigDefault in embulk-util-config
@ConfigInject Exec.get???()
ConfigSource#loadConfig ConfigMapper#map in embulk-util-config
TaskSource#loadTask TaskMapper#map in embulk-util-config
Task org.embulk.util.config.Task in embulk-util-config
ColumnConfig org.embulk.util.config.units.ColumnConfig in embulk-util-config
SchemaConfig org.embulk.util.config.units.SchemaConfig in embulk-util-config
LocalFile in PluginTask Use org.embulk.util.config.modules.LocalFileModule and org.embulk.util.config.units.LocalFile in embulk-util-config
Charset in PluginTask Use org.embulk.util.config.modules.CharsetModule in embulk-util-config
Column in PluginTask Use org.embulk.util.config.modules.ColumnModule in embulk-util-config
Schema in PluginTask Use org.embulk.util.config.modules.SchemaModule in embulk-util-config
Type in PluginTask Use org.embulk.util.config.modules.TypeModule in embulk-util-config
Timestamp in PluginTask (deprecated) Use org.embulk.util.config.modules.TimestampModule in embulk-util-config
DateTimeZone in PluginTask Use java.time.ZoneId with org.embulk.util.config.modules.ZoneIdModule in embulk-util-config
org.embulk.spi.json.* org.embulk.util.json.* in embulk-util-json
org.embulk.spi.util.* org.embulk.util.file.* in embulk-util-file, or org.embulk.util.text.* in embulk-util-text
RetryExecutor org.embulk.util.retryhelper.RetryExecutor in embulk-util-retryhelper
TimestampFormatter org.embulk.util.timestamp.TimestampFormatter in embulk-util-timestamp
TimestampParser org.embulk.util.timestamp.TimestampFormatter in embulk-util-timestamp

If your plugin depends on org.embulk:embulk-core for the main code

Remove org.embulk:embulk-core, and depend only on the following two instead.

compileOnly "org.embulk:embulk-api:0.10.29"
compileOnly "org.embulk:embulk-spi:0.10.29"

Once you have done the replacements explained above, org.embulk:embulk-core should no longer be required. (If you see an error by removing org.embulk:embulk-core, it means that you still have something to replace.)

Tests

Tests still need to depend on org.embulk:embulk-core, and in addition, org.embulk:embulk-deps.

testCompile "org.embulk:embulk-core:0.10.29"
testCompile "org.embulk:embulk-deps:0.10.29"

Release your plugin to RubyGems.org, and your preferred Maven repository in addition

Java-based Embulk plugins are to be released as Maven artifacts, in addition to Ruby gems. When releasing your plugin, run two tasks like that, to release a Maven artifact and a Ruby gem.

./gradlew publishMavenPublicationTo<???>Repository gemPush  # NOTE: "<???>" would depend on your configuration.