Github Travis-CI Bintray setup for Maven project

Github hosts thousands of open source repositories. Then why is then still so difficult to setup your repository to be automatically build and publish releases?

Yesterday and today I have spend approximately 6 hours to get this, what seems to be very trivial, working.
So therefore, so you do not have to wait so much time setting this up, a guided walkthrough…

The wanted setup

Getting started

assumption: the sources are already in a public Github repository and you want to release the binaries as open source software.

Setup Travis CI

First, create an account on Travic-CI. You can login with your Github account. See this tutorial for details. Once logged in, Travis will scan all your Github repositories and list them in your account settings. Find the repository you want Travis to build and activate it by flipping the switch. Travis will now add the required webhooks in the Github repository.

Setup Bintray

To publish artifacts to Bintray and account is required. Create an account. You can again use your Github account, this also allows scanning of your repositories, but it is not required.

Decide whether you want to publish artifacts on your personal account or whether as a organisation. If the latter, you have manually create an organisation.

What is confusing is that Bintray’s concept of a repository is not the same as on Github. Bintray’s repository is like a Maven repository and therefore can contain resources and versions for many projects.
If you do not know how to name your Bintray repository, since it should not have the name of your Github repo!, call it ‘maven’ since that seems to be used by many users.

Then, in your newly created repository, create a new package. Use the same name as your Github repository. Fill in additional info like licences, tags, etc. For the ‘website’ and ‘Version control’ use the url of your Github repo. If you have the issue tracker enabled on Github, add this url as well.

Configure your project to build on Travis CI

Travis CI requires a .travis.yml file to be added to your project. This file contains the build information for Travis CI, just like the Jenkinsfile does for Jenkins.

jdk: oraclejdk8 # JDK to use for the build.
language: java  # The programming language of the project.
sudo: false     # No need to run this as root

# For maven projects, the default installation phase can be skipped
install: true

# The branches to build.
# Can either include or exclude branches.
# Here only changes to the master branch will be build.
branches:
  only:
  - master

# Cache the maven repository to not need to download all dependencies on every build.
cache:
  directories:
  - "$HOME/.m2/repository"

Deploy artifacts to Bintray

Travis CI supports deploying artifacts to Bintray. Therefore a ‘deploy provider’ has to be added to the .travis.yml file.

deploy:
# Deploy releases to bintray
- provider: bintray
  skip_cleanup: true        # required to prevent Travis from cleaning the workspace before deployment
  file: target/bintray.json # defines the Bintray project to deploy to, and what to deploy and where to.
  user: <your-bintray-accountname> # fill in your account name
  key: $BINTRAY_API_KEY            # reference to your Bintray API key
  on: # only deploy when
    repo: <github account>/<github repo> # only for this repo
    tags: true                           # and only when a (release) tag was set

For Travis CI to be able to deploy to Bintray, it must authenticate on your behalf. That’s where the user and key are used for. Since you do not want to expose any secrets in a public repo, you can refer to a secret in Travis CI’s project environment.

Define Bintray descriptor A descriptor file is needed to tell Bintray where to deploy the file and which files to deploy to which location. Bintray can help with a lot of stuff, but I have not been able to anywhere in the UI or API a way for Bintray to create this file. So it needs to be created manually.

I decided to call it bintray.json and place in the in root of my project. Using the Maven Resources Plugin filtering some info from the Pom can be used to dynamically fill in the details for this file.

{
  "package": {
    "name": "${project.name}",
    "repo": "<your-bintray-repository-name>",
    "subject": "<your-bintray-account-or-organisation>",
    "website_url": "${project.url}",
    "issue_tracker_url": "${project.url}/issues",
    "vcs_url": "${project.url}",
    "licenses": ["<Your project license>"],
    "labels": ["<label 1>", "<label X>"],
    "public_download_numbers": true,
    "public_stats": true
  },
  "version": {
    "name": "${project.version}",
    "released": "${maven.build.timestamp}",
    "vcs_tag": "${project.version}"
  },
  "files":
  [
    {"includePattern": "pom.xml", "uploadPattern": "com/github/cloudyrock/mongock/mongock/${project.version}/mongock-${project.version}.pom"},
    {"includePattern": "mongock-core/target/(.*\.jar)", "uploadPattern": "com/github/cloudyrock/mongock/mongock-core/${project.version}/$1"},
    {"includePattern": "mongock-core/pom.xml", "uploadPattern": "com/github/cloudyrock/mongock/mongock-core/${project.version}/mongock-core-${project.version}.pom"},
    {"includePattern": "mongock-spring/target/(.*\.jar)", "uploadPattern": "com/github/cloudyrock/mongock/mongock-spring/${project.version}/$1"},
    {"includePattern": "mongock-spring/pom.xml", "uploadPattern": "com/github/cloudyrock/mongock/mongock-spring/${project.version}/mongock-spring-${project.version}.pom"}
  ],
  "publish": true
}

Fill in your own Bintray repo and subject and add labels to your liking.
In the files section you have to add an entry for all files that have to be uploaded to Bintray. As you can see, the includePattern can be a regex and the regex groups can be used in the uploadPattern. Make sure the uploadPattern is according to the Maven standard. For a multi-module project, do not forget to also deploy to parent-pom.

The package.name refers to the Bintray repository package created earlier. If you used the same name in the project Maven Pom, project.name can be used. Otherwise statically fill in the correct Bintray package name.

On every deployment, Bintray will create a new version with the project.version so make sure not to deploy the same version twice. In this setup, the release-tag will be used for the project version and therefore also for the Bintray package version.

Setup Maven Resource Plugin To enable resource filtering with Maven, for the bintray.json file described above, add this to the main project Pom in the build.plugins section:

<plugin>
  <artifactId>maven-resources-plugin</artifactId>
  <version>3.1.0</version>
  <inherited>false</inherited>
  <configuration>
    <outputDirectory>${basedir}/target</outputDirectory>
    <resources>
      <resource>
        <directory>${basedir}</directory>
        <include>bintray.json</include>
        <filtering>true</filtering>
      </resource>
    </resources>
  </configuration>
  <executions>
    <execution>
      <id>copy-resources</id>
      <phase>process-resources</phase>
      <goals>
        <goal>copy-resources</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Deploy artifacts to Github Release

Whenever a new release is created, I’d also like the release artifacts to be available in the Github Release. Travis CI supports deploying to Github. Since the project already deploys to Bintray, an additional deploy provided has to be added.

Since this requires setting up a Github OAuth key, the simplist way to do this, is let the Travis CLI do this for you.

The Travis CLI now has updated the .travis.yml file with the Github release provider containing the api_key. Copy the api_key value, go to your Travis CI project settings page again and add a GITHUB_OAUTH_API_KEY environment value using this value.

# Automatic create release on github repo
- provider: releases
  skip_cleanup: true             # required to prevent Travis from cleaning the workspace before deployment
  api_key: $GITHUB_OAUTH_API_KEY # reference to Github OAuth key
  file_glob: true
  file: # Add all artifacts to deploy to Github
    - "mongock-core/target/*.jar"
    - "mongock-spring/target/*.jar"
  on: # only deploy when
    repo: <github account>/<github repo> # only for this repo
    tags: true                           # and only when a (release) tag was set

Use release tag for versioning

I’d like to be able to release something easily, and tagging a commit with a version seems to be the simplest way to do that.

When Travis CI performs a build based on a new tag, the TRAVIS_TAG variable contains the tag. Then this must be used to update the project before creating the binaries which can be done using Maven’s versions plugin. After this, we must not forget to package the project to get binaries with this new version number.

# Set version to tag for release
before_deploy:
  - mvn versions:set -DnewVersion=$TRAVIS_TAG
  - mvn package -Dmaven.test.skip=true

Note: that this before_deploy phase is run for each deploy provider. It’s a old know issue (#2570) but apparently ‘by design’.
Since it runs for each provider I added the -Dmaven.test.skip=true to the mvn package command. It is already tested before reaching this phase anyway.

Note: if you’re using test-jars between modules, you have to add <skip>false</skip> to the maven-jar-plugin configuration otherwise the test-jar is not being generated causing the mvn package -Dmaven.test.skip=true task to fail.

Finally, for Travis CI to build on a new tag, the tag ‘branch’ must be enabled. This confused me quite a bit, but in the end I found this similar issue leading me to the solution to add the tag branch to the branch.only list which now looks like this:

branches:
  only:
  - update-mongo-java-driver
  - /^.*RELEASE$/ # enable building of release tags

Note: you can use regex patterns in the name. This setup allows to build any tag which ends with ‘RELEASE’.

Add badges to README

There are a number of badges you can add to your README.md file of your repo to indicate the status of your project or library.

License badge
See Shields.io for all possible ways to get a license badge. To create one based on your Github repository license, use the format https://img.shields.io/github/license/<github-account>/<github-repo>.svg.

Final result

After all this setting up, when creating a release on Github (I’d like to give it a proper name and some description), choose a version number which ends with ‘RELEASE’ like ‘2.1.0.RELEASE’ and Travis will start a build and deploy your artifacts to Bintray and added the artifact to your newly created release in Github.

See my first created release. The source zip/tar.gz have been added by Github itself. The jars have been added by the build. In Bintray, the build created this package version containing all jar and pom artifacts.

I hope this tutorial may help others to create an automated build and deploy setup for open source project/libraries. Even though afterwards I thought it was not that much work, while writing all the steps down and looking at the size of this post, it is still quite a lot to figure out.

Here is the final version of .travis.yaml (check my repo for the latest version)

jdk: oraclejdk8
language: java
sudo: false

# Skipping installation phase
install: true

branches:
  only:
  - update-mongo-java-driver
  - /^.*RELEASE$/ # enable building of release tags

cache:
  directories:
  - "$HOME/.m2/repository"

# Set version to tag for release
before_deploy:
  - mvn versions:set -DnewVersion=$TRAVIS_TAG
  - mvn package -Dmaven.test.skip=true

deploy:
# Deploy releases to bintray
- provider: bintray
  skip_cleanup: true
  file: target/bintray.json
  user: diversit
  key: $BINTRAY_API_KEY
  on:
    repo: diversit/mongock
    tags: true

# Automatic create release on github repo
- provider: releases
  skip_cleanup: true
  api_key: $GITHUB_OAUTH_API_KEY
  file_glob: true
  file:
    - "mongock-core/target/*.jar"
    - "mongock-spring/target/*.jar"
  on:
    repo: diversit/mongock
    tags: true