Migrating from Gradle to Bazel

We recently switched our build system, which we use for a collection of Java applications and libraries in a monorepo, from Gradle to Bazel. We had a number of issues with Gradle that led us to seek alternate build tools:

Our experiences with Gradle

Gradle is slow

One of our biggest issues with Gradle is the speed. For example, here are some comparisons between Gradle and Bazel:

With no files changed, rerunning the build (Bazel’s //… syntax means build everything):

% time ./gradlew compileJava compileTestJava
10.547 total

% time bazel build //...
0.520 total  

Running all of the tests takes half the time:

% time ./gradlew test
1:58.84 total

% time bazel test //...
1:01.14 total  

Changing a single file, recompiling, and rerunning dependent tests also takes half the time:

% time ./gradlew test
cpu 25.802 total

% time bazel test //...
12.860 total  

Gradle is slow to figure out which files have changed and which tests need to be run, which we believe is due to the number of subprojects (modules) we have inside this monorepo (about 20). Bazel handles this case much better. Gradle’s support for parallel execution is still incubating, and even with it on, we saw little improvement.

Gradle’s build language is Groovy

Perhaps Groovy is a great language on its own merits, but it’s a difficult build language for a team that isn’t familiar with it. Most of us are familiar with build languages in the host language (e.g. Rake and Ruby), or common scripting languages like Bash, Python, etc. Groovy adds yet another thing to learn, and has its own quirks.

For example, here’s how to execute a command and redirect standard output in a Gradle task:

task myTask(type: Exec) {  
  commandLine 'someCommand', 'param1'

  doFirst {
    standardOutput = new FileOutputStream('output.txt')
  }
}

This is pretty different from a simple someCommand param1 > output.txt. Gradle doesn’t prevent us from writing Bash scripts, but if you want custom commands to be run as dependencies of other tasks, it can be harder to write them separately.

Bazel’s build languages (Core and Skylark) are subsets of Python, so they are more familiar to our team. They also aren’t general-purpose languages, which force us to write scripts in scripting languages. That means we’re using the right tool for the job, instead of Gradle for everything.

Gradle is error-prone

We found Gradle to be full of gotchas. For example, we had a section in our build.gradle that was supposed to turn on compiler warnings:

subprojects {  
  compileJava {
    options.compilerArgs << "-Xlint:all"
    options.compilerArgs << "-Xlint:-processing"
    options.compilerArgs << "-Werror"
  }

  apply plugin: "java"
}

It turned out that since the apply plugin: "java" was after the compileJava section, the options were never applied. There was no error or warning; Gradle silently ignored our code.

The Gradle IntelliJ IDEA plugin also led to complications. It would sometimes fail to refresh after we made changes to our build files (even with auto-import turned on), and then it was hard to get it back into a good state. We’d often have to manually synchronize the project, or even restart IntelliJ.

Another difficulty was libraries which pulled in other, conflicting libraries (kafka -> slf4j-log4j12). We were able to fix this in Gradle with code like:

dependencies {  
  compile("org.apache.kafka:kafka_2.11:0.8.2.1") {
    exclude module: "slf4j-log4j12"
  }
}

However, this didn’t affect the IntelliJ Gradle plugin. To fix that one, we had to manually check in a file called .idea/libraries/Gradle__org_slf4j_slf4j_log4j12_1_6_1.xml:

<component name="libraryTable">  
  <library name="Gradle: org.slf4j:slf4j-log4j12:1.6.1">
    <CLASSES />
    <JAVADOC />
    <SOURCES />
  </library>
</component>  

This worked, but if you accidentally imported the project again, or made any other manual changes to dependencies, IntelliJ would remove this file and the error would return.

In the Bazel world, every dependency is explicitly stated. In our case, we simply left out the conflicting library and only included the good one.

More benefits of Bazel

Bazel brings more to the table than just fixing our issues with Gradle. Here are a few notable features:

Docker support

Bazel recently announced support for building Docker images directly in Bazel, without Dockerfiles.

With Bazel, we were able to create a macro that unified several of our Docker configurations. Now, our apps can build Docker images with only a few lines of config (and no Dockerfile):

app_docker_image(  
  java_binary = "myapp",
  main_class = "braintree.myapp.MyApplication",
)

Bazel query

Bazel supports a powerful query language. For example, if you change a library, you can query and run all of the dependent tests:

bazel test $(bazel query 'kind(test, rdeps(//..., //mylibrary))')  

You can also query and graph your dependencies:

bazel query 'deps(//:main)' --output graph > graph.in

dot -Tpng < graph.in > graph.png  

Test tagging

Bazel lets you specify the size of your test suites (small, medium, etc) and then it will alert you if the suite time exceeds the allotted time; this can help keep test times under control.

There are also a handful of special behavior tags, such as exclusive, manual, external, and flaky.

How we switched

Once we decided to switch, we had to do a number of things to cut over.

Porting the build files

The first step was porting our existing build.gradle files over to BUILD files. We went through a few iterations on scripting this, and the latest version is bazel-deps on GitHub.

With this tool, we were able to generate the majority of our WORKSPACE and BUILD files. We explicitly depend on less than 30 libraries (from Maven), but the transitive dependencies come out to over 200! For example, dropwizard alone brings in about 80 dependent libraries.

IntelliJ

We wrote a script to create an IntelliJ project from our Bazel BUILD files. We based it on the script that comes with Bazel, but modified it for our needs.

For example, our script generates an IntelliJ project with modules for each of our subprojects. This means we need to create a top-level modules.xml, a subproject.iml for each project, and then add dependent projects as project-level dependencies. Our script also adds Bazel-generated files as dependencies to the subprojects, such as generated protobuf classes.

Aliases

We added some aliases for common Bazel commands:

alias bzb="bazel build //..."  
alias bzt="bazel test //..."  

The aftermath

We’ve been on Bazel exclusively for a few weeks now, after running both Bazel and Gradle for a while. The cutover was a little painful as we figured things out like CI (Bazel is a memory hog by default, and our build boxes are memory-constrained). We also had to migrate all of our processes, tools, READMEs, etc. to Bazel. We inevitably missed a few items at first, which caused headaches for some members of our team. Thankfully, we seem to be past all of this, and we’re happily running Bazel now.

If you’re interested in more cool Bazel features, check out the Bazel Blog. We’d love to hear about your experiences with Bazel and Gradle.

Disclaimer: The opinions expressed in this post are those of the individual author and do not represent the opinions of Braintree or PayPal. This is not an endorsement of any product or service by Braintree or PayPal. In addition, an official study resulting in statistical proof was not conducted. This post originally appeared on the author's personal blog.
***
Paul Gross Paul Gross is a Lead Developer at Braintree. He previously worked at ThoughtWorks, a global IT consultancy, building custom software in diverse languages, including Java, .NET, Python, and Ruby. More posts by this author

You Might Also Like