Bazel has always had support for protocol buffers (protobuf) since the beginning. Both being a Google product, one would think that their integration would be seamless and the best experience. Unfortunately, it’s some of the worst part of the user experience with Bazel I’ve found. 😔

Let’s start with the basics; What rule should I adopt for protobufs?

Well first I Google “Bazel protobuf” and land on the protobuf reference page for Bazel which states:

If using Bazel, please load the rule from https://github.com/bazelbuild/rules_proto.

One may think the sensible rules_proto is a good starting point but the README.md states:

This repository is deprecated…we decided to move the implementation of the rules together with proto compiler into protobuf repository.

OK…🤔

Let’s go check protobuf.

The README.md claims one can install one of two ways by inserting the following into your MODULE.bazel without much explanation as to the difference. 🤷‍♂️

bazel_dep(name = "protobuf", version = <VERSION>)
#
# or
#
bazel_dep(name = "protobuf", version = <VERSION>,
          repo_name = "com_google_protobuf")

I decide to audit the source to see what’s going on. You quickly land on the rule definition for proto_library and see the following documentation for the rule. 🤦

proto_library = rule(
    _proto_library_impl,
    # TODO: proto_common docs are missing
    # TODO: ProtoInfo link doesn't work and docs are missing
    doc = """
<p>If using Bazel, please load the rule from
<a href="https://github.com/bazelbuild/rules_proto">
https://github.com/bazelbuild/rules_proto</a>.

Where is the protoc (protobuf compiler) ultimately coming from for the rule? I notice these interesting snippets in the rule.

toolchains.if_legacy_toolchain({
        "_proto_compiler": attr.label(
            cfg = "exec",
            executable = True,
            allow_files = True,
            default = configuration_field("proto", "proto_compiler"),

_incompatible_toolchain_resolution =
    getattr(native_proto_common,
            "INCOMPATIBLE_ENABLE_PROTO_TOOLCHAIN_RESOLUTION", False)

def _if_legacy_toolchain(legacy_attr_dict):
    if _incompatible_toolchain_resolution:
        return {}
    else:
        return legacy_attr_dict

Turns out that INCOMPATIBLE_ENABLE_PROTO_TOOLCHAIN_RESOLUTION is set from the command line ref.

–[no]incompatible_enable_proto_toolchain_resolution default: “false” If true, proto lang rules define toolchains from protobuf repository. Tags: loading_and_analysis, incompatible_change

I don’t have that in my .bazelrc so let’s ignore it. That means our _proto_compiler is coming from configuration_field("proto", "proto_compiler").

You then search the bazelbuild/bazel source to find where it’s defined.

@Option(
    name = "proto_compiler",
    defaultValue = ProtoConstants.DEFAULT_PROTOC_LABEL,
    converter = CoreOptionConverters.LabelConverter.class,
    documentationCategory = OptionDocumentationCategory.UNCATEGORIZED,
    effectTags = {OptionEffectTag.AFFECTS_OUTPUTS, OptionEffectTag.LOADING_AND_ANALYSIS},
    help = "The label of the proto-compiler.")
public Label protoCompiler;

// The flags need to point to @bazel_tools, because this is a canonical repo
// name when either bzlmod or WORKSPACE mode is used.
/** Default label for proto compiler.*/
public static final String DEFAULT_PROTOC_LABEL
        = "@bazel_tools//tools/proto:protoc";

Chasing down the ultimate target in the defining BUILD file you discover it’s an alias to "@com_google_protobuf//:protoc".

# Those aliases are needed to resolve the repository name correctly in both
# bzlmod and WORKSPACE mode. They are resolved in the namespace of MODULE.tools

alias(
    name = "protoc",
    actual = "@com_google_protobuf//:protoc",
)

😲 So we discovered why com_google_protobuf may want to be the repo_name in the bazel_dep rule. The repository name com_google_protobuf is hard-coded within the Bazel source code for the location to discover the protoc compiler.

You’ll have to trust me that the resolution to the compiler for the language toolchains such as java_proto_library is the same as well; just way more obfuscated.

The rabbit hole only goes deeper if you consider gRPC, other languages and then having to manage various runtimes (compatibility matrix) for your language across your codebases if they leave source of truth.

I feel like we discovered a lot but didn’t really learn or accomplish anything. 😩

Brighter Future?

Lots of interesting work is being done by the rule-authors SIG (Special Interest Group).

That doc has a great in-depth overview of the current state of affairs.

The most notable changes on the horizon are migrating protocol buffers to Bazel’s toolchain mechanism. This should make binding to protoc look like other toolchains in Bazel and no longer special case com_google_protobuf.

What are toolchains? In my mind effectively the capability to late bind a label to a target.

To me, a simple immediate improvement would be fixing the documentation around rules_proto and having a more clear path on how to adopt Bazel given some constraint (i.e. Bazel >= 7.0).

The latest blog post from Bazel on protobuf is from 2017!

The work Aspect Build is doing to improve the protobuf ecosystem is great as well. Their video series on “Never Compile Protoc Again” is excellent and served as a great resource for my previous post on minding your PATH.

---