grpc-java: protoc-gen-grpc-java not available on apple m1

What version of gRPC-Java are you using?

1.34.0

What is your environment?

osx 11.0.1

What did you expect to see?

build success

What did you see instead?

[INFO] --- protobuf-maven-plugin:0.6.1:compile (default-cli) @ grpc ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.720 s
[INFO] Finished at: 2020-12-03T16:37:02+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.xolstice.maven.plugins:protobuf-maven-plugin:0.6.1:compile (default-cli) on project grpc: Unable to resolve artifact: Missing:
[ERROR] ----------
[ERROR] 1) com.google.protobuf:protoc:exe:osx-aarch_64:3.12.0
[ERROR] 
[ERROR]   Try downloading the file manually from the project website.
[ERROR] 
[ERROR]   Then, install it using the command: 
[ERROR]       mvn install:install-file -DgroupId=com.google.protobuf -DartifactId=protoc -Dversion=3.12.0 -Dclassifier=osx-aarch_64 -Dpackaging=exe -Dfile=/path/to/file
[ERROR] 
[ERROR]   Alternatively, if you host your own repository you can deploy the file there: 
[ERROR]       mvn deploy:deploy-file -DgroupId=com.google.protobuf -DartifactId=protoc -Dversion=3.12.0 -Dclassifier=osx-aarch_64 -Dpackaging=exe -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
[ERROR] 
[ERROR]   Path to dependency: 
[ERROR]   	1) com.reinmind:grpc:jar:1.0-SNAPSHOT
[ERROR]   	2) com.google.protobuf:protoc:exe:osx-aarch_64:3.12.0
[ERROR] 
[ERROR] ----------
[ERROR] 1 required artifact is missing.
[ERROR] 
[ERROR] for artifact: 
[ERROR]   com.reinmind:grpc:jar:1.0-SNAPSHOT
[ERROR] 
[ERROR] from the specified remote repositories:
[ERROR]   aliyunmaven (https://maven.aliyun.com/repository/public, releases=true, snapshots=false)
[ERROR] 
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Steps to reproduce the bug

mvn protobuf:compile

About this issue

Original URL
State: open
Created 4 years ago
Reactions: 27
Comments: 93 (23 by maintainers)

Commits related to this issue

Added workaround for protoc on Apple Silicon Just a workaround. Track https://github.com/grpc/grpc-java/issues/7690 and https://github.com/protocolbuffers/protobuf/pull/8557 for permanent fix — committed to drasyl/drasyl by KevinRoebert 3 years ago
protoc-gen-grpc-java not available on apple m1. Add workaround as suggested here : https://github.com/grpc/grpc-java/issues/7690#issuecomment-760619366 — committed to azimbabu/grpc-up-and-running by azimbabu 2 years ago
Support Apple Chip for protoc * See https://github.com/grpc/grpc-java/issues/7690 — committed to emeraldhieu/vinci by emeraldhieu a year ago
Sync appropriate protoc artifact for m1 Fixes compilation on M1 macOS. Check https://github.com/grpc/grpc-java/issues/7690 for further info Signed-off-by: Aayush Gupta <aayushgupta219@gmail.com> — committed to theimpulson/seedvault by theimpulson a year ago
Sync appropriate protoc artifact for m1 Fixes compilation on M1 macOS. Check https://github.com/grpc/grpc-java/issues/7690 for further info Signed-off-by: Aayush Gupta <aayushgupta219@gmail.com> — committed to theimpulson/seedvault by theimpulson a year ago
Sync appropriate protoc artifact for m1 Fixes compilation on M1 macOS. Check https://github.com/grpc/grpc-java/issues/7690 for further info Signed-off-by: Aayush Gupta <aayushgupta219@gmail.com> — committed to theimpulson/seedvault by theimpulson a year ago
work-around old protobuf:protoc on Apple Silicon Use x86_64 protobuf compilers on macOS until protobuf is updated to a newer version (~3.21.0). See https://github.com/grpc/grpc-java/issues/7690 for m... — committed to tmancill/JVoiceXML by tmancill 10 months ago
work-around old protobuf:protoc on Apple Silicon Use x86_64 protobuf compilers on macOS until protobuf is updated to a newer version (~3.21.0). See https://github.com/grpc/grpc-java/issues/7690 for m... — committed to tmancill/JVoiceXML by tmancill 10 months ago
Sync appropriate protoc artifact for m1 Fixes compilation on M1 macOS. Check https://github.com/grpc/grpc-java/issues/7690 for further info Signed-off-by: Aayush Gupta <aayushgupta219@gmail.com> — committed to seedvault-app/seedvault by theimpulson a year ago
Sync appropriate protoc artifact for m1 Fixes compilation on M1 macOS. Check https://github.com/grpc/grpc-java/issues/7690 for further info Signed-off-by: Aayush Gupta <aayushgupta219@gmail.com> — committed to AndreiFlorea04/seedvault by AndreiFlorea04 a year ago

Most upvoted comments

To manually specify the classifier, for Gradle:

protobuf {
  protoc {
    artifact = 'com.google.protobuf:protoc:3.14.0:osx-x86_64'
  }
}

I’m less familiar with the Maven plugin, but I think it may be something like:

<protocArtifact>com.google.protobuf:protoc:3.14.0:exe:osx-x86_64</protocArtifact>

You can do the same for protoc-gen-grpc-java.

+158

ejona86 on Dec 17, 2020

for me, add this to ~/.m2/settings.xml, it worked:

<settings>
  ...
  <activeProfiles>
    <activeProfile>
      apple-silicon
    </activeProfile>
    ...
  </activeProfiles>
  <profiles>
    <profile>
      <id>apple-silicon</id>
      <properties>
        <os.detected.classifier>osx-x86_64</os.detected.classifier>
      </properties>
    </profile>
    ...
  </profiles>
  ...
</settings>

+81

hu-chia on Feb 3, 2021

@Jiachen-Zhang

protobuf {
  protoc {
    if (osdetector.os == "osx") {
      artifact = 'com.google.protobuf:protoc:3.14.0:osx-x86_64'
    } else {
      artifact = 'com.google.protobuf:protoc:3.14.0'
    }
  }
}

You can also use project properties to only change the behavior when explicitly requested.

+39

ejona86 on Jan 14, 2021

<settings>
  <activeProfiles>
    <activeProfile>
      apple-silicon
    </activeProfile>
  </activeProfiles>
  <profiles>
    <profile>
      <id>apple-silicon</id>
      <properties>
        <os.detected.classifier>osx-x86_64</os.detected.classifier>
      </properties>
    </profile>
  </profiles>
</settings>

@guyeu my seetings.xml file

+30

Uditmittal on Feb 24, 2021

@Uditmittal in my build.gradle file

protobuf {
    protoc {
        // for apple m1, please add protoc_platform=osx-x86_64 in $HOME/.gradle/gradle.properties
        if (project.hasProperty('protoc_platform')) {
            artifact = "com.google.protobuf:protoc:3.13.0:${protoc_platform}"
        } else {
            artifact = "com.google.protobuf:protoc:3.13.0"
        }
    }
}

in my ~/.gradle/gradle.properties file

protoc_platform=osx-x86_64

I think you can define and use variables in maven to implement this feature. Good luck

+16

zjc17 on Feb 23, 2021

/usr/sbin/softwareupdate --install-rosetta --agree-to-license It is now May 2023 and none of the above methods work. This command solved my problem.

+10

venshine on May 5, 2023

    <profiles>
        <profile>
            <id>mac</id>
            <activation>
                <os> 
                <family>mac</family>
                </os>
            </activation>
            <properties>
                <os.detected.classifier>osx-x86_64</os.detected.classifier>
            </properties>
        </profile>
    </profiles>

hengyunabc on Apr 14, 2023

@Uditmittal This is my org.xolstice.maven.plugins:protobuf-maven-plugin build plugin configuration, the ${os.detected.classifier} value could read from active profile from settings.xml.

      <configuration>
          <protocArtifact>
              com.google.protobuf:protoc:3.1.0:exe:${os.detected.classifier}
          </protocArtifact>
          <pluginId>grpc-java</pluginId>
          <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.0.0:exe:${os.detected.classifier}</pluginArtifact>
      </configuration>

hu-chia on Feb 25, 2021

@Jiachen-Zhang
protobuf {
  protoc {
    if (osdetector.os == "osx") {
      artifact = 'com.google.protobuf:protoc:3.14.0:osx-x86_64'
    } else {
      artifact = 'com.google.protobuf:protoc:3.14.0'
    }
  }
}
You can also use project properties to only change the behavior when explicitly requested.

Thanks!

Finally, I made it by

// for apple m1, please add protoc_platform=osx-x86_64 in $HOME/.gradle/gradle.properties
if (project.hasProperty('protoc_platform')) {
    artifact = "com.google.protobuf:protoc:3.13.0:${protoc_platform}"
} else {
    artifact = "com.google.protobuf:protoc:3.13.0"
}

zjc17 on Jan 15, 2021

https://github.com/protocolbuffers/protobuf/pull/8557 just went into protobuf based on my suggestion to just copy the current x86 binary to an arm64 name until we have actual M1 support. That doesn’t “fix” this, but it does resolve the main user-visible issues and doesn’t turn out to be too hacky.

I think that is as simple as making a copy of the “exe” and the hashes/signature in our upload_artifacts.sh script.

ejona86 on Jun 2, 2021

@ejona86 do you know when the protoc-gen-grpc-java_xxx_osx_aarch64 build of will be fixed? As of now osx_aarch and osx_x86-84 are the same binary which is not compatible with M1 CPU:

./protoc-gen-grpc-java-1.51.1-osx-aarch_64.exe 
zsh: bad CPU type in executable: ./protoc-gen-grpc-java-1.51.1-osx-aarch_64.exe

Source: https://repo1.maven.org/maven2/io/grpc/protoc-gen-grpc-java/1.51.1/

I suspect it’s not big deal for infra team to make a proper build for that artifact since protoc-xxx-osx-aarch_64.exe was built correctly: https://repo1.maven.org/maven2/com/google/protobuf/protoc/3.21.9/

Am I right that @deannagarcia could do that? https://github.com/protocolbuffers/protobuf/issues/9397#issuecomment-1124400911

If it’s impossible to provide correct aarch64 builds of protoc-gen-grpc-java please remove it from artifact publishing to not mislead end users.

dtretyakov on Dec 28, 2022

Yes, we do not produce binaries for arm64 macs.

Edit: Summary of the long issue: The easiest thing to do as a consumer is to install Rosetta. With it installed, the Intel binary works fine. If you’d like to contribute, see https://github.com/grpc/grpc-java/issues/7690#issuecomment-831617279 . Easiest plan is to build protobuf and the grpc plugin as universal binaries.

ejona86 on Oct 20, 2023

zsh: bad CPU type in executable: ./protoc-gen-grpc-java-osx-aarch_64.exe

can not executable on mac m2 chips version: protoc-gen-grpc-java-1.58.0-osx-aarch_64.exe

JackyAnn on Oct 18, 2023

Interactions with the protobuf plugins is resolved, but requires Rosetta. I reopened the issue to track native M1 support.

ejona86 on Mar 3, 2022

@ejona86 I’ve had a look at the repo and created a very simple PR #8680 . No idea if it works but maybe this gets things rolling?

kdubb on Nov 9, 2021

@ejona86 Where do I beg to get protoc-gen-grpc-java published as osx-aarch_64?

This is the lone holdout of 3 projects (including com.google.protobuf:protoc) required for building easily on the M1. The other projects seem to be following the advice of copying osx-x86_64 to osx-aarch_64 and it works well. I currently do this manually on my machine to get the required protoc-gen-grpc-java:osx-aarch_64 artifact.

kdubb on Nov 9, 2021

https://github.com/protocolbuffers/protobuf/issues/9397#issuecomment-1535044692 🥳

joschi on May 5, 2023

please adjust plugin configuration to something like this

<plugins>
            <!-- compile proto file into java files. -->
            <plugin>
                <groupId>com.github.os72</groupId>
                <artifactId>protoc-jar-maven-plugin</artifactId>
                <version>3.11.4</version>
                <executions>
                    <execution>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>run</goal>
                        </goals>
                        <configuration>
                            <protocVersion>3.21.1</protocVersion>
                            <includeStdTypes>true</includeStdTypes>
                            <pluginArtifact>io.grpc:protoc-gen-grpc-java:pom:1.47.0</pluginArtifact>
                            <includeMavenTypes>direct</includeMavenTypes>
                            <includeMavenTypes>transitive</includeMavenTypes>
                            <addProtoSources>all</addProtoSources>
                            <includeDirectories>
                                <include>src/main/resources</include>
                            </includeDirectories>
                            <inputDirectories>
                                <include>src/main/resources/protobuf/src</include>
                            </inputDirectories>
                            <outputTargets>
                                <outputTarget>
                                    <type>java</type>
                                    <outputDirectory>src/main/java</outputDirectory>
                                </outputTarget>
                            </outputTargets>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

my output

[INFO] ----------------< grpc-Oscar-Health:grpc-Oscar-Health >-----------------
[INFO] Building grpc-Oscar-Health 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- protoc-jar-maven-plugin:3.11.4:run (default) @ grpc-Oscar-Health ---
[INFO] Protoc version: 3.21.1
protoc-jar: protoc version: 3.21.1, detected platform: osx-aarch_64 (mac os x/aarch64)
protoc-jar: cached: /var/folders/46/zh5s3qfd0j7gr6x4ffj03s_h0000gn/T/protocjar.webcache/com/google/protobuf/protoc/maven-metadata.xml
protoc-jar: cached: /var/folders/46/zh5s3qfd0j7gr6x4ffj03s_h0000gn/T/protocjar.webcache/com/google/protobuf/protoc/3.21.1/protoc-3.21.1-osx-aarch_64.exe
protoc-jar: executing: [/var/folders/46/zh5s3qfd0j7gr6x4ffj03s_h0000gn/T/protocjar2051368796002136723/bin/protoc.exe, --version]
libprotoc 3.21.1
[INFO] Protoc command: /var/folders/46/zh5s3qfd0j7gr6x4ffj03s_h0000gn/T/protocjar2051368796002136723/bin/protoc.exe
[INFO] Additional include types: /var/folders/46/zh5s3qfd0j7gr6x4ffj03s_h0000gn/T/protocjar13749638867177729208/include
[INFO]     google/protobuf/any.proto
[INFO]     google/protobuf/api.proto
[INFO]     google/protobuf/compiler/plugin.proto
[INFO]     google/protobuf/descriptor.proto
[INFO]     google/protobuf/duration.proto
[INFO]     google/protobuf/empty.proto
[INFO]     google/protobuf/field_mask.proto
[INFO]     google/protobuf/source_context.proto
[INFO]     google/protobuf/struct.proto
[INFO]     google/protobuf/timestamp.proto
[INFO]     google/protobuf/type.proto
[INFO]     google/protobuf/wrappers.proto
[INFO]     grpc/lb/v1/load_balancer.proto
[INFO]     grpc/lookup/v1/rls_config.proto
[INFO]     grpc/lookup/v1/rls.proto
[INFO]     grpc/reflection/v1alpha/reflection.proto
[INFO]     grpc/health/v1/health.proto
[INFO]     grpc/channelz/v1/channelz.proto
[INFO]     grpc/binlog/v1/binarylog.proto
[INFO]     grpc/gcp/altscontext.proto
[INFO]     grpc/gcp/transport_security_common.proto
[INFO]     grpc/gcp/handshaker.proto
[INFO]     opencensus/proto/trace/v1/trace_config.proto
[INFO]     opencensus/proto/trace/v1/trace.proto
[INFO]     opencensus/proto/resource/v1/resource.proto
[INFO]     opencensus/proto/metrics/v1/metrics.proto
[INFO]     opencensus/proto/stats/v1/stats.proto
[INFO]     opencensus/proto/agent/trace/v1/trace_service.proto
[INFO]     opencensus/proto/agent/common/v1/common.proto
[INFO]     opencensus/proto/agent/metrics/v1/metrics_service.proto
[INFO]     google/api/monitoring.proto
[INFO]     google/api/distribution.proto
[INFO]     google/api/backend.proto
[INFO]     google/api/annotations.proto
[INFO]     google/api/label.proto
[INFO]     google/api/billing.proto
[INFO]     google/api/monitored_resource.proto
[INFO]     google/api/config_change.proto
[INFO]     google/api/consumer.proto
[INFO]     google/api/context.proto
[INFO]     google/api/experimental/experimental.proto
[INFO]     google/api/experimental/authorization_config.proto
[INFO]     google/api/control.proto
[INFO]     google/api/metric.proto
[INFO]     google/api/system_parameter.proto
[INFO]     google/api/usage.proto
[INFO]     google/api/httpbody.proto
[INFO]     google/api/http.proto
[INFO]     google/api/auth.proto
[INFO]     google/api/source_info.proto
[INFO]     google/api/endpoint.proto
[INFO]     google/api/logging.proto
[INFO]     google/api/service.proto
[INFO]     google/api/log.proto
[INFO]     google/api/documentation.proto
[INFO]     google/api/quota.proto
[INFO]     google/longrunning/operations.proto
[INFO]     google/cloud/audit/audit_log.proto
[INFO]     google/logging/type/log_severity.proto
[INFO]     google/logging/type/http_request.proto
[INFO]     google/type/date.proto
[INFO]     google/type/latlng.proto
[INFO]     google/type/timeofday.proto
[INFO]     google/type/color.proto
[INFO]     google/type/money.proto
[INFO]     google/type/dayofweek.proto
[INFO]     google/type/postal_address.proto
[INFO]     google/rpc/code.proto
[INFO]     google/rpc/status.proto
[INFO]     google/rpc/error_details.proto
[INFO] Input directories:
[INFO]     /Users/slachiewicz/work/grpc-java-7690/src/main/resources/protobuf/src
[INFO] Include directories:
[INFO]     /Users/slachiewicz/work/grpc-java-7690/src/main/resources
[INFO]     /var/folders/46/zh5s3qfd0j7gr6x4ffj03s_h0000gn/T/protocjar13749638867177729208/include
[INFO] Output targets:
[INFO]     java: /Users/slachiewicz/work/grpc-java-7690/src/main/java (add: main, clean: false, plugin: null, outputOptions: null)
[INFO]     Processing (java): book.proto
protoc-jar: executing: [/var/folders/46/zh5s3qfd0j7gr6x4ffj03s_h0000gn/T/protocjar2051368796002136723/bin/protoc.exe, -I/Users/slachiewicz/work/grpc-java-7690/src/main/resources, -I/var/folders/46/zh5s3qfd0j7gr6x4ffj03s_h0000gn/T/protocjar13749638867177729208/include, -I/Users/slachiewicz/work/grpc-java-7690/src/main/resources/protobuf/src, --java_out=/Users/slachiewicz/work/grpc-java-7690/src/main/java, /Users/slachiewicz/work/grpc-java-7690/src/main/resources/protobuf/src/book.proto]
[INFO] Adding generated sources (java): /Users/slachiewicz/work/grpc-java-7690/src/main/java

slachiewicz on Jul 12, 2022

@MochiXu, I expect you’re just missing a place in your build that you need to make the change. There’s not much way to help you without seeing your build. That said, that recommendation of mine is pretty old. Since protobuf v3.17.3, protobuf has been copying the osx-x86_64 artifact to osx-aarch_64 meaning you wouldn’t need configuration in your build. So these days I’d recommend you just upgrade to a newer protobuf.

ejona86 on May 20, 2022

Reopening because we’ve only a workaround currently. We still need to do something along the lines of https://github.com/grpc/grpc-java/issues/7690#issuecomment-831617279 for native M1 support. I thought we had another issue open for tracking native M1 support, but I can’t find it. Anyway, this will do.

ejona86 on Mar 2, 2022