You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/06/27 03:13:17 UTC
[GitHub] [pulsar-client-go] simoneau opened a new issue #302: Build with Bazel fails on MacOS as of 0.1.1.
simoneau opened a new issue #302:
URL: https://github.com/apache/pulsar-client-go/issues/302
One of the reasons we are excited about this package is that it removed any non-Go dependencies, making it easy to build on any environment. As far as I understand, this commit introduced a native dependency:
https://github.com/apache/pulsar-client-go/pull/270
Now when I build my project with bazel on MacOS, I see this error:
```
ERROR: /private/var/tmp/_bazel_matthew/da00a9bee09cb3d27c7323521da8cade/external/com_github_valyala_gozstd/BUILD.bazel:3:1: GoCompilePkg external/com_github_valyala_gozstd/darwin_amd64_stripped/go_default_library%/github.com/valyala/gozstd.a failed (Exit 1) builder failed: error executing command bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk external/go_sdk -installsuffix darwin_amd64 -tags integration -src external/com_github_valyala_gozstd/dict.go -src ... (remaining 55 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox
clang: error: no such file or directory: './libzstd_darwin_amd64.a'
compilepkg: error running subcommand: exit status 1
```
I see a reference that this shouldn't be an issue if "cgo is not enabled", but I'm not sure where to set this in my build. You could argue that this isn't an issue for this package and that it's really a bazel issue or an issue with my build settings. Close this issue if you must. Nevertheless, I wanted to let you know that seeing this build failure when I upgraded to 0.1.1 made me sad. I'll continue to see if there's a workaround on my end and will update this issue if I find anything.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] merlimat commented on issue #302: Build with Bazel fails on MacOS as of 0.1.1.
Posted by GitBox <gi...@apache.org>.
merlimat commented on issue #302:
URL: https://github.com/apache/pulsar-client-go/issues/302#issuecomment-650490872
@simoneau we switched to use C based compression because the performance is ~2x compared to pure Go implementation.
It’s only enabled if CGO_ENABLED=1, so just set it to 0 to completely disable it.
Having said that, there are few fixes in master, merged after 0.1.1 tag that should have fixed the compilation in either case. Please try it out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] nathanclayton commented on issue #302: Build with Bazel fails on MacOS as of 0.1.1.
Posted by GitBox <gi...@apache.org>.
nathanclayton commented on issue #302:
URL: https://github.com/apache/pulsar-client-go/issues/302#issuecomment-705792195
> Or, is Bazel regenerating the go code from the proto file? We're already shipping the generated code, so there should be no need to do that.
That's what it seems to be doing, attempting to regenerate the go code from the proto file using Google's protobuf libraries. I cloned the repo and renamed the `.proto` file extension and things seem to be working better now.
Are you using Gazelle as well? That seems to automatically create the regeneration rules when you run a `gazelle update`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] nathanclayton removed a comment on issue #302: Build with Bazel fails on MacOS as of 0.1.1.
Posted by GitBox <gi...@apache.org>.
nathanclayton removed a comment on issue #302:
URL: https://github.com/apache/pulsar-client-go/issues/302#issuecomment-705792195
> Or, is Bazel regenerating the go code from the proto file? We're already shipping the generated code, so there should be no need to do that.
That's what it seems to be doing, attempting to regenerate the go code from the proto file using Google's protobuf libraries. I cloned the repo and renamed the `.proto` file extension and things seem to be working better now.
Are you using Gazelle as well? That seems to automatically create the regeneration rules when you run a `gazelle update`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] nathanclayton commented on issue #302: Build with Bazel fails on MacOS as of 0.1.1.
Posted by GitBox <gi...@apache.org>.
nathanclayton commented on issue #302:
URL: https://github.com/apache/pulsar-client-go/issues/302#issuecomment-705792195
> Or, is Bazel regenerating the go code from the proto file? We're already shipping the generated code, so there should be no need to do that.
That's what it seems to be doing, attempting to regenerate the go code from the proto file using Google's protobuf libraries. I cloned the repo and renamed the `.proto` file extension and things seem to be working better now.
Are you using Gazelle as well? That seems to automatically create the regeneration rules when you run a `gazelle update`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] merlimat commented on issue #302: Build with Bazel fails on MacOS as of 0.1.1.
Posted by GitBox <gi...@apache.org>.
merlimat commented on issue #302:
URL: https://github.com/apache/pulsar-client-go/issues/302#issuecomment-656405627
@simoneau I think the CGO stuff is not the problem anymore here.
It appears to be a dependency issue. We switched from using `github.com/golang/protobuf` to `github.com/gogo/protobuf`. `go build` is picking up the correct dependencies. Maybe you still have the old deps in bazel?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] simoneau commented on issue #302: Build with Bazel fails on MacOS as of 0.1.1.
Posted by GitBox <gi...@apache.org>.
simoneau commented on issue #302:
URL: https://github.com/apache/pulsar-client-go/issues/302#issuecomment-656388451
@merlimat, thanks for your reply.
For us, portability and developer workflows are more important than raw compression speed.
A simple `go build ./...` seems to work fine, so I guess this is a bazel bug? Or limitation? I assume something in there is turning `CGO_ENABLED=1`?
I pulled master to see if it changed the situation:
`github.com/apache/pulsar-client-go v0.1.2-0.20200709023031-9cbe36f0e8d9`
This doesn't seem to help. Here is the full `--sandbox_debug` build, in case anyone who reads this is more wise in the ways of the build than I.
```
✗ bazel build //... --sandbox_debug
INFO: Analyzed 239 targets (0 packages loaded, 0 targets configured).
INFO: Found 239 targets...
ERROR: /private/var/tmp/_bazel_matthew/da00a9bee09cb3d27c7323521da8cade/external/com_github_apache_pulsar_client_go/pulsar/internal/BUILD.bazel:3:1: GoCompilePkg external/com_github_apache_pulsar_client_go/pulsar/internal/darwin_amd64_stripped/go_default_library%/github.com/apache/pulsar-client-go/pulsar/internal.a failed (Exit 1) sandbox-exec failed: error executing command
(cd /private/var/tmp/_bazel_matthew/da00a9bee09cb3d27c7323521da8cade/sandbox/darwin-sandbox/1457/execroot/__main__ && \
exec env - \
APPLE_SDK_PLATFORM=MacOSX \
APPLE_SDK_VERSION_OVERRIDE=10.15 \
CGO_ENABLED=1 \
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \
GOARCH=amd64 \
GOOS=darwin \
GOPATH='' \
GOROOT=external/go_sdk \
GOROOT_FINAL=GOROOT \
PATH=external/local_config_cc:/bin:/usr/bin \
SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk \
TMPDIR=/var/folders/z0/f6_lcbw12y74t8hr_zfxr3zr0000gp/T/ \
XCODE_VERSION_OVERRIDE=11.5.0.11E608c \
/usr/bin/sandbox-exec -f /private/var/tmp/_bazel_matthew/da00a9bee09cb3d27c7323521da8cade/sandbox/darwin-sandbox/1457/sandbox.sb /var/tmp/_bazel_matthew/install/4f12cbbb3a294bfa314e578758559d54/process-wrapper '--timeout=0' '--kill_delay=15' bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk external/go_sdk -installsuffix darwin_amd64 -src external/com_github_apache_pulsar_client_go/pulsar/internal/backoff.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/batch_builder.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/blocking_queue.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/buffer.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/checksum.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/client_handlers.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/closable.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/commands.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/connection.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/connection_pool.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/connection_reader.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/default_router.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/hash.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/lookup_service.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/rpc_client.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/semaphore.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/topic_name.go -src external/com_github_apache_pulsar_client_go/pulsar/internal/utils.go -arc 'github.com/apache/pulsar-client-go/pulsar/internal/auth=github.com/apache/pulsar-client-go/pulsar/internal/auth=bazel-out/darwin-fastbuild/bin/external/com_github_apache_pulsar_client_go/pulsar/internal/auth/darwin_amd64_stripped/go_default_library%/github.com/apache/pulsar-client-go/pulsar/internal/auth.a=bazel-out/darwin-fastbuild/bin/external/com_github_apache_pulsar_client_go/pulsar/internal/auth/darwin_amd64_stripped/go_default_library%/github.com/apache/pulsar-client-go/pulsar/internal/auth.x' -arc 'github.com/apache/pulsar-client-go/pulsar/internal/compression=github.com/apache/pulsar-client-go/pulsar/internal/compression=bazel-out/darwin-fastbuild/bin/external/com_github_apache_pulsar_client_go/pulsar/internal/compression/darwin_amd64_stripped/go_default_library%/github.com/apache/pulsar-client-go/pulsar/internal/compression.a=bazel-out/darwin-fastbuild/bin/external/com_github_apache_pulsar_client_go/pulsar/internal/compression/darwin_amd64_stripped/go_default_library%/github.com/apache/pulsar-client-go/pulsar/internal/compression.x' -arc 'github.com/apache/pulsar-client-go/pulsar/internal/pulsar_proto=github.com/apache/pulsar-client-go/pulsar/internal/pulsar_proto=bazel-out/darwin-fastbuild/bin/external/com_github_apache_pulsar_client_go/pulsar/internal/pulsar_proto/darwin_amd64_stripped/go_default_library%/github.com/apache/pulsar-client-go/pulsar/internal/pulsar_proto.a=bazel-out/darwin-fastbuild/bin/external/com_github_apache_pulsar_client_go/pulsar/internal/pulsar_proto/darwin_amd64_stripped/go_default_library%/github.com/apache/pulsar-client-go/pulsar/internal/pulsar_proto.x' -arc 'github.com/gogo/protobuf/proto=github.com/gogo/protobuf/proto=bazel-out/darwin-fastbuild/bin/external/com_github_gogo_protobuf/proto/darwin_amd64_stripped/go_default_library%/github.com/gogo/protobuf/proto.a=bazel-out/darwin-fastbuild/bin/external/com_github_gogo_protobuf/proto/darwin_amd64_stripped/go_default_library%/github.com/gogo/protobuf/proto.x' -arc 'github.com/pkg/errors=github.com/pkg/errors=bazel-out/darwin-fastbuild/bin/external/com_github_pkg_errors/darwin_amd64_stripped/go_default_library%/github.com/pkg/errors.a=bazel-out/darwin-fastbuild/bin/external/com_github_pkg_errors/darwin_amd64_stripped/go_default_library%/github.com/pkg/errors.x' -arc 'github.com/sirupsen/logrus=github.com/sirupsen/logrus=bazel-out/darwin-fastbuild/bin/external/com_github_sirupsen_logrus/darwin_amd64_stripped/go_default_library%/github.com/sirupsen/logrus.a=bazel-out/darwin-fastbuild/bin/external/com_github_sirupsen_logrus/darwin_amd64_stripped/go_default_library%/github.com/sirupsen/logrus.x' -arc 'github.com/spaolacci/murmur3=github.com/spaolacci/murmur3=bazel-out/darwin-fastbuild/bin/external/com_github_spaolacci_murmur3/darwin_amd64_stripped/go_default_library%/github.com/spaolacci/murmur3.a=bazel-out/darwin-fastbuild/bin/external/com_github_spaolacci_murmur3/darwin_amd64_stripped/go_default_library%/github.com/spaolacci/murmur3.x' -importpath github.com/apache/pulsar-client-go/pulsar/internal -p github.com/apache/pulsar-client-go/pulsar/internal -package_list bazel-out/host/bin/external/go_sdk/packages.txt -o bazel-out/darwin-fastbuild/bin/external/com_github_apache_pulsar_client_go/pulsar/internal/darwin_amd64_stripped/go_default_library%/github.com/apache/pulsar-client-go/pulsar/internal.a -nogo bazel-out/darwin-opt-exec-2B5CBBC6/bin/darwin_amd64_stripped/my_nogo -x bazel-out/darwin-fastbuild/bin/external/com_github_apache_pulsar_client_go/pulsar/internal/darwin_amd64_stripped/go_default_library%/github.com/apache/pulsar-client-go/pulsar/internal.x -gcflags '' -asmflags '')
compilepkg: error running subcommand: exit status 2
/private/var/tmp/_bazel_matthew/da00a9bee09cb3d27c7323521da8cade/sandbox/darwin-sandbox/1457/execroot/__main__/external/com_github_apache_pulsar_client_go/pulsar/internal/commands.go:206:28: smm.Size undefined (type *pulsar_proto.SingleMessageMetadata has no field or method Size)
/private/var/tmp/_bazel_matthew/da00a9bee09cb3d27c7323521da8cade/sandbox/darwin-sandbox/1457/execroot/__main__/external/com_github_apache_pulsar_client_go/pulsar/internal/commands.go:210:15: smm.MarshalToSizedBuffer undefined (type *pulsar_proto.SingleMessageMetadata has no field or method MarshalToSizedBuffer)
/private/var/tmp/_bazel_matthew/da00a9bee09cb3d27c7323521da8cade/sandbox/darwin-sandbox/1457/execroot/__main__/external/com_github_apache_pulsar_client_go/pulsar/internal/commands.go:236:19: cmdSend.MarshalToSizedBuffer undefined (type *pulsar_proto.BaseCommand has no field or method MarshalToSizedBuffer)
/private/var/tmp/_bazel_matthew/da00a9bee09cb3d27c7323521da8cade/sandbox/darwin-sandbox/1457/execroot/__main__/external/com_github_apache_pulsar_client_go/pulsar/internal/commands.go:251:22: msgMetadata.MarshalToSizedBuffer undefined (type *pulsar_proto.MessageMetadata has no field or method MarshalToSizedBuffer)
/private/var/tmp/_bazel_matthew/da00a9bee09cb3d27c7323521da8cade/sandbox/darwin-sandbox/1457/execroot/__main__/external/com_github_apache_pulsar_client_go/pulsar/internal/connection.go:386:23: cmd.Size undefined (type *pulsar_proto.BaseCommand has no field or method Size)
/private/var/tmp/_bazel_matthew/da00a9bee09cb3d27c7323521da8cade/sandbox/darwin-sandbox/1457/execroot/__main__/external/com_github_apache_pulsar_client_go/pulsar/internal/connection.go:396:15: cmd.MarshalToSizedBuffer undefined (type *pulsar_proto.BaseCommand has no field or method MarshalToSizedBuffer)
INFO: Elapsed time: 0.518s, Critical Path: 0.32s
INFO: 1 process: 1 darwin-sandbox.
FAILED: Build did NOT complete successfully
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] nathanclayton commented on issue #302: Build with Bazel fails on MacOS as of 0.1.1.
Posted by GitBox <gi...@apache.org>.
nathanclayton commented on issue #302:
URL: https://github.com/apache/pulsar-client-go/issues/302#issuecomment-706367641
@simoneau and @merlimat - This problem seems to be directly related to Bazel/Gazelle attempting to rebuild the protobuf files (even when defining the proto compiler as gogofast in the root configuration).
To fix this issue, in the `go_repository` that defines the pulsar client you need to add the line `build_file_proto_mode = "disable_global",`, and everything will build just fine.
For reference, my repository definitions are:
```
go_repository(
name = "com_github_apache_pulsar_client_go",
importpath = "github.com/apache/pulsar-client-go",
sum = "h1:7teu0FaXzzKPjDdUNjA7dVYKFjCy6OVX5as6nUww4qk=",
version = "v0.2.0",
build_file_proto_mode = "disable_global",
)
go_repository(
name = "com_github_apache_pulsar_client_go_oauth2",
importpath = "github.com/apache/pulsar-client-go/oauth2",
sum = "h1:RCaSqG1q6sN2kBDwF2FjGEkcQbZz7eJabcQaxPr2Yfc=",
version = "v0.0.0-20201001062620-12eedc37df57",
build_file_proto_mode = "disable_global",
)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] merlimat commented on issue #302: Build with Bazel fails on MacOS as of 0.1.1.
Posted by GitBox <gi...@apache.org>.
merlimat commented on issue #302:
URL: https://github.com/apache/pulsar-client-go/issues/302#issuecomment-656405932
Or, is Bazel regenerating the go code from the proto file? We're already shipping the generated code, so there should be no need to do that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org