You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/12/08 17:16:22 UTC
[GitHub] [arrow] lidavidm opened a new pull request #11906: WIP: [C++] Demonstrate C++ OTel instrumentation
lidavidm opened a new pull request #11906:
URL: https://github.com/apache/arrow/pull/11906
Instructions:
```
$ mkdir otlp
$ nvim otlp/config.dev.yml
exporters:
file:
path: /dev/stdout
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: "0.0.0.0:4318"
cors_allowed_origins:
- '*'
service:
pipelines:
traces:
receivers:
- otlp
exporters:
- file
$ docker run --rm -it -p 4317:4317 -p 4318:4318 -v (pwd)/otlp:/cfg otel/opentelemetry-collector:0.40.0 --config=/cfg/config.dev.yml
# In another terminal
$ pwd
/path/to/arrow/repo/root
$ mkdir build; cd build
$ cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -GNinja -DCMAKE_INSTALL_PREFIX=$(pwd)/../install -DCMAKE_INSTALL_LIBDIR=lib -DARROW_DATASET=ON -DARROW_PARQUET=ON -DARROW_JSON=ON -DARROW_CSV=ON -DARROW_WITH_SNAPPY=ON -DARROW_WITH_LZ4=ON -DARROW_S3=ON -DARROW_ENGINE=ON -DARROW_WITH_OPENTELEMETRY=ON
$ ninja install
$ cd ../r
$ export ARROW_HOME=$(pwd)/../install
$ export LD_LIBRARY_PATH=$(pwd)/../install/lib
$ env LIBARROW_DOWNLOAD=false LIBARROW_BUILD=false ARROW_R_DEV=true R CMD INSTALL .
$ R -e 'remotes::install_github("ursacomputing/arrowbench")'
$ env ARROW_TRACING_BACKEND=otlp_http R -e 'library(arrowbench);run_benchmark(tpc_h, engine=c("arrow"), query_id=c(1), format=c("parquet"), scale_factor=c(1))'
# Go look at the terminal running the OTLP collector
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on pull request #11906: WIP: [C++] Demonstrate C++ OTel instrumentation
Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11906:
URL: https://github.com/apache/arrow/pull/11906#issuecomment-989284780
We could enable the built-in stdout logger, though that logs in a format that's very not-amenable to parsing. We could also restore the JSON logger from the original PR, though that would need some updates to log in a spec-compliant way. Or we could use https://github.com/open-telemetry/opentelemetry-cpp/pull/1111.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm closed pull request #11906: WIP: [C++] Demonstrate C++ OTel instrumentation
Posted by GitBox <gi...@apache.org>.
lidavidm closed pull request #11906:
URL: https://github.com/apache/arrow/pull/11906
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on pull request #11906: WIP: [C++] Demonstrate C++ OTel instrumentation
Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11906:
URL: https://github.com/apache/arrow/pull/11906#issuecomment-990333820
In #11925 we now have two exporters that can be enabled by env var, with some samples of the output - hopefully that'll work for Conbench
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on pull request #11906: WIP: [C++] Demonstrate C++ OTel instrumentation
Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11906:
URL: https://github.com/apache/arrow/pull/11906#issuecomment-989010602
CC @westonpace @felipeblazing @jonkeane @mbrobbel
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jonkeane commented on pull request #11906: WIP: [C++] Demonstrate C++ OTel instrumentation
Posted by GitBox <gi...@apache.org>.
jonkeane commented on pull request #11906:
URL: https://github.com/apache/arrow/pull/11906#issuecomment-989287140
FWIW, stdout might be the easiest lift to get this into conbench (well one I merge https://github.com/ursacomputing/arrowbench/pull/43 we can have conbench consume the stdout from that).
Conbench could certainly be extended to look for a dump file and store that too, but it would be a slightly bigger lift.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] westonpace commented on pull request #11906: WIP: [C++] Demonstrate C++ OTel instrumentation
Posted by GitBox <gi...@apache.org>.
westonpace commented on pull request #11906:
URL: https://github.com/apache/arrow/pull/11906#issuecomment-989271720
Asking users to run a separate docker container to get logs feels a bit heavyweight, especially for Windows. I do get that this is more flexible but how much more work would it be to have a naive ability to dump to a file from Arrow? This can be a task for another day.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on pull request #11906: WIP: [C++] Demonstrate C++ OTel instrumentation
Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11906:
URL: https://github.com/apache/arrow/pull/11906#issuecomment-989973161
I filed ARROW-15044 for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #11906: WIP: [C++] Demonstrate C++ OTel instrumentation
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11906:
URL: https://github.com/apache/arrow/pull/11906#issuecomment-989010448
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
Thanks for opening a pull request!
If this is not a [minor PR](https://github.com/apache/arrow/blob/master/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW
Opening JIRAs ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
Then could you also rename pull request title in the following format?
ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
See also:
* [Other pull requests](https://github.com/apache/arrow/pulls/)
* [Contribution Guidelines - How to contribute patches](https://arrow.apache.org/docs/developers/contributing.html#how-to-contribute-patches)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on pull request #11906: WIP: [C++] Demonstrate C++ OTel instrumentation
Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11906:
URL: https://github.com/apache/arrow/pull/11906#issuecomment-989293166
Ok, I think we can hook up logging JSON to stdout/stderr then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org