You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "ThalysonR (via GitHub)" <gi...@apache.org> on 2023/06/08 01:32:29 UTC
[GitHub] [apisix] ThalysonR opened a new issue, #9270: help request: opentelemetry + grpc-transcode not working
ThalysonR opened a new issue, #9270:
URL: https://github.com/apache/apisix/issues/9270
### Description
I've been testing apisix and tried using otel plugin while transcoding to grpc and got an error. I did try using the otel plugin with http and it works fine, so I assume the problem is with the 2 plugins working together. This is the error i got:
```
2023/04/08 05:21:57 [error] 56#56: *4242 lua entry thread aborted: runtime error: ...deps/share/lua/5.1/opentelemetry/trace/exporter/otlp.lua:104: bad argument #1 to 'encode' (type 'opentelemetry.proto.trace.v1.TracesData' does not exists)
stack traceback:
coroutine 0:
[C]: in function 'encode'
...deps/share/lua/5.1/opentelemetry/trace/exporter/otlp.lua:104: in function 'export_spans'
...are/lua/5.1/opentelemetry/trace/batch_span_processor.lua:45: in function 'process_batches'
...are/lua/5.1/opentelemetry/trace/batch_span_processor.lua:77: in function <...are/lua/5.1/opentelemetry/trace/batch_span_processor.lua:57>, context: ngx.timer, client: 172.20.0.1, server: 0.0.0.0:9080
```
This is my config:
```
# config.yaml
plugins:
- grpc-transcode
- opentelemetry
opentelemetry:
resource:
service.name: APISIX
tenant.id: business_id
collector:
address: otel-collector:4318
request_timeout: 3
request_headers:
foo: bar
batch_span_processor:
drop_on_queue_full: false
max_queue_size: 6
batch_timeout: 2
inactive_timeout: 1
max_export_batch_size: 2
```
```
# apisix.yaml
routes:
- id: users
methods: [GET]
uri: /users
plugins:
grpc-transcode:
proto_id: "1"
service: users.UsersService
method: FindAll
opentelemetry:
sampler:
name: always_on
upstream:
scheme: grpc
type: roundrobin
nodes:
"users:5000": 1
```
The transcoding part works fine, I'm able to get the expected response from the grpc service, just missing the otel trace.
Am I maybe using the plugins incorrectly?
### Environment
- APISIX version (run `apisix version`): 3.2.0 docker image
- Operating system (run `uname -a`): Ubuntu WSL2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] shreemaan-abhishek commented on issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "shreemaan-abhishek (via GitHub)" <gi...@apache.org>.
shreemaan-abhishek commented on issue #9270:
URL: https://github.com/apache/apisix/issues/9270#issuecomment-1519026115
On it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] coffeebe4code commented on issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "coffeebe4code (via GitHub)" <gi...@apache.org>.
coffeebe4code commented on issue #9270:
URL: https://github.com/apache/apisix/issues/9270#issuecomment-1574270228
Any progress on this issue, also facing open telemetry issues with the same reported error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Jamel-jun commented on issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "Jamel-jun (via GitHub)" <gi...@apache.org>.
Jamel-jun commented on issue #9270:
URL: https://github.com/apache/apisix/issues/9270#issuecomment-1575896655
When will this problem be repaired? Now it may affect me in production, and I feel sad about this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] ThalysonR commented on issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "ThalysonR (via GitHub)" <gi...@apache.org>.
ThalysonR commented on issue #9270:
URL: https://github.com/apache/apisix/issues/9270#issuecomment-1520785711
@shreemaan-abhishek I was trying to get regular OTLP traces, the GRPC transcoding was being used to reach GRPC services
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kingluo commented on issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "kingluo (via GitHub)" <gi...@apache.org>.
kingluo commented on issue #9270:
URL: https://github.com/apache/apisix/issues/9270#issuecomment-1522071173
It's a bug.
grpc-transcode plugin will compile the configured proto but does not save its original one and restore it later.
https://github.com/apache/apisix/blob/1434335041c6e054f7b6afb0401aea0a069b0ad7/apisix/plugins/grpc-transcode/proto.lua#L100-L102
Then in turn it discards the otel proto compiled at startup:
https://github.com/yangxikun/opentelemetry-lua/blob/bb56b3c8e2163711a763c871ae702b36ee09c557/lib/opentelemetry/trace/exporter/pb.lua#L4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] shreemaan-abhishek commented on issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "shreemaan-abhishek (via GitHub)" <gi...@apache.org>.
shreemaan-abhishek commented on issue #9270:
URL: https://github.com/apache/apisix/issues/9270#issuecomment-1521087825
Thanks, I was able to reproduce this issue. Let me take a look for a fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kingluo commented on issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "kingluo (via GitHub)" <gi...@apache.org>.
kingluo commented on issue #9270:
URL: https://github.com/apache/apisix/issues/9270#issuecomment-1581835493
cc @coffeebe4code @Jamel-jun The bugfix was merged, please check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 closed issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 closed issue #9270: help request: opentelemetry + grpc-transcode not working
URL: https://github.com/apache/apisix/issues/9270
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] shreemaan-abhishek commented on issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "shreemaan-abhishek (via GitHub)" <gi...@apache.org>.
shreemaan-abhishek commented on issue #9270:
URL: https://github.com/apache/apisix/issues/9270#issuecomment-1520567336
Note that open telemetry only supports binary-encoded [OLTP over HTTP](https://opentelemetry.io/docs/reference/specification/protocol/otlp/#otlphttp).
So I wanted to know if you were using grpc-transcode to support OLTP over GRPC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 commented on issue #9270:
URL: https://github.com/apache/apisix/issues/9270#issuecomment-1596369113
considered resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Sn0rt commented on issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "Sn0rt (via GitHub)" <gi...@apache.org>.
Sn0rt commented on issue #9270:
URL: https://github.com/apache/apisix/issues/9270#issuecomment-1517317165
@shreemaan-abhishek help me talk a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] leslie-tsang commented on issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "leslie-tsang (via GitHub)" <gi...@apache.org>.
leslie-tsang commented on issue #9270:
URL: https://github.com/apache/apisix/issues/9270#issuecomment-1581756015
@ThalysonR Please take a look. :) #9606
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 closed issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 closed issue #9270: help request: opentelemetry + grpc-transcode not working
URL: https://github.com/apache/apisix/issues/9270
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] leslie-tsang commented on issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "leslie-tsang (via GitHub)" <gi...@apache.org>.
leslie-tsang commented on issue #9270:
URL: https://github.com/apache/apisix/issues/9270#issuecomment-1575910179
> When will this problem be repaired? Now it may affect me in production, and I feel sad about this.
Yes, @kingluo Please take a look. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] shreemaan-abhishek commented on issue #9270: help request: opentelemetry + grpc-transcode not working
Posted by "shreemaan-abhishek (via GitHub)" <gi...@apache.org>.
shreemaan-abhishek commented on issue #9270:
URL: https://github.com/apache/apisix/issues/9270#issuecomment-1521817034
@kingluo, should we support this or it is by design?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org