You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/08/15 07:36:20 UTC
[GitHub] [apisix] akalittle opened a new issue, #7674: help request: opentelemetry error in lua script
akalittle opened a new issue, #7674:
URL: https://github.com/apache/apisix/issues/7674
### Description
When I try to implement opentelemetry into my route.
There are some errors in the Lua script.
<img width="443" alt="image" src="https://user-images.githubusercontent.com/38489160/184594640-4c1f4d68-24b7-4c79-bb1e-7fc1399af226.png">
```yaml
plugins:
- api-breaker
- authz-keycloak
- basic-auth
- ...
- opentelemetry
pluginAttrs:
opentelemetry:
# trace_id_source: x-api-key
resource:
service.name: APISIX
# tenant.id: business_id
collector:
address: opentelemetry-deploy-opentelemetry-collector.apisix-istio.svc.cluster.local:4318
request_timeout: 3
batch_span_processor:
drop_on_queue_full: false
max_queue_size: 6
batch_timeout: 2
inactive_timeout: 1
max_export_batch_size: 2
```
```
2022/08/15 07:28:33 [error] 49#49: *9205 lua entry thread aborted: runtime error: bad argument #1 to '?' (invalid value)
stack traceback:
coroutine 0:
[C]: in function 'gsub'
...deps/share/lua/5.1/opentelemetry/trace/exporter/otlp.lua:18: in function 'hex2bytes'
...deps/share/lua/5.1/opentelemetry/trace/exporter/otlp.lua:50: in function 'export_spans'
...are/lua/5.1/opentelemetry/trace/batch_span_processor.lua:18: in function 'process_batches'
...are/lua/5.1/opentelemetry/trace/batch_span_processor.lua:46: in function <...are/lua/5.1/opentelemetry/trace/batch_span_processor.lua:29>, context: ngx.timer, client: 127.0.0.6, server: 0.0.0.0:9080
```
### Environment
apache/apisix:2.15.0-alpine
apache/apisix-ingress-controller:1.4.1
docker.io/bitnami/etcd:3.5.4-debian-11-r14
Kubernetes cluster version:
Client Version: v1.24.0
Kustomize Version: v4.5.4
Server Version: v1.21.12-gke.2200
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kingluo commented on issue #7674: help request: opentelemetry error in lua script
Posted by GitBox <gi...@apache.org>.
kingluo commented on issue #7674:
URL: https://github.com/apache/apisix/issues/7674#issuecomment-1216450883
@akalittle The `hex2bytes` assumes the input string is in valid hex format (`[0-9a-f]{32}`), and transforms each hex u8 number into char. The `char()` assumes the number ranges from 0~255, otherwise it would throw error.
It's likely your `span.ctx.trace_id` (https://github.com/yangxikun/opentelemetry-lua/blob/v0.1.3/lib/opentelemetry/trace/exporter/otlp.lua#L50) contains `-`, which means negative number and cause `char()` failed. It means your trace id comes from `x-request-id` in invalid format.
```lua
local function hex2bytes(str)
return (str:gsub('..', function (cc)
local n = tonumber(cc, 16)
if n then
print(n)
return string.char(n)
end
end))
end
hex2bytes("xx-1")
```
```
/usr/local/openresty-debug/luajit/bin/luajit: bad argument #1 to '?' (invalid value)
stack traceback:
[builtin#80]: at 0x5571dae60530
[C]: in function 'gsub'
/tmp/test_gsub.lua:2: in function 'hex2bytes'
/tmp/test_gsub.lua:11: in main chunk
[C]: at 0x5571dadfc320
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on issue #7674: help request: opentelemetry error in lua script
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on issue #7674:
URL: https://github.com/apache/apisix/issues/7674#issuecomment-1216078151
我来 follow 下这个问题
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on issue #7674: help request: opentelemetry error in lua script
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on issue #7674:
URL: https://github.com/apache/apisix/issues/7674#issuecomment-1216188788
could you print out the value of span.ctx.trace_id, span.ctx.span_id and span.parent_ctx.span_id before error occur?
https://github.com/yangxikun/opentelemetry-lua/blob/main/lib/opentelemetry/trace/exporter/otlp.lua#L75
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] akalittle closed issue #7674: help request: opentelemetry error in lua script
Posted by GitBox <gi...@apache.org>.
akalittle closed issue #7674: help request: opentelemetry error in lua script
URL: https://github.com/apache/apisix/issues/7674
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers commented on issue #7674: help request: opentelemetry error in lua script
Posted by GitBox <gi...@apache.org>.
tokers commented on issue #7674:
URL: https://github.com/apache/apisix/issues/7674#issuecomment-1216249581
> I'm sure I just use system generated trace_id. In my yaml file, I just simply use like below, I didn't use any other attributes, you can also see in the json object.
>
> ```yaml
> - name: opentelemetry
> enable: true
> config:
> sampler:
> name: always_on
> ```
>
> Here is a request header example: <img alt="image" width="767" src="https://user-images.githubusercontent.com/38489160/184805662-e1911917-f998-45e8-aba0-ee499e2f9694.png">
@akalittle It's not the config from the `ApisixRoute` but the APISIX `config-default.yaml`. By default the trace id is generated from x-request-id.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on issue #7674: help request: opentelemetry error in lua script
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on issue #7674:
URL: https://github.com/apache/apisix/issues/7674#issuecomment-1216186951
@akalittle could you print out the value of
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #7674: help request: opentelemetry error in lua script
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #7674:
URL: https://github.com/apache/apisix/issues/7674#issuecomment-1215051363
@akalittle Can you provide the routing configuration in text format? Do not provide images.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] juzhiyuan commented on issue #7674: help request: opentelemetry error in lua script
Posted by GitBox <gi...@apache.org>.
juzhiyuan commented on issue #7674:
URL: https://github.com/apache/apisix/issues/7674#issuecomment-1215026817
FYI: The related Slack link[1].
[1] https://the-asf.slack.com/archives/CUC5MN17A/p1660569995134259
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on issue #7674: help request: opentelemetry error in lua script
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on issue #7674:
URL: https://github.com/apache/apisix/issues/7674#issuecomment-1216078334
Let me follow this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] akalittle commented on issue #7674: help request: opentelemetry error in lua script
Posted by GitBox <gi...@apache.org>.
akalittle commented on issue #7674:
URL: https://github.com/apache/apisix/issues/7674#issuecomment-1218080705
It can be closed now.
[default config](https://github.com/apache/apisix/blob/master/conf/config-default.yaml#L506)
the default value is `trace_id_source ` is `x-request-id`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] akalittle commented on issue #7674: help request: opentelemetry error in lua script
Posted by GitBox <gi...@apache.org>.
akalittle commented on issue #7674:
URL: https://github.com/apache/apisix/issues/7674#issuecomment-1216158802
I'm sure I just use system generated trace_id.
In my yaml file, I just simply use like below, I didn't use any other attributes, you can also see in the json object.
```yaml
- name: opentelemetry
enable: true
config:
sampler:
name: always_on
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #7674: help request: opentelemetry error in lua script
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #7674:
URL: https://github.com/apache/apisix/issues/7674#issuecomment-1216123324
In any case, I think the immediate cause is that `string.char()` is converting number that over the range 0..255.
here is the test cases:
```
use t::APISIX 'no_plan';
add_block_preprocessor(sub {
my ($block) = @_;
if (!$block->request) {
$block->set_value("request", "GET /t");
}
if (!defined $block->response_body) {
$block->set_value("response_body", "passed\n");
}
if (!$block->no_error_log && !$block->error_log) {
$block->set_value("no_error_log", "[error]");
}
$block;
});
repeat_each(1);
no_long_string();
no_root_location();
log_level("info");
run_tests;
__DATA__
=== TEST 19: set additional_attributes
--- config
location /t {
content_by_lua_block {
local id_generator = require("opentelemetry.trace.id_generator")
local trace_id =id_generator.new_ids()
-- mock hex2bytes
local function hex2bytes(str)
return (str:gsub('..', function (cc)
local n = tonumber(cc, 16)
if n then
-- hack n to be over the range 0..255
return string.char(256)
end
end))
end
hex2bytes(trace_id)
ngx.say("done")
}
}
--- response_body
done
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #7674: help request: opentelemetry error in lua script
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #7674:
URL: https://github.com/apache/apisix/issues/7674#issuecomment-1216126760
@akalittle from your configuration, it looks like you commented out
```
opentelemetry:
trace_id_source: x-request-id
```
This means that the system-generated `trace_id`(https://github.com/yangxikun/opentelemetry-lua/blob/463a70993b2a87b9ab13a49322f6ec6089af55d2/lib/opentelemetry/trace/id_generator.lua#L22-L40) is used.
Please make sure that in your environment the trace_id is not obtained from the x-request-id of the request, but is system generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] akalittle commented on issue #7674: help request: opentelemetry error in lua script
Posted by GitBox <gi...@apache.org>.
akalittle commented on issue #7674:
URL: https://github.com/apache/apisix/issues/7674#issuecomment-1215131247
CRD:
```yaml
apiVersion: apisix.apache.org/v2beta3
kind: ApisixRoute
metadata:
name: status-tracker
namespace: launcher
spec:
http:
- name: status-tracker-rule
match:
hosts:
- api.xxx.cc
paths:
- /status-tracker*
plugins:
- name: proxy-rewrite
enable: true
config:
regex_uri:
- ^/status-tracker/(.*)
- /$1
- name: limit-count
enable: true
config:
count: 100
time_window: 60
- name: opentelemetry
enable: true
config:
sampler:
name: always_on
backends:
- serviceName: status-tracker
servicePort: 8000
# authentication:
# enable: true
# type: keyAuth
```
in json format:
```json
{
"uris": [
"/status-tracker*"
],
"name": "launcher_status-tracker_status-tracker-rule",
"desc": "Created by apisix-ingress-controller, DO NOT modify it manually",
"hosts": [
"api.xxx.cc"
],
"plugins": {
"limit-count": {
"allow_degradation": false,
"count": 100,
"key": "remote_addr",
"key_type": "var",
"policy": "local",
"rejected_code": 503,
"show_limit_quota_header": true,
"time_window": 60
},
"opentelemetry": {
"sampler": {
"name": "always_on",
"options": {
"fraction": 0,
"root": {
"name": "always_off"
}
}
}
},
"proxy-rewrite": {
"regex_uri": [
"^/status-tracker/(.*)",
"/$1"
],
"use_real_request_uri_unsafe": false
}
},
"upstream_id": "b407a3",
"labels": {
"managed-by": "apisix-ingress-controller"
},
"status": 1
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org