You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/10/17 06:49:43 UTC
[GitHub] [apisix] Gallardot opened a new pull request, #8099: fix(zipkin): send trace IDs with a reject sampling decision
Gallardot opened a new pull request, #8099:
URL: https://github.com/apache/apisix/pull/8099
### Description
https://github.com/apache/apisix/issues/7795
<!-- Please include a summary of the change and which issue is fixed. -->
<!-- Please also include relevant motivation and context. -->
Fixes # (issue)
### Checklist
- [ ] I have explained the need for this PR and the problem it solves
- [ ] I have explained the changes or the new features added to this PR
- [ ] I have added tests corresponding to this change
- [ ] I have updated the documentation to reflect this change
- [ ] I have verified that this change is backward compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first)
<!--
Note
1. Mark the PR as draft until it's ready to be reviewed.
2. Always add/update tests for any changes unless you have a good reason.
3. Always update the documentation to reflect the changes made in the PR.
4. Make a new commit to resolve conversations instead of `push -f`.
5. To resolve merge conflicts, merge master instead of rebasing.
6. Use "request review" to notify the reviewer after making changes.
7. Only a reviewer can mark a conversation as resolved.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #8099: fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r1000071567
##########
apisix/plugins/zipkin.lua:
##########
@@ -163,9 +165,9 @@ function _M.rewrite(plugin_conf, ctx)
-- pass the trace ids even the sample is rejected
-- see https://github.com/openzipkin/b3-propagation#why-send-
-- trace-ids-with-a-reject-sampling-decision
- core.request.set_header(ctx, "x-b3-traceid", trace_id)
- core.request.set_header(ctx, "x-b3-parentspanid", parent_span_id)
- core.request.set_header(ctx, "x-b3-spanid", request_span_id)
+ core.request.set_header(ctx, "x-b3-traceid", trace_id or to_hex(rand_bytes(16)))
Review Comment:
There is no standard way to do sampling in zipkin. But by reading the code of Kong / Envoy / Traefik, they both choose the "correct steps" we described in https://github.com/apache/apisix/pull/8099#discussion_r999020747.
The benefit of this way is that we don't need to duplicate header set operations in both sampling and no sampling path, especially when we need to introduce more flags in the future. The fake span id solution is not so popular.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #8099: [WIP]fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r1002897924
##########
apisix/plugins/zipkin/codec.lua:
##########
@@ -97,11 +97,13 @@ local function new_injector()
headers["x-b3-parentspanid"] = span_context.parent_id
and to_hex(span_context.parent_id) or nil
headers["x-b3-spanid"] = to_hex(span_context.span_id)
- -- when we call this function, we already start to sample
- headers["x-b3-sampled"] = "1"
+ headers["x-b3-sampled"] = span_context:get_baggage_item("x-b3-sampled")
for key, value in span_context:each_baggage_item() do
- -- XXX: https://github.com/opentracing/specification/issues/117
- headers["uberctx-"..key] = ngx.escape_uri(value)
+ -- skip x-b3-sampled baggage
+ if key ~= "x-b3-sampled" then
+ -- XXX: https://github.com/opentracing/specification/issues/117
+ headers["uberctx-"..key] = ngx.escape_uri(value)
Review Comment:
@Gallardot
Sorry, it is my mistake, I didn't realize it. The current solution is good enough to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Gallardot commented on a diff in pull request #8099: fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
Gallardot commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r999158464
##########
apisix/plugins/zipkin.lua:
##########
@@ -163,9 +165,9 @@ function _M.rewrite(plugin_conf, ctx)
-- pass the trace ids even the sample is rejected
-- see https://github.com/openzipkin/b3-propagation#why-send-
-- trace-ids-with-a-reject-sampling-decision
- core.request.set_header(ctx, "x-b3-traceid", trace_id)
- core.request.set_header(ctx, "x-b3-parentspanid", parent_span_id)
- core.request.set_header(ctx, "x-b3-spanid", request_span_id)
+ core.request.set_header(ctx, "x-b3-traceid", trace_id or to_hex(rand_bytes(16)))
Review Comment:
I'm sorry, I didn't get your point. @spacewander
Why do we need to generate a real root span when no sampling is required and the client does not pass a trace ID & span ID? Where does this real root span need to be used when no sampling is required? We do not need to report these spans to the zipkin server.
A fake trace ID & span ID are sufficient. Only `upstream headers` and `apisix's logs` need these ids
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #8099: [WIP]fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r1002703235
##########
apisix/plugins/zipkin/codec.lua:
##########
@@ -97,11 +97,13 @@ local function new_injector()
headers["x-b3-parentspanid"] = span_context.parent_id
and to_hex(span_context.parent_id) or nil
headers["x-b3-spanid"] = to_hex(span_context.span_id)
- -- when we call this function, we already start to sample
- headers["x-b3-sampled"] = "1"
+ headers["x-b3-sampled"] = span_context:get_baggage_item("x-b3-sampled")
for key, value in span_context:each_baggage_item() do
- -- XXX: https://github.com/opentracing/specification/issues/117
- headers["uberctx-"..key] = ngx.escape_uri(value)
+ -- skip x-b3-sampled baggage
+ if key ~= "x-b3-sampled" then
+ -- XXX: https://github.com/opentracing/specification/issues/117
+ headers["uberctx-"..key] = ngx.escape_uri(value)
Review Comment:
Why does the uberctx depend on "x-b3-sampled"?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #8099: fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r1006466361
##########
t/plugin/zipkin2.t:
##########
@@ -126,8 +126,9 @@ b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-0-05e3ac9a4f6e3b90
--- response_headers
x-b3-sampled: 0
x-b3-traceid: 80f198ee56343ba864fe8b2a57d3eff7
-x-b3-parentspanid: 05e3ac9a4f6e3b90
-x-b3-spanid: e457b5a2e4d86bd1
+x-b3-parentspanid: e457b5a2e4d86bd1
+--- raw_response_headers_like:
Review Comment:
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Gallardot commented on pull request #8099: fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
Gallardot commented on PR #8099:
URL: https://github.com/apache/apisix/pull/8099#issuecomment-1289921126
> Let's update the test file to make CI pass.
@spacewander All CI pass. PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #8099: fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r999020747
##########
apisix/plugins/zipkin.lua:
##########
@@ -163,9 +165,9 @@ function _M.rewrite(plugin_conf, ctx)
-- pass the trace ids even the sample is rejected
-- see https://github.com/openzipkin/b3-propagation#why-send-
-- trace-ids-with-a-reject-sampling-decision
- core.request.set_header(ctx, "x-b3-traceid", trace_id)
- core.request.set_header(ctx, "x-b3-parentspanid", parent_span_id)
- core.request.set_header(ctx, "x-b3-spanid", request_span_id)
+ core.request.set_header(ctx, "x-b3-traceid", trace_id or to_hex(rand_bytes(16)))
Review Comment:
Yes, I am wrong about the root span.
After some investigation, I figure out that:
Currently, APISIX handles root span like this:
1. check if need to sample or not
2. if not, set some upstream header and go to the next plugin
3. generate root span
The correct behavior should be:
1. check if need to sample or not
2. generate root span
3. set some upstream header (as the sampling should only affect the tracing data report)
In step 2 of APISIX, the root span is missing so we have to create a fake span ID in this PR.
Maybe we need to do some bigger changes to solve this problem?
For example, update
https://github.com/apache/apisix/blob/490e0ba995fb0bfe4c1b22eec424f58ebef39446/apisix/plugins/zipkin/codec.lua#L101
https://github.com/apache/apisix/blob/490e0ba995fb0bfe4c1b22eec424f58ebef39446/apisix/plugins/zipkin/reporter.lua#L58
to make it work correctly.
Otherwise, the generated fake span in this PR doesn't have a real span associated with it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Gallardot commented on a diff in pull request #8099: fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
Gallardot commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r1006321751
##########
t/plugin/zipkin2.t:
##########
@@ -126,8 +126,9 @@ b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-0-05e3ac9a4f6e3b90
--- response_headers
x-b3-sampled: 0
x-b3-traceid: 80f198ee56343ba864fe8b2a57d3eff7
-x-b3-parentspanid: 05e3ac9a4f6e3b90
-x-b3-spanid: e457b5a2e4d86bd1
+x-b3-parentspanid: e457b5a2e4d86bd1
+--- raw_response_headers_like:
Review Comment:
@spacewander Sorry, I'm not familiar with this testing framework. I need some help.
The expected test result is that the `x-b3-spanid` header exists and is not empty. Because `x-b3-spanid` header's value is a random value.
May be look like this ?
```
--- raw_response_headers_like: x-b3-spanid: .*
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #8099: fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r997736222
##########
apisix/plugins/zipkin.lua:
##########
@@ -163,9 +165,9 @@ function _M.rewrite(plugin_conf, ctx)
-- pass the trace ids even the sample is rejected
-- see https://github.com/openzipkin/b3-propagation#why-send-
-- trace-ids-with-a-reject-sampling-decision
- core.request.set_header(ctx, "x-b3-traceid", trace_id)
- core.request.set_header(ctx, "x-b3-parentspanid", parent_span_id)
- core.request.set_header(ctx, "x-b3-spanid", request_span_id)
+ core.request.set_header(ctx, "x-b3-traceid", trace_id or to_hex(rand_bytes(16)))
Review Comment:
I am confused. Why do we generate a trace ID when the sampling is rejected?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #8099: fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r997810495
##########
apisix/plugins/zipkin.lua:
##########
@@ -163,9 +165,9 @@ function _M.rewrite(plugin_conf, ctx)
-- pass the trace ids even the sample is rejected
-- see https://github.com/openzipkin/b3-propagation#why-send-
-- trace-ids-with-a-reject-sampling-decision
- core.request.set_header(ctx, "x-b3-traceid", trace_id)
- core.request.set_header(ctx, "x-b3-parentspanid", parent_span_id)
- core.request.set_header(ctx, "x-b3-spanid", request_span_id)
+ core.request.set_header(ctx, "x-b3-traceid", trace_id or to_hex(rand_bytes(16)))
Review Comment:
After reading the OpenTracing material, I can understand the reason.
The current implementation of APISIX is incomplete. For example, we don't have the ability to generate a root span. We require the client to pass a trace ID & span ID. Maybe we should add the ability to generate trace ID/span ID if not given instead of just generating a trace ID when the sampling is rejected
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Gallardot commented on a diff in pull request #8099: fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
Gallardot commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r998167475
##########
apisix/plugins/zipkin.lua:
##########
@@ -163,9 +165,9 @@ function _M.rewrite(plugin_conf, ctx)
-- pass the trace ids even the sample is rejected
-- see https://github.com/openzipkin/b3-propagation#why-send-
-- trace-ids-with-a-reject-sampling-decision
- core.request.set_header(ctx, "x-b3-traceid", trace_id)
- core.request.set_header(ctx, "x-b3-parentspanid", parent_span_id)
- core.request.set_header(ctx, "x-b3-spanid", request_span_id)
+ core.request.set_header(ctx, "x-b3-traceid", trace_id or to_hex(rand_bytes(16)))
Review Comment:
thanks, @spacewander
> For example, we don't have the ability to generate a root span. We require the client to pass a trace ID & span ID.
APISIX has the ability to generate root spans, even if the client does not pass a trace ID & span ID. If we build such a simple route `curl->apisix->service`. We will find the root span generated by APISIX in the zipkin server.
> Maybe we should add the ability to generate trace ID/span ID if not given instead of just generating a trace ID when the sampling is rejected
It is better to have this ability, but it is the same as the current PR effect, only the logical difference in the processing process.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Gallardot commented on a diff in pull request #8099: fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
Gallardot commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r1003151386
##########
apisix/plugins/zipkin/codec.lua:
##########
@@ -97,11 +97,13 @@ local function new_injector()
headers["x-b3-parentspanid"] = span_context.parent_id
and to_hex(span_context.parent_id) or nil
headers["x-b3-spanid"] = to_hex(span_context.span_id)
- -- when we call this function, we already start to sample
- headers["x-b3-sampled"] = "1"
+ headers["x-b3-sampled"] = span_context:get_baggage_item("x-b3-sampled")
for key, value in span_context:each_baggage_item() do
- -- XXX: https://github.com/opentracing/specification/issues/117
- headers["uberctx-"..key] = ngx.escape_uri(value)
+ -- skip x-b3-sampled baggage
+ if key ~= "x-b3-sampled" then
+ -- XXX: https://github.com/opentracing/specification/issues/117
+ headers["uberctx-"..key] = ngx.escape_uri(value)
Review Comment:
@spacewander I have no idea why the CI about Kafka is fail. Can you help to run it again? PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander merged pull request #8099: fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
spacewander merged PR #8099:
URL: https://github.com/apache/apisix/pull/8099
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #8099: fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r1005268568
##########
t/plugin/zipkin2.t:
##########
@@ -136,6 +137,7 @@ x-b3-spanid: e457b5a2e4d86bd1
b3: 0
--- response_headers
x-b3-sampled: 0
+--- raw_response_headers_unlike
Review Comment:
Let's use `raw_response_headers_like` too
##########
t/plugin/zipkin2.t:
##########
@@ -126,8 +126,9 @@ b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-0-05e3ac9a4f6e3b90
--- response_headers
x-b3-sampled: 0
x-b3-traceid: 80f198ee56343ba864fe8b2a57d3eff7
-x-b3-parentspanid: 05e3ac9a4f6e3b90
-x-b3-spanid: e457b5a2e4d86bd1
+x-b3-parentspanid: e457b5a2e4d86bd1
+--- raw_response_headers_like:
Review Comment:
It should be:
```
--- raw_response_headers_like
.*x-b3-spanid:.*
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Gallardot commented on a diff in pull request #8099: fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
Gallardot commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r1006374400
##########
t/plugin/zipkin2.t:
##########
@@ -136,6 +137,7 @@ x-b3-spanid: e457b5a2e4d86bd1
b3: 0
--- response_headers
x-b3-sampled: 0
+--- raw_response_headers_unlike
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Gallardot commented on a diff in pull request #8099: [WIP]fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
Gallardot commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r1002854199
##########
apisix/plugins/zipkin/codec.lua:
##########
@@ -97,11 +97,13 @@ local function new_injector()
headers["x-b3-parentspanid"] = span_context.parent_id
and to_hex(span_context.parent_id) or nil
headers["x-b3-spanid"] = to_hex(span_context.span_id)
- -- when we call this function, we already start to sample
- headers["x-b3-sampled"] = "1"
+ headers["x-b3-sampled"] = span_context:get_baggage_item("x-b3-sampled")
for key, value in span_context:each_baggage_item() do
- -- XXX: https://github.com/opentracing/specification/issues/117
- headers["uberctx-"..key] = ngx.escape_uri(value)
+ -- skip x-b3-sampled baggage
+ if key ~= "x-b3-sampled" then
+ -- XXX: https://github.com/opentracing/specification/issues/117
+ headers["uberctx-"..key] = ngx.escape_uri(value)
Review Comment:
The uberctx does not depend on "x-b3-sampled".
ref: https://github.com/apache/apisix/pull/8099/files#diff-d7b683eff2eae613bb495064140483bd2d223010b83563842f57433f1501cc64R205-R210
baggage is used to pass "x-b3-sampled", so we need to skip this item. Or any suggestions? Looking forward to your reply. @spacewander
ps: I will fix the CI later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #8099: fix(zipkin): send trace IDs with a reject sampling decision
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #8099:
URL: https://github.com/apache/apisix/pull/8099#discussion_r1004049257
##########
t/plugin/zipkin2.t:
##########
@@ -126,8 +126,9 @@ b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-0-05e3ac9a4f6e3b90
--- response_headers
x-b3-sampled: 0
x-b3-traceid: 80f198ee56343ba864fe8b2a57d3eff7
-x-b3-parentspanid: 05e3ac9a4f6e3b90
-x-b3-spanid: e457b5a2e4d86bd1
+x-b3-parentspanid: e457b5a2e4d86bd1
+--- raw_response_headers_unlike
+x-b3-spanid:
Review Comment:
It can pass even without x-b3-spanid.
I will recommend:
```
--- raw_response_headers_like
.*x-b3-spanid:.*
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org