You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/06/06 07:53:10 UTC
[GitHub] [apisix] gotosre opened a new issue, #7198: bug: apisix kafka-logger output json array, not json object
gotosre opened a new issue, #7198:
URL: https://github.com/apache/apisix/issues/7198
### Current Behavior
[
{第一条日志},
{第二条日志},
...
{第N条日志}
]
### Expected Behavior
{第一条日志}
{第二条日志}
### Error Logs
_No response_
### Steps to Reproduce
not clear
### Environment
apisix 2.9
kafka-logger metadata:
`{"count":"1","action":"get","node":{"key":"\/apisix\/plugin_metadata\/kafka-logger","value":{"log_format":{"response_time":"$upstream_response_time","x-screen-width":"$http_x_screen_width","x-request-id":"$request_id","x-channel":"$http_x_keep_channel","x-timezone":"$http_x_keep_timezone","request":"$request_uri","@timestamp":"$time_iso8601","body_bytes_sent":"$body_bytes_sent","http_upstream":"$upstream_addr","x-version-code":"$http_x_version_code","x-user-id":"$http_x_user_id","status":"$status","hostname":"$hostname","http2":"$http2","x-device-id":"$http_x_device_id","sreauth":"$http_sreauth","x-os":"$http_x_os","grafana-user-name":"$http_x_user_name","x-model":"$http_x_model","request_time":"$request_time","http_user_agent":"$http_user_agent","x-manufacturer":"$http_x_manufacturer","http_host":"$host","x-screen-height":"$http_x_screen_height","request_length":"$request_length","grafana-panel-id":"$http_x_panel_id","scheme":"$scheme","grafana-dashboard-id":"$http_x_dashboard_i
d","request_body":"$request_body","remote_addr":"$remote_addr","upstream_name":"$service_name","http_referer":"$http_referer","connection":"$connection","method":"$request_method","x-version-name":"$http_x_version_name","content-type":"$http_content_type","cookie":"$http_cookie","http_auth":"$http_authorization","x-os-version":"$http_x_os_version","http_x_forwarded_for":"$http_x_forwarded_for"}}}}`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] soulbird commented on issue #7198: bug: apisix kafka-logger output json array, not json object
Posted by GitBox <gi...@apache.org>.
soulbird commented on issue #7198:
URL: https://github.com/apache/apisix/issues/7198#issuecomment-1147195810
We use the method of uploading logs in batches instead of sending them individually, which will affect performance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander closed issue #7198: bug: apisix kafka-logger output json array, not json object
Posted by GitBox <gi...@apache.org>.
spacewander closed issue #7198: bug: apisix kafka-logger output json array, not json object
URL: https://github.com/apache/apisix/issues/7198
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] soulbird commented on issue #7198: bug: apisix kafka-logger output json array, not json object
Posted by GitBox <gi...@apache.org>.
soulbird commented on issue #7198:
URL: https://github.com/apache/apisix/issues/7198#issuecomment-1148151349
Apache APISIX uses a custom batch-processor: https://github.com/apache/apisix/blob/master/docs/en/latest/batch-processor.md. You can set `batch_max_size` to 1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #7198: bug: apisix kafka-logger output json array, not json object
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #7198:
URL: https://github.com/apache/apisix/issues/7198#issuecomment-1158687949
Consider solved. Feel free to reopen it if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gotosre commented on issue #7198: bug: apisix kafka-logger output json array, not json object
Posted by GitBox <gi...@apache.org>.
gotosre commented on issue #7198:
URL: https://github.com/apache/apisix/issues/7198#issuecomment-1147228085
> We use the method of uploading logs in batches instead of sending them individually, which will affect performance.
kafka producer support batch producing in native
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #7198: bug: apisix kafka-logger output json array, not json object
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #7198:
URL: https://github.com/apache/apisix/issues/7198#issuecomment-1148250528
Even the kafka's producer don't need to know the length of logs, the application that consumes the log still needs it. So we need json array.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] soulbird commented on issue #7198: bug: apisix kafka-logger output json array, not json object
Posted by GitBox <gi...@apache.org>.
soulbird commented on issue #7198:
URL: https://github.com/apache/apisix/issues/7198#issuecomment-1148156917
Maybe we can let users choose to use kafka's [batch method](https://github.com/doujiang24/lua-resty-kafka#send) or APISIX custom's batch-processor.cc @bzp2010 @spacewander
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gotosre commented on issue #7198: bug: apisix kafka-logger output json array, not json object
Posted by GitBox <gi...@apache.org>.
gotosre commented on issue #7198:
URL: https://github.com/apache/apisix/issues/7198#issuecomment-1148261820
> Apache APISIX uses a custom batch-processor: https://github.com/apache/apisix/blob/master/docs/en/latest/batch-processor.md. You can set `batch_max_size` to 1
love u.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gotosre commented on issue #7198: bug: apisix kafka-logger output json array, not json object
Posted by GitBox <gi...@apache.org>.
gotosre commented on issue #7198:
URL: https://github.com/apache/apisix/issues/7198#issuecomment-1148261549
> batch_max_size
ty，栓Q，it works
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] gotosre commented on issue #7198: bug: apisix kafka-logger output json array, not json object
Posted by GitBox <gi...@apache.org>.
gotosre commented on issue #7198:
URL: https://github.com/apache/apisix/issues/7198#issuecomment-1148263289
> Even the kafka's producer don't need to know the length of logs, the application that consumes the log still needs it. So we need json array.
but not all log collector support this style.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org