You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "zhouqt0528 (via GitHub)" <gi...@apache.org> on 2023/03/06 02:40:12 UTC
[GitHub] [skywalking] zhouqt0528 opened a new issue, #10485: [Bug]
zhouqt0528 opened a new issue, #10485:
URL: https://github.com/apache/skywalking/issues/10485
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar issues.
### Apache SkyWalking Component
OAP server (apache/skywalking)
### What happened
Our project wasing using skywalking to monitor Postgresql slow and found a problem: with "pg_sleep" as (select * from pg_sleep(1.4)) select * from "pg_sleep", This sql was not shown in Show statements, But this sql "with pg_sleep as (select * from pg_sleep(1.4)) select * from pg_sleep" display normally. I debuged running Skywalking oap, It could get a message("with pg_sleep as (select * from pg_sleep(1.4)) select * from pg_sleep"), But it is not stored in the Elasticsearch(We use elasticsearch as the store). That should be a problem.
NOTE:The Skywalking OAP did not show any problems
### What you expected to happen
I expect this type of sql to display normally in the Slow statements(ex sql: with "pg_sleep" as (select * from pg_sleep(1.4)) select * from "pg_sleep"),
Skywalking version: 9.3.0,
fluent-bit version: 2.0.9
fluent-bit log:
[{"date":1677836101.832795,"body":{"json":{"json":"{\"statement\":\" 1404.043 ms statement: with \"pg_sleep\" as (select * from pg_sleep(1.4)) select * from \"pg_sleep\";\",\"query_time\":1404,\"service\":\"postgresql::postgres:5432\",\"id\":\"34bacb9e-aea6-6246-49e5-14ace3d4848c\",\"layer\":\"POSTGRESQL\",\"time\":1677836101000}"}},"tags":{"data":[{"value":"SLOW_SQL","key":"LOG_KIND"}]},"service":"postgresql::postgres:5432","layer":"POSTGRESQL"}]
[2023/03/03 17:36:44] [ info] [output:http:http.1] 10.255.13.133:12800, HTTP status=200
### How to reproduce
Skywalking monitor Postgresql,
Enabling Postgresql slow
execute the following sql
sql:
with "pg_sleep" as (select * from pg_sleep(1.4)) select * from "pg_sleep"
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #10485:
URL: https://github.com/apache/skywalking/issues/10485#issuecomment-1455447419
If you want to address issues, you could debug the log receiver by yourself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] zhouqt0528 commented on issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
Posted by "zhouqt0528 (via GitHub)" <gi...@apache.org>.
zhouqt0528 commented on issue #10485:
URL: https://github.com/apache/skywalking/issues/10485#issuecomment-1455441068
The "\" in the log is removed in the window
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] zhouqt0528 commented on issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
Posted by "zhouqt0528 (via GitHub)" <gi...@apache.org>.
zhouqt0528 commented on issue #10485:
URL: https://github.com/apache/skywalking/issues/10485#issuecomment-1455459643
> If you want to address issues, you could debug the log receiver by yourself.
I debuged running Skywalking oap, It could get a slow sql json(json in the screenshot above), But the slow sql is not stored in the Elasticsearch. The problem is that Skywalking oap handle slow sql json
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #10485: [Bug] Some special Postgresql slow sql was not shown in Show statements
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #10485:
URL: https://github.com/apache/skywalking/issues/10485#issuecomment-1455399435
> fluent-bit log:
[{"date":1677836101.832795,"body":{"json":{"json":"{"statement":" 1404.043 ms statement: with "pg_sleep" as (select * from pg_sleep(1.4)) select * from "pg_sleep";","query_time":1404,"service":"postgresql::postgres:5432","id":"34bacb9e-aea6-6246-49e5-14ace3d4848c","layer":"POSTGRESQL","time":1677836101000}"}},"tags":{"data":[{"value":"SLOW_SQL","key":"LOG_KIND"}]},"service":"postgresql::postgres:5432","layer":"POSTGRESQL"}]
[2023/03/03 17:36:44] [ info] [output:http:http.1] 10.255.13.133:12800, HTTP status=200
This seems not a legal JSON. `"` should be represented as `\"` in the JSON AFAIK.
So, first of all, this could be expected, rather than a bug.
Maybe need to enhance fluentd config? @yswdqz
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] zhouqt0528 commented on issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
Posted by "zhouqt0528 (via GitHub)" <gi...@apache.org>.
zhouqt0528 commented on issue #10485:
URL: https://github.com/apache/skywalking/issues/10485#issuecomment-1455552006
> This description doesn't help. We never process logs based on the content(SQL text).
In the Skyealking OAP,this step is normal, as follows:
but execute to this method(closure.call()), The code didn't continue to go down
Normal logic should execute FilterSpec.extractor, But my debugging did not
my sql: with "pg_sleep" as (select * from pg_sleep(1.4)) select * from "pg_sleep";
flunt-bit log:
You can try the sql I provided
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #10485:
URL: https://github.com/apache/skywalking/issues/10485#issuecomment-1455470956
This description doesn't help. We never process logs based on the content(SQL text).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng closed issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng closed issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
URL: https://github.com/apache/skywalking/issues/10485
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] yswdqz commented on issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
Posted by "yswdqz (via GitHub)" <gi...@apache.org>.
yswdqz commented on issue #10485:
URL: https://github.com/apache/skywalking/issues/10485#issuecomment-1455648384
I have tested local, and the log of fluent-bit is :
```
[{"date":1678088131.451024,"tags":{"data":[{"key":"LOG_KIND","value":"SLOW_SQL"}]},"body":{"json":{"json":"{\"id\":\"cb9c1a5b-691e-fb2f-457a-9c72a392d9ed\",\"layer\":\"POSTGRESQL\",\"statement\":\"with \"pg_sleep\" as (select * from pg_sleep(1.4)) select * from \"pg_sleep\"\",\"time\":1678088131000,\"query_time\":1401,\"service\":\"postgresql::postgres:5432\"}"}},"layer":"POSTGRESQL","service":"postgresql::postgres:5432"}]
```
But your log is
And this seems to have some syntax problems
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] zhouqt0528 commented on issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
Posted by "zhouqt0528 (via GitHub)" <gi...@apache.org>.
zhouqt0528 commented on issue #10485:
URL: https://github.com/apache/skywalking/issues/10485#issuecomment-1455436592
> > fluent-bit log:
> > [{"date":1677836101.832795,"body":{"json":{"json":"{"statement":" 1404.043 ms statement: with "pg_sleep" as (select * from pg_sleep(1.4)) select * from "pg_sleep";","query_time":1404,"service":"postgresql::postgres:5432","id":"34bacb9e-aea6-6246-49e5-14ace3d4848c","layer":"POSTGRESQL","time":1677836101000}"}},"tags":{"data":[{"value":"SLOW_SQL","key":"LOG_KIND"}]},"service":"postgresql::postgres:5432","layer":"POSTGRESQL"}]
> > [2023/03/03 17:36:44] [ info] [output:http:http.1] 10.255.13.133:12800, HTTP status=200
>
> This seems not a legal JSON. `"` should be represented as `\"` in the JSON AFAIK. So, first of all, this could be expected, rather than a bug.
>
> Maybe need to enhance fluentd config? @yswdqz
This is log in flunt-bit:
[{"date":1677838462.133755,"body":{"json":{"json":"{\"statement\":\" 1408.590 ms statement: with \"pg_sleep\" as (select * from pg_sleep(1.4)) select * from \"pg_sleep\";\",\"query_time\":1408,\"service\":\"postgresql::postgres:5432\",\"id\":\"4545983c-b9c4-ebae-4f7e-9e9ca35918de\",\"layer\":\"POSTGRESQL\",\"time\":1677838462000}"}},"tags":{"data":[{"value":"SLOW_SQL","key":"LOG_KIND"}]},"service":"postgresql::postgres:5432","layer":"POSTGRESQL"}]
Skywalking oap can get messages normally and Flunt-bit and Skywalking OAP are normal(not show any problems)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng closed issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng closed issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
URL: https://github.com/apache/skywalking/issues/10485
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] zhouqt0528 commented on issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
Posted by "zhouqt0528 (via GitHub)" <gi...@apache.org>.
zhouqt0528 commented on issue #10485:
URL: https://github.com/apache/skywalking/issues/10485#issuecomment-1455442490
\"pg sleep\" in the log is replaced by "pg sleep" in the window
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] zhouqt0528 commented on issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
Posted by "zhouqt0528 (via GitHub)" <gi...@apache.org>.
zhouqt0528 commented on issue #10485:
URL: https://github.com/apache/skywalking/issues/10485#issuecomment-1455674693
> I have tested local, and the log of fluent-bit is :
>
> ```
> [{"date":1678088131.451024,"tags":{"data":[{"key":"LOG_KIND","value":"SLOW_SQL"}]},"body":{"json":{"json":"{\"id\":\"cb9c1a5b-691e-fb2f-457a-9c72a392d9ed\",\"layer\":\"POSTGRESQL\",\"statement\":\"with \"pg_sleep\" as (select * from pg_sleep(1.4)) select * from \"pg_sleep\"\",\"time\":1678088131000,\"query_time\":1401,\"service\":\"postgresql::postgres:5432\"}"}},"layer":"POSTGRESQL","service":"postgresql::postgres:5432"}]
> ```
>
> But your log is  And this seems to have some syntax problems
It has nothing to do with what you're saying，this sql with pg_sleep as (select * from pg_sleep(1.4)) select * from pg_sleep can display in Slow statements
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] zhouqt0528 commented on issue #10485: [Bug] Some special Postgresql slow sql was not shown in Show statements
Posted by "zhouqt0528 (via GitHub)" <gi...@apache.org>.
zhouqt0528 commented on issue #10485:
URL: https://github.com/apache/skywalking/issues/10485#issuecomment-1455399598
这是来自QQ邮箱的假期自动回复邮件。 您好，我最近正在休假中，无法亲自回复您的邮件。我将在假期结束后，尽快给您回复。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] zhouqt0528 commented on issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
Posted by "zhouqt0528 (via GitHub)" <gi...@apache.org>.
zhouqt0528 commented on issue #10485:
URL: https://github.com/apache/skywalking/issues/10485#issuecomment-1455459261
I debuged running Skywalking oap, It could get a slow sql json(json in the screenshot above), But the slow sql is not stored in the Elasticsearch. The problem is that Skywalking oap handle slow sql json
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] zhouqt0528 commented on issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
Posted by "zhouqt0528 (via GitHub)" <gi...@apache.org>.
zhouqt0528 commented on issue #10485:
URL: https://github.com/apache/skywalking/issues/10485#issuecomment-1455679353
Please help explain,
> > I have tested local, and the log of fluent-bit is :
> > ```
> > [{"date":1678088131.451024,"tags":{"data":[{"key":"LOG_KIND","value":"SLOW_SQL"}]},"body":{"json":{"json":"{\"id\":\"cb9c1a5b-691e-fb2f-457a-9c72a392d9ed\",\"layer\":\"POSTGRESQL\",\"statement\":\"with \"pg_sleep\" as (select * from pg_sleep(1.4)) select * from \"pg_sleep\"\",\"time\":1678088131000,\"query_time\":1401,\"service\":\"postgresql::postgres:5432\"}"}},"layer":"POSTGRESQL","service":"postgresql::postgres:5432"}]
> > ```
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > But your log is  And this seems to have some syntax problems
>
> It has nothing to do with what you're saying，this sql with pg_sleep as (select * from pg_sleep(1.4)) select * from pg_sleep can display in Slow statements
>
> 
Please help explain,Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #10485: [Fluentd Setup]Some special Postgresql slow sql was not shown in Show statements
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #10485:
URL: https://github.com/apache/skywalking/issues/10485#issuecomment-1455567228
@kezhenxu94 Any guess from LAL engine perspective?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org