You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "xccui (via GitHub)" <gi...@apache.org> on 2024/06/01 20:19:52 UTC
[I] [Bug] Serializers were created infinitely and caused OOM for 0.8 [paimon]
xccui opened a new issue, #3460:
URL: https://github.com/apache/paimon/issues/3460
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
0.8
### Compute Engine
Flink 1.18.1
### Minimal reproduce step
Serializers were created infinitely for the following recursive schema after bumping up Paimon to 0.8.
```
ARRAY <
ROW <
`null_value` STRING,
`number_value` DOUBLE,
`string_value` STRING,
`bool_value` BOOLEAN,
`struct_value` ROW <
`fields` ARRAY <
ROW <
`key` STRING,
`value` ROW <
`null_value` STRING,
`number_value` DOUBLE,
`string_value` STRING,
`bool_value` BOOLEAN,
`struct_value` ROW <
`fields` ARRAY <
ROW <
`key` STRING,
`value` ROW <
`null_value` STRING,
`number_value` DOUBLE,
`string_value` STRING,
`bool_value` BOOLEAN
>
> NOT NULL
>
>,
`list_value` ROW <
`values` ARRAY <
ROW <
`null_value` STRING,
`number_value` DOUBLE,
`string_value` STRING,
`bool_value` BOOLEAN
> NOT NULL
>
>
>
> NOT NULL
>
>,
`list_value` ROW <
`values` ARRAY <
ROW <
`null_value` STRING,
`number_value` DOUBLE,
`string_value` STRING,
`bool_value` BOOLEAN,
`struct_value` ROW <
`fields` ARRAY <
ROW <
`key` STRING,
`value` ROW <
`null_value` STRING,
`number_value` DOUBLE,
`string_value` STRING,
`bool_value` BOOLEAN
>
> NOT NULL
>
>,
`list_value` ROW <
`values` ARRAY <
ROW <
`null_value` STRING,
`number_value` DOUBLE,
`string_value` STRING,
`bool_value` BOOLEAN
> NOT NULL
>
>
> NOT NULL
>
>
> NOT NULL
>
```
### What doesn't meet your expectations?
The test case worked well with Paimon 0.7 but caused OOM with Paimon 0.8.

### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Serializers were created infinitely and caused OOM for 0.8 [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi closed issue #3460: [Bug] Serializers were created infinitely and caused OOM for 0.8
URL: https://github.com/apache/paimon/issues/3460
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Serializers were created infinitely and caused OOM for 0.8 [paimon]
Posted by "xccui (via GitHub)" <gi...@apache.org>.
xccui commented on issue #3460:
URL: https://github.com/apache/paimon/issues/3460#issuecomment-2148819000
I think the bug was introduced in https://github.com/apache/paimon/pull/3039 since everything's back to normal after I reverted it.
CC @JingsongLi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Serializers were created infinitely and caused OOM for 0.8 [paimon]
Posted by "wg1026688210 (via GitHub)" <gi...@apache.org>.
wg1026688210 commented on issue #3460:
URL: https://github.com/apache/paimon/issues/3460#issuecomment-2144597931
Hi~ @xccui Could you please specify when the OOM occurred? Was it during the data writing process?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Serializers were created infinitely and caused OOM for 0.8 [paimon]
Posted by "wg1026688210 (via GitHub)" <gi...@apache.org>.
wg1026688210 commented on issue #3460:
URL: https://github.com/apache/paimon/issues/3460#issuecomment-2151404084
> Hi @wg1026688210, it happened when I tried to write something to a Paimon table. Here's the screenshot for the stack.
>
> <img alt="image" width="812" src="https://private-user-images.githubusercontent.com/1452518/336093585-0eb163de-ab02-4faf-bf0d-9f01d3a267e3.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTc2NDkxMjAsIm5iZiI6MTcxNzY0ODgyMCwicGF0aCI6Ii8xNDUyNTE4LzMzNjA5MzU4NS0wZWIxNjNkZS1hYjAyLTRmYWYtYmYwZC05ZjAxZDNhMjY3ZTMucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDYwNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA2MDZUMDQ0MDIwWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NjU0MDQ1OWFiOWI5OGQyYTJmZTZhMjVkODM2ZWIwOGUyMTQ5Y2M3ZjQzNTBmMDBlOTc5ZGE5NGVjODBhNTYxOCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.PefzhrEkjssDPXzdO4wDcq3kep_qPANpRKGXSue0Mz4">
Hi ~ @xccui Could you provide the code reproducing this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Serializers were created infinitely and caused OOM for 0.8 [paimon]
Posted by "xccui (via GitHub)" <gi...@apache.org>.
xccui commented on issue #3460:
URL: https://github.com/apache/paimon/issues/3460#issuecomment-2145230406
Hi @wg1026688210, it happened when I tried to write something to a Paimon table. Here's the screenshot for the stack.
<img width="812" alt="image" src="https://github.com/apache/paimon/assets/1452518/0eb163de-ab02-4faf-bf0d-9f01d3a267e3">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org