You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "JackeyLee007 (via GitHub)" <gi...@apache.org> on 2024/11/08 02:24:30 UTC

[I] [Bug] Fail to create tag automatically with watermark [paimon]

JackeyLee007 opened a new issue, #4481:
URL: https://github.com/apache/paimon/issues/4481

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
   
   
   ### Paimon version
   
   0.9
   
   ### Compute Engine
   
   Flink-1.18.1
   Standalone cluster.
   
   
   ### Minimal reproduce step
   
   ```sql
   -- Target paimon table with tags create automatically from watermark 
   create table orders_with_watermark(
       id INT PRIMARY KEY NOT ENFORCED, 
       val int, 
       create_time TIMESTAMP(3), 
       update_time TIMESTAMP(3),
       WATERMARK FOR `update_time` AS `update_time` - INTERVAL '5' SECOND  
     ) 
     WITH (
       'tag.automatic-creation' = 'watermark',
       'tag.creation-period' = 'hourly',
       'tag.creation-delay' = '10 m',
       'tag.num-retained-max' = '90',
       'sink.watermark-time-zone' = 'Asia/Shanghai'
   );
   -- mock datasource 
   CREATE TEMPORARY TABLE datagen_table (
       id INT,
       name STRING,
       ts TIMESTAMP(3)
   ) WITH (
       'connector' = 'datagen',
       'rows-per-second' = '1',
       'fields.id.min' = '1',
       'fields.id.max' = '100',
       'fields.name.length' = '10',
       'fields.ts.kind' = 'random'
   );
   -- Run a stream job on Flink
   INSERT INTO  orders_with_watermark
     select id, 1, now(), now() 
     from datagen_table;
   ```
   
   
   ### What doesn't meet your expectations?
   
   The tags should be created automatically every hour, but none.
   No tag files could be found under the table folder and also no result when executing the following sql
   ```sql
   select * from `orders_with_watermark$tags` 
   ```
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] Fail to create tag automatically with watermark [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on issue #4481:
URL: https://github.com/apache/paimon/issues/4481#issuecomment-2464575656

   First, you can check if the job is heath.
   Second, you can check the watermark in Flink topology, does the source produces the watermark normally.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] Fail to create tag automatically with watermark [paimon]

Posted by "JackeyLee007 (via GitHub)" <gi...@apache.org>.
JackeyLee007 commented on issue #4481:
URL: https://github.com/apache/paimon/issues/4481#issuecomment-2466207377

   Sorry, it's my fault. Actually, the watermark should be emitted from the stream source which is the datagen_table for this case.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] Fail to create tag automatically with watermark [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi closed issue #4481: [Bug] Fail to create tag automatically with watermark
URL: https://github.com/apache/paimon/issues/4481


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] Fail to create tag automatically with watermark [paimon]

Posted by "JackeyLee007 (via GitHub)" <gi...@apache.org>.
JackeyLee007 commented on issue #4481:
URL: https://github.com/apache/paimon/issues/4481#issuecomment-2466207570

   Thanks for your hint. @JingsongLi 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] Fail to create tag automatically with watermark [paimon]

Posted by "JackeyLee007 (via GitHub)" <gi...@apache.org>.
JackeyLee007 closed issue #4481: [Bug] Fail to create tag automatically with watermark
URL: https://github.com/apache/paimon/issues/4481


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org