[GitHub] [incubator-paimon] leaves12138 opened a new issue, #1046: [Feature]

["leaves12138 (via GitHub)" <gi...@apache.org>]

leaves12138 opened a new issue, #1046:
URL: https://github.com/apache/incubator-paimon/issues/1046

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
   
   
   ### Motivation
   
   It is not convenient to test in many situation.
   For example, we need to search a bug while read file happens. Then we need
   1. Generate a mount of data
   2. Insert the data to a table
   3. Read the data and test.
   
   WHAT IF we create a special FileIO class, that accept a schema as parameter, and could generate data itself. We can distinct file by the name. If it ended with .orc, then we generate data of orc format. If it start with "manifest", we generate a manifest avro file according to the schema passed in.
   
   Then we have something like flink datagen and blackhole. we could search bugs or test performance more easily.
   We don't need to write a mount of data before testing the read funtion.
   
   ### Solution
   
   As mentioned above.
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-paimon] leaves12138 closed issue #1046: [Feature] datagen in memory

["leaves12138 (via GitHub)" <gi...@apache.org>]

leaves12138 closed issue #1046: [Feature] datagen in memory
URL: https://github.com/apache/incubator-paimon/issues/1046


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org