You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/04/28 02:57:19 UTC

[GitHub] [incubator-seatunnel] lordk911 opened a new issue, #1764: [Feature][jdbc spark] support query param

lordk911 opened a new issue, #1764:
URL: https://github.com/apache/incubator-seatunnel/issues/1764

   ### Search before asking
   
   - [X] I had searched in the [feature](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
   
   
   ### Description
   
   我尝试用spark从mysql的一张表中读数据，然后输出到控制台，发现卡住很久且内存溢出。
   是因为spark会读全量数据吗？是否可以加query参数，主动过滤掉一些数据数据。
   另外试了加transform，一样的情况，这样也做不到谓词下推吧？
   
   ### Usage Scenario
   
   _No response_
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] lordk911 commented on issue #1764: [Feature][jdbc spark] support query param

Posted by GitBox <gi...@apache.org>.
lordk911 commented on issue #1764:
URL: https://github.com/apache/incubator-seatunnel/issues/1764#issuecomment-1111853316

    I tried using Spark to read data from a table in mysql and sink it to the console, but it stuck for a long time and ran out of memory. 
   
   Is it because Spark reads full data? Whether the query parameter can be added to actively filter some data data.
   
   I also tried adding transform, helpless, that doesn't do the predicate push down, does it?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] lordk911 closed issue #1764: [Feature][jdbc spark] support query param

Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 closed issue #1764: [Feature][jdbc spark] support query param
URL: https://github.com/apache/incubator-seatunnel/issues/1764


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] ashulin commented on issue #1764: [Feature][jdbc spark] support query param

Posted by GitBox <gi...@apache.org>.
ashulin commented on issue #1764:
URL: https://github.com/apache/incubator-seatunnel/issues/1764#issuecomment-1112822507

   Currently source is instantiated directly rather than created by the SQL engine, so predicate pushdown is not supported.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] chenhu commented on issue #1764: [Feature][jdbc spark] support query param

Posted by GitBox <gi...@apache.org>.
chenhu commented on issue #1764:
URL: https://github.com/apache/incubator-seatunnel/issues/1764#issuecomment-1128335998

   https://seatunnel.apache.org/docs/2.1.1/command/usage
   
   @lordk911 u can use the -i option passing the params to the config file


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] ruanwenjun commented on issue #1764: [Feature][jdbc spark] support query param

Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on issue #1764:
URL: https://github.com/apache/incubator-seatunnel/issues/1764#issuecomment-1111777035

   @lordk911 We use English to describe the issue, please translate to English, thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org