[GitHub] [incubator-hudi] vinothchandar commented on issue #917: [HUDI-251] JDBC incremental load to HUDI with DeltaStreamer

[GitBox <gi...@apache.org>]

vinothchandar commented on issue #917: [HUDI-251] JDBC incremental load to HUDI with DeltaStreamer
URL: https://github.com/apache/incubator-hudi/pull/917#issuecomment-534587329
 
 
   >The other option is that when we set is_incremental as false then we pull all the data at once and then write it.
   
   this can be implemented by passing `--checkpoint null` or `--full-load` flag? This is actually a general issue for all sources.. it would be good to open a new JIRA for this and tackle separately.. For e.g, even if you have files on DFS, you want to probably have an option to do this.. For this PR, we can just focus on incremental pulling where the first run without checkpoint, pulls the entire table? 
   
   >>the interval we should be pulling the data every interval.
   On the interval, may be I was vague. apologies. What I meant was, the frequency at which we run DeltaStreamer is controlled by the user in non-continuous mode and #921 just added a flag to control this in continuous mode. Don't think we need to worry about it in this PR? 
   
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services