[GitHub] incubator-fluo issue #795: May need to add post-commit call back to loader

[keith-turner <gi...@git.apache.org>]

Github user keith-turner commented on the issue:

    https://github.com/apache/incubator-fluo/issues/795
  
    I described a solution without describing the problem very well.  The problem this issue is trying to solve is getting good throughput from a source of continuous information that needs feedback on when that information was successfully processed by Fluo.  The LoaderExecutor gives good throughput for committing transactions, but lacks the feedback mechanism.
    
    I was thinking about how to solve  this w/ the current 1.0.0 release.  One way would be to process batches with the loader executor.  For example process 10,000 transactions with the LoaderExecutor and then shut the LoaderExecutor down.  After its shut down, then feedback can be given to the information source that all 10,000 transactions completed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---