[GitHub] [incubator-ponymail-foal] sbp opened a new pull request #14: Ensure that all docs are migrated

[GitBox <gi...@apache.org>]

sbp opened a new pull request #14:
URL: https://github.com/apache/incubator-ponymail-foal/pull/14


   This PR fixes a bug where `migrate.py` would not migrate all documents. The problem is that in the loop iterating over all documents in the source database, documents are appended gradually into a buffer. When the buffer size exceeds 100, the whole buffer is bulk pushed and then cleared. This leads to a "leftovers" problem if the number of source document is not an exact multiple of 100, meaning that up to 99 documents may be left in the buffer after the final loop and never pushed because the buffer size did not exceed 100.
   
   The fix suggested in this PR is to add an extra check after the loop, bulk pushing the buffer again if the buffer size is greater than 0. The problem only exists for the mbox and source indices; attachments are handled one-by-one, and have no buffer.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-ponymail-foal] Humbedooh merged pull request #14: Ensure that all docs are migrated

[GitBox <gi...@apache.org>]

Humbedooh merged pull request #14:
URL: https://github.com/apache/incubator-ponymail-foal/pull/14


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org