[GitHub] [incubator-druid] jon-wei opened a new pull request #7598: Add tool for migrating from local deep storage/Derby metadata

[GitBox <gi...@apache.org>]

jon-wei opened a new pull request #7598: Add tool for migrating from local deep storage/Derby metadata
URL: https://github.com/apache/incubator-druid/pull/7598
 
 
   This PR adds a new tool under `services` meant to assist with the following use case:
   
   Users sometimes being evaluating Druid with a simple cluster that uses local deep storage and Derby. After using the evaluation cluster for some time and ingesting some segments, they wish to move to MySQL or PostgreSQL and/or use a different deep storage.
   
   This tool exports the contents of the following Druid tables:
   - segments
   - rules
   - config
   - datasource
   - supervisors
   
   These tables are chosen since they contain non-transient (like task locks), non-historical entities (like task logs), as the tool is intended for migrations where the user shuts the entire cluster down.
   
   The tool also allows users to specify a new S3 bucket/key, HDFS path, or new local filesystem path, and the entries from the segments table will be rewritten with new load specs. This is to assist with deep storage migration.
   
   Currently only migration from local deep storage combined with Derby metadata is supported (the use case described above), the tool could be later expanded to handle other use cases.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org