You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "c8679724 (via GitHub)" <gi...@apache.org> on 2024/07/13 03:40:49 UTC

[I] [Feature] Feature Request: Support Archiving Older Data to Object Storage (e.g., S3, Ceph) [skywalking]

c8679724 opened a new issue, #12437:
URL: https://github.com/apache/skywalking/issues/12437

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar feature requirement.
   
   
   ### Description
   
   ## Description
   Currently, SkyWalking does not support the archiving of data older than 6 months to external object storage solutions such as Amazon S3 or Ceph. This feature would be highly beneficial for organizations looking to manage their long-term data storage more efficiently and reduce the load on their primary storage systems.
   
   
   
   ## Proposed Solution
   Implement a feature that allows users to configure SkyWalking to automatically archive data older than a specified time period (e.g., 6 months) to object storage. This could involve:
   
   1. **Configuration Options:** Add configuration options in the SkyWalking configuration files to specify the object storage provider, credentials, and other necessary details.
   2. **Data Archiving Process:** Develop a background process that periodically checks for data older than the specified period and transfers it to the configured object storage.
   3. **Retrieval Mechanism:** Ensure that there is a mechanism to retrieve archived data when needed for analysis or other purposes.
   
   ## Additional Context
   - This feature would align with the growing need for cost-effective long-term data storage solutions.
   - It would also help in reducing the operational burden on system administrators by automating the data archiving process.
   
   ## References
   - [Amazon S3 Documentation](https://docs.aws.amazon.com/s3/index.html)
   - [Ceph Documentation](https://docs.ceph.com/docs/master/)
   
   Looking forward to your thoughts on this feature request. Thank you for considering this enhancement to SkyWalking.
   
   
   ### Use case
   
   ## Use Case
   Many users require long-term data retention for compliance, historical analysis, and other purposes. Storing this data in an external, scalable, and cost-effective storage solution like S3 or Ceph would help in managing storage costs and maintaining performance of the primary data storage systems.
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a pull request to implement this on your own?
   
   - [ ] Yes I am willing to submit a pull request on my own!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Feature] Feature Request: Support Archiving Older Data to Object Storage (e.g., S3, Ceph) [skywalking]

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #12437:
URL: https://github.com/apache/skywalking/issues/12437#issuecomment-2226788931

   We are not going to support that from SkyWalking generally. Your asking doesn't make sense. We don't have a component like ETL to do that. But all storage are open for you, you could use existing tools to do that on your own.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Feature] Feature Request: Support Archiving Older Data to Object Storage (e.g., S3, Ceph) [skywalking]

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #12437:
URL: https://github.com/apache/skywalking/issues/12437#issuecomment-2226789167

   The only possibility we support that is, this will be a part of BanyanDB feature, but not from SkyWalking OAP.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Feature] Feature Request: Support Archiving Older Data to Object Storage (e.g., S3, Ceph) [skywalking]

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng closed issue #12437: [Feature] Feature Request: Support Archiving Older Data to Object Storage (e.g., S3, Ceph)
URL: https://github.com/apache/skywalking/issues/12437


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org