You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "morningman (via GitHub)" <gi...@apache.org> on 2023/05/10 02:17:54 UTC
[GitHub] [doris] morningman opened a new issue, #19477: [Feature] support compress file for tvf hdfs() and s3()
morningman opened a new issue, #19477:
URL: https://github.com/apache/doris/issues/19477
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Description
Related discussion: https://github.com/apache/doris/discussions/18140
We can support it by add a new properties for tvf , like :
```
select * from hdfs("uri" = "xxx", ..., "compress_type" = "lz4", ...)
```
User can:
1. Specify compression explicitly by setting `"compression" = "xxx"`.
2. Doris can infer the compression type by the suffix of file name, eg, file1.gz
Currently, we only support reading compress file in csv format, and on BE side, we already support.
All need to do is to analyze the "compress_type" on FE side and pass it to BE.
Related class:
```
PropertyAnalyzer.analyzeCompressionType()
```
### Use case
_No response_
### Related issues
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] reswqa commented on issue #19477: [Feature] support compress file for tvf hdfs() and s3()
Posted by "reswqa (via GitHub)" <gi...@apache.org>.
reswqa commented on issue #19477:
URL: https://github.com/apache/doris/issues/19477#issuecomment-1541419905
Thanks for creating this! I'd like to pick up this~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morningman closed issue #19477: [Feature] support compress file for tvf hdfs() and s3()
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman closed issue #19477: [Feature] support compress file for tvf hdfs() and s3()
URL: https://github.com/apache/doris/issues/19477
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org