You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2023/01/04 10:47:35 UTC
[GitHub] [doris] nextdreamblue opened a new pull request, #15617: [fix](tvf) use virtual-hosted style when s3('uri'='s3://xxx')
nextdreamblue opened a new pull request, #15617:
URL: https://github.com/apache/doris/pull/15617
Signed-off-by: nextdreamblue <zx...@163.com>
# Proposed changes
Issue Number: close #xxx
## Problem summary
when S3 tvf, the S3 SDK uses the virtual-hosted style by default, but the code set forceVirtualHosted=false when uri='s3://'.
## Checklist(Required)
1. Does it affect the original behavior:
- [ ] Yes
- [x] No
- [ ] I don't know
2. Has unit tests been added:
- [ ] Yes
- [ ] No
- [x] No Need
3. Has document been added or modified:
- [ ] Yes
- [ ] No
- [x] No Need
4. Does it need to update dependencies:
- [ ] Yes
- [x] No
5. Are there any changes that cannot be rolled back:
- [ ] Yes (If Yes, please explain WHY)
- [x] No
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] nextdreamblue commented on pull request #15617: [fix](tvf) use virtual-hosted style when s3('uri'='s3://xxx')
Posted by GitBox <gi...@apache.org>.
nextdreamblue commented on PR #15617:
URL: https://github.com/apache/doris/pull/15617#issuecomment-1375044289
> > > Hi, I still get error like 'parse file failed, path = s3://file' after using your changes, but using 'http://' is ok.
> >
> >
> > s3默认使用的virtual-hosted style，即类似：s3://bucket.endpoint/file的形式，如果是s3://endpoint/bucket/file的形式应该会报你这种错误才对。您用的哪种方式呢
>
> ok, I got it. Is `"use_path_style"="true"` no use for `s3://bucket.endpoint/file` ? When I'm using`s3://bucket.endpoint/file`, it's ok whatever I set `"use_path_style"="true"` or `"use_path_style"="false"`
看社区本身的设计和之前的代码实现，应该就是s3协议只支持virtual-hosted style。非s3协议才由use_path_style来控制用什么方式去解析uri
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] BePPPower commented on pull request #15617: [fix](tvf) use virtual-hosted style when s3('uri'='s3://xxx')
Posted by GitBox <gi...@apache.org>.
BePPPower commented on PR #15617:
URL: https://github.com/apache/doris/pull/15617#issuecomment-1373366957
Hi, I still get error like 'parse file failed, path = s3://file' after using your changes, but using 'http://' is ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] nextdreamblue commented on pull request #15617: [fix](tvf) use virtual-hosted style when s3('uri'='s3://xxx')
Posted by GitBox <gi...@apache.org>.
nextdreamblue commented on PR #15617:
URL: https://github.com/apache/doris/pull/15617#issuecomment-1373488966
> Hi, I still get error like 'parse file failed, path = s3://file' after using your changes, but using 'http://' is ok.
s3默认使用的virtual-hosted style，即类似：s3://bucket.endpoint/file的形式，如果是s3://endpoint/bucket/file的形式应该会报你这种错误才对。您用的哪种方式呢
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #15617: [fix](tvf) use virtual-hosted style when s3('uri'='s3://xxx')
Posted by GitBox <gi...@apache.org>.
hello-stephen commented on PR #15617:
URL: https://github.com/apache/doris/pull/15617#issuecomment-1370961593
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 35.6 seconds
load time: 651 seconds
storage size: 17123659785 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230104135811_clickbench_pr_73554.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] BePPPower commented on pull request #15617: [fix](tvf) use virtual-hosted style when s3('uri'='s3://xxx')
Posted by GitBox <gi...@apache.org>.
BePPPower commented on PR #15617:
URL: https://github.com/apache/doris/pull/15617#issuecomment-1374390595
> > Hi, I still get error like 'parse file failed, path = s3://file' after using your changes, but using 'http://' is ok.
>
> s3默认使用的virtual-hosted style，即类似：s3://bucket.endpoint/file的形式，如果是s3://endpoint/bucket/file的形式应该会报你这种错误才对。您用的哪种方式呢
ok, I got it. Is `"use_path_style"="true"` no use for `s3://bucket.endpoint/file` ? When I'm using`s3://bucket.endpoint/file`, it's ok whatever I set `"use_path_style"="true"` or `"use_path_style"="false"`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15617: [fix](tvf) use virtual-hosted style when s3('uri'='s3://xxx')
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #15617:
URL: https://github.com/apache/doris/pull/15617#issuecomment-1375074960
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #15617: [fix](tvf) use virtual-hosted style when s3('uri'='s3://xxx')
Posted by GitBox <gi...@apache.org>.
yiguolei merged PR #15617:
URL: https://github.com/apache/doris/pull/15617
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org