You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "jojochuang (via GitHub)" <gi...@apache.org> on 2025/06/08 21:48:34 UTC
[PR] HDDS-13210. Add httpfs container to docker compose file. [ozone-docker]
jojochuang opened a new pull request, #46:
URL: https://github.com/apache/ozone-docker/pull/46
The change was copied from https://github.com/apache/ozone/pull/8556/files and verified to work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13210. Add httpfs container to docker compose file. [ozone-docker]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #46:
URL: https://github.com/apache/ozone-docker/pull/46#discussion_r2135244609
##########
docker-compose.yaml:
##########
@@ -73,3 +73,14 @@ services:
environment:
<<: *common-config
command: ["ozone","s3g"]
+ httpfs:
+ <<: *image
+ ports:
+ - 14000:14000
+ environment:
+ CORE-SITE.XML_fs.defaultFS: "ofs://om"
+ CORE-SITE.XML_hadoop.proxyuser.hadoop.hosts: "*"
+ CORE-SITE.XML_hadoop.proxyuser.hadoop.groups: "*"
Review Comment:
These need to be added in `common-config`. Otherwise I got the following error when trying to put a file:
```
httpfs-1 | 07:44:03,141 ERROR OzoneClientFactory:267 - Couldn't create RpcClient protocol exception:
httpfs-1 | org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hadoop is not allowed to impersonate hadoop
httpfs-1 | at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1584)
...
httpfs-1 | at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.getServiceInfo(OzoneManagerProtocolClientSideTranslatorPB.java:1880)
httpfs-1 | at org.apache.hadoop.ozone.client.rpc.RpcClient.<init>(RpcClient.java:260)
...
httpfs-1 | at org.apache.hadoop.fs.ozone.RootedOzoneClientAdapterImpl.<init>(RootedOzoneClientAdapterImpl.java:50)
httpfs-1 | at org.apache.hadoop.fs.ozone.RootedOzoneFileSystem.createAdapter(RootedOzoneFileSystem.java:108)
httpfs-1 | at org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.initialize(BasicRootedOzoneFileSystem.java:194)
...
httpfs-1 | at org.apache.ozone.lib.service.hadoop.FileSystemAccessService.createFileSystem(FileSystemAccessService.java:339)
...
httpfs-1 | at org.apache.ozone.fs.http.server.HttpFSServer.handleCreate(HttpFSServer.java:1229)
httpfs-1 | at org.apache.ozone.fs.http.server.HttpFSServer.put(HttpFSServer.java:901)
```
##########
docker-compose.yaml:
##########
@@ -73,3 +73,14 @@ services:
environment:
<<: *common-config
command: ["ozone","s3g"]
+ httpfs:
+ <<: *image
+ ports:
+ - 14000:14000
Review Comment:
nit: the rest of this file has indentation of 3 spaces, please use the same for the new service.
##########
docker-compose.yaml:
##########
@@ -73,3 +73,14 @@ services:
environment:
<<: *common-config
command: ["ozone","s3g"]
+ httpfs:
+ <<: *image
+ ports:
+ - 14000:14000
+ environment:
+ CORE-SITE.XML_fs.defaultFS: "ofs://om"
+ CORE-SITE.XML_hadoop.proxyuser.hadoop.hosts: "*"
+ CORE-SITE.XML_hadoop.proxyuser.hadoop.groups: "*"
+ OZONE-SITE.XML_hdds.scm.safemode.min.datanode: ${OZONE_SAFEMODE_MIN_DATANODES:-1}
Review Comment:
`hdds.scm.safemode.min.datanode` is not needed for `httpfs`, can be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13210. Add httpfs container to docker compose file. [ozone-docker]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #46:
URL: https://github.com/apache/ozone-docker/pull/46#issuecomment-2960170946
> Also I'm wondering instead of putting it in common-config, how about moving the proxy user configs to OM and have fs.defaultFS set at httpfs? I tested and that also works for me.
Yes, that's probably the most specific working config.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13210. Add httpfs container to docker compose file. [ozone-docker]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #46:
URL: https://github.com/apache/ozone-docker/pull/46#issuecomment-2960157037
Thanks @adoroszlai i had initially added them to common-config and then figured i should made it role specific.
Updated according to your suggestion.
Also I'm wondering instead of putting it in common-config, how about moving the proxy user configs to OM and have fs.defaultFS set at httpfs? I tested and that also works for me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13210. Add httpfs container to docker compose file. [ozone-docker]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #46:
URL: https://github.com/apache/ozone-docker/pull/46#issuecomment-2961232957
Merged. Thanks @adoroszlai for review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13210. Add httpfs container to docker compose file. [ozone-docker]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang merged PR #46:
URL: https://github.com/apache/ozone-docker/pull/46
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org