[GitHub] [ozone] adoroszlai commented on pull request #1667: HDDS-4525. Replace Hadoop variables and functions in Ozone shell scripts with Ozone-specific ones

[GitBox <gi...@apache.org>]

adoroszlai commented on pull request #1667:
URL: https://github.com/apache/ozone/pull/1667#issuecomment-745167615


   > Is it possible to instead changing to config and breaking the convention fix the envtoconf behaviour, I see that OZONE_CONFIG_DIR is added to docker-config files and that's how envtoconf came into the picture, previously we did not needed HADOOP_CONF_DIR there, we do we need OZONE_CONFIG_DIR in the docker-config files now?
   
   `HADOOP_CONF_DIR` is already defined in the docker images we use, hence it's not needed in `docker-config`, eg.:
   
   ```
   $ docker run -it --rm apache/ozone-runner:20200625-1 env | grep HADOOP
   HADOOP_LOG_DIR=/var/log/hadoop
   HADOOP_CONF_DIR=/etc/hadoop
   ```
   
   Also, `HADOOP_CONF_DIR` is an explicitly defined exception in `envtoconf`:
   
   https://github.com/apache/ozone/blob/09579756b0756fcaf1b22b860c1ee1eb927e82c2/hadoop-ozone/dist/src/main/dockerlibexec/envtoconf.py#L39
   
   We could fix this script, but it would only apply to `ozone-runner` containers, since it uses mounted Ozone binaries.  Both `ozone` and `hadoop` containers (used in `upgrade`, `ozone-mr`, etc. tests) come with `envtoconf.py` baked in.
   
   ```
   $ docker run -it --rm apache/hadoop:3 grep CONF_DIR /opt/envtoconf.py
       self.excluded_envs = ['HADOOP_CONF_DIR']
   $ docker run -it --rm apache/ozone:1.0.0 grep CONF_DIR /opt/hadoop/libexec/envtoconf.py
       self.excluded_envs = ['HADOOP_CONF_DIR']
   ```
   
   So we have 3 options:
   
   1. proper fix: update `envtoconf` in both Ozone and Hadoop and build new Docker images (`apache/ozone-runner`, `apache/hadoop`, `apache/ozone`, the latter two for multiple versions)
   2. hack: modify all `docker-compose.yaml` files to mount the fixed version of `envtoconf`
   3. workaround: use different variable name


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org