You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/11/11 10:27:34 UTC

[GitHub] [ozone] adoroszlai opened a new pull request #1572: HDDS-4450. Cannot run ozone if HADOOP_HOME points to Hadoop install

adoroszlai opened a new pull request #1572:
URL: https://github.com/apache/ozone/pull/1572

## What changes were proposed in this pull request?

If `HADOOP_HOME` does not point to Ozone install, make an attempt to find it relative to the script being run. This logic is copied from `start-ozone.sh`, where it was added for HDDS-1912 to fix similar problem.

https://issues.apache.org/jira/browse/HDDS-4450

## How was this patch tested?

Tested using `ozonescripts` compose environment after temporarily adding `HADOOP_HOME=/usr/local/hadoop` to `docker-config`. Verified that all three scripts (`ozone` and `stop-ozone.sh`, changed here, and `start-ozone.sh`, changed previously) work fine.

```
cd hadoop-ozone/dist/target/ozone-1.1.0-SNAPSHOT/compose/ozonescripts
docker-compose up -d
# <wait a bit>
./start.sh
...
./ps.sh
# <output with Ozone processes>
docker-compose exec scm bash
bash-4.2$ ozone freon ockg -n1 -t1 -F ONE
...
Successful executions: 1
bash-4.2$ echo $HADOOP_HOME
/usr/local/hadoop
bash-4.2$ exit
./stop.sh
...
./ps.sh
# <no more Ozone process>
```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] elek commented on pull request #1572: HDDS-4450. Cannot run ozone if HADOOP_HOME points to Hadoop install

Posted by GitBox <gi...@apache.org>.

elek commented on pull request #1572:
URL: https://github.com/apache/ozone/pull/1572#issuecomment-727906745


   I guess, it's an incompatible change, but should be acceptable IMHO. 
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] adoroszlai commented on pull request #1572: HDDS-4450. Cannot run ozone if HADOOP_HOME points to Hadoop install

Posted by GitBox <gi...@apache.org>.

adoroszlai commented on pull request #1572:
URL: https://github.com/apache/ozone/pull/1572#issuecomment-730253599


   Thanks @elek for merging it and @avijayanhwx for the review.
   
   > Based on my understanding HADOOP_HOME was usable earlier to point to any Ozone install, now it's forced to point a specific dir.
   
   The script is happy as long as it finds the `ozone-config.sh` file, does not care about specific directories, versions or instances.  HADOOP_HOME is only forced if it does not already point to any Ozone install, rather to Hadoop or something completely different or maybe a non-existent dir.  In this case, `ozone` (and `stop-ozone.sh`) did not work prior to this change.
   
   Also, as long as Ozone scripts are executed, not sourced, updating the variables does not affect the caller.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] adoroszlai edited a comment on pull request #1572: HDDS-4450. Cannot run ozone if HADOOP_HOME points to Hadoop install

Posted by GitBox <gi...@apache.org>.

adoroszlai edited a comment on pull request #1572:
URL: https://github.com/apache/ozone/pull/1572#issuecomment-730253599

Thanks @elek for merging it and @avijayanhwx for the review.

> Based on my understanding HADOOP_HOME was usable earlier to point to any Ozone install, now it's forced to point a specific dir.

The script is happy as long as it finds the `ozone-config.sh` file, does not care about specific directories, versions or instances. HADOOP_HOME is only forced if it does not already point to any Ozone install, rather to Hadoop or something completely different or maybe a non-existent dir. In this case, `ozone` (and `stop-ozone.sh`) did not work prior to this change.

Also, as long as Ozone scripts are executed, not sourced, updating the variables does not affect the caller.

Normally HADOOP_HOME and other environment variables can be defined for each component separately. However, in an Impala mini cluster (used for running tests), all components share the same environment. HADOOP_HOME points to Hadoop, so we need a way to override this "locally" for Ozone. This change allows that without extensive and potentially dangerous refactoring (renaming all HADOOP variables and updating Ozone's copy of hadoop functions etc.).

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] elek commented on pull request #1572: HDDS-4450. Cannot run ozone if HADOOP_HOME points to Hadoop install

Posted by GitBox <gi...@apache.org>.

elek commented on pull request #1572:
URL: https://github.com/apache/ozone/pull/1572#issuecomment-729926151


   > Can you tell me how this is an incompatible change?
   
   Based on my understanding HADOOP_HOME was usable earlier to point to any Ozone install, now it's forced to point a specific dir. 
   
   The chance is very small for this behavior is used by any cluster manager, but I would like to be sure it's safe. (earlier I accidentally committed incompatible changes.)  
   
   But as you are fine with it, let's merge it.
   
   Thanks the patch @adoroszlai and review @avijayanhwx  


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] elek commented on pull request #1572: HDDS-4450. Cannot run ozone if HADOOP_HOME points to Hadoop install

Posted by GitBox <gi...@apache.org>.

elek commented on pull request #1572:
URL: https://github.com/apache/ozone/pull/1572#issuecomment-732168684


   :heart: Thanks for the detailed explanation.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] elek merged pull request #1572: HDDS-4450. Cannot run ozone if HADOOP_HOME points to Hadoop install

Posted by GitBox <gi...@apache.org>.

elek merged pull request #1572:
URL: https://github.com/apache/ozone/pull/1572


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org