You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/12/16 10:43:51 UTC
[GitHub] [ozone] adoroszlai opened a new pull request #1712: HDDS-4601. envtoconf broken for .conf and few other formats
adoroszlai opened a new pull request #1712:
URL: https://github.com/apache/ozone/pull/1712
## What changes were proposed in this pull request?
Fix `envtoconf` for some output formats that are currently broken. These are not used in Ozone, nor covered by tests. The problem may be accidentally triggered by specific environment variable names, eg. `HBASE_CONF_DIR` or similar.
https://issues.apache.org/jira/browse/HDDS-4601
## How was this patch tested?
Test:
```
cd hadoop-ozone/dist/target/ozone-1.1.0-SNAPSHOT
OZONE_RUNNER_VERSION=20200625-1
docker run -it --rm -e DUMMY.CONF_key=value -v $(pwd)/libexec:/opt/hadoop/libexec apache/ozone:0.5.0 bash -c 'ls ${HADOOP_CONF_DIR}/dummy.conf'
docker run -it --rm -e DUMMY.SH_key=value -v $(pwd)/libexec:/opt/hadoop/libexec apache/ozone:1.0.0 bash -c 'ls ${HADOOP_CONF_DIR}/dummy.sh'
docker run -it --rm -e DUMMY.ENV_key=value -v $(pwd)/libexec/transformation.py:/opt/transformation.py apache/hadoop:3 bash -c 'ls ${HADOOP_CONF_DIR}/dummy.env'
docker run -it --rm -e DUMMY.CFG_key=value -v $(pwd):/opt/hadoop apache/ozone-runner:${OZONE_RUNNER_VERSION} bash -c 'ls ${HADOOP_CONF_DIR}/dummy.cfg'
```
Output:
```
/etc/hadoop/dummy.conf
/etc/hadoop/dummy.sh
/etc/hadoop/dummy.env
/etc/hadoop/dummy.cfg
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #1712: HDDS-4601. envtoconf broken for .conf and few other formats
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on pull request #1712:
URL: https://github.com/apache/ozone/pull/1712#issuecomment-746767407
Thanks @fapifta for the review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #1712: HDDS-4601. envtoconf broken for .conf and few other formats
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on pull request #1712:
URL: https://github.com/apache/ozone/pull/1712#issuecomment-746502308
(Answering the part of your comment from #1667 relevant to this PR.)
> The change itself in transformation.py is to go through the items instead of the collection means that we are transforming something different in those cases and we do it similarly as the one case you mentioned working well before this change also, I am not sure though, how this solves the problem.
@fapifta Here's a Python REPL session to show what's going on:
```python
>>> props = {"key": "value"}
>>> props
{'key': 'value'}
>>> props.keys()
['key']
>>> props.values()
['value']
>>> props.items()
[('key', 'value')]
>>> for x in props: print x
...
key
>>> for x in props.keys(): print x
...
key
>>> for x, y in props: print "{}={}".format(x, y)
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: too many values to unpack
>>> for x, y in props.items(): print "{}={}".format(x, y)
...
key=value
>>>
```
`props` is a map (dictionary). `for x in props` iterates keys, same as `for x in props.keys()`. `for x, y in props` also iterates keys (`in props`), but tries to extract key-value pairs, which fails with `ValueError`, since each key is only a single item. `for x, y in props.items()` correctly iterates key-value pairs.
Already used correctly in `to_properties`:
https://github.com/apache/ozone/blob/cbe8651217930181057f7a4e496ae20d891ad032/hadoop-ozone/dist/src/main/dockerlibexec/transformation.py#L81-L85
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] fapifta commented on pull request #1712: HDDS-4601. envtoconf broken for .conf and few other formats
Posted by GitBox <gi...@apache.org>.
fapifta commented on pull request #1712:
URL: https://github.com/apache/ozone/pull/1712#issuecomment-746643575
Thank you for making it clear what the exact problem is.
Looks good to me, +1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai merged pull request #1712: HDDS-4601. envtoconf broken for .conf and few other formats
Posted by GitBox <gi...@apache.org>.
adoroszlai merged pull request #1712:
URL: https://github.com/apache/ozone/pull/1712
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org