You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/30 15:08:24 UTC
[GitHub] [airflow] subkanthi opened a new pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,
subkanthi opened a new pull request #21222:
URL: https://github.com/apache/airflow/pull/21222
Helm: Fix elasticsearch URL in secret when username/password fields are empty,
closes: #20560
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a change in pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on a change in pull request #21222:
URL: https://github.com/apache/airflow/pull/21222#discussion_r800945294
##########
File path: chart/templates/secrets/elasticsearch-secret.yaml
##########
@@ -33,6 +33,14 @@ metadata:
type: Opaque
data:
{{- with .Values.elasticsearch.connection }}
- connection: {{ urlJoin (dict "scheme" (default "http" .scheme) "userinfo" (printf "%s:%s" (.user | urlquery) (.pass | urlquery)) "host" (printf "%s:%s" .host ((default 9200 .port) | toString) ) ) | b64enc | quote }}
- {{- end }}
+ {{- if .user }}
+ {{- if .pass }}
+ connection: {{ urlJoin (dict "scheme" (default "http" .scheme) "userinfo" (printf "%s:%s" (.user | urlquery) (.pass | urlquery)) "host" (printf "%s:%s" .host ((default 9200 .port) | toString) ) ) | b64enc | quote }}
+ {{- else}}
+ connection: {{ urlJoin (dict "scheme" (default "http" .scheme) "host" (printf "%s:%s" .host ((default 9200 .port) | toString))) | b64enc | quote }}
+ {{- end }}
Review comment:
```suggestion
{{- if and .user .pass }}
connection: {{ urlJoin (dict "scheme" (default "http" .scheme) "userinfo" (printf "%s:%s" (.user | urlquery) (.pass | urlquery)) "host" (printf "%s:%s" .host ((default 9200 .port) | toString) ) ) | b64enc | quote }}
```
This is simpler.
##########
File path: chart/tests/test_elasticsearch_secret.py
##########
@@ -132,3 +132,39 @@ def test_should_generate_secret_with_specified_schemes(self, scheme):
)
assert f"{scheme}://username:password@elastichostname:9200" == connection
+
+ def test_url_generated_when_user_is_empty(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"pass": "password", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://elastichostname:8080" == connection
+
+ def test_url_generated_when_password_is_empty(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"user": "admin", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://elastichostname:8080" == connection
+
+ def test_url_generated_with_valid_user_password(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"user": "admin", "pass": "pass", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://admin:pass@elastichostname:8080" == connection
Review comment:
You don't have a test where neither user/pass are set. It might be cleaner to parameterize these as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #21222:
URL: https://github.com/apache/airflow/pull/21222#discussion_r803873125
##########
File path: chart/tests/test_elasticsearch_secret.py
##########
@@ -132,3 +132,39 @@ def test_should_generate_secret_with_specified_schemes(self, scheme):
)
assert f"{scheme}://username:password@elastichostname:9200" == connection
+
+ def test_url_generated_when_user_is_empty(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"pass": "password", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://elastichostname:8080" == connection
+
+ def test_url_generated_when_password_is_empty(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"user": "admin", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://elastichostname:8080" == connection
+
+ def test_url_generated_with_valid_user_password(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"user": "admin", "pass": "pass", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://admin:pass@elastichostname:8080" == connection
Review comment:
Thanks , tuple definitely made the difference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #21222:
URL: https://github.com/apache/airflow/pull/21222#discussion_r801109387
##########
File path: chart/tests/test_elasticsearch_secret.py
##########
@@ -132,3 +132,39 @@ def test_should_generate_secret_with_specified_schemes(self, scheme):
)
assert f"{scheme}://username:password@elastichostname:9200" == connection
+
+ def test_url_generated_when_user_is_empty(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"pass": "password", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://elastichostname:8080" == connection
+
+ def test_url_generated_when_password_is_empty(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"user": "admin", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://elastichostname:8080" == connection
+
+ def test_url_generated_with_valid_user_password(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"user": "admin", "pass": "pass", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://admin:pass@elastichostname:8080" == connection
Review comment:
Thanks, Im actually having problems passing JSON in parametrized.expand, trying to figure out, for some reason the parameter only has the first key.
``` @parameterized.expand(
[
# When user is empty
({"enabled": True, "connection" : {"pass": "password", "host": "elastichostname", "port": 8080}}),
# When pass is empty
# ({"connection": {"user": "admin", "host": "elastichostname", "port": 8080}}),
# # When user and pass are empty
# ({"connection": {"host": "elastichostname", "port": 8080}}),
],
)```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi closed pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,
Posted by GitBox <gi...@apache.org>.
subkanthi closed pull request #21222:
URL: https://github.com/apache/airflow/pull/21222
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a change in pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on a change in pull request #21222:
URL: https://github.com/apache/airflow/pull/21222#discussion_r801116690
##########
File path: chart/tests/test_elasticsearch_secret.py
##########
@@ -132,3 +132,39 @@ def test_should_generate_secret_with_specified_schemes(self, scheme):
)
assert f"{scheme}://username:password@elastichostname:9200" == connection
+
+ def test_url_generated_when_user_is_empty(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"pass": "password", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://elastichostname:8080" == connection
+
+ def test_url_generated_when_password_is_empty(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"user": "admin", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://elastichostname:8080" == connection
+
+ def test_url_generated_with_valid_user_password(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"user": "admin", "pass": "pass", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://admin:pass@elastichostname:8080" == connection
Review comment:
I was kinda thinking something like this (untested, but might help?):
```
@parameterized([
({}, ""),
({"user": "admin", ""),
({"pass": "password", ""),
({"user": "admin", "pass": "password", "admin:password"),
])
def test_something(self, extra_conn_kwargs, expected_userinfo):
connection = self._get_connection({
"elasticsearch": {
"enabled": True,
"connection": {
"host": "elastichostname",
"port": 8080,
**extra_conn_kwargs
}
}
})
assert f"http://{expected_userinfo}@elastichostname:8080" == connection
````
I think not being a tuple might be your issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a change in pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on a change in pull request #21222:
URL: https://github.com/apache/airflow/pull/21222#discussion_r804016587
##########
File path: chart/tests/test_elasticsearch_secret.py
##########
@@ -132,3 +132,30 @@ def test_should_generate_secret_with_specified_schemes(self, scheme):
)
assert f"{scheme}://username:password@elastichostname:9200" == connection
+
+ @parameterized.expand(
+ [
+ # When both user and password are empty.
+ ({}, ""),
+ # When password is empty
+ ({"user": "admin"}, ""),
+ # When user is empty
+ ({"pass": "password"}, ""),
+ # Valid username/password
+ ({"user": "admin", "pass": "password"}, "admin:password"),
+ ],
+ )
+ def test_url_generated_when_user_pass_empty_combinations(self, extra_conn_kwargs, expected_user_info):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"host": "elastichostname", "port": 8080, **extra_conn_kwargs},
+ }
+ }
+ )
+
+ if not expected_user_info:
+ assert f"http://elastichostname:8080" == connection
Review comment:
```suggestion
assert "http://elastichostname:8080" == connection
```
Should fix the static check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham merged pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,
Posted by GitBox <gi...@apache.org>.
jedcunningham merged pull request #21222:
URL: https://github.com/apache/airflow/pull/21222
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #21222:
URL: https://github.com/apache/airflow/pull/21222#issuecomment-1035348537
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,
Posted by GitBox <gi...@apache.org>.
eladkal commented on pull request #21222:
URL: https://github.com/apache/airflow/pull/21222#issuecomment-1028829661
You'll need to rebase to fix the static checks. it was already fixed in main
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #21222:
URL: https://github.com/apache/airflow/pull/21222#discussion_r801109387
##########
File path: chart/tests/test_elasticsearch_secret.py
##########
@@ -132,3 +132,39 @@ def test_should_generate_secret_with_specified_schemes(self, scheme):
)
assert f"{scheme}://username:password@elastichostname:9200" == connection
+
+ def test_url_generated_when_user_is_empty(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"pass": "password", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://elastichostname:8080" == connection
+
+ def test_url_generated_when_password_is_empty(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"user": "admin", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://elastichostname:8080" == connection
+
+ def test_url_generated_with_valid_user_password(self):
+ connection = self._get_connection(
+ {
+ "elasticsearch": {
+ "enabled": True,
+ "connection": {"user": "admin", "pass": "pass", "host": "elastichostname", "port": 8080},
+ }
+ }
+ )
+
+ assert "http://admin:pass@elastichostname:8080" == connection
Review comment:
Thanks, Im actually having problems passing JSON in parametrized.expand
``` @parameterized.expand(
[
# When user is empty
({"enabled": True, "connection" : {"pass": "password", "host": "elastichostname", "port": 8080}}),
# When pass is empty
# ({"connection": {"user": "admin", "host": "elastichostname", "port": 8080}}),
# # When user and pass are empty
# ({"connection": {"host": "elastichostname", "port": 8080}}),
],
)```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #21222: Helm: Fix elasticsearch URL in secret when username/password fields are empty,
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #21222:
URL: https://github.com/apache/airflow/pull/21222#discussion_r803975711
##########
File path: chart/templates/secrets/elasticsearch-secret.yaml
##########
@@ -33,6 +33,14 @@ metadata:
type: Opaque
data:
{{- with .Values.elasticsearch.connection }}
- connection: {{ urlJoin (dict "scheme" (default "http" .scheme) "userinfo" (printf "%s:%s" (.user | urlquery) (.pass | urlquery)) "host" (printf "%s:%s" .host ((default 9200 .port) | toString) ) ) | b64enc | quote }}
- {{- end }}
+ {{- if .user }}
+ {{- if .pass }}
+ connection: {{ urlJoin (dict "scheme" (default "http" .scheme) "userinfo" (printf "%s:%s" (.user | urlquery) (.pass | urlquery)) "host" (printf "%s:%s" .host ((default 9200 .port) | toString) ) ) | b64enc | quote }}
+ {{- else}}
+ connection: {{ urlJoin (dict "scheme" (default "http" .scheme) "host" (printf "%s:%s" .host ((default 9200 .port) | toString))) | b64enc | quote }}
+ {{- end }}
Review comment:
Updated, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org