You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/09/14 23:00:18 UTC
[GitHub] [airflow] mik-laj opened a new pull request #18258: Improve coverage for airflow.security.kerberos module
mik-laj opened a new pull request #18258:
URL: https://github.com/apache/airflow/pull/18258
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #18258: Improve coverage for airflow.security.kerberos module
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #18258:
URL: https://github.com/apache/airflow/pull/18258
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed pull request #18258: Improve coverage for airflow.security.kerberos module
Posted by GitBox <gi...@apache.org>.
potiuk closed pull request #18258:
URL: https://github.com/apache/airflow/pull/18258
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #18258: Improve coverage for airflow.security.kerberos module
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #18258:
URL: https://github.com/apache/airflow/pull/18258#discussion_r708834881
##########
File path: tests/security/test_kerberos.py
##########
@@ -104,3 +105,255 @@ def test_args_from_cli(self):
)
assert ctx.value.code == 1
+
+
+class TestKerberosUnit(unittest.TestCase):
Review comment:
Should we use a pytest setup instead? `pytest.mark.paramtrize`, `caplog`, and `pytest.raises` are easier to use than `parameterized`, `assertLogs`, and `assertRaises` IMO. But this is minor and I’m OK if you feel the current approach is more familiar etc.
##########
File path: tests/security/test_kerberos.py
##########
@@ -104,3 +105,255 @@ def test_args_from_cli(self):
)
assert ctx.value.code == 1
+
+
+class TestKerberosUnit(unittest.TestCase):
+ @parameterized.expand(
+ [
+ (
+ {('kerberos', 'reinit_frequency'): '42'},
+ [
+ 'kinit',
+ '-f',
+ '-a',
+ '-r',
+ '42m',
+ '-k',
+ '-t',
+ 'keytab',
+ '-c',
+ '/tmp/airflow_krb5_ccache',
+ 'test-principal',
+ ],
+ ),
+ (
+ {('kerberos', 'forwardable'): 'True', ('kerberos', 'include_ip'): 'True'},
+ [
+ 'kinit',
+ '-f',
+ '-a',
+ '-r',
+ '3600m',
+ '-k',
+ '-t',
+ 'keytab',
+ '-c',
+ '/tmp/airflow_krb5_ccache',
+ 'test-principal',
+ ],
+ ),
+ (
+ {('kerberos', 'forwardable'): 'False', ('kerberos', 'include_ip'): 'False'},
+ [
+ 'kinit',
+ '-F',
+ '-A',
+ '-r',
+ '3600m',
+ '-k',
+ '-t',
+ 'keytab',
+ '-c',
+ '/tmp/airflow_krb5_ccache',
+ 'test-principal',
+ ],
+ ),
+ ]
+ )
+ def test_renew_from_kt(self, kerberos_config, expected_cmd):
+ with self.assertLogs(kerberos.log) as log_ctx, conf_vars(kerberos_config), mock.patch(
+ 'airflow.security.kerberos.subprocess'
+ ) as mock_subprocess, mock.patch(
+ 'airflow.security.kerberos.NEED_KRB181_WORKAROUND', None
+ ), mock.patch(
+ 'airflow.security.kerberos.open', mock.mock_open(read_data=b'X-CACHECONF:')
+ ), mock.patch(
+ 'time.sleep', return_value=None
+ ):
+ mock_subprocess.Popen.return_value.__enter__.return_value.returncode = 0
+ mock_subprocess.call.return_value = 0
+ renew_from_kt(principal="test-principal", keytab="keytab")
+
+ assert mock_subprocess.Popen.call_args[0][0] == expected_cmd
+
+ expected_cmd_text = " ".join(shlex.quote(f) for f in expected_cmd)
+ assert log_ctx.output == [
+ f'INFO:airflow.security.kerberos:Re-initialising kerberos from keytab: {expected_cmd_text}',
+ 'INFO:airflow.security.kerberos:Renewing kerberos ticket to work around kerberos 1.8.1: '
+ 'kinit -c /tmp/airflow_krb5_ccache -R',
+ ]
+
+ mock_subprocess.assert_has_calls(
+ [
+ mock.call.Popen(
+ expected_cmd,
+ bufsize=-1,
+ close_fds=True,
+ stderr=mock_subprocess.PIPE,
+ stdout=mock_subprocess.PIPE,
+ universal_newlines=True,
+ ),
+ mock.call.Popen().__enter__(),
+ mock.call.Popen().__enter__().wait(),
+ mock.call.Popen().__exit__(None, None, None),
+ mock.call.call(['kinit', '-c', '/tmp/airflow_krb5_ccache', '-R'], close_fds=True),
+ ]
+ )
Review comment:
`assert_has_calls` [is non-exhaustive](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_has_calls). Are we expecting `subprocess.Popen` to be called by anything else here? If not, we should use `assert mock_subprocess.mock_calls == [...]` isntead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #18258: Improve coverage for airflow.security.kerberos module
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #18258:
URL: https://github.com/apache/airflow/pull/18258#discussion_r709131658
##########
File path: tests/security/test_kerberos.py
##########
@@ -104,3 +105,255 @@ def test_args_from_cli(self):
)
assert ctx.value.code == 1
+
+
+class TestKerberosUnit(unittest.TestCase):
+ @parameterized.expand(
+ [
+ (
+ {('kerberos', 'reinit_frequency'): '42'},
+ [
+ 'kinit',
+ '-f',
+ '-a',
+ '-r',
+ '42m',
+ '-k',
+ '-t',
+ 'keytab',
+ '-c',
+ '/tmp/airflow_krb5_ccache',
+ 'test-principal',
+ ],
+ ),
+ (
+ {('kerberos', 'forwardable'): 'True', ('kerberos', 'include_ip'): 'True'},
+ [
+ 'kinit',
+ '-f',
+ '-a',
+ '-r',
+ '3600m',
+ '-k',
+ '-t',
+ 'keytab',
+ '-c',
+ '/tmp/airflow_krb5_ccache',
+ 'test-principal',
+ ],
+ ),
+ (
+ {('kerberos', 'forwardable'): 'False', ('kerberos', 'include_ip'): 'False'},
+ [
+ 'kinit',
+ '-F',
+ '-A',
+ '-r',
+ '3600m',
+ '-k',
+ '-t',
+ 'keytab',
+ '-c',
+ '/tmp/airflow_krb5_ccache',
+ 'test-principal',
+ ],
+ ),
+ ]
+ )
+ def test_renew_from_kt(self, kerberos_config, expected_cmd):
+ with self.assertLogs(kerberos.log) as log_ctx, conf_vars(kerberos_config), mock.patch(
+ 'airflow.security.kerberos.subprocess'
+ ) as mock_subprocess, mock.patch(
+ 'airflow.security.kerberos.NEED_KRB181_WORKAROUND', None
+ ), mock.patch(
+ 'airflow.security.kerberos.open', mock.mock_open(read_data=b'X-CACHECONF:')
+ ), mock.patch(
+ 'time.sleep', return_value=None
+ ):
+ mock_subprocess.Popen.return_value.__enter__.return_value.returncode = 0
+ mock_subprocess.call.return_value = 0
+ renew_from_kt(principal="test-principal", keytab="keytab")
+
+ assert mock_subprocess.Popen.call_args[0][0] == expected_cmd
+
+ expected_cmd_text = " ".join(shlex.quote(f) for f in expected_cmd)
+ assert log_ctx.output == [
+ f'INFO:airflow.security.kerberos:Re-initialising kerberos from keytab: {expected_cmd_text}',
+ 'INFO:airflow.security.kerberos:Renewing kerberos ticket to work around kerberos 1.8.1: '
+ 'kinit -c /tmp/airflow_krb5_ccache -R',
+ ]
+
+ mock_subprocess.assert_has_calls(
+ [
+ mock.call.Popen(
+ expected_cmd,
+ bufsize=-1,
+ close_fds=True,
+ stderr=mock_subprocess.PIPE,
+ stdout=mock_subprocess.PIPE,
+ universal_newlines=True,
+ ),
+ mock.call.Popen().__enter__(),
+ mock.call.Popen().__enter__().wait(),
+ mock.call.Popen().__exit__(None, None, None),
+ mock.call.call(['kinit', '-c', '/tmp/airflow_krb5_ccache', '-R'], close_fds=True),
+ ]
+ )
Review comment:
Updated. Thanks. I was concerned if mock.ANY would work in this situation, but it works.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #18258: Improve coverage for airflow.security.kerberos module
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #18258:
URL: https://github.com/apache/airflow/pull/18258#issuecomment-922347215
Reopened to rebuild.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #18258: Improve coverage for airflow.security.kerberos module
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #18258:
URL: https://github.com/apache/airflow/pull/18258#discussion_r709135191
##########
File path: tests/security/test_kerberos.py
##########
@@ -104,3 +105,255 @@ def test_args_from_cli(self):
)
assert ctx.value.code == 1
+
+
+class TestKerberosUnit(unittest.TestCase):
Review comment:
This is very subjective, but I checked the popularity of each approach and I can see that it is very close.
PS. We had discussions on the mailing list about this. https://lists.apache.org/thread.html/1e4df7d4b0cd9b2d2bb76a3336471aa85e852545dd41ada6d4e461b8%40%3Cdev.airflow.apache.org%3E Feel free to join it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org