You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/12/16 19:52:03 UTC
[GitHub] [iceberg] rubenvdg opened a new pull request, #6445: Python: Mock home and root folder when running `test_missing_uri`
rubenvdg opened a new pull request, #6445:
URL: https://github.com/apache/iceberg/pull/6445
This could be a way to alleviate this issue https://github.com/apache/iceberg/issues/6361.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] arminnajafi commented on pull request #6445: Python: Mock home and root folder when running `test_missing_uri`
Posted by GitBox <gi...@apache.org>.
arminnajafi commented on PR #6445:
URL: https://github.com/apache/iceberg/pull/6445#issuecomment-1385032472
https://github.com/apache/iceberg/pull/6607
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on a diff in pull request #6445: Python: Mock home and root folder when running `test_missing_uri`
Posted by GitBox <gi...@apache.org>.
Fokko commented on code in PR #6445:
URL: https://github.com/apache/iceberg/pull/6445#discussion_r1051172204
##########
python/tests/utils/test_config.py:
##########
@@ -39,7 +40,7 @@ def test_from_environment_variables_uppercase() -> None:
assert Config().get_catalog_config("PRODUCTION") == {"uri": "https://service.io/api"}
-def test_from_configuration_files(tmp_path_factory) -> None: # type: ignore
+def test_from_configuration_files(tmp_path_factory: pytest.TempPathFactory) -> None:
Review Comment:
Nice!
##########
python/tests/cli/test_console.py:
##########
@@ -134,14 +134,17 @@ def update_namespace_properties(
MOCK_ENVIRONMENT = {"PYICEBERG_CATALOG__PRODUCTION__URI": "test://doesnotexist"}
-def test_missing_uri() -> None:
- runner = CliRunner()
- result = runner.invoke(run, ["list"])
- assert result.exit_code == 1
- assert (
- result.output
- == "URI missing, please provide using --uri, the config or environment variable \nPYICEBERG_CATALOG__DEFAULT__URI\n"
- )
+def test_missing_uri(empty_home_dir_path: str) -> None:
Review Comment:
This looks good to me! I'd rather use the decorator `@mock.patch.dict(os.environ, MOCK_ENVIRONMENT)` similar as the test below, but I'm aware that we that we can't use the fixture on a global scope.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] arminnajafi commented on pull request #6445: Python: Mock home and root folder when running `test_missing_uri`
Posted by GitBox <gi...@apache.org>.
arminnajafi commented on PR #6445:
URL: https://github.com/apache/iceberg/pull/6445#issuecomment-1385007961
One more mock is needed to make it work:
```
with mock.patch('pyiceberg.catalog._ENV_CONFIG', Config()):
```
```
def test_missing_uri(empty_home_dir_path: str) -> None:
# mock to prevent parsing ~/.pyiceberg.yaml or {PYICEBERG_HOME}/.pyiceberg.yaml
with mock.patch.dict(os.environ, {"HOME": empty_home_dir_path, "PYICEBERG_HOME": empty_home_dir_path}):
with mock.patch('pyiceberg.catalog._ENV_CONFIG', Config()):
runner = CliRunner()
result = runner.invoke(run, ["list"])
assert result.exit_code == 1
assert (
result.output
== "URI missing, please provide using --uri, the config or environment variable \nPYICEBERG_CATALOG__DEFAULT__URI\n"
)
```
```
> make test
...
1519 passed, 22 deselected, 1 warning in 6.14s
```
I'll send a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rubenvdg commented on a diff in pull request #6445: Python: Mock home and root folder when running `test_missing_uri`
Posted by GitBox <gi...@apache.org>.
rubenvdg commented on code in PR #6445:
URL: https://github.com/apache/iceberg/pull/6445#discussion_r1051331332
##########
python/tests/cli/test_console.py:
##########
@@ -134,14 +134,17 @@ def update_namespace_properties(
MOCK_ENVIRONMENT = {"PYICEBERG_CATALOG__PRODUCTION__URI": "test://doesnotexist"}
-def test_missing_uri() -> None:
- runner = CliRunner()
- result = runner.invoke(run, ["list"])
- assert result.exit_code == 1
- assert (
- result.output
- == "URI missing, please provide using --uri, the config or environment variable \nPYICEBERG_CATALOG__DEFAULT__URI\n"
- )
+def test_missing_uri(empty_home_dir_path: str) -> None:
Review Comment:
Yeah me too. If you're adamant about it, could do the following, but I'd say it's a pretty terrible workaround
```
def test_missing_uri(empty_home_dir_path)
@mock.patch.dict(os.environ, {"HOME": empty_home_dir_path})
def _test_missing_uri():
...
_test_missing_uri()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rubenvdg commented on a diff in pull request #6445: Python: Mock home and root folder when running `test_missing_uri`
Posted by GitBox <gi...@apache.org>.
rubenvdg commented on code in PR #6445:
URL: https://github.com/apache/iceberg/pull/6445#discussion_r1051331332
##########
python/tests/cli/test_console.py:
##########
@@ -134,14 +134,17 @@ def update_namespace_properties(
MOCK_ENVIRONMENT = {"PYICEBERG_CATALOG__PRODUCTION__URI": "test://doesnotexist"}
-def test_missing_uri() -> None:
- runner = CliRunner()
- result = runner.invoke(run, ["list"])
- assert result.exit_code == 1
- assert (
- result.output
- == "URI missing, please provide using --uri, the config or environment variable \nPYICEBERG_CATALOG__DEFAULT__URI\n"
- )
+def test_missing_uri(empty_home_dir_path: str) -> None:
Review Comment:
Yeah me too. If you're adamant about it, could do the following, but I'd say it's a pretty terrible workaround
```
def test_missing_uri(empty_home_dir_path)
@mock.patch.dict(os.environ, {"HOME": empty_home_dir_path})
def _test_missing_uri():
...
_test_missing_uri()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on a diff in pull request #6445: Python: Mock home and root folder when running `test_missing_uri`
Posted by GitBox <gi...@apache.org>.
Fokko commented on code in PR #6445:
URL: https://github.com/apache/iceberg/pull/6445#discussion_r1051363319
##########
python/tests/cli/test_console.py:
##########
@@ -134,14 +134,17 @@ def update_namespace_properties(
MOCK_ENVIRONMENT = {"PYICEBERG_CATALOG__PRODUCTION__URI": "test://doesnotexist"}
-def test_missing_uri() -> None:
- runner = CliRunner()
- result = runner.invoke(run, ["list"])
- assert result.exit_code == 1
- assert (
- result.output
- == "URI missing, please provide using --uri, the config or environment variable \nPYICEBERG_CATALOG__DEFAULT__URI\n"
- )
+def test_missing_uri(empty_home_dir_path: str) -> None:
Review Comment:
But then we also should invoke the `make_temporary_home_folder`, I prefer the current solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko merged pull request #6445: Python: Mock home and root folder when running `test_missing_uri`
Posted by GitBox <gi...@apache.org>.
Fokko merged PR #6445:
URL: https://github.com/apache/iceberg/pull/6445
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #6445: Python: Mock home and root folder when running `test_missing_uri`
Posted by GitBox <gi...@apache.org>.
rdblue commented on PR #6445:
URL: https://github.com/apache/iceberg/pull/6445#issuecomment-1355564692
Looks good to me! @Fokko, can you take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rubenvdg commented on a diff in pull request #6445: Python: Mock home and root folder when running `test_missing_uri`
Posted by GitBox <gi...@apache.org>.
rubenvdg commented on code in PR #6445:
URL: https://github.com/apache/iceberg/pull/6445#discussion_r1051332065
##########
python/tests/cli/test_console.py:
##########
@@ -134,14 +134,17 @@ def update_namespace_properties(
MOCK_ENVIRONMENT = {"PYICEBERG_CATALOG__PRODUCTION__URI": "test://doesnotexist"}
-def test_missing_uri() -> None:
- runner = CliRunner()
- result = runner.invoke(run, ["list"])
- assert result.exit_code == 1
- assert (
- result.output
- == "URI missing, please provide using --uri, the config or environment variable \nPYICEBERG_CATALOG__DEFAULT__URI\n"
- )
+def test_missing_uri(empty_home_dir_path: str) -> None:
Review Comment:
Alternatively, we could do something like this in `conftest.py`:
```
def make_temporary_home_folder(tmp_path_factory) -> None:
home_path = str(tmp_path_factory.mktemp("home"))
os.environ["TMP_HOME_PATH"] = home_path
```
and then in the test
```
@mock.patch.dict(os.environ, {"HOME": os.environ["TMP_HOME_PATH"]})
def test_missing_uri():
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] arminnajafi commented on pull request #6445: Python: Mock home and root folder when running `test_missing_uri`
Posted by GitBox <gi...@apache.org>.
arminnajafi commented on PR #6445:
URL: https://github.com/apache/iceberg/pull/6445#issuecomment-1384992306
I don't think this change is not cutting it.
I think it because before we get to the patch line:
```
# mock to prevent parsing ~/.pyiceberg.yaml or {PYICEBERG_HOME}/.pyiceberg.yaml
with mock.patch.dict(os.environ, {"HOME": empty_home_dir_path, "PYICEBERG_HOME": empty_home_dir_path}):
```
Config is already loaded by the one of imports statements in the `test_console.py` file and below code already runs and read the real $HOME env.:
https://github.com/apache/iceberg/blob/596d7b876dcd9a9d5a5242622e3abaf8f069f3dc/python/pyiceberg/catalog/__init__.py#L51
`make test` fails for me with:
```
tests/cli/test_console.py:144: AssertionError
...
E AssertionError: assert 'You must specify a region.\n' == 'URI missing,...EFAULT__URI\n'
E + You must specify a region.
E - URI missing, please provide using --uri, the config or environment variable
E - PYICEBERG_CATALOG__DEFAULT__URI
```
```
> cat ~/.pyiceberg.yaml
catalog:
default:
type: glue
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org