You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "MonkeyCanCode (via GitHub)" <gi...@apache.org> on 2025/07/22 03:58:15 UTC
[PR] Enable python client debug flag [polaris]
MonkeyCanCode opened a new pull request, #2154:
URL: https://github.com/apache/polaris/pull/2154
Enable request tracing via money patching:
Original:
```
➜ polaris git:(python_client_debug_flag) ./polaris --profile dev principals list
{"name": "root", "clientId": "root", "properties": {}, "createTimestamp": 1753151091780, "lastUpdateTimestamp": 1753151091780, "entityVersion": 1}
```
With debug mode (dummy password from the getting start example):
```
➜ polaris git:(python_client_debug_flag) ./polaris --profile dev --debug principals list
Request: POST http://localhost:8181/api/catalog/v1/oauth/tokens
Headers: HTTPHeaderDict({'Content-Type': 'application/x-www-form-urlencoded'})
Body: grant_type=client_credentials&client_id=root&client_secret=s3cr3t&scope=PRINCIPAL_ROLE%3AALL
Request: GET http://localhost:8181/api/management/v1/principals
Headers: {'Accept': 'application/json', 'User-Agent': 'OpenAPI-Generator/1.0.0/python', 'Authorization': 'Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJwb2xhcmlzIiwic3ViIjoiMSIsImlhdCI6MTc1MzE1NjUwMiwiZXhwIjoxNzUzMTYwMTAyLCJqdGkiOiIyMjc3NTc4Yi1mOGQ2LTQ3NjctODc2Yy0xZjg0YjVhOTFjOWIiLCJhY3RpdmUiOnRydWUsImNsaWVudF9pZCI6InJvb3QiLCJwcmluY2lwYWxJZCI6MSwic2NvcGUiOiJQUklOQ0lQQUxfUk9MRTpBTEwifQ.CXHMjqgGtGZZ9hUtS_Tqi4hr2Zt5kIdifqWVH0sWO2Ycbju3J6f4KWu4TgBB7FT6PQUbFqIAZiNBOAkZRpCDygUPEZ3-bt3cbTt9LWfkQYRXFnJF7-fM42zSK3Go7t1AVfYShDLozexgBvZGg700SYdcffKdCpARke7IX8VjLl_0uqZ4ygtjWwmoIHxfVH8WCTXajdvTOH7MF6aPgdVr5_y4KiCfZoTu5DG9gSJLYYAu1sVfUtQ07hGWs6wJUENLt9ZBjBBJI80Um70Oq3U508MC4y10FBWIKRrs8655le976TfcRjY_BPxpJaekBQIrxP7qz1hqP8XGSPRuFTgWbQ'}
{"name": "root", "clientId": "root", "properties": {}, "createTimestamp": 1753151091780, "lastUpdateTimestamp": 1753151091780, "entityVersion": 1}
```
Also, as the current code will requires an user to provide an argument even if the type is bool. To fix that (assuming that is not intensional), a code fix is provided (otherwise, we will need to do `--debug XXX`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #2154:
URL: https://github.com/apache/polaris/pull/2154#discussion_r2223363658
##########
client/python/cli/options/parser.py:
##########
@@ -57,22 +57,24 @@ class Parser(object):
),
Argument(Arguments.PROFILE, str, hint="profile for token-based authentication"),
Argument(Arguments.PROXY, str, hint="proxy URL"),
+ Argument(Arguments.DEBUG, bool, hint="Enable debug mode"),
]
@staticmethod
def _build_parser() -> argparse.ArgumentParser:
parser = TreeHelpParser(description="Polaris CLI")
for arg in Parser._ROOT_ARGUMENTS:
+ kwargs = {"help": arg.hint}
if arg.default is not None:
- parser.add_argument(
Review Comment:
But it looks like we only changed it for the root arguments, while the other boolean arguments (e.g. `CASCADE`) are unchanged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #2154:
URL: https://github.com/apache/polaris/pull/2154#discussion_r2242442378
##########
client/python/cli/polaris_cli.py:
##########
@@ -72,11 +74,33 @@ def execute(args=None):
admin_api = PolarisDefaultApi(api_client)
command = Command.from_options(options)
+ if options.debug:
+ PolarisCli._enable_api_request_logging()
command.execute(admin_api)
except Exception as e:
PolarisCli._try_print_exception(e)
sys.exit(1)
+ @staticmethod
+ def _enable_api_request_logging():
+ # Store the original urlopen method
+ if not hasattr(urllib3.PoolManager, "original_urlopen"):
+ urllib3.PoolManager.original_urlopen = urllib3.PoolManager.urlopen
+
+ # Define the wrapper function
+ @functools.wraps(urllib3.PoolManager.original_urlopen)
+ def urlopen_wrapper(self, method, url, **kwargs):
+ sys.stderr.write(f"Request: {method} {url}\n")
Review Comment:
I don't know if there's a standard format for this kind of thing -- what you have looks good to me though. If we were taking the dry-run approach, it might be nice to print a runnable command like a curl.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #2154:
URL: https://github.com/apache/polaris/pull/2154#discussion_r2223150544
##########
client/python/cli/polaris_cli.py:
##########
@@ -77,6 +81,26 @@ def execute(args=None):
PolarisCli._try_print_exception(e)
sys.exit(1)
+ @staticmethod
+ def _enable_api_request_logging():
+ # Store the original urlopen method
+ if not hasattr(urllib3.PoolManager, "original_urlopen"):
+ urllib3.PoolManager.original_urlopen = urllib3.PoolManager.urlopen
+
+ # Define the wrapper function
+ @functools.wraps(urllib3.PoolManager.original_urlopen)
+ def urlopen_wrapper(self, method, url, **kwargs):
+ print(f"Request: {method} {url}")
+ if "headers" in kwargs:
+ print(f"Headers: {kwargs['headers']}")
Review Comment:
Does it make sense to use STDERR so you can still handle stdout if need be?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #2154:
URL: https://github.com/apache/polaris/pull/2154#discussion_r2223138585
##########
client/python/cli/options/parser.py:
##########
@@ -57,22 +57,24 @@ class Parser(object):
),
Argument(Arguments.PROFILE, str, hint="profile for token-based authentication"),
Argument(Arguments.PROXY, str, hint="proxy URL"),
+ Argument(Arguments.DEBUG, bool, hint="Enable debug mode"),
]
@staticmethod
def _build_parser() -> argparse.ArgumentParser:
parser = TreeHelpParser(description="Polaris CLI")
for arg in Parser._ROOT_ARGUMENTS:
+ kwargs = {"help": arg.hint}
if arg.default is not None:
- parser.add_argument(
Review Comment:
Is this needed because this is the first root arg that's binary?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #2154:
URL: https://github.com/apache/polaris/pull/2154#discussion_r2242436388
##########
client/python/cli/options/parser.py:
##########
@@ -57,22 +57,24 @@ class Parser(object):
),
Argument(Arguments.PROFILE, str, hint="profile for token-based authentication"),
Argument(Arguments.PROXY, str, hint="proxy URL"),
+ Argument(Arguments.DEBUG, bool, hint="Enable debug mode"),
Review Comment:
Oh, I see. For some commands it's not easy to support dry-run. I would be fine with just failing in that case, or if we want to be really fancy we can let the user manually run the command and paste the response into stdin.
Anyway, if this all sounds like too much work let's just skip it -- was only an idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on code in PR #2154:
URL: https://github.com/apache/polaris/pull/2154#discussion_r2244215639
##########
client/python/cli/polaris_cli.py:
##########
@@ -72,11 +74,33 @@ def execute(args=None):
admin_api = PolarisDefaultApi(api_client)
command = Command.from_options(options)
+ if options.debug:
+ PolarisCli._enable_api_request_logging()
command.execute(admin_api)
except Exception as e:
PolarisCli._try_print_exception(e)
sys.exit(1)
+ @staticmethod
+ def _enable_api_request_logging():
+ # Store the original urlopen method
+ if not hasattr(urllib3.PoolManager, "original_urlopen"):
+ urllib3.PoolManager.original_urlopen = urllib3.PoolManager.urlopen
+
+ # Define the wrapper function
+ @functools.wraps(urllib3.PoolManager.original_urlopen)
+ def urlopen_wrapper(self, method, url, **kwargs):
+ sys.stderr.write(f"Request: {method} {url}\n")
Review Comment:
Okay, I can do that in the next PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on code in PR #2154:
URL: https://github.com/apache/polaris/pull/2154#discussion_r2223340373
##########
client/python/cli/options/parser.py:
##########
@@ -57,22 +57,24 @@ class Parser(object):
),
Argument(Arguments.PROFILE, str, hint="profile for token-based authentication"),
Argument(Arguments.PROXY, str, hint="proxy URL"),
+ Argument(Arguments.DEBUG, bool, hint="Enable debug mode"),
]
@staticmethod
def _build_parser() -> argparse.ArgumentParser:
parser = TreeHelpParser(description="Polaris CLI")
for arg in Parser._ROOT_ARGUMENTS:
+ kwargs = {"help": arg.hint}
if arg.default is not None:
- parser.add_argument(
Review Comment:
So with current code, when we provide bool flag, it will still expected an argument. Thus, I made this change. When no bool flag is provided, it is False. If bool flag is ever provided, it is True (which matched to the behavior in add_arguments).
##########
client/python/cli/polaris_cli.py:
##########
@@ -65,6 +67,8 @@ def execute(args=None):
command = Command.from_options(options)
command.execute()
else:
+ if options.debug:
+ PolarisCli._enable_api_request_logging()
Review Comment:
k. Let me move this one.
##########
client/python/cli/polaris_cli.py:
##########
@@ -77,6 +81,26 @@ def execute(args=None):
PolarisCli._try_print_exception(e)
sys.exit(1)
+ @staticmethod
+ def _enable_api_request_logging():
+ # Store the original urlopen method
+ if not hasattr(urllib3.PoolManager, "original_urlopen"):
+ urllib3.PoolManager.original_urlopen = urllib3.PoolManager.urlopen
+
+ # Define the wrapper function
+ @functools.wraps(urllib3.PoolManager.original_urlopen)
+ def urlopen_wrapper(self, method, url, **kwargs):
+ print(f"Request: {method} {url}")
+ if "headers" in kwargs:
+ print(f"Headers: {kwargs['headers']}")
Review Comment:
Actually, that is a very good idea for CLI. I will make this change.
##########
client/python/cli/options/parser.py:
##########
@@ -57,22 +57,24 @@ class Parser(object):
),
Argument(Arguments.PROFILE, str, hint="profile for token-based authentication"),
Argument(Arguments.PROXY, str, hint="proxy URL"),
+ Argument(Arguments.DEBUG, bool, hint="Enable debug mode"),
Review Comment:
so a single mode flag and 3 modes supported? What will dry-run do? show the request?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #2154:
URL: https://github.com/apache/polaris/pull/2154#discussion_r2223358864
##########
client/python/cli/options/parser.py:
##########
@@ -57,22 +57,24 @@ class Parser(object):
),
Argument(Arguments.PROFILE, str, hint="profile for token-based authentication"),
Argument(Arguments.PROXY, str, hint="proxy URL"),
+ Argument(Arguments.DEBUG, bool, hint="Enable debug mode"),
Review Comment:
Yeah, just show the request. This isn't a blocker, I'm just wondering if it might be useful because
1. dry-run would be very useful for testing
2. the current mode ("debug"?) is a little fragile because it non-transactionally does the printing and executes the command
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on code in PR #2154:
URL: https://github.com/apache/polaris/pull/2154#discussion_r2224227690
##########
client/python/cli/options/parser.py:
##########
@@ -57,22 +57,24 @@ class Parser(object):
),
Argument(Arguments.PROFILE, str, hint="profile for token-based authentication"),
Argument(Arguments.PROXY, str, hint="proxy URL"),
+ Argument(Arguments.DEBUG, bool, hint="Enable debug mode"),
]
@staticmethod
def _build_parser() -> argparse.ArgumentParser:
parser = TreeHelpParser(description="Polaris CLI")
for arg in Parser._ROOT_ARGUMENTS:
+ kwargs = {"help": arg.hint}
if arg.default is not None:
- parser.add_argument(
Review Comment:
that is correct, the above only change for the root one as it is the one that is enforcing everything to have an value for a given argument. The non-root arguments should work with following:
```
if arg.type is bool:
del kwargs['type']
parser.add_argument(arg.get_flag_name(), **kwargs, action='store_true')
````
E.g.
```
➜ polaris git:(python_client_debug_flag) ✗ ./polaris --profile dev privileges catalog revoke --catalog quickstart_catalog --catalog-role quickstart_catalog_role --cascade CATALOG_MANAGE_CONTENT
Exception when communicating with the Polaris server. 501: Not Implemented
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on code in PR #2154:
URL: https://github.com/apache/polaris/pull/2154#discussion_r2244214932
##########
client/python/cli/options/parser.py:
##########
@@ -57,22 +57,24 @@ class Parser(object):
),
Argument(Arguments.PROFILE, str, hint="profile for token-based authentication"),
Argument(Arguments.PROXY, str, hint="proxy URL"),
+ Argument(Arguments.DEBUG, bool, hint="Enable debug mode"),
Review Comment:
Yes, maybe come back later for this implementation if this is helpful for debugging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode merged PR #2154:
URL: https://github.com/apache/polaris/pull/2154
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #2154:
URL: https://github.com/apache/polaris/pull/2154#discussion_r2242440098
##########
client/python/cli/polaris_cli.py:
##########
@@ -72,11 +74,33 @@ def execute(args=None):
admin_api = PolarisDefaultApi(api_client)
command = Command.from_options(options)
+ if options.debug:
+ PolarisCli._enable_api_request_logging()
command.execute(admin_api)
except Exception as e:
PolarisCli._try_print_exception(e)
sys.exit(1)
+ @staticmethod
+ def _enable_api_request_logging():
+ # Store the original urlopen method
+ if not hasattr(urllib3.PoolManager, "original_urlopen"):
+ urllib3.PoolManager.original_urlopen = urllib3.PoolManager.urlopen
+
+ # Define the wrapper function
+ @functools.wraps(urllib3.PoolManager.original_urlopen)
+ def urlopen_wrapper(self, method, url, **kwargs):
+ sys.stderr.write(f"Request: {method} {url}\n")
+ if "headers" in kwargs:
+ sys.stderr.write(f"Headers: {kwargs['headers']}\n")
+ if "body" in kwargs:
+ sys.stderr.write(f"Body: {kwargs['body']}\n")
+ sys.stderr.write("\n")
Review Comment:
nit: looks like this newline is not needed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #2154:
URL: https://github.com/apache/polaris/pull/2154#discussion_r2223148799
##########
client/python/cli/options/parser.py:
##########
@@ -57,22 +57,24 @@ class Parser(object):
),
Argument(Arguments.PROFILE, str, hint="profile for token-based authentication"),
Argument(Arguments.PROXY, str, hint="proxy URL"),
+ Argument(Arguments.DEBUG, bool, hint="Enable debug mode"),
Review Comment:
Thinking this over, it might be nice to have multiple modes, e.g. `default`, `debug`, `dry-run`. Sometimes you just want to see the command without running it, sometimes you want to see it and run it, and sometimes you just want to run the commands.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #2154:
URL: https://github.com/apache/polaris/pull/2154#discussion_r2223144445
##########
client/python/cli/polaris_cli.py:
##########
@@ -65,6 +67,8 @@ def execute(args=None):
command = Command.from_options(options)
command.execute()
else:
+ if options.debug:
+ PolarisCli._enable_api_request_logging()
Review Comment:
I think we should do this after the command is parsed but before it's executed... or, if possible, in the middle of command execution. As the logic is right now it looks like there are cases where we'd emit the logging but not actually make the request.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable python client debug flag [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on code in PR #2154:
URL: https://github.com/apache/polaris/pull/2154#discussion_r2224255394
##########
client/python/cli/options/parser.py:
##########
@@ -57,22 +57,24 @@ class Parser(object):
),
Argument(Arguments.PROFILE, str, hint="profile for token-based authentication"),
Argument(Arguments.PROXY, str, hint="proxy URL"),
+ Argument(Arguments.DEBUG, bool, hint="Enable debug mode"),
Review Comment:
so looking at implementing the dry-run mode, it turns out a bit more work will be needed. For example, for a single update principal property request, it will need to run following requests:
1. get principal
2. update principal
Now as there is no native way (as far as I know) to do this dry-run, we will then need to do another monkey patching on the urlopen. Now the challenge is as a single CLI request will be convert to multiple actual API requests, we will then need to create proper mock responses for each of the request to move forward until we hit the last one.
If this is the feature we would like to support (and no other better solution other than doing a bunch mock responses), I can proceed with those implementation. The only catch that I can think off is later on if we changed API on diff version, we will need to keep those up to date (e.g. have mock for v1 and v2 as they may have diff response etc.). Unlike the debug mode, where we are really just enrich the output but showing request/header/body.
Let me know what you think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org