You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by GitBox <gi...@apache.org> on 2021/07/27 14:51:13 UTC
[GitHub] [ranger] mateusz-szczesny opened a new pull request #111: Parse URL using urlib.parse or urlparse instead of os package
mateusz-szczesny opened a new pull request #111:
URL: https://github.com/apache/ranger/pull/111
Working on windows produces bug when given Ranger URL has not leading slash which is not common behavior in base URL structure.
```python
from apache_ranger.client.ranger_client import RangerService, RangerClient
ranger_url = "http://dummy.hostname.com:6080"
ranger_auth = ("dummy.username", "dummy.password")
ranger = RangerClient(ranger_url, ranger_auth)
for role in ranger.find_roles():
print(str(role.id))
```
Above code leads to such error on Windows like systems.
```
Traceback (most recent call last):
File "C:\Users\szczesny\Desktop\cdpctl\main.py", line 8, in <module>
for role in ranger.find_roles():
File "C:\Users\szczesny\AppData\Local\pypoetry\Cache\virtualenvs\cdpctl-1KAmzwEP-py3.9\lib\site-packages\apache_ranger\client\ranger_client.py", line 327, in find_roles
resp = self.__call_api(RangerClient.FIND_ROLES, filter)
File "C:\Users\szczesny\AppData\Local\pypoetry\Cache\virtualenvs\cdpctl-1KAmzwEP-py3.9\lib\site-packages\apache_ranger\client\ranger_client.py", line 427, in __call_api
raise RangerServiceException(api, response)
File "C:\Users\szczesny\AppData\Local\pypoetry\Cache\virtualenvs\cdpctl-1KAmzwEP-py3.9\lib\site-packages\apache_ranger\exceptions.py", line 39, in __init__
respJson = response.json()
File "C:\Users\szczesny\AppData\Local\pypoetry\Cache\virtualenvs\cdpctl-1KAmzwEP-py3.9\lib\site-packages\requests\models.py", line 910, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\szczesny\AppData\Local\Programs\Python\Python39\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Users\szczesny\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\szczesny\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
```
It can be fixed by manually adding leading slash, but it is just an ugly workaround and may lead to problem when hostname along with port will come from env vars.
```python
[...]
ranger_url = "http://dummy.hostname.com:6080/"
[...]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] JeevansSP commented on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
JeevansSP commented on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1071060114
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] mateusz-szczesny commented on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
mateusz-szczesny commented on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1058967297
i see this problem has been already solved, thus closing 👏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] JeevansSP edited a comment on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
JeevansSP edited a comment on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1070548466
about the python client it is the latest one i just did a pip install
i used a helm chart which i have deployed to my k8s cluster
have been trying to debug it for weeks now
i tried this too https://github.com/apache/ranger/commit/e1ec22406b915e5930ead8ef79128d74122d2c7e but it did not work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] JeevansSP edited a comment on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
JeevansSP edited a comment on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1070530577
also for the code given by @mateusz-szczesny , your workaround did not work :( kindly help
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] JeevansSP commented on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
JeevansSP commented on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1071104503
idk if this helps but what i found is the error remains the same even with wrong user and pass, but if i change the host or port then it gives a diff error(max retries exeeded)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] JeevansSP commented on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
JeevansSP commented on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1070620836
Thanks i have created an issue :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] yungParrot commented on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
yungParrot commented on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1070544626
@JeevansSP its hard to tell what causes this problem... what version of ranger are you using? maybe an older version which doesn't have this issue fixed? if it still occurs you should probably create a new issue 🤔
---
btw when formating your code with ``` you can specify the syntax highlighting e.g. ```python so
this:
```
x = 123
print(x)
```
will look like this:
```python
x = 123
print(x)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] JeevansSP commented on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
JeevansSP commented on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1070548466
about the python client it is the latest one i just did a pip install
so this the link to my helm chart which i have deployed
https://raw.githubusercontent.com/starburstdata/starburst-deploy/main/helm/ranger.yaml
i have been trying to debug it for weeks now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] mateusz-szczesny closed pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
mateusz-szczesny closed pull request #111:
URL: https://github.com/apache/ranger/pull/111
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] JeevansSP commented on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
JeevansSP commented on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1070523172
`from apache_ranger.model.ranger_service import *
from apache_ranger.client.ranger_client import *
from apache_ranger.model.ranger_policy import *
ranger_url = "http://*******.com:****/"
ranger_auth = ("*********", "**********")
ranger = RangerClient(ranger_url, ranger_auth)
policy = RangerPolicy()
policy.service = "starburst-enterprise"
policy.name = "test_policy"
policy.resources = { "database": RangerPolicyResource({ "values": ["hive"] }),
"schema": RangerPolicyResource({"values":["data_mesh"]}),
"table": RangerPolicyResource({ "values": ["uber_rides"] }),
"column": RangerPolicyResource({ "values": ["*"] }) }
allowItem1 = RangerPolicyItem()
allowItem1.groups = [ "Data Consumer" ]
allowItem1.accesses = [ RangerPolicyItemAccess({ "type": "select" }) ]
denyItem1 = RangerPolicyItem()
denyItem1.groups = [ "Data Consumer" ]
denyItem1.accesses = [ RangerPolicyItemAccess({ "type": "drop" }) ]
policy.policyItems = [ allowItem1 ]
policy.denyPolicyItems = [ denyItem1 ]
print('Creating policy: name=' + policy.name)
created_policy = ranger.create_policy(policy)
print(' created policy: name=' + created_policy.name + ', id=' + str(created_policy.id))
`
Hello, my Code works fine upto line 9 with or without the leading slash but anytime i try to create a policy, service etc , anything that requires a JSON file i get this error
`<Response [400]>
Traceback (most recent call last):
File "c:\Users\j.shimoga.prakash\Desktop\pls work - Copy\test.py", line 38, in <module>
created_policy = ranger.create_policy(policy)
File "C:\Users\j.shimoga.prakash\Desktop\pls work - Copy\venv\lib\site-packages\apache_ranger\client\ranger_client.py", line 124, in create_policy
resp = self.__call_api(RangerClient.CREATE_POLICY, params, policy)
File "C:\Users\j.shimoga.prakash\Desktop\pls work - Copy\venv\lib\site-packages\apache_ranger\client\ranger_client.py", line 333, in __call_api
raise RangerServiceException(api, response)
File "C:\Users\j.shimoga.prakash\Desktop\pls work - Copy\venv\lib\site-packages\apache_ranger\exceptions.py", line 39, in __init__
respJson = response.json()
File "C:\Users\j.shimoga.prakash\Desktop\pls work - Copy\venv\lib\site-packages\requests\models.py", line 910, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\j.shimoga.prakash\Desktop\pls work - Copy\venv\lib\site-packages\simplejson\__init__.py", line 525, in loads
return _default_decoder.decode(s)
File "C:\Users\j.shimoga.prakash\Desktop\pls work - Copy\venv\lib\site-packages\simplejson\decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "C:\Users\j.shimoga.prakash\Desktop\pls work - Copy\venv\lib\site-packages\simplejson\decoder.py", line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)`
Can someone please help me out?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] JeevansSP edited a comment on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
JeevansSP edited a comment on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1070548466
about the python client it is the latest one i just did a pip install
so this the link to my helm chart which i have deployed
https://raw.githubusercontent.com/starburstdata/starburst-deploy/main/helm/ranger.yaml
i have been trying to debug it for weeks now
i tried this too https://github.com/apache/ranger/commit/e1ec22406b915e5930ead8ef79128d74122d2c7e but it did not work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] JeevansSP commented on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
JeevansSP commented on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1071060114
hey i created the issue but i dont think anyones ever gonna reply :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] JeevansSP commented on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
JeevansSP commented on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1070524748
@mateusz-szczesny can you kindly look into this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] JeevansSP commented on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
JeevansSP commented on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1070582352
how do i create a new issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] JeevansSP edited a comment on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
JeevansSP edited a comment on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1070530577
also for the code given by @mateusz-szczesny , adding a forward slash does not work :( kindly help
i tried your workaround but it does not work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] JeevansSP commented on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
JeevansSP commented on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1070530577
also for the code given by @mateusz-szczesny , adding a forward slash does not work :( kindly help
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] yungParrot commented on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
yungParrot commented on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1070596729
@JeevansSP hmmm I see that there is no issue tab on github so maybe you could [look on Jira?](https://issues.apache.org/jira/projects/RANGER/issues/) 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] JeevansSP edited a comment on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
JeevansSP edited a comment on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1070523172
```
from apache_ranger.model.ranger_service import *
from apache_ranger.client.ranger_client import *
from apache_ranger.model.ranger_policy import *
ranger_url = "http://*******.com:****/"
ranger_auth = ("*******", "**********")
ranger = RangerClient(ranger_url, ranger_auth)
policy = RangerPolicy()
policy.service = "starburst-enterprise"
policy.name = "test_policy"
policy.resources = { "database": RangerPolicyResource({ "values": ["hive"] }),
"schema": RangerPolicyResource({"values":["data_mesh"]}),
"table": RangerPolicyResource({ "values": ["uber_rides"] }),
"column": RangerPolicyResource({ "values": ["*"] }) }
allowItem1 = RangerPolicyItem()
allowItem1.groups = [ "Data Consumer" ]
allowItem1.accesses = [ RangerPolicyItemAccess({ "type": "select" }) ]
denyItem1 = RangerPolicyItem()
denyItem1.groups = [ "Data Consumer" ]
denyItem1.accesses = [ RangerPolicyItemAccess({ "type": "drop" }) ]
policy.policyItems = [ allowItem1 ]
policy.denyPolicyItems = [ denyItem1 ]
print('Creating policy: name=' + policy.name)
created_policy = ranger.create_policy(policy)
print(' created policy: name=' + created_policy.name + ', id=' + str(created_policy.id))
```
Hello, my Code works fine upto line 9 with or without the leading slash but anytime i try to create a policy, service etc , anything that requires a JSON file i get this error
`<Response [400]>
Traceback (most recent call last):
File "c:\Users\j.shimoga.prakash\Desktop\pls work - Copy\test.py", line 38, in <module>
created_policy = ranger.create_policy(policy)
File "C:\Users\j.shimoga.prakash\Desktop\pls work - Copy\venv\lib\site-packages\apache_ranger\client\ranger_client.py", line 124, in create_policy
resp = self.__call_api(RangerClient.CREATE_POLICY, params, policy)
File "C:\Users\j.shimoga.prakash\Desktop\pls work - Copy\venv\lib\site-packages\apache_ranger\client\ranger_client.py", line 333, in __call_api
raise RangerServiceException(api, response)
File "C:\Users\j.shimoga.prakash\Desktop\pls work - Copy\venv\lib\site-packages\apache_ranger\exceptions.py", line 39, in __init__
respJson = response.json()
File "C:\Users\j.shimoga.prakash\Desktop\pls work - Copy\venv\lib\site-packages\requests\models.py", line 910, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\j.shimoga.prakash\Desktop\pls work - Copy\venv\lib\site-packages\simplejson\__init__.py", line 525, in loads
return _default_decoder.decode(s)
File "C:\Users\j.shimoga.prakash\Desktop\pls work - Copy\venv\lib\site-packages\simplejson\decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "C:\Users\j.shimoga.prakash\Desktop\pls work - Copy\venv\lib\site-packages\simplejson\decoder.py", line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)`
Can someone please help me out?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] yungParrot commented on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
yungParrot commented on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1070568807
@JeevansSP if it's the latest version then I would just create a new issue...
good luck 🤠
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ranger] yungParrot commented on pull request #111: Parse URL using urlib.parse or urlparse instead of os package
Posted by GitBox <gi...@apache.org>.
yungParrot commented on pull request #111:
URL: https://github.com/apache/ranger/pull/111#issuecomment-1058966193
lets goooo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org