You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@knox.apache.org by "Kevin Risden (JIRA)" <ji...@apache.org> on 2019/01/07 18:13:00 UTC
[jira] [Comment Edited] (KNOX-1727) Values should not be forced in
query parameters when proxying through Knox
[ https://issues.apache.org/jira/browse/KNOX-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16736124#comment-16736124 ]
Kevin Risden edited comment on KNOX-1727 at 1/7/19 6:12 PM:
------------------------------------------------------------
Well this breaks the integration tests with WebHDFS and I'm not 100% sure why off the top of my head right now. I need to dig in further. It might only be with SecureKnoxShellTest which is pretty new. Not sure why it fails with this change.
{code:java}
2019-01-07 13:12:03,618 INFO gateway - Initializing AclsAuthz Provider for: WEBHDFS
2019-01-07 13:12:03,618 INFO gateway - Access Granted: true
2019-01-07 13:12:03,710 WARN gateway - Possible identity spoofing attempt - impersonation parameter removed: user.name
2019-01-07 13:12:03,765 INFO webhdfs - 127.0.0.1 PUT /webhdfs/v1/user/guest/example/README?op=CREATE&namenoderpcaddress=127.0.0.1%3A60701createflag&createparent=true&overwrite=false&permission=755&user.name=guest 500
org.apache.knox.gateway.shell.KnoxShellException: org.apache.knox.gateway.shell.ErrorResponse: http://localhost:60722/gateway/secure/webhdfs/data/v1/webhdfs/v1/user/guest/example/README?_=AAAACAAAABAAAACwy6RpcI-MPdTkT0EO82gb1W5VLqbkwdA3ypLGlvDq0VU10IX7FGWD363CCZr08j2WFNGWuAPKmz-jU8MU7-4T9XAKxKKFD8andIiBbqDtOPBYtWyHDXjc5l4xHm81BOg4UVoK-MuA6N4iNV3rCKXXZiY6Pty9z13Rg2NJMorKzdl5bidRN-Ni0fvr-wkDvcoJotnZewTCVbL6To2-UEWJ6_D9AtmXkgB5SWm1TmSOtT9JtbH8QemArP2KXr7Kpub8fUKNm3eks50: HTTP/1.1 400 Bad Request
at org.apache.knox.gateway.shell.AbstractRequest.now(AbstractRequest.java:81)
at org.apache.knox.gateway.shell.AbstractRequest$now.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:115)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:119)
at org.apache.knox.gateway.ShellTest.SecureWebHdfsPutGet.run(SecureWebHdfsPutGet.groovy:32)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:441)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:497)
at org.apache.knox.gateway.SecureKnoxShellTest.webhdfsPutGet(SecureKnoxShellTest.java:312)
at org.apache.knox.gateway.SecureKnoxShellTest.testCachedTicket(SecureKnoxShellTest.java:285)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: org.apache.knox.gateway.shell.ErrorResponse: http://localhost:60722/gateway/secure/webhdfs/data/v1/webhdfs/v1/user/guest/example/README?_=AAAACAAAABAAAACwy6RpcI-MPdTkT0EO82gb1W5VLqbkwdA3ypLGlvDq0VU10IX7FGWD363CCZr08j2WFNGWuAPKmz-jU8MU7-4T9XAKxKKFD8andIiBbqDtOPBYtWyHDXjc5l4xHm81BOg4UVoK-MuA6N4iNV3rCKXXZiY6Pty9z13Rg2NJMorKzdl5bidRN-Ni0fvr-wkDvcoJotnZewTCVbL6To2-UEWJ6_D9AtmXkgB5SWm1TmSOtT9JtbH8QemArP2KXr7Kpub8fUKNm3eks50: HTTP/1.1 400 Bad Request
at org.apache.knox.gateway.shell.KnoxSession.lambda$executeNow$0(KnoxSession.java:457)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at org.apache.knox.gateway.shell.KnoxSession.executeNow(KnoxSession.java:448)
at org.apache.knox.gateway.shell.AbstractRequest.execute(AbstractRequest.java:50)
at org.apache.knox.gateway.shell.hdfs.Put$Request.access$1600(Put.java:39)
at org.apache.knox.gateway.shell.hdfs.Put$Request$1.call(Put.java:125)
at org.apache.knox.gateway.shell.hdfs.Put$Request$1.call(Put.java:96)
at org.apache.knox.gateway.shell.AbstractRequest.now(AbstractRequest.java:79)
... 33 more
{code}
was (Author: risdenk):
Well this breaks the integration tests with WebHDFS and I'm not 100% sure why off the top of my head right now. I need to dig in further. It might only be with SecureKnoxShellTest which is pretty new. Not sure why it failes with these changes.
{code:java}
2019-01-07 13:12:03,618 INFO gateway - Initializing AclsAuthz Provider for: WEBHDFS
2019-01-07 13:12:03,618 INFO gateway - Access Granted: true
2019-01-07 13:12:03,710 WARN gateway - Possible identity spoofing attempt - impersonation parameter removed: user.name
2019-01-07 13:12:03,765 INFO webhdfs - 127.0.0.1 PUT /webhdfs/v1/user/guest/example/README?op=CREATE&namenoderpcaddress=127.0.0.1%3A60701createflag&createparent=true&overwrite=false&permission=755&user.name=guest 500
org.apache.knox.gateway.shell.KnoxShellException: org.apache.knox.gateway.shell.ErrorResponse: http://localhost:60722/gateway/secure/webhdfs/data/v1/webhdfs/v1/user/guest/example/README?_=AAAACAAAABAAAACwy6RpcI-MPdTkT0EO82gb1W5VLqbkwdA3ypLGlvDq0VU10IX7FGWD363CCZr08j2WFNGWuAPKmz-jU8MU7-4T9XAKxKKFD8andIiBbqDtOPBYtWyHDXjc5l4xHm81BOg4UVoK-MuA6N4iNV3rCKXXZiY6Pty9z13Rg2NJMorKzdl5bidRN-Ni0fvr-wkDvcoJotnZewTCVbL6To2-UEWJ6_D9AtmXkgB5SWm1TmSOtT9JtbH8QemArP2KXr7Kpub8fUKNm3eks50: HTTP/1.1 400 Bad Request
at org.apache.knox.gateway.shell.AbstractRequest.now(AbstractRequest.java:81)
at org.apache.knox.gateway.shell.AbstractRequest$now.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:115)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:119)
at org.apache.knox.gateway.ShellTest.SecureWebHdfsPutGet.run(SecureWebHdfsPutGet.groovy:32)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:441)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:497)
at org.apache.knox.gateway.SecureKnoxShellTest.webhdfsPutGet(SecureKnoxShellTest.java:312)
at org.apache.knox.gateway.SecureKnoxShellTest.testCachedTicket(SecureKnoxShellTest.java:285)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: org.apache.knox.gateway.shell.ErrorResponse: http://localhost:60722/gateway/secure/webhdfs/data/v1/webhdfs/v1/user/guest/example/README?_=AAAACAAAABAAAACwy6RpcI-MPdTkT0EO82gb1W5VLqbkwdA3ypLGlvDq0VU10IX7FGWD363CCZr08j2WFNGWuAPKmz-jU8MU7-4T9XAKxKKFD8andIiBbqDtOPBYtWyHDXjc5l4xHm81BOg4UVoK-MuA6N4iNV3rCKXXZiY6Pty9z13Rg2NJMorKzdl5bidRN-Ni0fvr-wkDvcoJotnZewTCVbL6To2-UEWJ6_D9AtmXkgB5SWm1TmSOtT9JtbH8QemArP2KXr7Kpub8fUKNm3eks50: HTTP/1.1 400 Bad Request
at org.apache.knox.gateway.shell.KnoxSession.lambda$executeNow$0(KnoxSession.java:457)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at org.apache.knox.gateway.shell.KnoxSession.executeNow(KnoxSession.java:448)
at org.apache.knox.gateway.shell.AbstractRequest.execute(AbstractRequest.java:50)
at org.apache.knox.gateway.shell.hdfs.Put$Request.access$1600(Put.java:39)
at org.apache.knox.gateway.shell.hdfs.Put$Request$1.call(Put.java:125)
at org.apache.knox.gateway.shell.hdfs.Put$Request$1.call(Put.java:96)
at org.apache.knox.gateway.shell.AbstractRequest.now(AbstractRequest.java:79)
... 33 more
{code}
> Values should not be forced in query parameters when proxying through Knox
> --------------------------------------------------------------------------
>
> Key: KNOX-1727
> URL: https://issues.apache.org/jira/browse/KNOX-1727
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
> Reporter: Robert Levas
> Assignee: Kevin Risden
> Priority: Major
> Fix For: 1.3.0
>
> Attachments: KNOX-1727.patch
>
>
> Values should not be forced in query parameters when proxying through Knox.
> For example if the following query is received by Knox to be forwarded:
> {noformat}
> GET /some_topology/some_service/some_path?param1=value1¶m2¶m3=value3
> {noformat}
> Knox should not force all query parameters to have (empty) values, like
> {noformat}
> GET /some_topology/some_service/some_path?param1=value1¶m2=¶m3=value3
> {noformat}
> Notice {{param2}} vs {{param2=}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)