You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/02/23 04:47:35 UTC

[GitHub] [flink] walterddr opened a new pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory

walterddr opened a new pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory
URL: https://github.com/apache/flink/pull/11189
 
 
   ## What is the purpose of the change
   
   YARNSessionFIFOSecuredITCase is currently no loading the right test factory and utilizes the NoOpSecurityContextFactory for instantiation. It needs to be fixed.
   
   ## Brief change log
   
     - Added TestHadoopSecurityContextFactory that works with the TestHadoopModuleFactory
     - Altered ITCase.
   
   ## Verifying this change
   
   This change is already covered by existing tests, such as *(please describe tests)*.
   
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): no
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no
     - The S3 file system connector: no
   
   ## Documentation
   
     - Does this pull request introduce a new feature? no
     - If yes, how is the feature documented? not applicable
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-590057578
 
 
   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "SUCCESS",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/150192639",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5476",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6033",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "PENDING",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/152199400",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 21899c1dde35fb0bd66121b382f4fcffff1ea040 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/150192639) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5476) 
   * 26f49aa572e93aad54402bcf66621721005af2d9 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/152199400) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6033) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r385664752
 
 

 ##########
 File path: flink-yarn-tests/src/test/java/org/apache/flink/yarn/util/TestHadoopSecurityContextFactory.java
 ##########
 @@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.yarn.util;
+
+import org.apache.flink.runtime.security.SecurityConfiguration;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContext;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContextFactory;
+import org.apache.flink.runtime.security.contexts.SecurityContext;
+import org.apache.flink.runtime.security.contexts.SecurityContextFactory;
+
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ * Test hadoop context module factory associated with {@link TestHadoopModuleFactory}.
+ */
+public class TestHadoopSecurityContextFactory implements SecurityContextFactory {
+
+	@Override
+	public boolean isCompatibleWith(SecurityConfiguration securityConfig) {
+		if (!securityConfig.getSecurityModuleFactories().contains(TestHadoopModuleFactory.class.getCanonicalName())) {
+			return false;
+		}
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				HadoopSecurityContextFactory.class.getClassLoader());
+			return true;
+		} catch (ClassNotFoundException e) {
+			return false;
+		}
+	}
+
+	@Override
+	public SecurityContext createContext(SecurityConfiguration securityConfig) {
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				TestHadoopSecurityContextFactory.class.getClassLoader());
+			UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
+			return new HadoopSecurityContext(loginUser);
+		} catch (Exception e) {
+			return null;
 
 Review comment:
   Shouldn't we fail with an exception here?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r386453071
 
 

 ##########
 File path: flink-yarn-tests/src/test/java/org/apache/flink/yarn/util/TestHadoopSecurityContextFactory.java
 ##########
 @@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.yarn.util;
+
+import org.apache.flink.runtime.security.SecurityConfiguration;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContext;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContextFactory;
+import org.apache.flink.runtime.security.contexts.SecurityContext;
+import org.apache.flink.runtime.security.contexts.SecurityContextFactory;
+
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ * Test hadoop context module factory associated with {@link TestHadoopModuleFactory}.
+ */
+public class TestHadoopSecurityContextFactory implements SecurityContextFactory {
+
+	@Override
+	public boolean isCompatibleWith(SecurityConfiguration securityConfig) {
+		if (!securityConfig.getSecurityModuleFactories().contains(TestHadoopModuleFactory.class.getCanonicalName())) {
+			return false;
+		}
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				HadoopSecurityContextFactory.class.getClassLoader());
+			return true;
+		} catch (ClassNotFoundException e) {
+			return false;
+		}
+	}
+
+	@Override
+	public SecurityContext createContext(SecurityConfiguration securityConfig) {
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				TestHadoopSecurityContextFactory.class.getClassLoader());
+			UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
+			return new HadoopSecurityContext(loginUser);
+		} catch (Exception e) {
+			return null;
 
 Review comment:
   I think this should be better expressed in the interface. E.g. one could add a `@Nullable` to the the return type stating that this method returns `null` if it cannot instantiate a `Context`. Using `Optional` would be even better. However, the problem with this kind of approach is that we are losing the context of `e`. `e` might contain helpful information why the context could not be instantiated. Hence, I would suggest to make the contract so that `createContext` always needs to return something. If not, then it should thrown an exception. How the service loader implementation reacts to it, should not be the problem of this class here.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r385665310
 
 

 ##########
 File path: flink-yarn-tests/src/test/java/org/apache/flink/yarn/util/TestHadoopSecurityContextFactory.java
 ##########
 @@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.yarn.util;
+
+import org.apache.flink.runtime.security.SecurityConfiguration;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContext;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContextFactory;
+import org.apache.flink.runtime.security.contexts.SecurityContext;
+import org.apache.flink.runtime.security.contexts.SecurityContextFactory;
+
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ * Test hadoop context module factory associated with {@link TestHadoopModuleFactory}.
+ */
+public class TestHadoopSecurityContextFactory implements SecurityContextFactory {
+
+	@Override
+	public boolean isCompatibleWith(SecurityConfiguration securityConfig) {
+		if (!securityConfig.getSecurityModuleFactories().contains(TestHadoopModuleFactory.class.getCanonicalName())) {
+			return false;
+		}
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				HadoopSecurityContextFactory.class.getClassLoader());
+			return true;
+		} catch (ClassNotFoundException e) {
+			return false;
+		}
+	}
+
+	@Override
+	public SecurityContext createContext(SecurityConfiguration securityConfig) {
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				TestHadoopSecurityContextFactory.class.getClassLoader());
 
 Review comment:
   Why is this lookup for the class different from the one in `isCompatibleWith`. I would suggest to remove this duplicate code.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r394867702
 
 

 ##########
 File path: flink-yarn-tests/src/test/java/org/apache/flink/yarn/util/TestHadoopSecurityContextFactory.java
 ##########
 @@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.yarn.util;
+
+import org.apache.flink.runtime.security.SecurityConfiguration;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContext;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContextFactory;
+import org.apache.flink.runtime.security.contexts.SecurityContext;
+import org.apache.flink.runtime.security.contexts.SecurityContextFactory;
+
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ * Test hadoop context module factory associated with {@link TestHadoopModuleFactory}.
+ */
+public class TestHadoopSecurityContextFactory implements SecurityContextFactory {
+
+	@Override
+	public boolean isCompatibleWith(SecurityConfiguration securityConfig) {
+		if (!securityConfig.getSecurityModuleFactories().contains(TestHadoopModuleFactory.class.getCanonicalName())) {
+			return false;
+		}
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				HadoopSecurityContextFactory.class.getClassLoader());
+			return true;
+		} catch (ClassNotFoundException e) {
+			return false;
+		}
+	}
+
+	@Override
+	public SecurityContext createContext(SecurityConfiguration securityConfig) {
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				TestHadoopSecurityContextFactory.class.getClassLoader());
+			UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
+			return new HadoopSecurityContext(loginUser);
 
 Review comment:
   Why can't we use the `HadoopSecurityContextFactory` instead of the this one? The factories look pretty much the same.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] tillrohrmann closed pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
tillrohrmann closed pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] walterddr commented on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory

Posted by GitBox <gi...@apache.org>.
walterddr commented on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-590425281
 
 
   @tillrohrmann @aljoscha could you kindly take a look? this is a follow up fix for #10836

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-590057578
 
 
   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "PENDING",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/150192639",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5476",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 21899c1dde35fb0bd66121b382f4fcffff1ea040 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/150192639) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5476) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-590057578
 
 
   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "SUCCESS",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/150192639",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5476",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 21899c1dde35fb0bd66121b382f4fcffff1ea040 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/150192639) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5476) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r394935236
 
 

 ##########
 File path: flink-yarn-tests/src/test/java/org/apache/flink/yarn/util/TestHadoopSecurityContextFactory.java
 ##########
 @@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.yarn.util;
+
+import org.apache.flink.runtime.security.SecurityConfiguration;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContext;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContextFactory;
+import org.apache.flink.runtime.security.contexts.SecurityContext;
+import org.apache.flink.runtime.security.contexts.SecurityContextFactory;
+
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ * Test hadoop context module factory associated with {@link TestHadoopModuleFactory}.
+ */
+public class TestHadoopSecurityContextFactory implements SecurityContextFactory {
+
+	@Override
+	public boolean isCompatibleWith(SecurityConfiguration securityConfig) {
+		if (!securityConfig.getSecurityModuleFactories().contains(TestHadoopModuleFactory.class.getCanonicalName())) {
+			return false;
+		}
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				HadoopSecurityContextFactory.class.getClassLoader());
+			return true;
+		} catch (ClassNotFoundException e) {
+			return false;
+		}
+	}
+
+	@Override
+	public SecurityContext createContext(SecurityConfiguration securityConfig) {
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				TestHadoopSecurityContextFactory.class.getClassLoader());
+			UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
+			return new HadoopSecurityContext(loginUser);
 
 Review comment:
   Ok, I understand now. The problem seems to be that `HadoopModuleFactory` loads the Hadoop configuration via `HadoopUtils.getHadoopConfiguration` and you need a way to inject the testing Yarn configuration. Instead of introducing new factories, I think the right approach would be to load the Yarn configuration outside of the `HadoopModuleFactory`. That way, it would be reusable. This would be a follow up ticket, though.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r394865266
 
 

 ##########
 File path: flink-runtime/src/main/java/org/apache/flink/runtime/security/contexts/HadoopSecurityContextFactory.java
 ##########
 @@ -53,17 +54,13 @@ public boolean isCompatibleWith(SecurityConfiguration securityConfig) {
 	}
 
 	@Override
-	public SecurityContext createContext(SecurityConfiguration securityConfig) {
+	public SecurityContext createContext(SecurityConfiguration securityConfig) throws SecurityContextInitializeException {
 		try {
-			Class.forName(
-				"org.apache.hadoop.security.UserGroupInformation",
-				false,
-				HadoopSecurityContextFactory.class.getClassLoader());
 			UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
 
 Review comment:
   I think the reflective lookup of the class `UserGroupInformation` is useless if we directly depend on `UserGroupInformation` here. If Hadoop is not on the classpath, then this whole class won't load due to this. I think this defies the purpose of the reflective lookup.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] walterddr commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r395071975
 
 

 ##########
 File path: flink-runtime/src/main/java/org/apache/flink/runtime/security/contexts/HadoopSecurityContextFactory.java
 ##########
 @@ -53,17 +54,13 @@ public boolean isCompatibleWith(SecurityConfiguration securityConfig) {
 	}
 
 	@Override
-	public SecurityContext createContext(SecurityConfiguration securityConfig) {
+	public SecurityContext createContext(SecurityConfiguration securityConfig) throws SecurityContextInitializeException {
 		try {
-			Class.forName(
-				"org.apache.hadoop.security.UserGroupInformation",
-				false,
-				HadoopSecurityContextFactory.class.getClassLoader());
 			UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
 
 Review comment:
   yes this is indeed a problem from way back. +1 to address it in separate JIRA

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot commented on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory

Posted by GitBox <gi...@apache.org>.
flinkbot commented on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-590026675
 
 
   Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress of the review.
   
   
   ## Automated Checks
   Last check on commit 21899c1dde35fb0bd66121b382f4fcffff1ea040 (Sun Feb 23 04:50:26 UTC 2020)
   
   **Warnings:**
    * No documentation files were touched! Remember to keep the Flink docs up to date!
   
   
   <sub>Mention the bot in a comment to re-run the automated checks.</sub>
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
    The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
    - `@flinkbot approve all` to approve all aspects
    - `@flinkbot approve-until architecture` to approve everything until `architecture`
    - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
    - `@flinkbot disapprove architecture` to remove an approval you gave earlier
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r394866598
 
 

 ##########
 File path: flink-yarn-tests/src/test/java/org/apache/flink/yarn/util/TestHadoopSecurityContextFactory.java
 ##########
 @@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.yarn.util;
+
+import org.apache.flink.runtime.security.SecurityConfiguration;
+import org.apache.flink.runtime.security.SecurityContextInitializeException;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContext;
+import org.apache.flink.runtime.security.contexts.SecurityContext;
+import org.apache.flink.runtime.security.contexts.SecurityContextFactory;
+
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ * Test hadoop context module factory associated with {@link TestHadoopModuleFactory}.
+ */
+public class TestHadoopSecurityContextFactory implements SecurityContextFactory {
+
+	@Override
+	public boolean isCompatibleWith(SecurityConfiguration securityConfig) {
+		if (!securityConfig.getSecurityModuleFactories().contains(TestHadoopModuleFactory.class.getCanonicalName())) {
+			return false;
+		}
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				TestHadoopSecurityContextFactory.class.getClassLoader());
 
 Review comment:
   I think this test is useless since we directly depend on `UserGroupInformation` in the `createContext` method. If this class is not on the classpath, then `TestHadoopSecurityContextFactory` should not load.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-590057578
 
 
   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "DELETED",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/150192639",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5476",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6033",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "DELETED",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/152199400",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6401",
       "triggerID" : "ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446",
       "status" : "SUCCESS",
       "url" : "https://travis-ci.com/github/flink-ci/flink/builds/154066510",
       "triggerID" : "ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446 Travis: [SUCCESS](https://travis-ci.com/github/flink-ci/flink/builds/154066510) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6401) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-590057578
 
 
   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "DELETED",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/150192639",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5476",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6033",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "SUCCESS",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/152199400",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 26f49aa572e93aad54402bcf66621721005af2d9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/152199400) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6033) 
   * ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] walterddr commented on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory

Posted by GitBox <gi...@apache.org>.
walterddr commented on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-592059882
 
 
   any additional comments? I would merge it if there's no objection. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-590057578
 
 
   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "DELETED",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/150192639",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5476",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6033",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "SUCCESS",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/152199400",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 26f49aa572e93aad54402bcf66621721005af2d9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/152199400) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6033) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] walterddr commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r385844823
 
 

 ##########
 File path: flink-yarn-tests/src/test/java/org/apache/flink/yarn/util/TestHadoopSecurityContextFactory.java
 ##########
 @@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.yarn.util;
+
+import org.apache.flink.runtime.security.SecurityConfiguration;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContext;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContextFactory;
+import org.apache.flink.runtime.security.contexts.SecurityContext;
+import org.apache.flink.runtime.security.contexts.SecurityContextFactory;
+
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ * Test hadoop context module factory associated with {@link TestHadoopModuleFactory}.
+ */
+public class TestHadoopSecurityContextFactory implements SecurityContextFactory {
+
+	@Override
+	public boolean isCompatibleWith(SecurityConfiguration securityConfig) {
+		if (!securityConfig.getSecurityModuleFactories().contains(TestHadoopModuleFactory.class.getCanonicalName())) {
+			return false;
+		}
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				HadoopSecurityContextFactory.class.getClassLoader());
+			return true;
+		} catch (ClassNotFoundException e) {
+			return false;
+		}
+	}
+
+	@Override
+	public SecurityContext createContext(SecurityConfiguration securityConfig) {
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				TestHadoopSecurityContextFactory.class.getClassLoader());
+			UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
+			return new HadoopSecurityContext(loginUser);
 
 Review comment:
   TestHadoopSecurityContext

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-590057578
 
 
   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "DELETED",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/150192639",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5476",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6033",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "SUCCESS",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/152199400",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 26f49aa572e93aad54402bcf66621721005af2d9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/152199400) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6033) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot commented on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory

Posted by GitBox <gi...@apache.org>.
flinkbot commented on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-590057578
 
 
   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 21899c1dde35fb0bd66121b382f4fcffff1ea040 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] walterddr commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r385844624
 
 

 ##########
 File path: flink-yarn-tests/src/test/java/org/apache/flink/yarn/util/TestHadoopSecurityContextFactory.java
 ##########
 @@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.yarn.util;
+
+import org.apache.flink.runtime.security.SecurityConfiguration;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContext;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContextFactory;
+import org.apache.flink.runtime.security.contexts.SecurityContext;
+import org.apache.flink.runtime.security.contexts.SecurityContextFactory;
+
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ * Test hadoop context module factory associated with {@link TestHadoopModuleFactory}.
+ */
+public class TestHadoopSecurityContextFactory implements SecurityContextFactory {
+
+	@Override
+	public boolean isCompatibleWith(SecurityConfiguration securityConfig) {
+		if (!securityConfig.getSecurityModuleFactories().contains(TestHadoopModuleFactory.class.getCanonicalName())) {
+			return false;
+		}
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				HadoopSecurityContextFactory.class.getClassLoader());
 
 Review comment:
   This is suppose to be `TestHadoopSecurityContextFactory`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] walterddr commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r395072254
 
 

 ##########
 File path: flink-yarn-tests/src/test/java/org/apache/flink/yarn/util/TestHadoopSecurityContextFactory.java
 ##########
 @@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.yarn.util;
+
+import org.apache.flink.runtime.security.SecurityConfiguration;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContext;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContextFactory;
+import org.apache.flink.runtime.security.contexts.SecurityContext;
+import org.apache.flink.runtime.security.contexts.SecurityContextFactory;
+
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ * Test hadoop context module factory associated with {@link TestHadoopModuleFactory}.
+ */
+public class TestHadoopSecurityContextFactory implements SecurityContextFactory {
+
+	@Override
+	public boolean isCompatibleWith(SecurityConfiguration securityConfig) {
+		if (!securityConfig.getSecurityModuleFactories().contains(TestHadoopModuleFactory.class.getCanonicalName())) {
+			return false;
+		}
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				HadoopSecurityContextFactory.class.getClassLoader());
+			return true;
+		} catch (ClassNotFoundException e) {
+			return false;
+		}
+	}
+
+	@Override
+	public SecurityContext createContext(SecurityConfiguration securityConfig) {
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				TestHadoopSecurityContextFactory.class.getClassLoader());
+			UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
+			return new HadoopSecurityContext(loginUser);
 
 Review comment:
   +1

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r394865508
 
 

 ##########
 File path: flink-runtime/src/main/java/org/apache/flink/runtime/security/contexts/HadoopSecurityContextFactory.java
 ##########
 @@ -53,17 +54,13 @@ public boolean isCompatibleWith(SecurityConfiguration securityConfig) {
 	}
 
 	@Override
-	public SecurityContext createContext(SecurityConfiguration securityConfig) {
+	public SecurityContext createContext(SecurityConfiguration securityConfig) throws SecurityContextInitializeException {
 		try {
-			Class.forName(
-				"org.apache.hadoop.security.UserGroupInformation",
-				false,
-				HadoopSecurityContextFactory.class.getClassLoader());
 			UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
 
 Review comment:
   This problem hasn't been introduced with this PR, though.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] walterddr commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r385845102
 
 

 ##########
 File path: flink-yarn-tests/src/test/java/org/apache/flink/yarn/util/TestHadoopSecurityContextFactory.java
 ##########
 @@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.yarn.util;
+
+import org.apache.flink.runtime.security.SecurityConfiguration;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContext;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContextFactory;
+import org.apache.flink.runtime.security.contexts.SecurityContext;
+import org.apache.flink.runtime.security.contexts.SecurityContextFactory;
+
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ * Test hadoop context module factory associated with {@link TestHadoopModuleFactory}.
+ */
+public class TestHadoopSecurityContextFactory implements SecurityContextFactory {
+
+	@Override
+	public boolean isCompatibleWith(SecurityConfiguration securityConfig) {
+		if (!securityConfig.getSecurityModuleFactories().contains(TestHadoopModuleFactory.class.getCanonicalName())) {
+			return false;
+		}
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				HadoopSecurityContextFactory.class.getClassLoader());
+			return true;
+		} catch (ClassNotFoundException e) {
+			return false;
+		}
+	}
+
+	@Override
+	public SecurityContext createContext(SecurityConfiguration securityConfig) {
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				TestHadoopSecurityContextFactory.class.getClassLoader());
+			UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
+			return new HadoopSecurityContext(loginUser);
+		} catch (Exception e) {
+			return null;
 
 Review comment:
   the new `SecurityUtils` with SPI will fallback to the next `SecurityContextFactory`. if no security context factory were found. exception will be thrown there. see #10836 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-590057578
 
 
   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "DELETED",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/150192639",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5476",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6033",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "DELETED",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/152199400",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6401",
       "triggerID" : "ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446",
       "status" : "PENDING",
       "url" : "https://travis-ci.com/github/flink-ci/flink/builds/154066510",
       "triggerID" : "ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/154066510) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6401) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-590057578
 
 
   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "SUCCESS",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/150192639",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5476",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 21899c1dde35fb0bd66121b382f4fcffff1ea040 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/150192639) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5476) 
   * 26f49aa572e93aad54402bcf66621721005af2d9 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-590057578
 
 
   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "DELETED",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/150192639",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5476",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6033",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "status" : "SUCCESS",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/152199400",
       "triggerID" : "26f49aa572e93aad54402bcf66621721005af2d9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6401",
       "triggerID" : "ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446",
       "status" : "PENDING",
       "url" : "https://travis-ci.com/github/flink-ci/flink/builds/154066510",
       "triggerID" : "ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 26f49aa572e93aad54402bcf66621721005af2d9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/152199400) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6033) 
   * ae5df2e71a3f0c6fd4dba82b7629d7a7fc74a446 Travis: [PENDING](https://travis-ci.com/github/flink-ci/flink/builds/154066510) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=6401) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] tillrohrmann commented on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-601112703
 
 
   Merging this PR after Travis gives green light.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] walterddr commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r390735933
 
 

 ##########
 File path: flink-runtime/src/main/java/org/apache/flink/runtime/security/contexts/SecurityContextFactory.java
 ##########
 @@ -47,5 +48,5 @@ default boolean isCompatibleWith(final SecurityConfiguration securityConfig) {
 	 * @param securityConfig security configuration used to create context.
 	 * @return the security context object.
 	 */
-	SecurityContext createContext(SecurityConfiguration securityConfig);
+	SecurityContext createContext(SecurityConfiguration securityConfig) throws SecurityContextInitializeException;
 
 Review comment:
   I altered the API to throw explicit exception instead here, to quote the suggestion: 
   > we are losing the context of `e`. `e` might contain helpful information why the context could not be instantiated
   
   Since the API explicitly requires an override of the `isCompatibleWith` method explicitly to check through, I think throwing an exception makes sense here 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loding correct context factory
URL: https://github.com/apache/flink/pull/11189#issuecomment-590057578
 
 
   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "status" : "PENDING",
       "url" : "https://travis-ci.com/flink-ci/flink/builds/150192639",
       "triggerID" : "21899c1dde35fb0bd66121b382f4fcffff1ea040",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 21899c1dde35fb0bd66121b382f4fcffff1ea040 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/150192639) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] walterddr commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory

Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #11189: [FLINK-16236][yarn][hotfix] fix Yarn secure test not loading correct context factory
URL: https://github.com/apache/flink/pull/11189#discussion_r389076773
 
 

 ##########
 File path: flink-yarn-tests/src/test/java/org/apache/flink/yarn/util/TestHadoopSecurityContextFactory.java
 ##########
 @@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.yarn.util;
+
+import org.apache.flink.runtime.security.SecurityConfiguration;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContext;
+import org.apache.flink.runtime.security.contexts.HadoopSecurityContextFactory;
+import org.apache.flink.runtime.security.contexts.SecurityContext;
+import org.apache.flink.runtime.security.contexts.SecurityContextFactory;
+
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ * Test hadoop context module factory associated with {@link TestHadoopModuleFactory}.
+ */
+public class TestHadoopSecurityContextFactory implements SecurityContextFactory {
+
+	@Override
+	public boolean isCompatibleWith(SecurityConfiguration securityConfig) {
+		if (!securityConfig.getSecurityModuleFactories().contains(TestHadoopModuleFactory.class.getCanonicalName())) {
+			return false;
+		}
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				HadoopSecurityContextFactory.class.getClassLoader());
+			return true;
+		} catch (ClassNotFoundException e) {
+			return false;
+		}
+	}
+
+	@Override
+	public SecurityContext createContext(SecurityConfiguration securityConfig) {
+		try {
+			Class.forName(
+				"org.apache.hadoop.security.UserGroupInformation",
+				false,
+				TestHadoopSecurityContextFactory.class.getClassLoader());
+			UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
+			return new HadoopSecurityContext(loginUser);
 
 Review comment:
   this is no longer needed. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services