[GitHub] HyukjinKwon commented on a change in pull request #23337: [SPARK-26019][PYSPARK] Allow insecure py4j gateways

[GitBox <gi...@apache.org>]

HyukjinKwon commented on a change in pull request #23337: [SPARK-26019][PYSPARK] Allow insecure py4j gateways
URL: https://github.com/apache/spark/pull/23337#discussion_r242381329
 
 

 ##########
 File path: python/pyspark/tests.py
 ##########
 @@ -2381,6 +2382,36 @@ def test_startTime(self):
         with SparkContext() as sc:
             self.assertGreater(sc.startTime, 0)
 
+    def test_forbid_insecure_gateway(self):
+        # By default, we fail immediately if you try to create a SparkContext
+        # with an insecure gateway
+        gateway = _launch_gateway(insecure=True)
+        with self.assertRaises(Exception) as context:
+            SparkContext(gateway=gateway)
+        self.assertIn("insecure py4j gateway", context.exception.message)
+        self.assertIn("spark.python.allowInsecurePy4j", context.exception.message)
+        self.assertIn("removed in Spark 3.0", context.exception.message)
+
+    def test_allow_insecure_gateway_with_conf(self):
+        with SparkContext._lock:
+            SparkContext._gateway = None
+            SparkContext._jvm = None
+        gateway = _launch_gateway(insecure=True)
+        conf = SparkConf()
+        conf.set("spark.python.allowInsecurePy4j", "true")
+        print("entering allow insecure test")
+        with SparkContext(conf=conf, gateway=gateway) as sc:
+            print("sc created, about to create accum")
+            a = sc.accumulator(1)
+            rdd = sc.parallelize([1, 2, 3])
+
+            def f(x):
+                a.add(x)
+
+            rdd.foreach(f)
 
 Review comment:
   `lambda x: a.add(x)`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org