[jira] [Created] (SPARK-16357) For Spark shuffle RPC encryption using 3DES on Sparksql, throughput test of 2 stream lasts 2.68X time than power test.

["zeweichen (JIRA)" <ji...@apache.org>]

zeweichen created SPARK-16357:
---------------------------------

             Summary: For Spark shuffle RPC encryption using 3DES on Sparksql, throughput test of 2 stream lasts 2.68X time than power test.
                 Key: SPARK-16357
                 URL: https://issues.apache.org/jira/browse/SPARK-16357
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 1.6.1
         Environment: Apache Spark 1.6.1 with local patch for supporting Q7 subquery feature 
Apache Spark 1.6.1 with Spark shuffle RPC encryption
BigBench 1.0.1
            Reporter: zeweichen
            Priority: Minor
             Fix For: 2.1.0


During our performance test of Spark shuffle RPC encryption using 3DES on Sparksql, we found that throughput test of 2 stream lasts 2.68X time than power test, which is much larger than it should be. Reason is that q30 cost 0.5 hour in stream1,  but q30 last 2.5 hours in stream 0 which is much longer than it should be.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org