[jira] [Created] (SPARK-32952) Test failure on IBM Z: CoalesceShufflePartitionsSuite: - determining the number of reducers: complex query 1

["Michael Munday (Jira)" <ji...@apache.org>]

Michael Munday created SPARK-32952:
--------------------------------------

             Summary: Test failure on IBM Z: CoalesceShufflePartitionsSuite: - determining the number of reducers: complex query 1
                 Key: SPARK-32952
                 URL: https://issues.apache.org/jira/browse/SPARK-32952
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.0.1
         Environment: *{color:#FF0000}{color}*Linux on IBM Z (s390x).
            Reporter: Michael Munday


I'm seeing the test 'CoalesceShufflePartitionsSuite: - determining the number of reducers: complex query 1' fail on IBM Z with the wrong number of partitions (1 instead of 2). It's strange because none of the other tests fail.

I'd be grateful for any hints as to how the number of partitions is calculated. Could that calculation be affected by incorrect unsafe code? Is there a way to trace the calculation?

{noformat}
CoalesceShufflePartitionsSuite:
- determining the number of reducers: aggregate operator(minNumPostShufflePartitions: 5)
- determining the number of reducers: join operator(minNumPostShufflePartitions: 5)
- determining the number of reducers: complex query 1(minNumPostShufflePartitions: 5)
- determining the number of reducers: complex query 2(minNumPostShufflePartitions: 5)
- determining the number of reducers: plan already partitioned(minNumPostShufflePartitions: 5)
- determining the number of reducers: aggregate operator
- determining the number of reducers: join operator
- determining the number of reducers: complex query 1 *** FAILED ***
 1 did not equal 2 (CoalesceShufflePartitionsSuite.scala:221)
- determining the number of reducers: complex query 2
- determining the number of reducers: plan already partitioned
- SPARK-24705 adaptive query execution works correctly when exchange reuse enabled
- Do not reduce the number of shuffle partition for repartition
- Union two datasets with different pre-shuffle partition number{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org