[jira] [Created] (SPARK-14291) Stuck job when using bitwise and

["Jarno Seppanen (JIRA)" <ji...@apache.org>]

Jarno Seppanen created SPARK-14291:
--------------------------------------

             Summary: Stuck job when using bitwise and
                 Key: SPARK-14291
                 URL: https://issues.apache.org/jira/browse/SPARK-14291
             Project: Spark
          Issue Type: Bug
          Components: PySpark
    Affects Versions: 1.6.0
         Environment: Amazon EMR release 4.4.0
            Reporter: Jarno Seppanen


DataFrame operations hang when using bitwise and operation.

Minimal PySpark example:
{noformat}
>>> foo = sqlContext.createDataFrame([(i,) for i in range(10)], ['x'])
>>> foo.filter(foo.x.bitwiseAND(1) == 1).count()
[Stage 217:====================================================>(959 + 1) / 960]
{noformat}

The job never finishes, it's stuck at the last step.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org