[jira] [Commented] (SPARK-9387) Support comparing UnsafeRow and normal InternalRow

["Reynold Xin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643654#comment-14643654 ] 

Reynold Xin commented on SPARK-9387:
------------------------------------

Here's the equals method we can add to InternalRow for comparing them:
{code}

  override def equals(o: Any): Boolean = {
    if (!o.isInstanceOf[InternalRow]) {
      return false
    }

    val other = o.asInstanceOf[InternalRow]
    if (other eq null) {
      return false
    }

    val len = numFields
    if (len != other.numFields) {
      return false
    }

    var i = 0
    while (i < len) {
      if (isNullAt(i)) {
        if (!other.isNullAt(i)) {
          return false
        }
      } else {
        get(i) match {
          case v: Boolean =>
            if (v != other.getBoolean(i)) return false

          case v: Byte =>
            if (v != other.getByte(i)) return false

          case v: Short =>
            if (v != other.getShort(i)) return false

          case v: Int =>
            if (v != other.getInt(i)) return false

          case v: Long =>
            if (v != other.getLong(i)) return false

          case v: Decimal =>
            if (v != other.getDecimal(i)) return false

          case v: Array[Byte] =>
            val o = other.get(i)
            if (!java.util.Arrays.equals(v, o.asInstanceOf[Array[Byte]])) {
              return false
            }

          case v: Float =>
            val o = other.getFloat(i)
            if (java.lang.Float.isNaN(v)) {
              if (!java.lang.Float.isNaN(o)) {
                return false
              }
            } else {
              if (v != o) {
                return false
              }
            }

          case v: Double =>
            val o = other.getDouble(i)
            if (java.lang.Double.isNaN(v)) {
              if (!java.lang.Double.isNaN(o)) {
                return false
              }
            } else {
              if (v != o) {
                return false
              }
            }

          case v: UTF8String =>
            if (v != other.getUTF8String(i)) return false

          case v: InternalRow =>
            if (v != other.getStruct(i, v.numFields)) return false

          case v =>
            if (v != other.genericGet(i)) {
              return false
            }
        }
      }
      i += 1
    }
    true
  }
{code}

Also need to update UnsafeRow's equals to call this one.

One problem is hash code ... the hash code generated from UnsafeRow is different from the one generated from other InternalRows.


> Support comparing UnsafeRow and normal InternalRow
> --------------------------------------------------
>
>                 Key: SPARK-9387
>                 URL: https://issues.apache.org/jira/browse/SPARK-9387
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>            Reporter: Reynold Xin
>            Assignee: Reynold Xin
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org