You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by rdblue <gi...@git.apache.org> on 2018/05/04 22:44:33 UTC

[GitHub] spark pull request #21242: [SPARK-23657][SQL] Document and expose the intern...

GitHub user rdblue opened a pull request:

    https://github.com/apache/spark/pull/21242

    [SPARK-23657][SQL] Document and expose the internal data API

    ## What changes were proposed in this pull request?
    
    This makes the `InternalRow`, `ArrayData`, and `MapData` classes public and adds package documentation for the internal representation used by Spark SQL.
    
    The motivation for this change is to document the internal API because it has been leaked as a public API and used in the v2 DataSource classes.
    
    ## How was this patch tested?
    
    Existing tests. This is a refactor and adds documentation.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/rdblue/spark SPARK-23657-expose-internal-data-apis

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/21242.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #21242
    
----
commit f738127bc287632ede185daa3345b1f0de8b0ccc
Author: Ryan Blue <bl...@...>
Date:   2018-05-04T21:12:22Z

    Move InternalRow to org.apache.spark.sql.catalyst.data.

commit 747867c719029ddc86d44e7149e98e031a3468ea
Author: Ryan Blue <bl...@...>
Date:   2018-05-04T21:20:31Z

    Move ArrayData to org.apache.spark.sql.catalyst.data.

commit 7d108e9f1e603977b21773c664ba506b65ed0682
Author: Ryan Blue <bl...@...>
Date:   2018-05-04T21:34:25Z

    Move MapData to org.apache.spark.sql.catalyst.data.

commit 45a5c8d5e90df5e659bea107c73fc6e1a015ad7d
Author: Ryan Blue <bl...@...>
Date:   2018-05-04T21:36:05Z

    Move SpecializedGetters to org.apache.spark.sql.catalyst.data.

commit 465852a7f32c568b89d4ee988262b53a26710f97
Author: Ryan Blue <bl...@...>
Date:   2018-05-04T22:38:18Z

    Clean up the public API for InternalRow, ArrayData, and MapData.
    
    This adds SpecializedSetters, with helper functions that are common
    between InternalRow and ArrayData. This class also defines a default get
    method that was previously implemented by UnsafeArrayData and UnsafeRow.
    
    The package.scala for data has a high-level overview of the internal
    data API.

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90337/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90226/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90337 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90337/testReport)** for PR 21242 at commit [`220cb3d`](https://github.com/apache/spark/commit/220cb3d0e49df7382a5206be609b3c585b5aa2e1).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2932/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90231/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90229/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90383/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3015/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3044/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90229 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90229/testReport)** for PR 21242 at commit [`32eb1bc`](https://github.com/apache/spark/commit/32eb1bc0045a982e12daefa78f05c786a284fe4c).
     * This patch **fails Scala style tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90379 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90379/testReport)** for PR 21242 at commit [`5c8e390`](https://github.com/apache/spark/commit/5c8e390262bc005eb165e6854abe10a7b271200e).
     * This patch **fails to generate documentation**.
     * This patch **does not merge cleanly**.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by rdblue <gi...@git.apache.org>.
Github user rdblue commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    @rxin, after a few attempts at cleanly exposing these classes, I tend to agree that it isn't going to be worth it. But the problem is that we need an API for data sources to produce. What's worse is that they can already produce this data, but there's no documentation or guidance for doing so. What do you suggest as a fix?
    
    I'm fine using `InternalRow` as is in the v2 sources, and creating a markdown doc that describes the internal data formats that should be used.
    
    Even if we just add docs and don't expose these classes, the internals are going to need to change. `ArrayData`, for example, exposes `array` that is used in places to get the backing array. That prevents the objects from getting reused by readers and increases both allocations and gc costs. There is no real reason to use that method besides the fact that it happens to be available.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90374/testReport)** for PR 21242 at commit [`ad0765a`](https://github.com/apache/spark/commit/ad0765a0aae78bf892a97ed664e0ac1e9bb81f37).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90383 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90383/testReport)** for PR 21242 at commit [`64da297`](https://github.com/apache/spark/commit/64da2971a1f083926df35fe1366bcba84d97c7b7).
     * This patch **fails Spark unit tests**.
     * This patch **does not merge cleanly**.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90374/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90339/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90339 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90339/testReport)** for PR 21242 at commit [`b05a10b`](https://github.com/apache/spark/commit/b05a10b633d13b4b24da140be48619aacd7f5723).
     * This patch **fails to build**.
     * This patch **does not merge cleanly**.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3051/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90379 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90379/testReport)** for PR 21242 at commit [`5c8e390`](https://github.com/apache/spark/commit/5c8e390262bc005eb165e6854abe10a7b271200e).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90383 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90383/testReport)** for PR 21242 at commit [`64da297`](https://github.com/apache/spark/commit/64da2971a1f083926df35fe1366bcba84d97c7b7).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90231 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90231/testReport)** for PR 21242 at commit [`a4a68f1`](https://github.com/apache/spark/commit/a4a68f14f23bfd0e771912228e9cb7b405dbb7b2).
     * This patch **fails to build**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3048/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90337 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90337/testReport)** for PR 21242 at commit [`220cb3d`](https://github.com/apache/spark/commit/220cb3d0e49df7382a5206be609b3c585b5aa2e1).
     * This patch **fails to build**.
     * This patch **does not merge cleanly**.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90226 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90226/testReport)** for PR 21242 at commit [`465852a`](https://github.com/apache/spark/commit/465852a7f32c568b89d4ee988262b53a26710f97).
     * This patch **fails Scala style tests**.
     * This patch merges cleanly.
     * This patch adds the following public classes _(experimental)_:
      * `abstract class ArrayData extends SpecializedGetters with SpecializedSetters with Serializable `
      * `abstract class InternalRow extends SpecializedGetters with SpecializedSetters with Serializable `


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2934/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2936/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90339 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90339/testReport)** for PR 21242 at commit [`b05a10b`](https://github.com/apache/spark/commit/b05a10b633d13b4b24da140be48619aacd7f5723).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90229 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90229/testReport)** for PR 21242 at commit [`32eb1bc`](https://github.com/apache/spark/commit/32eb1bc0045a982e12daefa78f05c786a284fe4c).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90374 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90374/testReport)** for PR 21242 at commit [`ad0765a`](https://github.com/apache/spark/commit/ad0765a0aae78bf892a97ed664e0ac1e9bb81f37).
     * This patch **fails to generate documentation**.
     * This patch **does not merge cleanly**.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90226 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90226/testReport)** for PR 21242 at commit [`465852a`](https://github.com/apache/spark/commit/465852a7f32c568b89d4ee988262b53a26710f97).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90379/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    **[Test build #90231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90231/testReport)** for PR 21242 at commit [`a4a68f1`](https://github.com/apache/spark/commit/a4a68f14f23bfd0e771912228e9cb7b405dbb7b2).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3013/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the issue:

    https://github.com/apache/spark/pull/21242
  
    Thanks Ryan. I'm not a fan of just exposing internal classes like this. The APIs haven't really been designed or audited for the purpose of external consumption. If we want to expose the internal APIs, we should revisit their APIs to make sure they are good, and potentially narrow down the exposure.



---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org