You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by samelamin <gi...@git.apache.org> on 2017/03/29 19:27:47 UTC

[GitHub] spark pull request #17472: [SPARK-19999]: Fix for flakey tests due to java.n...

GitHub user samelamin opened a pull request:

    https://github.com/apache/spark/pull/17472

    [SPARK-19999]: Fix for flakey tests due to java.nio.Bits.unaligned() \u2026

    \u2026doesn't return true on ppc
    
    ## What changes were proposed in this pull request?
    check architecture
    
    ## How was this patch tested?
    
    unit test
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/samelamin/spark SPARK-19999

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/17472.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #17472
    
----
commit 7044e8d728078da37d22069955287cfae2593021
Author: samelamin <sa...@discovery.com>
Date:   2017-03-29T15:59:04Z

    [SPARK-19999]: Fix for flakey tests due to java.nio.Bits.unaligned() doesn't return true on ppc

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Fix for flakey tests due to java.n...

Posted by samelamin <gi...@git.apache.org>.
Github user samelamin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108783754
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -46,16 +46,22 @@
       private static final boolean unaligned;
       static {
         boolean _unaligned;
    +    String arch = System.getProperty("os.arch", "");
         // use reflection to access unaligned field
         try {
           Class<?> bitsClass =
             Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
           Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
           unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    +
    +      //Since java.nio.Bits.unaligned() doesn't return true on ppc (See JDK-8165231)
    +      if(arch.matches("^(ppc64le|ppc64)$")){
    --- End diff --
    
    moved up top.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Workaround JDK-8165231 to identify PPC64 ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75386/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by samelamin <gi...@git.apache.org>.
Github user samelamin commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    ok @gatorsmile i set default indentation to 2, how does it look now? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    I do not think we have such a default steup. You can manually change the indentation of InteiiJ 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Fix for flakey tests due to java.n...

Posted by samelamin <gi...@git.apache.org>.
Github user samelamin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108879329
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -46,18 +46,22 @@
       private static final boolean unaligned;
       static {
         boolean _unaligned;
    -    // use reflection to access unaligned field
    -    try {
    -      Class<?> bitsClass =
    -        Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
    -      Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
    -      unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    -    } catch (Throwable t) {
    -      // We at least know x86 and x64 support unaligned access.
    -      String arch = System.getProperty("os.arch", "");
    -      //noinspection DynamicRegexReplaceableByCompiledPattern
    -      _unaligned = arch.matches("^(i[3-6]86|x86(_64)?|x64|amd64|aarch64)$");
    +    String arch = System.getProperty("os.arch", "");
    +    if (arch.matches("^(ppc64le | ppc64)$")) {
    --- End diff --
    
    thats fine but itll end up being arch.startsWith("ppc64") || arch.startsWith("ppc64le")
    
    I thought this was less code, but startsWith is more readable
    
    Are you happy with that approach? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Fix for flakey tests due to java.n...

Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108773559
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -46,16 +46,22 @@
       private static final boolean unaligned;
       static {
         boolean _unaligned;
    +    String arch = System.getProperty("os.arch", "");
         // use reflection to access unaligned field
         try {
           Class<?> bitsClass =
             Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
           Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
           unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    +
    +      //Since java.nio.Bits.unaligned() doesn't return true on ppc (See JDK-8165231)
    +      if(arch.matches("^(ppc64le|ppc64)$")){
    --- End diff --
    
    Would it be possible to add appropriate space between words?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Fix for flakey tests due to java.n...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108878462
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -46,18 +46,22 @@
       private static final boolean unaligned;
       static {
         boolean _unaligned;
    -    // use reflection to access unaligned field
    -    try {
    -      Class<?> bitsClass =
    -        Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
    -      Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
    -      unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    -    } catch (Throwable t) {
    -      // We at least know x86 and x64 support unaligned access.
    -      String arch = System.getProperty("os.arch", "");
    -      //noinspection DynamicRegexReplaceableByCompiledPattern
    -      _unaligned = arch.matches("^(i[3-6]86|x86(_64)?|x64|amd64|aarch64)$");
    +    String arch = System.getProperty("os.arch", "");
    +    if (arch.matches("^(ppc64le | ppc64)$")) {
    --- End diff --
    
    Still, why not `arch.startsWith("ppc64")`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    **[Test build #75380 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75380/testReport)** for PR 17472 at commit [`77777e9`](https://github.com/apache/spark/commit/77777e96a627d72ff7a74043a14e7baa9d70bb04).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Fix for flakey tests due to java.n...

Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108773661
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -46,16 +46,22 @@
       private static final boolean unaligned;
       static {
         boolean _unaligned;
    +    String arch = System.getProperty("os.arch", "");
         // use reflection to access unaligned field
         try {
           Class<?> bitsClass =
             Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
           Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
           unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    +
    +      //Since java.nio.Bits.unaligned() doesn't return true on ppc (See JDK-8165231)
    +      if(arch.matches("^(ppc64le|ppc64)$")){
    +       _unaligned=true;
    --- End diff --
    
    Spark uses two spaces for an indent.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Fix for flakey tests due to java.n...

Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108787682
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -45,21 +45,27 @@
     
       private static final boolean unaligned;
       static {
    -    boolean _unaligned;
    -    // use reflection to access unaligned field
    -    try {
    -      Class<?> bitsClass =
    -        Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
    -      Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
    -      unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    -    } catch (Throwable t) {
    -      // We at least know x86 and x64 support unaligned access.
    +      boolean _unaligned;
    --- End diff --
    
    Indent issues.
    
    Please follow the style guide: https://github.com/databricks/scala-style-guide#indent


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Fix for flakey tests due to java.n...

Posted by samelamin <gi...@git.apache.org>.
Github user samelamin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108789142
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -45,21 +45,27 @@
     
       private static final boolean unaligned;
       static {
    -    boolean _unaligned;
    -    // use reflection to access unaligned field
    -    try {
    -      Class<?> bitsClass =
    -        Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
    -      Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
    -      unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    -    } catch (Throwable t) {
    -      // We at least know x86 and x64 support unaligned access.
    +      boolean _unaligned;
    --- End diff --
    
    I think its to do with my local setup, i dont see the issues in Intellij, sorry! :( 
    
    ill revert my changes and try again


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Workaround JDK-8165231 to identify PPC64 ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    **[Test build #75386 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75386/testReport)** for PR 17472 at commit [`632161b`](https://github.com/apache/spark/commit/632161b299ac37a598083b1c2995e9becbddc33c).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    **[Test build #75372 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75372/testReport)** for PR 17472 at commit [`bf7cc24`](https://github.com/apache/spark/commit/bf7cc24f213a2cf043a579846859647da850f1f8).
     * This patch **fails to build**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Fix for flakey tests due to java.n...

Posted by samelamin <gi...@git.apache.org>.
Github user samelamin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108849902
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -46,18 +46,22 @@
       private static final boolean unaligned;
       static {
         boolean _unaligned;
    -    // use reflection to access unaligned field
    -    try {
    -      Class<?> bitsClass =
    -        Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
    -      Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
    -      unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    -    } catch (Throwable t) {
    -      // We at least know x86 and x64 support unaligned access.
    -      String arch = System.getProperty("os.arch", "");
    -      //noinspection DynamicRegexReplaceableByCompiledPattern
    -      _unaligned = arch.matches("^(i[3-6]86|x86(_64)?|x64|amd64|aarch64)$");
    +    if (arch.matches("^(ppc64le | ppc64)$")) {
    +      // Since java.nio.Bits.unaligned() doesn't return true on ppc (See JDK-8165231), but ppc64 and ppc64le support it
    +      _unaligned = true;
    +    } else {
    +      try {
    +        Class<?> bitsClass =
    +          Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
    +        Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
    +        unalignedMethod.setAccessible(true);
    +        _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    +      } catch (Throwable t) {
    +        // We at least know x86 and x64 support unaligned access.
    +        String arch = System.getProperty("os.arch", "");
    --- End diff --
    
    your absolutely right, I got caught off while I was trying to fix the indentation issues. good catch! 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by samelamin <gi...@git.apache.org>.
Github user samelamin commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    jenkins test this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Workaround JDK-8165231 to identify PPC64 ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    **[Test build #75386 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75386/testReport)** for PR 17472 at commit [`632161b`](https://github.com/apache/spark/commit/632161b299ac37a598083b1c2995e9becbddc33c).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by samelamin <gi...@git.apache.org>.
Github user samelamin commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    Can someone guide me why the tests failed? is it another flakey test or? because the below message doesnt make sense to me :( 
    
    > Traceback (most recent call last):
    >   File "./dev/run-tests-jenkins.py", line 226, in <module>
    >     main()
    >   File "./dev/run-tests-jenkins.py", line 213, in main
    >     test_result_code, test_result_note = run_tests(tests_timeout)
    >   File "./dev/run-tests-jenkins.py", line 140, in run_tests
    >     test_result_note = ' * This patch **fails %s**.' % failure_note_by_errcode[test_result_code]
    > KeyError: -9


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    **[Test build #75372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75372/testReport)** for PR 17472 at commit [`bf7cc24`](https://github.com/apache/spark/commit/bf7cc24f213a2cf043a579846859647da850f1f8).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Accommodate a new architecture tha...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108884076
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -46,18 +46,22 @@
       private static final boolean unaligned;
       static {
         boolean _unaligned;
    -    // use reflection to access unaligned field
    -    try {
    -      Class<?> bitsClass =
    -        Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
    -      Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
    -      unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    -    } catch (Throwable t) {
    -      // We at least know x86 and x64 support unaligned access.
    -      String arch = System.getProperty("os.arch", "");
    -      //noinspection DynamicRegexReplaceableByCompiledPattern
    -      _unaligned = arch.matches("^(i[3-6]86|x86(_64)?|x64|amd64|aarch64)$");
    +    String arch = System.getProperty("os.arch", "");
    +    if (arch.matches("^(ppc64le | ppc64)$")) {
    --- End diff --
    
    Sorry, I'm writing one thing and thinking another. I mean alignment, not endianness. Which architectures do you know allow unaligned access? I'd presume all PPC does, and I assume the JDK issue means "PPC64 (big-endian) but also PPC64 little-endian".  OK, to be conservative, maybe just check the strings "ppc64" and "ppc64le" as you intended.
    
    However your regex doesn't work. You have extra whitespace. Just instead check `arch.equals(...) || arch.equals(...)`
    
    Also the PR title shoudl match the JIRA title. I was commenting that it differed here in referring to a flaky test, but it isn't. To be very clear, I propose you rename both to perhaps: "Workaround JDK-8165231 to identify PPC64 architectures as supporting unaligned access"


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Accommodate a new architecture tha...

Posted by samelamin <gi...@git.apache.org>.
Github user samelamin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108881923
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -46,18 +46,22 @@
       private static final boolean unaligned;
       static {
         boolean _unaligned;
    -    // use reflection to access unaligned field
    -    try {
    -      Class<?> bitsClass =
    -        Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
    -      Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
    -      unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    -    } catch (Throwable t) {
    -      // We at least know x86 and x64 support unaligned access.
    -      String arch = System.getProperty("os.arch", "");
    -      //noinspection DynamicRegexReplaceableByCompiledPattern
    -      _unaligned = arch.matches("^(i[3-6]86|x86(_64)?|x64|amd64|aarch64)$");
    +    String arch = System.getProperty("os.arch", "");
    +    if (arch.matches("^(ppc64le | ppc64)$")) {
    --- End diff --
    
    yeah sorry complete fail, didnt see that lol
    
    With regards to a big-endian PPC64 arch , I dont think so but I am not 100% sure. that said as it stands it isn't supported and the test fails. So happy to change the PR or even close it if you do not see any value it adds 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    Clear code comments can help code reading. : )


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Workaround JDK-8165231 to identify PPC64 ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Workaround JDK-8165231 to identify PPC64 ...

Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Workaround JDK-8165231 to identify PPC64 ...

Posted by samelamin <gi...@git.apache.org>.
Github user samelamin commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    @gatorsmile @srowen how does it look now? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75380/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75372/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by samelamin <gi...@git.apache.org>.
Github user samelamin commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    @gatorsmile I think my Intellij has some annoying auto indenting changes. Does spark have some project wide settings I can auto import? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by samelamin <gi...@git.apache.org>.
Github user samelamin commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    @gatorsmile moved the comment per your suggestion, but to be honest if the comment is unclear surly the first thing someone will do is check that JIRA? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Workaround JDK-8165231 to identify...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/17472


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Fix for flakey tests due to java.n...

Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108837689
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -46,18 +46,22 @@
       private static final boolean unaligned;
       static {
         boolean _unaligned;
    -    // use reflection to access unaligned field
    -    try {
    -      Class<?> bitsClass =
    -        Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
    -      Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
    -      unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    -    } catch (Throwable t) {
    -      // We at least know x86 and x64 support unaligned access.
    -      String arch = System.getProperty("os.arch", "");
    -      //noinspection DynamicRegexReplaceableByCompiledPattern
    -      _unaligned = arch.matches("^(i[3-6]86|x86(_64)?|x64|amd64|aarch64)$");
    +    if (arch.matches("^(ppc64le | ppc64)$")) {
    +      // Since java.nio.Bits.unaligned() doesn't return true on ppc (See JDK-8165231), but ppc64 and ppc64le support it
    +      _unaligned = true;
    +    } else {
    +      try {
    +        Class<?> bitsClass =
    +          Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
    +        Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
    +        unalignedMethod.setAccessible(true);
    +        _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    +      } catch (Throwable t) {
    +        // We at least know x86 and x64 support unaligned access.
    +        String arch = System.getProperty("os.arch", "");
    --- End diff --
    
    Should we define `arch` before the `if` statement, now?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Fix for flakey tests due to java.n...

Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108802741
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -46,18 +46,22 @@
       private static final boolean unaligned;
       static {
         boolean _unaligned;
    -    // use reflection to access unaligned field
    -    try {
    -      Class<?> bitsClass =
    -        Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
    -      Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
    -      unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    -    } catch (Throwable t) {
    -      // We at least know x86 and x64 support unaligned access.
    -      String arch = System.getProperty("os.arch", "");
    -      //noinspection DynamicRegexReplaceableByCompiledPattern
    -      _unaligned = arch.matches("^(i[3-6]86|x86(_64)?|x64|amd64|aarch64)$");
    +    if (arch.matches("^(ppc64le | ppc64)$")) {
    +      // Since java.nio.Bits.unaligned() doesn't return true on ppc (See JDK-8165231), but ppc64 and ppc64le support it
    --- End diff --
    
    It is longer than 101 characters. It will fail the style test.
    
    You can check it in your local environment using the command:
    > dev/lint-scala



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Accommodate a new architecture tha...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108880865
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -46,18 +46,22 @@
       private static final boolean unaligned;
       static {
         boolean _unaligned;
    -    // use reflection to access unaligned field
    -    try {
    -      Class<?> bitsClass =
    -        Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
    -      Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
    -      unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    -    } catch (Throwable t) {
    -      // We at least know x86 and x64 support unaligned access.
    -      String arch = System.getProperty("os.arch", "");
    -      //noinspection DynamicRegexReplaceableByCompiledPattern
    -      _unaligned = arch.matches("^(i[3-6]86|x86(_64)?|x64|amd64|aarch64)$");
    +    String arch = System.getProperty("os.arch", "");
    +    if (arch.matches("^(ppc64le | ppc64)$")) {
    --- End diff --
    
    Both of those start with ppc64, what do you mean?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Fix for flakey tests due to java.n...

Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108797960
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -46,18 +46,22 @@
       private static final boolean unaligned;
       static {
         boolean _unaligned;
    -    // use reflection to access unaligned field
    -    try {
    -      Class<?> bitsClass =
    -        Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
    -      Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
    -      unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    -    } catch (Throwable t) {
    -      // We at least know x86 and x64 support unaligned access.
    -      String arch = System.getProperty("os.arch", "");
    -      //noinspection DynamicRegexReplaceableByCompiledPattern
    -      _unaligned = arch.matches("^(i[3-6]86|x86(_64)?|x64|amd64|aarch64)$");
    +    if (arch.matches("^(ppc64le | ppc64)$")) {
    +      _unaligned = true;
    +    } else {
    +      //Since java.nio.Bits.unaligned() doesn't return true on ppc (See JDK-8165231)
    --- End diff --
    
    Moving this comment to line 50?
    
    Maybe you also need to explain it more clearly.  
    > Since java.nio.Bits.unaligned() doesn't return true on ppc (See JDK-8165231), but ppc64 and ppc64le support it


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Fix for flakey tests due to java.n...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108775406
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -46,16 +46,22 @@
       private static final boolean unaligned;
       static {
         boolean _unaligned;
    +    String arch = System.getProperty("os.arch", "");
         // use reflection to access unaligned field
         try {
           Class<?> bitsClass =
             Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
           Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
           unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    +
    +      //Since java.nio.Bits.unaligned() doesn't return true on ppc (See JDK-8165231)
    +      if(arch.matches("^(ppc64le|ppc64)$")){
    --- End diff --
    
    I think you can perform this check up top? you don't need to do all the reflection here if the arch matches. Also is it true that anything starting with "ppc64" should be special-cased? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Workaround JDK-8165231 to identify PPC64 ...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    Merged to master


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17472: [SPARK-19999]: Accommodate a new architecture tha...

Posted by samelamin <gi...@git.apache.org>.
Github user samelamin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17472#discussion_r108884506
  
    --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java ---
    @@ -46,18 +46,22 @@
       private static final boolean unaligned;
       static {
         boolean _unaligned;
    -    // use reflection to access unaligned field
    -    try {
    -      Class<?> bitsClass =
    -        Class.forName("java.nio.Bits", false, ClassLoader.getSystemClassLoader());
    -      Method unalignedMethod = bitsClass.getDeclaredMethod("unaligned");
    -      unalignedMethod.setAccessible(true);
    -      _unaligned = Boolean.TRUE.equals(unalignedMethod.invoke(null));
    -    } catch (Throwable t) {
    -      // We at least know x86 and x64 support unaligned access.
    -      String arch = System.getProperty("os.arch", "");
    -      //noinspection DynamicRegexReplaceableByCompiledPattern
    -      _unaligned = arch.matches("^(i[3-6]86|x86(_64)?|x64|amd64|aarch64)$");
    +    String arch = System.getProperty("os.arch", "");
    +    if (arch.matches("^(ppc64le | ppc64)$")) {
    --- End diff --
    
    haha its fine, called it rubber duck programming eh
    
    Sure I will make those changes, but I have no rights to rename the JIRA
    
    So I will rename this PR, can you get yourself or someone to rename the JIRA? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by samelamin <gi...@git.apache.org>.
Github user samelamin commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    @kiszk @srowen updated per your commets


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17472: [SPARK-19999]: Fix for flakey tests due to java.nio.Bits...

Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:

    https://github.com/apache/spark/pull/17472
  
    ok to test


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org