You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by GitBox <gi...@apache.org> on 2020/09/08 08:40:54 UTC
[GitHub] [spark-website] zhengruifeng opened a new pull request #289: Release 3.0.1
zhengruifeng opened a new pull request #289:
URL: https://github.com/apache/spark-website/pull/289
This is generated by the following.
```
$ ruby --version
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]
$ jekyll --version
jekyll 4.0.0
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] dongjoon-hyun commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-691114428
@zhengruifeng . Sorry, I'm not sure SparkR release process either.
`SparkR` package info has `Maintainer` field like the following.
- https://github.com/apache/spark/blob/master/R/pkg/DESCRIPTION#L6-L11
```
Authors@R: c(person("Shivaram", "Venkataraman", role = c("aut", "cre"),
email = "shivaram@cs.berkeley.edu"),
person("Xiangrui", "Meng", role = "aut",
email = "meng@databricks.com"),
person("Felix", "Cheung", role = "aut",
email = "felixcheung@apache.org"),
person(family = "The Apache Software Foundation", role = c("aut", "cph")))
```
@shivaram , @mengxr , @felixcheung . Could you give us some advice?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] cloud-fan commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r484764228
##########
File path: releases/_posts/2020-09-08-spark-release-3-0-1.md
##########
@@ -0,0 +1,50 @@
+---
+layout: post
+title: Spark Release 3.0.1
+categories: []
+tags: []
+status: publish
+type: post
+published: true
+meta:
+ _edit_last: '4'
+ _wpas_done_all: '1'
+---
+
+Spark 3.0.1 is a maintenance release containing stability fixes. This release is based on the branch-3.0 maintenance branch of Spark. We strongly recommend all 3.0 users to upgrade to this stable release.
+
+### Notable changes
Review comment:
shall we mention SPARK-31990? it's a correctness issue
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] viirya commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r485342723
##########
File path: site/developer-tools.html
##########
@@ -505,8 +505,8 @@ <h4>Solving a binary incompatibility</h4>
<p>For the problem described above, we might add the following:</p>
-<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// [SPARK-zz][CORE] Fix an issue
-</span><span class="nc">ProblemFilters</span><span class="o">.</span><span class="n">exclude</span><span class="o">[</span><span class="kt">DirectMissingMethodProblem</span><span class="o">](</span><span class="s">"org.apache.spark.SomeClass.this"</span><span class="o">)</span></code></pre></figure>
+<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// [SPARK-zz][CORE] Fix an issue</span>
+<span class="nv">ProblemFilters</span><span class="o">.</span><span class="py">exclude</span><span class="o">[</span><span class="kt">DirectMissingMethodProblem</span><span class="o">](</span><span class="s">"org.apache.spark.SomeClass.this"</span><span class="o">)</span></code></pre></figure>
Review comment:
I tried it locally. Just run `jekyll build` can reproduce these changes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-689409181
@dongjoon-hyun Thanks for pointing out it! I forgot to upload it to CRAN.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-689359165
@maropu doc update was done by Wencheng in https://github.com/apache/spark-website/commit/928fe4c88a691cd1b566c255c5a3243c607f3c8b
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] viirya commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r485302593
##########
File path: site/developer-tools.html
##########
@@ -505,8 +505,8 @@ <h4>Solving a binary incompatibility</h4>
<p>For the problem described above, we might add the following:</p>
-<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// [SPARK-zz][CORE] Fix an issue
-</span><span class="nc">ProblemFilters</span><span class="o">.</span><span class="n">exclude</span><span class="o">[</span><span class="kt">DirectMissingMethodProblem</span><span class="o">](</span><span class="s">"org.apache.spark.SomeClass.this"</span><span class="o">)</span></code></pre></figure>
+<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// [SPARK-zz][CORE] Fix an issue</span>
+<span class="nv">ProblemFilters</span><span class="o">.</span><span class="py">exclude</span><span class="o">[</span><span class="kt">DirectMissingMethodProblem</span><span class="o">](</span><span class="s">"org.apache.spark.SomeClass.this"</span><span class="o">)</span></code></pre></figure>
Review comment:
btw, do we have `class="py"`? I don't see `py` in the css in current site, or I missed it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng edited a comment on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng edited a comment on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-689359165
@maropu doc update was done by @cloud-fan in https://github.com/apache/spark-website/commit/928fe4c88a691cd1b566c255c5a3243c607f3c8b
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-690895923
@dongjoon-hyun I had uploaded the `SparkR_3.0.1.tar.gz` two days ago, according **Publish to CRAN** in [release-process](https://spark.apache.org/release-process.html). But I am not sure whether this upload is done. Do you know how to check its status? Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-688728799
also cc @dongjoon-hyun
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] maropu commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r484854079
##########
File path: releases/_posts/2020-09-08-spark-release-3-0-1.md
##########
@@ -0,0 +1,52 @@
+---
+layout: post
+title: Spark Release 3.0.1
+categories: []
+tags: []
+status: publish
+type: post
+published: true
+meta:
+ _edit_last: '4'
+ _wpas_done_all: '1'
+---
+
+Spark 3.0.1 is a maintenance release containing stability fixes. This release is based on the branch-3.0 maintenance branch of Spark. We strongly recommend all 3.0 users to upgrade to this stable release.
+
+### Notable changes
+ - [[SPARK-26905]](https://issues.apache.org/jira/browse/SPARK-26905): Revisit reserved/non-reserved keywords based on the ANSI SQL standard
+ - [[SPARK-31220]](https://issues.apache.org/jira/browse/SPARK-31220): repartition obeys spark.sql.adaptive.coalescePartitions.initialPartitionNum when spark.sql.adaptive.enabled
Review comment:
Ah, ok.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r485390473
##########
File path: site/developer-tools.html
##########
@@ -505,8 +505,8 @@ <h4>Solving a binary incompatibility</h4>
<p>For the problem described above, we might add the following:</p>
-<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// [SPARK-zz][CORE] Fix an issue
-</span><span class="nc">ProblemFilters</span><span class="o">.</span><span class="n">exclude</span><span class="o">[</span><span class="kt">DirectMissingMethodProblem</span><span class="o">](</span><span class="s">"org.apache.spark.SomeClass.this"</span><span class="o">)</span></code></pre></figure>
+<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// [SPARK-zz][CORE] Fix an issue</span>
+<span class="nv">ProblemFilters</span><span class="o">.</span><span class="py">exclude</span><span class="o">[</span><span class="kt">DirectMissingMethodProblem</span><span class="o">](</span><span class="s">"org.apache.spark.SomeClass.this"</span><span class="o">)</span></code></pre></figure>
Review comment:
Thank you for confirming
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] viirya commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r485301988
##########
File path: site/developer-tools.html
##########
@@ -505,8 +505,8 @@ <h4>Solving a binary incompatibility</h4>
<p>For the problem described above, we might add the following:</p>
-<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// [SPARK-zz][CORE] Fix an issue
-</span><span class="nc">ProblemFilters</span><span class="o">.</span><span class="n">exclude</span><span class="o">[</span><span class="kt">DirectMissingMethodProblem</span><span class="o">](</span><span class="s">"org.apache.spark.SomeClass.this"</span><span class="o">)</span></code></pre></figure>
+<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// [SPARK-zz][CORE] Fix an issue</span>
+<span class="nv">ProblemFilters</span><span class="o">.</span><span class="py">exclude</span><span class="o">[</span><span class="kt">DirectMissingMethodProblem</span><span class="o">](</span><span class="s">"org.apache.spark.SomeClass.this"</span><span class="o">)</span></code></pre></figure>
Review comment:
why `nc` -> `nv`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] cloud-fan commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-689544665
I'm merging it. We can announce the release after the Spark R release is done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-690895923
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r485289490
##########
File path: releases/_posts/2020-09-08-spark-release-3-0-1.md
##########
@@ -0,0 +1,52 @@
+---
+layout: post
+title: Spark Release 3.0.1
+categories: []
+tags: []
+status: publish
+type: post
+published: true
+meta:
+ _edit_last: '4'
+ _wpas_done_all: '1'
+---
+
+Spark 3.0.1 is a maintenance release containing stability fixes. This release is based on the branch-3.0 maintenance branch of Spark. We strongly recommend all 3.0 users to upgrade to this stable release.
+
+### Notable changes
+ - [[SPARK-26905]](https://issues.apache.org/jira/browse/SPARK-26905): Revisit reserved/non-reserved keywords based on the ANSI SQL standard
+ - [[SPARK-31220]](https://issues.apache.org/jira/browse/SPARK-31220): repartition obeys spark.sql.adaptive.coalescePartitions.initialPartitionNum when spark.sql.adaptive.enabled
+ - [[SPARK-31703]](https://issues.apache.org/jira/browse/SPARK-31703): Changes made by SPARK-26985 break reading parquet files correctly in BigEndian architectures (AIX + LinuxPPC64)
+ - [[SPARK-31915]](https://issues.apache.org/jira/browse/SPARK-31915): Resolve the grouping column properly per the case sensitivity in grouped and cogrouped pandas UDFs
+ - [[SPARK-31923]](https://issues.apache.org/jira/browse/SPARK-31923): Event log cannot be generated when some internal accumulators use unexpected types
+ - [[SPARK-31935]](https://issues.apache.org/jira/browse/SPARK-31935): Hadoop file system config should be effective in data source options
+ - [[SPARK-31968]](https://issues.apache.org/jira/browse/SPARK-31968): write.partitionBy() creates duplicate subdirectories when user provides duplicate columns
+ - [[SPARK-31983]](https://issues.apache.org/jira/browse/SPARK-31983): Tables of structured streaming tab show wrong result for duration column
+ - [[SPARK-31990]](https://issues.apache.org/jira/browse/SPARK-31990): Streaming's state store compatibility is broken
+ - [[SPARK-32003]](https://issues.apache.org/jira/browse/SPARK-32003): Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost
+ - [[SPARK-32038]](https://issues.apache.org/jira/browse/SPARK-32038): Regression in handling NaN values in COUNT(DISTINCT)
+ - [[SPARK-32073]](https://issues.apache.org/jira/browse/SPARK-32073): Drop R < 3.5 support
+ - [[SPARK-32092]](https://issues.apache.org/jira/browse/SPARK-32092): CrossvalidatorModel does not save all submodels (it saves only 3)
+ - [[SPARK-32136]](https://issues.apache.org/jira/browse/SPARK-32136): Spark producing incorrect groupBy results when key is a struct with nullable properties
+ - [[SPARK-32148]](https://issues.apache.org/jira/browse/SPARK-32148): LEFT JOIN generating non-deterministic and unexpected result (regression in Spark 3.0)
+ - [[SPARK-32220]](https://issues.apache.org/jira/browse/SPARK-32220): Cartesian Product Hint cause data error
+ - [[SPARK-32310]](https://issues.apache.org/jira/browse/SPARK-32310): ML params default value parity
+ - [[SPARK-32339]](https://issues.apache.org/jira/browse/SPARK-32339): Improve MLlib BLAS native acceleration docs
+ - [[SPARK-32424]](https://issues.apache.org/jira/browse/SPARK-32424): Fix silent data change for timestamp parsing if overflow happens
+ - [[SPARK-32451]](https://issues.apache.org/jira/browse/SPARK-32451): Support Apache Arrow 1.0.0 in SparkR
+ - [[SPARK-32456]](https://issues.apache.org/jira/browse/SPARK-32456): Check the Distinct by assuming it as Aggregate for Structured Streaming
+ - [[SPARK-32608]](https://issues.apache.org/jira/browse/SPARK-32608): Script Transform DELIMIT value should be formatted
+ - [[SPARK-32646]](https://issues.apache.org/jira/browse/SPARK-32646): ORC predicate pushdown should work with case-insensitive analysis
+ - [[SPARK-32658]](https://issues.apache.org/jira/browse/SPARK-32658): Partition length number overflow in PartitionWriterStream
+ - [[SPARK-32676]](https://issues.apache.org/jira/browse/SPARK-32676): Fix double caching in KMeans/BiKMeans
+
+
+### Known issues
+ - [[SPARK-31511]](https://issues.apache.org/jira/browse/SPARK-31511): Make BytesToBytesMap iterator() thread-safe
Review comment:
The four known issues here were marked fixed in the tickets. So I add this phrase for them all.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] shivaram commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
shivaram commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-691143254
@zhengruifeng sorry for the delay -- I'll get to this today
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] shivaram commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
shivaram commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-691143254
@zhengruifeng sorry for the delay -- I'll get to this today
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] viirya commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r485312315
##########
File path: site/developer-tools.html
##########
@@ -505,8 +505,8 @@ <h4>Solving a binary incompatibility</h4>
<p>For the problem described above, we might add the following:</p>
-<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// [SPARK-zz][CORE] Fix an issue
-</span><span class="nc">ProblemFilters</span><span class="o">.</span><span class="n">exclude</span><span class="o">[</span><span class="kt">DirectMissingMethodProblem</span><span class="o">](</span><span class="s">"org.apache.spark.SomeClass.this"</span><span class="o">)</span></code></pre></figure>
+<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// [SPARK-zz][CORE] Fix an issue</span>
+<span class="nv">ProblemFilters</span><span class="o">.</span><span class="py">exclude</span><span class="o">[</span><span class="kt">DirectMissingMethodProblem</span><span class="o">](</span><span class="s">"org.apache.spark.SomeClass.this"</span><span class="o">)</span></code></pre></figure>
Review comment:
oh ok. then it should be fine, i don't see change in developer-tools related to this, maybe related to jekyll.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] dongjoon-hyun commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-689370090
BTW, @zhengruifeng . Are we going to skip uploading SparkR 3.0.1?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] maropu commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r484795335
##########
File path: releases/_posts/2020-09-08-spark-release-3-0-1.md
##########
@@ -0,0 +1,52 @@
+---
+layout: post
+title: Spark Release 3.0.1
+categories: []
+tags: []
+status: publish
+type: post
+published: true
+meta:
+ _edit_last: '4'
+ _wpas_done_all: '1'
+---
+
+Spark 3.0.1 is a maintenance release containing stability fixes. This release is based on the branch-3.0 maintenance branch of Spark. We strongly recommend all 3.0 users to upgrade to this stable release.
+
+### Notable changes
+ - [[SPARK-26905]](https://issues.apache.org/jira/browse/SPARK-26905): Revisit reserved/non-reserved keywords based on the ANSI SQL standard
+ - [[SPARK-31220]](https://issues.apache.org/jira/browse/SPARK-31220): repartition obeys spark.sql.adaptive.coalescePartitions.initialPartitionNum when spark.sql.adaptive.enabled
Review comment:
Could we split this `Notable changes` section into sub-secions like `SQL`, `ML`, ...?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] cloud-fan commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r484851836
##########
File path: releases/_posts/2020-09-08-spark-release-3-0-1.md
##########
@@ -0,0 +1,52 @@
+---
+layout: post
+title: Spark Release 3.0.1
+categories: []
+tags: []
+status: publish
+type: post
+published: true
+meta:
+ _edit_last: '4'
+ _wpas_done_all: '1'
+---
+
+Spark 3.0.1 is a maintenance release containing stability fixes. This release is based on the branch-3.0 maintenance branch of Spark. We strongly recommend all 3.0 users to upgrade to this stable release.
+
+### Notable changes
+ - [[SPARK-26905]](https://issues.apache.org/jira/browse/SPARK-26905): Revisit reserved/non-reserved keywords based on the ANSI SQL standard
+ - [[SPARK-31220]](https://issues.apache.org/jira/browse/SPARK-31220): repartition obeys spark.sql.adaptive.coalescePartitions.initialPartitionNum when spark.sql.adaptive.enabled
Review comment:
seems like we don't have sub-sections in other patch releases either.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-688718055
cc @srowen @cloud-fan @HyukjinKwon @gatorsmile @viirya @maropu
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] shivaram commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
shivaram commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-691143254
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-690895923
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] dongjoon-hyun commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-691114428
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] cloud-fan commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r484765415
##########
File path: releases/_posts/2020-09-08-spark-release-3-0-1.md
##########
@@ -0,0 +1,50 @@
+---
+layout: post
+title: Spark Release 3.0.1
+categories: []
+tags: []
+status: publish
+type: post
+published: true
+meta:
+ _edit_last: '4'
+ _wpas_done_all: '1'
+---
+
+Spark 3.0.1 is a maintenance release containing stability fixes. This release is based on the branch-3.0 maintenance branch of Spark. We strongly recommend all 3.0 users to upgrade to this stable release.
+
+### Notable changes
Review comment:
and SPARK-32658, a shuffle data corruption issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r484751245
##########
File path: site/release-process.html
##########
@@ -429,7 +429,7 @@ <h4>Update the Spark Website</h4>
$ cp -R _site spark-website/site/docs/1.1.1
# Update the "latest" link
-$ cd spark/site/docs
+$ cd spark-website/site/docs
Review comment:
here should be dir `spark-website/`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] dongjoon-hyun commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-691114428
@zhengruifeng . Sorry, I'm not sure SparkR release process either.
`SparkR` package info has `Maintainer` field like the following.
- https://github.com/apache/spark/blob/master/R/pkg/DESCRIPTION#L6-L11
```
Authors@R: c(person("Shivaram", "Venkataraman", role = c("aut", "cre"),
email = "shivaram@cs.berkeley.edu"),
person("Xiangrui", "Meng", role = "aut",
email = "meng@databricks.com"),
person("Felix", "Cheung", role = "aut",
email = "felixcheung@apache.org"),
person(family = "The Apache Software Foundation", role = c("aut", "cph")))
```
@shivaram , @mengxr , @felixcheung . Could you give us some advice?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r485309360
##########
File path: site/developer-tools.html
##########
@@ -505,8 +505,8 @@ <h4>Solving a binary incompatibility</h4>
<p>For the problem described above, we might add the following:</p>
-<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// [SPARK-zz][CORE] Fix an issue
-</span><span class="nc">ProblemFilters</span><span class="o">.</span><span class="n">exclude</span><span class="o">[</span><span class="kt">DirectMissingMethodProblem</span><span class="o">](</span><span class="s">"org.apache.spark.SomeClass.this"</span><span class="o">)</span></code></pre></figure>
+<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// [SPARK-zz][CORE] Fix an issue</span>
+<span class="nv">ProblemFilters</span><span class="o">.</span><span class="py">exclude</span><span class="o">[</span><span class="kt">DirectMissingMethodProblem</span><span class="o">](</span><span class="s">"org.apache.spark.SomeClass.this"</span><span class="o">)</span></code></pre></figure>
Review comment:
@viirya those changes in the second commit(3b416ecb027c9262a28ad778d50634d6bd582f81) were made by running `jekyll build` in dir `spark-website`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] cloud-fan merged pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
cloud-fan merged pull request #289:
URL: https://github.com/apache/spark-website/pull/289
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] maropu commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r484800121
##########
File path: releases/_posts/2020-09-08-spark-release-3-0-1.md
##########
@@ -0,0 +1,52 @@
+---
+layout: post
+title: Spark Release 3.0.1
+categories: []
+tags: []
+status: publish
+type: post
+published: true
+meta:
+ _edit_last: '4'
+ _wpas_done_all: '1'
+---
+
+Spark 3.0.1 is a maintenance release containing stability fixes. This release is based on the branch-3.0 maintenance branch of Spark. We strongly recommend all 3.0 users to upgrade to this stable release.
+
+### Notable changes
+ - [[SPARK-26905]](https://issues.apache.org/jira/browse/SPARK-26905): Revisit reserved/non-reserved keywords based on the ANSI SQL standard
+ - [[SPARK-31220]](https://issues.apache.org/jira/browse/SPARK-31220): repartition obeys spark.sql.adaptive.coalescePartitions.initialPartitionNum when spark.sql.adaptive.enabled
+ - [[SPARK-31703]](https://issues.apache.org/jira/browse/SPARK-31703): Changes made by SPARK-26985 break reading parquet files correctly in BigEndian architectures (AIX + LinuxPPC64)
+ - [[SPARK-31915]](https://issues.apache.org/jira/browse/SPARK-31915): Resolve the grouping column properly per the case sensitivity in grouped and cogrouped pandas UDFs
+ - [[SPARK-31923]](https://issues.apache.org/jira/browse/SPARK-31923): Event log cannot be generated when some internal accumulators use unexpected types
+ - [[SPARK-31935]](https://issues.apache.org/jira/browse/SPARK-31935): Hadoop file system config should be effective in data source options
+ - [[SPARK-31968]](https://issues.apache.org/jira/browse/SPARK-31968): write.partitionBy() creates duplicate subdirectories when user provides duplicate columns
+ - [[SPARK-31983]](https://issues.apache.org/jira/browse/SPARK-31983): Tables of structured streaming tab show wrong result for duration column
+ - [[SPARK-31990]](https://issues.apache.org/jira/browse/SPARK-31990): Streaming's state store compatibility is broken
+ - [[SPARK-32003]](https://issues.apache.org/jira/browse/SPARK-32003): Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost
+ - [[SPARK-32038]](https://issues.apache.org/jira/browse/SPARK-32038): Regression in handling NaN values in COUNT(DISTINCT)
+ - [[SPARK-32073]](https://issues.apache.org/jira/browse/SPARK-32073): Drop R < 3.5 support
+ - [[SPARK-32092]](https://issues.apache.org/jira/browse/SPARK-32092): CrossvalidatorModel does not save all submodels (it saves only 3)
+ - [[SPARK-32136]](https://issues.apache.org/jira/browse/SPARK-32136): Spark producing incorrect groupBy results when key is a struct with nullable properties
+ - [[SPARK-32148]](https://issues.apache.org/jira/browse/SPARK-32148): LEFT JOIN generating non-deterministic and unexpected result (regression in Spark 3.0)
+ - [[SPARK-32220]](https://issues.apache.org/jira/browse/SPARK-32220): Cartesian Product Hint cause data error
+ - [[SPARK-32310]](https://issues.apache.org/jira/browse/SPARK-32310): ML params default value parity
+ - [[SPARK-32339]](https://issues.apache.org/jira/browse/SPARK-32339): Improve MLlib BLAS native acceleration docs
+ - [[SPARK-32424]](https://issues.apache.org/jira/browse/SPARK-32424): Fix silent data change for timestamp parsing if overflow happens
+ - [[SPARK-32451]](https://issues.apache.org/jira/browse/SPARK-32451): Support Apache Arrow 1.0.0 in SparkR
+ - [[SPARK-32456]](https://issues.apache.org/jira/browse/SPARK-32456): Check the Distinct by assuming it as Aggregate for Structured Streaming
+ - [[SPARK-32608]](https://issues.apache.org/jira/browse/SPARK-32608): Script Transform DELIMIT value should be formatted
+ - [[SPARK-32646]](https://issues.apache.org/jira/browse/SPARK-32646): ORC predicate pushdown should work with case-insensitive analysis
+ - [[SPARK-32658]](https://issues.apache.org/jira/browse/SPARK-32658): Partition length number overflow in PartitionWriterStream
+ - [[SPARK-32676]](https://issues.apache.org/jira/browse/SPARK-32676): Fix double caching in KMeans/BiKMeans
+
+
+### Known issues
+ - [[SPARK-31511]](https://issues.apache.org/jira/browse/SPARK-31511): Make BytesToBytesMap iterator() thread-safe
Review comment:
To follow [the 3.0.0 release note](https://spark.apache.org/releases/spark-release-3-0-0.html), how about adding a phrase `This will be fixed in Spark 3.0.2.` for already-fixed issues?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-690895923
@dongjoon-hyun I had uploaded the `SparkR_3.0.1.tar.gz` two days ago, according **Publish to CRAN** in [release-process](https://spark.apache.org/release-process.html). But I am not sure whether this upload is done. Do you know how to check its status? Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng commented on pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on pull request #289:
URL: https://github.com/apache/spark-website/pull/289#issuecomment-689910659
Thanks all!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
[GitHub] [spark-website] zhengruifeng commented on a change in pull request #289: Release 3.0.1
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on a change in pull request #289:
URL: https://github.com/apache/spark-website/pull/289#discussion_r484793702
##########
File path: site/docs/latest
##########
@@ -1 +1 @@
-3.0.0
\ No newline at end of file
+3.0.1
Review comment:
```
ln -s 3.0.1 latest
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org