You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/09/25 09:13:45 UTC
[GitHub] [beam] iemejia opened a new pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
iemejia opened a new pull request #12937:
URL: https://github.com/apache/beam/pull/12937
R: @aromanenko-dev
CC: @ibzib
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-698828394
# [Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=h1) Report
> Merging [#12937](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d1261ad2c183f065fe907f1d3f8b1adaac8e81f8?el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12937 +/- ##
=======================================
Coverage 82.32% 82.32%
=======================================
Files 455 455
Lines 54603 54603
=======================================
+ Hits 44950 44952 +2
+ Misses 9653 9651 -2
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/12937/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `83.75% <0.00%> (-0.29%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/12937/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `89.20% <0.00%> (+0.44%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=footer). Last update [d1261ad...8f293a4](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-698828394
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-698828394
# [Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=h1) Report
> Merging [#12937](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d1261ad2c183f065fe907f1d3f8b1adaac8e81f8?el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12937 +/- ##
=======================================
Coverage 82.32% 82.32%
=======================================
Files 455 455
Lines 54603 54603
=======================================
+ Hits 44950 44952 +2
+ Misses 9653 9651 -2
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/12937/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `83.75% <0.00%> (-0.29%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/12937/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `89.20% <0.00%> (+0.44%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=footer). Last update [d1261ad...8f293a4](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-698828394
# [Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=h1) Report
> Merging [#12937](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d1261ad2c183f065fe907f1d3f8b1adaac8e81f8?el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12937 +/- ##
=======================================
Coverage 82.32% 82.32%
=======================================
Files 455 455
Lines 54603 54603
=======================================
+ Hits 44950 44952 +2
+ Misses 9653 9651 -2
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/12937/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `83.75% <0.00%> (-0.29%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/12937/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `89.20% <0.00%> (+0.44%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=footer). Last update [d1261ad...8f293a4](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-698828394
# [Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=h1) Report
> Merging [#12937](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d1261ad2c183f065fe907f1d3f8b1adaac8e81f8?el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12937 +/- ##
=======================================
Coverage 82.32% 82.32%
=======================================
Files 455 455
Lines 54603 54603
=======================================
+ Hits 44950 44952 +2
+ Misses 9653 9651 -2
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/12937/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `83.75% <0.00%> (-0.29%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/12937/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `89.20% <0.00%> (+0.44%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=footer). Last update [d1261ad...8f293a4](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] echauchot commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
echauchot commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-720419358
Sorry guys I was on a sick leave so I could not take a look at this PR. Tell me if you need any followup comments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-698820081
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-698899734
Run Spark StructuredStreaming ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-699607631
@ibzib The issues on the portable streaming tests are part of the flakes you mentioned at some point?
Apart of this I am really surprised of the issues with the Structured Streaming runner seems they could have broken something due to some backport, that error does not make sense at all. Probably needs more investigation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-706386165
> I wonder if we should get this one merged then given that the failing tests are unrelated to the version upgrade. Opinions?
Fine with me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-698981068
Run Java Spark PortableValidatesRunner Streaming
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-698828394
# [Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=h1) Report
> Merging [#12937](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d1261ad2c183f065fe907f1d3f8b1adaac8e81f8?el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12937 +/- ##
=======================================
Coverage 82.32% 82.32%
=======================================
Files 455 455
Lines 54603 54603
=======================================
+ Hits 44950 44952 +2
+ Misses 9653 9651 -2
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/12937/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `83.75% <0.00%> (-0.29%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/12937/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `89.20% <0.00%> (+0.44%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=footer). Last update [d1261ad...8f293a4](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-707868614
> Fine with me.
+1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-704275344
I wonder if we should get this one merged then given that the failing tests are unrelated to the version upgrade. Opinions?
@aromanenko-dev @ibzib
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-698820081
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-698828394
# [Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=h1) Report
> Merging [#12937](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d1261ad2c183f065fe907f1d3f8b1adaac8e81f8?el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12937 +/- ##
=======================================
Coverage 82.32% 82.32%
=======================================
Files 455 455
Lines 54603 54603
=======================================
+ Hits 44950 44952 +2
+ Misses 9653 9651 -2
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/12937/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `83.75% <0.00%> (-0.29%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/12937/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `89.20% <0.00%> (+0.44%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=footer). Last update [d1261ad...8f293a4](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-700174016
> @ibzib The issues on the portable streaming tests are part of the flakes you mentioned at some point?
Yeah, windowing is handled incorrectly: BEAM-10784
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-704273168
Just for info it seems the errors on Spark Structured Streaming runner are not related to this PR, it seems the runner has its Validates Runner tests broken since https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/2098/
Filled BEAM-11023 for further tracking.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-700525027
@echauchot The exceptions with Structured Streaming runner seems too weird, it is like if Spark was generating invalid code. I doubt that this is an issue in the runner translation, but maybe we need to double check. maybe you have a different insight?
```
Caused by: java.lang.IllegalAccessError: org/apache/beam/sdk/transforms/GroupByKeyTest$BadEqualityKey
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown Source)
at org.apache.spark.sql.execution.ObjectOperator$$anonfun$serializeObjectToRow$1.apply(objects.scala:155)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev merged pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
aromanenko-dev merged pull request #12937:
URL: https://github.com/apache/beam/pull/12937
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] echauchot commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
echauchot commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-700606662
@iemejia I'll take a look within a few days, thanks for raising this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-698820364
Run Python Spark ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia edited a comment on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
iemejia edited a comment on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-704273168
Just for info it seems the errors on Spark Structured Streaming runner are not related to this PR, it seems the runner has its Validates Runner tests broken in master since https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/2098/
Filled BEAM-11023 for further tracking.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12937: [BEAM-10969] Upgrade Spark to version 2.4.7
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12937:
URL: https://github.com/apache/beam/pull/12937#issuecomment-698828394
# [Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=h1) Report
> Merging [#12937](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d1261ad2c183f065fe907f1d3f8b1adaac8e81f8?el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12937 +/- ##
=======================================
Coverage 82.32% 82.32%
=======================================
Files 455 455
Lines 54603 54603
=======================================
+ Hits 44950 44952 +2
+ Misses 9653 9651 -2
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/12937/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `83.75% <0.00%> (-0.29%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/12937/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `89.20% <0.00%> (+0.44%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=footer). Last update [d1261ad...8f293a4](https://codecov.io/gh/apache/beam/pull/12937?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org