You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/09/29 11:29:20 UTC

[GitHub] [beam] nielsbasjes opened a new pull request #12962: [BEAM-9379] Update calcite to 1.25 (WIP/INCOMPLETE)

nielsbasjes opened a new pull request #12962:
URL: https://github.com/apache/beam/pull/12962


   In these commits I have done the following:
   1. Update the vendored calcite build to version 1.25
   1. Use that new Calcite version in Beam.
   1. Attempt (!!) to fix all the changes that have occurred in Calcite over the last year.
   
   To test this I have run these to install the new calcite locally 
   ```
   ./gradlew -p vendor/calcite-1_25_0  check
   ./gradlew -p vendor/calcite-1_25_0  -PvendoredDependenciesOnly  publishToMavenLocal 
   ```
   
   Current state of this initial pull request:
   1. WORK IN PROGRESS. And I have currently reached the boundaries of my knowledge of this part of the system. Help is needed.
   1. It probably won't build in CI because the vendored Calcite has not yet been released.
   1. There are lots of changes that I did to make it compile that I'm not sure are correct. My current knowledge of this is too limited at this time.
   1. If I run this locally (i.e. ./gradlew build -p sdks/java/extensions/sql/  ) it all builds and about 10% of the tests fail (i.e. about 90% pass)
   
   R:@amaliujia
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [x] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
    - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
    - [x] Update `CHANGES.md` with noteworthy changes.
    - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
   
   Post-Commit Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2
   --- | --- | --- | --- | --- | --- | ---
   Go | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | ---
   Java | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/i
 con)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[![Build Status](htt
 ps://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/)
   Python | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[![Build Status](https://ci-beam
 .apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/) | ---
   XLang | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/) | ---
   
   Pre-Commit Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   --- |Java | Python | Go | Website | Whitespace | Typescript
   --- | --- | --- | --- | --- | --- | ---
   Non-portable | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/) <br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/be
 am_PreCommit_Go_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/)
   Portable | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | --- | --- | ---
   
   See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
   
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/cee7388365970774969e63fe90fdf3cdd804e79f?el=desc) will **decrease** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #12962      +/-   ##
   ==========================================
   - Coverage   82.34%   82.33%   -0.01%     
   ==========================================
     Files         455      455              
     Lines       54650    54647       -3     
   ==========================================
   - Hits        45002    44995       -7     
   - Misses       9648     9652       +4     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `88.68% <0.00%> (-1.23%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `90.90% <0.00%> (-0.76%)` | :arrow_down: |
   | [...nners/portability/fn\_api\_runner/worker\_handlers.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3dvcmtlcl9oYW5kbGVycy5weQ==) | `80.57% <0.00%> (-0.18%)` | :arrow_down: |
   | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.98% <0.00%> (-0.17%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/gcp/bigquery.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5LnB5) | `80.56% <0.00%> (-0.15%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `94.31% <0.00%> (-0.14%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/fileio.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZWlvLnB5) | `95.80% <0.00%> (-0.05%)` | :arrow_down: |
   | [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `98.90% <0.00%> (+0.02%)` | :arrow_up: |
   | [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `89.20% <0.00%> (+0.44%)` | :arrow_up: |
   | [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `79.53% <0.00%> (+0.67%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [cee7388...19d11b7](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] nielsbasjes commented on a change in pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

nielsbasjes commented on a change in pull request #12962:
URL: https://github.com/apache/beam/pull/12962#discussion_r497363201



##########
File path: vendor/calcite-1_25_0/build.gradle
##########
@@ -18,29 +18,33 @@
 
 plugins { id 'org.apache.beam.vendor-java' }
 
-description = "Apache Beam :: Vendored Dependencies :: Calcite 1.20.0"
+description = "Apache Beam :: Vendored Dependencies :: Calcite 1.25.0"
 
 group = "org.apache.beam"
 version = "0.2"

Review comment:
       Yes




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] nielsbasjes edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

nielsbasjes edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-739500944


   I am in need of advice / assistance.
   
   What I ran into is that some tests in the current state of this pull request fail over a change in Calcite.
   I am specifically talking about https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/jdbc/src/test/java/org/apache/beam/sdk/extensions/sql/jdbc/BeamSqlLineTest.java#L63 and many other tests in this file that do a `CREATE EXTERNAL TABLE` or a `DROP TABLE`.
   
   I tracked the source of this exception back to Calcite where a few months ago this default method was added in Calcite by @julianhyde : https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/parser/SqlParserImplFactory.java#L58
   
   This default implementation simply returns a `DdlExecutor` (a new interface) that always fails with a `UnsupportedOperationException` if no valid implementation has been provided.
   Looks good so far.
   
   The problem I have is that my knowledge is lacking how to correctly override this method to return the implementation for Beam that makes it possible to do a `CREATE EXTERNAL TABLE`.
   
   A pointer on how I should fix this correctly is highly appreciated.
   
   ----
   
   I found in the generated code of BeamSqlParserImpl this code (I shortened it a bit)
   ```
       public static final SqlParserImplFactory FACTORY = new SqlParserImplFactory() {
           public SqlAbstractParserImpl getParser(Reader reader) {
               final BeamSqlParserImpl parser = new BeamSqlParserImpl(reader);
   ...
               return parser;
           }
       };
   ```
   which is generated from a template in Calcite itself and (as far as I have been able to find so far) does not have a way of implementing a non-default method for `DdlExecutor getDdlExecutor()`.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) (de8ff95) into [master](https://codecov.io/gh/apache/beam/commit/80248d00abeabdbcfc81d876638c5437fa118e41?el=desc) (80248d0) will **decrease** coverage by `0.02%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #12962      +/-   ##
   ==========================================
   - Coverage   82.72%   82.69%   -0.03%     
   ==========================================
     Files         466      466              
     Lines       57518    57518              
   ==========================================
   - Hits        47582    47567      -15     
   - Misses       9936     9951      +15     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `90.47% <0.00%> (-4.77%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `91.91% <0.00%> (-2.21%)` | :arrow_down: |
   | [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `74.30% <0.00%> (-1.12%)` | :arrow_down: |
   | [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `86.45% <0.00%> (-1.05%)` | :arrow_down: |
   | [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `85.34% <0.00%> (-0.53%)` | :arrow_down: |
   | [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `89.92% <0.00%> (-0.36%)` | :arrow_down: |
   | [.../python/apache\_beam/typehints/trivial\_inference.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3RyaXZpYWxfaW5mZXJlbmNlLnB5) | `89.18% <0.00%> (-0.27%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.81% <0.00%> (-0.27%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.44% <0.00%> (-0.26%)` | :arrow_down: |
   | [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.66% <0.00%> (-0.18%)` | :arrow_down: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [80248d0...de8ff95](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] vectorijk commented on a change in pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

vectorijk commented on a change in pull request #12962:
URL: https://github.com/apache/beam/pull/12962#discussion_r496966029



##########
File path: vendor/calcite-1_25_0/build.gradle
##########
@@ -18,29 +18,33 @@
 
 plugins { id 'org.apache.beam.vendor-java' }
 
-description = "Apache Beam :: Vendored Dependencies :: Calcite 1.20.0"
+description = "Apache Beam :: Vendored Dependencies :: Calcite 1.25.0"
 
 group = "org.apache.beam"
 version = "0.2"
 
-def calcite_version = "1.20.0"
+def calcite_version = "1.25.0"
 def avatica_version = "1.16.0"
-def prefix = "org.apache.beam.vendor.calcite.v1_20_0"
+def prefix = "org.apache.beam.vendor.calcite.v1_25_0"
 
 List<String> packagesToRelocate = [
         "com.esri",
+        "com.fasterxml",
         "com.google.common",
-        "com.google.thirdparty",
         "com.google.protobuf",
-        "com.fasterxml",
+        "com.google.thirdparty",
         "com.jayway",
         "com.yahoo",
+        "javax.annotation",
+        "net.minidev",
         "org.apache.calcite",
         "org.apache.commons",
         "org.apache.http",
+        "org.apiguardian.api",
         "org.codehaus",
+        "org.objectweb",
         "org.pentaho",
-        "org.yaml"
+        "org.yaml",
 ]
 
 vendorJava(

Review comment:
       as discussed in #11058, there are some logging libraries that might need to be put into runtimeDepencies. for you reference: https://github.com/vectorijk/beam/blob/calcite-1-25/vendor/calcite-1_25_0/build.gradle




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) (19d11b7) into [master](https://codecov.io/gh/apache/beam/commit/8ab1955550931fad398048a5990ebea868406587?el=desc) (8ab1955) will **increase** coverage by `47.53%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master   #12962       +/-   ##
   ===========================================
   + Coverage   34.80%   82.33%   +47.53%     
   ===========================================
     Files         437      455       +18     
     Lines       55008    54647      -361     
   ===========================================
   + Hits        19144    44995    +25851     
   + Misses      35864     9652    -26212     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [conftest.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-Y29uZnRlc3QucHk=) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
   | [sdks/python/apache\_beam/utils/urns.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvdXJucy5weQ==) | `88.88% <0.00%> (-0.18%)` | :arrow_down: |
   | [setup.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
   | [sdks/python/apache\_beam/runners/internal/names.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcm5hbC9uYW1lcy5weQ==) | `100.00% <0.00%> (ø)` | |
   | [...hon/apache\_beam/runners/dataflow/internal/names.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9uYW1lcy5weQ==) | `100.00% <0.00%> (ø)` | |
   | [...beam/runners/portability/local\_job\_service\_main.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9sb2NhbF9qb2Jfc2VydmljZV9tYWluLnB5) | `0.00% <0.00%> (ø)` | |
   | [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
   | [.../internal/clients/bigquery/bigquery\_v2\_messages.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvYmlncXVlcnkvYmlncXVlcnlfdjJfbWVzc2FnZXMucHk=) | `100.00% <0.00%> (ø)` | |
   | [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | | |
   | [sdks/python/apache\_beam/io/gcp/spanner.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL3NwYW5uZXIucHk=) | | |
   | ... and [351 more](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [30f9a60...aee483f](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kennknowles commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

kennknowles commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-701703218


   https://scans.gradle.com/s/ofcfnu774du4y/tests/:sdks:java:extensions:sql:zetasql:test/org.apache.beam.sdk.extensions.sql.zetasql.ZetaSqlTimeFunctionsTest/testFormatDate#1
   
   this one looks easier - just a change of capital to lowercase CC @robinyqiu 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] vectorijk commented on a change in pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

vectorijk commented on a change in pull request #12962:
URL: https://github.com/apache/beam/pull/12962#discussion_r496964747



##########
File path: vendor/calcite-1_25_0/build.gradle
##########
@@ -18,29 +18,33 @@
 
 plugins { id 'org.apache.beam.vendor-java' }
 
-description = "Apache Beam :: Vendored Dependencies :: Calcite 1.20.0"
+description = "Apache Beam :: Vendored Dependencies :: Calcite 1.25.0"
 
 group = "org.apache.beam"
 version = "0.2"

Review comment:
       let's start from 0.1?

##########
File path: vendor/calcite-1_25_0/build.gradle
##########
@@ -18,29 +18,33 @@
 
 plugins { id 'org.apache.beam.vendor-java' }
 
-description = "Apache Beam :: Vendored Dependencies :: Calcite 1.20.0"
+description = "Apache Beam :: Vendored Dependencies :: Calcite 1.25.0"
 
 group = "org.apache.beam"
 version = "0.2"
 
-def calcite_version = "1.20.0"
+def calcite_version = "1.25.0"
 def avatica_version = "1.16.0"
-def prefix = "org.apache.beam.vendor.calcite.v1_20_0"
+def prefix = "org.apache.beam.vendor.calcite.v1_25_0"
 
 List<String> packagesToRelocate = [
         "com.esri",
+        "com.fasterxml",
         "com.google.common",
-        "com.google.thirdparty",
         "com.google.protobuf",
-        "com.fasterxml",
+        "com.google.thirdparty",
         "com.jayway",
         "com.yahoo",
+        "javax.annotation",
+        "net.minidev",
         "org.apache.calcite",
         "org.apache.commons",
         "org.apache.http",
+        "org.apiguardian.api",
         "org.codehaus",
+        "org.objectweb",
         "org.pentaho",
-        "org.yaml"
+        "org.yaml",
 ]
 
 vendorJava(

Review comment:
       some logging libraries need to be put into runtimeDepencies. for you reference: https://github.com/vectorijk/beam/blob/calcite-1-25/vendor/calcite-1_25_0/build.gradle




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kennknowles commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

kennknowles commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-701704059


   https://scans.gradle.com/s/ofcfnu774du4y/tests/:sdks:java:extensions:sql:test/org.apache.beam.sdk.extensions.sql.impl.udf.BeamSqlUdfExpressionTest/testSHA1?anchor=e30&focused-exception-line=0-1-0#1
   
   Switch from `byte[]` to avatica's `ByteString`


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] nielsbasjes commented on pull request #12962: [BEAM-9379] Update calcite to 1.25 (WIP/INCOMPLETE)

Posted by GitBox <gi...@apache.org>.

nielsbasjes commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700642647


   R: @amaliujia


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kennknowles commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

kennknowles commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-701703858


   https://scans.gradle.com/s/ofcfnu774du4y/tests/:sdks:java:extensions:sql:test/org.apache.beam.sdk.extensions.sql.BeamSqlDslAggregationTest/testWindowOnNonTimestampField#1
   
   This is just a bad test. It checks the exact string instead of the key elements of the string so it is sensitive to irrelevant changes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) (19d11b7) into [master](https://codecov.io/gh/apache/beam/commit/8ab1955550931fad398048a5990ebea868406587?el=desc) (8ab1955) will **increase** coverage by `47.53%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master   #12962       +/-   ##
   ===========================================
   + Coverage   34.80%   82.33%   +47.53%     
   ===========================================
     Files         437      455       +18     
     Lines       55008    54647      -361     
   ===========================================
   + Hits        19144    44995    +25851     
   + Misses      35864     9652    -26212     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [conftest.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-Y29uZnRlc3QucHk=) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
   | [sdks/python/apache\_beam/utils/urns.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvdXJucy5weQ==) | `88.88% <0.00%> (-0.18%)` | :arrow_down: |
   | [setup.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
   | [sdks/python/apache\_beam/runners/internal/names.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcm5hbC9uYW1lcy5weQ==) | `100.00% <0.00%> (ø)` | |
   | [...hon/apache\_beam/runners/dataflow/internal/names.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9uYW1lcy5weQ==) | `100.00% <0.00%> (ø)` | |
   | [...beam/runners/portability/local\_job\_service\_main.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9sb2NhbF9qb2Jfc2VydmljZV9tYWluLnB5) | `0.00% <0.00%> (ø)` | |
   | [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
   | [.../internal/clients/bigquery/bigquery\_v2\_messages.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvYmlncXVlcnkvYmlncXVlcnlfdjJfbWVzc2FnZXMucHk=) | `100.00% <0.00%> (ø)` | |
   | [...ython/apache\_beam/io/gcp/bigquery\_read\_internal.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X3JlYWRfaW50ZXJuYWwucHk=) | | |
   | [...\_beam/runners/interactive/user\_pipeline\_tracker.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS91c2VyX3BpcGVsaW5lX3RyYWNrZXIucHk=) | | |
   | ... and [351 more](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [30f9a60...eec087e](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) (19d11b7) into [master](https://codecov.io/gh/apache/beam/commit/8ab1955550931fad398048a5990ebea868406587?el=desc) (8ab1955) will **increase** coverage by `47.53%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master   #12962       +/-   ##
   ===========================================
   + Coverage   34.80%   82.33%   +47.53%     
   ===========================================
     Files         437      455       +18     
     Lines       55008    54647      -361     
   ===========================================
   + Hits        19144    44995    +25851     
   + Misses      35864     9652    -26212     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [conftest.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-Y29uZnRlc3QucHk=) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
   | [sdks/python/apache\_beam/utils/urns.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvdXJucy5weQ==) | `88.88% <0.00%> (-0.18%)` | :arrow_down: |
   | [setup.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
   | [sdks/python/apache\_beam/runners/internal/names.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcm5hbC9uYW1lcy5weQ==) | `100.00% <0.00%> (ø)` | |
   | [...hon/apache\_beam/runners/dataflow/internal/names.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9uYW1lcy5weQ==) | `100.00% <0.00%> (ø)` | |
   | [...beam/runners/portability/local\_job\_service\_main.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9sb2NhbF9qb2Jfc2VydmljZV9tYWluLnB5) | `0.00% <0.00%> (ø)` | |
   | [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
   | [.../internal/clients/bigquery/bigquery\_v2\_messages.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvYmlncXVlcnkvYmlncXVlcnlfdjJfbWVzc2FnZXMucHk=) | `100.00% <0.00%> (ø)` | |
   | [sdks/python/apache\_beam/utils/sharded\_key.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc2hhcmRlZF9rZXkucHk=) | | |
   | [...ython/apache\_beam/io/gcp/bigquery\_read\_internal.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X3JlYWRfaW50ZXJuYWwucHk=) | | |
   | ... and [351 more](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [30f9a60...f94f6cf](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] commented on pull request #12962: [BEAM-9379] Update calcite to 1.25 (WIP/INCOMPLETE)

Posted by GitBox <gi...@apache.org>.

codecov[bot] commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/cee7388365970774969e63fe90fdf3cdd804e79f?el=desc) will **decrease** coverage by `0.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #12962      +/-   ##
   ==========================================
   - Coverage   82.34%   82.33%   -0.02%     
   ==========================================
     Files         455      455              
     Lines       54650    54650              
   ==========================================
   - Hits        45002    44996       -6     
   - Misses       9648     9654       +6     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `88.68% <0.00%> (-1.23%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `90.90% <0.00%> (-0.76%)` | :arrow_down: |
   | [...nners/portability/fn\_api\_runner/worker\_handlers.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3dvcmtlcl9oYW5kbGVycy5weQ==) | `80.57% <0.00%> (-0.18%)` | :arrow_down: |
   | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.98% <0.00%> (-0.17%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `94.58% <0.00%> (+0.13%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [cee7388...f498320](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kennknowles commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

kennknowles commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-701703585


   https://scans.gradle.com/s/ofcfnu774du4y/tests/:sdks:java:extensions:sql:test/org.apache.beam.sdk.extensions.sql.impl.rel.BeamMatchRelTest/matchPrevFunctionTest#1
   
   Seems there is something special and missing from `BeamMatchRel`


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] apilloud commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

apilloud commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-749173309


   Thanks for all the work you've done on this so far! Hopefully you saw [my response to your email to dev@](https://lists.apache.org/thread.html/ra9bfbc1f6db11dbc3d033fdc0e1015110032b1a88ce20be0301cee3f%40%3Cdev.beam.apache.org%3E) about about the DDL issues. The other two things you are seeing are probably around changes that need to be made to the type translation code in [BeamCalcRel.java](https://github.com/apache/beam/blob/3bb232fb098700de408f574585dfe74bbaff7230/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamCalcRel.java).
   
   I don't have time to pick this up right now, but I do have time planned for this in Late January/Early February. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) (de8ff95) into [master](https://codecov.io/gh/apache/beam/commit/80248d00abeabdbcfc81d876638c5437fa118e41?el=desc) (80248d0) will **decrease** coverage by `0.02%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #12962      +/-   ##
   ==========================================
   - Coverage   82.72%   82.69%   -0.03%     
   ==========================================
     Files         466      466              
     Lines       57518    57518              
   ==========================================
   - Hits        47582    47567      -15     
   - Misses       9936     9951      +15     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `90.47% <0.00%> (-4.77%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `91.91% <0.00%> (-2.21%)` | :arrow_down: |
   | [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `74.30% <0.00%> (-1.12%)` | :arrow_down: |
   | [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `86.45% <0.00%> (-1.05%)` | :arrow_down: |
   | [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `85.34% <0.00%> (-0.53%)` | :arrow_down: |
   | [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `89.92% <0.00%> (-0.36%)` | :arrow_down: |
   | [.../python/apache\_beam/typehints/trivial\_inference.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3RyaXZpYWxfaW5mZXJlbmNlLnB5) | `89.18% <0.00%> (-0.27%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.81% <0.00%> (-0.27%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.44% <0.00%> (-0.26%)` | :arrow_down: |
   | [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.66% <0.00%> (-0.18%)` | :arrow_down: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [80248d0...de8ff95](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) (19d11b7) into [master](https://codecov.io/gh/apache/beam/commit/8ab1955550931fad398048a5990ebea868406587?el=desc) (8ab1955) will **increase** coverage by `47.53%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master   #12962       +/-   ##
   ===========================================
   + Coverage   34.80%   82.33%   +47.53%     
   ===========================================
     Files         437      455       +18     
     Lines       55008    54647      -361     
   ===========================================
   + Hits        19144    44995    +25851     
   + Misses      35864     9652    -26212     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [conftest.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-Y29uZnRlc3QucHk=) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
   | [sdks/python/apache\_beam/utils/urns.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvdXJucy5weQ==) | `88.88% <0.00%> (-0.18%)` | :arrow_down: |
   | [setup.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
   | [sdks/python/apache\_beam/runners/internal/names.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcm5hbC9uYW1lcy5weQ==) | `100.00% <0.00%> (ø)` | |
   | [...hon/apache\_beam/runners/dataflow/internal/names.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9uYW1lcy5weQ==) | `100.00% <0.00%> (ø)` | |
   | [...beam/runners/portability/local\_job\_service\_main.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9sb2NhbF9qb2Jfc2VydmljZV9tYWluLnB5) | `0.00% <0.00%> (ø)` | |
   | [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
   | [.../internal/clients/bigquery/bigquery\_v2\_messages.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvYmlncXVlcnkvYmlncXVlcnlfdjJfbWVzc2FnZXMucHk=) | `100.00% <0.00%> (ø)` | |
   | [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | | |
   | [sdks/python/apache\_beam/utils/sharded\_key.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc2hhcmRlZF9rZXkucHk=) | | |
   | ... and [351 more](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [30f9a60...c7b4b7f](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] aaltay commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

aaltay commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-747825843


   > I am in need of advice / assistance.
   
   @tysonjh @amaliujia @kennknowles - Who could help on this PR?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) (19d11b7) into [master](https://codecov.io/gh/apache/beam/commit/8ab1955550931fad398048a5990ebea868406587?el=desc) (8ab1955) will **increase** coverage by `47.53%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master   #12962       +/-   ##
   ===========================================
   + Coverage   34.80%   82.33%   +47.53%     
   ===========================================
     Files         437      455       +18     
     Lines       55008    54647      -361     
   ===========================================
   + Hits        19144    44995    +25851     
   + Misses      35864     9652    -26212     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [conftest.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-Y29uZnRlc3QucHk=) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
   | [sdks/python/apache\_beam/utils/urns.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvdXJucy5weQ==) | `88.88% <0.00%> (-0.18%)` | :arrow_down: |
   | [setup.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
   | [sdks/python/apache\_beam/runners/internal/names.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcm5hbC9uYW1lcy5weQ==) | `100.00% <0.00%> (ø)` | |
   | [...hon/apache\_beam/runners/dataflow/internal/names.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9uYW1lcy5weQ==) | `100.00% <0.00%> (ø)` | |
   | [...beam/runners/portability/local\_job\_service\_main.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9sb2NhbF9qb2Jfc2VydmljZV9tYWluLnB5) | `0.00% <0.00%> (ø)` | |
   | [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
   | [.../internal/clients/bigquery/bigquery\_v2\_messages.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvYmlncXVlcnkvYmlncXVlcnlfdjJfbWVzc2FnZXMucHk=) | `100.00% <0.00%> (ø)` | |
   | [...he\_beam/transforms/combinefn\_lifecycle\_pipeline.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9jb21iaW5lZm5fbGlmZWN5Y2xlX3BpcGVsaW5lLnB5) | | |
   | [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | | |
   | ... and [351 more](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [30f9a60...b28c39f](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kennknowles commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

kennknowles commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700893568


   Can you publish a build scan by running your second step with `--scan` ? Then we can see what is broken.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] nielsbasjes commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

nielsbasjes commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-701269350


   > Can you publish a build scan by running your second step with `--scan` ? Then we can see what is broken.
   
   https://scans.gradle.com/s/ofcfnu774du4y


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/cee7388365970774969e63fe90fdf3cdd804e79f?el=desc) will **decrease** coverage by `0.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #12962      +/-   ##
   ==========================================
   - Coverage   82.34%   82.33%   -0.02%     
   ==========================================
     Files         455      455              
     Lines       54650    54650              
   ==========================================
   - Hits        45002    44996       -6     
   - Misses       9648     9654       +6     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `88.68% <0.00%> (-1.23%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `90.90% <0.00%> (-0.76%)` | :arrow_down: |
   | [...nners/portability/fn\_api\_runner/worker\_handlers.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3dvcmtlcl9oYW5kbGVycy5weQ==) | `80.57% <0.00%> (-0.18%)` | :arrow_down: |
   | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.98% <0.00%> (-0.17%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `94.58% <0.00%> (+0.13%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [cee7388...19d11b7](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] nielsbasjes commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

nielsbasjes commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-912580440


   All of this was fixed in https://github.com/apache/beam/pull/14729


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) (19d11b7) into [master](https://codecov.io/gh/apache/beam/commit/80248d00abeabdbcfc81d876638c5437fa118e41?el=desc) (80248d0) will **decrease** coverage by `0.38%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #12962      +/-   ##
   ==========================================
   - Coverage   82.72%   82.33%   -0.39%     
   ==========================================
     Files         466      455      -11     
     Lines       57518    54647    -2871     
   ==========================================
   - Hits        47582    44995    -2587     
   + Misses       9936     9652     -284     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/utils/profiler.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvcHJvZmlsZXIucHk=) | `32.11% <0.00%> (-54.35%)` | :arrow_down: |
   | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `83.33% <0.00%> (-11.91%)` | :arrow_down: |
   | [sdks/python/apache\_beam/typehints/row\_type.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3Jvd190eXBlLnB5) | `81.81% <0.00%> (-4.55%)` | :arrow_down: |
   | [sdks/python/apache\_beam/dataframe/frame\_base.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lX2Jhc2UucHk=) | `85.31% <0.00%> (-3.16%)` | :arrow_down: |
   | [sdks/python/apache\_beam/transforms/core.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9jb3JlLnB5) | `88.63% <0.00%> (-2.92%)` | :arrow_down: |
   | [...ython/apache\_beam/io/gcp/tests/bigquery\_matcher.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL3Rlc3RzL2JpZ3F1ZXJ5X21hdGNoZXIucHk=) | `79.83% <0.00%> (-2.87%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/mongodbio.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbW9uZ29kYmlvLnB5) | `93.15% <0.00%> (-2.80%)` | :arrow_down: |
   | [sdks/python/apache\_beam/metrics/metricbase.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWV0cmljcy9tZXRyaWNiYXNlLnB5) | `87.87% <0.00%> (-2.51%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/gcp/bigtableio.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3RhYmxlaW8ucHk=) | `42.22% <0.00%> (-2.46%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/textio.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdGV4dGlvLnB5) | `93.30% <0.00%> (-2.40%)` | :arrow_down: |
   | ... and [131 more](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [80248d0...de8ff95](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] nielsbasjes commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

nielsbasjes commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-754695051


   @apilloud Yes, I read and followed the links from your email. 
   I tried to understand this and given my current knowledge of this I failed in fixing the issues at hand.
   When you pick this up in a few weeks remember I'm still willing to help out where I can.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] iemejia commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

iemejia commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-706286868


   Can we consider moving this one to 1.26 because of https://lists.apache.org/thread.html/r0b0fbe2038388175951ce1028182d980f9e9a7328be13d52dab70bb3%40%3Cannounce.apache.org%3E
   
   Probably Beam's use case of Calcite is not impacted by this but I can envision the automatic vulnerability tools complaining on this soon.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kennknowles commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

kennknowles commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-701702697


   https://scans.gradle.com/s/ofcfnu774du4y/tests/:sdks:java:extensions:sql:test/org.apache.beam.sdk.extensions.sql.BeamSqlDslUdfUdafTest/testUdfWithListInput#1
   
   Seems parsing of arrays changed?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] apilloud commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

apilloud commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-912634356


   Thanks for all the work you did to make this happen!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] nielsbasjes closed pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

nielsbasjes closed pull request #12962:
URL: https://github.com/apache/beam/pull/12962


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kennknowles commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

kennknowles commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-701702361


   https://scans.gradle.com/s/ofcfnu774du4y/tests/:sdks:java:extensions:sql:test/org.apache.beam.sdk.extensions.sql.BeamSqlDslSqlStdOperatorsTest/testArithmeticOperator#1
   
   Looks like Calcite switched to `java.math.BigDecimal`?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] nielsbasjes commented on a change in pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

nielsbasjes commented on a change in pull request #12962:
URL: https://github.com/apache/beam/pull/12962#discussion_r497365708



##########
File path: vendor/calcite-1_25_0/build.gradle
##########
@@ -18,29 +18,33 @@
 
 plugins { id 'org.apache.beam.vendor-java' }
 
-description = "Apache Beam :: Vendored Dependencies :: Calcite 1.20.0"
+description = "Apache Beam :: Vendored Dependencies :: Calcite 1.25.0"
 
 group = "org.apache.beam"
 version = "0.2"
 
-def calcite_version = "1.20.0"
+def calcite_version = "1.25.0"
 def avatica_version = "1.16.0"
-def prefix = "org.apache.beam.vendor.calcite.v1_20_0"
+def prefix = "org.apache.beam.vendor.calcite.v1_25_0"
 
 List<String> packagesToRelocate = [
         "com.esri",
+        "com.fasterxml",
         "com.google.common",
-        "com.google.thirdparty",
         "com.google.protobuf",
-        "com.fasterxml",
+        "com.google.thirdparty",
         "com.jayway",
         "com.yahoo",
+        "javax.annotation",
+        "net.minidev",
         "org.apache.calcite",
         "org.apache.commons",
         "org.apache.http",
+        "org.apiguardian.api",
         "org.codehaus",
+        "org.objectweb",
         "org.pentaho",
-        "org.yaml"
+        "org.yaml",
 ]
 
 vendorJava(

Review comment:
       I have copied (+ updated version) of the runtime dependencies part of this.
   As this is uncharted territory for me I'm unsure if this is correct now.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] nielsbasjes commented on a change in pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

nielsbasjes commented on a change in pull request #12962:
URL: https://github.com/apache/beam/pull/12962#discussion_r497362928



##########
File path: vendor/calcite-1_25_0/build.gradle
##########
@@ -18,29 +18,33 @@
 
 plugins { id 'org.apache.beam.vendor-java' }
 
-description = "Apache Beam :: Vendored Dependencies :: Calcite 1.20.0"
+description = "Apache Beam :: Vendored Dependencies :: Calcite 1.25.0"
 
 group = "org.apache.beam"
 version = "0.2"

Review comment:
       > Can you publish a build scan by running your second step with `--scan` ? Then we can see what is broken.
   
   https://scans.gradle.com/s/ofcfnu774du4y




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] nielsbasjes commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

nielsbasjes commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-739500944

I am in need of advice / assistance.

What I ran into is that some tests in the current state of this pull request fail over a change in Calcite.
I am specifically talking about https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/jdbc/src/test/java/org/apache/beam/sdk/extensions/sql/jdbc/BeamSqlLineTest.java#L63 and many other tests in this file that do a `CREATE EXTERNAL TABLE` or a `DROP TABLE`.

I tracked the source of this exception back to Calcite where a few months ago this default method was added in Calcite by @julianhyde : https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/parser/SqlParserImplFactory.java#L58

This default implementation simply returns a `DdlExecutor` (a new interface) that always fails with a `UnsupportedOperationException` if no valid implementation has been provided.
Looks good so far.

The problem I have is that my knowledge is lacking how to correctly override this method to return the implementation for Beam that makes it possible to do a `CREATE EXTERNAL TABLE`.

A pointer on how I should fix this correctly is highly appreciated.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) (19d11b7) into [master](https://codecov.io/gh/apache/beam/commit/8ab1955550931fad398048a5990ebea868406587?el=desc) (8ab1955) will **increase** coverage by `47.53%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master   #12962       +/-   ##
   ===========================================
   + Coverage   34.80%   82.33%   +47.53%     
   ===========================================
     Files         437      455       +18     
     Lines       55008    54647      -361     
   ===========================================
   + Hits        19144    44995    +25851     
   + Misses      35864     9652    -26212     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [conftest.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-Y29uZnRlc3QucHk=) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
   | [sdks/python/apache\_beam/utils/urns.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvdXJucy5weQ==) | `88.88% <0.00%> (-0.18%)` | :arrow_down: |
   | [setup.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
   | [sdks/python/apache\_beam/runners/internal/names.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcm5hbC9uYW1lcy5weQ==) | `100.00% <0.00%> (ø)` | |
   | [...hon/apache\_beam/runners/dataflow/internal/names.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9uYW1lcy5weQ==) | `100.00% <0.00%> (ø)` | |
   | [...beam/runners/portability/local\_job\_service\_main.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9sb2NhbF9qb2Jfc2VydmljZV9tYWluLnB5) | `0.00% <0.00%> (ø)` | |
   | [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
   | [.../internal/clients/bigquery/bigquery\_v2\_messages.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvYmlncXVlcnkvYmlncXVlcnlfdjJfbWVzc2FnZXMucHk=) | `100.00% <0.00%> (ø)` | |
   | [...les/complete/juliaset/juliaset/juliaset\_test\_it.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvanVsaWFzZXQvanVsaWFzZXQvanVsaWFzZXRfdGVzdF9pdC5weQ==) | | |
   | [...ython/apache\_beam/io/gcp/bigquery\_read\_internal.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X3JlYWRfaW50ZXJuYWwucHk=) | | |
   | ... and [351 more](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [30f9a60...76c44b5](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [BEAM-9379] Update calcite to 1.25 (WIP/INCOMPLETE)

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/cee7388365970774969e63fe90fdf3cdd804e79f?el=desc) will **decrease** coverage by `0.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #12962      +/-   ##
   ==========================================
   - Coverage   82.34%   82.33%   -0.02%     
   ==========================================
     Files         455      455              
     Lines       54650    54650              
   ==========================================
   - Hits        45002    44996       -6     
   - Misses       9648     9654       +6     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `88.68% <0.00%> (-1.23%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `90.90% <0.00%> (-0.76%)` | :arrow_down: |
   | [...nners/portability/fn\_api\_runner/worker\_handlers.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3dvcmtlcl9oYW5kbGVycy5weQ==) | `80.57% <0.00%> (-0.18%)` | :arrow_down: |
   | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.98% <0.00%> (-0.17%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `94.58% <0.00%> (+0.13%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [cee7388...f498320](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] vectorijk commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

vectorijk commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700912215


   > It might be a huge amount of work. We can just add the new vendored module and then release it, but we will want to know that we have enough people available to also port to it.
   
   agree on this. we could vendor module first and use released library for porting work.
   
   last few weeks. I did a testing on my branch https://github.com/vectorijk/beam/tree/calcite-1-25 and run
   `./gradlew -Ppublishing -PvendoredDependenciesOnly -PjavaLinkageArtifactIds=beam-vendor-calcite-1_25_0:0.1-SNAPSHOT :checkJavaLinkage`
   it still has some issues unresolved.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [BEAM-9379] Update calcite to 1.25 (WIP/INCOMPLETE)

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/cee7388365970774969e63fe90fdf3cdd804e79f?el=desc) will **decrease** coverage by `0.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #12962      +/-   ##
   ==========================================
   - Coverage   82.34%   82.33%   -0.02%     
   ==========================================
     Files         455      455              
     Lines       54650    54650              
   ==========================================
   - Hits        45002    44996       -6     
   - Misses       9648     9654       +6     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `88.68% <0.00%> (-1.23%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `90.90% <0.00%> (-0.76%)` | :arrow_down: |
   | [...nners/portability/fn\_api\_runner/worker\_handlers.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3dvcmtlcl9oYW5kbGVycy5weQ==) | `80.57% <0.00%> (-0.18%)` | :arrow_down: |
   | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.98% <0.00%> (-0.17%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `94.58% <0.00%> (+0.13%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [cee7388...f498320](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) (19d11b7) into [master](https://codecov.io/gh/apache/beam/commit/3d6cc0ed9ed537229b27b5dbe73288f21b0e351c?el=desc) (3d6cc0e) will **decrease** coverage by `0.15%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #12962      +/-   ##
   ==========================================
   - Coverage   82.48%   82.33%   -0.16%     
   ==========================================
     Files         455      455              
     Lines       54876    54647     -229     
   ==========================================
   - Hits        45266    44995     -271     
   - Misses       9610     9652      +42     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/utils/profiler.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvcHJvZmlsZXIucHk=) | `32.11% <0.00%> (-54.35%)` | :arrow_down: |
   | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `83.33% <0.00%> (-9.53%)` | :arrow_down: |
   | [...ython/apache\_beam/io/gcp/tests/bigquery\_matcher.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL3Rlc3RzL2JpZ3F1ZXJ5X21hdGNoZXIucHk=) | `79.83% <0.00%> (-2.87%)` | :arrow_down: |
   | [conftest.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-Y29uZnRlc3QucHk=) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
   | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `88.68% <0.00%> (-0.84%)` | :arrow_down: |
   | [sdks/python/apache\_beam/utils/retry.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvcmV0cnkucHk=) | `86.27% <0.00%> (-0.77%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `90.90% <0.00%> (-0.76%)` | :arrow_down: |
   | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `90.77% <0.00%> (-0.53%)` | :arrow_down: |
   | [sdks/python/apache\_beam/metrics/metricbase.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWV0cmljcy9tZXRyaWNiYXNlLnB5) | `87.87% <0.00%> (-0.36%)` | :arrow_down: |
   | [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `89.45% <0.00%> (-0.36%)` | :arrow_down: |
   | ... and [36 more](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [372a4f3...8f79ca3](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) (19d11b7) into [master](https://codecov.io/gh/apache/beam/commit/3d6cc0ed9ed537229b27b5dbe73288f21b0e351c?el=desc) (3d6cc0e) will **decrease** coverage by `0.15%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #12962      +/-   ##
   ==========================================
   - Coverage   82.48%   82.33%   -0.16%     
   ==========================================
     Files         455      455              
     Lines       54876    54647     -229     
   ==========================================
   - Hits        45266    44995     -271     
   - Misses       9610     9652      +42     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [sdks/python/apache\_beam/utils/profiler.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvcHJvZmlsZXIucHk=) | `32.11% <0.00%> (-54.35%)` | :arrow_down: |
   | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `83.33% <0.00%> (-9.53%)` | :arrow_down: |
   | [...ython/apache\_beam/io/gcp/tests/bigquery\_matcher.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL3Rlc3RzL2JpZ3F1ZXJ5X21hdGNoZXIucHk=) | `79.83% <0.00%> (-2.87%)` | :arrow_down: |
   | [conftest.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-Y29uZnRlc3QucHk=) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
   | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `88.68% <0.00%> (-0.84%)` | :arrow_down: |
   | [sdks/python/apache\_beam/utils/retry.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvcmV0cnkucHk=) | `86.27% <0.00%> (-0.77%)` | :arrow_down: |
   | [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `90.90% <0.00%> (-0.76%)` | :arrow_down: |
   | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `90.77% <0.00%> (-0.53%)` | :arrow_down: |
   | [sdks/python/apache\_beam/metrics/metricbase.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWV0cmljcy9tZXRyaWNiYXNlLnB5) | `87.87% <0.00%> (-0.36%)` | :arrow_down: |
   | [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `89.45% <0.00%> (-0.36%)` | :arrow_down: |
   | ... and [36 more](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [45ce942...38e8111](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kennknowles commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.25

Posted by GitBox <gi...@apache.org>.

kennknowles commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700894541


   It might be a huge amount of work. We can just add the new vendored module and then release it, but we will want to know that we have enough people available to also port to it.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] nielsbasjes commented on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

nielsbasjes commented on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-748063151


   Yes please. 
   I have tried over the last week and the main problems that remain are beyond me.
   I'm stuck and unable to proceed.
   At this point I do not have the understanding on how Calcite and Beam (should) work together to fix this.
   So I'm totally fine if someone takes over from here.
   
   At this point I see these `big` categories of tests that fail:
   
   1. Tests that try to create a table fail with `DDL not supported`.
   2. Tests that have nested records fail with class cast exceptions in the `SchemaCoder::encode`. 
       To me it seems an error exists in  handling the nesting of schemas.
   3. The `BeamCalcRel::castOutputTime` tries to call `java.time.LocalDate.ofEpochDay(long)` yet the underlying Calcite code goes searching for `java.time.LocalDate.ofEpochDay(java.lang.Integer)` which doesn't exist. 
       To me this seems an internal change of calcite how they choose data types.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #12962: [WIP][BEAM-9379] Update calcite to 1.26

Posted by GitBox <gi...@apache.org>.

codecov[bot] edited a comment on pull request #12962:
URL: https://github.com/apache/beam/pull/12962#issuecomment-700648248


   # [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=h1) Report
   > Merging [#12962](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=desc) (19d11b7) into [master](https://codecov.io/gh/apache/beam/commit/8ab1955550931fad398048a5990ebea868406587?el=desc) (8ab1955) will **increase** coverage by `47.53%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/12962/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master   #12962       +/-   ##
   ===========================================
   + Coverage   34.80%   82.33%   +47.53%     
   ===========================================
     Files         437      455       +18     
     Lines       55008    54647      -361     
   ===========================================
   + Hits        19144    44995    +25851     
   + Misses      35864     9652    -26212     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [conftest.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-Y29uZnRlc3QucHk=) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
   | [sdks/python/apache\_beam/utils/urns.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvdXJucy5weQ==) | `88.88% <0.00%> (-0.18%)` | :arrow_down: |
   | [setup.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
   | [sdks/python/apache\_beam/runners/internal/names.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcm5hbC9uYW1lcy5weQ==) | `100.00% <0.00%> (ø)` | |
   | [...hon/apache\_beam/runners/dataflow/internal/names.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9uYW1lcy5weQ==) | `100.00% <0.00%> (ø)` | |
   | [...beam/runners/portability/local\_job\_service\_main.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9sb2NhbF9qb2Jfc2VydmljZV9tYWluLnB5) | `0.00% <0.00%> (ø)` | |
   | [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
   | [.../internal/clients/bigquery/bigquery\_v2\_messages.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvYmlncXVlcnkvYmlncXVlcnlfdjJfbWVzc2FnZXMucHk=) | `100.00% <0.00%> (ø)` | |
   | [.../python/apache\_beam/io/gcp/resource\_identifiers.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL3Jlc291cmNlX2lkZW50aWZpZXJzLnB5) | | |
   | [...ks/python/apache\_beam/internal/metrics/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9fX2luaXRfXy5weQ==) | | |
   | ... and [351 more](https://codecov.io/gh/apache/beam/pull/12962/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=footer). Last update [30f9a60...8104136](https://codecov.io/gh/apache/beam/pull/12962?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org