You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2021/03/10 02:55:44 UTC
[GitHub] [flink] KevinyhZou opened a new pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
KevinyhZou opened a new pull request #15068:
URL: https://github.com/apache/flink/pull/15068
## What is the purpose of the change
Bug fix for array out of bounds exception while running a hive streaming job with partitioned table source, the partition feilds is not found in the fields provided by the context(HiveContinuousPartitionFetcherContext) , so we add the field names and types to it.
## Brief change log
Get the partiton field name and types from catalog base table, and put them into the context (HiveContinuousPartitionFetcherContext) while to get hive partitions.
## Verifying this change
This change added tests and can be verified as follows:
- Change HiveTableSourceITCase#testStreamPartitionReadByPartitionName to use it as a test
- Tested manually by running a flink streaming job to hive partitioned table source
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on a change in pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on a change in pull request #15068:
URL: https://github.com/apache/flink/pull/15068#discussion_r588139880
##########
File path: flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/connectors/hive/HiveTableSourceITCase.java
##########
@@ -572,7 +572,8 @@ public void testStreamPartitionReadByPartitionName() throws Exception {
.commit("pt_year='2020',pt_mon='09',pt_day='03'");
TableResult result =
- tEnv.executeSql("select * from hive.source_db.stream_partition_name_test");
+ tEnv.executeSql(
+ "select * from hive.source_db.stream_partition_name_test where pt_year = '2020'");
Review comment:
Hi, the test does not pass when add a `where` condition, it got a `TestTimedOutException` , and it pass with the patch and no exception throws. So should we set the test sql like `select x,y from .... where ....` ? @leonardBang
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-795025084
> @KevinyhZou The PR generally LGTM, could you fix my last minor comment and rebase to latest master?
hello, I have rebased to latest master and create a new test case `testStreamReadWithProjectPushDown` in `HiveTableSourceITCase`. please review the code. @leonardBang
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789459239
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "624bf85a6820617768b5a7a20032e2001641cd27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031",
"triggerID" : "624bf85a6820617768b5a7a20032e2001641cd27",
"triggerType" : "PUSH"
}, {
"hash" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14174",
"triggerID" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"triggerType" : "PUSH"
}, {
"hash" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14197",
"triggerID" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"triggerType" : "PUSH"
}, {
"hash" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14354",
"triggerID" : "793981283",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 2a67bce07c78b6fc8a86184dfd5a7c499afffd7d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14197) Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14354)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-794778900
> @KevinyhZou The PR generally LGTM, could you fix my last minor comment and rebase to latest master?
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789459239
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "624bf85a6820617768b5a7a20032e2001641cd27",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031",
"triggerID" : "624bf85a6820617768b5a7a20032e2001641cd27",
"triggerType" : "PUSH"
}, {
"hash" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14174",
"triggerID" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 624bf85a6820617768b5a7a20032e2001641cd27 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031)
* 6684fedae17839fbf78ea0850b76cf0c4161714a Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14174)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] leonardBang commented on a change in pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
leonardBang commented on a change in pull request #15068:
URL: https://github.com/apache/flink/pull/15068#discussion_r588122708
##########
File path: flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/connectors/hive/HiveTableSourceITCase.java
##########
@@ -572,7 +572,8 @@ public void testStreamPartitionReadByPartitionName() throws Exception {
.commit("pt_year='2020',pt_mon='09',pt_day='03'");
TableResult result =
- tEnv.executeSql("select * from hive.source_db.stream_partition_name_test");
+ tEnv.executeSql(
+ "select * from hive.source_db.stream_partition_name_test where pt_year = '2020'");
Review comment:
I think this test can pass without this patch, we should change to `select x, y from...` thus the selected fields push down.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789459239
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "624bf85a6820617768b5a7a20032e2001641cd27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031",
"triggerID" : "624bf85a6820617768b5a7a20032e2001641cd27",
"triggerType" : "PUSH"
}, {
"hash" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14174",
"triggerID" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"triggerType" : "PUSH"
}, {
"hash" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14197",
"triggerID" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"triggerType" : "PUSH"
}, {
"hash" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14354",
"triggerID" : "793981283",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 2a67bce07c78b6fc8a86184dfd5a7c499afffd7d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14197) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14354)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-791364085
hello, the test sql has changed to `select x, y from ... where ...`, please review the code @leonardBang thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-794778900
> @KevinyhZou The PR generally LGTM, could you fix my last minor comment and rebase to latest master?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789459239
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "624bf85a6820617768b5a7a20032e2001641cd27",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031",
"triggerID" : "624bf85a6820617768b5a7a20032e2001641cd27",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 624bf85a6820617768b5a7a20032e2001641cd27 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789459239
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "624bf85a6820617768b5a7a20032e2001641cd27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031",
"triggerID" : "624bf85a6820617768b5a7a20032e2001641cd27",
"triggerType" : "PUSH"
}, {
"hash" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14174",
"triggerID" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"triggerType" : "PUSH"
}, {
"hash" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14197",
"triggerID" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"triggerType" : "PUSH"
}, {
"hash" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14354",
"triggerID" : "793981283",
"triggerType" : "MANUAL"
}, {
"hash" : "c108e647882bfd48bfb21edad2aab8cdc4b020a1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c108e647882bfd48bfb21edad2aab8cdc4b020a1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2a67bce07c78b6fc8a86184dfd5a7c499afffd7d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14197) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14354)
* c108e647882bfd48bfb21edad2aab8cdc4b020a1 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] leonardBang commented on a change in pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
leonardBang commented on a change in pull request #15068:
URL: https://github.com/apache/flink/pull/15068#discussion_r588122708
##########
File path: flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/connectors/hive/HiveTableSourceITCase.java
##########
@@ -572,7 +572,8 @@ public void testStreamPartitionReadByPartitionName() throws Exception {
.commit("pt_year='2020',pt_mon='09',pt_day='03'");
TableResult result =
- tEnv.executeSql("select * from hive.source_db.stream_partition_name_test");
+ tEnv.executeSql(
+ "select * from hive.source_db.stream_partition_name_test where pt_year = '2020'");
Review comment:
I think this test can pass without this patch, we should change to `select x, y from...` thus the selected fields can be pushed down.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou removed a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou removed a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-794782166
OK,
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789459239
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "624bf85a6820617768b5a7a20032e2001641cd27",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "624bf85a6820617768b5a7a20032e2001641cd27",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 624bf85a6820617768b5a7a20032e2001641cd27 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-791364085
hello, the test sql has been changed to `select x, y from ... where ...`, please review the code @leonardBang thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on a change in pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on a change in pull request #15068:
URL: https://github.com/apache/flink/pull/15068#discussion_r590975723
##########
File path: flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/connectors/hive/HiveTableSourceITCase.java
##########
@@ -591,8 +591,7 @@ public void testStreamPartitionReadByPartitionName() throws Exception {
Assert.assertEquals(
Arrays.asList(
- Row.of(i, "new_add", 11 + i, "2020", "10", "0" + i).toString(),
- Row.of(i, "new_add_1", 11 + i, "2020", "10", "0" + i).toString()),
+ Row.of(i, "new_add").toString(), Row.of(i, "new_add_1").toString()),
Review comment:
Hello, it not missed. As we have changed to `select x,y... where ..` , So we only assertEquals the column x, y. The `Row.of(i, "new_add").toString(), Row.of(i, "new_add_1").toString()),` is 2 line.
##########
File path: flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/connectors/hive/HiveTableSourceITCase.java
##########
@@ -591,8 +591,7 @@ public void testStreamPartitionReadByPartitionName() throws Exception {
Assert.assertEquals(
Arrays.asList(
- Row.of(i, "new_add", 11 + i, "2020", "10", "0" + i).toString(),
- Row.of(i, "new_add_1", 11 + i, "2020", "10", "0" + i).toString()),
+ Row.of(i, "new_add").toString(), Row.of(i, "new_add_1").toString()),
Review comment:
Hello, it not missed. As we have changed to `select x,y... where ..` , So we only assertEquals the column x, y. The `Row.of(i, "new_add").toString(), Row.of(i, "new_add_1").toString())` is 2 line.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] lirui-apache commented on a change in pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
lirui-apache commented on a change in pull request #15068:
URL: https://github.com/apache/flink/pull/15068#discussion_r590942798
##########
File path: flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/connectors/hive/HiveTableSourceITCase.java
##########
@@ -572,7 +572,8 @@ public void testStreamPartitionReadByPartitionName() throws Exception {
.commit("pt_year='2020',pt_mon='09',pt_day='03'");
TableResult result =
- tEnv.executeSql("select * from hive.source_db.stream_partition_name_test");
+ tEnv.executeSql(
+ "select * from hive.source_db.stream_partition_name_test where pt_year = '2020'");
Review comment:
I see. Since we filter partition by a literal, the partition value doesn't need to be selected from the source. Thanks for the explanations.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] lirui-apache commented on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
lirui-apache commented on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-794790329
Instead of changing a test, I think it's better to add a new test case for project pushdown.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wuchong merged pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
wuchong merged pull request #15068:
URL: https://github.com/apache/flink/pull/15068
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou closed pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou closed pull request #15068:
URL: https://github.com/apache/flink/pull/15068
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-794782166
OK,
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789459239
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "624bf85a6820617768b5a7a20032e2001641cd27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031",
"triggerID" : "624bf85a6820617768b5a7a20032e2001641cd27",
"triggerType" : "PUSH"
}, {
"hash" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14174",
"triggerID" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"triggerType" : "PUSH"
}, {
"hash" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14197",
"triggerID" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"triggerType" : "PUSH"
}, {
"hash" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14354",
"triggerID" : "793981283",
"triggerType" : "MANUAL"
}, {
"hash" : "c108e647882bfd48bfb21edad2aab8cdc4b020a1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14382",
"triggerID" : "c108e647882bfd48bfb21edad2aab8cdc4b020a1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c108e647882bfd48bfb21edad2aab8cdc4b020a1 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14382)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789459239
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "624bf85a6820617768b5a7a20032e2001641cd27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031",
"triggerID" : "624bf85a6820617768b5a7a20032e2001641cd27",
"triggerType" : "PUSH"
}, {
"hash" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14174",
"triggerID" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"triggerType" : "PUSH"
}, {
"hash" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14197",
"triggerID" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2a67bce07c78b6fc8a86184dfd5a7c499afffd7d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14197)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789459239
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "624bf85a6820617768b5a7a20032e2001641cd27",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031",
"triggerID" : "624bf85a6820617768b5a7a20032e2001641cd27",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 624bf85a6820617768b5a7a20032e2001641cd27 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15068: [FLINK-21523][Connectors / Hive] ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789449204
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 624bf85a6820617768b5a7a20032e2001641cd27 (Wed Mar 03 05:40:18 UTC 2021)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
* **Invalid pull request title: No valid Jira ID provided**
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-793981283
@flinkbot
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-794782166
Hello, the PR was closed by accident. can it be open again or I
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789459239
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "624bf85a6820617768b5a7a20032e2001641cd27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031",
"triggerID" : "624bf85a6820617768b5a7a20032e2001641cd27",
"triggerType" : "PUSH"
}, {
"hash" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14174",
"triggerID" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"triggerType" : "PUSH"
}, {
"hash" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14197",
"triggerID" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6684fedae17839fbf78ea0850b76cf0c4161714a Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14174)
* 2a67bce07c78b6fc8a86184dfd5a7c499afffd7d Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14197)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789459239
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "624bf85a6820617768b5a7a20032e2001641cd27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031",
"triggerID" : "624bf85a6820617768b5a7a20032e2001641cd27",
"triggerType" : "PUSH"
}, {
"hash" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14174",
"triggerID" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"triggerType" : "PUSH"
}, {
"hash" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6684fedae17839fbf78ea0850b76cf0c4161714a Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14174)
* 2a67bce07c78b6fc8a86184dfd5a7c499afffd7d UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-791364085
hello, the test sql has benn changed to `select x, y from ... where ...`, please review the code @leonardBang thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-794809557
> Instead of changing a test, I think it's better to add a new test case for project pushdown.
shoud we open a new test case for project pushdown. @leonardBang
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789459239
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "624bf85a6820617768b5a7a20032e2001641cd27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031",
"triggerID" : "624bf85a6820617768b5a7a20032e2001641cd27",
"triggerType" : "PUSH"
}, {
"hash" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14174",
"triggerID" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"triggerType" : "PUSH"
}, {
"hash" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14197",
"triggerID" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"triggerType" : "PUSH"
}, {
"hash" : "2a67bce07c78b6fc8a86184dfd5a7c499afffd7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14354",
"triggerID" : "793981283",
"triggerType" : "MANUAL"
}, {
"hash" : "c108e647882bfd48bfb21edad2aab8cdc4b020a1",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14382",
"triggerID" : "c108e647882bfd48bfb21edad2aab8cdc4b020a1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2a67bce07c78b6fc8a86184dfd5a7c499afffd7d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14197) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14354)
* c108e647882bfd48bfb21edad2aab8cdc4b020a1 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14382)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789459239
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "624bf85a6820617768b5a7a20032e2001641cd27",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031",
"triggerID" : "624bf85a6820617768b5a7a20032e2001641cd27",
"triggerType" : "PUSH"
}, {
"hash" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 624bf85a6820617768b5a7a20032e2001641cd27 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031)
* 6684fedae17839fbf78ea0850b76cf0c4161714a UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-789459239
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "624bf85a6820617768b5a7a20032e2001641cd27",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14031",
"triggerID" : "624bf85a6820617768b5a7a20032e2001641cd27",
"triggerType" : "PUSH"
}, {
"hash" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14174",
"triggerID" : "6684fedae17839fbf78ea0850b76cf0c4161714a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6684fedae17839fbf78ea0850b76cf0c4161714a Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14174)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-794826274
> > > Instead of changing a test, I think it's better to add a new test case for project pushdown.
> >
> >
> > shoud we open a new test case for project pushdown. @leonardBang
>
> I agree @lirui-apache that we'd better add a test `testStreamReadWithProjectPushDown`
OK, I will add a new test case
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on a change in pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on a change in pull request #15068:
URL: https://github.com/apache/flink/pull/15068#discussion_r590370037
##########
File path: flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/connectors/hive/HiveTableSourceITCase.java
##########
@@ -572,7 +572,8 @@ public void testStreamPartitionReadByPartitionName() throws Exception {
.commit("pt_year='2020',pt_mon='09',pt_day='03'");
TableResult result =
- tEnv.executeSql("select * from hive.source_db.stream_partition_name_test");
+ tEnv.executeSql(
+ "select * from hive.source_db.stream_partition_name_test where pt_year = '2020'");
Review comment:
From the debug log, it is the same problem: partition key field not found. the projected fields do not include the partition column both in `select x, y from ..` and `selecrt * from .. where` .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] lirui-apache commented on a change in pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
lirui-apache commented on a change in pull request #15068:
URL: https://github.com/apache/flink/pull/15068#discussion_r589362359
##########
File path: flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/connectors/hive/HiveTableSourceITCase.java
##########
@@ -572,7 +572,8 @@ public void testStreamPartitionReadByPartitionName() throws Exception {
.commit("pt_year='2020',pt_mon='09',pt_day='03'");
TableResult result =
- tEnv.executeSql("select * from hive.source_db.stream_partition_name_test");
+ tEnv.executeSql(
+ "select * from hive.source_db.stream_partition_name_test where pt_year = '2020'");
Review comment:
> the test does not pass when add a where condition, it got a TestTimedOutException
Do we know why is that? It sounds like another hidden issue to me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-790922292
ok, I will fix it by change getProducedTableSchema() to getTableSchema() and provide the test case . Thanks for your suggestion @leonardBang
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on a change in pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on a change in pull request #15068:
URL: https://github.com/apache/flink/pull/15068#discussion_r588139880
##########
File path: flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/connectors/hive/HiveTableSourceITCase.java
##########
@@ -572,7 +572,8 @@ public void testStreamPartitionReadByPartitionName() throws Exception {
.commit("pt_year='2020',pt_mon='09',pt_day='03'");
TableResult result =
- tEnv.executeSql("select * from hive.source_db.stream_partition_name_test");
+ tEnv.executeSql(
+ "select * from hive.source_db.stream_partition_name_test where pt_year = '2020'");
Review comment:
Hi, the test does not pass when add a `where` condition, it got a `TestTimedOutException` , and it pass with the patch and no exception throws. So would we set the test sql like `select x,y from .... where ....` ? @leonardBang
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-797311986
> Thanks for the nice work @KevinyhZou , LGTM
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] leonardBang commented on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
leonardBang commented on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-794823398
> > Instead of changing a test, I think it's better to add a new test case for project pushdown.
>
> shoud we open a new test case for project pushdown. @leonardBang
I agree @lirui-apache that we'd better add a test `testStreamReadWithProjectPushDown`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-794826274
> > > Instead of changing a test, I think it's better to add a new test case for project pushdown.
> >
> >
> > shoud we open a new test case for project pushdown. @leonardBang
>
> I agree @lirui-apache that we'd better add a test `testStreamReadWithProjectPushDown`
OK, I will and a new test case
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] leonardBang commented on a change in pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
leonardBang commented on a change in pull request #15068:
URL: https://github.com/apache/flink/pull/15068#discussion_r590902615
##########
File path: flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/connectors/hive/HiveTableSourceITCase.java
##########
@@ -591,8 +591,7 @@ public void testStreamPartitionReadByPartitionName() throws Exception {
Assert.assertEquals(
Arrays.asList(
- Row.of(i, "new_add", 11 + i, "2020", "10", "0" + i).toString(),
- Row.of(i, "new_add_1", 11 + i, "2020", "10", "0" + i).toString()),
+ Row.of(i, "new_add").toString(), Row.of(i, "new_add_1").toString()),
Review comment:
Looks like the we missed one line here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-791240766
Hi, I have use `getTableSchema()` to pass full field names/types while construct HivePartitionFetcherContextBase, and also change `HiveTableSourceITCase#testStreamPartitionReadByPartitionName` as a test . please review the code, thanks. @leonardBang
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou edited a comment on pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou edited a comment on pull request #15068:
URL: https://github.com/apache/flink/pull/15068#issuecomment-793981283
@flinkbot run azure
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KevinyhZou commented on a change in pull request #15068: [FLINK-21523][Connectors / Hive] Bug fix: ArrayIndexOutOfBoundsException occurs while run a hive strea…
Posted by GitBox <gi...@apache.org>.
KevinyhZou commented on a change in pull request #15068:
URL: https://github.com/apache/flink/pull/15068#discussion_r590975723
##########
File path: flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/connectors/hive/HiveTableSourceITCase.java
##########
@@ -591,8 +591,7 @@ public void testStreamPartitionReadByPartitionName() throws Exception {
Assert.assertEquals(
Arrays.asList(
- Row.of(i, "new_add", 11 + i, "2020", "10", "0" + i).toString(),
- Row.of(i, "new_add_1", 11 + i, "2020", "10", "0" + i).toString()),
+ Row.of(i, "new_add").toString(), Row.of(i, "new_add_1").toString()),
Review comment:
Hello, it not missed. As we have changed to `select x,y... where ..` , So we only assertEquals the column x, y. The `Row.of(i, "new_add").toString(), Row.of(i, "new_add_1").toString())` is two Row which is the same as before .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org