You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/06/01 03:38:00 UTC
[GitHub] [iceberg] chixiaopao opened a new issue, #4924: flink sql insert iceberg table array, then select the table array values is same.
chixiaopao opened a new issue, #4924:
URL: https://github.com/apache/iceberg/issues/4924
flink version: 1.14.4
iceberg version: 0.13.1
idea test use hadoop catalog.
example：
```
env.executeSql(
s"""
|CREATE CATALOG `$catalog` WITH (
| 'type'='iceberg',
| 'catalog-type'='hadoop',
| 'warehouse'='$basePath',
| 'property-version'='1'
|)
|""".stripMargin
)
env.useCatalog(catalog)
env.useDatabase(database)
```
create table :
```
env.executeSql(
s"""
|CREATE TABLE IF NOT EXISTS `$catalog`.`$database`.`ods__cs__session_text_slice_d_i`
|(
| `dateId` STRING COMMENT '分区时间',
| `tagGcode` ARRAY<STRING> COMMENT '标签gcode列表',
| `windowsTagIds` ARRAY<STRING> COMMENT '窗口标签ID列表',
| `windowTag` STRING COMMENT '窗口标签',
| `create_time` TIMESTAMP COMMENT '数据创建时间'
|) PARTITIONED BY (dateId)
|WITH (
| 'format-version' = '2',
| 'write.upsert.enabled' = 'false',
| 'write.distribution-mode' = 'hash',
| 'write.metadata.delete-after-commit.enabled' = 'true',
| 'write.parquet.row-group-size-bytes' = '134217728',
| 'write.target-file-size-bytes' = '536870912',
| 'history.expire.max-snapshot-age-ms' = '360000',
| 'write.metadata.previous-versions-max' = '5',
| 'write.parallelism' = '1'
|)
|""".stripMargin
)
```
insert data:
```
env.executeSql(
s"""
|INSERT INTO `$catalog`.`$database`.`ods__cs__session_text_slice_d_i` VALUES(
|'2022-05-31',
|Array['gcode'],
|Array['window_tag_id'],
|'window_tag',
|to_timestamp('2022-05-31 18:00:00')
|)
|""".stripMargin
)
```
select:
```
env.executeSql(
s"""
|INSERT INTO `$catalog`.`$database`.`ods__cs__session_text_slice_d_i` VALUES(
|'2022-05-31',
|Array['gcode'],
|Array['window_tag_id'],
|'window_tag',
|to_timestamp('2022-05-31 18:00:00')
|)
|""".stripMargin
)
```
result:
```
+--------------------------------+--------------------------------+
| tagGcode | windowsTagIds |
+--------------------------------+--------------------------------+
| [gcode] | [gcode] |
+--------------------------------+--------------------------------+
```
why? is this bug?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] kbendick commented on issue #4924: flink sql insert iceberg table array, then select the table array values is same.
Posted by GitBox <gi...@apache.org>.
kbendick commented on issue #4924:
URL: https://github.com/apache/iceberg/issues/4924#issuecomment-1144316555
Oh I understand now. That’s my fault I didn’t see the result had the same value.
Can you share either a screenshot of the generated DAG for the insert (and for the read) or attach the output of running EXPLAIN? Explain is documented here: https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/dev/table/sql/explain/#explain-statements
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] kbendick commented on issue #4924: flink sql insert iceberg table array, then select the table array values is same.
Posted by GitBox <gi...@apache.org>.
kbendick commented on issue #4924:
URL: https://github.com/apache/iceberg/issues/4924#issuecomment-1145178256
No worries! Thanks for closing the issue 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] chixiaopao commented on issue #4924: flink sql insert iceberg table array, then select the table array values is same.
Posted by GitBox <gi...@apache.org>.
chixiaopao commented on issue #4924:
URL: https://github.com/apache/iceberg/issues/4924#issuecomment-1144314603
> I'm not sure I fully understand your question. The type is declared as `ARRAY<STRING>` for both of those fields, and an array value was inserted. So that's what's being returned.
>
> What behavior are you expecting?
im so sorry about my pool english, i try to express it once more.
I have created a table named `test`, It contains two columns of type array named `tagGcode` `windowsTagIds`,then insert the value(Array['gcode'],Array['window_tag_id']),when I query this table I expect the following result:
+--------------------------------+--------------------------------+
| tagGcode | windowsTagIds |
+--------------------------------+--------------------------------+
| [gcode] | [window_tag_id] |
+--------------------------------+--------------------------------+
but the actual result：
+--------------------------------+--------------------------------+
| tagGcode | windowsTagIds |
+--------------------------------+--------------------------------+
| [gcode] | [gcode] |
+--------------------------------+--------------------------------+
I'm not sure if it's flink's bug or iceberg's bug.
looking forward to your answer thank you very much!@kbendick 😊
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] chixiaopao commented on issue #4924: flink sql insert iceberg table array, then select the table array values is same.
Posted by GitBox <gi...@apache.org>.
chixiaopao commented on issue #4924:
URL: https://github.com/apache/iceberg/issues/4924#issuecomment-1144880889
> Also what happens when you simply select *?
I'm sorry I can't reproduce this problem, it's most likely caused by the version of flink we maintain internally. so I will close this issue. Thank you very much for your generous answer!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] kbendick commented on issue #4924: flink sql insert iceberg table array, then select the table array values is same.
Posted by GitBox <gi...@apache.org>.
kbendick commented on issue #4924:
URL: https://github.com/apache/iceberg/issues/4924#issuecomment-1144046800
I'm not sure I fully understand your question. The type is declared as `ARRAY<STRING>` for both of those fields, and an array value was inserted. So that's what's being returned.
What behavior are you expecting?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] xiaows08 commented on issue #4924: flink sql insert iceberg table array, then select the table array values is same.
Posted by GitBox <gi...@apache.org>.
xiaows08 commented on issue #4924:
URL: https://github.com/apache/iceberg/issues/4924#issuecomment-1143171784
I have the same problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] kbendick commented on issue #4924: flink sql insert iceberg table array, then select the table array values is same.
Posted by GitBox <gi...@apache.org>.
kbendick commented on issue #4924:
URL: https://github.com/apache/iceberg/issues/4924#issuecomment-1144316756
Also what happens when you simply select *?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] chixiaopao closed issue #4924: flink sql insert iceberg table array, then select the table array values is same.
Posted by GitBox <gi...@apache.org>.
chixiaopao closed issue #4924: flink sql insert iceberg table array, then select the table array values is same.
URL: https://github.com/apache/iceberg/issues/4924
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org