You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "felixYyu (via GitHub)" <gi...@apache.org> on 2023/05/02 15:10:34 UTC
[GitHub] [iceberg] felixYyu opened a new issue, #5069: Support create table `PRIMARY KEY` column via Spark sql?
felixYyu opened a new issue, #5069:
URL: https://github.com/apache/iceberg/issues/5069
Flink Sql could create PK table, but Spark sql could't. There will be duplicate data when synchronizing data through spark.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] cbrianbet commented on issue #5069: Support create table `PRIMARY KEY` column via Spark sql?
Posted by "cbrianbet (via GitHub)" <gi...@apache.org>.
cbrianbet commented on issue #5069:
URL: https://github.com/apache/iceberg/issues/5069#issuecomment-1727845133
Hello
Has anyone been able to find a solution to this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] kris-sea commented on issue #5069: Support create table `PRIMARY KEY` column via Spark sql?
Posted by GitBox <gi...@apache.org>.
kris-sea commented on issue #5069:
URL: https://github.com/apache/iceberg/issues/5069#issuecomment-1215041789
I guess @felixYyu is trying to create an iceberg table by executing flink SQL `PRIMARY KEY(col) NOT ENFORCED`, and trying to add few records into the iceberg table by spark SQL, right ?
In this case, the current `PRIMARY KEY` definitions from flink sql just add the equality keys into the iceberg table metadata. And for iceberg table's equality key fields, the engines won't enforce the uniqueness. that's why we don't see the uniqueness rows after we inserted few records by spark SQL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] commented on issue #5069: Support create table `PRIMARY KEY` column via Spark sql?
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #5069:
URL: https://github.com/apache/iceberg/issues/5069#issuecomment-1471014292
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] closed issue #5069: Support create table `PRIMARY KEY` column via Spark sql?
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #5069: Support create table `PRIMARY KEY` column via Spark sql?
URL: https://github.com/apache/iceberg/issues/5069
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on issue #5069: Support create table `PRIMARY KEY` column via Spark sql?
Posted by GitBox <gi...@apache.org>.
rdblue commented on issue #5069:
URL: https://github.com/apache/iceberg/issues/5069#issuecomment-1163837120
@felixYyu, is this valid Spark syntax? If not, I don't see how Iceberg would add it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] xuzhiwen1255 commented on issue #5069: Support create table `PRIMARY KEY` column via Spark sql?
Posted by GitBox <gi...@apache.org>.
xuzhiwen1255 commented on issue #5069:
URL: https://github.com/apache/iceberg/issues/5069#issuecomment-1214918111
@rdblue Maybe we can add it through options in Spark SQL
Such as this
{
The create table t1 (...). The options (pk = 'col)
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] felixYyu commented on issue #5069: Support create table primitive key column via Spark sql?
Posted by GitBox <gi...@apache.org>.
felixYyu commented on issue #5069:
URL: https://github.com/apache/iceberg/issues/5069#issuecomment-1161197507
cc: @rdblue @RussellSpitzer @kbendick I think should be added primitive key to column via Spark sql?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support create table `PRIMARY KEY` column via Spark sql? [iceberg]
Posted by "W-I-D-EE (via GitHub)" <gi...@apache.org>.
W-I-D-EE commented on issue #5069:
URL: https://github.com/apache/iceberg/issues/5069#issuecomment-1758634548
No but i would love to hear one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jhchee commented on issue #5069: Support create table `PRIMARY KEY` column via Spark sql?
Posted by "jhchee (via GitHub)" <gi...@apache.org>.
jhchee commented on issue #5069:
URL: https://github.com/apache/iceberg/issues/5069#issuecomment-1531619984
This ticket shouldn't be closed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] commented on issue #5069: Support create table `PRIMARY KEY` column via Spark sql?
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #5069:
URL: https://github.com/apache/iceberg/issues/5069#issuecomment-1499780608
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] felixYyu commented on issue #5069: Support create table `PRIMARY KEY` column via Spark sql?
Posted by GitBox <gi...@apache.org>.
felixYyu commented on issue #5069:
URL: https://github.com/apache/iceberg/issues/5069#issuecomment-1163858387
@rdblue, It is not valid Spark syntax, However, there is no way to build a primary key if you create a table using spark syntax.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] waywtdcc commented on issue #5069: Support create table `PRIMARY KEY` column via Spark sql?
Posted by "waywtdcc (via GitHub)" <gi...@apache.org>.
waywtdcc commented on issue #5069:
URL: https://github.com/apache/iceberg/issues/5069#issuecomment-1707704470
Hello, can sparksql ddl define primary key now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org