You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "MonsterChenzhuo (via GitHub)" <gi...@apache.org> on 2023/11/07 10:07:05 UTC
[PR] [WIP] [flink] support longtext data type mapping bytes [incubator-paimon]
MonsterChenzhuo opened a new pull request, #2281:
URL: https://github.com/apache/incubator-paimon/pull/2281
<!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
### Purpose
support longtext data type mapping bytes
<!-- What is the purpose of the change -->
### Tests
<!-- List UT and IT cases to verify this change -->
### API and Format
<!-- Does this change affect API or storage format -->
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [flink] support longtext data type mapping bytes [incubator-paimon]
Posted by "yuzelin (via GitHub)" <gi...@apache.org>.
yuzelin merged PR #2281:
URL: https://github.com/apache/incubator-paimon/pull/2281
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [flink] support longtext data type mapping bytes [incubator-paimon]
Posted by "MonsterChenzhuo (via GitHub)" <gi...@apache.org>.
MonsterChenzhuo commented on code in PR #2281:
URL: https://github.com/apache/incubator-paimon/pull/2281#discussion_r1410512893
##########
paimon-flink/paimon-flink-cdc/src/main/java/org/apache/paimon/flink/action/cdc/mysql/MySqlTypeUtils.java:
##########
@@ -286,6 +286,10 @@ public static DataType toDataType(
return DataTypes.STRING();
// MySQL BINARY and VARBINARY are stored as bytes in JSON. We convert them to
// DataTypes.VARBINARY to retain the length information
Review Comment:
done.
##########
paimon-flink/paimon-flink-cdc/src/main/java/org/apache/paimon/flink/action/cdc/TypeMapping.java:
##########
@@ -70,6 +70,7 @@ public enum TypeMappingMode {
TINYINT1_NOT_BOOL,
TO_NULLABLE,
TO_STRING,
+ LONGTEXT_TO_BYTES,
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] [flink] support longtext data type mapping bytes [incubator-paimon]
Posted by "MonsterChenzhuo (via GitHub)" <gi...@apache.org>.
MonsterChenzhuo commented on code in PR #2281:
URL: https://github.com/apache/incubator-paimon/pull/2281#discussion_r1384825959
##########
paimon-flink/paimon-flink-cdc/src/main/java/org/apache/paimon/flink/action/cdc/mysql/MySqlTypeUtils.java:
##########
@@ -262,6 +263,9 @@ public static DataType toDataType(
case TEXT:
case MEDIUMTEXT:
case LONGTEXT:
+ return length == null || length == 0 || typeMapping.containsMode(LONGTEXT_TO_BYTES)
+ ? DataTypes.VARCHAR(length)
+ : DataTypes.STRING();
Review Comment:
There's an issue with my VPN, so I won't be able to submit the code today.ðŸ˜
##########
paimon-flink/paimon-flink-cdc/src/main/java/org/apache/paimon/flink/action/cdc/mysql/MySqlTypeUtils.java:
##########
@@ -262,6 +263,9 @@ public static DataType toDataType(
case TEXT:
case MEDIUMTEXT:
case LONGTEXT:
+ return length == null || length == 0 || typeMapping.containsMode(LONGTEXT_TO_BYTES)
+ ? DataTypes.VARCHAR(length)
+ : DataTypes.STRING();
Review Comment:
There's an issue with my VPN, so I won't be able to submit the code today.ðŸ˜
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [flink] support longtext data type mapping bytes [incubator-paimon]
Posted by "MonsterChenzhuo (via GitHub)" <gi...@apache.org>.
MonsterChenzhuo commented on PR #2281:
URL: https://github.com/apache/incubator-paimon/pull/2281#issuecomment-1832335838
@yuzelin @zhuangchong PTAL,thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] [flink] support longtext data type mapping bytes [incubator-paimon]
Posted by "MonsterChenzhuo (via GitHub)" <gi...@apache.org>.
MonsterChenzhuo commented on code in PR #2281:
URL: https://github.com/apache/incubator-paimon/pull/2281#discussion_r1384777336
##########
paimon-flink/paimon-flink-cdc/src/main/java/org/apache/paimon/flink/action/cdc/mysql/MySqlTypeUtils.java:
##########
@@ -262,6 +263,9 @@ public static DataType toDataType(
case TEXT:
case MEDIUMTEXT:
case LONGTEXT:
+ return length == null || length == 0 || typeMapping.containsMode(LONGTEXT_TO_BYTES)
+ ? DataTypes.VARCHAR(length)
+ : DataTypes.STRING();
Review Comment:
Sorry, this is WIP and the code has not been submitted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [flink] support longtext data type mapping bytes [incubator-paimon]
Posted by "yuzelin (via GitHub)" <gi...@apache.org>.
yuzelin commented on code in PR #2281:
URL: https://github.com/apache/incubator-paimon/pull/2281#discussion_r1410432992
##########
paimon-flink/paimon-flink-cdc/src/main/java/org/apache/paimon/flink/action/cdc/mysql/MySqlTypeUtils.java:
##########
@@ -286,6 +286,10 @@ public static DataType toDataType(
return DataTypes.STRING();
// MySQL BINARY and VARBINARY are stored as bytes in JSON. We convert them to
// DataTypes.VARBINARY to retain the length information
Review Comment:
This comment is for `case BINARY|VARBINARY`, please adjust the place of new code.
##########
paimon-flink/paimon-flink-cdc/src/main/java/org/apache/paimon/flink/action/cdc/TypeMapping.java:
##########
@@ -70,6 +70,7 @@ public enum TypeMappingMode {
TINYINT1_NOT_BOOL,
TO_NULLABLE,
TO_STRING,
+ LONGTEXT_TO_BYTES,
Review Comment:
Update comment above `TypeMappingMode`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [flink] support longtext data type mapping bytes [incubator-paimon]
Posted by "MonsterChenzhuo (via GitHub)" <gi...@apache.org>.
MonsterChenzhuo commented on PR #2281:
URL: https://github.com/apache/incubator-paimon/pull/2281#issuecomment-1833543958
@yuzelin PTAL,thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] [flink] support longtext data type mapping bytes [incubator-paimon]
Posted by "zhuangchong (via GitHub)" <gi...@apache.org>.
zhuangchong commented on code in PR #2281:
URL: https://github.com/apache/incubator-paimon/pull/2281#discussion_r1384731889
##########
paimon-flink/paimon-flink-cdc/src/main/java/org/apache/paimon/flink/action/cdc/mysql/MySqlTypeUtils.java:
##########
@@ -262,6 +263,9 @@ public static DataType toDataType(
case TEXT:
case MEDIUMTEXT:
case LONGTEXT:
+ return length == null || length == 0 || typeMapping.containsMode(LONGTEXT_TO_BYTES)
+ ? DataTypes.VARCHAR(length)
+ : DataTypes.STRING();
Review Comment:
DataTypes.VARBINARY?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [flink] support longtext data type mapping bytes [incubator-paimon]
Posted by "MonsterChenzhuo (via GitHub)" <gi...@apache.org>.
MonsterChenzhuo closed pull request #2281: [flink] support longtext data type mapping bytes
URL: https://github.com/apache/incubator-paimon/pull/2281
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [flink] support longtext data type mapping bytes [incubator-paimon]
Posted by "MonsterChenzhuo (via GitHub)" <gi...@apache.org>.
MonsterChenzhuo closed pull request #2281: [flink] support longtext data type mapping bytes
URL: https://github.com/apache/incubator-paimon/pull/2281
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org