You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by "fuweng11 (via GitHub)" <gi...@apache.org> on 2024/06/06 11:56:28 UTC
[PR] [INLONG-10370][Manager] Support configuration of kV data format [inlong]
fuweng11 opened a new pull request, #10371:
URL: https://github.com/apache/inlong/pull/10371
<!-- Prepare a Pull Request
Change the title of pull request refer to the following example:
[INLONG-XYZ][Component] Title of the pull request
-->
<!-- Specify the issue this pull request going to fix.
The following *XYZ* should be replaced by the actual [GitHub Issue](https://github.com/apache/inlong/issues) number)-->
Fixes #10370
### Motivation
Support configuration of kV data format.
### Modifications
Support configuration of kV data format.
\
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-10370][Manager] Support configuration of kV data format [inlong]
Posted by "EMsnap (via GitHub)" <gi...@apache.org>.
EMsnap merged PR #10371:
URL: https://github.com/apache/inlong/pull/10371
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-10370][Manager] Support configuration of kV data format [inlong]
Posted by "vernedeng (via GitHub)" <gi...@apache.org>.
vernedeng commented on code in PR #10371:
URL: https://github.com/apache/inlong/pull/10371#discussion_r1629389254
##########
inlong-manager/manager-pojo/src/main/java/org/apache/inlong/manager/pojo/source/kafka/KafkaSource.java:
##########
@@ -90,6 +90,9 @@ public class KafkaSource extends StreamSource {
@ApiModelProperty(value = "Data separator")
private String dataSeparator;
+ @ApiModelProperty(value = "Kv separator")
Review Comment:
KV
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-10370][Manager] Support configuration of kV data format [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #10371:
URL: https://github.com/apache/inlong/pull/10371#discussion_r1629401071
##########
inlong-manager/manager-pojo/src/main/java/org/apache/inlong/manager/pojo/sort/node/base/ExtractNodeProvider.java:
##########
@@ -131,6 +136,21 @@ default Format parsingFormat(
case RAW:
format = new RawFormat();
break;
+ case KV:
+ if (StringUtils.isNumeric(separatorStr)) {
Review Comment:
Fixed.
##########
inlong-manager/manager-pojo/src/main/java/org/apache/inlong/manager/pojo/source/kafka/KafkaSourceRequest.java:
##########
@@ -80,10 +78,13 @@ public class KafkaSourceRequest extends SourceRequest {
private String primaryKey;
@ApiModelProperty(value = "Data encoding format: UTF-8, GBK")
- private String dataEncoding = StandardCharsets.UTF_8.toString();
+ private String dataEncoding;
@ApiModelProperty(value = "Data separator")
- private String dataSeparator = String.valueOf((int) '|');
+ private String dataSeparator;
+
+ @ApiModelProperty(value = "Kv separator")
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-10370][Manager] Support configuration of kV data format [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #10371:
URL: https://github.com/apache/inlong/pull/10371#discussion_r1629401535
##########
inlong-manager/manager-pojo/src/main/java/org/apache/inlong/manager/pojo/source/pulsar/PulsarSource.java:
##########
@@ -73,6 +73,9 @@ public class PulsarSource extends StreamSource {
@ApiModelProperty(value = "Data separator")
private String dataSeparator;
+ @ApiModelProperty(value = "Kv separator")
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-10370][Manager] Support configuration of kV data format [inlong]
Posted by "vernedeng (via GitHub)" <gi...@apache.org>.
vernedeng commented on code in PR #10371:
URL: https://github.com/apache/inlong/pull/10371#discussion_r1629387538
##########
inlong-manager/manager-pojo/src/main/java/org/apache/inlong/manager/pojo/sort/node/base/ExtractNodeProvider.java:
##########
@@ -131,6 +136,21 @@ default Format parsingFormat(
case RAW:
format = new RawFormat();
break;
+ case KV:
+ if (StringUtils.isNumeric(separatorStr)) {
Review Comment:
Extract to one method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-10370][Manager] Support configuration of kV data format [inlong]
Posted by "vernedeng (via GitHub)" <gi...@apache.org>.
vernedeng commented on code in PR #10371:
URL: https://github.com/apache/inlong/pull/10371#discussion_r1629398698
##########
inlong-manager/manager-pojo/src/main/java/org/apache/inlong/manager/pojo/source/pulsar/PulsarSource.java:
##########
@@ -73,6 +73,9 @@ public class PulsarSource extends StreamSource {
@ApiModelProperty(value = "Data separator")
private String dataSeparator;
+ @ApiModelProperty(value = "Kv separator")
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-10370][Manager] Support configuration of kV data format [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #10371:
URL: https://github.com/apache/inlong/pull/10371#discussion_r1629402448
##########
inlong-manager/manager-pojo/src/main/java/org/apache/inlong/manager/pojo/source/kafka/KafkaSource.java:
##########
@@ -90,6 +90,9 @@ public class KafkaSource extends StreamSource {
@ApiModelProperty(value = "Data separator")
private String dataSeparator;
+ @ApiModelProperty(value = "Kv separator")
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org