You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by GitBox <gi...@apache.org> on 2022/08/25 11:26:04 UTC

[GitHub] [inlong] e-mhui opened a new pull request, #5706: [INLONG-5703][Manager] Get separator from enum 'DataSeparator' and remove fieldDelimiter from properties of PulsarSource

e-mhui opened a new pull request, #5706:
URL: https://github.com/apache/inlong/pull/5706

   ### Prepare a Pull Request
   
   [INLONG-5703][Manager] Get separator from enum 'DataSeparator' and remove fieldDelimiter from properties of PulsarSource
   
   - Fixes #5703 
   
   ### Motivation
   
   - DataSeparator is an ascii code, so we should get separator from enum `DataSeparator `
   - if not remove fieldDelimiter, it will be parse into sql option. for example:
   
   ```
   "properties": {
     "properties.auth-params": "token:eyJrZXlJZCI6InB1bHNhci1vcDQ4OG5nYThtZW8iLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJwdWxzYXItb3A0ODhuZ2E4bWVvX2F1dG8tZGV2MTI1NzMwNTE1ODIwMjIwODE0MjI0NzIwIn0.cG3huJk1tcVEQSZvz-7zS-yr3XIypgVNJzFdLpREYdI",
     "properties.auth-plugin-classname": "org.apache.pulsar.client.impl.auth.AuthenticationToken",
     "fieldDelimiter": "|"
   }
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [inlong] healchow commented on a diff in pull request #5706: [INLONG-5703][Manager] Get separator from enum 'DataSeparator' and remove fieldDelimiter from properties of PulsarSource

Posted by GitBox <gi...@apache.org>.
healchow commented on code in PR #5706:
URL: https://github.com/apache/inlong/pull/5706#discussion_r954904892


##########
inlong-manager/manager-pojo/src/main/java/org/apache/inlong/manager/pojo/source/pulsar/PulsarSource.java:
##########
@@ -62,6 +64,15 @@ public class PulsarSource extends StreamSource {
     @ApiModelProperty("Primary key, needed when serialization type is csv, json, avro")
     private String primaryKey;
 
+    @ApiModelProperty(value = "Data encoding format: UTF-8, GBK")
+    private String dataEncoding = StandardCharsets.UTF_8.toString();
+
+    @ApiModelProperty(value = "Data separator, stored as ASCII code")
+    private String dataSeparator = DataSeparator.VERTICAL_BAR.getAsciiCode().toString();

Review Comment:
   Just set the default value in the `PulsarSourceRequest`, other classes do not need it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [inlong] dockerzhang merged pull request #5706: [INLONG-5703][Manager] Add separator-related fields for some sources

Posted by GitBox <gi...@apache.org>.
dockerzhang merged PR #5706:
URL: https://github.com/apache/inlong/pull/5706


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [inlong] e-mhui commented on a diff in pull request #5706: [INLONG-5703][Manager] Get separator from enum 'DataSeparator' and remove fieldDelimiter from properties of PulsarSource

Posted by GitBox <gi...@apache.org>.
e-mhui commented on code in PR #5706:
URL: https://github.com/apache/inlong/pull/5706#discussion_r954982348


##########
inlong-manager/manager-pojo/src/main/java/org/apache/inlong/manager/pojo/source/pulsar/PulsarSource.java:
##########
@@ -62,6 +64,15 @@ public class PulsarSource extends StreamSource {
     @ApiModelProperty("Primary key, needed when serialization type is csv, json, avro")
     private String primaryKey;
 
+    @ApiModelProperty(value = "Data encoding format: UTF-8, GBK")
+    private String dataEncoding = StandardCharsets.UTF_8.toString();
+
+    @ApiModelProperty(value = "Data separator, stored as ASCII code")
+    private String dataSeparator = DataSeparator.VERTICAL_BAR.getAsciiCode().toString();

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org