You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/08/22 03:06:58 UTC

[GitHub] [incubator-seatunnel] ashulin opened a new issue, #2490: [Feature][Api] Enable the SPI factory classes to improve the entire process.

ashulin opened a new issue, #2490:
URL: https://github.com/apache/incubator-seatunnel/issues/2490

### Search before asking

- [X] I had searched in the [feature](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.

### Description

Currently, the way we parse `RowType` is inconsistent.

These factories should be enabled to allow users to configure more easily and support more features;

Below I briefly explain the factory:
1. CatalogFactory: a feature with the same target as the user-defined schema, used to obtain the structured information of the table;
2. DeserializationFormatFactory & SerializationFormatFactory: used to support kafka, file and other unstructured connectors to process data in various formats;
3. TableSourceFactory: It is used to support multi-table reading and metadata column features. It is an upper-level extension of source;
4. TableSinkFactory: The same level as TableSourceFactory, but there are currently no supported features;

### Usage Scenario

1. Provides a unified schema.
2. Support for metadata columns.
3. Support for CDC connector.
4. Support for format SPI.

### Related issues

https://github.com/apache/incubator-seatunnel/issues/2469

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] Carl-Zhou-CN commented on issue #2490: [Feature][Core] Enable the SPI factory classes to improve the entire process.

Posted by GitBox <gi...@apache.org>.

Carl-Zhou-CN commented on issue #2490:
URL: https://github.com/apache/incubator-seatunnel/issues/2490#issuecomment-1286572699

   @ashulin hi, is it possible to identify the source of meta data in the schema definition and treat it as a normal field,Load when it needs to be processed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #2490: [Feature][Core] Enable the SPI factory classes to improve the entire process.

Posted by GitBox <gi...@apache.org>.

github-actions[bot] commented on issue #2490:
URL: https://github.com/apache/incubator-seatunnel/issues/2490#issuecomment-1366277408

   This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] ashulin commented on issue #2490: [Feature][Core] Enable the SPI factory classes to improve the entire process.

Posted by GitBox <gi...@apache.org>.

ashulin commented on issue #2490:
URL: https://github.com/apache/incubator-seatunnel/issues/2490#issuecomment-1290249256

   > @ashulin hi, is it possible to identify the source of meta data in the schema definition and treat it as a normal field,Load when it needs to be processed
   
   @Carl-Zhou-CN That's how it's designed right now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] github-actions[bot] closed issue #2490: [Feature][Core] Enable the SPI factory classes to improve the entire process.

Posted by GitBox <gi...@apache.org>.

github-actions[bot] closed issue #2490: [Feature][Core] Enable the SPI factory classes to improve the entire process.
URL: https://github.com/apache/incubator-seatunnel/issues/2490


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #2490: [Feature][Core] Enable the SPI factory classes to improve the entire process.

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on issue #2490:
URL: https://github.com/apache/incubator-seatunnel/issues/2490#issuecomment-1445234913

   This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] hailin0 commented on issue #2490: [Feature][Core] Enable the SPI factory classes to improve the entire process.

Posted by GitBox <gi...@apache.org>.

hailin0 commented on issue #2490:
URL: https://github.com/apache/incubator-seatunnel/issues/2490#issuecomment-1308820151

   To support cdc, we also need to support RowKind(delete/update) based on primary key define in sink


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #2490: [Feature][Core] Enable the SPI factory classes to improve the entire process.

Posted by GitBox <gi...@apache.org>.

github-actions[bot] commented on issue #2490:
URL: https://github.com/apache/incubator-seatunnel/issues/2490#issuecomment-1360549841

   This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] github-actions[bot] closed issue #2490: [Feature][Core] Enable the SPI factory classes to improve the entire process.

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] closed issue #2490: [Feature][Core] Enable the SPI factory classes to improve the entire process.
URL: https://github.com/apache/incubator-seatunnel/issues/2490


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org