You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "Hisoka-X (via GitHub)" <gi...@apache.org> on 2025/08/18 08:18:47 UTC

[I] [Feature] Connector prepare for RAG [seatunnel]

Hisoka-X opened a new issue, #9713:
URL: https://github.com/apache/seatunnel/issues/9713

   ### Search before asking
   
   - [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
   
   
   ### Description
   
   As a multimodal data integration tool, we hope that SeaTunnel can support parsing complex file types, converting their contents into structured file streams, and ultimately writing them into a vector library through embedding. This issue tracks related tasks.
   
   1. Support parse markdown to structured data (Parser + Normalization).
   2. Support parse word to structured data (Parser + Normalization).
   3. Support parse pdf to structured data (Parser + Normalization).
   4. Support text splitter transform (Chunking).
   5. Write a document to introduce the entire RAG Ready data processing process.
   
   For chunking please refer 
   Please refer https://docs.dify.ai/en/guides/knowledge-base/create-knowledge-and-upload-documents/chunking-and-cleaning-text
   and 
   https://docs.llamaindex.ai/en/stable/examples/node_parsers/semantic_chunking/
   
   
   ### Usage Scenario
   
   _No response_
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature] Connector prepare for RAG [seatunnel]

Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on issue #9713:
URL: https://github.com/apache/seatunnel/issues/9713#issuecomment-3200677781

   > [@Hisoka-X](https://github.com/Hisoka-X)
   > 
   > Is it alright if I collaborate with [@joonseolee](https://github.com/joonseolee) on the abstraction task and also take on tickets 1 through 3 together?
   
   Sure! Thanks @joonseolee @iinow 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature] Connector prepare for RAG [seatunnel]

Posted by "joonseolee (via GitHub)" <gi...@apache.org>.
joonseolee commented on issue #9713:
URL: https://github.com/apache/seatunnel/issues/9713#issuecomment-3200646961

   @Hisoka-X 
   
   Before working on this, I think it’s important to have a discussion about abstraction first.  
   Is it okay for the person who originally created this to just go ahead and handle the abstraction work as well?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature] Connector prepare for RAG [seatunnel]

Posted by "iinow (via GitHub)" <gi...@apache.org>.
iinow commented on issue #9713:
URL: https://github.com/apache/seatunnel/issues/9713#issuecomment-3200658131

   @Hisoka-X 
   
   Is it alright if I collaborate with @joonseolee  on the abstraction task and also take on tickets 1 through 3 together?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org