You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "Hisoka-X (via GitHub)" <gi...@apache.org> on 2025/08/18 08:18:47 UTC
[I] [Feature] Connector prepare for RAG [seatunnel]
Hisoka-X opened a new issue, #9713:
URL: https://github.com/apache/seatunnel/issues/9713
### Search before asking
- [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
### Description
As a multimodal data integration tool, we hope that SeaTunnel can support parsing complex file types, converting their contents into structured file streams, and ultimately writing them into a vector library through embedding. This issue tracks related tasks.
1. Support parse markdown to structured data (Parser + Normalization).
2. Support parse word to structured data (Parser + Normalization).
3. Support parse pdf to structured data (Parser + Normalization).
4. Support text splitter transform (Chunking).
5. Write a document to introduce the entire RAG Ready data processing process.
For chunking please refer
Please refer https://docs.dify.ai/en/guides/knowledge-base/create-knowledge-and-upload-documents/chunking-and-cleaning-text
and
https://docs.llamaindex.ai/en/stable/examples/node_parsers/semantic_chunking/
### Usage Scenario
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature] Connector prepare for RAG [seatunnel]
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on issue #9713:
URL: https://github.com/apache/seatunnel/issues/9713#issuecomment-3200677781
> [@Hisoka-X](https://github.com/Hisoka-X)
>
> Is it alright if I collaborate with [@joonseolee](https://github.com/joonseolee) on the abstraction task and also take on tickets 1 through 3 together?
Sure! Thanks @joonseolee @iinow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature] Connector prepare for RAG [seatunnel]
Posted by "joonseolee (via GitHub)" <gi...@apache.org>.
joonseolee commented on issue #9713:
URL: https://github.com/apache/seatunnel/issues/9713#issuecomment-3200646961
@Hisoka-X
Before working on this, I think it’s important to have a discussion about abstraction first.
Is it okay for the person who originally created this to just go ahead and handle the abstraction work as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature] Connector prepare for RAG [seatunnel]
Posted by "iinow (via GitHub)" <gi...@apache.org>.
iinow commented on issue #9713:
URL: https://github.com/apache/seatunnel/issues/9713#issuecomment-3200658131
@Hisoka-X
Is it alright if I collaborate with @joonseolee on the abstraction task and also take on tickets 1 through 3 together?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org