You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@wayang.apache.org by "kamir (via GitHub)" <gi...@apache.org> on 2024/09/05 09:21:17 UTC
[I] Restructure the Kafka related operators [incubator-wayang]
kamir opened a new issue, #469:
URL: https://github.com/apache/incubator-wayang/issues/469
Proposal:
We create a **wayang-kafka** module inside of **wayang-platforms**.
This leads to a cleaner approach to add these operators.
Right now we edit both **wayang-java** and **wayang-spark**, where both of them contain versions of operators related to reading and writing to Kafka.
Conceptually, having all implementations for the specific APIs in **wayang-kafka** would be neat.
Then you could have wayang-kafka/src/main/java/JavaKafkaOperator wayang-kafka/src/main/java/SparkKafkaOperator etc. and the respective platforms themselves don't have to concern about Kafka.
It is planned to implement this as soon as the initial demos for Kafka integration are ready.
Target: Release 1.1 - Q1-2025.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Restructure the Kafka related operators [incubator-wayang]
Posted by "2pk03 (via GitHub)" <gi...@apache.org>.
2pk03 commented on issue #469:
URL: https://github.com/apache/incubator-wayang/issues/469#issuecomment-2331338267
+1 - that's a great idea! There we can also put our REST API or pure storage modules.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Restructure the Kafka related operators [wayang]
Posted by "zkaoudi (via GitHub) via commits" <co...@wayang.apache.org>.
zkaoudi commented on issue #469:
URL: https://github.com/apache/wayang/issues/469#issuecomment-6558256404
Thank you for bringing this up @Kaileshwar16. It has been forgotten.
This should come up as another [DISCUSS] item in the dev list. There are pros and cons for both approaches and it's good to be discussed outside this issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Restructure the Kafka related operators [incubator-wayang]
Posted by "zkaoudi (via GitHub)" <gi...@apache.org>.
zkaoudi commented on issue #469:
URL: https://github.com/apache/incubator-wayang/issues/469#issuecomment-2331270817
To take this further, we create another module called wayang-io (or some other name) and inside this parent module we place the wayang-kafka module that @kamir mentioned above. The reason for that being: Kafka is not a processing platform, only reads or writes data. So inside wayang-io we can have all the data sources and the data sinks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Restructure the Kafka related operators [wayang]
Posted by "Kaileshwar16 (via GitHub) via commits" <co...@wayang.apache.org>.
Kaileshwar16 commented on issue #469:
URL: https://github.com/apache/wayang/issues/469#issuecomment-6556431657
@zkaoudi @novatechflow, Is the proposed `wayang-io` structure still the preferred approach, given that the original 1.1 release target has passed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org