You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/06/18 21:10:27 UTC
[GitHub] [pulsar] Anonymitaet opened a new issue #10971: [Doc] Transaction
Anonymitaet opened a new issue #10971:
URL: https://github.com/apache/pulsar/issues/10971
Recently, Pulsar 2.8 has been released and one of the highlights is transaction API, which supports atomic writes and acknowledgments across multiple topics. It completes our vision of making Pulsar a complete unified messaging and streaming platform.
The [transaction docs](https://pulsar.apache.org/docs/en/next/transactions/) need to be improved, so I've re-architectured the doc and drafted one [here](https://docs.google.com/document/d/1rChR2ZScYxCDKUmkQTg6inbuhyfrz_jJfdQOFtwO8P0/edit#).
Feel free to comment, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet edited a comment on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
Anonymitaet edited a comment on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-865877663
Hi @newur thanks for your suggestions.
After discussing w/ @congbobo184, we've adjusted the doc architecture and want to make the following changes:
1. Move the following contents to the `What is txn` chapter since they are general explanations and appropriate for stream processing (like Pulsar Flink connector, Pulsar Spark connector, and Function).
2. Add `Use Case` chapter
(1) Move the following contents to the Pulsar Flink connector doc and add the link here.
(2) Currently, only the Pulsar Flink connector supports txn. Pulsar function and Pulsar spark connector may support txn in the future. At that time, we'll add txn explanations for them to their corresponding doc and then add links to them here.
Does this make sense to you?
Besides, "Today’s businesses need to handle ever-increasing amounts of live data." demonstrates the fact (processing a lot of streaming data is in great demand), I think it is OK to say that. :-D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] newur edited a comment on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
newur edited a comment on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-865272604
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] newur commented on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
newur commented on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-864523880
Really good idea to update the docu on this one. I was wondering a lot about how this feature works and missing examples, after reading the release notes and the blog post.
For me the last chapter in the draft, called 'Integration' only made it clear in which case the transactional feature applies. Maybe wanna move that chapter/picture a bit up?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-866448895
>Sounds good. Maybe you can add a brace to the picture from which start to which end the transaction applies (I think I understand it from the text now, but would be nice anyway).
Thanks for your suggestion. I think the directions of these arrows already point out the start and end (from left to right), correct?
> Nit: I think writes the resulting messages to another topic is more like writes the result to one or more topics? At least there are multiple topics in the picture, and the sentence about the source is in a similar fashion.
Thanks for reminding me, I've updated it to `writes the result to one or more topics`.
> Currently, only the Pulsar Flink connector supports txn. Pulsar function and Pulsar spark connector may support txn in the future.
> Hu? I was under the impression I can write my own transaction in a pulsar function? Thought there were some code examples either on the pulsar side or in the linked blog post, no?
Hi @congbobo184 for this question, could you please help explain? thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] newur edited a comment on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
newur edited a comment on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-865272604
@Anonymitaet I would say so, yes. For me transactions are all about the boundaries, so I want to know where does it start and till which point can it rollback. And pictures are pretty useful to illustrate borders. :D
Btw. I would drop statement like "Today’s businesses need to handle ever-increasing amounts of live data." This probably sounds great in marketing, but is fluff for most people who will actually read the documentation. Or actually implement a transactional boundary. ^^ Just my 2 cents of course
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-863870313
@congbobo184 since you are the lead developer of the transaction, could you please help review overall and answer the questions marked in the doc?
It would be much appreciated if you could provide any feedback by EOD 6/23 GMT+8, many thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] newur commented on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
newur commented on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-865272604
@Anonymitaet I would say so, yes. For me transaction are all about the boundaries, so I want to know where does it start and till which point can it rollback. And pictures are pretty useful to illustrate borders. :D
Btw. I would drop statement like "Today’s businesses need to handle ever-increasing amounts of live data." This probably sounds great in marketing, but is fluff for most people who will actually read the documentation. Or actually implement a transactional boundary. ^^ Just my 2 cents of course
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-864845975
@newur thanks for your feedback. Does it make sense to move the contents of `integration` to the `what feature do transactions provide` section under the `overview` chapter?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] newur edited a comment on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
newur edited a comment on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-865272604
@Anonymitaet I would say so, yes. For me transactions are all about the boundaries, so I want to know where does it start and till which point can it rollback. And pictures are pretty useful to illustrate borders. :D
Btw. I would drop statements like "Today’s businesses need to handle ever-increasing amounts of live data." This probably sounds great in marketing, but is fluff for most people who will actually read the documentation. Or actually implement a transactional boundary. ^^ Just my 2 cents of course
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-865877663
Hi @newur thanks for your suggestions.
After discussing w/ @congbobo184, we've adjusted the doc architecture and want to make the following changes:
1. Move the following contents to the `What is txn` chapter since they are general explanations and appropriate for stream processing (like Pulsar Flink connector, Pulsar Spark connector, and Function)
2. Add `Use Case` chapter
(1) Currently, only the Pulsar Flink connector supports txn. Pulsar function and Pulsar spark connector may support txn in the future, we can add links to them later.
(2) Move the following contents to the Pulsar Flink connector guide and add the link here
Does this make sense to you?
Besides, "Today’s businesses need to handle ever-increasing amounts of live data." demonstrates the fact (processing a lot of streaming data is in great demand), I think it is OK to say that. :-D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] newur edited a comment on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
newur edited a comment on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-865910225
Sounds good. Maybe you can add a brace to the picture from which start to which end the transaction applies (I think I understand it from the text now, but would be nice anyway).
Nit: I think `writes the resulting messages to another topic` is more like `writes the result to one or more topics`? At least there are multiple topics in the picture, and the sentence about the source is in a similar fashion.
> Currently, only the Pulsar Flink connector supports txn. Pulsar function and Pulsar spark connector may support txn in the future.
Hu? I was under the impression I can write my own transaction in a pulsar function? Thought there were some code examples either on the pulsar side or in the linked blog post, no?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] newur edited a comment on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
newur edited a comment on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-865910225
Sounds good. Maybe you can add a brace to the picture from which start to which end the transaction applies (I think I understand it from the text now, but would be nice anyway).
Nit: I think `writes the resulting messages to another topic` is more like `writes the result to one or more topics`?
> Currently, only the Pulsar Flink connector supports txn. Pulsar function and Pulsar spark connector may support txn in the future.
Hu? I was under the impression I can write my own transaction in a pulsar function? Thought there were some code examples either on the pulsar side or in the linked blog post, no?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui closed issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
codelipenghui closed issue #10971:
URL: https://github.com/apache/pulsar/issues/10971
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] newur commented on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
newur commented on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-865910225
Sounds good. Maybe you can add a brace to the picture from which start to which end the transaction applies (I think I understand it from the text now, but would be nice anyway).
Nit: I think `writes the resulting messages to another topic` is more like `writes the result to one or more topics`?
> Currently, only the Pulsar Flink connector supports txn. Pulsar function and Pulsar spark connector may support txn in the future.
Hu? I was under the impression I can write my own transaction in a pulsar function? Thought there were some code examples either on the pulsar side or in the linked blog post, no?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] newur edited a comment on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
newur edited a comment on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-865910225
Sounds good. Maybe you can add a brace to the picture from which start to which end the transaction applies (I think I understand it from the text now, but would be nice anyway).
Nit: I think `writes the resulting messages to another topic` is more like `writes the result to one or more topics`? At least there are multiple topics in the picture, and the sentence about the source in a similar fashion.
> Currently, only the Pulsar Flink connector supports txn. Pulsar function and Pulsar spark connector may support txn in the future.
Hu? I was under the impression I can write my own transaction in a pulsar function? Thought there were some code examples either on the pulsar side or in the linked blog post, no?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] newur commented on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
newur commented on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-865272604
@Anonymitaet I would say so, yes. For me transaction are all about the boundaries, so I want to know where does it start and till which point can it rollback. And pictures are pretty useful to illustrate borders. :D
Btw. I would drop statement like "Today’s businesses need to handle ever-increasing amounts of live data." This probably sounds great in marketing, but is fluff for most people who will actually read the documentation. Or actually implement a transactional boundary. ^^ Just my 2 cents of course
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on issue #10971: [Doc] Transaction
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on issue #10971:
URL: https://github.com/apache/pulsar/issues/10971#issuecomment-864845975
@newur thanks for your feedback. Does it make sense to move the contents of `integration` to the `what feature do transactions provide` section under the `overview` chapter?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org