You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by fubupc <fu...@gmail.com> on 2013/11/23 06:15:28 UTC
How acknowledge mechanism work?
Hi everybody, I asked this question before in storm ustlist but not got
answer. So I posted it here again...
My question is about the tracking algorithm in
Guaranteeing-message-processing<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fnathanmarz%2Fstorm%2Fwiki%2FGuaranteeing-message-processing&sa=D&sntz=1&usg=AFQjCNE0e9AjEDnrBLDrzwfi8YZZfBkBNA>
.
In that article, it said "An acker task stores a map from a spout tuple id
to a pair of values. The first value is the task id that created the spout
tuple which is used later on to send completion messages. The second value
is a 64 bit number called the "ack val". The ack val is a representation of
the state of the entire tuple tree, no matter how big or how small. It is
simply the *xor of all tuple ids* that have been created and/or acked in
the tree."
But when I try to test some simple topology with debug mode ON I found some
interesting log messages:
2013-10-29 11:09:00 task [INFO] Emitting: spout-1 default [SOMEMESSAGE]
2013-10-29 11:09:00 task [INFO] Emitting: spout-1 __ack_init
[-7415442840981449264 -678585584408453014 4]
As you can see: after spout emit out a tuple, it send a message to acker.
So far so good. But the problem is there are* two *64 bit number in the
message. After some experiment I think the first one (-7415442840981449264) is
the tuple id and the other (-678585584408453014) is used for ack val
calculation. The third number 4 is the task id.
So it's conflict with the statement above "*xor of all tuple ids"* in the
Guaranteeing-message-processing<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fnathanmarz%2Fstorm%2Fwiki%2FGuaranteeing-message-processing&sa=D&sntz=1&usg=AFQjCNE0e9AjEDnrBLDrzwfi8YZZfBkBNA>
documentation,
right? The documentation is wrong or I misunderstand something?
Thanks!
Re: How acknowledge mechanism work?
Posted by fubupc <fu...@gmail.com>.
OK. Thank you!
2013/11/24 James Xu <xu...@gmail.com>
> You are right.
>
> On 2013年11月23日, at 下午11:51, fubupc <fu...@gmail.com> wrote:
>
> Thanks very much for your detail explanation!
>
> So in my previous example: [-7415442840981449264 -678585584408453014 4]
> -7415442840981449264 is the LOGICAL spout tuple root-id.
> -678585584408453014 is the PYSHICAL spout tuple id.
> right?
>
>
>
> Btw, the article Acking-framework-implementation<https://github.com/nathanmarz/storm/wiki/Acking-framework-implementation>
> said:
>
> *...The acker is actually a regular bolt, with its execute method
> <https://github.com/nathanmarz/storm/blob/46c3ba7/storm-core/src/clj/backtype/storm/daemon/acker.clj#L36> defined
> withing mk-acker-bolt. When a new tupletree is born, the spout sends the
> XORed edge-ids of each tuple recipient, which the acker records in
> its pending ledger. Every time an executor acks a tuple, the acker receives
> a partial checksum that is the XOR of the tuple's own edge-id (clearing it
> from the ledger) and the edge-id of each downstream tuple the executor
> emitted (thus entering them into the ledger)....*
>
> It mentioned an interesting statement: edge id.
>
> It seems "edge id" can explain "ack algorithm" as well.
>
>
>
>
>
>
> 2013/11/23 James Xu <xu...@gmail.com>
>
>> When you want programmatically send ONE spout tuple out, there actually
>> could be MANY spout tuple instances sent. Because
>> (1) there may be many bolts subscribe to the same spout
>> (2) All-Grouping requests the spout tuple to be sent to all the tasks of
>> the downstream bolts.
>>
>> Let’s take the (1) case as an example:
>>
>> S1 ―――― B1
>> |
>> |―― B2
>>
>> when S1 wants to send a LOGICAL spout tuple, there will be three ids:
>>
>> 1. TUPLE_ID1: tuple-id for the PHYSICAL tuple sent to B1
>> 2. TUPLE_ID2: tuple-id for the PHYSICAL tuple sent to B2
>> 3. ROOT_ID: the root-id for this LOGICAL spout tuple ― for tracking the
>> tuple tree
>>
>> so when a spout tuple is sent out, the following is sent to ACKER:
>> [ROOT_ID, (TUPLE_ID1 xor TUPLE_ID2), taskId]
>>
>> On 2013年11月23日, at 下午10:42, fubupc <fu...@gmail.com> wrote:
>>
>> Why is the initial ackval not the spout tuple id? At first there is only
>> one tuple *created and/or acked.*
>>
>>
>> 2013/11/23 James Xu <xu...@gmail.com>
>>
>>> There is no conflict. As you have quoted:
>>>
>>> *"An acker task stores a map from a spout tuple id to a pair of values.
>>> The first value is the task id that created the spout tuple which is used
>>> later on to send completion messages. The second value is a 64 bit number
>>> called the "ack val". The ack val is a representation of the state of the
>>> entire tuple tree, no matter how big or how small. It is simply the xor of
>>> all tuple ids that have been created and/or acked in the tree.**”*
>>>
>>> So storm needs three variables to track the tuple tree of one
>>> spout tuple:
>>> 1. the spout tuple id (-7415442840981449264)
>>> 2. the ackval(-678585584408453014)
>>> 3. the task id(4)
>>>
>>> On 2013年11月23日, at 下午1:15, fubupc <fu...@gmail.com> wrote:
>>>
>>> Hi everybody, I asked this question before in storm ustlist but not got
>>> answer. So I posted it here again...
>>>
>>> My question is about the tracking algorithm in
>>> Guaranteeing-message-processing<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fnathanmarz%2Fstorm%2Fwiki%2FGuaranteeing-message-processing&sa=D&sntz=1&usg=AFQjCNE0e9AjEDnrBLDrzwfi8YZZfBkBNA>
>>> .
>>>
>>> In that article, it said "An acker task stores a map from a spout tuple
>>> id to a pair of values. The first value is the task id that created the
>>> spout tuple which is used later on to send completion messages. The second
>>> value is a 64 bit number called the "ack val". The ack val is a
>>> representation of the state of the entire tuple tree, no matter how big or
>>> how small. It is simply the *xor of all tuple ids* that have been
>>> created and/or acked in the tree."
>>>
>>> But when I try to test some simple topology with debug mode ON I found
>>> some interesting log messages:
>>> 2013-10-29 11:09:00 task [INFO] Emitting: spout-1 default [SOMEMESSAGE]
>>> 2013-10-29 11:09:00 task [INFO] Emitting: spout-1 __ack_init
>>> [-7415442840981449264 -678585584408453014 4]
>>> As you can see: after spout emit out a tuple, it send a message to
>>> acker. So far so good. But the problem is there are* two *64 bit number
>>> in the message. After some experiment I think the first one (
>>> -7415442840981449264) is the tuple id and the other (
>>> -678585584408453014) is used for ack val calculation. The third number
>>> 4 is the task id.
>>>
>>> So it's conflict with the statement above "*xor of all tuple ids"* in
>>> the Guaranteeing-message-processing<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fnathanmarz%2Fstorm%2Fwiki%2FGuaranteeing-message-processing&sa=D&sntz=1&usg=AFQjCNE0e9AjEDnrBLDrzwfi8YZZfBkBNA> documentation,
>>> right? The documentation is wrong or I misunderstand something?
>>>
>>> Thanks!
>>>
>>>
>>>
>>
>>
>
>
Re: How acknowledge mechanism work?
Posted by James Xu <xu...@gmail.com>.
You are right.
On 2013年11月23日, at 下午11:51, fubupc <fu...@gmail.com> wrote:
> Thanks very much for your detail explanation!
>
> So in my previous example: [-7415442840981449264 -678585584408453014 4]
> -7415442840981449264 is the LOGICAL spout tuple root-id.
> -678585584408453014 is the PYSHICAL spout tuple id.
> right?
>
>
>
> Btw, the article Acking-framework-implementation said:
>
> ...The acker is actually a regular bolt, with its execute method defined withing mk-acker-bolt. When a new tupletree is born, the spout sends the XORed edge-ids of each tuple recipient, which the acker records in its pending ledger. Every time an executor acks a tuple, the acker receives a partial checksum that is the XOR of the tuple's own edge-id (clearing it from the ledger) and the edge-id of each downstream tuple the executor emitted (thus entering them into the ledger)....
>
> It mentioned an interesting statement: edge id.
>
> It seems "edge id" can explain "ack algorithm" as well.
>
>
>
>
>
>
> 2013/11/23 James Xu <xu...@gmail.com>
> When you want programmatically send ONE spout tuple out, there actually could be MANY spout tuple instances sent. Because
> (1) there may be many bolts subscribe to the same spout
> (2) All-Grouping requests the spout tuple to be sent to all the tasks of the downstream bolts.
>
> Let’s take the (1) case as an example:
>
> S1 ———— B1
> |
> |—— B2
>
> when S1 wants to send a LOGICAL spout tuple, there will be three ids:
>
> 1. TUPLE_ID1: tuple-id for the PHYSICAL tuple sent to B1
> 2. TUPLE_ID2: tuple-id for the PHYSICAL tuple sent to B2
> 3. ROOT_ID: the root-id for this LOGICAL spout tuple — for tracking the tuple tree
>
> so when a spout tuple is sent out, the following is sent to ACKER:
> [ROOT_ID, (TUPLE_ID1 xor TUPLE_ID2), taskId]
>
> On 2013年11月23日, at 下午10:42, fubupc <fu...@gmail.com> wrote:
>
>> Why is the initial ackval not the spout tuple id? At first there is only one tuple created and/or acked.
>>
>>
>> 2013/11/23 James Xu <xu...@gmail.com>
>> There is no conflict. As you have quoted:
>>
>> "An acker task stores a map from a spout tuple id to a pair of values. The first value is the task id that created the spout tuple which is used later on to send completion messages. The second value is a 64 bit number called the "ack val". The ack val is a representation of the state of the entire tuple tree, no matter how big or how small. It is simply the xor of all tuple ids that have been created and/or acked in the tree.”
>>
>> So storm needs three variables to track the tuple tree of one spout tuple:
>> 1. the spout tuple id (-7415442840981449264)
>> 2. the ackval(-678585584408453014)
>> 3. the task id(4)
>>
>> On 2013年11月23日, at 下午1:15, fubupc <fu...@gmail.com> wrote:
>>
>>> Hi everybody, I asked this question before in storm ustlist but not got answer. So I posted it here again...
>>>
>>> My question is about the tracking algorithm in Guaranteeing-message-processing.
>>>
>>> In that article, it said "An acker task stores a map from a spout tuple id to a pair of values. The first value is the task id that created the spout tuple which is used later on to send completion messages. The second value is a 64 bit number called the "ack val". The ack val is a representation of the state of the entire tuple tree, no matter how big or how small. It is simply the xor of all tuple ids that have been created and/or acked in the tree."
>>>
>>> But when I try to test some simple topology with debug mode ON I found some interesting log messages:
>>> 2013-10-29 11:09:00 task [INFO] Emitting: spout-1 default [SOMEMESSAGE]
>>> 2013-10-29 11:09:00 task [INFO] Emitting: spout-1 __ack_init [-7415442840981449264 -678585584408453014 4]
>>> As you can see: after spout emit out a tuple, it send a message to acker. So far so good. But the problem is there are two 64 bit number in the message. After some experiment I think the first one (-7415442840981449264) is the tuple id and the other (-678585584408453014) is used for ack val calculation. The third number 4 is the task id.
>>>
>>> So it's conflict with the statement above "xor of all tuple ids" in the Guaranteeing-message-processing documentation, right? The documentation is wrong or I misunderstand something?
>>>
>>> Thanks!
>>
>>
>
>
Re: How acknowledge mechanism work?
Posted by fubupc <fu...@gmail.com>.
Thanks very much for your detail explanation!
So in my previous example: [-7415442840981449264 -678585584408453014 4]
-7415442840981449264 is the LOGICAL spout tuple root-id.
-678585584408453014 is the PYSHICAL spout tuple id.
right?
Btw, the article
Acking-framework-implementation<https://github.com/nathanmarz/storm/wiki/Acking-framework-implementation>
said:
*...The acker is actually a regular bolt, with its execute method
<https://github.com/nathanmarz/storm/blob/46c3ba7/storm-core/src/clj/backtype/storm/daemon/acker.clj#L36>
defined
withing mk-acker-bolt. When a new tupletree is born, the spout sends the
XORed edge-ids of each tuple recipient, which the acker records in
its pending ledger. Every time an executor acks a tuple, the acker receives
a partial checksum that is the XOR of the tuple's own edge-id (clearing it
from the ledger) and the edge-id of each downstream tuple the executor
emitted (thus entering them into the ledger)....*
It mentioned an interesting statement: edge id.
It seems "edge id" can explain "ack algorithm" as well.
2013/11/23 James Xu <xu...@gmail.com>
> When you want programmatically send ONE spout tuple out, there actually
> could be MANY spout tuple instances sent. Because
> (1) there may be many bolts subscribe to the same spout
> (2) All-Grouping requests the spout tuple to be sent to all the tasks of
> the downstream bolts.
>
> Let’s take the (1) case as an example:
>
> S1 ―――― B1
> |
> |―― B2
>
> when S1 wants to send a LOGICAL spout tuple, there will be three ids:
>
> 1. TUPLE_ID1: tuple-id for the PHYSICAL tuple sent to B1
> 2. TUPLE_ID2: tuple-id for the PHYSICAL tuple sent to B2
> 3. ROOT_ID: the root-id for this LOGICAL spout tuple ― for tracking the
> tuple tree
>
> so when a spout tuple is sent out, the following is sent to ACKER:
> [ROOT_ID, (TUPLE_ID1 xor TUPLE_ID2), taskId]
>
> On 2013年11月23日, at 下午10:42, fubupc <fu...@gmail.com> wrote:
>
> Why is the initial ackval not the spout tuple id? At first there is only
> one tuple *created and/or acked.*
>
>
> 2013/11/23 James Xu <xu...@gmail.com>
>
>> There is no conflict. As you have quoted:
>>
>> *"An acker task stores a map from a spout tuple id to a pair of values.
>> The first value is the task id that created the spout tuple which is used
>> later on to send completion messages. The second value is a 64 bit number
>> called the "ack val". The ack val is a representation of the state of the
>> entire tuple tree, no matter how big or how small. It is simply the xor of
>> all tuple ids that have been created and/or acked in the tree.**”*
>>
>> So storm needs three variables to track the tuple tree of one spout tuple:
>> 1. the spout tuple id (-7415442840981449264)
>> 2. the ackval(-678585584408453014)
>> 3. the task id(4)
>>
>> On 2013年11月23日, at 下午1:15, fubupc <fu...@gmail.com> wrote:
>>
>> Hi everybody, I asked this question before in storm ustlist but not got
>> answer. So I posted it here again...
>>
>> My question is about the tracking algorithm in
>> Guaranteeing-message-processing<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fnathanmarz%2Fstorm%2Fwiki%2FGuaranteeing-message-processing&sa=D&sntz=1&usg=AFQjCNE0e9AjEDnrBLDrzwfi8YZZfBkBNA>
>> .
>>
>> In that article, it said "An acker task stores a map from a spout tuple
>> id to a pair of values. The first value is the task id that created the
>> spout tuple which is used later on to send completion messages. The second
>> value is a 64 bit number called the "ack val". The ack val is a
>> representation of the state of the entire tuple tree, no matter how big or
>> how small. It is simply the *xor of all tuple ids* that have been
>> created and/or acked in the tree."
>>
>> But when I try to test some simple topology with debug mode ON I found
>> some interesting log messages:
>> 2013-10-29 11:09:00 task [INFO] Emitting: spout-1 default [SOMEMESSAGE]
>> 2013-10-29 11:09:00 task [INFO] Emitting: spout-1 __ack_init
>> [-7415442840981449264 -678585584408453014 4]
>> As you can see: after spout emit out a tuple, it send a message to
>> acker. So far so good. But the problem is there are* two *64 bit number
>> in the message. After some experiment I think the first one (
>> -7415442840981449264) is the tuple id and the other (-678585584408453014) is
>> used for ack val calculation. The third number 4 is the task id.
>>
>> So it's conflict with the statement above "*xor of all tuple ids"* in the
>> Guaranteeing-message-processing<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fnathanmarz%2Fstorm%2Fwiki%2FGuaranteeing-message-processing&sa=D&sntz=1&usg=AFQjCNE0e9AjEDnrBLDrzwfi8YZZfBkBNA> documentation,
>> right? The documentation is wrong or I misunderstand something?
>>
>> Thanks!
>>
>>
>>
>
>
Re: How acknowledge mechanism work?
Posted by James Xu <xu...@gmail.com>.
When you want programmatically send ONE spout tuple out, there actually could be MANY spout tuple instances sent. Because
(1) there may be many bolts subscribe to the same spout
(2) All-Grouping requests the spout tuple to be sent to all the tasks of the downstream bolts.
Let’s take the (1) case as an example:
S1 ———— B1
|
|—— B2
when S1 wants to send a LOGICAL spout tuple, there will be three ids:
1. TUPLE_ID1: tuple-id for the PHYSICAL tuple sent to B1
2. TUPLE_ID2: tuple-id for the PHYSICAL tuple sent to B2
3. ROOT_ID: the root-id for this LOGICAL spout tuple — for tracking the tuple tree
so when a spout tuple is sent out, the following is sent to ACKER:
[ROOT_ID, (TUPLE_ID1 xor TUPLE_ID2), taskId]
On 2013年11月23日, at 下午10:42, fubupc <fu...@gmail.com> wrote:
> Why is the initial ackval not the spout tuple id? At first there is only one tuple created and/or acked.
>
>
> 2013/11/23 James Xu <xu...@gmail.com>
> There is no conflict. As you have quoted:
>
> "An acker task stores a map from a spout tuple id to a pair of values. The first value is the task id that created the spout tuple which is used later on to send completion messages. The second value is a 64 bit number called the "ack val". The ack val is a representation of the state of the entire tuple tree, no matter how big or how small. It is simply the xor of all tuple ids that have been created and/or acked in the tree.”
>
> So storm needs three variables to track the tuple tree of one spout tuple:
> 1. the spout tuple id (-7415442840981449264)
> 2. the ackval(-678585584408453014)
> 3. the task id(4)
>
> On 2013年11月23日, at 下午1:15, fubupc <fu...@gmail.com> wrote:
>
>> Hi everybody, I asked this question before in storm ustlist but not got answer. So I posted it here again...
>>
>> My question is about the tracking algorithm in Guaranteeing-message-processing.
>>
>> In that article, it said "An acker task stores a map from a spout tuple id to a pair of values. The first value is the task id that created the spout tuple which is used later on to send completion messages. The second value is a 64 bit number called the "ack val". The ack val is a representation of the state of the entire tuple tree, no matter how big or how small. It is simply the xor of all tuple ids that have been created and/or acked in the tree."
>>
>> But when I try to test some simple topology with debug mode ON I found some interesting log messages:
>> 2013-10-29 11:09:00 task [INFO] Emitting: spout-1 default [SOMEMESSAGE]
>> 2013-10-29 11:09:00 task [INFO] Emitting: spout-1 __ack_init [-7415442840981449264 -678585584408453014 4]
>> As you can see: after spout emit out a tuple, it send a message to acker. So far so good. But the problem is there are two 64 bit number in the message. After some experiment I think the first one (-7415442840981449264) is the tuple id and the other (-678585584408453014) is used for ack val calculation. The third number 4 is the task id.
>>
>> So it's conflict with the statement above "xor of all tuple ids" in the Guaranteeing-message-processing documentation, right? The documentation is wrong or I misunderstand something?
>>
>> Thanks!
>
>
Re: How acknowledge mechanism work?
Posted by fubupc <fu...@gmail.com>.
Why is the initial ackval not the spout tuple id? At first there is only
one tuple *created and/or acked.*
2013/11/23 James Xu <xu...@gmail.com>
> There is no conflict. As you have quoted:
>
> *"An acker task stores a map from a spout tuple id to a pair of values.
> The first value is the task id that created the spout tuple which is used
> later on to send completion messages. The second value is a 64 bit number
> called the "ack val". The ack val is a representation of the state of the
> entire tuple tree, no matter how big or how small. It is simply the xor of
> all tuple ids that have been created and/or acked in the tree.**”*
>
> So storm needs three variables to track the tuple tree of one spout tuple:
> 1. the spout tuple id (-7415442840981449264)
> 2. the ackval(-678585584408453014)
> 3. the task id(4)
>
> On 2013年11月23日, at 下午1:15, fubupc <fu...@gmail.com> wrote:
>
> Hi everybody, I asked this question before in storm ustlist but not got
> answer. So I posted it here again...
>
> My question is about the tracking algorithm in
> Guaranteeing-message-processing<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fnathanmarz%2Fstorm%2Fwiki%2FGuaranteeing-message-processing&sa=D&sntz=1&usg=AFQjCNE0e9AjEDnrBLDrzwfi8YZZfBkBNA>
> .
>
> In that article, it said "An acker task stores a map from a spout tuple
> id to a pair of values. The first value is the task id that created the
> spout tuple which is used later on to send completion messages. The second
> value is a 64 bit number called the "ack val". The ack val is a
> representation of the state of the entire tuple tree, no matter how big or
> how small. It is simply the *xor of all tuple ids* that have been created
> and/or acked in the tree."
>
> But when I try to test some simple topology with debug mode ON I found
> some interesting log messages:
> 2013-10-29 11:09:00 task [INFO] Emitting: spout-1 default [SOMEMESSAGE]
> 2013-10-29 11:09:00 task [INFO] Emitting: spout-1 __ack_init
> [-7415442840981449264 -678585584408453014 4]
> As you can see: after spout emit out a tuple, it send a message to acker.
> So far so good. But the problem is there are* two *64 bit number in the
> message. After some experiment I think the first one (
> -7415442840981449264) is the tuple id and the other (-678585584408453014) is
> used for ack val calculation. The third number 4 is the task id.
>
> So it's conflict with the statement above "*xor of all tuple ids"* in the
> Guaranteeing-message-processing<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fnathanmarz%2Fstorm%2Fwiki%2FGuaranteeing-message-processing&sa=D&sntz=1&usg=AFQjCNE0e9AjEDnrBLDrzwfi8YZZfBkBNA> documentation,
> right? The documentation is wrong or I misunderstand something?
>
> Thanks!
>
>
>
Re: How acknowledge mechanism work?
Posted by James Xu <xu...@gmail.com>.
There is no conflict. As you have quoted:
"An acker task stores a map from a spout tuple id to a pair of values. The first value is the task id that created the spout tuple which is used later on to send completion messages. The second value is a 64 bit number called the "ack val". The ack val is a representation of the state of the entire tuple tree, no matter how big or how small. It is simply the xor of all tuple ids that have been created and/or acked in the tree.”
So storm needs three variables to track the tuple tree of one spout tuple:
1. the spout tuple id (-7415442840981449264)
2. the ackval(-678585584408453014)
3. the task id(4)
On 2013年11月23日, at 下午1:15, fubupc <fu...@gmail.com> wrote:
> Hi everybody, I asked this question before in storm ustlist but not got answer. So I posted it here again...
>
> My question is about the tracking algorithm in Guaranteeing-message-processing.
>
> In that article, it said "An acker task stores a map from a spout tuple id to a pair of values. The first value is the task id that created the spout tuple which is used later on to send completion messages. The second value is a 64 bit number called the "ack val". The ack val is a representation of the state of the entire tuple tree, no matter how big or how small. It is simply the xor of all tuple ids that have been created and/or acked in the tree."
>
> But when I try to test some simple topology with debug mode ON I found some interesting log messages:
> 2013-10-29 11:09:00 task [INFO] Emitting: spout-1 default [SOMEMESSAGE]
> 2013-10-29 11:09:00 task [INFO] Emitting: spout-1 __ack_init [-7415442840981449264 -678585584408453014 4]
> As you can see: after spout emit out a tuple, it send a message to acker. So far so good. But the problem is there are two 64 bit number in the message. After some experiment I think the first one (-7415442840981449264) is the tuple id and the other (-678585584408453014) is used for ack val calculation. The third number 4 is the task id.
>
> So it's conflict with the statement above "xor of all tuple ids" in the Guaranteeing-message-processing documentation, right? The documentation is wrong or I misunderstand something?
>
> Thanks!