You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Michael Pearce <Mi...@ig.com> on 2017/01/05 20:29:52 UTC
Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
@Jay.
On KIP-82 are you and Jun now on board that Headers are the better of the two options/kips?
If so could you reply on that thread this is the case.
We really would like to get KIP-82 going/moving on an active discussion around the last few implementation details, rather than where we’re at that seems were held up on getting consensus that the concept of KIP-82 is worth it.
If we don’t push on this one.
Either way we really need a solution of 82 or 87 Apr/May time release (or when ever the next one is after the imminent on in Feb)
Cheers
Mike
On 20/12/2016, 05:55, "Michael Pearce" <Mi...@ig.com> wrote:
Agreed. As I said there is still a use case just I wouldn't be pushing for it, the. Need for it reduces for me.
Sent using OWA for iPhone
________________________________________
From: radai <ra...@gmail.com>
Sent: Tuesday, December 20, 2016 3:51:33 AM
To: dev@kafka.apache.org
Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
i didnt mean to sound as insisting. what i actually mean is it would still
be a valid issue but of much less concern.
On Mon, Dec 19, 2016 at 7:50 PM, radai <ra...@gmail.com> wrote:
> this kip fixes a "bug" (quirk?) that arises when people implement headers
> "in V" (in the payload part of a message).
> if you have proper headers you obviously dont need to to stick them in V
> and so wont run into this, but its still a valid issue.
>
> On Mon, Dec 19, 2016 at 3:06 PM, Jay Kreps <ja...@confluent.io> wrote:
>
>> Makes sense!
>>
>> -Jay
>>
>> On Mon, Dec 19, 2016 at 2:40 PM, Michael Pearce <Mi...@ig.com>
>> wrote:
>>
>> > Wow just read that def over tired. Hopefully it makes sense. Or you get
>> > the gist at least.
>> >
>> > ________________________________________
>> > From: Michael Pearce <Mi...@ig.com>
>> > Sent: Monday, December 19, 2016 9:19:02 PM
>> > To: dev@kafka.apache.org
>> > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
>> >
>> > Hi Jay,
>> >
>> > Agreed this stemmed as offshoot from KIP-82.
>> >
>> > Which our main driver for was to be able to have some headers for a null
>> > value as such for our routing, audit, tracing and a few other bits which
>> > currently we are forced to do with a message wrapper, if we all agreed
>> on
>> > KIP-82 that we need native headers and look to implement that the push
>> for
>> > this would dissipate.
>> >
>> > This KIP would allow for though one use case that comes to mind we could
>> > see which is to have business data with a delete. Though as said this
>> isn't
>> > something we are pushing for think really we would have.
>> >
>> > As such in summary yes, if you want to fully support KIP-82 and we can
>> get
>> > that agreed in principle and a target release version, I think quite a
>> few
>> > guys at LinkedIn are quite pro it too ;) I'm happy to drop this one.
>> >
>> > Cheers
>> > Mike
>> >
>> > Sent using OWA for iPhone
>> > ________________________________________
>> > From: Jay Kreps <ja...@confluent.io>
>> > Sent: Monday, December 19, 2016 8:51:23 PM
>> > To: dev@kafka.apache.org
>> > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
>> >
>> > Hey Michael,
>> >
>> > Here is the compatibility concern I have:
>> >
>> > 1. You have a consumer app that relies on value == null to indicate a
>> > delete (current semantics).
>> > 2. You upgrade Kafka and your clients.
>> > 3. Some producer starts using the tombstone field in combination with
>> > non-null.
>> >
>> > I share Ismael's dislike of setting tombstones on records with null
>> values.
>> > This makes sense as a transitional state, but as an end state its a bit
>> > weird. You'd expect to be able to mix null values and tombstones, and
>> have
>> > the null values remain and the tombstones get compacted. However what
>> will
>> > happen is both will be compacted and upon debugging this you'll learn
>> that
>> > we sometimes use null in the value to indicate tombstone. Ismael's
>> solution
>> > is a bigger compatibility break, though, so not sure if that is better.
>> >
>> > My other question is the relationship to KIP-82. My read is that this
>> KIP
>> > solves some but not all of the problems KIP-82 is intended for. KIP-82,
>> on
>> > the other hand, seems to address most of the motivating uses for this
>> KIP.
>> > The exception is maybe item (5) on the list where you want to
>> simultaneous
>> > delete and convey some information to subscribers, but I couldn't
>> construct
>> > a concrete examples for that one. Do we need to rationalize these two
>> KIPs?
>> > That is, do you still advocate this proposal if we do KIP-82 and vice
>> > versa? As you may have noticed I'm somewhat emotionally invested in the
>> > simplicity of the core data model, so my default position is let's try
>> to
>> > avoid stuffing more stuff in, but if we have to add stuff I like each of
>> > these individually more than doing both. :-)
>> >
>> > -Jay
>> >
>> >
>> >
>> >
>> > On Fri, Dec 16, 2016 at 12:16 PM, Michael Pearce <Michael.Pearce@ig.com
>> >
>> > wrote:
>> >
>> > > Hi Jay
>> > >
>> > > I disagree here that we are breaking any compatibility, we went
>> through
>> > > this on the discussion thread in fact with the help of that thread is
>> how
>> > > the kip came to the solution.
>> > >
>> > > Also on the supported combinations front you mention, we are not
>> taking
>> > > anything away afaik.
>> > >
>> > > Currently supported are only are:
>> > > Null value = delete
>> > > Non-null value = non delete
>> > >
>> > > With this kip we would support
>> > > Null value + tombstone = delete
>> > > Non null value + tombstone = delete
>> > > Non null value + no tombstone = non delete
>> > >
>> > > As for the alternative idea, this is simply a new policy, how can
>> there
>> > be
>> > > confusion here? For this policy it would be explicit that tombstone
>> > marker
>> > > would need to be set for a delete.
>> > >
>> > > I'm going to vent a little now as starting to get quite frustrated.
>> > >
>> > > We are going round in circles on kip-82 as per use cases there is now
>> > many
>> > > use cases, how many more are needed? just because confluent don't see
>> > these
>> > > doesn't mean they aren't real use cases other have, this is the point
>> of
>> > > the Apache foundation, it shouldn't be the view of just one
>> organisation.
>> > > It really is getting a feeling of the NIH syndrome. Rather than it
>> being
>> > > constructive on discussion of the implementation detail.
>> > >
>> > > kip-87 spawned from as on the kip call we all agreed this was needed.
>> And
>> > > would at least allow a custom wrapper be supported in a compacted
>> topic,
>> > > allowing meta data. Which again now I feel we are spinning wheels, and
>> > > simply finding reasons not support it.
>> > >
>> > > Cheers
>> > > Mike
>> > >
>> > >
>> > >
>> > > Sent using OWA for iPad
>> > > ________________________________________
>> > > From: Jay Kreps <ja...@confluent.io>
>> > > Sent: Friday, December 16, 2016 7:09:23 PM
>> > > To: dev@kafka.apache.org
>> > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
>> > >
>> > > Hey Michael,
>> > >
>> > > I do think it might have been better had we started with a separate
>> > concept
>> > > of null vs delete. Given that we are where we are, I'm not sure that
>> the
>> > > possible cures we explored so far are better than the disease.
>> > >
>> > > I apologize for coming to this late, but I didn't really understand
>> the
>> > > implications of the KIP and that we'd be breaking compatibility with
>> > > existing apps until the vote had begun, and in my defense I'm not sure
>> > the
>> > > other folks voting did either.
>> > >
>> > > I think we all agree there are many existing apps that are built with
>> the
>> > > assumption of "null value non-tombstone" and it isn't possible to
>> > > disambiguate these from tombstones on the producer. It isn't that
>> anyone
>> > is
>> > > saying we have to support all four possibilities at once, it's that we
>> > > simply can't orphan one of the existing combinations or our users will
>> > eat
>> > > us!
>> > >
>> > > If I've understood your alternate solution of adding another setting
>> for
>> > > compaction, I think this does fix the compatibility problem, but it
>> adds
>> > an
>> > > odd mode the user has to add on all their topics. While the current
>> state
>> > > is easily explainable, the resulting state where the meaning of
>> tombstone
>> > > and null are overlapping and ambiguous and dependent on a compaction
>> > > setting that could change out of band or not be in sync with your
>> code in
>> > > some environment seems worse to me then where we currently are. I
>> think
>> > the
>> > > question is how would this combination be explained to users and does
>> it
>> > > make sense?
>> > >
>> > > -Jay
>> > >
>> > >
>> > >
>> > > On Fri, Dec 16, 2016 at 9:25 AM, Michael Pearce <
>> Michael.Pearce@ig.com>
>> > > wrote:
>> > >
>> > > > Hi Chaps,
>> > > >
>> > > > Can we either get one more +1 binding (we have 2 already) on the
>> > > existing?
>> > > >
>> > > > Or have some response on the below possible alternative. We are
>> keen to
>> > > > get working on this, so we make next feature release.
>> > > >
>> > > > Cheers
>> > > > Mike
>> > > >
>> > > >
>> > > > On 13/12/2016, 16:32, "Michael Pearce" <Mi...@ig.com>
>> wrote:
>> > > >
>> > > > Hi Ismael
>> > > >
>> > > > Did you see our email this morning, what's your thoughts on this
>> > > > approach to instead we simply have a brand new policy?
>> > > >
>> > > > Cheers
>> > > > Mike
>> > > >
>> > > >
>> > > > Sent using OWA for iPhone
>> > > > ________________________________________
>> > > > From: ismaelj@gmail.com <is...@gmail.com> on behalf of Ismael
>> > > Juma <
>> > > > ismael@juma.me.uk>
>> > > > Sent: Tuesday, December 13, 2016 11:30:05 AM
>> > > > To: dev@kafka.apache.org
>> > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
>> > > >
>> > > > Yes, this is actually tricky to do in a way where we both have
>> the
>> > > > desired
>> > > > semantics and maintain compatibility. When someone creates a
>> > > > `ProducerRecord` with a `null` value today, the producer doesn't
>> > know
>> > > > if
>> > > > it's meant to be a tombstone or not. For V3 messages, it's easy
>> > when
>> > > > the
>> > > > constructor that takes a tombstone is used. However, if any
>> other
>> > > > constructor is used, it's not clear. A couple of options I can
>> > think
>> > > > of,
>> > > > none of them particularly nice:
>> > > >
>> > > > 1. Have a third state where tombstone = unknown and the broker
>> > would
>> > > > set
>> > > > the tombstone bit if the value was null and the topic was
>> > compacted.
>> > > > People
>> > > > that wanted to pass a non-null value for the tombstone would
>> have
>> > to
>> > > > use
>> > > > the constructor that takes a tombstone. The drawbacks: third
>> state
>> > > for
>> > > > tombstone in message format, message conversion at the broker
>> for a
>> > > > common
>> > > > case.
>> > > >
>> > > > 2. Extend MetadataResponse to optionally include topic configs,
>> > which
>> > > > would
>> > > > make it possible for the producer to be smarter about setting
>> the
>> > > > tombstone. It would only do it if a tombstone was not passed
>> > > > explicitly,
>> > > > the value was null and the topic was compacted. The main
>> drawback
>> > is
>> > > > that
>> > > > the producer would be getting a bit more data for each topic
>> even
>> > > > though it
>> > > > probably won't use it most of the time. Extending
>> MetadataResponse
>> > to
>> > > > return topic configs would be useful for other reasons as well,
>> so
>> > > that
>> > > > part seems OK.
>> > > >
>> > > > In addition, for both proposals, we could consider adding
>> warnings
>> > to
>> > > > the
>> > > > documentation that the behaviour of the constructors that don't
>> > take
>> > > a
>> > > > tombstone would change in the next major release so that
>> tombstone
>> > =
>> > > > false.
>> > > > Not sure if this would be worth it though.
>> > > >
>> > > > Ismael
>> > > >
>> > > > On Sun, Dec 11, 2016 at 11:15 PM, Ewen Cheslack-Postava <
>> > > > ewen@confluent.io>
>> > > > wrote:
>> > > >
>> > > > > Michael,
>> > > > >
>> > > > > It kind of depends on how you want to interpret the tombstone
>> > flag.
>> > > > If it's
>> > > > > purely a producer-facing Kafka-level thing that we treat as
>> > > internal
>> > > > to the
>> > > > > broker and log cleaner once the record is sent, then your
>> > approach
>> > > > makes
>> > > > > sense. You're just moving copying the null-indicates-delete
>> > > behavior
>> > > > of the
>> > > > > old constructor into the tombstone flag.
>> > > > >
>> > > > > However, if you want this change to more generally decouple
>> the
>> > > idea
>> > > > of
>> > > > > deletion and null values, then you are sometimes converting
>> what
>> > > > might be a
>> > > > > completely valid null value that doesn't indicate deletion
>> into a
>> > > > > tombstone. Downstream applications could potentially handle
>> these
>> > > > cases
>> > > > > differently given the separation of deletion from value.
>> > > > >
>> > > > > I guess the question is if we want to try to support the
>> latter
>> > > even
>> > > > for
>> > > > > topics where we have older produce requests. An example where
>> > this
>> > > > could
>> > > > > come up is in something like a CDC Connector. If we try to
>> > support
>> > > > the
>> > > > > semantic difference, a connector might write changes to Kafka
>> > using
>> > > > the
>> > > > > tombstone flag to indicate when a row was truly deleted (vs an
>> > > > update that
>> > > > > sets it to null but still present; this probably makes more
>> sense
>> > > > for CDC
>> > > > > from document stores or extracting single columns). There are
>> > > various
>> > > > > reasons we might want to maintain the full log and not turn
>> > > > compaction on
>> > > > > (or just use a time-based retention policy), but downstream
>> > > > applications
>> > > > > might care to know the difference between a delete and a null
>> > > value.
>> > > > In
>> > > > > fact both versions of the same log (compacted and
>> time-retention)
>> > > > could be
>> > > > > useful and I don't think it'll be uncommon to maintain both or
>> > use
>> > > > KIP-71
>> > > > > to maintain a hybrid compacted/retention topic.
>> > > > >
>> > > > > -Ewen
>> > > > >
>> > > > > On Sun, Dec 11, 2016 at 1:18 PM, Michael Pearce <
>> > > > Michael.Pearce@ig.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Jay,
>> > > > > >
>> > > > > > Why wouldn't that work, the tombstone value is only looked
>> at
>> > by
>> > > > the
>> > > > > > broker, on a topic configured for compaction as such is
>> benign
>> > on
>> > > > non
>> > > > > > compacted topics. This is as much as sending a null value
>> > > currently
>> > > > > >
>> > > > > >
>> > > > > > Regards
>> > > > > > Mike
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > Sent using OWA for iPhone
>> > > > > > ________________________________________
>> > > > > > From: Jay Kreps <ja...@confluent.io>
>> > > > > > Sent: Sunday, December 11, 2016 8:58:53 PM
>> > > > > > To: dev@kafka.apache.org
>> > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
>> > > > > >
>> > > > > > Hey Michael,
>> > > > > >
>> > > > > > I'm not quite sure that works as that would translate ALL
>> null
>> > > > values to
>> > > > > > tombstones, even for non-compacted topics that use null as
>> an
>> > > > acceptable
>> > > > > > value sent by the producer and expected by the consumer.
>> > > > > >
>> > > > > > -Jay
>> > > > > >
>> > > > > > On Sun, Dec 11, 2016 at 3:26 AM, Michael Pearce <
>> > > > Michael.Pearce@ig.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi Ewen,
>> > > > > > >
>> > > > > > > I think the easiest way to show this is with code.
>> > > > > > >
>> > > > > > > As you can see we keep the existing behaviour for
>> > code/binaries
>> > > > calling
>> > > > > > > the pre-existing constructors, whereby if the value is
>> null
>> > the
>> > > > > tombstone
>> > > > > > > is set to true.
>> > > > > > >
>> > > > > > > Regards
>> > > > > > > Mike
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > /**
>> > > > > > > * Creates a record with a specified timestamp to be
>> sent
>> > > to
>> > > > a
>> > > > > > > specified topic and partition
>> > > > > > > *
>> > > > > > > * @param topic The topic the record will be appended
>> to
>> > > > > > > * @param partition The partition to which the record
>> > > should
>> > > > be
>> > > > > sent
>> > > > > > > * @param timestamp The timestamp of the record
>> > > > > > > * @param tombstone if the record should be treated
>> as a
>> > > > tombstone
>> > > > > if
>> > > > > > > the topic is compacted
>> > > > > > > * @param key The key that will be included in the
>> record
>> > > > > > > * @param value The record contents
>> > > > > > > */
>> > > > > > > public ProducerRecord(String topic, Integer partition,
>> > > > Boolean
>> > > > > > > tombstone, Long timestamp, K key, V value) {
>> > > > > > > if (topic == null)
>> > > > > > > throw new IllegalArgumentException("Topic
>> cannot
>> > > be
>> > > > > null.");
>> > > > > > > if (timestamp != null && timestamp < 0)
>> > > > > > > throw new IllegalArgumentException(
>> > > > > > > String.format("Invalid timestamp: %d.
>> > > > Timestamp
>> > > > > > should
>> > > > > > > always be non-negative or null.", timestamp));
>> > > > > > > if (partition != null && partition < 0)
>> > > > > > > throw new IllegalArgumentException(
>> > > > > > > String.format("Invalid partition: %d.
>> > > > Partition
>> > > > > > number
>> > > > > > > should always be non-negative or null.", partition));
>> > > > > > > if (!tombstone && value == null){
>> > > > > > > throw new IllegalArgumentException(
>> > > > > > > String.format("Tombstone must be true
>> if
>> > > null
>> > > > > > value"));
>> > > > > > > }
>> > > > > > > this.topic = topic;
>> > > > > > > this.partition = partition;
>> > > > > > > this.tombstone = tombstone;
>> > > > > > > this.key = key;
>> > > > > > > this.value = value;
>> > > > > > > this.timestamp = timestamp;
>> > > > > > > }
>> > > > > > >
>> > > > > > > public ProducerRecord(String topic, Integer partition,
>> > Long
>> > > > > > timestamp,
>> > > > > > > K key, V value) {
>> > > > > > > this(topic, partition, value == null, timestamp,
>> key,
>> > > > value);
>> > > > > > > }
>> > > > > > > ________________________________________
>> > > > > > > From: Ewen Cheslack-Postava <ew...@confluent.io>
>> > > > > > > Sent: Sunday, December 11, 2016 5:45 AM
>> > > > > > > To: dev@kafka.apache.org
>> > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
>> > > > > > >
>> > > > > > > It seemed like this was addressed in the migration
>> section,
>> > > > wasn't it?
>> > > > > > The
>> > > > > > > V2 vs V3 requests and the fact that the broker will
>> downgrade
>> > > the
>> > > > > message
>> > > > > > > format (losing zero copy) if you issues a V2 request to a
>> > > broker
>> > > > using
>> > > > > V3
>> > > > > > > format handles compatibility, does it not? The existing
>> > > > consumers won't
>> > > > > > see
>> > > > > > > the extra metadata in the value, but they will get a null
>> > > > instead and
>> > > > > > treat
>> > > > > > > it as a tombstone. Certainly there is a performance
>> impact,
>> > but
>> > > > it
>> > > > > seemed
>> > > > > > > compatible.
>> > > > > > >
>> > > > > > > I'm worried about this though:
>> > > > > > >
>> > > > > > >
>> > > > > > > - *NOTE *: With the new version of producer client
>> using
>> > > > > > ProduceRequest
>> > > > > > > V3 (magic byte = 2), a non tombstone (tombstone bit not
>> > set)
>> > > > and
>> > > > > null
>> > > > > > > value
>> > > > > > > should not be allowed as the older version of consumer
>> > using
>> > > > > > > FetchRequest
>> > > > > > > V2 will think of this as a tombstone when its actually
>> > not.
>> > > > > > >
>> > > > > > >
>> > > > > > > Unless I'm misunderstanding, this ends up breaking binary
>> > > > compatibility
>> > > > > > for
>> > > > > > > the Java API. It sounds like this suggests that passing a
>> > null
>> > > > value to
>> > > > > > the
>> > > > > > > existing ProducerRecord constructors would generate an
>> > > exception
>> > > > since
>> > > > > > you
>> > > > > > > didn't explicitly enable the tombstone (via whatever new
>> > > > constructor is
>> > > > > > > provided). But that means you can't swap in a newer client
>> > jar
>> > > > without
>> > > > > > > recompiling and get the same behavior if your app deletes
>> > keys
>> > > > using
>> > > > > the
>> > > > > > > current approach because your app will start throwing
>> > > > exceptions. Maybe
>> > > > > > > this is a tradeoff we're ok with, but we've tried pretty
>> hard
>> > > to
>> > > > avoid
>> > > > > > > breaking compatibility like this so far -- it makes
>> picking
>> > up
>> > > > bug
>> > > > > fixes
>> > > > > > in
>> > > > > > > the clients harder for users of frameworks that have to
>> pin
>> > to
>> > > > earlier
>> > > > > > > default versions for compatibility.
>> > > > > > >
>> > > > > > > But then later the KIP says:
>> > > > > > >
>> > > > > > >
>> > > > > > > - The old constructors for ProducerRecord without this
>> > > > parameter
>> > > > > will
>> > > > > > be
>> > > > > > > kept but updated so that their default behaviour if
>> > setting
>> > > > null
>> > > > > value
>> > > > > > > will
>> > > > > > > be the tombstone will be set to true to keep existing
>> > > > behaviour.
>> > > > > > >
>> > > > > > >
>> > > > > > > So maybe I am misinterpreting something.
>> > > > > > >
>> > > > > > > And just a nit re: motivation section, item 6 would be
>> > clearer
>> > > > for a
>> > > > > > union
>> > > > > > > schema with null (or optional schemas in other formats),
>> e.g.
>> > > > [null,
>> > > > > > > string], in which case losing the schema truly is losing
>> > > > information
>> > > > > > > (whereas null is already the only valid value for a pure
>> null
>> > > > schema).
>> > > > > > >
>> > > > > > > -Ewen
>> > > > > > >
>> > > > > > >
>> > > > > > > On Sat, Dec 10, 2016 at 9:24 PM, Michael Pearce <
>> > > > Michael.Pearce@ig.com
>> > > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Hi Jay,
>> > > > > > > >
>> > > > > > > > Good point this detail is missing in the KIP write up.
>> Ive
>> > > > added this
>> > > > > > > now.
>> > > > > > > >
>> > > > > > > > Essentially simply just upgrading the clients we do not
>> > > expect
>> > > > any
>> > > > > > client
>> > > > > > > > app code change needed.
>> > > > > > > >
>> > > > > > > > Cheers
>> > > > > > > > Mike
>> > > > > > > > ________________________________________
>> > > > > > > > From: Jay Kreps <ja...@confluent.io>
>> > > > > > > > Sent: Saturday, December 10, 2016 10:51 PM
>> > > > > > > > To: dev@kafka.apache.org
>> > > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone
>> Flag
>> > > > > > > >
>> > > > > > > > Michael,
>> > > > > > > >
>> > > > > > > > The compatibility section goes through the migration
>> path,
>> > > but
>> > > > isn't
>> > > > > > the
>> > > > > > > > bigger compatibility issue with existing apps? There are
>> > many
>> > > > > (probably
>> > > > > > > > thousands) of apps in production that use this feature
>> and
>> > > > send null
>> > > > > to
>> > > > > > > > mean delete. It seems like this would break
>> compatibility
>> > > with
>> > > > them,
>> > > > > > and
>> > > > > > > > they would have to be rewritten to right?
>> > > > > > > >
>> > > > > > > > -Jay
>> > > > > > > >
>> > > > > > > > On Thu, Dec 8, 2016 at 12:12 AM, Michael Pearce <
>> > > > > Michael.Pearce@ig.com
>> > > > > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Hi Jun,
>> > > > > > > > >
>> > > > > > > > > 4) On v3 we honour the tombstone. As such we expect
>> it to
>> > > be
>> > > > set
>> > > > > > > > correctly
>> > > > > > > > > as per the KIP.
>> > > > > > > > >
>> > > > > > > > > 4.1) why would we want to produce an error when its
>> v3?
>> > > This
>> > > > is the
>> > > > > > > exact
>> > > > > > > > > purpose to support non-null tombstone's
>> > > > > > > > > 4.2) again here im unclear on the question on the v3,
>> > > produce
>> > > > > request
>> > > > > > > we
>> > > > > > > > > expect the tombstone flag to be set correctly.
>> > > > > > > > >
>> > > > > > > > > 4.4) compaction only occurs on compacted topics, the
>> bit
>> > > > makes no
>> > > > > > > > > difference and not looked at on un-compacted
>> (time/size
>> > > based
>> > > > > > > eviction).
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On 06/12/2016, 20:08, "Jun Rao" <ju...@confluent.io>
>> > wrote:
>> > > > > > > > >
>> > > > > > > > > Hi, Michael,
>> > > > > > > > >
>> > > > > > > > > 4. Then, I think I misunderstood this point. Could
>> > you
>> > > > document
>> > > > > > the
>> > > > > > > > > following points in the wiki?
>> > > > > > > > > 4.1 If producer V3 sets tombstone, but provides a
>> > > > non-null
>> > > > > value,
>> > > > > > > > does
>> > > > > > > > > the
>> > > > > > > > > send() get an error or does the producer
>> > automatically
>> > > > set the
>> > > > > > > value
>> > > > > > > > to
>> > > > > > > > > null?
>> > > > > > > > > 4.2 If producer V3 doesn't set tombstone, but
>> > provides
>> > > a
>> > > > null
>> > > > > > > value,
>> > > > > > > > > does
>> > > > > > > > > the send() get an error or does the producer
>> > > > automatically sets
>> > > > > > the
>> > > > > > > > > tombstone?
>> > > > > > > > > 4.3 Does the broker only expect messages that (a)
>> > have
>> > > no
>> > > > > > tombstone
>> > > > > > > > and
>> > > > > > > > > non-null value; (b) have tombstone and null value
>> and
>> > > > reject
>> > > > > the
>> > > > > > > > > messages
>> > > > > > > > > with an error code otherwise?
>> > > > > > > > > 4.4 Do 4.1, 4.2, 4.3 depend on whether the topic
>> is
>> > > > compacted
>> > > > > on
>> > > > > > > > not?
>> > > > > > > > >
>> > > > > > > > > Thanks,
>> > > > > > > > >
>> > > > > > > > > Jun
>> > > > > > > > >
>> > > > > > > > > On Tue, Dec 6, 2016 at 10:35 AM, Michael Pearce <
>> > > > > > > > Michael.Pearce@ig.com
>> > > > > > > > > >
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Not at all. This only acts on compacted topics
>> > just
>> > > > as what
>> > > > > > > occurs
>> > > > > > > > > today
>> > > > > > > > > >
>> > > > > > > > > > Sent using OWA for iPhone
>> > > > > > > > > > ________________________________________
>> > > > > > > > > > From: Jun Rao <ju...@confluent.io>
>> > > > > > > > > > Sent: Tuesday, December 6, 2016 6:25:28 PM
>> > > > > > > > > > To: dev@kafka.apache.org
>> > > > > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction
>> > Tombstone
>> > > > Flag
>> > > > > > > > > >
>> > > > > > > > > > Hi, Michael,
>> > > > > > > > > >
>> > > > > > > > > > 4. Hmm, does that mean the new client library
>> can
>> > > > never send
>> > > > > a
>> > > > > > > null
>> > > > > > > > > message
>> > > > > > > > > > even to a regular topic? This seems like a
>> change
>> > of
>> > > > the
>> > > > > > existing
>> > > > > > > > > behavior.
>> > > > > > > > > >
>> > > > > > > > > > Thanks,
>> > > > > > > > > >
>> > > > > > > > > > Jun
>> > > > > > > > > >
>> > > > > > > > > > On Tue, Dec 6, 2016 at 9:51 AM, Michael Pearce <
>> > > > > > > > > Michael.Pearce@ig.com>
>> > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > Hi Jun,
>> > > > > > > > > > >
>> > > > > > > > > > > Re 4) That's because we expect the tombstone
>> > value
>> > > > to be
>> > > > > set
>> > > > > > > > > correctly if
>> > > > > > > > > > > message bit is 2, as such if an older client
>> > sends
>> > > > in on
>> > > > > old
>> > > > > > > > > message the
>> > > > > > > > > > > message is upcast and the bit is set
>> correctly.
>> > And
>> > > > such no
>> > > > > > > > longer
>> > > > > > > > > need
>> > > > > > > > > > to
>> > > > > > > > > > > check the value. Mayuresh can you confirm my
>> > > > thinking and
>> > > > > > > > > understanding
>> > > > > > > > > > of
>> > > > > > > > > > > what we've discussed?
>> > > > > > > > > > >
>> > > > > > > > > > > The second point I understand what you're
>> getting
>> > > at
>> > > > now my
>> > > > > > > > > apologies.
>> > > > > > > > > > Yes
>> > > > > > > > > > > this makes sense to save on touching the
>> message,
>> > > if
>> > > > we're
>> > > > > > the
>> > > > > > > > > only kip
>> > > > > > > > > > > going in, in this release.
>> > > > > > > > > > >
>> > > > > > > > > > > Cheers
>> > > > > > > > > > > Mike
>> > > > > > > > > > >
>> > > > > > > > > > > Sent using OWA for iPhone
>> > > > > > > > > > > ________________________________________
>> > > > > > > > > > > From: Jun Rao <ju...@confluent.io>
>> > > > > > > > > > > Sent: Tuesday, December 6, 2016 5:22:13 PM
>> > > > > > > > > > > To: dev@kafka.apache.org
>> > > > > > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction
>> > > > Tombstone Flag
>> > > > > > > > > > >
>> > > > > > > > > > > Hi, Michael,
>> > > > > > > > > > >
>> > > > > > > > > > > 4. Is this updated in the wiki? The text "If
>> the
>> > > > magic byte
>> > > > > > on
>> > > > > > > > > message is
>> > > > > > > > > > > 2, the broker should use the tombstone bit for
>> > log
>> > > > > > compaction."
>> > > > > > > > > doesn't
>> > > > > > > > > > > seem to have changed.
>> > > > > > > > > > >
>> > > > > > > > > > > 2. My point is that if we change the message
>> > format
>> > > > just
>> > > > > for
>> > > > > > > this
>> > > > > > > > > KIP, we
>> > > > > > > > > > > should consider whether it's worth optimizing
>> the
>> > > > down
>> > > > > > > conversion
>> > > > > > > > > path
>> > > > > > > > > > > (i.e., decide whether a conversion is needed
>> by
>> > > just
>> > > > > looking
>> > > > > > at
>> > > > > > > > the
>> > > > > > > > > > > tombstone bit in the wrapper message) since
>> > > > tombstone will
>> > > > > be
>> > > > > > > > used
>> > > > > > > > > > rarely.
>> > > > > > > > > > > However, if the message format change here is
>> > > > combined with
>> > > > > > > other
>> > > > > > > > > KIPs,
>> > > > > > > > > > > then this optimization likely won't be needed.
>> > The
>> > > > latter
>> > > > > > > > probably
>> > > > > > > > > makes
>> > > > > > > > > > > the code simpler. Jiangjie, Mayuresh, what do
>> you
>> > > > think?
>> > > > > > > > > > >
>> > > > > > > > > > > Other than those, +1 from me,
>> > > > > > > > > > >
>> > > > > > > > > > > Thanks,
>> > > > > > > > > > >
>> > > > > > > > > > > Jun
>> > > > > > > > > > >
>> > > > > > > > > > > On Tue, Dec 6, 2016 at 8:54 AM, Michael
>> Pearce <
>> > > > > > > > > Michael.Pearce@ig.com>
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > Hi Jun
>> > > > > > > > > > > >
>> > > > > > > > > > > > do we have your vote on this now?
>> > > > > > > > > > > >
>> > > > > > > > > > > > Any other concerns?
>> > > > > > > > > > > >
>> > > > > > > > > > > > Cheers
>> > > > > > > > > > > > Mike
>> > > > > > > > > > > >
>> > > > > > > > > > > > Sent using OWA for iPhone
>> > > > > > > > > > > > ________________________________________
>> > > > > > > > > > > > From: Michael Pearce <Michael.Pearce@ig.com
>> >
>> > > > > > > > > > > > Sent: Saturday, December 3, 2016 1:37:45 AM
>> > > > > > > > > > > > To: dev@kafka.apache.org
>> > > > > > > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction
>> > > > Tombstone
>> > > > > Flag
>> > > > > > > > > > > >
>> > > > > > > > > > > > Hi Jun,
>> > > > > > > > > > > >
>> > > > > > > > > > > > Have updated. Thanks again for the feedback.
>> > > > > > > > > > > >
>> > > > > > > > > > > > Agree yes we should align up when it gets to
>> > > that,
>> > > > I
>> > > > > assume
>> > > > > > > > > you've
>> > > > > > > > > > > flagged
>> > > > > > > > > > > > the same in the other KIP?
>> > > > > > > > > > > >
>> > > > > > > > > > > > I think for now let's get this KIP approved,
>> > then
>> > > > we can
>> > > > > > > start
>> > > > > > > > > the work
>> > > > > > > > > > > to
>> > > > > > > > > > > > get an initial PR, then we can discuss how
>> to
>> > > > align the
>> > > > > two
>> > > > > > > if
>> > > > > > > > > needed.
>> > > > > > > > > > > >
>> > > > > > > > > > > > Cheers,
>> > > > > > > > > > > > Mike
>> > > > > > > > > > > >
>> > > > > > > > > > > > On 02/12/2016, 18:26, "Jun Rao" <
>> > > jun@confluent.io>
>> > > > > wrote:
>> > > > > > > > > > > >
>> > > > > > > > > > > > Hi, Michael,
>> > > > > > > > > > > >
>> > > > > > > > > > > > For 2), this is fine. Could you update
>> the
>> > > KIP
>> > > > wiki
>> > > > > to
>> > > > > > > make
>> > > > > > > > > this
>> > > > > > > > > > and
>> > > > > > > > > > > > other
>> > > > > > > > > > > > points clearer? Other than that, the KIP
>> > > looks
>> > > > good
>> > > > > to
>> > > > > > > me.
>> > > > > > > > > > > >
>> > > > > > > > > > > > An orthogonal thing is that there are
>> other
>> > > > KIPs such
>> > > > > > as
>> > > > > > > > > KIP-98
>> > > > > > > > > > that
>> > > > > > > > > > > > also
>> > > > > > > > > > > > intend to change the message format. If
>> > they
>> > > > all get
>> > > > > > > > > approved, we
>> > > > > > > > > > > > should
>> > > > > > > > > > > > think about whether it's better to just
>> > bump
>> > > > up the
>> > > > > > magic
>> > > > > > > > > byte once
>> > > > > > > > > > > to
>> > > > > > > > > > > > incorporate multiple format changes
>> like we
>> > > > did in
>> > > > > > > > > KIP-31/KIP-32.
>> > > > > > > > > > > >
>> > > > > > > > > > > > Thanks,
>> > > > > > > > > > > >
>> > > > > > > > > > > > Jun
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > On Fri, Dec 2, 2016 at 3:19 AM, Michael
>> > > Pearce
>> > > > <
>> > > > > > > > > > > Michael.Pearce@ig.com>
>> > > > > > > > > > > > wrote:
>> > > > > > > > > > > >
>> > > > > > > > > > > > > Hi Jao
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Thanks for the response. Sorry for
>> slow
>> > > > reply, both
>> > > > > > > with
>> > > > > > > > > personal
>> > > > > > > > > > > > sickness
>> > > > > > > > > > > > > and also battling some critical issues
>> > > > encountered
>> > > > > > > since
>> > > > > > > > > > upgrading
>> > > > > > > > > > > to
>> > > > > > > > > > > > > 0.10.1.0
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 1) Thans for spotting, Document error
>> > where
>> > > > we
>> > > > > > branched
>> > > > > > > > > this KIP
>> > > > > > > > > > > from
>> > > > > > > > > > > > > KIP-82, will get that removed.
>> > > > > > > > > > > > > 2) Intent is to do this just at the
>> > record
>> > > > message
>> > > > > > > level.
>> > > > > > > > > > > > > 3) Thanks for spotting, Will ensure
>> this
>> > is
>> > > > > > corrected.
>> > > > > > > > > > > > > 4) As per discussion thread we will
>> > support
>> > > > > > tombstone +
>> > > > > > > > > null
>> > > > > > > > > > value,
>> > > > > > > > > > > > > tombstone + non null value, no
>> tombstone
>> > +
>> > > > null
>> > > > > > value.
>> > > > > > > > > > > > > 5) I believe this was in the
>> discussion
>> > > > thread,
>> > > > > > > @Mayuresh
>> > > > > > > > > is this
>> > > > > > > > > > > > > something we've overlooked? I thought
>> we
>> > > > would down
>> > > > > > > > > convert and
>> > > > > > > > > > > > remove the
>> > > > > > > > > > > > > value so the old consumer had existing
>> > > > behavior, or
>> > > > > > is
>> > > > > > > > > there
>> > > > > > > > > > > > something we
>> > > > > > > > > > > > > haven't thought about?
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Cheers
>> > > > > > > > > > > > > Mike
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > On 30/11/2016, 18:12, "Jun Rao" <
>> > > > jun@confluent.io>
>> > > > > > > > wrote:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Hi, Michael,
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Thanks for the KIP. A few comments
>> > > below.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 1. The message format change
>> contains
>> > > > > > > "HeadersLength
>> > > > > > > > > > Headers".
>> > > > > > > > > > > > Is that
>> > > > > > > > > > > > > intended?
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 2. For compressed messageset, is
>> the
>> > > > tombstone
>> > > > > > bit
>> > > > > > > > > only set
>> > > > > > > > > > at
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > shallow
>> > > > > > > > > > > > > level? Do we always leave that
>> bit in
>> > > the
>> > > > > wrapper
>> > > > > > > > > message
>> > > > > > > > > > > unset?
>> > > > > > > > > > > > An
>> > > > > > > > > > > > > alternative is to set the
>> tombstone
>> > bit
>> > > > in the
>> > > > > > > > wrapper
>> > > > > > > > > if at
>> > > > > > > > > > > > least one
>> > > > > > > > > > > > > inner message has the tombstone
>> bit
>> > > set.
>> > > > This
>> > > > > > makes
>> > > > > > > > > things a
>> > > > > > > > > > > bit
>> > > > > > > > > > > > more
>> > > > > > > > > > > > > complicated, but we could
>> potentially
>> > > > exploit
>> > > > > > that
>> > > > > > > > for
>> > > > > > > > > > > > optimizing down
>> > > > > > > > > > > > > conversion. For example, we only
>> need
>> > > to
>> > > > > convert
>> > > > > > > > > messages
>> > > > > > > > > > with
>> > > > > > > > > > > > magic 2
>> > > > > > > > > > > > > to
>> > > > > > > > > > > > > magic 1 if the wrapper's tombstone
>> > bit
>> > > > is set
>> > > > > > > > > (conversion is
>> > > > > > > > > > > > always
>> > > > > > > > > > > > > needed
>> > > > > > > > > > > > > from magic 2 to magic 0). Not
>> sure if
>> > > the
>> > > > > > > > optimization
>> > > > > > > > > is
>> > > > > > > > > > worth
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > complexity though.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 3. The referencing of the new
>> version
>> > > of
>> > > > > > > > > > > > ProducerRequest/FetchRequest
>> > > > > > > > > > > > > is
>> > > > > > > > > > > > > inconsistent (v4 vs v3). Since our
>> > > > convention
>> > > > > > > starts
>> > > > > > > > at
>> > > > > > > > > > version
>> > > > > > > > > > > > at 0, I
>> > > > > > > > > > > > > think the new version would be 3.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 4. "If the magic byte on message
>> is
>> > 2,
>> > > > the
>> > > > > broker
>> > > > > > > > > should use
>> > > > > > > > > > > the
>> > > > > > > > > > > > > tombstone
>> > > > > > > > > > > > > bit for log compaction." What
>> about
>> > > null
>> > > > value?
>> > > > > > My
>> > > > > > > > > > > understanding
>> > > > > > > > > > > > is
>> > > > > > > > > > > > > that
>> > > > > > > > > > > > > null value will be treated the
>> same
>> > as
>> > > > setting
>> > > > > > the
>> > > > > > > > > tombstone
>> > > > > > > > > > > bit.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 5. For the migration path, it
>> would
>> > be
>> > > > useful
>> > > > > to
>> > > > > > > > > describe the
>> > > > > > > > > > > > down
>> > > > > > > > > > > > > conversion path to consumers
>> (i.e.,
>> > > > brokers on
>> > > > > > > > message
>> > > > > > > > > format
>> > > > > > > > > > > > 0.10.2
>> > > > > > > > > > > > > and
>> > > > > > > > > > > > > consumers on older version).
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Thanks,
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Jun
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > On Tue, Nov 29, 2016 at 3:18 AM,
>> > > Michael
>> > > > > Pearce <
>> > > > > > > > > > > > Michael.Pearce@ig.com
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > > Hi All,
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > We have been discussing in the
>> > below
>> > > > thread
>> > > > > and
>> > > > > > > > final
>> > > > > > > > > > changes
>> > > > > > > > > > > > have
>> > > > > > > > > > > > > been
>> > > > > > > > > > > > > > made to the KIP wiki based on
>> these
>> > > > > > discussions.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > We would now like to put to the
>> > vote
>> > > > the
>> > > > > > > following
>> > > > > > > > > KIP:
>> > > > > > > > > > > > > > https://cwiki.apache.org/
>> > > > > > > > > confluence/display/KAFKA/KIP-
>> > > > > > > > > > 87+-+
>> > > > > > > > > > > > > > Add+Compaction+Tombstone+Flag
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > This kip is for having a
>> distinct
>> > > > compaction
>> > > > > > > > > attribute
>> > > > > > > > > > > > "tombstone"
>> > > > > > > > > > > > > flag
>> > > > > > > > > > > > > > instead of relying on null
>> value,
>> > > > allowing
>> > > > > > > non-null
>> > > > > > > > > value
>> > > > > > > > > > > > delete
>> > > > > > > > > > > > > messages.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Many thanks,
>> > > > > > > > > > > > > > Michael
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > On 22/11/2016, 15:52, "Michael
>> > > Pearce"
>> > > > <
>> > > > > > > > > > > Michael.Pearce@ig.com>
>> > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Hi Mayuresh,
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > LGTM. Ive just made one
>> small
>> > > > adjustment
>> > > > > > > > > updating the
>> > > > > > > > > > > wire
>> > > > > > > > > > > > > protocol to
>> > > > > > > > > > > > > > show the magic byte bump.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Do we think we're good to
>> put
>> > to
>> > > a
>> > > > vote?
>> > > > > Is
>> > > > > > > > > there any
>> > > > > > > > > > > > other bits
>> > > > > > > > > > > > > > needing discussion?
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Cheers
>> > > > > > > > > > > > > > Mike
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > On 21/11/2016, 18:26,
>> "Mayuresh
>> > > > Gharat" <
>> > > > > > > > > > > > > gharatmayuresh15@gmail.com>
>> > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Hi Michael,
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > I have updated the
>> > migration
>> > > > section
>> > > > > of
>> > > > > > > the
>> > > > > > > > > KIP.
>> > > > > > > > > > Can
>> > > > > > > > > > > > you
>> > > > > > > > > > > > > please
>> > > > > > > > > > > > > > take a look?
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Thanks,
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Mayuresh
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > On Fri, Nov 18, 2016 at
>> > 9:07
>> > > > AM,
>> > > > > > Mayuresh
>> > > > > > > > > Gharat <
>> > > > > > > > > > > > > > gharatmayuresh15@gmail.com
>> > > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Hi Michael,
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > That whilst sending
>> > > > tombstone and
>> > > > > non
>> > > > > > > > null
>> > > > > > > > > value,
>> > > > > > > > > > > the
>> > > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > can expect
>> > > > > > > > > > > > > > > only to receive the
>> > > non-null
>> > > > > message
>> > > > > > > only
>> > > > > > > > > in step
>> > > > > > > > > > > > (3) is
>> > > > > > > > > > > > > this
>> > > > > > > > > > > > > > correct?
>> > > > > > > > > > > > > > > ---> I do agree with
>> you
>> > > > here.
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Becket, Ismael : can
>> you
>> > > guys
>> > > > > review
>> > > > > > > the
>> > > > > > > > > > migration
>> > > > > > > > > > > > plan
>> > > > > > > > > > > > > listed
>> > > > > > > > > > > > > > above using
>> > > > > > > > > > > > > > > magic byte?
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Thanks,
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Mayuresh
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > On Fri, Nov 18, 2016
>> at
>> > > 8:58
>> > > > AM,
>> > > > > > > Michael
>> > > > > > > > > Pearce <
>> > > > > > > > > > > > > > Michael.Pearce@ig.com>
>> > > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >> Many thanks for this
>> > > > Mayuresh. I
>> > > > > > don't
>> > > > > > > > > have any
>> > > > > > > > > > > > > objections.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> I assume we should
>> > state:
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> That whilst sending
>> > > > tombstone and
>> > > > > > non
>> > > > > > > > null
>> > > > > > > > > > value,
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > can expect
>> > > > > > > > > > > > > > >> only to receive the
>> > > non-null
>> > > > > message
>> > > > > > > > only
>> > > > > > > > > in
>> > > > > > > > > > step
>> > > > > > > > > > > > (3) is
>> > > > > > > > > > > > > this
>> > > > > > > > > > > > > > correct?
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Cheers
>> > > > > > > > > > > > > > >> Mike
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Sent using OWA for
>> > iPhone
>> > > > > > > > > > > > > > >>
>> > > > ______________________________
>> > > > > > > > __________
>> > > > > > > > > > > > > > >> From: Mayuresh
>> Gharat <
>> > > > > > > > > > gharatmayuresh15@gmail.com
>> > > > > > > > > > > >
>> > > > > > > > > > > > > > >> Sent: Thursday,
>> November
>> > > > 17, 2016
>> > > > > > > > 5:18:41
>> > > > > > > > > PM
>> > > > > > > > > > > > > > >> To:
>> > dev@kafka.apache.org
>> > > > > > > > > > > > > > >> Subject: Re:
>> [DISCUSS]
>> > > > KIP-87 -
>> > > > > Add
>> > > > > > > > > Compaction
>> > > > > > > > > > > > Tombstone
>> > > > > > > > > > > > > Flag
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Hi Ismael,
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Thanks for the
>> > > explanation.
>> > > > > > > > > > > > > > >> Specially I like this
>> > part
>> > > > where
>> > > > > in
>> > > > > > > you
>> > > > > > > > > > mentioned
>> > > > > > > > > > > > we can
>> > > > > > > > > > > > > get
>> > > > > > > > > > > > > > rid of the
>> > > > > > > > > > > > > > >> older null value
>> support
>> > > > for log
>> > > > > > > > > compaction
>> > > > > > > > > > later
>> > > > > > > > > > > > on,
>> > > > > > > > > > > > > here :
>> > > > > > > > > > > > > > >> We can't change
>> > semantics
>> > > > of the
>> > > > > > > message
>> > > > > > > > > format
>> > > > > > > > > > > > without
>> > > > > > > > > > > > > having
>> > > > > > > > > > > > > > a long
>> > > > > > > > > > > > > > >> transition period.
>> And
>> > we
>> > > > can't
>> > > > > rely
>> > > > > > > > > > > > > > >> on people reading
>> > > > documentation or
>> > > > > > > > acting
>> > > > > > > > > on a
>> > > > > > > > > > > > warning for
>> > > > > > > > > > > > > > something so
>> > > > > > > > > > > > > > >> fundamental. As
>> such, my
>> > > > take is
>> > > > > > that
>> > > > > > > we
>> > > > > > > > > need to
>> > > > > > > > > > > > bump the
>> > > > > > > > > > > > > magic
>> > > > > > > > > > > > > > byte. The
>> > > > > > > > > > > > > > >> good news is
>> > > > > > > > > > > > > > >> that we don't have to
>> > > > support all
>> > > > > > > > versions
>> > > > > > > > > > > forever.
>> > > > > > > > > > > > We
>> > > > > > > > > > > > > have
>> > > > > > > > > > > > > > said that we
>> > > > > > > > > > > > > > >> will support direct
>> > > > upgrades for 2
>> > > > > > > > years.
>> > > > > > > > > That
>> > > > > > > > > > > > means that
>> > > > > > > > > > > > > > message format
>> > > > > > > > > > > > > > >> version n could, in
>> > > theory,
>> > > > be
>> > > > > > > removed 2
>> > > > > > > > > years
>> > > > > > > > > > > > after the
>> > > > > > > > > > > > > it's
>> > > > > > > > > > > > > > introduced.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Just a heads up, I
>> would
>> > > > like to
>> > > > > > > mention
>> > > > > > > > > that
>> > > > > > > > > > even
>> > > > > > > > > > > > without
>> > > > > > > > > > > > > > bumping magic
>> > > > > > > > > > > > > > >> byte, we will *NOT*
>> > loose
>> > > > zero
>> > > > > copy
>> > > > > > as
>> > > > > > > > in
>> > > > > > > > > the
>> > > > > > > > > > > > client(x+1)
>> > > > > > > > > > > > > in my
>> > > > > > > > > > > > > > >> explanation
>> > > > > > > > > > > > > > >> above will convert
>> > > > internally a
>> > > > > null
>> > > > > > > > > value to
>> > > > > > > > > > > have a
>> > > > > > > > > > > > > tombstone
>> > > > > > > > > > > > > > bit set and
>> > > > > > > > > > > > > > >> a tombstone bit set
>> to
>> > > have
>> > > > a null
>> > > > > > > value
>> > > > > > > > > > > > automatically
>> > > > > > > > > > > > > > internally and by
>> > > > > > > > > > > > > > >> the time we move to
>> > > version
>> > > > (x+2),
>> > > > > > the
>> > > > > > > > > clients
>> > > > > > > > > > > > would have
>> > > > > > > > > > > > > > upgraded.
>> > > > > > > > > > > > > > >> Obviously if we
>> support
>> > a
>> > > > request
>> > > > > > from
>> > > > > > > > > > > consumer(x),
>> > > > > > > > > > > > we
>> > > > > > > > > > > > > will
>> > > > > > > > > > > > > > loose zero
>> > > > > > > > > > > > > > >> copy
>> > > > > > > > > > > > > > >> but that is the same
>> > case
>> > > > with
>> > > > > magic
>> > > > > > > > byte.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> But if magic byte
>> bump
>> > > > makes life
>> > > > > > > easier
>> > > > > > > > > for
>> > > > > > > > > > > > transition
>> > > > > > > > > > > > > for the
>> > > > > > > > > > > > > > above
>> > > > > > > > > > > > > > >> reasons that you
>> > > explained,
>> > > > I am
>> > > > > OK
>> > > > > > > with
>> > > > > > > > > it
>> > > > > > > > > > since
>> > > > > > > > > > > > we are
>> > > > > > > > > > > > > going
>> > > > > > > > > > > > > > to meet the
>> > > > > > > > > > > > > > >> end goal down the
>> road
>> > :)
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> On a side note can we
>> > > > update the
>> > > > > doc
>> > > > > > > > here
>> > > > > > > > > on
>> > > > > > > > > > magic
>> > > > > > > > > > > > byte
>> > > > > > > > > > > > > to say
>> > > > > > > > > > > > > > that "*it
>> > > > > > > > > > > > > > >> should be bumped
>> > whenever
>> > > > the
>> > > > > > message
>> > > > > > > > > format is
>> > > > > > > > > > > > changed
>> > > > > > > > > > > > > or the
>> > > > > > > > > > > > > > >> interpretation of
>> > message
>> > > > format
>> > > > > > > (usage
>> > > > > > > > > of the
>> > > > > > > > > > > > reserved
>> > > > > > > > > > > > > bits as
>> > > > > > > > > > > > > > well) is
>> > > > > > > > > > > > > > >> changed*".
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Hi Michael,
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Here is the update
>> plan
>> > > > that we
>> > > > > > > > discussed
>> > > > > > > > > > offline
>> > > > > > > > > > > > > yesterday :
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Currently the
>> magic-byte
>> > > > which
>> > > > > > > > > corresponds to
>> > > > > > > > > > the
>> > > > > > > > > > > > > > "message.format.version"
>> > > > > > > > > > > > > > >> is set to 1.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> 1) On broker it will
>> be
>> > > set
>> > > > to 1
>> > > > > > > > > initially.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> 2) When a producer
>> > client
>> > > > sends a
>> > > > > > > > message
>> > > > > > > > > with
>> > > > > > > > > > > > magic-byte
>> > > > > > > > > > > > > = 2,
>> > > > > > > > > > > > > > since the
>> > > > > > > > > > > > > > >> broker is on
>> magic-byte
>> > =
>> > > > 1, we
>> > > > > will
>> > > > > > > > down
>> > > > > > > > > > convert
>> > > > > > > > > > > > it,
>> > > > > > > > > > > > > which
>> > > > > > > > > > > > > > means if the
>> > > > > > > > > > > > > > >> tombstone bit is set,
>> > the
>> > > > value
>> > > > > will
>> > > > > > > be
>> > > > > > > > > set to
>> > > > > > > > > > > > null. A
>> > > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > >> understanding
>> > magic-byte =
>> > > > 1, will
>> > > > > > > still
>> > > > > > > > > work
>> > > > > > > > > > with
>> > > > > > > > > > > > this. A
>> > > > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > >> working
>> > > > > > > > > > > > > > >> with magic-byte =2
>> will
>> > > > also be
>> > > > > able
>> > > > > > > to
>> > > > > > > > > > understand
>> > > > > > > > > > > > this,
>> > > > > > > > > > > > > since
>> > > > > > > > > > > > > > it
>> > > > > > > > > > > > > > >> understands the
>> > tombstone.
>> > > > > > > > > > > > > > >> Now there is still
>> the
>> > > > question of
>> > > > > > > > > supporting a
>> > > > > > > > > > > > > non-tombstone
>> > > > > > > > > > > > > > and null
>> > > > > > > > > > > > > > >> value from producer
>> > client
>> > > > with
>> > > > > > > > > magic-byte = 2.*
>> > > > > > > > > > > (I
>> > > > > > > > > > > > am
>> > > > > > > > > > > > > not sure
>> > > > > > > > > > > > > > if we
>> > > > > > > > > > > > > > >> should support this.
>> > > > Ismael/Becket
>> > > > > > can
>> > > > > > > > > comment
>> > > > > > > > > > > > here)*
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> 3) When almost all
>> the
>> > > > clients
>> > > > > have
>> > > > > > > > > upgraded,
>> > > > > > > > > > the
>> > > > > > > > > > > > > > message.format.version
>> > > > > > > > > > > > > > >> on
>> > > > > > > > > > > > > > >> the broker can be
>> > changed
>> > > > to 2,
>> > > > > > where
>> > > > > > > in
>> > > > > > > > > the
>> > > > > > > > > > down
>> > > > > > > > > > > > > conversion in
>> > > > > > > > > > > > > > the above
>> > > > > > > > > > > > > > >> step will not
>> happen. If
>> > > at
>> > > > this
>> > > > > > point
>> > > > > > > > we
>> > > > > > > > > get a
>> > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > request from a
>> > > > > > > > > > > > > > >> older consumer, we
>> might
>> > > > have to
>> > > > > > down
>> > > > > > > > > convert
>> > > > > > > > > > > where
>> > > > > > > > > > > > in we
>> > > > > > > > > > > > > loose
>> > > > > > > > > > > > > > zero copy,
>> > > > > > > > > > > > > > >> but these cases
>> should
>> > be
>> > > > rare.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Becket can you review
>> > this
>> > > > plan
>> > > > > and
>> > > > > > > add
>> > > > > > > > > more
>> > > > > > > > > > > > details if I
>> > > > > > > > > > > > > have
>> > > > > > > > > > > > > > >> missed/wronged
>> > something,
>> > > > before
>> > > > > we
>> > > > > > > put
>> > > > > > > > > it on
>> > > > > > > > > > KIP.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Thanks,
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Mayuresh
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> On Wed, Nov 16, 2016
>> at
>> > > > 11:07 PM,
>> > > > > > > > Michael
>> > > > > > > > > > Pearce <
>> > > > > > > > > > > > > > Michael.Pearce@ig.com>
>> > > > > > > > > > > > > > >> wrote:
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> > Thanks guys, for
>> > > > discussing this
>> > > > > > > > > offline and
>> > > > > > > > > > > > getting
>> > > > > > > > > > > > > some
>> > > > > > > > > > > > > > consensus.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > So its clear for
>> > myself
>> > > > and
>> > > > > others
>> > > > > > > > what
>> > > > > > > > > is
>> > > > > > > > > > > > proposed now
>> > > > > > > > > > > > > (i
>> > > > > > > > > > > > > > think i
>> > > > > > > > > > > > > > >> > understand, but
>> want
>> > to
>> > > > make
>> > > > > sure)
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > Could i ask either
>> > > > directly
>> > > > > update
>> > > > > > > the
>> > > > > > > > > kip to
>> > > > > > > > > > > > detail the
>> > > > > > > > > > > > > > migration
>> > > > > > > > > > > > > > >> > strategy, or
>> > (re-)state
>> > > > your
>> > > > > > offline
>> > > > > > > > > discussed
>> > > > > > > > > > > and
>> > > > > > > > > > > > > agreed
>> > > > > > > > > > > > > > migration
>> > > > > > > > > > > > > > >> > strategy based on a
>> > > magic
>> > > > byte
>> > > > > is
>> > > > > > in
>> > > > > > > > > this
>> > > > > > > > > > > thread.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > The main original
>> > driver
>> > > > for the
>> > > > > > KIP
>> > > > > > > > > was to
>> > > > > > > > > > > > support
>> > > > > > > > > > > > > > compaction where
>> > > > > > > > > > > > > > >> value
>> > > > > > > > > > > > > > >> > isn't null, based
>> off
>> > > the
>> > > > > > > discussions
>> > > > > > > > on
>> > > > > > > > > > KIP-82
>> > > > > > > > > > > > thread.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > We should be able
>> to
>> > > > support
>> > > > > > > > > non-tombstone +
>> > > > > > > > > > > null
>> > > > > > > > > > > > value
>> > > > > > > > > > > > > by the
>> > > > > > > > > > > > > > >> completion
>> > > > > > > > > > > > > > >> > of the KIP, as we
>> > noted
>> > > > when
>> > > > > > > > discussing
>> > > > > > > > > this
>> > > > > > > > > > > kip,
>> > > > > > > > > > > > having
>> > > > > > > > > > > > > > logic based on
>> > > > > > > > > > > > > > >> a
>> > > > > > > > > > > > > > >> > null value isn't
>> very
>> > > > clean and
>> > > > > > also
>> > > > > > > > > separates
>> > > > > > > > > > > the
>> > > > > > > > > > > > > concerns.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > As discussed
>> already
>> > > > though we
>> > > > > can
>> > > > > > > > > split this
>> > > > > > > > > > > into
>> > > > > > > > > > > > > KIP-87a
>> > > > > > > > > > > > > > and KIP-87b
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > Where we look to
>> > deliver
>> > > > KIP-87a
>> > > > > > on
>> > > > > > > a
>> > > > > > > > > > compacted
>> > > > > > > > > > > > topic
>> > > > > > > > > > > > > (to
>> > > > > > > > > > > > > > address the
>> > > > > > > > > > > > > > >> > immediate issues)
>> > > > > > > > > > > > > > >> > * tombstone + null
>> > value
>> > > > > > > > > > > > > > >> > * tombstone +
>> non-null
>> > > > value
>> > > > > > > > > > > > > > >> > * non-tombstone +
>> > > > non-null value
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > Then we can discuss
>> > once
>> > > > KIP-87a
>> > > > > > is
>> > > > > > > > > completed
>> > > > > > > > > > > > options
>> > > > > > > > > > > > > later
>> > > > > > > > > > > > > > and how we
>> > > > > > > > > > > > > > >> > support the second
>> > part
>> > > > KIP-87b
>> > > > > to
>> > > > > > > > > deliver:
>> > > > > > > > > > > > > > >> > * non-tombstone +
>> null
>> > > > value
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > Cheers
>> > > > > > > > > > > > > > >> > Mike
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> >
>> > > > ______________________________
>> > > > > > > > > __________
>> > > > > > > > > > > > > > >> > From: Becket Qin <
>> > > > > > > > becket.qin@gmail.com>
>> > > > > > > > > > > > > > >> > Sent: Thursday,
>> > November
>> > > > 17,
>> > > > > 2016
>> > > > > > > 1:43
>> > > > > > > > > AM
>> > > > > > > > > > > > > > >> > To:
>> > > dev@kafka.apache.org
>> > > > > > > > > > > > > > >> > Subject: Re:
>> [DISCUSS]
>> > > > KIP-87 -
>> > > > > > Add
>> > > > > > > > > Compaction
>> > > > > > > > > > > > > Tombstone Flag
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > Renu, Mayuresh and
>> I
>> > had
>> > > > an
>> > > > > > offline
>> > > > > > > > > > discussion,
>> > > > > > > > > > > > and
>> > > > > > > > > > > > > following
>> > > > > > > > > > > > > > is a brief
>> > > > > > > > > > > > > > >> > summary.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > 1. We agreed that
>> not
>> > > > bumping up
>> > > > > > > magic
>> > > > > > > > > value
>> > > > > > > > > > may
>> > > > > > > > > > > > result
>> > > > > > > > > > > > > in
>> > > > > > > > > > > > > > losing zero
>> > > > > > > > > > > > > > >> copy
>> > > > > > > > > > > > > > >> > during migration.
>> > > > > > > > > > > > > > >> > 2. Given that
>> bumping
>> > up
>> > > > magic
>> > > > > > value
>> > > > > > > > is
>> > > > > > > > > almost
>> > > > > > > > > > > > free and
>> > > > > > > > > > > > > has
>> > > > > > > > > > > > > > benefit of
>> > > > > > > > > > > > > > >> > avoiding potential
>> > > > performance
>> > > > > > > issue.
>> > > > > > > > > It is
>> > > > > > > > > > > > probably
>> > > > > > > > > > > > > worth
>> > > > > > > > > > > > > > doing.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > One issue we still
>> > need
>> > > > to think
>> > > > > > > about
>> > > > > > > > > is
>> > > > > > > > > > > whether
>> > > > > > > > > > > > we
>> > > > > > > > > > > > > want to
>> > > > > > > > > > > > > > support a
>> > > > > > > > > > > > > > >> > non-tombstone
>> message
>> > > > with null
>> > > > > > > value.
>> > > > > > > > > > > > > > >> > Currently it is not
>> > > > supported by
>> > > > > > > > Kafka.
>> > > > > > > > > If we
>> > > > > > > > > > > > allow a
>> > > > > > > > > > > > > > non-tombstone null
>> > > > > > > > > > > > > > >> > value message to
>> exist
>> > > > after
>> > > > > > KIP-87.
>> > > > > > > > The
>> > > > > > > > > > problem
>> > > > > > > > > > > > is
>> > > > > > > > > > > > > that such
>> > > > > > > > > > > > > > message
>> > > > > > > > > > > > > > >> will
>> > > > > > > > > > > > > > >> > not be supported by
>> > the
>> > > > > consumers
>> > > > > > > > prior
>> > > > > > > > > to
>> > > > > > > > > > > KIP-87.
>> > > > > > > > > > > > > Because a
>> > > > > > > > > > > > > > null value
>> > > > > > > > > > > > > > >> > will always be
>> > > > interpreted to a
>> > > > > > > > > tombstone.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > One option is that
>> we
>> > > > keep the
>> > > > > > > current
>> > > > > > > > > way,
>> > > > > > > > > > i.e.
>> > > > > > > > > > > > do not
>> > > > > > > > > > > > > > support such
>> > > > > > > > > > > > > > >> > message. It would
>> be
>> > > good
>> > > > to
>> > > > > know
>> > > > > > if
>> > > > > > > > > there is
>> > > > > > > > > > a
>> > > > > > > > > > > > > concrete use
>> > > > > > > > > > > > > > case for
>> > > > > > > > > > > > > > >> such
>> > > > > > > > > > > > > > >> > message. If there
>> is
>> > > not,
>> > > > we can
>> > > > > > > > > probably just
>> > > > > > > > > > > not
>> > > > > > > > > > > > > support it.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > Thanks,
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > JIangjie (Becket)
>> Qin
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > On Wed, Nov 16,
>> 2016
>> > at
>> > > > 1:28 PM,
>> > > > > > > > > Mayuresh
>> > > > > > > > > > > Gharat <
>> > > > > > > > > > > > > > >> >
>> > > > gharatmayuresh15@gmail.com
>> > > > > > > > > > > > > > >> > > wrote:
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > > Hi Ismael,
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > This is
>> something I
>> > > can
>> > > > think
>> > > > > of
>> > > > > > > for
>> > > > > > > > > > migration
>> > > > > > > > > > > > plan:
>> > > > > > > > > > > > > > >> > > So the migration
>> > plan
>> > > > can look
>> > > > > > > > > something
>> > > > > > > > > > like
>> > > > > > > > > > > > this,
>> > > > > > > > > > > > > with up
>> > > > > > > > > > > > > > >> conversion :
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > 1) Currently lets
>> > say
>> > > > we have
>> > > > > > > Broker
>> > > > > > > > > at
>> > > > > > > > > > > version
>> > > > > > > > > > > > x.
>> > > > > > > > > > > > > > >> > > 2) Currently we
>> have
>> > > > clients
>> > > > > at
>> > > > > > > > > version x.
>> > > > > > > > > > > > > > >> > > 3) a) We move the
>> > > > version to
>> > > > > > > > > Broker(x+1) :
>> > > > > > > > > > > > supports
>> > > > > > > > > > > > > both
>> > > > > > > > > > > > > > tombstone and
>> > > > > > > > > > > > > > >> > null
>> > > > > > > > > > > > > > >> > > for log
>> compaction.
>> > > > > > > > > > > > > > >> > > b) We upgrade
>> > the
>> > > > client
>> > > > > to
>> > > > > > > > > version
>> > > > > > > > > > > > client(x+1) :
>> > > > > > > > > > > > > if in
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > >> producer
>> > > > > > > > > > > > > > >> > > client(x+1) the
>> > value
>> > > > is set
>> > > > > to
>> > > > > > > > null,
>> > > > > > > > > we
>> > > > > > > > > > will
>> > > > > > > > > > > > > automatically
>> > > > > > > > > > > > > > set the
>> > > > > > > > > > > > > > >> > > Tombstone bit
>> > > > internally. If
>> > > > > the
>> > > > > > > > > producer
>> > > > > > > > > > > > client(x+1)
>> > > > > > > > > > > > > sets
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > >> tombstone
>> > > > > > > > > > > > > > >> > > itself, well and
>> > good.
>> > > > For
>> > > > > > > producer
>> > > > > > > > > > client(x),
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > broker
>> > > > > > > > > > > > > > will up
>> > > > > > > > > > > > > > >> convert
>> > > > > > > > > > > > > > >> > > to have the
>> > tombstone
>> > > > bit.
>> > > > > > > > > Broker(x+1) is
>> > > > > > > > > > > > supporting
>> > > > > > > > > > > > > both.
>> > > > > > > > > > > > > > Consumer
>> > > > > > > > > > > > > > >> > > client(x+1) will
>> be
>> > > > aware of
>> > > > > > this
>> > > > > > > > and
>> > > > > > > > > should
>> > > > > > > > > > > be
>> > > > > > > > > > > > able
>> > > > > > > > > > > > > to
>> > > > > > > > > > > > > > handle this.
>> > > > > > > > > > > > > > >> For
>> > > > > > > > > > > > > > >> > > consumer
>> client(x)
>> > we
>> > > > will
>> > > > > down
>> > > > > > > > > convert the
>> > > > > > > > > > > > message
>> > > > > > > > > > > > > on the
>> > > > > > > > > > > > > > broker
>> > > > > > > > > > > > > > >> side.
>> > > > > > > > > > > > > > >> > > c) At this
>> point
>> > > we
>> > > > will
>> > > > > > have
>> > > > > > > to
>> > > > > > > > > > specify a
>> > > > > > > > > > > > > warning or
>> > > > > > > > > > > > > > clearly
>> > > > > > > > > > > > > > >> specify
>> > > > > > > > > > > > > > >> > > in docs that this
>> > > > behavior is
>> > > > > > > about
>> > > > > > > > > to be
>> > > > > > > > > > > > changed for
>> > > > > > > > > > > > > log
>> > > > > > > > > > > > > > compaction.
>> > > > > > > > > > > > > > >> > > 4) a) In next
>> > release
>> > > > of the
>> > > > > > > > > Broker(x+2), we
>> > > > > > > > > > > > say that
>> > > > > > > > > > > > > only
>> > > > > > > > > > > > > > Tombstone
>> > > > > > > > > > > > > > >> is
>> > > > > > > > > > > > > > >> > > used for log
>> > > compaction
>> > > > on the
>> > > > > > > > Broker
>> > > > > > > > > side.
>> > > > > > > > > > > > > Clients(x+1)
>> > > > > > > > > > > > > > still is
>> > > > > > > > > > > > > > >> > > supported.
>> > > > > > > > > > > > > > >> > > b) We upgrade
>> > the
>> > > > client
>> > > > > to
>> > > > > > > > > version
>> > > > > > > > > > > > client(x+2) :
>> > > > > > > > > > > > > if
>> > > > > > > > > > > > > > value is set
>> > > > > > > > > > > > > > >> to
>> > > > > > > > > > > > > > >> > > null, tombstone
>> will
>> > > > not be
>> > > > > set
>> > > > > > > > > > automatically.
>> > > > > > > > > > > > The
>> > > > > > > > > > > > > client
>> > > > > > > > > > > > > > will have to
>> > > > > > > > > > > > > > >> > call
>> > > > > > > > > > > > > > >> > > setTombstone() to
>> > > > actually set
>> > > > > > the
>> > > > > > > > > > tombstone.
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > We should compare
>> > this
>> > > > > migration
>> > > > > > > > plan
>> > > > > > > > > with
>> > > > > > > > > > the
>> > > > > > > > > > > > > migration
>> > > > > > > > > > > > > > plan for
>> > > > > > > > > > > > > > >> magic
>> > > > > > > > > > > > > > >> > > byte bump and do
>> > > > whatever
>> > > > > looks
>> > > > > > > > good.
>> > > > > > > > > > > > > > >> > > I am just worried
>> > that
>> > > > if we
>> > > > > go
>> > > > > > > down
>> > > > > > > > > magic
>> > > > > > > > > > > byte
>> > > > > > > > > > > > route,
>> > > > > > > > > > > > > > unless I am
>> > > > > > > > > > > > > > >> > missing
>> > > > > > > > > > > > > > >> > > something, it
>> sounds
>> > > > like
>> > > > > kafka
>> > > > > > > will
>> > > > > > > > > be
>> > > > > > > > > > stuck
>> > > > > > > > > > > > with
>> > > > > > > > > > > > > > supporting both
>> > > > > > > > > > > > > > >> null
>> > > > > > > > > > > > > > >> > > value and
>> tombstone
>> > > bit
>> > > > for
>> > > > > log
>> > > > > > > > > compaction
>> > > > > > > > > > for
>> > > > > > > > > > > > life
>> > > > > > > > > > > > > long,
>> > > > > > > > > > > > > > which does
>> > > > > > > > > > > > > > >> not
>> > > > > > > > > > > > > > >> > > look like a good
>> end
>> > > > state.
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > Thanks,
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > Mayuresh
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > On Wed, Nov 16,
>> 2016
>> > > at
>> > > > 9:32
>> > > > > AM,
>> > > > > > > > > Mayuresh
>> > > > > > > > > > > > Gharat <
>> > > > > > > > > > > > > > >> > >
>> > > > gharatmayuresh15@gmail.com
>> > > > > > > > > > > > > > >> > > > wrote:
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > > Hi Ismael,
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > That's a very
>> good
>> > > > point
>> > > > > > which I
>> > > > > > > > > might
>> > > > > > > > > > have
>> > > > > > > > > > > > not
>> > > > > > > > > > > > > > considered earlier.
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > Here is a plan
>> > that
>> > > I
>> > > > can
>> > > > > > think
>> > > > > > > > of:
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > Stage 1) The
>> > broker
>> > > > from now
>> > > > > > on,
>> > > > > > > > up
>> > > > > > > > > > converts
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > message
>> > > > > > > > > > > > > > to have the
>> > > > > > > > > > > > > > >> > > > tombstone
>> marker.
>> > > The
>> > > > log
>> > > > > > > > compaction
>> > > > > > > > > > thread
>> > > > > > > > > > > > does log
>> > > > > > > > > > > > > > compaction
>> > > > > > > > > > > > > > >> based
>> > > > > > > > > > > > > > >> > on
>> > > > > > > > > > > > > > >> > > > both null and
>> > > > tombstone
>> > > > > > marker.
>> > > > > > > > > This is
>> > > > > > > > > > our
>> > > > > > > > > > > > > transition
>> > > > > > > > > > > > > > period.
>> > > > > > > > > > > > > > >> > > > Stage 2) The
>> next
>> > > > release we
>> > > > > > > only
>> > > > > > > > > say that
>> > > > > > > > > > > log
>> > > > > > > > > > > > > compaction
>> > > > > > > > > > > > > > is based
>> > > > > > > > > > > > > > >> on
>> > > > > > > > > > > > > > >> > > > tombstone
>> marker.
>> > > > (Open
>> > > > > source
>> > > > > > > > > kafka makes
>> > > > > > > > > > > > this as a
>> > > > > > > > > > > > > > policy). By
>> > > > > > > > > > > > > > >> this
>> > > > > > > > > > > > > > >> > > time,
>> > > > > > > > > > > > > > >> > > > the
>> organization
>> > > > which is
>> > > > > > moving
>> > > > > > > > to
>> > > > > > > > > this
>> > > > > > > > > > > > release
>> > > > > > > > > > > > > will be
>> > > > > > > > > > > > > > sure that
>> > > > > > > > > > > > > > >> they
>> > > > > > > > > > > > > > >> > > > have gone
>> through
>> > > the
>> > > > entire
>> > > > > > > > > transition
>> > > > > > > > > > > > period.
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > My only goal of
>> > > doing
>> > > > this
>> > > > > is
>> > > > > > > that
>> > > > > > > > > Kafka
>> > > > > > > > > > > > clearly
>> > > > > > > > > > > > > > specifies the end
>> > > > > > > > > > > > > > >> > state
>> > > > > > > > > > > > > > >> > > > about what log
>> > > > compaction
>> > > > > > means
>> > > > > > > > (is
>> > > > > > > > > it
>> > > > > > > > > > null
>> > > > > > > > > > > > value
>> > > > > > > > > > > > > or a
>> > > > > > > > > > > > > > tombstone
>> > > > > > > > > > > > > > >> > marker,
>> > > > > > > > > > > > > > >> > > > but not both).
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > What do you
>> think?
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > Thanks,
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > Mayuresh
>> > > > > > > > > > > > > > >> > > > .
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > On Wed, Nov 16,
>> > 2016
>> > > > at 9:17
>> > > > > > AM,
>> > > > > > > > > Ismael
>> > > > > > > > > > > Juma <
>> > > > > > > > > > > > > > ismael@juma.me.uk>
>> > > > > > > > > > > > > > >> > wrote:
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > >> One comment
>> > below.
>> > > > > > > > > > > > > > >> > > >>
>> > > > > > > > > > > > > > >> > > >> On Wed, Nov
>> 16,
>> > > 2016
>> > > > at
>> > > > > 5:08
>> > > > > > > PM,
>> > > > > > > > > Mayuresh
>> > > > > > > > > > > > Gharat <
>> > > > > > > > > > > > > > >> > > >>
>> > > > gharatmayuresh15@gmail.com
>> > > > > > > > > > > > > > >> > > >> > wrote:
>> > > > > > > > > > > > > > >> > > >>
>> > > > > > > > > > > > > > >> > > >> > - If we
>> > don't
>> > > > bump up
>> > > > > > the
>> > > > > > > > > magic
>> > > > > > > > > > byte,
>> > > > > > > > > > > > on the
>> > > > > > > > > > > > > broker
>> > > > > > > > > > > > > > side, the
>> > > > > > > > > > > > > > >> > > broker
>> > > > > > > > > > > > > > >> > > >> > will
>> always
>> > > > have to
>> > > > > look
>> > > > > > > at
>> > > > > > > > > both
>> > > > > > > > > > > > tombstone
>> > > > > > > > > > > > > bit and
>> > > > > > > > > > > > > > the value
>> > > > > > > > > > > > > > >> when
>> > > > > > > > > > > > > > >> > > do
>> > > > > > > > > > > > > > >> > > >> the
>> > > > > > > > > > > > > > >> > > >> >
>> compaction.
>> > > > Assuming
>> > > > > we
>> > > > > > do
>> > > > > > > > > not bump
>> > > > > > > > > > up
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > magic
>> > > > > > > > > > > > > > byte,
>> > > > > > > > > > > > > > >> > > >> > imagine
>> the
>> > > > broker
>> > > > > sees
>> > > > > > a
>> > > > > > > > > message
>> > > > > > > > > > > which
>> > > > > > > > > > > > does
>> > > > > > > > > > > > > not
>> > > > > > > > > > > > > > have a
>> > > > > > > > > > > > > > >> tombstone
>> > > > > > > > > > > > > > >> > > bit
>> > > > > > > > > > > > > > >> > > >> > set. The
>> > > broker
>> > > > does
>> > > > > not
>> > > > > > > > know
>> > > > > > > > > when
>> > > > > > > > > > the
>> > > > > > > > > > > > > message was
>> > > > > > > > > > > > > > produced
>> > > > > > > > > > > > > > >> (i.e.
>> > > > > > > > > > > > > > >> > > >> > whether
>> > > > > > > > > > > > > > >> > > >> > the
>> message
>> > > has
>> > > > been
>> > > > > up
>> > > > > > > > > converted or
>> > > > > > > > > > > > not), it
>> > > > > > > > > > > > > has
>> > > > > > > > > > > > > > to take a
>> > > > > > > > > > > > > > >> > further
>> > > > > > > > > > > > > > >> > > >> > look at
>> > > > > > > > > > > > > > >> > > >> > the
>> value to
>> > > > see if it
>> > > > > > is
>> > > > > > > > > null or
>> > > > > > > > > > not
>> > > > > > > > > > > in
>> > > > > > > > > > > > > order to
>> > > > > > > > > > > > > > determine
>> > > > > > > > > > > > > > >> if it
>> > > > > > > > > > > > > > >> > > is
>> > > > > > > > > > > > > > >> > > >> a
>> > > > > > > > > > > > > > >> > > >> >
>> tombstone.
>> > The
>> > > > same
>> > > > > > logic
>> > > > > > > > has
>> > > > > > > > > to be
>> > > > > > > > > > > put
>> > > > > > > > > > > > on the
>> > > > > > > > > > > > > > consumer as
>> > > > > > > > > > > > > > >> well
>> > > > > > > > > > > > > > >> > > >> because
>> > > > > > > > > > > > > > >> > > >> > the
>> > > > > > > > > > > > > > >> > > >> > consumer
>> > does
>> > > > not know
>> > > > > > if
>> > > > > > > > the
>> > > > > > > > > > message
>> > > > > > > > > > > > has
>> > > > > > > > > > > > > been up
>> > > > > > > > > > > > > > converted or
>> > > > > > > > > > > > > > >> > not.
>> > > > > > > > > > > > > > >> > > >> > - If
>> we
>> > > > upconvert
>> > > > > > while
>> > > > > > > > > > appending,
>> > > > > > > > > > > > this is
>> > > > > > > > > > > > > not
>> > > > > > > > > > > > > > the case,
>> > > > > > > > > > > > > > >> > right?
>> > > > > > > > > > > > > > >> > > >>
>> > > > > > > > > > > > > > >> > > >>
>> > > > > > > > > > > > > > >> > > >> If I
>> understand
>> > you
>> > > > > > correctly,
>> > > > > > > > > this is
>> > > > > > > > > > not
>> > > > > > > > > > > > > sufficient
>> > > > > > > > > > > > > > because the
>> > > > > > > > > > > > > > >> log
>> > > > > > > > > > > > > > >> > > may
>> > > > > > > > > > > > > > >> > > >> have messages
>> > > > appended
>> > > > > before
>> > > > > > > it
>> > > > > > > > > was
>> > > > > > > > > > > > upgraded to
>> > > > > > > > > > > > > include
>> > > > > > > > > > > > > > KIP-87.
>> > > > > > > > > > > > > > >> > > >>
>> > > > > > > > > > > > > > >> > > >> Ismael
>> > > > > > > > > > > > > > >> > > >>
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > --
>> > > > > > > > > > > > > > >> > > > -Regards,
>> > > > > > > > > > > > > > >> > > > Mayuresh R.
>> Gharat
>> > > > > > > > > > > > > > >> > > > (862) 250-7125
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > --
>> > > > > > > > > > > > > > >> > > -Regards,
>> > > > > > > > > > > > > > >> > > Mayuresh R.
>> Gharat
>> > > > > > > > > > > > > > >> > > (862) 250-7125
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > The information
>> > > contained
>> > > > in
>> > > > > this
>> > > > > > > > email
>> > > > > > > > > is
>> > > > > > > > > > > > strictly
>> > > > > > > > > > > > > > confidential and for
>> > > > > > > > > > > > > > >> > the use of the
>> > addressee
>> > > > only,
>> > > > > > > unless
>> > > > > > > > > > otherwise
>> > > > > > > > > > > > > indicated. If
>> > > > > > > > > > > > > > you are
>> > > > > > > > > > > > > > >> not
>> > > > > > > > > > > > > > >> > the intended
>> > recipient,
>> > > > please
>> > > > > do
>> > > > > > > not
>> > > > > > > > > read,
>> > > > > > > > > > > copy,
>> > > > > > > > > > > > use or
>> > > > > > > > > > > > > > disclose to
>> > > > > > > > > > > > > > >> others
>> > > > > > > > > > > > > > >> > this message or any
>> > > > attachment.
>> > > > > > > Please
>> > > > > > > > > also
>> > > > > > > > > > > > notify the
>> > > > > > > > > > > > > sender
>> > > > > > > > > > > > > > by
>> > > > > > > > > > > > > > >> replying
>> > > > > > > > > > > > > > >> > to this email or by
>> > > > telephone
>> > > > > > > (+44(020
>> > > > > > > > > 7896
>> > > > > > > > > > > 0011)
>> > > > > > > > > > > > and
>> > > > > > > > > > > > > then
>> > > > > > > > > > > > > > delete the
>> > > > > > > > > > > > > > >> email
>> > > > > > > > > > > > > > >> > and any copies of
>> it.
>> > > > Opinions,
>> > > > > > > > > conclusion
>> > > > > > > > > > (etc)
>> > > > > > > > > > > > that
>> > > > > > > > > > > > > do not
>> > > > > > > > > > > > > > relate to
>> > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > >> > official business
>> of
>> > > this
>> > > > > company
>> > > > > > > > shall
>> > > > > > > > > be
>> > > > > > > > > > > > understood as
>> > > > > > > > > > > > > > neither given
>> > > > > > > > > > > > > > >> nor
>> > > > > > > > > > > > > > >> > endorsed by it. IG
>> is
>> > a
>> > > > trading
>> > > > > > name
>> > > > > > > > of
>> > > > > > > > > IG
>> > > > > > > > > > > Markets
>> > > > > > > > > > > > > Limited (a
>> > > > > > > > > > > > > > company
>> > > > > > > > > > > > > > >> > registered in
>> England
>> > > and
>> > > > Wales,
>> > > > > > > > company
>> > > > > > > > > > number
>> > > > > > > > > > > > > 04008957) and
>> > > > > > > > > > > > > > IG Index
>> > > > > > > > > > > > > > >> > Limited (a company
>> > > > registered in
>> > > > > > > > > England and
>> > > > > > > > > > > > Wales,
>> > > > > > > > > > > > > company
>> > > > > > > > > > > > > > number
>> > > > > > > > > > > > > > >> > 01190902).
>> Registered
>> > > > address at
>> > > > > > > > Cannon
>> > > > > > > > > Bridge
>> > > > > > > > > > > > House, 25
>> > > > > > > > > > > > > > Dowgate Hill,
>> > > > > > > > > > > > > > >> > London EC4R 2YA.
>> Both
>> > IG
>> > > > Markets
>> > > > > > > > Limited
>> > > > > > > > > > > (register
>> > > > > > > > > > > > > number
>> > > > > > > > > > > > > > 195355) and IG
>> > > > > > > > > > > > > > >> > Index Limited
>> > (register
>> > > > number
>> > > > > > > 114059)
>> > > > > > > > > are
>> > > > > > > > > > > > authorised
>> > > > > > > > > > > > > and
>> > > > > > > > > > > > > > regulated by
>> > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > >> > Financial Conduct
>> > > > Authority.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> --
>> > > > > > > > > > > > > > >> -Regards,
>> > > > > > > > > > > > > > >> Mayuresh R. Gharat
>> > > > > > > > > > > > > > >> (862) 250-7125
>> > > > > > > > > > > > > > >> The information
>> > contained
>> > > > in this
>> > > > > > > email
>> > > > > > > > is
>> > > > > > > > > > > strictly
>> > > > > > > > > > > > > > confidential and for
>> > > > > > > > > > > > > > >> the use of the
>> addressee
>> > > > only,
>> > > > > > unless
>> > > > > > > > > otherwise
>> > > > > > > > > > > > > indicated. If
>> > > > > > > > > > > > > > you are not
>> > > > > > > > > > > > > > >> the intended
>> recipient,
>> > > > please do
>> > > > > > not
>> > > > > > > > > read,
>> > > > > > > > > > copy,
>> > > > > > > > > > > > use or
>> > > > > > > > > > > > > > disclose to others
>> > > > > > > > > > > > > > >> this message or any
>> > > > attachment.
>> > > > > > Please
>> > > > > > > > > also
>> > > > > > > > > > notify
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > sender
>> > > > > > > > > > > > > > by replying
>> > > > > > > > > > > > > > >> to this email or by
>> > > > telephone
>> > > > > > (+44(020
>> > > > > > > > > 7896
>> > > > > > > > > > 0011)
>> > > > > > > > > > > > and then
>> > > > > > > > > > > > > > delete the email
>> > > > > > > > > > > > > > >> and any copies of it.
>> > > > Opinions,
>> > > > > > > > > conclusion (etc)
>> > > > > > > > > > > > that do
>> > > > > > > > > > > > > not
>> > > > > > > > > > > > > > relate to the
>> > > > > > > > > > > > > > >> official business of
>> > this
>> > > > company
>> > > > > > > shall
>> > > > > > > > be
>> > > > > > > > > > > > understood as
>> > > > > > > > > > > > > > neither given nor
>> > > > > > > > > > > > > > >> endorsed by it. IG
>> is a
>> > > > trading
>> > > > > name
>> > > > > > > of
>> > > > > > > > IG
>> > > > > > > > > > Markets
>> > > > > > > > > > > > > Limited (a
>> > > > > > > > > > > > > > company
>> > > > > > > > > > > > > > >> registered in England
>> > and
>> > > > Wales,
>> > > > > > > company
>> > > > > > > > > number
>> > > > > > > > > > > > 04008957)
>> > > > > > > > > > > > > and
>> > > > > > > > > > > > > > IG Index
>> > > > > > > > > > > > > > >> Limited (a company
>> > > > registered in
>> > > > > > > England
>> > > > > > > > > and
>> > > > > > > > > > > Wales,
>> > > > > > > > > > > > > company
>> > > > > > > > > > > > > > number
>> > > > > > > > > > > > > > >> 01190902). Registered
>> > > > address at
>> > > > > > > Cannon
>> > > > > > > > > Bridge
>> > > > > > > > > > > > House, 25
>> > > > > > > > > > > > > > Dowgate Hill,
>> > > > > > > > > > > > > > >> London EC4R 2YA.
>> Both IG
>> > > > Markets
>> > > > > > > Limited
>> > > > > > > > > > (register
>> > > > > > > > > > > > number
>> > > > > > > > > > > > > > 195355) and IG
>> > > > > > > > > > > > > > >> Index Limited
>> (register
>> > > > number
>> > > > > > 114059)
>> > > > > > > > are
>> > > > > > > > > > > > authorised and
>> > > > > > > > > > > > > > regulated by the
>> > > > > > > > > > > > > > >> Financial Conduct
>> > > Authority.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > --
>> > > > > > > > > > > > > > > -Regards,
>> > > > > > > > > > > > > > > Mayuresh R. Gharat
>> > > > > > > > > > > > > > > (862) 250-7125
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > --
>> > > > > > > > > > > > > > -Regards,
>> > > > > > > > > > > > > > Mayuresh R. Gharat
>> > > > > > > > > > > > > > (862) 250-7125
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > The information contained in
>> > this
>> > > > email
>> > > > > is
>> > > > > > > > > strictly
>> > > > > > > > > > > > confidential
>> > > > > > > > > > > > > and
>> > > > > > > > > > > > > > for the use of the addressee
>> only,
>> > > > unless
>> > > > > > > otherwise
>> > > > > > > > > > > indicated.
>> > > > > > > > > > > > If
>> > > > > > > > > > > > > you are
>> > > > > > > > > > > > > > not the intended recipient,
>> please
>> > do
>> > > > not
>> > > > > read,
>> > > > > > > > > copy, use
>> > > > > > > > > > or
>> > > > > > > > > > > > > disclose to
>> > > > > > > > > > > > > > others this message or any
>> > > attachment.
>> > > > Please
>> > > > > > > also
>> > > > > > > > > notify
>> > > > > > > > > > the
>> > > > > > > > > > > > sender
>> > > > > > > > > > > > > by
>> > > > > > > > > > > > > > replying to this email or by
>> > > telephone
>> > > > > (+44(020
>> > > > > > > > 7896
>> > > > > > > > > 0011)
>> > > > > > > > > > > and
>> > > > > > > > > > > > then
>> > > > > > > > > > > > > delete
>> > > > > > > > > > > > > > the email and any copies of it.
>> > > > Opinions,
>> > > > > > > > conclusion
>> > > > > > > > > (etc)
>> > > > > > > > > > > > that do
>> > > > > > > > > > > > > not
>> > > > > > > > > > > > > > relate to the official business
>> of
>> > > this
>> > > > > company
>> > > > > > > > > shall be
>> > > > > > > > > > > > understood
>> > > > > > > > > > > > > as
>> > > > > > > > > > > > > > neither given nor endorsed by
>> it.
>> > IG
>> > > > is a
>> > > > > > trading
>> > > > > > > > > name of
>> > > > > > > > > > IG
>> > > > > > > > > > > > Markets
>> > > > > > > > > > > > > > Limited (a company registered in
>> > > > England and
>> > > > > > > Wales,
>> > > > > > > > > company
>> > > > > > > > > > > > number
>> > > > > > > > > > > > > > 04008957) and IG Index Limited
>> (a
>> > > > company
>> > > > > > > > registered
>> > > > > > > > > in
>> > > > > > > > > > > > England and
>> > > > > > > > > > > > > Wales,
>> > > > > > > > > > > > > > company number 01190902).
>> > Registered
>> > > > address
>> > > > > at
>> > > > > > > > > Cannon
>> > > > > > > > > > Bridge
>> > > > > > > > > > > > House,
>> > > > > > > > > > > > > 25
>> > > > > > > > > > > > > > Dowgate Hill, London EC4R 2YA.
>> Both
>> > > IG
>> > > > > Markets
>> > > > > > > > > Limited
>> > > > > > > > > > > > (register
>> > > > > > > > > > > > > number
>> > > > > > > > > > > > > > 195355) and IG Index Limited
>> > > (register
>> > > > number
>> > > > > > > > > 114059) are
>> > > > > > > > > > > > authorised
>> > > > > > > > > > > > > and
>> > > > > > > > > > > > > > regulated by the Financial
>> Conduct
>> > > > Authority.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > The information contained in this
>> email
>> > is
>> > > > strictly
>> > > > > > > > > confidential
>> > > > > > > > > > > and
>> > > > > > > > > > > > for
>> > > > > > > > > > > > > the use of the addressee only, unless
>> > > > otherwise
>> > > > > > > > indicated.
>> > > > > > > > > If you
>> > > > > > > > > > > > are not
>> > > > > > > > > > > > > the intended recipient, please do not
>> > read,
>> > > > copy,
>> > > > > use
>> > > > > > > or
>> > > > > > > > > disclose
>> > > > > > > > > > > to
>> > > > > > > > > > > > others
>> > > > > > > > > > > > > this message or any attachment. Please
>> > also
>> > > > notify
>> > > > > > the
>> > > > > > > > > sender by
>> > > > > > > > > > > > replying
>> > > > > > > > > > > > > to this email or by telephone (+44(020
>> > 7896
>> > > > 0011
>> > > > > <020%207896%200011>) and
>> > > > > > > > then
>> > > > > > > > > delete
>> > > > > > > > > > > > the email
>> > > > > > > > > > > > > and any copies of it. Opinions,
>> > conclusion
>> > > > (etc)
>> > > > > that
>> > > > > > > do
>> > > > > > > > > not
>> > > > > > > > > > relate
>> > > > > > > > > > > > to the
>> > > > > > > > > > > > > official business of this company
>> shall
>> > be
>> > > > > understood
>> > > > > > > as
>> > > > > > > > > neither
>> > > > > > > > > > > > given nor
>> > > > > > > > > > > > > endorsed by it. IG is a trading name
>> of
>> > IG
>> > > > Markets
>> > > > > > > > Limited
>> > > > > > > > > (a
>> > > > > > > > > > > company
>> > > > > > > > > > > > > registered in England and Wales,
>> company
>> > > > number
>> > > > > > > 04008957)
>> > > > > > > > > and IG
>> > > > > > > > > > > > Index
>> > > > > > > > > > > > > Limited (a company registered in
>> England
>> > > and
>> > > > Wales,
>> > > > > > > > company
>> > > > > > > > > > number
>> > > > > > > > > > > > > 01190902). Registered address at
>> Cannon
>> > > > Bridge
>> > > > > House,
>> > > > > > > 25
>> > > > > > > > > Dowgate
>> > > > > > > > > > > > Hill,
>> > > > > > > > > > > > > London EC4R 2YA. Both IG Markets
>> Limited
>> > > > (register
>> > > > > > > number
>> > > > > > > > > 195355)
>> > > > > > > > > > > > and IG
>> > > > > > > > > > > > > Index Limited (register number 114059)
>> > are
>> > > > > authorised
>> > > > > > > and
>> > > > > > > > > > regulated
>> > > > > > > > > > > > by the
>> > > > > > > > > > > > > Financial Conduct Authority.
>> > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > The information contained in this email is
>> > > strictly
>> > > > > > > > confidential
>> > > > > > > > > and
>> > > > > > > > > > for
>> > > > > > > > > > > > the use of the addressee only, unless
>> otherwise
>> > > > > indicated.
>> > > > > > If
>> > > > > > > > > you are
>> > > > > > > > > > not
>> > > > > > > > > > > > the intended recipient, please do not read,
>> > copy,
>> > > > use or
>> > > > > > > > > disclose to
>> > > > > > > > > > > others
>> > > > > > > > > > > > this message or any attachment. Please also
>> > > notify
>> > > > the
>> > > > > > sender
>> > > > > > > > by
>> > > > > > > > > > replying
>> > > > > > > > > > > > to this email or by telephone (+44(020 7896
>> > 0011
>> > > > > <020%207896%200011>) and then
>> > > > > > > > > delete the
>> > > > > > > > > > > email
>> > > > > > > > > > > > and any copies of it. Opinions, conclusion
>> > (etc)
>> > > > that do
>> > > > > > not
>> > > > > > > > > relate to
>> > > > > > > > > > > the
>> > > > > > > > > > > > official business of this company shall be
>> > > > understood as
>> > > > > > > > neither
>> > > > > > > > > given
>> > > > > > > > > > > nor
>> > > > > > > > > > > > endorsed by it. IG is a trading name of IG
>> > > Markets
>> > > > > Limited
>> > > > > > (a
>> > > > > > > > > company
>> > > > > > > > > > > > registered in England and Wales, company
>> number
>> > > > 04008957)
>> > > > > > and
>> > > > > > > > IG
>> > > > > > > > > Index
>> > > > > > > > > > > > Limited (a company registered in England and
>> > > Wales,
>> > > > > company
>> > > > > > > > > number
>> > > > > > > > > > > > 01190902). Registered address at Cannon
>> Bridge
>> > > > House, 25
>> > > > > > > > Dowgate
>> > > > > > > > > Hill,
>> > > > > > > > > > > > London EC4R 2YA. Both IG Markets Limited
>> > > (register
>> > > > number
>> > > > > > > > > 195355) and
>> > > > > > > > > > IG
>> > > > > > > > > > > > Index Limited (register number 114059) are
>> > > > authorised and
>> > > > > > > > > regulated by
>> > > > > > > > > > > the
>> > > > > > > > > > > > Financial Conduct Authority.
>> > > > > > > > > > > >
>> > > > > > > > > > > The information contained in this email is
>> > strictly
>> > > > > > > confidential
>> > > > > > > > > and for
>> > > > > > > > > > > the use of the addressee only, unless
>> otherwise
>> > > > indicated.
>> > > > > If
>> > > > > > > you
>> > > > > > > > > are not
>> > > > > > > > > > > the intended recipient, please do not read,
>> copy,
>> > > > use or
>> > > > > > > disclose
>> > > > > > > > > to
>> > > > > > > > > > others
>> > > > > > > > > > > this message or any attachment. Please also
>> > notify
>> > > > the
>> > > > > sender
>> > > > > > > by
>> > > > > > > > > replying
>> > > > > > > > > > > to this email or by telephone (+44(020 7896
>> 0011
>> > > > > <020%207896%200011>) and then
>> > > > > > > delete
>> > > > > > > > > the
>> > > > > > > > > > email
>> > > > > > > > > > > and any copies of it. Opinions, conclusion
>> (etc)
>> > > > that do
>> > > > > not
>> > > > > > > > > relate to
>> > > > > > > > > > the
>> > > > > > > > > > > official business of this company shall be
>> > > > understood as
>> > > > > > > neither
>> > > > > > > > > given
>> > > > > > > > > > nor
>> > > > > > > > > > > endorsed by it. IG is a trading name of IG
>> > Markets
>> > > > Limited
>> > > > > (a
>> > > > > > > > > company
>> > > > > > > > > > > registered in England and Wales, company
>> number
>> > > > 04008957)
>> > > > > and
>> > > > > > > IG
>> > > > > > > > > Index
>> > > > > > > > > > > Limited (a company registered in England and
>> > Wales,
>> > > > company
>> > > > > > > > number
>> > > > > > > > > > > 01190902). Registered address at Cannon Bridge
>> > > > House, 25
>> > > > > > > Dowgate
>> > > > > > > > > Hill,
>> > > > > > > > > > > London EC4R 2YA. Both IG Markets Limited
>> > (register
>> > > > number
>> > > > > > > 195355)
>> > > > > > > > > and IG
>> > > > > > > > > > > Index Limited (register number 114059) are
>> > > > authorised and
>> > > > > > > > > regulated by
>> > > > > > > > > > the
>> > > > > > > > > > > Financial Conduct Authority.
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > The information contained in this email is
>> strictly
>> > > > > > confidential
>> > > > > > > > and
>> > > > > > > > > for
>> > > > > > > > > > the use of the addressee only, unless otherwise
>> > > > indicated. If
>> > > > > > you
>> > > > > > > > > are not
>> > > > > > > > > > the intended recipient, please do not read,
>> copy,
>> > use
>> > > > or
>> > > > > > disclose
>> > > > > > > > to
>> > > > > > > > > others
>> > > > > > > > > > this message or any attachment. Please also
>> notify
>> > > the
>> > > > sender
>> > > > > > by
>> > > > > > > > > replying
>> > > > > > > > > > to this email or by telephone (+44(020 7896 0011
>> > > > > <020%207896%200011>) and then
>> > > > > > delete
>> > > > > > > > > the email
>> > > > > > > > > > and any copies of it. Opinions, conclusion (etc)
>> > that
>> > > > do not
>> > > > > > > relate
>> > > > > > > > > to the
>> > > > > > > > > > official business of this company shall be
>> > understood
>> > > > as
>> > > > > > neither
>> > > > > > > > > given nor
>> > > > > > > > > > endorsed by it. IG is a trading name of IG
>> Markets
>> > > > Limited (a
>> > > > > > > > company
>> > > > > > > > > > registered in England and Wales, company number
>> > > > 04008957) and
>> > > > > > IG
>> > > > > > > > > Index
>> > > > > > > > > > Limited (a company registered in England and
>> Wales,
>> > > > company
>> > > > > > > number
>> > > > > > > > > > 01190902). Registered address at Cannon Bridge
>> > House,
>> > > > 25
>> > > > > > Dowgate
>> > > > > > > > > Hill,
>> > > > > > > > > > London EC4R 2YA. Both IG Markets Limited
>> (register
>> > > > number
>> > > > > > 195355)
>> > > > > > > > > and IG
>> > > > > > > > > > Index Limited (register number 114059) are
>> > authorised
>> > > > and
>> > > > > > > regulated
>> > > > > > > > > by the
>> > > > > > > > > > Financial Conduct Authority.
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > The information contained in this email is strictly
>> > > > confidential
>> > > > > and
>> > > > > > > for
>> > > > > > > > > the use of the addressee only, unless otherwise
>> > indicated.
>> > > > If you
>> > > > > are
>> > > > > > > not
>> > > > > > > > > the intended recipient, please do not read, copy, use
>> or
>> > > > disclose
>> > > > > to
>> > > > > > > > others
>> > > > > > > > > this message or any attachment. Please also notify the
>> > > > sender by
>> > > > > > > replying
>> > > > > > > > > to this email or by telephone (+44(020 7896 0011
>> > > > > <020%207896%200011>) and then delete the
>> > > > > > > > email
>> > > > > > > > > and any copies of it. Opinions, conclusion (etc) that
>> do
>> > > not
>> > > > relate
>> > > > > > to
>> > > > > > > > the
>> > > > > > > > > official business of this company shall be understood
>> as
>> > > > neither
>> > > > > > given
>> > > > > > > > nor
>> > > > > > > > > endorsed by it. IG is a trading name of IG Markets
>> > Limited
>> > > (a
>> > > > > company
>> > > > > > > > > registered in England and Wales, company number
>> 04008957)
>> > > > and IG
>> > > > > > Index
>> > > > > > > > > Limited (a company registered in England and Wales,
>> > company
>> > > > number
>> > > > > > > > > 01190902). Registered address at Cannon Bridge House,
>> 25
>> > > > Dowgate
>> > > > > > Hill,
>> > > > > > > > > London EC4R 2YA. Both IG Markets Limited (register
>> number
>> > > > 195355)
>> > > > > and
>> > > > > > > IG
>> > > > > > > > > Index Limited (register number 114059) are authorised
>> and
>> > > > regulated
>> > > > > > by
>> > > > > > > > the
>> > > > > > > > > Financial Conduct Authority.
>> > > > > > > > >
>> > > > > > > > The information contained in this email is strictly
>> > > > confidential and
>> > > > > > for
>> > > > > > > > the use of the addressee only, unless otherwise
>> indicated.
>> > If
>> > > > you are
>> > > > > > not
>> > > > > > > > the intended recipient, please do not read, copy, use or
>> > > > disclose to
>> > > > > > > others
>> > > > > > > > this message or any attachment. Please also notify the
>> > sender
>> > > > by
>> > > > > > replying
>> > > > > > > > to this email or by telephone (+44(020 7896 0011
>> > > > <020%207896%200011>)
>> > > > > and then delete the
>> > > > > > > email
>> > > > > > > > and any copies of it. Opinions, conclusion (etc) that do
>> > not
>> > > > relate
>> > > > > to
>> > > > > > > the
>> > > > > > > > official business of this company shall be understood as
>> > > > neither
>> > > > > given
>> > > > > > > nor
>> > > > > > > > endorsed by it. IG is a trading name of IG Markets
>> Limited
>> > (a
>> > > > company
>> > > > > > > > registered in England and Wales, company number
>> 04008957)
>> > and
>> > > > IG
>> > > > > Index
>> > > > > > > > Limited (a company registered in England and Wales,
>> company
>> > > > number
>> > > > > > > > 01190902). Registered address at Cannon Bridge House, 25
>> > > > Dowgate
>> > > > > Hill,
>> > > > > > > > London EC4R 2YA. Both IG Markets Limited (register
>> number
>> > > > 195355) and
>> > > > > > IG
>> > > > > > > > Index Limited (register number 114059) are authorised
>> and
>> > > > regulated
>> > > > > by
>> > > > > > > the
>> > > > > > > > Financial Conduct Authority.
>> > > > > > > >
>> > > > > > > The information contained in this email is strictly
>> > > confidential
>> > > > and
>> > > > > for
>> > > > > > > the use of the addressee only, unless otherwise
>> indicated. If
>> > > > you are
>> > > > > not
>> > > > > > > the intended recipient, please do not read, copy, use or
>> > > > disclose to
>> > > > > > others
>> > > > > > > this message or any attachment. Please also notify the
>> sender
>> > > by
>> > > > > replying
>> > > > > > > to this email or by telephone (+44(020 7896 0011
>> > > > <020%207896%200011>)
>> > > > > and then delete the
>> > > > > > email
>> > > > > > > and any copies of it. Opinions, conclusion (etc) that do
>> not
>> > > > relate to
>> > > > > > the
>> > > > > > > official business of this company shall be understood as
>> > > neither
>> > > > given
>> > > > > > nor
>> > > > > > > endorsed by it. IG is a trading name of IG Markets
>> Limited (a
>> > > > company
>> > > > > > > registered in England and Wales, company number 04008957)
>> and
>> > > IG
>> > > > Index
>> > > > > > > Limited (a company registered in England and Wales,
>> company
>> > > > number
>> > > > > > > 01190902). Registered address at Cannon Bridge House, 25
>> > > Dowgate
>> > > > Hill,
>> > > > > > > London EC4R 2YA. Both IG Markets Limited (register number
>> > > > 195355) and
>> > > > > IG
>> > > > > > > Index Limited (register number 114059) are authorised and
>> > > > regulated by
>> > > > > > the
>> > > > > > > Financial Conduct Authority.
>> > > > > > >
>> > > > > > The information contained in this email is strictly
>> > confidential
>> > > > and for
>> > > > > > the use of the addressee only, unless otherwise indicated.
>> If
>> > you
>> > > > are not
>> > > > > > the intended recipient, please do not read, copy, use or
>> > disclose
>> > > > to
>> > > > > others
>> > > > > > this message or any attachment. Please also notify the
>> sender
>> > by
>> > > > replying
>> > > > > > to this email or by telephone (+44(020 7896 0011
>> > > > <020%207896%200011>)
>> > > > > and then delete the email
>> > > > > > and any copies of it. Opinions, conclusion (etc) that do not
>> > > > relate to
>> > > > > the
>> > > > > > official business of this company shall be understood as
>> > neither
>> > > > given
>> > > > > nor
>> > > > > > endorsed by it. IG is a trading name of IG Markets Limited
>> (a
>> > > > company
>> > > > > > registered in England and Wales, company number 04008957)
>> and
>> > IG
>> > > > Index
>> > > > > > Limited (a company registered in England and Wales, company
>> > > number
>> > > > > > 01190902). Registered address at Cannon Bridge House, 25
>> > Dowgate
>> > > > Hill,
>> > > > > > London EC4R 2YA. Both IG Markets Limited (register number
>> > 195355)
>> > > > and IG
>> > > > > > Index Limited (register number 114059) are authorised and
>> > > > regulated by
>> > > > > the
>> > > > > > Financial Conduct Authority.
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > > The information contained in this email is strictly confidential and
>> > for
>> > > > the use of the addressee only, unless otherwise indicated. If you
>> are
>> > not
>> > > > the intended recipient, please do not read, copy, use or disclose to
>> > > others
>> > > > this message or any attachment. Please also notify the sender by
>> > replying
>> > > > to this email or by telephone (+44(020 7896 0011) and then delete
>> the
>> > > email
>> > > > and any copies of it. Opinions, conclusion (etc) that do not relate
>> to
>> > > the
>> > > > official business of this company shall be understood as neither
>> given
>> > > nor
>> > > > endorsed by it. IG is a trading name of IG Markets Limited (a
>> company
>> > > > registered in England and Wales, company number 04008957) and IG
>> Index
>> > > > Limited (a company registered in England and Wales, company number
>> > > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate
>> Hill,
>> > > > London EC4R 2YA. Both IG Markets Limited (register number 195355)
>> and
>> > IG
>> > > > Index Limited (register number 114059) are authorised and regulated
>> by
>> > > the
>> > > > Financial Conduct Authority.
>> > > >
>> > > The information contained in this email is strictly confidential and
>> for
>> > > the use of the addressee only, unless otherwise indicated. If you are
>> not
>> > > the intended recipient, please do not read, copy, use or disclose to
>> > others
>> > > this message or any attachment. Please also notify the sender by
>> replying
>> > > to this email or by telephone (+44(020 7896 0011) and then delete the
>> > email
>> > > and any copies of it. Opinions, conclusion (etc) that do not relate to
>> > the
>> > > official business of this company shall be understood as neither given
>> > nor
>> > > endorsed by it. IG is a trading name of IG Markets Limited (a company
>> > > registered in England and Wales, company number 04008957) and IG Index
>> > > Limited (a company registered in England and Wales, company number
>> > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
>> > > London EC4R 2YA. Both IG Markets Limited (register number 195355) and
>> IG
>> > > Index Limited (register number 114059) are authorised and regulated by
>> > the
>> > > Financial Conduct Authority.
>> > >
>> > The information contained in this email is strictly confidential and for
>> > the use of the addressee only, unless otherwise indicated. If you are
>> not
>> > the intended recipient, please do not read, copy, use or disclose to
>> others
>> > this message or any attachment. Please also notify the sender by
>> replying
>> > to this email or by telephone (+44(020 7896 0011) and then delete the
>> email
>> > and any copies of it. Opinions, conclusion (etc) that do not relate to
>> the
>> > official business of this company shall be understood as neither given
>> nor
>> > endorsed by it. IG is a trading name of IG Markets Limited (a company
>> > registered in England and Wales, company number 04008957) and IG Index
>> > Limited (a company registered in England and Wales, company number
>> > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
>> > London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG
>> > Index Limited (register number 114059) are authorised and regulated by
>> the
>> > Financial Conduct Authority.
>> >
>>
>
>
The information contained in this email is strictly confidential and for the use of the addressee only, unless otherwise indicated. If you are not the intended recipient, please do not read, copy, use or disclose to others this message or any attachment. Please also notify the sender by replying to this email or by telephone (+44(020 7896 0011) and then delete the email and any copies of it. Opinions, conclusion (etc) that do not relate to the official business of this company shall be understood as neither given nor endorsed by it. IG is a trading name of IG Markets Limited (a company registered in England and Wales, company number 04008957) and IG Index Limited (a company registered in England and Wales, company number 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG Index Limited (register number 114059) are authorised and regulated by the Financial Conduct Authority.
Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
Posted by Michael Pearce <Mi...@ig.com>.
Hi All,
Based on KIP-82 result.
I am no longer actively going to push further on this.
Based on this, if no one actively wants to take the helm on this, I will move KIP-87 to the Dormant/Inactive KIP section for now, I will do this next week, if no one steps in.
Cheers
Mike
On 05/01/2017, 20:29, "Michael Pearce" <Mi...@ig.com> wrote:
@Jay.
On KIP-82 are you and Jun now on board that Headers are the better of the two options/kips?
If so could you reply on that thread this is the case.
We really would like to get KIP-82 going/moving on an active discussion around the last few implementation details, rather than where we’re at that seems were held up on getting consensus that the concept of KIP-82 is worth it.
If we don’t push on this one.
Either way we really need a solution of 82 or 87 Apr/May time release (or when ever the next one is after the imminent on in Feb)
Cheers
Mike
On 20/12/2016, 05:55, "Michael Pearce" <Mi...@ig.com> wrote:
Agreed. As I said there is still a use case just I wouldn't be pushing for it, the. Need for it reduces for me.
Sent using OWA for iPhone
________________________________________
From: radai <ra...@gmail.com>
Sent: Tuesday, December 20, 2016 3:51:33 AM
To: dev@kafka.apache.org
Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
i didnt mean to sound as insisting. what i actually mean is it would still
be a valid issue but of much less concern.
On Mon, Dec 19, 2016 at 7:50 PM, radai <ra...@gmail.com> wrote:
> this kip fixes a "bug" (quirk?) that arises when people implement headers
> "in V" (in the payload part of a message).
> if you have proper headers you obviously dont need to to stick them in V
> and so wont run into this, but its still a valid issue.
>
> On Mon, Dec 19, 2016 at 3:06 PM, Jay Kreps <ja...@confluent.io> wrote:
>
>> Makes sense!
>>
>> -Jay
>>
>> On Mon, Dec 19, 2016 at 2:40 PM, Michael Pearce <Mi...@ig.com>
>> wrote:
>>
>> > Wow just read that def over tired. Hopefully it makes sense. Or you get
>> > the gist at least.
>> >
>> > ________________________________________
>> > From: Michael Pearce <Mi...@ig.com>
>> > Sent: Monday, December 19, 2016 9:19:02 PM
>> > To: dev@kafka.apache.org
>> > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
>> >
>> > Hi Jay,
>> >
>> > Agreed this stemmed as offshoot from KIP-82.
>> >
>> > Which our main driver for was to be able to have some headers for a null
>> > value as such for our routing, audit, tracing and a few other bits which
>> > currently we are forced to do with a message wrapper, if we all agreed
>> on
>> > KIP-82 that we need native headers and look to implement that the push
>> for
>> > this would dissipate.
>> >
>> > This KIP would allow for though one use case that comes to mind we could
>> > see which is to have business data with a delete. Though as said this
>> isn't
>> > something we are pushing for think really we would have.
>> >
>> > As such in summary yes, if you want to fully support KIP-82 and we can
>> get
>> > that agreed in principle and a target release version, I think quite a
>> few
>> > guys at LinkedIn are quite pro it too ;) I'm happy to drop this one.
>> >
>> > Cheers
>> > Mike
>> >
>> > Sent using OWA for iPhone
>> > ________________________________________
>> > From: Jay Kreps <ja...@confluent.io>
>> > Sent: Monday, December 19, 2016 8:51:23 PM
>> > To: dev@kafka.apache.org
>> > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
>> >
>> > Hey Michael,
>> >
>> > Here is the compatibility concern I have:
>> >
>> > 1. You have a consumer app that relies on value == null to indicate a
>> > delete (current semantics).
>> > 2. You upgrade Kafka and your clients.
>> > 3. Some producer starts using the tombstone field in combination with
>> > non-null.
>> >
>> > I share Ismael's dislike of setting tombstones on records with null
>> values.
>> > This makes sense as a transitional state, but as an end state its a bit
>> > weird. You'd expect to be able to mix null values and tombstones, and
>> have
>> > the null values remain and the tombstones get compacted. However what
>> will
>> > happen is both will be compacted and upon debugging this you'll learn
>> that
>> > we sometimes use null in the value to indicate tombstone. Ismael's
>> solution
>> > is a bigger compatibility break, though, so not sure if that is better.
>> >
>> > My other question is the relationship to KIP-82. My read is that this
>> KIP
>> > solves some but not all of the problems KIP-82 is intended for. KIP-82,
>> on
>> > the other hand, seems to address most of the motivating uses for this
>> KIP.
>> > The exception is maybe item (5) on the list where you want to
>> simultaneous
>> > delete and convey some information to subscribers, but I couldn't
>> construct
>> > a concrete examples for that one. Do we need to rationalize these two
>> KIPs?
>> > That is, do you still advocate this proposal if we do KIP-82 and vice
>> > versa? As you may have noticed I'm somewhat emotionally invested in the
>> > simplicity of the core data model, so my default position is let's try
>> to
>> > avoid stuffing more stuff in, but if we have to add stuff I like each of
>> > these individually more than doing both. :-)
>> >
>> > -Jay
>> >
>> >
>> >
>> >
>> > On Fri, Dec 16, 2016 at 12:16 PM, Michael Pearce <Michael.Pearce@ig.com
>> >
>> > wrote:
>> >
>> > > Hi Jay
>> > >
>> > > I disagree here that we are breaking any compatibility, we went
>> through
>> > > this on the discussion thread in fact with the help of that thread is
>> how
>> > > the kip came to the solution.
>> > >
>> > > Also on the supported combinations front you mention, we are not
>> taking
>> > > anything away afaik.
>> > >
>> > > Currently supported are only are:
>> > > Null value = delete
>> > > Non-null value = non delete
>> > >
>> > > With this kip we would support
>> > > Null value + tombstone = delete
>> > > Non null value + tombstone = delete
>> > > Non null value + no tombstone = non delete
>> > >
>> > > As for the alternative idea, this is simply a new policy, how can
>> there
>> > be
>> > > confusion here? For this policy it would be explicit that tombstone
>> > marker
>> > > would need to be set for a delete.
>> > >
>> > > I'm going to vent a little now as starting to get quite frustrated.
>> > >
>> > > We are going round in circles on kip-82 as per use cases there is now
>> > many
>> > > use cases, how many more are needed? just because confluent don't see
>> > these
>> > > doesn't mean they aren't real use cases other have, this is the point
>> of
>> > > the Apache foundation, it shouldn't be the view of just one
>> organisation.
>> > > It really is getting a feeling of the NIH syndrome. Rather than it
>> being
>> > > constructive on discussion of the implementation detail.
>> > >
>> > > kip-87 spawned from as on the kip call we all agreed this was needed.
>> And
>> > > would at least allow a custom wrapper be supported in a compacted
>> topic,
>> > > allowing meta data. Which again now I feel we are spinning wheels, and
>> > > simply finding reasons not support it.
>> > >
>> > > Cheers
>> > > Mike
>> > >
>> > >
>> > >
>> > > Sent using OWA for iPad
>> > > ________________________________________
>> > > From: Jay Kreps <ja...@confluent.io>
>> > > Sent: Friday, December 16, 2016 7:09:23 PM
>> > > To: dev@kafka.apache.org
>> > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
>> > >
>> > > Hey Michael,
>> > >
>> > > I do think it might have been better had we started with a separate
>> > concept
>> > > of null vs delete. Given that we are where we are, I'm not sure that
>> the
>> > > possible cures we explored so far are better than the disease.
>> > >
>> > > I apologize for coming to this late, but I didn't really understand
>> the
>> > > implications of the KIP and that we'd be breaking compatibility with
>> > > existing apps until the vote had begun, and in my defense I'm not sure
>> > the
>> > > other folks voting did either.
>> > >
>> > > I think we all agree there are many existing apps that are built with
>> the
>> > > assumption of "null value non-tombstone" and it isn't possible to
>> > > disambiguate these from tombstones on the producer. It isn't that
>> anyone
>> > is
>> > > saying we have to support all four possibilities at once, it's that we
>> > > simply can't orphan one of the existing combinations or our users will
>> > eat
>> > > us!
>> > >
>> > > If I've understood your alternate solution of adding another setting
>> for
>> > > compaction, I think this does fix the compatibility problem, but it
>> adds
>> > an
>> > > odd mode the user has to add on all their topics. While the current
>> state
>> > > is easily explainable, the resulting state where the meaning of
>> tombstone
>> > > and null are overlapping and ambiguous and dependent on a compaction
>> > > setting that could change out of band or not be in sync with your
>> code in
>> > > some environment seems worse to me then where we currently are. I
>> think
>> > the
>> > > question is how would this combination be explained to users and does
>> it
>> > > make sense?
>> > >
>> > > -Jay
>> > >
>> > >
>> > >
>> > > On Fri, Dec 16, 2016 at 9:25 AM, Michael Pearce <
>> Michael.Pearce@ig.com>
>> > > wrote:
>> > >
>> > > > Hi Chaps,
>> > > >
>> > > > Can we either get one more +1 binding (we have 2 already) on the
>> > > existing?
>> > > >
>> > > > Or have some response on the below possible alternative. We are
>> keen to
>> > > > get working on this, so we make next feature release.
>> > > >
>> > > > Cheers
>> > > > Mike
>> > > >
>> > > >
>> > > > On 13/12/2016, 16:32, "Michael Pearce" <Mi...@ig.com>
>> wrote:
>> > > >
>> > > > Hi Ismael
>> > > >
>> > > > Did you see our email this morning, what's your thoughts on this
>> > > > approach to instead we simply have a brand new policy?
>> > > >
>> > > > Cheers
>> > > > Mike
>> > > >
>> > > >
>> > > > Sent using OWA for iPhone
>> > > > ________________________________________
>> > > > From: ismaelj@gmail.com <is...@gmail.com> on behalf of Ismael
>> > > Juma <
>> > > > ismael@juma.me.uk>
>> > > > Sent: Tuesday, December 13, 2016 11:30:05 AM
>> > > > To: dev@kafka.apache.org
>> > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
>> > > >
>> > > > Yes, this is actually tricky to do in a way where we both have
>> the
>> > > > desired
>> > > > semantics and maintain compatibility. When someone creates a
>> > > > `ProducerRecord` with a `null` value today, the producer doesn't
>> > know
>> > > > if
>> > > > it's meant to be a tombstone or not. For V3 messages, it's easy
>> > when
>> > > > the
>> > > > constructor that takes a tombstone is used. However, if any
>> other
>> > > > constructor is used, it's not clear. A couple of options I can
>> > think
>> > > > of,
>> > > > none of them particularly nice:
>> > > >
>> > > > 1. Have a third state where tombstone = unknown and the broker
>> > would
>> > > > set
>> > > > the tombstone bit if the value was null and the topic was
>> > compacted.
>> > > > People
>> > > > that wanted to pass a non-null value for the tombstone would
>> have
>> > to
>> > > > use
>> > > > the constructor that takes a tombstone. The drawbacks: third
>> state
>> > > for
>> > > > tombstone in message format, message conversion at the broker
>> for a
>> > > > common
>> > > > case.
>> > > >
>> > > > 2. Extend MetadataResponse to optionally include topic configs,
>> > which
>> > > > would
>> > > > make it possible for the producer to be smarter about setting
>> the
>> > > > tombstone. It would only do it if a tombstone was not passed
>> > > > explicitly,
>> > > > the value was null and the topic was compacted. The main
>> drawback
>> > is
>> > > > that
>> > > > the producer would be getting a bit more data for each topic
>> even
>> > > > though it
>> > > > probably won't use it most of the time. Extending
>> MetadataResponse
>> > to
>> > > > return topic configs would be useful for other reasons as well,
>> so
>> > > that
>> > > > part seems OK.
>> > > >
>> > > > In addition, for both proposals, we could consider adding
>> warnings
>> > to
>> > > > the
>> > > > documentation that the behaviour of the constructors that don't
>> > take
>> > > a
>> > > > tombstone would change in the next major release so that
>> tombstone
>> > =
>> > > > false.
>> > > > Not sure if this would be worth it though.
>> > > >
>> > > > Ismael
>> > > >
>> > > > On Sun, Dec 11, 2016 at 11:15 PM, Ewen Cheslack-Postava <
>> > > > ewen@confluent.io>
>> > > > wrote:
>> > > >
>> > > > > Michael,
>> > > > >
>> > > > > It kind of depends on how you want to interpret the tombstone
>> > flag.
>> > > > If it's
>> > > > > purely a producer-facing Kafka-level thing that we treat as
>> > > internal
>> > > > to the
>> > > > > broker and log cleaner once the record is sent, then your
>> > approach
>> > > > makes
>> > > > > sense. You're just moving copying the null-indicates-delete
>> > > behavior
>> > > > of the
>> > > > > old constructor into the tombstone flag.
>> > > > >
>> > > > > However, if you want this change to more generally decouple
>> the
>> > > idea
>> > > > of
>> > > > > deletion and null values, then you are sometimes converting
>> what
>> > > > might be a
>> > > > > completely valid null value that doesn't indicate deletion
>> into a
>> > > > > tombstone. Downstream applications could potentially handle
>> these
>> > > > cases
>> > > > > differently given the separation of deletion from value.
>> > > > >
>> > > > > I guess the question is if we want to try to support the
>> latter
>> > > even
>> > > > for
>> > > > > topics where we have older produce requests. An example where
>> > this
>> > > > could
>> > > > > come up is in something like a CDC Connector. If we try to
>> > support
>> > > > the
>> > > > > semantic difference, a connector might write changes to Kafka
>> > using
>> > > > the
>> > > > > tombstone flag to indicate when a row was truly deleted (vs an
>> > > > update that
>> > > > > sets it to null but still present; this probably makes more
>> sense
>> > > > for CDC
>> > > > > from document stores or extracting single columns). There are
>> > > various
>> > > > > reasons we might want to maintain the full log and not turn
>> > > > compaction on
>> > > > > (or just use a time-based retention policy), but downstream
>> > > > applications
>> > > > > might care to know the difference between a delete and a null
>> > > value.
>> > > > In
>> > > > > fact both versions of the same log (compacted and
>> time-retention)
>> > > > could be
>> > > > > useful and I don't think it'll be uncommon to maintain both or
>> > use
>> > > > KIP-71
>> > > > > to maintain a hybrid compacted/retention topic.
>> > > > >
>> > > > > -Ewen
>> > > > >
>> > > > > On Sun, Dec 11, 2016 at 1:18 PM, Michael Pearce <
>> > > > Michael.Pearce@ig.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Jay,
>> > > > > >
>> > > > > > Why wouldn't that work, the tombstone value is only looked
>> at
>> > by
>> > > > the
>> > > > > > broker, on a topic configured for compaction as such is
>> benign
>> > on
>> > > > non
>> > > > > > compacted topics. This is as much as sending a null value
>> > > currently
>> > > > > >
>> > > > > >
>> > > > > > Regards
>> > > > > > Mike
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > Sent using OWA for iPhone
>> > > > > > ________________________________________
>> > > > > > From: Jay Kreps <ja...@confluent.io>
>> > > > > > Sent: Sunday, December 11, 2016 8:58:53 PM
>> > > > > > To: dev@kafka.apache.org
>> > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
>> > > > > >
>> > > > > > Hey Michael,
>> > > > > >
>> > > > > > I'm not quite sure that works as that would translate ALL
>> null
>> > > > values to
>> > > > > > tombstones, even for non-compacted topics that use null as
>> an
>> > > > acceptable
>> > > > > > value sent by the producer and expected by the consumer.
>> > > > > >
>> > > > > > -Jay
>> > > > > >
>> > > > > > On Sun, Dec 11, 2016 at 3:26 AM, Michael Pearce <
>> > > > Michael.Pearce@ig.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi Ewen,
>> > > > > > >
>> > > > > > > I think the easiest way to show this is with code.
>> > > > > > >
>> > > > > > > As you can see we keep the existing behaviour for
>> > code/binaries
>> > > > calling
>> > > > > > > the pre-existing constructors, whereby if the value is
>> null
>> > the
>> > > > > tombstone
>> > > > > > > is set to true.
>> > > > > > >
>> > > > > > > Regards
>> > > > > > > Mike
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > /**
>> > > > > > > * Creates a record with a specified timestamp to be
>> sent
>> > > to
>> > > > a
>> > > > > > > specified topic and partition
>> > > > > > > *
>> > > > > > > * @param topic The topic the record will be appended
>> to
>> > > > > > > * @param partition The partition to which the record
>> > > should
>> > > > be
>> > > > > sent
>> > > > > > > * @param timestamp The timestamp of the record
>> > > > > > > * @param tombstone if the record should be treated
>> as a
>> > > > tombstone
>> > > > > if
>> > > > > > > the topic is compacted
>> > > > > > > * @param key The key that will be included in the
>> record
>> > > > > > > * @param value The record contents
>> > > > > > > */
>> > > > > > > public ProducerRecord(String topic, Integer partition,
>> > > > Boolean
>> > > > > > > tombstone, Long timestamp, K key, V value) {
>> > > > > > > if (topic == null)
>> > > > > > > throw new IllegalArgumentException("Topic
>> cannot
>> > > be
>> > > > > null.");
>> > > > > > > if (timestamp != null && timestamp < 0)
>> > > > > > > throw new IllegalArgumentException(
>> > > > > > > String.format("Invalid timestamp: %d.
>> > > > Timestamp
>> > > > > > should
>> > > > > > > always be non-negative or null.", timestamp));
>> > > > > > > if (partition != null && partition < 0)
>> > > > > > > throw new IllegalArgumentException(
>> > > > > > > String.format("Invalid partition: %d.
>> > > > Partition
>> > > > > > number
>> > > > > > > should always be non-negative or null.", partition));
>> > > > > > > if (!tombstone && value == null){
>> > > > > > > throw new IllegalArgumentException(
>> > > > > > > String.format("Tombstone must be true
>> if
>> > > null
>> > > > > > value"));
>> > > > > > > }
>> > > > > > > this.topic = topic;
>> > > > > > > this.partition = partition;
>> > > > > > > this.tombstone = tombstone;
>> > > > > > > this.key = key;
>> > > > > > > this.value = value;
>> > > > > > > this.timestamp = timestamp;
>> > > > > > > }
>> > > > > > >
>> > > > > > > public ProducerRecord(String topic, Integer partition,
>> > Long
>> > > > > > timestamp,
>> > > > > > > K key, V value) {
>> > > > > > > this(topic, partition, value == null, timestamp,
>> key,
>> > > > value);
>> > > > > > > }
>> > > > > > > ________________________________________
>> > > > > > > From: Ewen Cheslack-Postava <ew...@confluent.io>
>> > > > > > > Sent: Sunday, December 11, 2016 5:45 AM
>> > > > > > > To: dev@kafka.apache.org
>> > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
>> > > > > > >
>> > > > > > > It seemed like this was addressed in the migration
>> section,
>> > > > wasn't it?
>> > > > > > The
>> > > > > > > V2 vs V3 requests and the fact that the broker will
>> downgrade
>> > > the
>> > > > > message
>> > > > > > > format (losing zero copy) if you issues a V2 request to a
>> > > broker
>> > > > using
>> > > > > V3
>> > > > > > > format handles compatibility, does it not? The existing
>> > > > consumers won't
>> > > > > > see
>> > > > > > > the extra metadata in the value, but they will get a null
>> > > > instead and
>> > > > > > treat
>> > > > > > > it as a tombstone. Certainly there is a performance
>> impact,
>> > but
>> > > > it
>> > > > > seemed
>> > > > > > > compatible.
>> > > > > > >
>> > > > > > > I'm worried about this though:
>> > > > > > >
>> > > > > > >
>> > > > > > > - *NOTE *: With the new version of producer client
>> using
>> > > > > > ProduceRequest
>> > > > > > > V3 (magic byte = 2), a non tombstone (tombstone bit not
>> > set)
>> > > > and
>> > > > > null
>> > > > > > > value
>> > > > > > > should not be allowed as the older version of consumer
>> > using
>> > > > > > > FetchRequest
>> > > > > > > V2 will think of this as a tombstone when its actually
>> > not.
>> > > > > > >
>> > > > > > >
>> > > > > > > Unless I'm misunderstanding, this ends up breaking binary
>> > > > compatibility
>> > > > > > for
>> > > > > > > the Java API. It sounds like this suggests that passing a
>> > null
>> > > > value to
>> > > > > > the
>> > > > > > > existing ProducerRecord constructors would generate an
>> > > exception
>> > > > since
>> > > > > > you
>> > > > > > > didn't explicitly enable the tombstone (via whatever new
>> > > > constructor is
>> > > > > > > provided). But that means you can't swap in a newer client
>> > jar
>> > > > without
>> > > > > > > recompiling and get the same behavior if your app deletes
>> > keys
>> > > > using
>> > > > > the
>> > > > > > > current approach because your app will start throwing
>> > > > exceptions. Maybe
>> > > > > > > this is a tradeoff we're ok with, but we've tried pretty
>> hard
>> > > to
>> > > > avoid
>> > > > > > > breaking compatibility like this so far -- it makes
>> picking
>> > up
>> > > > bug
>> > > > > fixes
>> > > > > > in
>> > > > > > > the clients harder for users of frameworks that have to
>> pin
>> > to
>> > > > earlier
>> > > > > > > default versions for compatibility.
>> > > > > > >
>> > > > > > > But then later the KIP says:
>> > > > > > >
>> > > > > > >
>> > > > > > > - The old constructors for ProducerRecord without this
>> > > > parameter
>> > > > > will
>> > > > > > be
>> > > > > > > kept but updated so that their default behaviour if
>> > setting
>> > > > null
>> > > > > value
>> > > > > > > will
>> > > > > > > be the tombstone will be set to true to keep existing
>> > > > behaviour.
>> > > > > > >
>> > > > > > >
>> > > > > > > So maybe I am misinterpreting something.
>> > > > > > >
>> > > > > > > And just a nit re: motivation section, item 6 would be
>> > clearer
>> > > > for a
>> > > > > > union
>> > > > > > > schema with null (or optional schemas in other formats),
>> e.g.
>> > > > [null,
>> > > > > > > string], in which case losing the schema truly is losing
>> > > > information
>> > > > > > > (whereas null is already the only valid value for a pure
>> null
>> > > > schema).
>> > > > > > >
>> > > > > > > -Ewen
>> > > > > > >
>> > > > > > >
>> > > > > > > On Sat, Dec 10, 2016 at 9:24 PM, Michael Pearce <
>> > > > Michael.Pearce@ig.com
>> > > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Hi Jay,
>> > > > > > > >
>> > > > > > > > Good point this detail is missing in the KIP write up.
>> Ive
>> > > > added this
>> > > > > > > now.
>> > > > > > > >
>> > > > > > > > Essentially simply just upgrading the clients we do not
>> > > expect
>> > > > any
>> > > > > > client
>> > > > > > > > app code change needed.
>> > > > > > > >
>> > > > > > > > Cheers
>> > > > > > > > Mike
>> > > > > > > > ________________________________________
>> > > > > > > > From: Jay Kreps <ja...@confluent.io>
>> > > > > > > > Sent: Saturday, December 10, 2016 10:51 PM
>> > > > > > > > To: dev@kafka.apache.org
>> > > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone
>> Flag
>> > > > > > > >
>> > > > > > > > Michael,
>> > > > > > > >
>> > > > > > > > The compatibility section goes through the migration
>> path,
>> > > but
>> > > > isn't
>> > > > > > the
>> > > > > > > > bigger compatibility issue with existing apps? There are
>> > many
>> > > > > (probably
>> > > > > > > > thousands) of apps in production that use this feature
>> and
>> > > > send null
>> > > > > to
>> > > > > > > > mean delete. It seems like this would break
>> compatibility
>> > > with
>> > > > them,
>> > > > > > and
>> > > > > > > > they would have to be rewritten to right?
>> > > > > > > >
>> > > > > > > > -Jay
>> > > > > > > >
>> > > > > > > > On Thu, Dec 8, 2016 at 12:12 AM, Michael Pearce <
>> > > > > Michael.Pearce@ig.com
>> > > > > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Hi Jun,
>> > > > > > > > >
>> > > > > > > > > 4) On v3 we honour the tombstone. As such we expect
>> it to
>> > > be
>> > > > set
>> > > > > > > > correctly
>> > > > > > > > > as per the KIP.
>> > > > > > > > >
>> > > > > > > > > 4.1) why would we want to produce an error when its
>> v3?
>> > > This
>> > > > is the
>> > > > > > > exact
>> > > > > > > > > purpose to support non-null tombstone's
>> > > > > > > > > 4.2) again here im unclear on the question on the v3,
>> > > produce
>> > > > > request
>> > > > > > > we
>> > > > > > > > > expect the tombstone flag to be set correctly.
>> > > > > > > > >
>> > > > > > > > > 4.4) compaction only occurs on compacted topics, the
>> bit
>> > > > makes no
>> > > > > > > > > difference and not looked at on un-compacted
>> (time/size
>> > > based
>> > > > > > > eviction).
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On 06/12/2016, 20:08, "Jun Rao" <ju...@confluent.io>
>> > wrote:
>> > > > > > > > >
>> > > > > > > > > Hi, Michael,
>> > > > > > > > >
>> > > > > > > > > 4. Then, I think I misunderstood this point. Could
>> > you
>> > > > document
>> > > > > > the
>> > > > > > > > > following points in the wiki?
>> > > > > > > > > 4.1 If producer V3 sets tombstone, but provides a
>> > > > non-null
>> > > > > value,
>> > > > > > > > does
>> > > > > > > > > the
>> > > > > > > > > send() get an error or does the producer
>> > automatically
>> > > > set the
>> > > > > > > value
>> > > > > > > > to
>> > > > > > > > > null?
>> > > > > > > > > 4.2 If producer V3 doesn't set tombstone, but
>> > provides
>> > > a
>> > > > null
>> > > > > > > value,
>> > > > > > > > > does
>> > > > > > > > > the send() get an error or does the producer
>> > > > automatically sets
>> > > > > > the
>> > > > > > > > > tombstone?
>> > > > > > > > > 4.3 Does the broker only expect messages that (a)
>> > have
>> > > no
>> > > > > > tombstone
>> > > > > > > > and
>> > > > > > > > > non-null value; (b) have tombstone and null value
>> and
>> > > > reject
>> > > > > the
>> > > > > > > > > messages
>> > > > > > > > > with an error code otherwise?
>> > > > > > > > > 4.4 Do 4.1, 4.2, 4.3 depend on whether the topic
>> is
>> > > > compacted
>> > > > > on
>> > > > > > > > not?
>> > > > > > > > >
>> > > > > > > > > Thanks,
>> > > > > > > > >
>> > > > > > > > > Jun
>> > > > > > > > >
>> > > > > > > > > On Tue, Dec 6, 2016 at 10:35 AM, Michael Pearce <
>> > > > > > > > Michael.Pearce@ig.com
>> > > > > > > > > >
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Not at all. This only acts on compacted topics
>> > just
>> > > > as what
>> > > > > > > occurs
>> > > > > > > > > today
>> > > > > > > > > >
>> > > > > > > > > > Sent using OWA for iPhone
>> > > > > > > > > > ________________________________________
>> > > > > > > > > > From: Jun Rao <ju...@confluent.io>
>> > > > > > > > > > Sent: Tuesday, December 6, 2016 6:25:28 PM
>> > > > > > > > > > To: dev@kafka.apache.org
>> > > > > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction
>> > Tombstone
>> > > > Flag
>> > > > > > > > > >
>> > > > > > > > > > Hi, Michael,
>> > > > > > > > > >
>> > > > > > > > > > 4. Hmm, does that mean the new client library
>> can
>> > > > never send
>> > > > > a
>> > > > > > > null
>> > > > > > > > > message
>> > > > > > > > > > even to a regular topic? This seems like a
>> change
>> > of
>> > > > the
>> > > > > > existing
>> > > > > > > > > behavior.
>> > > > > > > > > >
>> > > > > > > > > > Thanks,
>> > > > > > > > > >
>> > > > > > > > > > Jun
>> > > > > > > > > >
>> > > > > > > > > > On Tue, Dec 6, 2016 at 9:51 AM, Michael Pearce <
>> > > > > > > > > Michael.Pearce@ig.com>
>> > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > Hi Jun,
>> > > > > > > > > > >
>> > > > > > > > > > > Re 4) That's because we expect the tombstone
>> > value
>> > > > to be
>> > > > > set
>> > > > > > > > > correctly if
>> > > > > > > > > > > message bit is 2, as such if an older client
>> > sends
>> > > > in on
>> > > > > old
>> > > > > > > > > message the
>> > > > > > > > > > > message is upcast and the bit is set
>> correctly.
>> > And
>> > > > such no
>> > > > > > > > longer
>> > > > > > > > > need
>> > > > > > > > > > to
>> > > > > > > > > > > check the value. Mayuresh can you confirm my
>> > > > thinking and
>> > > > > > > > > understanding
>> > > > > > > > > > of
>> > > > > > > > > > > what we've discussed?
>> > > > > > > > > > >
>> > > > > > > > > > > The second point I understand what you're
>> getting
>> > > at
>> > > > now my
>> > > > > > > > > apologies.
>> > > > > > > > > > Yes
>> > > > > > > > > > > this makes sense to save on touching the
>> message,
>> > > if
>> > > > we're
>> > > > > > the
>> > > > > > > > > only kip
>> > > > > > > > > > > going in, in this release.
>> > > > > > > > > > >
>> > > > > > > > > > > Cheers
>> > > > > > > > > > > Mike
>> > > > > > > > > > >
>> > > > > > > > > > > Sent using OWA for iPhone
>> > > > > > > > > > > ________________________________________
>> > > > > > > > > > > From: Jun Rao <ju...@confluent.io>
>> > > > > > > > > > > Sent: Tuesday, December 6, 2016 5:22:13 PM
>> > > > > > > > > > > To: dev@kafka.apache.org
>> > > > > > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction
>> > > > Tombstone Flag
>> > > > > > > > > > >
>> > > > > > > > > > > Hi, Michael,
>> > > > > > > > > > >
>> > > > > > > > > > > 4. Is this updated in the wiki? The text "If
>> the
>> > > > magic byte
>> > > > > > on
>> > > > > > > > > message is
>> > > > > > > > > > > 2, the broker should use the tombstone bit for
>> > log
>> > > > > > compaction."
>> > > > > > > > > doesn't
>> > > > > > > > > > > seem to have changed.
>> > > > > > > > > > >
>> > > > > > > > > > > 2. My point is that if we change the message
>> > format
>> > > > just
>> > > > > for
>> > > > > > > this
>> > > > > > > > > KIP, we
>> > > > > > > > > > > should consider whether it's worth optimizing
>> the
>> > > > down
>> > > > > > > conversion
>> > > > > > > > > path
>> > > > > > > > > > > (i.e., decide whether a conversion is needed
>> by
>> > > just
>> > > > > looking
>> > > > > > at
>> > > > > > > > the
>> > > > > > > > > > > tombstone bit in the wrapper message) since
>> > > > tombstone will
>> > > > > be
>> > > > > > > > used
>> > > > > > > > > > rarely.
>> > > > > > > > > > > However, if the message format change here is
>> > > > combined with
>> > > > > > > other
>> > > > > > > > > KIPs,
>> > > > > > > > > > > then this optimization likely won't be needed.
>> > The
>> > > > latter
>> > > > > > > > probably
>> > > > > > > > > makes
>> > > > > > > > > > > the code simpler. Jiangjie, Mayuresh, what do
>> you
>> > > > think?
>> > > > > > > > > > >
>> > > > > > > > > > > Other than those, +1 from me,
>> > > > > > > > > > >
>> > > > > > > > > > > Thanks,
>> > > > > > > > > > >
>> > > > > > > > > > > Jun
>> > > > > > > > > > >
>> > > > > > > > > > > On Tue, Dec 6, 2016 at 8:54 AM, Michael
>> Pearce <
>> > > > > > > > > Michael.Pearce@ig.com>
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > Hi Jun
>> > > > > > > > > > > >
>> > > > > > > > > > > > do we have your vote on this now?
>> > > > > > > > > > > >
>> > > > > > > > > > > > Any other concerns?
>> > > > > > > > > > > >
>> > > > > > > > > > > > Cheers
>> > > > > > > > > > > > Mike
>> > > > > > > > > > > >
>> > > > > > > > > > > > Sent using OWA for iPhone
>> > > > > > > > > > > > ________________________________________
>> > > > > > > > > > > > From: Michael Pearce <Michael.Pearce@ig.com
>> >
>> > > > > > > > > > > > Sent: Saturday, December 3, 2016 1:37:45 AM
>> > > > > > > > > > > > To: dev@kafka.apache.org
>> > > > > > > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction
>> > > > Tombstone
>> > > > > Flag
>> > > > > > > > > > > >
>> > > > > > > > > > > > Hi Jun,
>> > > > > > > > > > > >
>> > > > > > > > > > > > Have updated. Thanks again for the feedback.
>> > > > > > > > > > > >
>> > > > > > > > > > > > Agree yes we should align up when it gets to
>> > > that,
>> > > > I
>> > > > > assume
>> > > > > > > > > you've
>> > > > > > > > > > > flagged
>> > > > > > > > > > > > the same in the other KIP?
>> > > > > > > > > > > >
>> > > > > > > > > > > > I think for now let's get this KIP approved,
>> > then
>> > > > we can
>> > > > > > > start
>> > > > > > > > > the work
>> > > > > > > > > > > to
>> > > > > > > > > > > > get an initial PR, then we can discuss how
>> to
>> > > > align the
>> > > > > two
>> > > > > > > if
>> > > > > > > > > needed.
>> > > > > > > > > > > >
>> > > > > > > > > > > > Cheers,
>> > > > > > > > > > > > Mike
>> > > > > > > > > > > >
>> > > > > > > > > > > > On 02/12/2016, 18:26, "Jun Rao" <
>> > > jun@confluent.io>
>> > > > > wrote:
>> > > > > > > > > > > >
>> > > > > > > > > > > > Hi, Michael,
>> > > > > > > > > > > >
>> > > > > > > > > > > > For 2), this is fine. Could you update
>> the
>> > > KIP
>> > > > wiki
>> > > > > to
>> > > > > > > make
>> > > > > > > > > this
>> > > > > > > > > > and
>> > > > > > > > > > > > other
>> > > > > > > > > > > > points clearer? Other than that, the KIP
>> > > looks
>> > > > good
>> > > > > to
>> > > > > > > me.
>> > > > > > > > > > > >
>> > > > > > > > > > > > An orthogonal thing is that there are
>> other
>> > > > KIPs such
>> > > > > > as
>> > > > > > > > > KIP-98
>> > > > > > > > > > that
>> > > > > > > > > > > > also
>> > > > > > > > > > > > intend to change the message format. If
>> > they
>> > > > all get
>> > > > > > > > > approved, we
>> > > > > > > > > > > > should
>> > > > > > > > > > > > think about whether it's better to just
>> > bump
>> > > > up the
>> > > > > > magic
>> > > > > > > > > byte once
>> > > > > > > > > > > to
>> > > > > > > > > > > > incorporate multiple format changes
>> like we
>> > > > did in
>> > > > > > > > > KIP-31/KIP-32.
>> > > > > > > > > > > >
>> > > > > > > > > > > > Thanks,
>> > > > > > > > > > > >
>> > > > > > > > > > > > Jun
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > On Fri, Dec 2, 2016 at 3:19 AM, Michael
>> > > Pearce
>> > > > <
>> > > > > > > > > > > Michael.Pearce@ig.com>
>> > > > > > > > > > > > wrote:
>> > > > > > > > > > > >
>> > > > > > > > > > > > > Hi Jao
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Thanks for the response. Sorry for
>> slow
>> > > > reply, both
>> > > > > > > with
>> > > > > > > > > personal
>> > > > > > > > > > > > sickness
>> > > > > > > > > > > > > and also battling some critical issues
>> > > > encountered
>> > > > > > > since
>> > > > > > > > > > upgrading
>> > > > > > > > > > > to
>> > > > > > > > > > > > > 0.10.1.0
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 1) Thans for spotting, Document error
>> > where
>> > > > we
>> > > > > > branched
>> > > > > > > > > this KIP
>> > > > > > > > > > > from
>> > > > > > > > > > > > > KIP-82, will get that removed.
>> > > > > > > > > > > > > 2) Intent is to do this just at the
>> > record
>> > > > message
>> > > > > > > level.
>> > > > > > > > > > > > > 3) Thanks for spotting, Will ensure
>> this
>> > is
>> > > > > > corrected.
>> > > > > > > > > > > > > 4) As per discussion thread we will
>> > support
>> > > > > > tombstone +
>> > > > > > > > > null
>> > > > > > > > > > value,
>> > > > > > > > > > > > > tombstone + non null value, no
>> tombstone
>> > +
>> > > > null
>> > > > > > value.
>> > > > > > > > > > > > > 5) I believe this was in the
>> discussion
>> > > > thread,
>> > > > > > > @Mayuresh
>> > > > > > > > > is this
>> > > > > > > > > > > > > something we've overlooked? I thought
>> we
>> > > > would down
>> > > > > > > > > convert and
>> > > > > > > > > > > > remove the
>> > > > > > > > > > > > > value so the old consumer had existing
>> > > > behavior, or
>> > > > > > is
>> > > > > > > > > there
>> > > > > > > > > > > > something we
>> > > > > > > > > > > > > haven't thought about?
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Cheers
>> > > > > > > > > > > > > Mike
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > On 30/11/2016, 18:12, "Jun Rao" <
>> > > > jun@confluent.io>
>> > > > > > > > wrote:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Hi, Michael,
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Thanks for the KIP. A few comments
>> > > below.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 1. The message format change
>> contains
>> > > > > > > "HeadersLength
>> > > > > > > > > > Headers".
>> > > > > > > > > > > > Is that
>> > > > > > > > > > > > > intended?
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 2. For compressed messageset, is
>> the
>> > > > tombstone
>> > > > > > bit
>> > > > > > > > > only set
>> > > > > > > > > > at
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > shallow
>> > > > > > > > > > > > > level? Do we always leave that
>> bit in
>> > > the
>> > > > > wrapper
>> > > > > > > > > message
>> > > > > > > > > > > unset?
>> > > > > > > > > > > > An
>> > > > > > > > > > > > > alternative is to set the
>> tombstone
>> > bit
>> > > > in the
>> > > > > > > > wrapper
>> > > > > > > > > if at
>> > > > > > > > > > > > least one
>> > > > > > > > > > > > > inner message has the tombstone
>> bit
>> > > set.
>> > > > This
>> > > > > > makes
>> > > > > > > > > things a
>> > > > > > > > > > > bit
>> > > > > > > > > > > > more
>> > > > > > > > > > > > > complicated, but we could
>> potentially
>> > > > exploit
>> > > > > > that
>> > > > > > > > for
>> > > > > > > > > > > > optimizing down
>> > > > > > > > > > > > > conversion. For example, we only
>> need
>> > > to
>> > > > > convert
>> > > > > > > > > messages
>> > > > > > > > > > with
>> > > > > > > > > > > > magic 2
>> > > > > > > > > > > > > to
>> > > > > > > > > > > > > magic 1 if the wrapper's tombstone
>> > bit
>> > > > is set
>> > > > > > > > > (conversion is
>> > > > > > > > > > > > always
>> > > > > > > > > > > > > needed
>> > > > > > > > > > > > > from magic 2 to magic 0). Not
>> sure if
>> > > the
>> > > > > > > > optimization
>> > > > > > > > > is
>> > > > > > > > > > worth
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > complexity though.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 3. The referencing of the new
>> version
>> > > of
>> > > > > > > > > > > > ProducerRequest/FetchRequest
>> > > > > > > > > > > > > is
>> > > > > > > > > > > > > inconsistent (v4 vs v3). Since our
>> > > > convention
>> > > > > > > starts
>> > > > > > > > at
>> > > > > > > > > > version
>> > > > > > > > > > > > at 0, I
>> > > > > > > > > > > > > think the new version would be 3.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 4. "If the magic byte on message
>> is
>> > 2,
>> > > > the
>> > > > > broker
>> > > > > > > > > should use
>> > > > > > > > > > > the
>> > > > > > > > > > > > > tombstone
>> > > > > > > > > > > > > bit for log compaction." What
>> about
>> > > null
>> > > > value?
>> > > > > > My
>> > > > > > > > > > > understanding
>> > > > > > > > > > > > is
>> > > > > > > > > > > > > that
>> > > > > > > > > > > > > null value will be treated the
>> same
>> > as
>> > > > setting
>> > > > > > the
>> > > > > > > > > tombstone
>> > > > > > > > > > > bit.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 5. For the migration path, it
>> would
>> > be
>> > > > useful
>> > > > > to
>> > > > > > > > > describe the
>> > > > > > > > > > > > down
>> > > > > > > > > > > > > conversion path to consumers
>> (i.e.,
>> > > > brokers on
>> > > > > > > > message
>> > > > > > > > > format
>> > > > > > > > > > > > 0.10.2
>> > > > > > > > > > > > > and
>> > > > > > > > > > > > > consumers on older version).
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Thanks,
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Jun
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > On Tue, Nov 29, 2016 at 3:18 AM,
>> > > Michael
>> > > > > Pearce <
>> > > > > > > > > > > > Michael.Pearce@ig.com
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > > Hi All,
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > We have been discussing in the
>> > below
>> > > > thread
>> > > > > and
>> > > > > > > > final
>> > > > > > > > > > changes
>> > > > > > > > > > > > have
>> > > > > > > > > > > > > been
>> > > > > > > > > > > > > > made to the KIP wiki based on
>> these
>> > > > > > discussions.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > We would now like to put to the
>> > vote
>> > > > the
>> > > > > > > following
>> > > > > > > > > KIP:
>> > > > > > > > > > > > > > https://cwiki.apache.org/
>> > > > > > > > > confluence/display/KAFKA/KIP-
>> > > > > > > > > > 87+-+
>> > > > > > > > > > > > > > Add+Compaction+Tombstone+Flag
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > This kip is for having a
>> distinct
>> > > > compaction
>> > > > > > > > > attribute
>> > > > > > > > > > > > "tombstone"
>> > > > > > > > > > > > > flag
>> > > > > > > > > > > > > > instead of relying on null
>> value,
>> > > > allowing
>> > > > > > > non-null
>> > > > > > > > > value
>> > > > > > > > > > > > delete
>> > > > > > > > > > > > > messages.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Many thanks,
>> > > > > > > > > > > > > > Michael
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > On 22/11/2016, 15:52, "Michael
>> > > Pearce"
>> > > > <
>> > > > > > > > > > > Michael.Pearce@ig.com>
>> > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Hi Mayuresh,
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > LGTM. Ive just made one
>> small
>> > > > adjustment
>> > > > > > > > > updating the
>> > > > > > > > > > > wire
>> > > > > > > > > > > > > protocol to
>> > > > > > > > > > > > > > show the magic byte bump.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Do we think we're good to
>> put
>> > to
>> > > a
>> > > > vote?
>> > > > > Is
>> > > > > > > > > there any
>> > > > > > > > > > > > other bits
>> > > > > > > > > > > > > > needing discussion?
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Cheers
>> > > > > > > > > > > > > > Mike
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > On 21/11/2016, 18:26,
>> "Mayuresh
>> > > > Gharat" <
>> > > > > > > > > > > > > gharatmayuresh15@gmail.com>
>> > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Hi Michael,
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > I have updated the
>> > migration
>> > > > section
>> > > > > of
>> > > > > > > the
>> > > > > > > > > KIP.
>> > > > > > > > > > Can
>> > > > > > > > > > > > you
>> > > > > > > > > > > > > please
>> > > > > > > > > > > > > > take a look?
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Thanks,
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Mayuresh
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > On Fri, Nov 18, 2016 at
>> > 9:07
>> > > > AM,
>> > > > > > Mayuresh
>> > > > > > > > > Gharat <
>> > > > > > > > > > > > > > gharatmayuresh15@gmail.com
>> > > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Hi Michael,
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > That whilst sending
>> > > > tombstone and
>> > > > > non
>> > > > > > > > null
>> > > > > > > > > value,
>> > > > > > > > > > > the
>> > > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > can expect
>> > > > > > > > > > > > > > > only to receive the
>> > > non-null
>> > > > > message
>> > > > > > > only
>> > > > > > > > > in step
>> > > > > > > > > > > > (3) is
>> > > > > > > > > > > > > this
>> > > > > > > > > > > > > > correct?
>> > > > > > > > > > > > > > > ---> I do agree with
>> you
>> > > > here.
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Becket, Ismael : can
>> you
>> > > guys
>> > > > > review
>> > > > > > > the
>> > > > > > > > > > migration
>> > > > > > > > > > > > plan
>> > > > > > > > > > > > > listed
>> > > > > > > > > > > > > > above using
>> > > > > > > > > > > > > > > magic byte?
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Thanks,
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Mayuresh
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > On Fri, Nov 18, 2016
>> at
>> > > 8:58
>> > > > AM,
>> > > > > > > Michael
>> > > > > > > > > Pearce <
>> > > > > > > > > > > > > > Michael.Pearce@ig.com>
>> > > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >> Many thanks for this
>> > > > Mayuresh. I
>> > > > > > don't
>> > > > > > > > > have any
>> > > > > > > > > > > > > objections.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> I assume we should
>> > state:
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> That whilst sending
>> > > > tombstone and
>> > > > > > non
>> > > > > > > > null
>> > > > > > > > > > value,
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > can expect
>> > > > > > > > > > > > > > >> only to receive the
>> > > non-null
>> > > > > message
>> > > > > > > > only
>> > > > > > > > > in
>> > > > > > > > > > step
>> > > > > > > > > > > > (3) is
>> > > > > > > > > > > > > this
>> > > > > > > > > > > > > > correct?
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Cheers
>> > > > > > > > > > > > > > >> Mike
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Sent using OWA for
>> > iPhone
>> > > > > > > > > > > > > > >>
>> > > > ______________________________
>> > > > > > > > __________
>> > > > > > > > > > > > > > >> From: Mayuresh
>> Gharat <
>> > > > > > > > > > gharatmayuresh15@gmail.com
>> > > > > > > > > > > >
>> > > > > > > > > > > > > > >> Sent: Thursday,
>> November
>> > > > 17, 2016
>> > > > > > > > 5:18:41
>> > > > > > > > > PM
>> > > > > > > > > > > > > > >> To:
>> > dev@kafka.apache.org
>> > > > > > > > > > > > > > >> Subject: Re:
>> [DISCUSS]
>> > > > KIP-87 -
>> > > > > Add
>> > > > > > > > > Compaction
>> > > > > > > > > > > > Tombstone
>> > > > > > > > > > > > > Flag
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Hi Ismael,
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Thanks for the
>> > > explanation.
>> > > > > > > > > > > > > > >> Specially I like this
>> > part
>> > > > where
>> > > > > in
>> > > > > > > you
>> > > > > > > > > > mentioned
>> > > > > > > > > > > > we can
>> > > > > > > > > > > > > get
>> > > > > > > > > > > > > > rid of the
>> > > > > > > > > > > > > > >> older null value
>> support
>> > > > for log
>> > > > > > > > > compaction
>> > > > > > > > > > later
>> > > > > > > > > > > > on,
>> > > > > > > > > > > > > here :
>> > > > > > > > > > > > > > >> We can't change
>> > semantics
>> > > > of the
>> > > > > > > message
>> > > > > > > > > format
>> > > > > > > > > > > > without
>> > > > > > > > > > > > > having
>> > > > > > > > > > > > > > a long
>> > > > > > > > > > > > > > >> transition period.
>> And
>> > we
>> > > > can't
>> > > > > rely
>> > > > > > > > > > > > > > >> on people reading
>> > > > documentation or
>> > > > > > > > acting
>> > > > > > > > > on a
>> > > > > > > > > > > > warning for
>> > > > > > > > > > > > > > something so
>> > > > > > > > > > > > > > >> fundamental. As
>> such, my
>> > > > take is
>> > > > > > that
>> > > > > > > we
>> > > > > > > > > need to
>> > > > > > > > > > > > bump the
>> > > > > > > > > > > > > magic
>> > > > > > > > > > > > > > byte. The
>> > > > > > > > > > > > > > >> good news is
>> > > > > > > > > > > > > > >> that we don't have to
>> > > > support all
>> > > > > > > > versions
>> > > > > > > > > > > forever.
>> > > > > > > > > > > > We
>> > > > > > > > > > > > > have
>> > > > > > > > > > > > > > said that we
>> > > > > > > > > > > > > > >> will support direct
>> > > > upgrades for 2
>> > > > > > > > years.
>> > > > > > > > > That
>> > > > > > > > > > > > means that
>> > > > > > > > > > > > > > message format
>> > > > > > > > > > > > > > >> version n could, in
>> > > theory,
>> > > > be
>> > > > > > > removed 2
>> > > > > > > > > years
>> > > > > > > > > > > > after the
>> > > > > > > > > > > > > it's
>> > > > > > > > > > > > > > introduced.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Just a heads up, I
>> would
>> > > > like to
>> > > > > > > mention
>> > > > > > > > > that
>> > > > > > > > > > even
>> > > > > > > > > > > > without
>> > > > > > > > > > > > > > bumping magic
>> > > > > > > > > > > > > > >> byte, we will *NOT*
>> > loose
>> > > > zero
>> > > > > copy
>> > > > > > as
>> > > > > > > > in
>> > > > > > > > > the
>> > > > > > > > > > > > client(x+1)
>> > > > > > > > > > > > > in my
>> > > > > > > > > > > > > > >> explanation
>> > > > > > > > > > > > > > >> above will convert
>> > > > internally a
>> > > > > null
>> > > > > > > > > value to
>> > > > > > > > > > > have a
>> > > > > > > > > > > > > tombstone
>> > > > > > > > > > > > > > bit set and
>> > > > > > > > > > > > > > >> a tombstone bit set
>> to
>> > > have
>> > > > a null
>> > > > > > > value
>> > > > > > > > > > > > automatically
>> > > > > > > > > > > > > > internally and by
>> > > > > > > > > > > > > > >> the time we move to
>> > > version
>> > > > (x+2),
>> > > > > > the
>> > > > > > > > > clients
>> > > > > > > > > > > > would have
>> > > > > > > > > > > > > > upgraded.
>> > > > > > > > > > > > > > >> Obviously if we
>> support
>> > a
>> > > > request
>> > > > > > from
>> > > > > > > > > > > consumer(x),
>> > > > > > > > > > > > we
>> > > > > > > > > > > > > will
>> > > > > > > > > > > > > > loose zero
>> > > > > > > > > > > > > > >> copy
>> > > > > > > > > > > > > > >> but that is the same
>> > case
>> > > > with
>> > > > > magic
>> > > > > > > > byte.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> But if magic byte
>> bump
>> > > > makes life
>> > > > > > > easier
>> > > > > > > > > for
>> > > > > > > > > > > > transition
>> > > > > > > > > > > > > for the
>> > > > > > > > > > > > > > above
>> > > > > > > > > > > > > > >> reasons that you
>> > > explained,
>> > > > I am
>> > > > > OK
>> > > > > > > with
>> > > > > > > > > it
>> > > > > > > > > > since
>> > > > > > > > > > > > we are
>> > > > > > > > > > > > > going
>> > > > > > > > > > > > > > to meet the
>> > > > > > > > > > > > > > >> end goal down the
>> road
>> > :)
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> On a side note can we
>> > > > update the
>> > > > > doc
>> > > > > > > > here
>> > > > > > > > > on
>> > > > > > > > > > magic
>> > > > > > > > > > > > byte
>> > > > > > > > > > > > > to say
>> > > > > > > > > > > > > > that "*it
>> > > > > > > > > > > > > > >> should be bumped
>> > whenever
>> > > > the
>> > > > > > message
>> > > > > > > > > format is
>> > > > > > > > > > > > changed
>> > > > > > > > > > > > > or the
>> > > > > > > > > > > > > > >> interpretation of
>> > message
>> > > > format
>> > > > > > > (usage
>> > > > > > > > > of the
>> > > > > > > > > > > > reserved
>> > > > > > > > > > > > > bits as
>> > > > > > > > > > > > > > well) is
>> > > > > > > > > > > > > > >> changed*".
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Hi Michael,
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Here is the update
>> plan
>> > > > that we
>> > > > > > > > discussed
>> > > > > > > > > > offline
>> > > > > > > > > > > > > yesterday :
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Currently the
>> magic-byte
>> > > > which
>> > > > > > > > > corresponds to
>> > > > > > > > > > the
>> > > > > > > > > > > > > > "message.format.version"
>> > > > > > > > > > > > > > >> is set to 1.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> 1) On broker it will
>> be
>> > > set
>> > > > to 1
>> > > > > > > > > initially.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> 2) When a producer
>> > client
>> > > > sends a
>> > > > > > > > message
>> > > > > > > > > with
>> > > > > > > > > > > > magic-byte
>> > > > > > > > > > > > > = 2,
>> > > > > > > > > > > > > > since the
>> > > > > > > > > > > > > > >> broker is on
>> magic-byte
>> > =
>> > > > 1, we
>> > > > > will
>> > > > > > > > down
>> > > > > > > > > > convert
>> > > > > > > > > > > > it,
>> > > > > > > > > > > > > which
>> > > > > > > > > > > > > > means if the
>> > > > > > > > > > > > > > >> tombstone bit is set,
>> > the
>> > > > value
>> > > > > will
>> > > > > > > be
>> > > > > > > > > set to
>> > > > > > > > > > > > null. A
>> > > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > >> understanding
>> > magic-byte =
>> > > > 1, will
>> > > > > > > still
>> > > > > > > > > work
>> > > > > > > > > > with
>> > > > > > > > > > > > this. A
>> > > > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > >> working
>> > > > > > > > > > > > > > >> with magic-byte =2
>> will
>> > > > also be
>> > > > > able
>> > > > > > > to
>> > > > > > > > > > understand
>> > > > > > > > > > > > this,
>> > > > > > > > > > > > > since
>> > > > > > > > > > > > > > it
>> > > > > > > > > > > > > > >> understands the
>> > tombstone.
>> > > > > > > > > > > > > > >> Now there is still
>> the
>> > > > question of
>> > > > > > > > > supporting a
>> > > > > > > > > > > > > non-tombstone
>> > > > > > > > > > > > > > and null
>> > > > > > > > > > > > > > >> value from producer
>> > client
>> > > > with
>> > > > > > > > > magic-byte = 2.*
>> > > > > > > > > > > (I
>> > > > > > > > > > > > am
>> > > > > > > > > > > > > not sure
>> > > > > > > > > > > > > > if we
>> > > > > > > > > > > > > > >> should support this.
>> > > > Ismael/Becket
>> > > > > > can
>> > > > > > > > > comment
>> > > > > > > > > > > > here)*
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> 3) When almost all
>> the
>> > > > clients
>> > > > > have
>> > > > > > > > > upgraded,
>> > > > > > > > > > the
>> > > > > > > > > > > > > > message.format.version
>> > > > > > > > > > > > > > >> on
>> > > > > > > > > > > > > > >> the broker can be
>> > changed
>> > > > to 2,
>> > > > > > where
>> > > > > > > in
>> > > > > > > > > the
>> > > > > > > > > > down
>> > > > > > > > > > > > > conversion in
>> > > > > > > > > > > > > > the above
>> > > > > > > > > > > > > > >> step will not
>> happen. If
>> > > at
>> > > > this
>> > > > > > point
>> > > > > > > > we
>> > > > > > > > > get a
>> > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > request from a
>> > > > > > > > > > > > > > >> older consumer, we
>> might
>> > > > have to
>> > > > > > down
>> > > > > > > > > convert
>> > > > > > > > > > > where
>> > > > > > > > > > > > in we
>> > > > > > > > > > > > > loose
>> > > > > > > > > > > > > > zero copy,
>> > > > > > > > > > > > > > >> but these cases
>> should
>> > be
>> > > > rare.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Becket can you review
>> > this
>> > > > plan
>> > > > > and
>> > > > > > > add
>> > > > > > > > > more
>> > > > > > > > > > > > details if I
>> > > > > > > > > > > > > have
>> > > > > > > > > > > > > > >> missed/wronged
>> > something,
>> > > > before
>> > > > > we
>> > > > > > > put
>> > > > > > > > > it on
>> > > > > > > > > > KIP.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Thanks,
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> Mayuresh
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> On Wed, Nov 16, 2016
>> at
>> > > > 11:07 PM,
>> > > > > > > > Michael
>> > > > > > > > > > Pearce <
>> > > > > > > > > > > > > > Michael.Pearce@ig.com>
>> > > > > > > > > > > > > > >> wrote:
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> > Thanks guys, for
>> > > > discussing this
>> > > > > > > > > offline and
>> > > > > > > > > > > > getting
>> > > > > > > > > > > > > some
>> > > > > > > > > > > > > > consensus.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > So its clear for
>> > myself
>> > > > and
>> > > > > others
>> > > > > > > > what
>> > > > > > > > > is
>> > > > > > > > > > > > proposed now
>> > > > > > > > > > > > > (i
>> > > > > > > > > > > > > > think i
>> > > > > > > > > > > > > > >> > understand, but
>> want
>> > to
>> > > > make
>> > > > > sure)
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > Could i ask either
>> > > > directly
>> > > > > update
>> > > > > > > the
>> > > > > > > > > kip to
>> > > > > > > > > > > > detail the
>> > > > > > > > > > > > > > migration
>> > > > > > > > > > > > > > >> > strategy, or
>> > (re-)state
>> > > > your
>> > > > > > offline
>> > > > > > > > > discussed
>> > > > > > > > > > > and
>> > > > > > > > > > > > > agreed
>> > > > > > > > > > > > > > migration
>> > > > > > > > > > > > > > >> > strategy based on a
>> > > magic
>> > > > byte
>> > > > > is
>> > > > > > in
>> > > > > > > > > this
>> > > > > > > > > > > thread.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > The main original
>> > driver
>> > > > for the
>> > > > > > KIP
>> > > > > > > > > was to
>> > > > > > > > > > > > support
>> > > > > > > > > > > > > > compaction where
>> > > > > > > > > > > > > > >> value
>> > > > > > > > > > > > > > >> > isn't null, based
>> off
>> > > the
>> > > > > > > discussions
>> > > > > > > > on
>> > > > > > > > > > KIP-82
>> > > > > > > > > > > > thread.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > We should be able
>> to
>> > > > support
>> > > > > > > > > non-tombstone +
>> > > > > > > > > > > null
>> > > > > > > > > > > > value
>> > > > > > > > > > > > > by the
>> > > > > > > > > > > > > > >> completion
>> > > > > > > > > > > > > > >> > of the KIP, as we
>> > noted
>> > > > when
>> > > > > > > > discussing
>> > > > > > > > > this
>> > > > > > > > > > > kip,
>> > > > > > > > > > > > having
>> > > > > > > > > > > > > > logic based on
>> > > > > > > > > > > > > > >> a
>> > > > > > > > > > > > > > >> > null value isn't
>> very
>> > > > clean and
>> > > > > > also
>> > > > > > > > > separates
>> > > > > > > > > > > the
>> > > > > > > > > > > > > concerns.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > As discussed
>> already
>> > > > though we
>> > > > > can
>> > > > > > > > > split this
>> > > > > > > > > > > into
>> > > > > > > > > > > > > KIP-87a
>> > > > > > > > > > > > > > and KIP-87b
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > Where we look to
>> > deliver
>> > > > KIP-87a
>> > > > > > on
>> > > > > > > a
>> > > > > > > > > > compacted
>> > > > > > > > > > > > topic
>> > > > > > > > > > > > > (to
>> > > > > > > > > > > > > > address the
>> > > > > > > > > > > > > > >> > immediate issues)
>> > > > > > > > > > > > > > >> > * tombstone + null
>> > value
>> > > > > > > > > > > > > > >> > * tombstone +
>> non-null
>> > > > value
>> > > > > > > > > > > > > > >> > * non-tombstone +
>> > > > non-null value
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > Then we can discuss
>> > once
>> > > > KIP-87a
>> > > > > > is
>> > > > > > > > > completed
>> > > > > > > > > > > > options
>> > > > > > > > > > > > > later
>> > > > > > > > > > > > > > and how we
>> > > > > > > > > > > > > > >> > support the second
>> > part
>> > > > KIP-87b
>> > > > > to
>> > > > > > > > > deliver:
>> > > > > > > > > > > > > > >> > * non-tombstone +
>> null
>> > > > value
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > Cheers
>> > > > > > > > > > > > > > >> > Mike
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> >
>> > > > ______________________________
>> > > > > > > > > __________
>> > > > > > > > > > > > > > >> > From: Becket Qin <
>> > > > > > > > becket.qin@gmail.com>
>> > > > > > > > > > > > > > >> > Sent: Thursday,
>> > November
>> > > > 17,
>> > > > > 2016
>> > > > > > > 1:43
>> > > > > > > > > AM
>> > > > > > > > > > > > > > >> > To:
>> > > dev@kafka.apache.org
>> > > > > > > > > > > > > > >> > Subject: Re:
>> [DISCUSS]
>> > > > KIP-87 -
>> > > > > > Add
>> > > > > > > > > Compaction
>> > > > > > > > > > > > > Tombstone Flag
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > Renu, Mayuresh and
>> I
>> > had
>> > > > an
>> > > > > > offline
>> > > > > > > > > > discussion,
>> > > > > > > > > > > > and
>> > > > > > > > > > > > > following
>> > > > > > > > > > > > > > is a brief
>> > > > > > > > > > > > > > >> > summary.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > 1. We agreed that
>> not
>> > > > bumping up
>> > > > > > > magic
>> > > > > > > > > value
>> > > > > > > > > > may
>> > > > > > > > > > > > result
>> > > > > > > > > > > > > in
>> > > > > > > > > > > > > > losing zero
>> > > > > > > > > > > > > > >> copy
>> > > > > > > > > > > > > > >> > during migration.
>> > > > > > > > > > > > > > >> > 2. Given that
>> bumping
>> > up
>> > > > magic
>> > > > > > value
>> > > > > > > > is
>> > > > > > > > > almost
>> > > > > > > > > > > > free and
>> > > > > > > > > > > > > has
>> > > > > > > > > > > > > > benefit of
>> > > > > > > > > > > > > > >> > avoiding potential
>> > > > performance
>> > > > > > > issue.
>> > > > > > > > > It is
>> > > > > > > > > > > > probably
>> > > > > > > > > > > > > worth
>> > > > > > > > > > > > > > doing.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > One issue we still
>> > need
>> > > > to think
>> > > > > > > about
>> > > > > > > > > is
>> > > > > > > > > > > whether
>> > > > > > > > > > > > we
>> > > > > > > > > > > > > want to
>> > > > > > > > > > > > > > support a
>> > > > > > > > > > > > > > >> > non-tombstone
>> message
>> > > > with null
>> > > > > > > value.
>> > > > > > > > > > > > > > >> > Currently it is not
>> > > > supported by
>> > > > > > > > Kafka.
>> > > > > > > > > If we
>> > > > > > > > > > > > allow a
>> > > > > > > > > > > > > > non-tombstone null
>> > > > > > > > > > > > > > >> > value message to
>> exist
>> > > > after
>> > > > > > KIP-87.
>> > > > > > > > The
>> > > > > > > > > > problem
>> > > > > > > > > > > > is
>> > > > > > > > > > > > > that such
>> > > > > > > > > > > > > > message
>> > > > > > > > > > > > > > >> will
>> > > > > > > > > > > > > > >> > not be supported by
>> > the
>> > > > > consumers
>> > > > > > > > prior
>> > > > > > > > > to
>> > > > > > > > > > > KIP-87.
>> > > > > > > > > > > > > Because a
>> > > > > > > > > > > > > > null value
>> > > > > > > > > > > > > > >> > will always be
>> > > > interpreted to a
>> > > > > > > > > tombstone.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > One option is that
>> we
>> > > > keep the
>> > > > > > > current
>> > > > > > > > > way,
>> > > > > > > > > > i.e.
>> > > > > > > > > > > > do not
>> > > > > > > > > > > > > > support such
>> > > > > > > > > > > > > > >> > message. It would
>> be
>> > > good
>> > > > to
>> > > > > know
>> > > > > > if
>> > > > > > > > > there is
>> > > > > > > > > > a
>> > > > > > > > > > > > > concrete use
>> > > > > > > > > > > > > > case for
>> > > > > > > > > > > > > > >> such
>> > > > > > > > > > > > > > >> > message. If there
>> is
>> > > not,
>> > > > we can
>> > > > > > > > > probably just
>> > > > > > > > > > > not
>> > > > > > > > > > > > > support it.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > Thanks,
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > JIangjie (Becket)
>> Qin
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > On Wed, Nov 16,
>> 2016
>> > at
>> > > > 1:28 PM,
>> > > > > > > > > Mayuresh
>> > > > > > > > > > > Gharat <
>> > > > > > > > > > > > > > >> >
>> > > > gharatmayuresh15@gmail.com
>> > > > > > > > > > > > > > >> > > wrote:
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >> > > Hi Ismael,
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > This is
>> something I
>> > > can
>> > > > think
>> > > > > of
>> > > > > > > for
>> > > > > > > > > > migration
>> > > > > > > > > > > > plan:
>> > > > > > > > > > > > > > >> > > So the migration
>> > plan
>> > > > can look
>> > > > > > > > > something
>> > > > > > > > > > like
>> > > > > > > > > > > > this,
>> > > > > > > > > > > > > with up
>> > > > > > > > > > > > > > >> conversion :
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > 1) Currently lets
>> > say
>> > > > we have
>> > > > > > > Broker
>> > > > > > > > > at
>> > > > > > > > > > > version
>> > > > > > > > > > > > x.
>> > > > > > > > > > > > > > >> > > 2) Currently we
>> have
>> > > > clients
>> > > > > at
>> > > > > > > > > version x.
>> > > > > > > > > > > > > > >> > > 3) a) We move the
>> > > > version to
>> > > > > > > > > Broker(x+1) :
>> > > > > > > > > > > > supports
>> > > > > > > > > > > > > both
>> > > > > > > > > > > > > > tombstone and
>> > > > > > > > > > > > > > >> > null
>> > > > > > > > > > > > > > >> > > for log
>> compaction.
>> > > > > > > > > > > > > > >> > > b) We upgrade
>> > the
>> > > > client
>> > > > > to
>> > > > > > > > > version
>> > > > > > > > > > > > client(x+1) :
>> > > > > > > > > > > > > if in
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > >> producer
>> > > > > > > > > > > > > > >> > > client(x+1) the
>> > value
>> > > > is set
>> > > > > to
>> > > > > > > > null,
>> > > > > > > > > we
>> > > > > > > > > > will
>> > > > > > > > > > > > > automatically
>> > > > > > > > > > > > > > set the
>> > > > > > > > > > > > > > >> > > Tombstone bit
>> > > > internally. If
>> > > > > the
>> > > > > > > > > producer
>> > > > > > > > > > > > client(x+1)
>> > > > > > > > > > > > > sets
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > >> tombstone
>> > > > > > > > > > > > > > >> > > itself, well and
>> > good.
>> > > > For
>> > > > > > > producer
>> > > > > > > > > > client(x),
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > broker
>> > > > > > > > > > > > > > will up
>> > > > > > > > > > > > > > >> convert
>> > > > > > > > > > > > > > >> > > to have the
>> > tombstone
>> > > > bit.
>> > > > > > > > > Broker(x+1) is
>> > > > > > > > > > > > supporting
>> > > > > > > > > > > > > both.
>> > > > > > > > > > > > > > Consumer
>> > > > > > > > > > > > > > >> > > client(x+1) will
>> be
>> > > > aware of
>> > > > > > this
>> > > > > > > > and
>> > > > > > > > > should
>> > > > > > > > > > > be
>> > > > > > > > > > > > able
>> > > > > > > > > > > > > to
>> > > > > > > > > > > > > > handle this.
>> > > > > > > > > > > > > > >> For
>> > > > > > > > > > > > > > >> > > consumer
>> client(x)
>> > we
>> > > > will
>> > > > > down
>> > > > > > > > > convert the
>> > > > > > > > > > > > message
>> > > > > > > > > > > > > on the
>> > > > > > > > > > > > > > broker
>> > > > > > > > > > > > > > >> side.
>> > > > > > > > > > > > > > >> > > c) At this
>> point
>> > > we
>> > > > will
>> > > > > > have
>> > > > > > > to
>> > > > > > > > > > specify a
>> > > > > > > > > > > > > warning or
>> > > > > > > > > > > > > > clearly
>> > > > > > > > > > > > > > >> specify
>> > > > > > > > > > > > > > >> > > in docs that this
>> > > > behavior is
>> > > > > > > about
>> > > > > > > > > to be
>> > > > > > > > > > > > changed for
>> > > > > > > > > > > > > log
>> > > > > > > > > > > > > > compaction.
>> > > > > > > > > > > > > > >> > > 4) a) In next
>> > release
>> > > > of the
>> > > > > > > > > Broker(x+2), we
>> > > > > > > > > > > > say that
>> > > > > > > > > > > > > only
>> > > > > > > > > > > > > > Tombstone
>> > > > > > > > > > > > > > >> is
>> > > > > > > > > > > > > > >> > > used for log
>> > > compaction
>> > > > on the
>> > > > > > > > Broker
>> > > > > > > > > side.
>> > > > > > > > > > > > > Clients(x+1)
>> > > > > > > > > > > > > > still is
>> > > > > > > > > > > > > > >> > > supported.
>> > > > > > > > > > > > > > >> > > b) We upgrade
>> > the
>> > > > client
>> > > > > to
>> > > > > > > > > version
>> > > > > > > > > > > > client(x+2) :
>> > > > > > > > > > > > > if
>> > > > > > > > > > > > > > value is set
>> > > > > > > > > > > > > > >> to
>> > > > > > > > > > > > > > >> > > null, tombstone
>> will
>> > > > not be
>> > > > > set
>> > > > > > > > > > automatically.
>> > > > > > > > > > > > The
>> > > > > > > > > > > > > client
>> > > > > > > > > > > > > > will have to
>> > > > > > > > > > > > > > >> > call
>> > > > > > > > > > > > > > >> > > setTombstone() to
>> > > > actually set
>> > > > > > the
>> > > > > > > > > > tombstone.
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > We should compare
>> > this
>> > > > > migration
>> > > > > > > > plan
>> > > > > > > > > with
>> > > > > > > > > > the
>> > > > > > > > > > > > > migration
>> > > > > > > > > > > > > > plan for
>> > > > > > > > > > > > > > >> magic
>> > > > > > > > > > > > > > >> > > byte bump and do
>> > > > whatever
>> > > > > looks
>> > > > > > > > good.
>> > > > > > > > > > > > > > >> > > I am just worried
>> > that
>> > > > if we
>> > > > > go
>> > > > > > > down
>> > > > > > > > > magic
>> > > > > > > > > > > byte
>> > > > > > > > > > > > route,
>> > > > > > > > > > > > > > unless I am
>> > > > > > > > > > > > > > >> > missing
>> > > > > > > > > > > > > > >> > > something, it
>> sounds
>> > > > like
>> > > > > kafka
>> > > > > > > will
>> > > > > > > > > be
>> > > > > > > > > > stuck
>> > > > > > > > > > > > with
>> > > > > > > > > > > > > > supporting both
>> > > > > > > > > > > > > > >> null
>> > > > > > > > > > > > > > >> > > value and
>> tombstone
>> > > bit
>> > > > for
>> > > > > log
>> > > > > > > > > compaction
>> > > > > > > > > > for
>> > > > > > > > > > > > life
>> > > > > > > > > > > > > long,
>> > > > > > > > > > > > > > which does
>> > > > > > > > > > > > > > >> not
>> > > > > > > > > > > > > > >> > > look like a good
>> end
>> > > > state.
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > Thanks,
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > Mayuresh
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > On Wed, Nov 16,
>> 2016
>> > > at
>> > > > 9:32
>> > > > > AM,
>> > > > > > > > > Mayuresh
>> > > > > > > > > > > > Gharat <
>> > > > > > > > > > > > > > >> > >
>> > > > gharatmayuresh15@gmail.com
>> > > > > > > > > > > > > > >> > > > wrote:
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > > Hi Ismael,
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > That's a very
>> good
>> > > > point
>> > > > > > which I
>> > > > > > > > > might
>> > > > > > > > > > have
>> > > > > > > > > > > > not
>> > > > > > > > > > > > > > considered earlier.
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > Here is a plan
>> > that
>> > > I
>> > > > can
>> > > > > > think
>> > > > > > > > of:
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > Stage 1) The
>> > broker
>> > > > from now
>> > > > > > on,
>> > > > > > > > up
>> > > > > > > > > > converts
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > message
>> > > > > > > > > > > > > > to have the
>> > > > > > > > > > > > > > >> > > > tombstone
>> marker.
>> > > The
>> > > > log
>> > > > > > > > compaction
>> > > > > > > > > > thread
>> > > > > > > > > > > > does log
>> > > > > > > > > > > > > > compaction
>> > > > > > > > > > > > > > >> based
>> > > > > > > > > > > > > > >> > on
>> > > > > > > > > > > > > > >> > > > both null and
>> > > > tombstone
>> > > > > > marker.
>> > > > > > > > > This is
>> > > > > > > > > > our
>> > > > > > > > > > > > > transition
>> > > > > > > > > > > > > > period.
>> > > > > > > > > > > > > > >> > > > Stage 2) The
>> next
>> > > > release we
>> > > > > > > only
>> > > > > > > > > say that
>> > > > > > > > > > > log
>> > > > > > > > > > > > > compaction
>> > > > > > > > > > > > > > is based
>> > > > > > > > > > > > > > >> on
>> > > > > > > > > > > > > > >> > > > tombstone
>> marker.
>> > > > (Open
>> > > > > source
>> > > > > > > > > kafka makes
>> > > > > > > > > > > > this as a
>> > > > > > > > > > > > > > policy). By
>> > > > > > > > > > > > > > >> this
>> > > > > > > > > > > > > > >> > > time,
>> > > > > > > > > > > > > > >> > > > the
>> organization
>> > > > which is
>> > > > > > moving
>> > > > > > > > to
>> > > > > > > > > this
>> > > > > > > > > > > > release
>> > > > > > > > > > > > > will be
>> > > > > > > > > > > > > > sure that
>> > > > > > > > > > > > > > >> they
>> > > > > > > > > > > > > > >> > > > have gone
>> through
>> > > the
>> > > > entire
>> > > > > > > > > transition
>> > > > > > > > > > > > period.
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > My only goal of
>> > > doing
>> > > > this
>> > > > > is
>> > > > > > > that
>> > > > > > > > > Kafka
>> > > > > > > > > > > > clearly
>> > > > > > > > > > > > > > specifies the end
>> > > > > > > > > > > > > > >> > state
>> > > > > > > > > > > > > > >> > > > about what log
>> > > > compaction
>> > > > > > means
>> > > > > > > > (is
>> > > > > > > > > it
>> > > > > > > > > > null
>> > > > > > > > > > > > value
>> > > > > > > > > > > > > or a
>> > > > > > > > > > > > > > tombstone
>> > > > > > > > > > > > > > >> > marker,
>> > > > > > > > > > > > > > >> > > > but not both).
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > What do you
>> think?
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > Thanks,
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > Mayuresh
>> > > > > > > > > > > > > > >> > > > .
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > On Wed, Nov 16,
>> > 2016
>> > > > at 9:17
>> > > > > > AM,
>> > > > > > > > > Ismael
>> > > > > > > > > > > Juma <
>> > > > > > > > > > > > > > ismael@juma.me.uk>
>> > > > > > > > > > > > > > >> > wrote:
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > >> One comment
>> > below.
>> > > > > > > > > > > > > > >> > > >>
>> > > > > > > > > > > > > > >> > > >> On Wed, Nov
>> 16,
>> > > 2016
>> > > > at
>> > > > > 5:08
>> > > > > > > PM,
>> > > > > > > > > Mayuresh
>> > > > > > > > > > > > Gharat <
>> > > > > > > > > > > > > > >> > > >>
>> > > > gharatmayuresh15@gmail.com
>> > > > > > > > > > > > > > >> > > >> > wrote:
>> > > > > > > > > > > > > > >> > > >>
>> > > > > > > > > > > > > > >> > > >> > - If we
>> > don't
>> > > > bump up
>> > > > > > the
>> > > > > > > > > magic
>> > > > > > > > > > byte,
>> > > > > > > > > > > > on the
>> > > > > > > > > > > > > broker
>> > > > > > > > > > > > > > side, the
>> > > > > > > > > > > > > > >> > > broker
>> > > > > > > > > > > > > > >> > > >> > will
>> always
>> > > > have to
>> > > > > look
>> > > > > > > at
>> > > > > > > > > both
>> > > > > > > > > > > > tombstone
>> > > > > > > > > > > > > bit and
>> > > > > > > > > > > > > > the value
>> > > > > > > > > > > > > > >> when
>> > > > > > > > > > > > > > >> > > do
>> > > > > > > > > > > > > > >> > > >> the
>> > > > > > > > > > > > > > >> > > >> >
>> compaction.
>> > > > Assuming
>> > > > > we
>> > > > > > do
>> > > > > > > > > not bump
>> > > > > > > > > > up
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > magic
>> > > > > > > > > > > > > > byte,
>> > > > > > > > > > > > > > >> > > >> > imagine
>> the
>> > > > broker
>> > > > > sees
>> > > > > > a
>> > > > > > > > > message
>> > > > > > > > > > > which
>> > > > > > > > > > > > does
>> > > > > > > > > > > > > not
>> > > > > > > > > > > > > > have a
>> > > > > > > > > > > > > > >> tombstone
>> > > > > > > > > > > > > > >> > > bit
>> > > > > > > > > > > > > > >> > > >> > set. The
>> > > broker
>> > > > does
>> > > > > not
>> > > > > > > > know
>> > > > > > > > > when
>> > > > > > > > > > the
>> > > > > > > > > > > > > message was
>> > > > > > > > > > > > > > produced
>> > > > > > > > > > > > > > >> (i.e.
>> > > > > > > > > > > > > > >> > > >> > whether
>> > > > > > > > > > > > > > >> > > >> > the
>> message
>> > > has
>> > > > been
>> > > > > up
>> > > > > > > > > converted or
>> > > > > > > > > > > > not), it
>> > > > > > > > > > > > > has
>> > > > > > > > > > > > > > to take a
>> > > > > > > > > > > > > > >> > further
>> > > > > > > > > > > > > > >> > > >> > look at
>> > > > > > > > > > > > > > >> > > >> > the
>> value to
>> > > > see if it
>> > > > > > is
>> > > > > > > > > null or
>> > > > > > > > > > not
>> > > > > > > > > > > in
>> > > > > > > > > > > > > order to
>> > > > > > > > > > > > > > determine
>> > > > > > > > > > > > > > >> if it
>> > > > > > > > > > > > > > >> > > is
>> > > > > > > > > > > > > > >> > > >> a
>> > > > > > > > > > > > > > >> > > >> >
>> tombstone.
>> > The
>> > > > same
>> > > > > > logic
>> > > > > > > > has
>> > > > > > > > > to be
>> > > > > > > > > > > put
>> > > > > > > > > > > > on the
>> > > > > > > > > > > > > > consumer as
>> > > > > > > > > > > > > > >> well
>> > > > > > > > > > > > > > >> > > >> because
>> > > > > > > > > > > > > > >> > > >> > the
>> > > > > > > > > > > > > > >> > > >> > consumer
>> > does
>> > > > not know
>> > > > > > if
>> > > > > > > > the
>> > > > > > > > > > message
>> > > > > > > > > > > > has
>> > > > > > > > > > > > > been up
>> > > > > > > > > > > > > > converted or
>> > > > > > > > > > > > > > >> > not.
>> > > > > > > > > > > > > > >> > > >> > - If
>> we
>> > > > upconvert
>> > > > > > while
>> > > > > > > > > > appending,
>> > > > > > > > > > > > this is
>> > > > > > > > > > > > > not
>> > > > > > > > > > > > > > the case,
>> > > > > > > > > > > > > > >> > right?
>> > > > > > > > > > > > > > >> > > >>
>> > > > > > > > > > > > > > >> > > >>
>> > > > > > > > > > > > > > >> > > >> If I
>> understand
>> > you
>> > > > > > correctly,
>> > > > > > > > > this is
>> > > > > > > > > > not
>> > > > > > > > > > > > > sufficient
>> > > > > > > > > > > > > > because the
>> > > > > > > > > > > > > > >> log
>> > > > > > > > > > > > > > >> > > may
>> > > > > > > > > > > > > > >> > > >> have messages
>> > > > appended
>> > > > > before
>> > > > > > > it
>> > > > > > > > > was
>> > > > > > > > > > > > upgraded to
>> > > > > > > > > > > > > include
>> > > > > > > > > > > > > > KIP-87.
>> > > > > > > > > > > > > > >> > > >>
>> > > > > > > > > > > > > > >> > > >> Ismael
>> > > > > > > > > > > > > > >> > > >>
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > > > --
>> > > > > > > > > > > > > > >> > > > -Regards,
>> > > > > > > > > > > > > > >> > > > Mayuresh R.
>> Gharat
>> > > > > > > > > > > > > > >> > > > (862) 250-7125
>> > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > > --
>> > > > > > > > > > > > > > >> > > -Regards,
>> > > > > > > > > > > > > > >> > > Mayuresh R.
>> Gharat
>> > > > > > > > > > > > > > >> > > (862) 250-7125
>> > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > >> > The information
>> > > contained
>> > > > in
>> > > > > this
>> > > > > > > > email
>> > > > > > > > > is
>> > > > > > > > > > > > strictly
>> > > > > > > > > > > > > > confidential and for
>> > > > > > > > > > > > > > >> > the use of the
>> > addressee
>> > > > only,
>> > > > > > > unless
>> > > > > > > > > > otherwise
>> > > > > > > > > > > > > indicated. If
>> > > > > > > > > > > > > > you are
>> > > > > > > > > > > > > > >> not
>> > > > > > > > > > > > > > >> > the intended
>> > recipient,
>> > > > please
>> > > > > do
>> > > > > > > not
>> > > > > > > > > read,
>> > > > > > > > > > > copy,
>> > > > > > > > > > > > use or
>> > > > > > > > > > > > > > disclose to
>> > > > > > > > > > > > > > >> others
>> > > > > > > > > > > > > > >> > this message or any
>> > > > attachment.
>> > > > > > > Please
>> > > > > > > > > also
>> > > > > > > > > > > > notify the
>> > > > > > > > > > > > > sender
>> > > > > > > > > > > > > > by
>> > > > > > > > > > > > > > >> replying
>> > > > > > > > > > > > > > >> > to this email or by
>> > > > telephone
>> > > > > > > (+44(020
>> > > > > > > > > 7896
>> > > > > > > > > > > 0011)
>> > > > > > > > > > > > and
>> > > > > > > > > > > > > then
>> > > > > > > > > > > > > > delete the
>> > > > > > > > > > > > > > >> email
>> > > > > > > > > > > > > > >> > and any copies of
>> it.
>> > > > Opinions,
>> > > > > > > > > conclusion
>> > > > > > > > > > (etc)
>> > > > > > > > > > > > that
>> > > > > > > > > > > > > do not
>> > > > > > > > > > > > > > relate to
>> > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > >> > official business
>> of
>> > > this
>> > > > > company
>> > > > > > > > shall
>> > > > > > > > > be
>> > > > > > > > > > > > understood as
>> > > > > > > > > > > > > > neither given
>> > > > > > > > > > > > > > >> nor
>> > > > > > > > > > > > > > >> > endorsed by it. IG
>> is
>> > a
>> > > > trading
>> > > > > > name
>> > > > > > > > of
>> > > > > > > > > IG
>> > > > > > > > > > > Markets
>> > > > > > > > > > > > > Limited (a
>> > > > > > > > > > > > > > company
>> > > > > > > > > > > > > > >> > registered in
>> England
>> > > and
>> > > > Wales,
>> > > > > > > > company
>> > > > > > > > > > number
>> > > > > > > > > > > > > 04008957) and
>> > > > > > > > > > > > > > IG Index
>> > > > > > > > > > > > > > >> > Limited (a company
>> > > > registered in
>> > > > > > > > > England and
>> > > > > > > > > > > > Wales,
>> > > > > > > > > > > > > company
>> > > > > > > > > > > > > > number
>> > > > > > > > > > > > > > >> > 01190902).
>> Registered
>> > > > address at
>> > > > > > > > Cannon
>> > > > > > > > > Bridge
>> > > > > > > > > > > > House, 25
>> > > > > > > > > > > > > > Dowgate Hill,
>> > > > > > > > > > > > > > >> > London EC4R 2YA.
>> Both
>> > IG
>> > > > Markets
>> > > > > > > > Limited
>> > > > > > > > > > > (register
>> > > > > > > > > > > > > number
>> > > > > > > > > > > > > > 195355) and IG
>> > > > > > > > > > > > > > >> > Index Limited
>> > (register
>> > > > number
>> > > > > > > 114059)
>> > > > > > > > > are
>> > > > > > > > > > > > authorised
>> > > > > > > > > > > > > and
>> > > > > > > > > > > > > > regulated by
>> > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > >> > Financial Conduct
>> > > > Authority.
>> > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >> --
>> > > > > > > > > > > > > > >> -Regards,
>> > > > > > > > > > > > > > >> Mayuresh R. Gharat
>> > > > > > > > > > > > > > >> (862) 250-7125
>> > > > > > > > > > > > > > >> The information
>> > contained
>> > > > in this
>> > > > > > > email
>> > > > > > > > is
>> > > > > > > > > > > strictly
>> > > > > > > > > > > > > > confidential and for
>> > > > > > > > > > > > > > >> the use of the
>> addressee
>> > > > only,
>> > > > > > unless
>> > > > > > > > > otherwise
>> > > > > > > > > > > > > indicated. If
>> > > > > > > > > > > > > > you are not
>> > > > > > > > > > > > > > >> the intended
>> recipient,
>> > > > please do
>> > > > > > not
>> > > > > > > > > read,
>> > > > > > > > > > copy,
>> > > > > > > > > > > > use or
>> > > > > > > > > > > > > > disclose to others
>> > > > > > > > > > > > > > >> this message or any
>> > > > attachment.
>> > > > > > Please
>> > > > > > > > > also
>> > > > > > > > > > notify
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > sender
>> > > > > > > > > > > > > > by replying
>> > > > > > > > > > > > > > >> to this email or by
>> > > > telephone
>> > > > > > (+44(020
>> > > > > > > > > 7896
>> > > > > > > > > > 0011)
>> > > > > > > > > > > > and then
>> > > > > > > > > > > > > > delete the email
>> > > > > > > > > > > > > > >> and any copies of it.
>> > > > Opinions,
>> > > > > > > > > conclusion (etc)
>> > > > > > > > > > > > that do
>> > > > > > > > > > > > > not
>> > > > > > > > > > > > > > relate to the
>> > > > > > > > > > > > > > >> official business of
>> > this
>> > > > company
>> > > > > > > shall
>> > > > > > > > be
>> > > > > > > > > > > > understood as
>> > > > > > > > > > > > > > neither given nor
>> > > > > > > > > > > > > > >> endorsed by it. IG
>> is a
>> > > > trading
>> > > > > name
>> > > > > > > of
>> > > > > > > > IG
>> > > > > > > > > > Markets
>> > > > > > > > > > > > > Limited (a
>> > > > > > > > > > > > > > company
>> > > > > > > > > > > > > > >> registered in England
>> > and
>> > > > Wales,
>> > > > > > > company
>> > > > > > > > > number
>> > > > > > > > > > > > 04008957)
>> > > > > > > > > > > > > and
>> > > > > > > > > > > > > > IG Index
>> > > > > > > > > > > > > > >> Limited (a company
>> > > > registered in
>> > > > > > > England
>> > > > > > > > > and
>> > > > > > > > > > > Wales,
>> > > > > > > > > > > > > company
>> > > > > > > > > > > > > > number
>> > > > > > > > > > > > > > >> 01190902). Registered
>> > > > address at
>> > > > > > > Cannon
>> > > > > > > > > Bridge
>> > > > > > > > > > > > House, 25
>> > > > > > > > > > > > > > Dowgate Hill,
>> > > > > > > > > > > > > > >> London EC4R 2YA.
>> Both IG
>> > > > Markets
>> > > > > > > Limited
>> > > > > > > > > > (register
>> > > > > > > > > > > > number
>> > > > > > > > > > > > > > 195355) and IG
>> > > > > > > > > > > > > > >> Index Limited
>> (register
>> > > > number
>> > > > > > 114059)
>> > > > > > > > are
>> > > > > > > > > > > > authorised and
>> > > > > > > > > > > > > > regulated by the
>> > > > > > > > > > > > > > >> Financial Conduct
>> > > Authority.
>> > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > --
>> > > > > > > > > > > > > > > -Regards,
>> > > > > > > > > > > > > > > Mayuresh R. Gharat
>> > > > > > > > > > > > > > > (862) 250-7125
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > --
>> > > > > > > > > > > > > > -Regards,
>> > > > > > > > > > > > > > Mayuresh R. Gharat
>> > > > > > > > > > > > > > (862) 250-7125
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > The information contained in
>> > this
>> > > > email
>> > > > > is
>> > > > > > > > > strictly
>> > > > > > > > > > > > confidential
>> > > > > > > > > > > > > and
>> > > > > > > > > > > > > > for the use of the addressee
>> only,
>> > > > unless
>> > > > > > > otherwise
>> > > > > > > > > > > indicated.
>> > > > > > > > > > > > If
>> > > > > > > > > > > > > you are
>> > > > > > > > > > > > > > not the intended recipient,
>> please
>> > do
>> > > > not
>> > > > > read,
>> > > > > > > > > copy, use
>> > > > > > > > > > or
>> > > > > > > > > > > > > disclose to
>> > > > > > > > > > > > > > others this message or any
>> > > attachment.
>> > > > Please
>> > > > > > > also
>> > > > > > > > > notify
>> > > > > > > > > > the
>> > > > > > > > > > > > sender
>> > > > > > > > > > > > > by
>> > > > > > > > > > > > > > replying to this email or by
>> > > telephone
>> > > > > (+44(020
>> > > > > > > > 7896
>> > > > > > > > > 0011)
>> > > > > > > > > > > and
>> > > > > > > > > > > > then
>> > > > > > > > > > > > > delete
>> > > > > > > > > > > > > > the email and any copies of it.
>> > > > Opinions,
>> > > > > > > > conclusion
>> > > > > > > > > (etc)
>> > > > > > > > > > > > that do
>> > > > > > > > > > > > > not
>> > > > > > > > > > > > > > relate to the official business
>> of
>> > > this
>> > > > > company
>> > > > > > > > > shall be
>> > > > > > > > > > > > understood
>> > > > > > > > > > > > > as
>> > > > > > > > > > > > > > neither given nor endorsed by
>> it.
>> > IG
>> > > > is a
>> > > > > > trading
>> > > > > > > > > name of
>> > > > > > > > > > IG
>> > > > > > > > > > > > Markets
>> > > > > > > > > > > > > > Limited (a company registered in
>> > > > England and
>> > > > > > > Wales,
>> > > > > > > > > company
>> > > > > > > > > > > > number
>> > > > > > > > > > > > > > 04008957) and IG Index Limited
>> (a
>> > > > company
>> > > > > > > > registered
>> > > > > > > > > in
>> > > > > > > > > > > > England and
>> > > > > > > > > > > > > Wales,
>> > > > > > > > > > > > > > company number 01190902).
>> > Registered
>> > > > address
>> > > > > at
>> > > > > > > > > Cannon
>> > > > > > > > > > Bridge
>> > > > > > > > > > > > House,
>> > > > > > > > > > > > > 25
>> > > > > > > > > > > > > > Dowgate Hill, London EC4R 2YA.
>> Both
>> > > IG
>> > > > > Markets
>> > > > > > > > > Limited
>> > > > > > > > > > > > (register
>> > > > > > > > > > > > > number
>> > > > > > > > > > > > > > 195355) and IG Index Limited
>> > > (register
>> > > > number
>> > > > > > > > > 114059) are
>> > > > > > > > > > > > authorised
>> > > > > > > > > > > > > and
>> > > > > > > > > > > > > > regulated by the Financial
>> Conduct
>> > > > Authority.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > The information contained in this
>> email
>> > is
>> > > > strictly
>> > > > > > > > > confidential
>> > > > > > > > > > > and
>> > > > > > > > > > > > for
>> > > > > > > > > > > > > the use of the addressee only, unless
>> > > > otherwise
>> > > > > > > > indicated.
>> > > > > > > > > If you
>> > > > > > > > > > > > are not
>> > > > > > > > > > > > > the intended recipient, please do not
>> > read,
>> > > > copy,
>> > > > > use
>> > > > > > > or
>> > > > > > > > > disclose
>> > > > > > > > > > > to
>> > > > > > > > > > > > others
>> > > > > > > > > > > > > this message or any attachment. Please
>> > also
>> > > > notify
>> > > > > > the
>> > > > > > > > > sender by
>> > > > > > > > > > > > replying
>> > > > > > > > > > > > > to this email or by telephone (+44(020
>> > 7896
>> > > > 0011
>> > > > > <020%207896%200011>) and
>> > > > > > > > then
>> > > > > > > > > delete
>> > > > > > > > > > > > the email
>> > > > > > > > > > > > > and any copies of it. Opinions,
>> > conclusion
>> > > > (etc)
>> > > > > that
>> > > > > > > do
>> > > > > > > > > not
>> > > > > > > > > > relate
>> > > > > > > > > > > > to the
>> > > > > > > > > > > > > official business of this company
>> shall
>> > be
>> > > > > understood
>> > > > > > > as
>> > > > > > > > > neither
>> > > > > > > > > > > > given nor
>> > > > > > > > > > > > > endorsed by it. IG is a trading name
>> of
>> > IG
>> > > > Markets
>> > > > > > > > Limited
>> > > > > > > > > (a
>> > > > > > > > > > > company
>> > > > > > > > > > > > > registered in England and Wales,
>> company
>> > > > number
>> > > > > > > 04008957)
>> > > > > > > > > and IG
>> > > > > > > > > > > > Index
>> > > > > > > > > > > > > Limited (a company registered in
>> England
>> > > and
>> > > > Wales,
>> > > > > > > > company
>> > > > > > > > > > number
>> > > > > > > > > > > > > 01190902). Registered address at
>> Cannon
>> > > > Bridge
>> > > > > House,
>> > > > > > > 25
>> > > > > > > > > Dowgate
>> > > > > > > > > > > > Hill,
>> > > > > > > > > > > > > London EC4R 2YA. Both IG Markets
>> Limited
>> > > > (register
>> > > > > > > number
>> > > > > > > > > 195355)
>> > > > > > > > > > > > and IG
>> > > > > > > > > > > > > Index Limited (register number 114059)
>> > are
>> > > > > authorised
>> > > > > > > and
>> > > > > > > > > > regulated
>> > > > > > > > > > > > by the
>> > > > > > > > > > > > > Financial Conduct Authority.
>> > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > The information contained in this email is
>> > > strictly
>> > > > > > > > confidential
>> > > > > > > > > and
>> > > > > > > > > > for
>> > > > > > > > > > > > the use of the addressee only, unless
>> otherwise
>> > > > > indicated.
>> > > > > > If
>> > > > > > > > > you are
>> > > > > > > > > > not
>> > > > > > > > > > > > the intended recipient, please do not read,
>> > copy,
>> > > > use or
>> > > > > > > > > disclose to
>> > > > > > > > > > > others
>> > > > > > > > > > > > this message or any attachment. Please also
>> > > notify
>> > > > the
>> > > > > > sender
>> > > > > > > > by
>> > > > > > > > > > replying
>> > > > > > > > > > > > to this email or by telephone (+44(020 7896
>> > 0011
>> > > > > <020%207896%200011>) and then
>> > > > > > > > > delete the
>> > > > > > > > > > > email
>> > > > > > > > > > > > and any copies of it. Opinions, conclusion
>> > (etc)
>> > > > that do
>> > > > > > not
>> > > > > > > > > relate to
>> > > > > > > > > > > the
>> > > > > > > > > > > > official business of this company shall be
>> > > > understood as
>> > > > > > > > neither
>> > > > > > > > > given
>> > > > > > > > > > > nor
>> > > > > > > > > > > > endorsed by it. IG is a trading name of IG
>> > > Markets
>> > > > > Limited
>> > > > > > (a
>> > > > > > > > > company
>> > > > > > > > > > > > registered in England and Wales, company
>> number
>> > > > 04008957)
>> > > > > > and
>> > > > > > > > IG
>> > > > > > > > > Index
>> > > > > > > > > > > > Limited (a company registered in England and
>> > > Wales,
>> > > > > company
>> > > > > > > > > number
>> > > > > > > > > > > > 01190902). Registered address at Cannon
>> Bridge
>> > > > House, 25
>> > > > > > > > Dowgate
>> > > > > > > > > Hill,
>> > > > > > > > > > > > London EC4R 2YA. Both IG Markets Limited
>> > > (register
>> > > > number
>> > > > > > > > > 195355) and
>> > > > > > > > > > IG
>> > > > > > > > > > > > Index Limited (register number 114059) are
>> > > > authorised and
>> > > > > > > > > regulated by
>> > > > > > > > > > > the
>> > > > > > > > > > > > Financial Conduct Authority.
>> > > > > > > > > > > >
>> > > > > > > > > > > The information contained in this email is
>> > strictly
>> > > > > > > confidential
>> > > > > > > > > and for
>> > > > > > > > > > > the use of the addressee only, unless
>> otherwise
>> > > > indicated.
>> > > > > If
>> > > > > > > you
>> > > > > > > > > are not
>> > > > > > > > > > > the intended recipient, please do not read,
>> copy,
>> > > > use or
>> > > > > > > disclose
>> > > > > > > > > to
>> > > > > > > > > > others
>> > > > > > > > > > > this message or any attachment. Please also
>> > notify
>> > > > the
>> > > > > sender
>> > > > > > > by
>> > > > > > > > > replying
>> > > > > > > > > > > to this email or by telephone (+44(020 7896
>> 0011
>> > > > > <020%207896%200011>) and then
>> > > > > > > delete
>> > > > > > > > > the
>> > > > > > > > > > email
>> > > > > > > > > > > and any copies of it. Opinions, conclusion
>> (etc)
>> > > > that do
>> > > > > not
>> > > > > > > > > relate to
>> > > > > > > > > > the
>> > > > > > > > > > > official business of this company shall be
>> > > > understood as
>> > > > > > > neither
>> > > > > > > > > given
>> > > > > > > > > > nor
>> > > > > > > > > > > endorsed by it. IG is a trading name of IG
>> > Markets
>> > > > Limited
>> > > > > (a
>> > > > > > > > > company
>> > > > > > > > > > > registered in England and Wales, company
>> number
>> > > > 04008957)
>> > > > > and
>> > > > > > > IG
>> > > > > > > > > Index
>> > > > > > > > > > > Limited (a company registered in England and
>> > Wales,
>> > > > company
>> > > > > > > > number
>> > > > > > > > > > > 01190902). Registered address at Cannon Bridge
>> > > > House, 25
>> > > > > > > Dowgate
>> > > > > > > > > Hill,
>> > > > > > > > > > > London EC4R 2YA. Both IG Markets Limited
>> > (register
>> > > > number
>> > > > > > > 195355)
>> > > > > > > > > and IG
>> > > > > > > > > > > Index Limited (register number 114059) are
>> > > > authorised and
>> > > > > > > > > regulated by
>> > > > > > > > > > the
>> > > > > > > > > > > Financial Conduct Authority.
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > The information contained in this email is
>> strictly
>> > > > > > confidential
>> > > > > > > > and
>> > > > > > > > > for
>> > > > > > > > > > the use of the addressee only, unless otherwise
>> > > > indicated. If
>> > > > > > you
>> > > > > > > > > are not
>> > > > > > > > > > the intended recipient, please do not read,
>> copy,
>> > use
>> > > > or
>> > > > > > disclose
>> > > > > > > > to
>> > > > > > > > > others
>> > > > > > > > > > this message or any attachment. Please also
>> notify
>> > > the
>> > > > sender
>> > > > > > by
>> > > > > > > > > replying
>> > > > > > > > > > to this email or by telephone (+44(020 7896 0011
>> > > > > <020%207896%200011>) and then
>> > > > > > delete
>> > > > > > > > > the email
>> > > > > > > > > > and any copies of it. Opinions, conclusion (etc)
>> > that
>> > > > do not
>> > > > > > > relate
>> > > > > > > > > to the
>> > > > > > > > > > official business of this company shall be
>> > understood
>> > > > as
>> > > > > > neither
>> > > > > > > > > given nor
>> > > > > > > > > > endorsed by it. IG is a trading name of IG
>> Markets
>> > > > Limited (a
>> > > > > > > > company
>> > > > > > > > > > registered in England and Wales, company number
>> > > > 04008957) and
>> > > > > > IG
>> > > > > > > > > Index
>> > > > > > > > > > Limited (a company registered in England and
>> Wales,
>> > > > company
>> > > > > > > number
>> > > > > > > > > > 01190902). Registered address at Cannon Bridge
>> > House,
>> > > > 25
>> > > > > > Dowgate
>> > > > > > > > > Hill,
>> > > > > > > > > > London EC4R 2YA. Both IG Markets Limited
>> (register
>> > > > number
>> > > > > > 195355)
>> > > > > > > > > and IG
>> > > > > > > > > > Index Limited (register number 114059) are
>> > authorised
>> > > > and
>> > > > > > > regulated
>> > > > > > > > > by the
>> > > > > > > > > > Financial Conduct Authority.
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > The information contained in this email is strictly
>> > > > confidential
>> > > > > and
>> > > > > > > for
>> > > > > > > > > the use of the addressee only, unless otherwise
>> > indicated.
>> > > > If you
>> > > > > are
>> > > > > > > not
>> > > > > > > > > the intended recipient, please do not read, copy, use
>> or
>> > > > disclose
>> > > > > to
>> > > > > > > > others
>> > > > > > > > > this message or any attachment. Please also notify the
>> > > > sender by
>> > > > > > > replying
>> > > > > > > > > to this email or by telephone (+44(020 7896 0011
>> > > > > <020%207896%200011>) and then delete the
>> > > > > > > > email
>> > > > > > > > > and any copies of it. Opinions, conclusion (etc) that
>> do
>> > > not
>> > > > relate
>> > > > > > to
>> > > > > > > > the
>> > > > > > > > > official business of this company shall be understood
>> as
>> > > > neither
>> > > > > > given
>> > > > > > > > nor
>> > > > > > > > > endorsed by it. IG is a trading name of IG Markets
>> > Limited
>> > > (a
>> > > > > company
>> > > > > > > > > registered in England and Wales, company number
>> 04008957)
>> > > > and IG
>> > > > > > Index
>> > > > > > > > > Limited (a company registered in England and Wales,
>> > company
>> > > > number
>> > > > > > > > > 01190902). Registered address at Cannon Bridge House,
>> 25
>> > > > Dowgate
>> > > > > > Hill,
>> > > > > > > > > London EC4R 2YA. Both IG Markets Limited (register
>> number
>> > > > 195355)
>> > > > > and
>> > > > > > > IG
>> > > > > > > > > Index Limited (register number 114059) are authorised
>> and
>> > > > regulated
>> > > > > > by
>> > > > > > > > the
>> > > > > > > > > Financial Conduct Authority.
>> > > > > > > > >
>> > > > > > > > The information contained in this email is strictly
>> > > > confidential and
>> > > > > > for
>> > > > > > > > the use of the addressee only, unless otherwise
>> indicated.
>> > If
>> > > > you are
>> > > > > > not
>> > > > > > > > the intended recipient, please do not read, copy, use or
>> > > > disclose to
>> > > > > > > others
>> > > > > > > > this message or any attachment. Please also notify the
>> > sender
>> > > > by
>> > > > > > replying
>> > > > > > > > to this email or by telephone (+44(020 7896 0011
>> > > > <020%207896%200011>)
>> > > > > and then delete the
>> > > > > > > email
>> > > > > > > > and any copies of it. Opinions, conclusion (etc) that do
>> > not
>> > > > relate
>> > > > > to
>> > > > > > > the
>> > > > > > > > official business of this company shall be understood as
>> > > > neither
>> > > > > given
>> > > > > > > nor
>> > > > > > > > endorsed by it. IG is a trading name of IG Markets
>> Limited
>> > (a
>> > > > company
>> > > > > > > > registered in England and Wales, company number
>> 04008957)
>> > and
>> > > > IG
>> > > > > Index
>> > > > > > > > Limited (a company registered in England and Wales,
>> company
>> > > > number
>> > > > > > > > 01190902). Registered address at Cannon Bridge House, 25
>> > > > Dowgate
>> > > > > Hill,
>> > > > > > > > London EC4R 2YA. Both IG Markets Limited (register
>> number
>> > > > 195355) and
>> > > > > > IG
>> > > > > > > > Index Limited (register number 114059) are authorised
>> and
>> > > > regulated
>> > > > > by
>> > > > > > > the
>> > > > > > > > Financial Conduct Authority.
>> > > > > > > >
>> > > > > > > The information contained in this email is strictly
>> > > confidential
>> > > > and
>> > > > > for
>> > > > > > > the use of the addressee only, unless otherwise
>> indicated. If
>> > > > you are
>> > > > > not
>> > > > > > > the intended recipient, please do not read, copy, use or
>> > > > disclose to
>> > > > > > others
>> > > > > > > this message or any attachment. Please also notify the
>> sender
>> > > by
>> > > > > replying
>> > > > > > > to this email or by telephone (+44(020 7896 0011
>> > > > <020%207896%200011>)
>> > > > > and then delete the
>> > > > > > email
>> > > > > > > and any copies of it. Opinions, conclusion (etc) that do
>> not
>> > > > relate to
>> > > > > > the
>> > > > > > > official business of this company shall be understood as
>> > > neither
>> > > > given
>> > > > > > nor
>> > > > > > > endorsed by it. IG is a trading name of IG Markets
>> Limited (a
>> > > > company
>> > > > > > > registered in England and Wales, company number 04008957)
>> and
>> > > IG
>> > > > Index
>> > > > > > > Limited (a company registered in England and Wales,
>> company
>> > > > number
>> > > > > > > 01190902). Registered address at Cannon Bridge House, 25
>> > > Dowgate
>> > > > Hill,
>> > > > > > > London EC4R 2YA. Both IG Markets Limited (register number
>> > > > 195355) and
>> > > > > IG
>> > > > > > > Index Limited (register number 114059) are authorised and
>> > > > regulated by
>> > > > > > the
>> > > > > > > Financial Conduct Authority.
>> > > > > > >
>> > > > > > The information contained in this email is strictly
>> > confidential
>> > > > and for
>> > > > > > the use of the addressee only, unless otherwise indicated.
>> If
>> > you
>> > > > are not
>> > > > > > the intended recipient, please do not read, copy, use or
>> > disclose
>> > > > to
>> > > > > others
>> > > > > > this message or any attachment. Please also notify the
>> sender
>> > by
>> > > > replying
>> > > > > > to this email or by telephone (+44(020 7896 0011
>> > > > <020%207896%200011>)
>> > > > > and then delete the email
>> > > > > > and any copies of it. Opinions, conclusion (etc) that do not
>> > > > relate to
>> > > > > the
>> > > > > > official business of this company shall be understood as
>> > neither
>> > > > given
>> > > > > nor
>> > > > > > endorsed by it. IG is a trading name of IG Markets Limited
>> (a
>> > > > company
>> > > > > > registered in England and Wales, company number 04008957)
>> and
>> > IG
>> > > > Index
>> > > > > > Limited (a company registered in England and Wales, company
>> > > number
>> > > > > > 01190902). Registered address at Cannon Bridge House, 25
>> > Dowgate
>> > > > Hill,
>> > > > > > London EC4R 2YA. Both IG Markets Limited (register number
>> > 195355)
>> > > > and IG
>> > > > > > Index Limited (register number 114059) are authorised and
>> > > > regulated by
>> > > > > the
>> > > > > > Financial Conduct Authority.
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > > The information contained in this email is strictly confidential and
>> > for
>> > > > the use of the addressee only, unless otherwise indicated. If you
>> are
>> > not
>> > > > the intended recipient, please do not read, copy, use or disclose to
>> > > others
>> > > > this message or any attachment. Please also notify the sender by
>> > replying
>> > > > to this email or by telephone (+44(020 7896 0011) and then delete
>> the
>> > > email
>> > > > and any copies of it. Opinions, conclusion (etc) that do not relate
>> to
>> > > the
>> > > > official business of this company shall be understood as neither
>> given
>> > > nor
>> > > > endorsed by it. IG is a trading name of IG Markets Limited (a
>> company
>> > > > registered in England and Wales, company number 04008957) and IG
>> Index
>> > > > Limited (a company registered in England and Wales, company number
>> > > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate
>> Hill,
>> > > > London EC4R 2YA. Both IG Markets Limited (register number 195355)
>> and
>> > IG
>> > > > Index Limited (register number 114059) are authorised and regulated
>> by
>> > > the
>> > > > Financial Conduct Authority.
>> > > >
>> > > The information contained in this email is strictly confidential and
>> for
>> > > the use of the addressee only, unless otherwise indicated. If you are
>> not
>> > > the intended recipient, please do not read, copy, use or disclose to
>> > others
>> > > this message or any attachment. Please also notify the sender by
>> replying
>> > > to this email or by telephone (+44(020 7896 0011) and then delete the
>> > email
>> > > and any copies of it. Opinions, conclusion (etc) that do not relate to
>> > the
>> > > official business of this company shall be understood as neither given
>> > nor
>> > > endorsed by it. IG is a trading name of IG Markets Limited (a company
>> > > registered in England and Wales, company number 04008957) and IG Index
>> > > Limited (a company registered in England and Wales, company number
>> > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
>> > > London EC4R 2YA. Both IG Markets Limited (register number 195355) and
>> IG
>> > > Index Limited (register number 114059) are authorised and regulated by
>> > the
>> > > Financial Conduct Authority.
>> > >
>> > The information contained in this email is strictly confidential and for
>> > the use of the addressee only, unless otherwise indicated. If you are
>> not
>> > the intended recipient, please do not read, copy, use or disclose to
>> others
>> > this message or any attachment. Please also notify the sender by
>> replying
>> > to this email or by telephone (+44(020 7896 0011) and then delete the
>> email
>> > and any copies of it. Opinions, conclusion (etc) that do not relate to
>> the
>> > official business of this company shall be understood as neither given
>> nor
>> > endorsed by it. IG is a trading name of IG Markets Limited (a company
>> > registered in England and Wales, company number 04008957) and IG Index
>> > Limited (a company registered in England and Wales, company number
>> > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
>> > London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG
>> > Index Limited (register number 114059) are authorised and regulated by
>> the
>> > Financial Conduct Authority.
>> >
>>
>
>
The information contained in this email is strictly confidential and for the use of the addressee only, unless otherwise indicated. If you are not the intended recipient, please do not read, copy, use or disclose to others this message or any attachment. Please also notify the sender by replying to this email or by telephone (+44(020 7896 0011) and then delete the email and any copies of it. Opinions, conclusion (etc) that do not relate to the official business of this company shall be understood as neither given nor endorsed by it. IG is a trading name of IG Markets Limited (a company registered in England and Wales, company number 04008957) and IG Index Limited (a company registered in England and Wales, company number 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG Index Limited (register number 114059) are authorised and regulated by the Financial Conduct Authority.
The information contained in this email is strictly confidential and for the use of the addressee only, unless otherwise indicated. If you are not the intended recipient, please do not read, copy, use or disclose to others this message or any attachment. Please also notify the sender by replying to this email or by telephone (+44(020 7896 0011) and then delete the email and any copies of it. Opinions, conclusion (etc) that do not relate to the official business of this company shall be understood as neither given nor endorsed by it. IG is a trading name of IG Markets Limited (a company registered in England and Wales, company number 04008957) and IG Index Limited (a company registered in England and Wales, company number 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG Index Limited (register number 114059) are authorised and regulated by the Financial Conduct Authority.