You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Vishal Santoshi <vi...@gmail.com> on 2019/10/15 20:00:52 UTC
Mirror Maker 2.0 cluster and starting from latest offset and other queries
2 queries
1. I am trying to configure MM2 to start replicating from the head (
latest of the topic ) . Should auto.offset.reset = latest in
mm2.properties be enough ? Unfortunately MM2 will start from the EARLIEST.
2. I do not have "Authorizer is configured on the broker " and see this
exception java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.SecurityDisabledException: No Authorizer is
configured on the broker From
https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-Config,ACLSync
*sync.topic.acls = false* should do it but does not..
Any ideas?
Re: Mirror Maker 2.0 cluster and starting from latest offset and
other queries
Posted by Piotr Nowojski <pi...@ververica.com>.
No problem, cheers :)
Piotrek
> On 17 Oct 2019, at 19:56, Vishal Santoshi <vi...@gmail.com> wrote:
>
> oh shit.. sorry wrong wrong forum :)
>
> On Thu, Oct 17, 2019 at 1:41 PM Piotr Nowojski <piotr@ververica.com <ma...@ververica.com>> wrote:
> Hi,
>
> It sounds like setting up Mirror Maker has very little to do with Flink. Shouldn’t you try asking this question on the Kafka mailing list?
>
> Piotrek
>
>> On 16 Oct 2019, at 16:06, Vishal Santoshi <vishal.santoshi@gmail.com <ma...@gmail.com>> wrote:
>>
>> 1. still no clue, apart from the fact that ConsumerConfig gets it from somewhere ( need to override it and have tried both auto.offset.reset =latest and consumer.auto.offset.reset = latest
>>
>> [2019-10-16 13:50:34,260] INFO ConsumerConfig values:
>> allow.auto.create.topics = true
>> auto.commit.interval.ms <http://auto.commit.interval.ms/> = 5000
>> auto.offset.reset = earliest
>>
>> 2. sync.topic.acls.enabled = false
>>
>>
>>
>>
>> On Tue, Oct 15, 2019 at 4:00 PM Vishal Santoshi <vishal.santoshi@gmail.com <ma...@gmail.com>> wrote:
>> 2 queries
>>
>> 1. I am trying to configure MM2 to start replicating from the head ( latest of the topic ) . Should auto.offset.reset = latest in mm2.properties be enough ? Unfortunately MM2 will start from the EARLIEST.
>>
>> 2. I do not have "Authorizer is configured on the broker " and see this exception java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.SecurityDisabledException: No Authorizer is configured on the broker From https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-Config,ACLSync <https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-Config,ACLSync>
>> sync.topic.acls = false should do it but does not..
>>
>>
>> Any ideas?
>>
>>
>
Re: Mirror Maker 2.0 cluster and starting from latest offset and
other queries
Posted by Vishal Santoshi <vi...@gmail.com>.
oh shit.. sorry wrong wrong forum :)
On Thu, Oct 17, 2019 at 1:41 PM Piotr Nowojski <pi...@ververica.com> wrote:
> Hi,
>
> It sounds like setting up Mirror Maker has very little to do with Flink.
> Shouldn’t you try asking this question on the Kafka mailing list?
>
> Piotrek
>
> On 16 Oct 2019, at 16:06, Vishal Santoshi <vi...@gmail.com>
> wrote:
>
> 1. still no clue, apart from the fact that ConsumerConfig gets it from
> somewhere ( need to override it and have tried both auto.offset.reset
> =latest and consumer.auto.offset.reset = latest
>
> [2019-10-16 13:50:34,260] INFO ConsumerConfig values:
> allow.auto.create.topics = true
> auto.commit.interval.ms = 5000
> auto.offset.reset = earliest
>
> 2. sync.topic.acls.enabled = false
>
>
>
>
> On Tue, Oct 15, 2019 at 4:00 PM Vishal Santoshi <vi...@gmail.com>
> wrote:
>
>> 2 queries
>>
>> 1. I am trying to configure MM2 to start replicating from the head (
>> latest of the topic ) . Should auto.offset.reset = latest in
>> mm2.properties be enough ? Unfortunately MM2 will start from the EARLIEST.
>>
>> 2. I do not have "Authorizer is configured on the broker " and see
>> this exception java.util.concurrent.ExecutionException:
>> org.apache.kafka.common.errors.SecurityDisabledException: No Authorizer is
>> configured on the broker From
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-Config,ACLSync
>> *sync.topic.acls = false* should do it but does not..
>>
>>
>> Any ideas?
>>
>>
>>
>
Re: Mirror Maker 2.0 cluster and starting from latest offset and
other queries
Posted by Piotr Nowojski <pi...@ververica.com>.
Hi,
It sounds like setting up Mirror Maker has very little to do with Flink. Shouldn’t you try asking this question on the Kafka mailing list?
Piotrek
> On 16 Oct 2019, at 16:06, Vishal Santoshi <vi...@gmail.com> wrote:
>
> 1. still no clue, apart from the fact that ConsumerConfig gets it from somewhere ( need to override it and have tried both auto.offset.reset =latest and consumer.auto.offset.reset = latest
>
> [2019-10-16 13:50:34,260] INFO ConsumerConfig values:
> allow.auto.create.topics = true
> auto.commit.interval.ms <http://auto.commit.interval.ms/> = 5000
> auto.offset.reset = earliest
>
> 2. sync.topic.acls.enabled = false
>
>
>
>
> On Tue, Oct 15, 2019 at 4:00 PM Vishal Santoshi <vishal.santoshi@gmail.com <ma...@gmail.com>> wrote:
> 2 queries
>
> 1. I am trying to configure MM2 to start replicating from the head ( latest of the topic ) . Should auto.offset.reset = latest in mm2.properties be enough ? Unfortunately MM2 will start from the EARLIEST.
>
> 2. I do not have "Authorizer is configured on the broker " and see this exception java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.SecurityDisabledException: No Authorizer is configured on the broker From https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-Config,ACLSync <https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-Config,ACLSync>
> sync.topic.acls = false should do it but does not..
>
>
> Any ideas?
>
>
Re: Mirror Maker 2.0 cluster and starting from latest offset and
other queries
Posted by Vishal Santoshi <vi...@gmail.com>.
1. still no clue, apart from the fact that ConsumerConfig gets it from
somewhere ( need to override it and have tried both auto.offset.reset
=latest and consumer.auto.offset.reset = latest
[2019-10-16 13:50:34,260] INFO ConsumerConfig values:
allow.auto.create.topics = true
auto.commit.interval.ms = 5000
auto.offset.reset = earliest
2. sync.topic.acls.enabled = false
On Tue, Oct 15, 2019 at 4:00 PM Vishal Santoshi <vi...@gmail.com>
wrote:
> 2 queries
>
> 1. I am trying to configure MM2 to start replicating from the head (
> latest of the topic ) . Should auto.offset.reset = latest in
> mm2.properties be enough ? Unfortunately MM2 will start from the EARLIEST.
>
> 2. I do not have "Authorizer is configured on the broker " and see this
> exception java.util.concurrent.ExecutionException:
> org.apache.kafka.common.errors.SecurityDisabledException: No Authorizer is
> configured on the broker From
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-Config,ACLSync
>
> *sync.topic.acls = false* should do it but does not..
>
>
>
> Any ideas?
>
>
>