You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by K Fred <yu...@gmail.com> on 2018/11/03 14:45:18 UTC
FlinkRunner JAAS verify failed in Flink cluster
Hi,
I am running into a very strange issue that
'Could not find a "KafkaClient" entry in the JAAS configuration. System
property "java.security.auth.login.config" is
/etc/kafka/kafka_sink_jaas.conf'
on a single node flink cluster when i consume record from kafka using beam
KafkaIO.
The JAAS file contains 'KafkaClient', but the flink cluster cannot find the
entry. Does anyone know the reason caused?
Thanks!
--
Fred
Re: FlinkRunner JAAS verify failed in Flink cluster
Posted by K Fred <yu...@gmail.com>.
Ok. l will post this solution to stackoverflow. The same question with
this on stackoverflow's link is below:
https://stackoverflow.com/q/53137481/7034070
On Wed, Nov 7, 2018, 18:09 Maximilian Michels <mx...@apache.org> wrote:
> Posting solutions to the mailing list is a great way to share knowledge
> because the mailing lists are archived and indexed by your favorite
> search engine.
>
> Replicating this to Stackoverflow could help to further increase
> visibility, as the site is generally more appealing to a wider range of
> people. So why not :)
>
> On 06.11.18 18:56, Austin Bennett wrote:
> > Related to another thread:
> >
> > Is there a value in posting issues that get put here (with follow up
> > solutions, like this thread, which indeed was excellent to have shared
> > the solution with the list) in Stack Overflow? Again, for ease of
> > discoverability, for those that face similar issues. Not sure how would
> > formalize, but bringing up nonetheless.
> >
> >
> >
> > On Tue, Nov 6, 2018 at 1:49 AM Maximilian Michels <mxm@apache.org
> > <ma...@apache.org>> wrote:
> >
> > Hi Fred,
> >
> > I see! Thanks for posting your solution here.
> >
> > Best,
> > Max
> >
> > On 06.11.18 03:49, K Fred wrote:
> > > Hi Max,
> > >
> > > I have resolved this issue. It's caused by the flink cluster
> > kerberos
> > > configuration. Just need to set some config on flink-conf.yaml
> > can make
> > > it work fine!
> > >
> > > The settings is below:
> > >
> > > security.kerberos.login.use-ticket-cache: false
> > > security.kerberos.login.keytab: /etc/kafka/kafka.keytab
> > > security.kerberos.login.principal: kafka@HADOOP.COM
> > <ma...@HADOOP.COM>
> > > <mailto:kafka@HADOOP.COM <ma...@HADOOP.COM>>
> > > security.kerberos.login.contexts: Client,KafkaClient
> > >
> > >
> > > Thanks,
> > > Fred.
> > >
> > > On Tue, Nov 6, 2018 at 2:56 AM Maximilian Michels <mxm@apache.org
> > <ma...@apache.org>
> > > <mailto:mxm@apache.org <ma...@apache.org>>> wrote:
> > >
> > > Hi Fred,
> > >
> > > Just to double check: Are you running this from a cluster or
> > your local
> > > machine? Asking because the stack trace indicates that the
> > exception
> > > occurs during job submission through the Flink command-line
> > client. So
> > > the machine you're running this on should also have the file
> > located in
> > > /etc.
> > >
> > > Thanks,
> > > Max
> > >
> > > On 05.11.18 12:26, K Fred wrote:
> > > > Hi Max,
> > > >
> > > > Yeah, The config is always located on the remote cluster.
> The
> > > exception
> > > > looks like that my application can find the config file,
> but
> > > cannot find
> > > > out the config's KafkaClient entry. So i guess the reason
> > may be
> > > related
> > > > to flink cluster some settings!
> > > >
> > > > /These code depict some stack trace below:/
> > > >
> > >
> >
> -----------------------------------------------------------------------------
> > > > The program finished with the following exception:
> > > >
> > > >
> > org.apache.flink.client.program.ProgramInvocationException: The main
> > > > method caused an error.
> > > > at
> > > >
> > >
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:545)
> > > > at
> > > >
> > >
> >
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420)
> > > > at
> > >
> >
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)
> > > > at
> > > >
> > >
> >
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:785)
> > > > at
> > >
> >
> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:279)
> > > > at
> > org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:214)
> > > > at
> > > >
> > >
> >
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1025)
> > > > at
> > > >
> > >
> >
> org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1101)
> > > > at
> > > >
> > >
> >
> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
> > > > at
> > >
> > org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1101)
> > > > Caused by: java.lang.RuntimeException: Error while
> translating
> > > > UnboundedSource:
> > > org.apache.beam.sdk.io.kafka.KafkaUnboundedSource@7bc6d27a
> > > > at
> > > >
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:225)
> > > > at
> > > >
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$ReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:273)
> > > > at
> > > >
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.applyStreamingTransform(FlinkStreamingPipelineTranslator.java:122)
> > > > at
> > > >
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.visitPrimitiveTransform(FlinkStreamingPipelineTranslator.java:101)
> > > > at
> > > >
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> > > > at
> > > >
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> > > > at
> > > >
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> > > > at
> > > >
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)
> > > > at
> > > >
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)
> > > > at
> > >
> >
> org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)
> > > > at
> > > >
> > >
> >
> org.apache.beam.runners.flink.FlinkPipelineTranslator.translate(FlinkPipelineTranslator.java:38)
> > > > at
> > > >
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.translate(FlinkStreamingPipelineTranslator.java:53)
> > > > at
> > > >
> > >
> >
> org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.translate(FlinkPipelineExecutionEnvironment.java:101)
> > > > at
> > >
> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:105)
> > > > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
> > > > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
> > > > at
> > >
> > ac.cn.iie.process.RelationProcess.process(RelationProcess.java:119)
> > > > at ac.cn.iie.Bootstrap.main(Bootstrap.java:16)
> > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > > > at
> > > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > > > at
> > > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > > at
> > > >
> > >
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528)
> > > > ... 9 more
> > > > Caused by: org.apache.kafka.common.KafkaException: Failed
> to
> > > construct
> > > > kafka consumer
> > > > at
> > > >
> > >
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:781)
> > > > at
> > > >
> > >
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:604)
> > > > at
> > > >
> > >
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:587)
> > > > at
> > > >
> > >
> >
> org.apache.beam.sdk.io.kafka.KafkaUnboundedSource.split(KafkaUnboundedSource.java:64)
> > > > at
> > > >
> > org.apache.beam.runners.flink.translation.wrappers.streaming.io
> > <
> http://org.apache.beam.runners.flink.translation.wrappers.streaming.io>
> > >
> > <http://streaming.io
> >.UnboundedSourceWrapper.<init>(UnboundedSourceWrapper.java:150)
> > > > at
> > > >
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:206)
> > > > ... 31 more
> > > > Caused by: java.lang.IllegalArgumentException: Could not
> > find a
> > > > 'KafkaClient' entry in the JAAS configuration. System
> property
> > > > 'java.security.auth.login.config' is
> > /etc/kafka/kafka_sink_jaas.conf
> > > > at
> > > >
> > >
> >
> org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:131)
> > > > at
> > >
> >
> org.apache.kafka.common.security.JaasContext.load(JaasContext.java:96)
> > > > at
> > >
> >
> org.apache.kafka.common.security.JaasContext.load(JaasContext.java:78)
> > > > at
> > > >
> > >
> >
> org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:103)
> > > > at
> > > >
> > >
> >
> org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:61)
> > > > at
> > > >
> > >
> >
> org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:86)
> > > > at
> > > >
> > >
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:702)
> > > > ... 36 more
> > > >
> > > > Thanks!
> > > >
> > > > On Mon, Nov 5, 2018 at 6:29 PM Maximilian Michels
> > <mxm@apache.org <ma...@apache.org>
> > > <mailto:mxm@apache.org <ma...@apache.org>>
> > > > <mailto:mxm@apache.org <ma...@apache.org>
> > <mailto:mxm@apache.org <ma...@apache.org>>>> wrote:
> > > >
> > > > Hi Fred,
> > > >
> > > > This is hard to debug without further information.
> Maybe a
> > > stack trace
> > > > would help. Are you sure the config is also located on
> > the remote
> > > > cluster?
> > > >
> > > > Thanks,
> > > > Max
> > > >
> > > > On 03.11.18 15:45, K Fred wrote:
> > > > > Hi,
> > > > >
> > > > > I am running into a very strange issue that
> > > > > 'Could not find a "KafkaClient" entry in the JAAS
> > > configuration.
> > > > System
> > > > > property "java.security.auth.login.config" is
> > > > > /etc/kafka/kafka_sink_jaas.conf'
> > > > > on a single node flink cluster when i consume
> > record from
> > > kafka
> > > > using
> > > > > beam KafkaIO.
> > > > >
> > > > > The JAAS file contains 'KafkaClient', but the flink
> > cluster
> > > > cannot find
> > > > > the entry. Does anyone know the reason caused?
> > > > >
> > > > > Thanks!
> > > > > --
> > > > >
> > > > > Fred
> > > > >
> > > >
> > > > --
> > > >
> > > > Fred
> > > >
> > >
> > > --
> > >
> > > Fred
> > >
> >
>
--
Fred
Re: FlinkRunner JAAS verify failed in Flink cluster
Posted by Maximilian Michels <mx...@apache.org>.
Posting solutions to the mailing list is a great way to share knowledge
because the mailing lists are archived and indexed by your favorite
search engine.
Replicating this to Stackoverflow could help to further increase
visibility, as the site is generally more appealing to a wider range of
people. So why not :)
On 06.11.18 18:56, Austin Bennett wrote:
> Related to another thread:
>
> Is there a value in posting issues that get put here (with follow up
> solutions, like this thread, which indeed was excellent to have shared
> the solution with the list) in Stack Overflow? Again, for ease of
> discoverability, for those that face similar issues. Not sure how would
> formalize, but bringing up nonetheless.
>
>
>
> On Tue, Nov 6, 2018 at 1:49 AM Maximilian Michels <mxm@apache.org
> <ma...@apache.org>> wrote:
>
> Hi Fred,
>
> I see! Thanks for posting your solution here.
>
> Best,
> Max
>
> On 06.11.18 03:49, K Fred wrote:
> > Hi Max,
> >
> > I have resolved this issue. It's caused by the flink cluster
> kerberos
> > configuration. Just need to set some config on flink-conf.yaml
> can make
> > it work fine!
> >
> > The settings is below:
> >
> > security.kerberos.login.use-ticket-cache: false
> > security.kerberos.login.keytab: /etc/kafka/kafka.keytab
> > security.kerberos.login.principal: kafka@HADOOP.COM
> <ma...@HADOOP.COM>
> > <mailto:kafka@HADOOP.COM <ma...@HADOOP.COM>>
> > security.kerberos.login.contexts: Client,KafkaClient
> >
> >
> > Thanks,
> > Fred.
> >
> > On Tue, Nov 6, 2018 at 2:56 AM Maximilian Michels <mxm@apache.org
> <ma...@apache.org>
> > <mailto:mxm@apache.org <ma...@apache.org>>> wrote:
> >
> > Hi Fred,
> >
> > Just to double check: Are you running this from a cluster or
> your local
> > machine? Asking because the stack trace indicates that the
> exception
> > occurs during job submission through the Flink command-line
> client. So
> > the machine you're running this on should also have the file
> located in
> > /etc.
> >
> > Thanks,
> > Max
> >
> > On 05.11.18 12:26, K Fred wrote:
> > > Hi Max,
> > >
> > > Yeah, The config is always located on the remote cluster. The
> > exception
> > > looks like that my application can find the config file, but
> > cannot find
> > > out the config's KafkaClient entry. So i guess the reason
> may be
> > related
> > > to flink cluster some settings!
> > >
> > > /These code depict some stack trace below:/
> > >
> >
> -----------------------------------------------------------------------------
> > > The program finished with the following exception:
> > >
> > >
> org.apache.flink.client.program.ProgramInvocationException: The main
> > > method caused an error.
> > > at
> > >
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:545)
> > > at
> > >
> >
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420)
> > > at
> >
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)
> > > at
> > >
> >
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:785)
> > > at
> >
> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:279)
> > > at
> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:214)
> > > at
> > >
> >
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1025)
> > > at
> > >
> >
> org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1101)
> > > at
> > >
> >
> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
> > > at
> >
> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1101)
> > > Caused by: java.lang.RuntimeException: Error while translating
> > > UnboundedSource:
> > org.apache.beam.sdk.io.kafka.KafkaUnboundedSource@7bc6d27a
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:225)
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$ReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:273)
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.applyStreamingTransform(FlinkStreamingPipelineTranslator.java:122)
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.visitPrimitiveTransform(FlinkStreamingPipelineTranslator.java:101)
> > > at
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> > > at
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> > > at
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> > > at
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)
> > > at
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)
> > > at
> >
> org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkPipelineTranslator.translate(FlinkPipelineTranslator.java:38)
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.translate(FlinkStreamingPipelineTranslator.java:53)
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.translate(FlinkPipelineExecutionEnvironment.java:101)
> > > at
> >
> org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:105)
> > > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
> > > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
> > > at
> >
> ac.cn.iie.process.RelationProcess.process(RelationProcess.java:119)
> > > at ac.cn.iie.Bootstrap.main(Bootstrap.java:16)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > > at
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > at
> > >
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528)
> > > ... 9 more
> > > Caused by: org.apache.kafka.common.KafkaException: Failed to
> > construct
> > > kafka consumer
> > > at
> > >
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:781)
> > > at
> > >
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:604)
> > > at
> > >
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:587)
> > > at
> > >
> >
> org.apache.beam.sdk.io.kafka.KafkaUnboundedSource.split(KafkaUnboundedSource.java:64)
> > > at
> > >
> org.apache.beam.runners.flink.translation.wrappers.streaming.io
> <http://org.apache.beam.runners.flink.translation.wrappers.streaming.io>
> >
> <http://streaming.io>.UnboundedSourceWrapper.<init>(UnboundedSourceWrapper.java:150)
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:206)
> > > ... 31 more
> > > Caused by: java.lang.IllegalArgumentException: Could not
> find a
> > > 'KafkaClient' entry in the JAAS configuration. System property
> > > 'java.security.auth.login.config' is
> /etc/kafka/kafka_sink_jaas.conf
> > > at
> > >
> >
> org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:131)
> > > at
> >
> org.apache.kafka.common.security.JaasContext.load(JaasContext.java:96)
> > > at
> >
> org.apache.kafka.common.security.JaasContext.load(JaasContext.java:78)
> > > at
> > >
> >
> org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:103)
> > > at
> > >
> >
> org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:61)
> > > at
> > >
> >
> org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:86)
> > > at
> > >
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:702)
> > > ... 36 more
> > >
> > > Thanks!
> > >
> > > On Mon, Nov 5, 2018 at 6:29 PM Maximilian Michels
> <mxm@apache.org <ma...@apache.org>
> > <mailto:mxm@apache.org <ma...@apache.org>>
> > > <mailto:mxm@apache.org <ma...@apache.org>
> <mailto:mxm@apache.org <ma...@apache.org>>>> wrote:
> > >
> > > Hi Fred,
> > >
> > > This is hard to debug without further information. Maybe a
> > stack trace
> > > would help. Are you sure the config is also located on
> the remote
> > > cluster?
> > >
> > > Thanks,
> > > Max
> > >
> > > On 03.11.18 15:45, K Fred wrote:
> > > > Hi,
> > > >
> > > > I am running into a very strange issue that
> > > > 'Could not find a "KafkaClient" entry in the JAAS
> > configuration.
> > > System
> > > > property "java.security.auth.login.config" is
> > > > /etc/kafka/kafka_sink_jaas.conf'
> > > > on a single node flink cluster when i consume
> record from
> > kafka
> > > using
> > > > beam KafkaIO.
> > > >
> > > > The JAAS file contains 'KafkaClient', but the flink
> cluster
> > > cannot find
> > > > the entry. Does anyone know the reason caused?
> > > >
> > > > Thanks!
> > > > --
> > > >
> > > > Fred
> > > >
> > >
> > > --
> > >
> > > Fred
> > >
> >
> > --
> >
> > Fred
> >
>
Re: FlinkRunner JAAS verify failed in Flink cluster
Posted by Austin Bennett <wh...@gmail.com>.
Related to another thread:
Is there a value in posting issues that get put here (with follow up
solutions, like this thread, which indeed was excellent to have shared the
solution with the list) in Stack Overflow? Again, for ease of
discoverability, for those that face similar issues. Not sure how would
formalize, but bringing up nonetheless.
On Tue, Nov 6, 2018 at 1:49 AM Maximilian Michels <mx...@apache.org> wrote:
> Hi Fred,
>
> I see! Thanks for posting your solution here.
>
> Best,
> Max
>
> On 06.11.18 03:49, K Fred wrote:
> > Hi Max,
> >
> > I have resolved this issue. It's caused by the flink cluster kerberos
> > configuration. Just need to set some config on flink-conf.yaml can make
> > it work fine!
> >
> > The settings is below:
> >
> > security.kerberos.login.use-ticket-cache: false
> > security.kerberos.login.keytab: /etc/kafka/kafka.keytab
> > security.kerberos.login.principal: kafka@HADOOP.COM
> > <ma...@HADOOP.COM>
> > security.kerberos.login.contexts: Client,KafkaClient
> >
> >
> > Thanks,
> > Fred.
> >
> > On Tue, Nov 6, 2018 at 2:56 AM Maximilian Michels <mxm@apache.org
> > <ma...@apache.org>> wrote:
> >
> > Hi Fred,
> >
> > Just to double check: Are you running this from a cluster or your
> local
> > machine? Asking because the stack trace indicates that the exception
> > occurs during job submission through the Flink command-line client.
> So
> > the machine you're running this on should also have the file located
> in
> > /etc.
> >
> > Thanks,
> > Max
> >
> > On 05.11.18 12:26, K Fred wrote:
> > > Hi Max,
> > >
> > > Yeah, The config is always located on the remote cluster. The
> > exception
> > > looks like that my application can find the config file, but
> > cannot find
> > > out the config's KafkaClient entry. So i guess the reason may be
> > related
> > > to flink cluster some settings!
> > >
> > > /These code depict some stack trace below:/
> > >
> >
> -----------------------------------------------------------------------------
> > > The program finished with the following exception:
> > >
> > > org.apache.flink.client.program.ProgramInvocationException: The
> main
> > > method caused an error.
> > > at
> > >
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:545)
> > > at
> > >
> >
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420)
> > > at
> >
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)
> > > at
> > >
> >
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:785)
> > > at
> >
> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:279)
> > > at
> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:214)
> > > at
> > >
> >
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1025)
> > > at
> > >
> >
> org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1101)
> > > at
> > >
> >
> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
> > > at
> > org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1101)
> > > Caused by: java.lang.RuntimeException: Error while translating
> > > UnboundedSource:
> > org.apache.beam.sdk.io.kafka.KafkaUnboundedSource@7bc6d27a
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:225)
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$ReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:273)
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.applyStreamingTransform(FlinkStreamingPipelineTranslator.java:122)
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.visitPrimitiveTransform(FlinkStreamingPipelineTranslator.java:101)
> > > at
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> > > at
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> > > at
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> > > at
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)
> > > at
> > >
> >
> org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)
> > > at
> > org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkPipelineTranslator.translate(FlinkPipelineTranslator.java:38)
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.translate(FlinkStreamingPipelineTranslator.java:53)
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.translate(FlinkPipelineExecutionEnvironment.java:101)
> > > at
> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:105)
> > > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
> > > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
> > > at
> > ac.cn.iie.process.RelationProcess.process(RelationProcess.java:119)
> > > at ac.cn.iie.Bootstrap.main(Bootstrap.java:16)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > > at
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > at
> > >
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528)
> > > ... 9 more
> > > Caused by: org.apache.kafka.common.KafkaException: Failed to
> > construct
> > > kafka consumer
> > > at
> > >
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:781)
> > > at
> > >
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:604)
> > > at
> > >
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:587)
> > > at
> > >
> >
> org.apache.beam.sdk.io.kafka.KafkaUnboundedSource.split(KafkaUnboundedSource.java:64)
> > > at
> > > org.apache.beam.runners.flink.translation.wrappers.streaming.io
> > <http://streaming.io
> >.UnboundedSourceWrapper.<init>(UnboundedSourceWrapper.java:150)
> > > at
> > >
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:206)
> > > ... 31 more
> > > Caused by: java.lang.IllegalArgumentException: Could not find a
> > > 'KafkaClient' entry in the JAAS configuration. System property
> > > 'java.security.auth.login.config' is
> /etc/kafka/kafka_sink_jaas.conf
> > > at
> > >
> >
> org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:131)
> > > at
> >
> org.apache.kafka.common.security.JaasContext.load(JaasContext.java:96)
> > > at
> >
> org.apache.kafka.common.security.JaasContext.load(JaasContext.java:78)
> > > at
> > >
> >
> org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:103)
> > > at
> > >
> >
> org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:61)
> > > at
> > >
> >
> org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:86)
> > > at
> > >
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:702)
> > > ... 36 more
> > >
> > > Thanks!
> > >
> > > On Mon, Nov 5, 2018 at 6:29 PM Maximilian Michels <mxm@apache.org
> > <ma...@apache.org>
> > > <mailto:mxm@apache.org <ma...@apache.org>>> wrote:
> > >
> > > Hi Fred,
> > >
> > > This is hard to debug without further information. Maybe a
> > stack trace
> > > would help. Are you sure the config is also located on the
> remote
> > > cluster?
> > >
> > > Thanks,
> > > Max
> > >
> > > On 03.11.18 15:45, K Fred wrote:
> > > > Hi,
> > > >
> > > > I am running into a very strange issue that
> > > > 'Could not find a "KafkaClient" entry in the JAAS
> > configuration.
> > > System
> > > > property "java.security.auth.login.config" is
> > > > /etc/kafka/kafka_sink_jaas.conf'
> > > > on a single node flink cluster when i consume record from
> > kafka
> > > using
> > > > beam KafkaIO.
> > > >
> > > > The JAAS file contains 'KafkaClient', but the flink cluster
> > > cannot find
> > > > the entry. Does anyone know the reason caused?
> > > >
> > > > Thanks!
> > > > --
> > > >
> > > > Fred
> > > >
> > >
> > > --
> > >
> > > Fred
> > >
> >
> > --
> >
> > Fred
> >
>
Re: FlinkRunner JAAS verify failed in Flink cluster
Posted by Maximilian Michels <mx...@apache.org>.
Hi Fred,
I see! Thanks for posting your solution here.
Best,
Max
On 06.11.18 03:49, K Fred wrote:
> Hi Max,
>
> I have resolved this issue. It's caused by the flink cluster kerberos
> configuration. Just need to set some config on flink-conf.yaml can make
> it work fine!
>
> The settings is below:
>
> security.kerberos.login.use-ticket-cache: false
> security.kerberos.login.keytab: /etc/kafka/kafka.keytab
> security.kerberos.login.principal: kafka@HADOOP.COM
> <ma...@HADOOP.COM>
> security.kerberos.login.contexts: Client,KafkaClient
>
>
> Thanks,
> Fred.
>
> On Tue, Nov 6, 2018 at 2:56 AM Maximilian Michels <mxm@apache.org
> <ma...@apache.org>> wrote:
>
> Hi Fred,
>
> Just to double check: Are you running this from a cluster or your local
> machine? Asking because the stack trace indicates that the exception
> occurs during job submission through the Flink command-line client. So
> the machine you're running this on should also have the file located in
> /etc.
>
> Thanks,
> Max
>
> On 05.11.18 12:26, K Fred wrote:
> > Hi Max,
> >
> > Yeah, The config is always located on the remote cluster. The
> exception
> > looks like that my application can find the config file, but
> cannot find
> > out the config's KafkaClient entry. So i guess the reason may be
> related
> > to flink cluster some settings!
> >
> > /These code depict some stack trace below:/
> >
> -----------------------------------------------------------------------------
> > The program finished with the following exception:
> >
> > org.apache.flink.client.program.ProgramInvocationException: The main
> > method caused an error.
> > at
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:545)
> > at
> >
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420)
> > at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)
> > at
> >
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:785)
> > at
> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:279)
> > at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:214)
> > at
> >
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1025)
> > at
> >
> org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1101)
> > at
> >
> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
> > at
> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1101)
> > Caused by: java.lang.RuntimeException: Error while translating
> > UnboundedSource:
> org.apache.beam.sdk.io.kafka.KafkaUnboundedSource@7bc6d27a
> > at
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:225)
> > at
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$ReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:273)
> > at
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.applyStreamingTransform(FlinkStreamingPipelineTranslator.java:122)
> > at
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.visitPrimitiveTransform(FlinkStreamingPipelineTranslator.java:101)
> > at
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> > at
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> > at
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> > at
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)
> > at
> >
> org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)
> > at
> org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)
> > at
> >
> org.apache.beam.runners.flink.FlinkPipelineTranslator.translate(FlinkPipelineTranslator.java:38)
> > at
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.translate(FlinkStreamingPipelineTranslator.java:53)
> > at
> >
> org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.translate(FlinkPipelineExecutionEnvironment.java:101)
> > at
> org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:105)
> > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
> > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
> > at
> ac.cn.iie.process.RelationProcess.process(RelationProcess.java:119)
> > at ac.cn.iie.Bootstrap.main(Bootstrap.java:16)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528)
> > ... 9 more
> > Caused by: org.apache.kafka.common.KafkaException: Failed to
> construct
> > kafka consumer
> > at
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:781)
> > at
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:604)
> > at
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:587)
> > at
> >
> org.apache.beam.sdk.io.kafka.KafkaUnboundedSource.split(KafkaUnboundedSource.java:64)
> > at
> > org.apache.beam.runners.flink.translation.wrappers.streaming.io
> <http://streaming.io>.UnboundedSourceWrapper.<init>(UnboundedSourceWrapper.java:150)
> > at
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:206)
> > ... 31 more
> > Caused by: java.lang.IllegalArgumentException: Could not find a
> > 'KafkaClient' entry in the JAAS configuration. System property
> > 'java.security.auth.login.config' is /etc/kafka/kafka_sink_jaas.conf
> > at
> >
> org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:131)
> > at
> org.apache.kafka.common.security.JaasContext.load(JaasContext.java:96)
> > at
> org.apache.kafka.common.security.JaasContext.load(JaasContext.java:78)
> > at
> >
> org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:103)
> > at
> >
> org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:61)
> > at
> >
> org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:86)
> > at
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:702)
> > ... 36 more
> >
> > Thanks!
> >
> > On Mon, Nov 5, 2018 at 6:29 PM Maximilian Michels <mxm@apache.org
> <ma...@apache.org>
> > <mailto:mxm@apache.org <ma...@apache.org>>> wrote:
> >
> > Hi Fred,
> >
> > This is hard to debug without further information. Maybe a
> stack trace
> > would help. Are you sure the config is also located on the remote
> > cluster?
> >
> > Thanks,
> > Max
> >
> > On 03.11.18 15:45, K Fred wrote:
> > > Hi,
> > >
> > > I am running into a very strange issue that
> > > 'Could not find a "KafkaClient" entry in the JAAS
> configuration.
> > System
> > > property "java.security.auth.login.config" is
> > > /etc/kafka/kafka_sink_jaas.conf'
> > > on a single node flink cluster when i consume record from
> kafka
> > using
> > > beam KafkaIO.
> > >
> > > The JAAS file contains 'KafkaClient', but the flink cluster
> > cannot find
> > > the entry. Does anyone know the reason caused?
> > >
> > > Thanks!
> > > --
> > >
> > > Fred
> > >
> >
> > --
> >
> > Fred
> >
>
> --
>
> Fred
>
Re: FlinkRunner JAAS verify failed in Flink cluster
Posted by K Fred <yu...@gmail.com>.
Hi Max,
I have resolved this issue. It's caused by the flink cluster kerberos
configuration. Just need to set some config on flink-conf.yaml can make it
work fine!
The settings is below:
security.kerberos.login.use-ticket-cache: false
security.kerberos.login.keytab: /etc/kafka/kafka.keytab
security.kerberos.login.principal: kafka@HADOOP.COM
security.kerberos.login.contexts: Client,KafkaClient
Thanks,
Fred.
On Tue, Nov 6, 2018 at 2:56 AM Maximilian Michels <mx...@apache.org> wrote:
> Hi Fred,
>
> Just to double check: Are you running this from a cluster or your local
> machine? Asking because the stack trace indicates that the exception
> occurs during job submission through the Flink command-line client. So
> the machine you're running this on should also have the file located in
> /etc.
>
> Thanks,
> Max
>
> On 05.11.18 12:26, K Fred wrote:
> > Hi Max,
> >
> > Yeah, The config is always located on the remote cluster. The exception
> > looks like that my application can find the config file, but cannot find
> > out the config's KafkaClient entry. So i guess the reason may be related
> > to flink cluster some settings!
> >
> > /These code depict some stack trace below:/
> >
> -----------------------------------------------------------------------------
> > The program finished with the following exception:
> >
> > org.apache.flink.client.program.ProgramInvocationException: The main
> > method caused an error.
> > at
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:545)
> > at
> >
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420)
> > at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)
> > at
> >
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:785)
> > at
> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:279)
> > at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:214)
> > at
> >
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1025)
> > at
> >
> org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1101)
> > at
> >
> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
> > at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1101)
> > Caused by: java.lang.RuntimeException: Error while translating
> > UnboundedSource:
> org.apache.beam.sdk.io.kafka.KafkaUnboundedSource@7bc6d27a
> > at
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:225)
> > at
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$ReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:273)
> > at
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.applyStreamingTransform(FlinkStreamingPipelineTranslator.java:122)
> > at
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.visitPrimitiveTransform(FlinkStreamingPipelineTranslator.java:101)
> > at
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> > at
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> > at
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> > at
> >
> org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)
> > at
> >
> org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)
> > at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)
> > at
> >
> org.apache.beam.runners.flink.FlinkPipelineTranslator.translate(FlinkPipelineTranslator.java:38)
> > at
> >
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.translate(FlinkStreamingPipelineTranslator.java:53)
> > at
> >
> org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.translate(FlinkPipelineExecutionEnvironment.java:101)
> > at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:105)
> > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
> > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
> > at ac.cn.iie.process.RelationProcess.process(RelationProcess.java:119)
> > at ac.cn.iie.Bootstrap.main(Bootstrap.java:16)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528)
> > ... 9 more
> > Caused by: org.apache.kafka.common.KafkaException: Failed to construct
> > kafka consumer
> > at
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:781)
> > at
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:604)
> > at
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:587)
> > at
> >
> org.apache.beam.sdk.io.kafka.KafkaUnboundedSource.split(KafkaUnboundedSource.java:64)
> > at
> > org.apache.beam.runners.flink.translation.wrappers.streaming.io
> .UnboundedSourceWrapper.<init>(UnboundedSourceWrapper.java:150)
> > at
> >
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:206)
> > ... 31 more
> > Caused by: java.lang.IllegalArgumentException: Could not find a
> > 'KafkaClient' entry in the JAAS configuration. System property
> > 'java.security.auth.login.config' is /etc/kafka/kafka_sink_jaas.conf
> > at
> >
> org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:131)
> > at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:96)
> > at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:78)
> > at
> >
> org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:103)
> > at
> >
> org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:61)
> > at
> >
> org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:86)
> > at
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:702)
> > ... 36 more
> >
> > Thanks!
> >
> > On Mon, Nov 5, 2018 at 6:29 PM Maximilian Michels <mxm@apache.org
> > <ma...@apache.org>> wrote:
> >
> > Hi Fred,
> >
> > This is hard to debug without further information. Maybe a stack
> trace
> > would help. Are you sure the config is also located on the remote
> > cluster?
> >
> > Thanks,
> > Max
> >
> > On 03.11.18 15:45, K Fred wrote:
> > > Hi,
> > >
> > > I am running into a very strange issue that
> > > 'Could not find a "KafkaClient" entry in the JAAS configuration.
> > System
> > > property "java.security.auth.login.config" is
> > > /etc/kafka/kafka_sink_jaas.conf'
> > > on a single node flink cluster when i consume record from kafka
> > using
> > > beam KafkaIO.
> > >
> > > The JAAS file contains 'KafkaClient', but the flink cluster
> > cannot find
> > > the entry. Does anyone know the reason caused?
> > >
> > > Thanks!
> > > --
> > >
> > > Fred
> > >
> >
> > --
> >
> > Fred
> >
>
--
Fred
Re: FlinkRunner JAAS verify failed in Flink cluster
Posted by Maximilian Michels <mx...@apache.org>.
Hi Fred,
Just to double check: Are you running this from a cluster or your local
machine? Asking because the stack trace indicates that the exception
occurs during job submission through the Flink command-line client. So
the machine you're running this on should also have the file located in
/etc.
Thanks,
Max
On 05.11.18 12:26, K Fred wrote:
> Hi Max,
>
> Yeah, The config is always located on the remote cluster. The exception
> looks like that my application can find the config file, but cannot find
> out the config's KafkaClient entry. So i guess the reason may be related
> to flink cluster some settings!
>
> /These code depict some stack trace below:/
> -----------------------------------------------------------------------------
> The program finished with the following exception:
>
> org.apache.flink.client.program.ProgramInvocationException: The main
> method caused an error.
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:545)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420)
> at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:785)
> at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:279)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:214)
> at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1025)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1101)
> at
> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1101)
> Caused by: java.lang.RuntimeException: Error while translating
> UnboundedSource: org.apache.beam.sdk.io.kafka.KafkaUnboundedSource@7bc6d27a
> at
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:225)
> at
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$ReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:273)
> at
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.applyStreamingTransform(FlinkStreamingPipelineTranslator.java:122)
> at
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.visitPrimitiveTransform(FlinkStreamingPipelineTranslator.java:101)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)
> at
> org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)
> at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)
> at
> org.apache.beam.runners.flink.FlinkPipelineTranslator.translate(FlinkPipelineTranslator.java:38)
> at
> org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.translate(FlinkStreamingPipelineTranslator.java:53)
> at
> org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.translate(FlinkPipelineExecutionEnvironment.java:101)
> at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:105)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
> at ac.cn.iie.process.RelationProcess.process(RelationProcess.java:119)
> at ac.cn.iie.Bootstrap.main(Bootstrap.java:16)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528)
> ... 9 more
> Caused by: org.apache.kafka.common.KafkaException: Failed to construct
> kafka consumer
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:781)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:604)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:587)
> at
> org.apache.beam.sdk.io.kafka.KafkaUnboundedSource.split(KafkaUnboundedSource.java:64)
> at
> org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapper.<init>(UnboundedSourceWrapper.java:150)
> at
> org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:206)
> ... 31 more
> Caused by: java.lang.IllegalArgumentException: Could not find a
> 'KafkaClient' entry in the JAAS configuration. System property
> 'java.security.auth.login.config' is /etc/kafka/kafka_sink_jaas.conf
> at
> org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:131)
> at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:96)
> at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:78)
> at
> org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:103)
> at
> org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:61)
> at
> org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:86)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:702)
> ... 36 more
>
> Thanks!
>
> On Mon, Nov 5, 2018 at 6:29 PM Maximilian Michels <mxm@apache.org
> <ma...@apache.org>> wrote:
>
> Hi Fred,
>
> This is hard to debug without further information. Maybe a stack trace
> would help. Are you sure the config is also located on the remote
> cluster?
>
> Thanks,
> Max
>
> On 03.11.18 15:45, K Fred wrote:
> > Hi,
> >
> > I am running into a very strange issue that
> > 'Could not find a "KafkaClient" entry in the JAAS configuration.
> System
> > property "java.security.auth.login.config" is
> > /etc/kafka/kafka_sink_jaas.conf'
> > on a single node flink cluster when i consume record from kafka
> using
> > beam KafkaIO.
> >
> > The JAAS file contains 'KafkaClient', but the flink cluster
> cannot find
> > the entry. Does anyone know the reason caused?
> >
> > Thanks!
> > --
> >
> > Fred
> >
>
> --
>
> Fred
>
Re: FlinkRunner JAAS verify failed in Flink cluster
Posted by K Fred <yu...@gmail.com>.
Hi Max,
Yeah, The config is always located on the remote cluster. The exception
looks like that my application can find the config file, but cannot find
out the config's KafkaClient entry. So i guess the reason may be related to
flink cluster some settings!
*These code depict some stack trace below:*
-----------------------------------------------------------------------------
The program finished with the following exception:
org.apache.flink.client.program.ProgramInvocationException: The main method
caused an error.
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:545)
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)
at
org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:785)
at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:279)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:214)
at
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1025)
at
org.apache.flink.client.cli.CliFrontend.lambda$main$9(CliFrontend.java:1101)
at
org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1101)
Caused by: java.lang.RuntimeException: Error while translating
UnboundedSource: org.apache.beam.sdk.io.kafka.KafkaUnboundedSource@7bc6d27a
at
org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:225)
at
org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$ReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:273)
at
org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.applyStreamingTransform(FlinkStreamingPipelineTranslator.java:122)
at
org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.visitPrimitiveTransform(FlinkStreamingPipelineTranslator.java:101)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)
at
org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)
at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)
at
org.apache.beam.runners.flink.FlinkPipelineTranslator.translate(FlinkPipelineTranslator.java:38)
at
org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.translate(FlinkStreamingPipelineTranslator.java:53)
at
org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.translate(FlinkPipelineExecutionEnvironment.java:101)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:105)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
at ac.cn.iie.process.RelationProcess.process(RelationProcess.java:119)
at ac.cn.iie.Bootstrap.main(Bootstrap.java:16)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528)
... 9 more
Caused by: org.apache.kafka.common.KafkaException: Failed to construct
kafka consumer
at
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:781)
at
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:604)
at
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:587)
at
org.apache.beam.sdk.io.kafka.KafkaUnboundedSource.split(KafkaUnboundedSource.java:64)
at
org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapper.<init>(UnboundedSourceWrapper.java:150)
at
org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:206)
... 31 more
Caused by: java.lang.IllegalArgumentException: Could not find a
'KafkaClient' entry in the JAAS configuration. System property
'java.security.auth.login.config' is /etc/kafka/kafka_sink_jaas.conf
at
org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:131)
at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:96)
at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:78)
at
org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:103)
at
org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:61)
at
org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:86)
at
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:702)
... 36 more
Thanks!
On Mon, Nov 5, 2018 at 6:29 PM Maximilian Michels <mx...@apache.org> wrote:
> Hi Fred,
>
> This is hard to debug without further information. Maybe a stack trace
> would help. Are you sure the config is also located on the remote cluster?
>
> Thanks,
> Max
>
> On 03.11.18 15:45, K Fred wrote:
> > Hi,
> >
> > I am running into a very strange issue that
> > 'Could not find a "KafkaClient" entry in the JAAS configuration. System
> > property "java.security.auth.login.config" is
> > /etc/kafka/kafka_sink_jaas.conf'
> > on a single node flink cluster when i consume record from kafka using
> > beam KafkaIO.
> >
> > The JAAS file contains 'KafkaClient', but the flink cluster cannot find
> > the entry. Does anyone know the reason caused?
> >
> > Thanks!
> > --
> >
> > Fred
> >
>
--
Fred
Re: FlinkRunner JAAS verify failed in Flink cluster
Posted by Maximilian Michels <mx...@apache.org>.
Hi Fred,
This is hard to debug without further information. Maybe a stack trace
would help. Are you sure the config is also located on the remote cluster?
Thanks,
Max
On 03.11.18 15:45, K Fred wrote:
> Hi,
>
> I am running into a very strange issue that
> 'Could not find a "KafkaClient" entry in the JAAS configuration. System
> property "java.security.auth.login.config" is
> /etc/kafka/kafka_sink_jaas.conf'
> on a single node flink cluster when i consume record from kafka using
> beam KafkaIO.
>
> The JAAS file contains 'KafkaClient', but the flink cluster cannot find
> the entry. Does anyone know the reason caused?
>
> Thanks!
> --
>
> Fred
>