You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flume.apache.org by "venkat Bandakadi (JIRA)" <ji...@apache.org> on 2018/03/05 21:42:00 UTC
[jira] [Commented] (FLUME-3220) Kite data set sink polling for
connection very frequently even if I dont have event
[ https://issues.apache.org/jira/browse/FLUME-3220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386807#comment-16386807 ]
venkat Bandakadi commented on FLUME-3220:
-----------------------------------------
My sample kite sink
dataIngestPDANotification.sinks.pda_notification_sink.type = org.apache.flume.sink.kite.DatasetSink
dataIngestPDANotification.sinks.pda_notification_sink.channel = mc_pda
dataIngestPDANotification.sinks.pda_notification_sink.kite.dataset.uri = dataset:hive:db/table
dataIngestPDANotification.sinks.pda_notification_sink.kite.batchSize= 1000
dataIngestPDANotification.sinks.pda_notification_sink.kite.rollInterval= 0
dataIngestPDANotification.sinks.pda_notification_sink.kite.flushable.commitOnBatch = true
dataIngestPDANotification.sinks.pda_notification_sink.kite.syncable.syncOnBatch = true
dataIngestPDANotification.sinks.pda_notification_sink.kite.entityParser = avro
dataIngestPDANotification.sinks.pda_notification_sink.auth.kerberosPrincipal = [id@Domain.com|mailto:id@Domain.com]
dataIngestPDANotification.sinks.pda_notification_sink.auth.kerberosKeytab = keytab
> Kite data set sink polling for connection very frequently even if I dont have event
> -----------------------------------------------------------------------------------
>
> Key: FLUME-3220
> URL: https://issues.apache.org/jira/browse/FLUME-3220
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: 1.8.0
> Reporter: venkat Bandakadi
> Priority: Major
> Labels: performance
>
> I see a lot of connection in a second even if don't see the event in my channel , due to this issue my hive meta store went down. , when i stopped service it come up
> [ingRunner-DefaultSinkProcessor] metastore INFO Trying to connect to metastore with URI thrift://xxxxx:9083
> [ingRunner-DefaultSinkProcessor] metastore INFO Opened a connection to metastore, current connections: 33028396
> [ingRunner-DefaultSinkProcessor] metastore INFO Connected to metastore.
> [ingRunner-DefaultSinkProcessor] metastore INFO Trying to connect to metastore with URI thrift://xxxxxx:9083
> [ingRunner-DefaultSinkProcessor] metastore INFO Opened a connection to metastore, current connections: 33028397
> [ingRunner-DefaultSinkProcessor] metastore INFO Connected to metastore.
> [ingRunner-DefaultSinkProcessor] metastore INFO Trying to connect to metastore with URI thrift://xxxxxxx:9083
> *****
> [ingRunner-DefaultSinkProcessor] metastore INFO Opened a connection to metastore, current connections: 33028402
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@flume.apache.org
For additional commands, e-mail: issues-help@flume.apache.org