You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Jay Kreps (JIRA)" <ji...@apache.org> on 2012/11/13 19:00:12 UTC
[jira] [Created] (KAFKA-608) getTopicMetadata does not respect
producer config settings
Jay Kreps created KAFKA-608:
-------------------------------
Summary: getTopicMetadata does not respect producer config settings
Key: KAFKA-608
URL: https://issues.apache.org/jira/browse/KAFKA-608
Project: Kafka
Issue Type: Bug
Reporter: Jay Kreps
ProducerPool.scala contains the following code:
object ProducerPool{
def createSyncProducer(configOpt: Option[ProducerConfig], broker: Broker): SyncProducer = {
val props = new Properties()
props.put("host", broker.host)
props.put("port", broker.port.toString)
if(configOpt.isDefined)
props.putAll(configOpt.get.props.props)
new SyncProducer(new SyncProducerConfig(props))
}
}
Note also, that ClientUtils.getTopicMetadata() does the following:
ProducerPool.createSyncProducer(None, brokers(i))
As a result there is no way to control the socket settings for the get metadata request.
My recommendation is that we require the config to be specified in the
Note that this creates a new sync producer without using ANY of the settings the user had given for the producer. In particular the socket timeout is defaulted to 500ms.
This causes unit tests to fail a lot since a newly started test may easily timeout on a 500ms request.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-608) getTopicMetadata does not respect
producer config settings
Posted by "Neha Narkhede (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neha Narkhede updated KAFKA-608:
--------------------------------
Attachment: kafka-608-v1.patch
Attaching a patch that makes the following changes -
1. Fixes the metadata api calls to use the right producer config
2. Fixes the auto create topic issue to ignore the exception that complains that the topic was already created. This is fine since many brokers might be creating the topic at the same time.
3. Bumped up the ack timeout on the producer to 1500 as default
> getTopicMetadata does not respect producer config settings
> ----------------------------------------------------------
>
> Key: KAFKA-608
> URL: https://issues.apache.org/jira/browse/KAFKA-608
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Jay Kreps
> Assignee: Neha Narkhede
> Priority: Blocker
> Fix For: 0.8
>
> Attachments: kafka-608-v1.patch
>
>
> ProducerPool.scala contains the following code:
> object ProducerPool{
> def createSyncProducer(configOpt: Option[ProducerConfig], broker: Broker): SyncProducer = {
> val props = new Properties()
> props.put("host", broker.host)
> props.put("port", broker.port.toString)
> if(configOpt.isDefined)
> props.putAll(configOpt.get.props.props)
> new SyncProducer(new SyncProducerConfig(props))
> }
> }
> Note also, that ClientUtils.getTopicMetadata() does the following:
> ProducerPool.createSyncProducer(None, brokers(i))
> As a result there is no way to control the socket settings for the get metadata request.
> My recommendation is that we require the config to be specified in the
> Note that this creates a new sync producer without using ANY of the settings the user had given for the producer. In particular the socket timeout is defaulted to 500ms.
> This causes unit tests to fail a lot since a newly started test may easily timeout on a 500ms request.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (KAFKA-608) getTopicMetadata does not respect
producer config settings
Posted by "Neha Narkhede (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neha Narkhede reassigned KAFKA-608:
-----------------------------------
Assignee: Neha Narkhede
> getTopicMetadata does not respect producer config settings
> ----------------------------------------------------------
>
> Key: KAFKA-608
> URL: https://issues.apache.org/jira/browse/KAFKA-608
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Jay Kreps
> Assignee: Neha Narkhede
> Priority: Blocker
> Fix For: 0.8
>
>
> ProducerPool.scala contains the following code:
> object ProducerPool{
> def createSyncProducer(configOpt: Option[ProducerConfig], broker: Broker): SyncProducer = {
> val props = new Properties()
> props.put("host", broker.host)
> props.put("port", broker.port.toString)
> if(configOpt.isDefined)
> props.putAll(configOpt.get.props.props)
> new SyncProducer(new SyncProducerConfig(props))
> }
> }
> Note also, that ClientUtils.getTopicMetadata() does the following:
> ProducerPool.createSyncProducer(None, brokers(i))
> As a result there is no way to control the socket settings for the get metadata request.
> My recommendation is that we require the config to be specified in the
> Note that this creates a new sync producer without using ANY of the settings the user had given for the producer. In particular the socket timeout is defaulted to 500ms.
> This causes unit tests to fail a lot since a newly started test may easily timeout on a 500ms request.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-608) getTopicMetadata does not respect
producer config settings
Posted by "Neha Narkhede (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neha Narkhede updated KAFKA-608:
--------------------------------
Status: Patch Available (was: Open)
> getTopicMetadata does not respect producer config settings
> ----------------------------------------------------------
>
> Key: KAFKA-608
> URL: https://issues.apache.org/jira/browse/KAFKA-608
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Jay Kreps
> Priority: Blocker
> Fix For: 0.8
>
>
> ProducerPool.scala contains the following code:
> object ProducerPool{
> def createSyncProducer(configOpt: Option[ProducerConfig], broker: Broker): SyncProducer = {
> val props = new Properties()
> props.put("host", broker.host)
> props.put("port", broker.port.toString)
> if(configOpt.isDefined)
> props.putAll(configOpt.get.props.props)
> new SyncProducer(new SyncProducerConfig(props))
> }
> }
> Note also, that ClientUtils.getTopicMetadata() does the following:
> ProducerPool.createSyncProducer(None, brokers(i))
> As a result there is no way to control the socket settings for the get metadata request.
> My recommendation is that we require the config to be specified in the
> Note that this creates a new sync producer without using ANY of the settings the user had given for the producer. In particular the socket timeout is defaulted to 500ms.
> This causes unit tests to fail a lot since a newly started test may easily timeout on a 500ms request.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-608) getTopicMetadata does not respect
producer config settings
Posted by "Neha Narkhede (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neha Narkhede updated KAFKA-608:
--------------------------------
Status: In Progress (was: Patch Available)
> getTopicMetadata does not respect producer config settings
> ----------------------------------------------------------
>
> Key: KAFKA-608
> URL: https://issues.apache.org/jira/browse/KAFKA-608
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Jay Kreps
> Assignee: Neha Narkhede
> Priority: Blocker
> Fix For: 0.8
>
>
> ProducerPool.scala contains the following code:
> object ProducerPool{
> def createSyncProducer(configOpt: Option[ProducerConfig], broker: Broker): SyncProducer = {
> val props = new Properties()
> props.put("host", broker.host)
> props.put("port", broker.port.toString)
> if(configOpt.isDefined)
> props.putAll(configOpt.get.props.props)
> new SyncProducer(new SyncProducerConfig(props))
> }
> }
> Note also, that ClientUtils.getTopicMetadata() does the following:
> ProducerPool.createSyncProducer(None, brokers(i))
> As a result there is no way to control the socket settings for the get metadata request.
> My recommendation is that we require the config to be specified in the
> Note that this creates a new sync producer without using ANY of the settings the user had given for the producer. In particular the socket timeout is defaulted to 500ms.
> This causes unit tests to fail a lot since a newly started test may easily timeout on a 500ms request.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-608) getTopicMetadata does not respect
producer config settings
Posted by "Neha Narkhede (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neha Narkhede updated KAFKA-608:
--------------------------------
Status: Patch Available (was: In Progress)
> getTopicMetadata does not respect producer config settings
> ----------------------------------------------------------
>
> Key: KAFKA-608
> URL: https://issues.apache.org/jira/browse/KAFKA-608
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Jay Kreps
> Assignee: Neha Narkhede
> Priority: Blocker
> Fix For: 0.8
>
> Attachments: kafka-608-v1.patch
>
>
> ProducerPool.scala contains the following code:
> object ProducerPool{
> def createSyncProducer(configOpt: Option[ProducerConfig], broker: Broker): SyncProducer = {
> val props = new Properties()
> props.put("host", broker.host)
> props.put("port", broker.port.toString)
> if(configOpt.isDefined)
> props.putAll(configOpt.get.props.props)
> new SyncProducer(new SyncProducerConfig(props))
> }
> }
> Note also, that ClientUtils.getTopicMetadata() does the following:
> ProducerPool.createSyncProducer(None, brokers(i))
> As a result there is no way to control the socket settings for the get metadata request.
> My recommendation is that we require the config to be specified in the
> Note that this creates a new sync producer without using ANY of the settings the user had given for the producer. In particular the socket timeout is defaulted to 500ms.
> This causes unit tests to fail a lot since a newly started test may easily timeout on a 500ms request.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-608) getTopicMetadata does not respect
producer config settings
Posted by "Jay Kreps (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jay Kreps updated KAFKA-608:
----------------------------
Priority: Blocker (was: Major)
Affects Version/s: 0.8
Fix Version/s: 0.8
> getTopicMetadata does not respect producer config settings
> ----------------------------------------------------------
>
> Key: KAFKA-608
> URL: https://issues.apache.org/jira/browse/KAFKA-608
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Jay Kreps
> Priority: Blocker
> Fix For: 0.8
>
>
> ProducerPool.scala contains the following code:
> object ProducerPool{
> def createSyncProducer(configOpt: Option[ProducerConfig], broker: Broker): SyncProducer = {
> val props = new Properties()
> props.put("host", broker.host)
> props.put("port", broker.port.toString)
> if(configOpt.isDefined)
> props.putAll(configOpt.get.props.props)
> new SyncProducer(new SyncProducerConfig(props))
> }
> }
> Note also, that ClientUtils.getTopicMetadata() does the following:
> ProducerPool.createSyncProducer(None, brokers(i))
> As a result there is no way to control the socket settings for the get metadata request.
> My recommendation is that we require the config to be specified in the
> Note that this creates a new sync producer without using ANY of the settings the user had given for the producer. In particular the socket timeout is defaulted to 500ms.
> This causes unit tests to fail a lot since a newly started test may easily timeout on a 500ms request.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira