[GitHub] jiangpengcheng opened a new issue #3458: producer.send() is not full asynchronous

[GitBox <gi...@apache.org>]

jiangpengcheng opened a new issue #3458: producer.send() is not full asynchronous
URL: https://github.com/apache/incubator-openwhisk/issues/3458
 
 
   When producer try to send message to a topic, the kafka client will try to fetch the topic metadata with timeout=60s, it is a blocking request so it may cause some problems, for example, when send message to a non-exists topic, it will hang up for 60 seconds, this may not happen in a real environment, but I think it's better to avoiding this
   
   ## Steps to reproduce the issue:
   
   1.   Create a test App:
   ```scala
   object TestKafka extends App {
       implicit val actorSystem = ActorSystem("controller-actor-system")
       implicit val logging = new AkkaLogging(actorSystem.log)
       def requiredProperties =
           Map(WhiskConfig.controllerInstances -> null) ++
               ExecManifest.requiredProperties ++
               SpiLoader.get[LoadBalancerProvider].requiredProperties ++
               EntitlementProvider.requiredProperties
   
       val config = new WhiskConfig(requiredProperties)
   
       private val messagingProvider = SpiLoader.get[MessagingProvider]
       private val messageProducer = messagingProvider.getProducer(config)
   
       logging.info(this, "before send message")
       messageProducer.send("invoker23", PingMessage(InstanceId(23, Some("23")))) //topic invoker23 doesn't exist
       logging.info(this, "after send message")
   }
   ```
   2. Run this application
   
   ## What we expect is :
   
   ```
   [2018-03-19T15:43:03.790Z] [INFO] Slf4jLogger started
   [2018-03-19T15:43:03.891Z] [INFO] [??] [WhiskConfig] reading properties from file C:\Users\platform-0420\IdeaProjects\lambda\whisk.properties
   [2018-03-19T15:43:04.332Z] [INFO] [??] [TestKafka] before send message
   [2018-03-19T15:43:04.406Z] [INFO] [??] [TestKafka] after send message
   [2018-03-19T15:44:04.409Z] [ERROR] [??] [KafkaProducerConnector] sending message on topic 'test' failed: Failed to update metadata after 60000 ms.
   [2018-03-19T15:44:04.481Z] [INFO] [??] [KafkaProducerConnector] old consumer closed
   ```
   
   
   ## But the actual results and outputs:
   
   ```
   [2018-03-19T15:44:52.490Z] [INFO] Slf4jLogger started
   [2018-03-19T15:44:52.580Z] [INFO] [??] [WhiskConfig] reading properties from file C:\Users\platform-0420\IdeaProjects\lambda\whisk.properties
   [2018-03-19T15:44:52.874Z] [INFO] [??] [TestKafka] before send message
   [2018-03-19T15:45:52.932Z] [INFO] [??] [TestKafka] after send message
   [2018-03-19T15:45:52.933Z] [ERROR] [??] [KafkaProducerConnector] sending message on topic 'test' failed: Failed to update metadata after 60000 ms.
   [2018-03-19T15:45:52.939Z] [INFO] [??] [KafkaProducerConnector] old consumer closed
   ```
   
   the difference is the time gap between print `after send message` and `before send message`
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services