[jira] [Commented] (KAFKA-10569) Running aggregate queries on KSQL client side is getting to ERROR Shutdown broker because all log dirs in ...

["Matthias J. Sax (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-10569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17206536#comment-17206536 ] 

Matthias J. Sax commented on KAFKA-10569:
-----------------------------------------

[~petregordan@gmail.com] – ksqlDB is not part of Apache Kafka but its own project. Of course, ksqlDB uses KafkaStreams internally. Thus, I am wondering (without reading the full ticket description yet) to what extend this is a KafkaStreams issue or if you should open a ticket for ksqlDB (they track issue via GitHub: [https://github.com/confluentinc/ksql/issues].)

If it is a KafkaStreams issue, if would be great to describe it in terms of the KafakStreams API and also update the "component" of this ticket to `KafkaStreams` (instead of `clients`). If it's a ksqlDB issue, we should close this ticket as "invalid" though.

> Running aggregate queries on KSQL client side is getting to ERROR Shutdown broker because all log dirs in ...
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-10569
>                 URL: https://issues.apache.org/jira/browse/KAFKA-10569
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients
>    Affects Versions: 2.5.0
>         Environment: local
>            Reporter: Petre Gordan
>            Priority: Critical
>         Attachments: KSQLDBServerSideErrors.txt, KafkaClusterLogs.txt, ProductsOrders.txt, ZiikeeperSideLog.txt, kafka-server-start.bat, ksql, ksql-server-start, ksql-server.properties, schema-registry.properties, server.properties, zookeeper-server-start.bat, zookeeper.properties
>
>
> Working on Windows 10 and confluent-5.5.0-2.12.zip and kafka_2.12-2.5.0. I'm running locally:
>  * in powershell, zookeeper with: *bin\windows\zookeeper-server-start.bat config\zookeeper.properties*
>  * in powershell, kafka-server
> *bin\windows\kafka-server-start.bat config\server.properties*
>  * in bash (with ubuntu) ksqldb server
> sudo bin/ksql-server-start etc/ksqldb/ksql-server.properties
>  * in bash (with ubuntu) ksql client
> sudo bin/ksql [http://0.0.0.0:8088|http://0.0.0.0:8088/]
> After all of these are sorted, than, I'm starting to practice the Kafka. So, I'm creating tables, streams, making inserts, and all good. I can do small queries like: 
> select * from products emit changes;, etc.
> All good until at this step.
> When, I'm trying to run every type of aggregate query, than is showing the result after a while, but into the end, after I will press Ctrl+C to terminate that and to do other query, everything is down.
> For example, see the attached .sql script, and after I will run that script the products table and orders stream are created with success and populated with success.
> After that if I run this query:
> select ProductRowKey, count(ProductRowKey) from orders group by ProductRowKey emit changes;
> I can see the results, all good, but into the end if I will press Ctrl + C, than everything is down.
>  
> Looking into the logs and taking based on the time history the main raised warning and issues are:
>  * first is raised this: 
> Query terminated due to exception:org.eclipse.jetty.io.EofException (io.confluent.ksql.rest.server.resources.streaming.QueryStreamWriter:95)
>  * than this: INFO stream-client [_confluent-ksql-default_transient State transition from RUNNING to PENDING_SHUTDOWN (org.apache.kafka.streams.KafkaStreams:285)
>  * than these:
> INFO stream-thread [_confluent-ksql-default_transient Informed to shut down (org.apache.kafka.streams.processor.internals.StreamThread:1116)
>  State transition from RUNNING to PENDING_SHUTDOWN (org.apache.kafka.streams.processor.internals.StreamThread:221)
>  Shutdown complete (org.apache.kafka.streams.processor.internals.StreamThread:1150)
>  * than this:
> INFO stream-thread [qtp2032891036-47] Deleting obsolete state directory 0_0 for task 0_0 as 1ms has elapsed
>  * than this:
> WARN Could not clean up the schema registry for query: _confluent-ksql-default_transient_
>  * than this:
> WARN [Producer clientId=producer-1] Connection to node 0 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient:763)
> all of these from above on the KSQLDB server side logs,
>  * finally this on the Kafka cluster side:
> ERROR Shutdown broker because all log dirs in ........ have failed (kafka.log.LogManager)
>  
> And after that everything is down. Please see all the attached files to get all the info.
>  
> Please help me with these.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)