You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Federico D'Ambrosio <fe...@gmail.com> on 2020/04/17 09:07:13 UTC
Cannot terminate query on KSQL
Hi,
We deployed 30-something queries which convert json events to avro, but
lately I've noticed some of the queries are consuming very slowly,
accumulating quite a bit of lag.
In order to reset those streams, I tried to terminate those queries, to
then recreate the streams, but the TERMINATE command from KSQL cli just
hangs without any feedback.
Ksql log is not much helpful, really just showing the POST request being
sent.
How should we proceed? Is there anyway to force deletion of a query/stream?
What could be causing this issue?
Thank you,
Federico D'Ambrosio
Re: Cannot terminate query on KSQL
Posted by Federico D'Ambrosio “fedexist” <fe...@gmail.com>.
Hi,
thank you for your reply, I opened an issue here: https://github.com/confluentinc/ksql/issues/5107 <https://github.com/confluentinc/ksql/issues/5107>
> Il giorno 17 apr 2020, alle ore 21:01, Guozhang Wang <wa...@gmail.com> ha scritto:
>
> Hi Federico,
>
> I'd think you could re-post this question here:
> https://github.com/confluentinc/ksql/issues.
>
> Cheers,
> Guozhang
>
>
> On Fri, Apr 17, 2020 at 8:57 AM Federico D'Ambrosio <fe...@gmail.com>
> wrote:
>
>> So, I tried with another query which hangs and I get this, after its
>> corresponding TERMINATE command:
>>
>> [2020-04-17 15:25:15,796] INFO stream-client
>>
>> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43]
>> State transition from RUNNING to PENDING_SHUTDOWN
>> (org.apache.kafka.streams.KafkaStreams:263)
>> [2020-04-17 15:25:15,801] INFO stream-thread
>>
>> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-319]
>> Informed to shut down
>> (org.apache.kafka.streams.processor.internals.StreamThread:1211)
>> [2020-04-17 15:25:15,801] INFO stream-thread
>>
>> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-319]
>> State transition from RUNNING to PENDING_SHUTDOWN
>> (org.apache.kafka.streams.processor.internals.StreamThread:220)
>> [2020-04-17 15:25:15,801] INFO stream-thread
>>
>> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
>> Informed to shut down
>> (org.apache.kafka.streams.processor.internals.StreamThread:1211)
>> [2020-04-17 15:25:15,801] INFO stream-thread
>>
>> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
>> State transition from RUNNING to PENDING_SHUTDOWN
>> (org.apache.kafka.streams.processor.internals.StreamThread:220)
>> [2020-04-17 15:25:15,801] INFO stream-thread
>>
>> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
>> Informed to shut down
>> (org.apache.kafka.streams.processor.internals.StreamThread:1211)
>> [2020-04-17 15:25:15,801] INFO stream-thread
>>
>> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
>> State transition from RUNNING to PENDING_SHUTDOWN
>> (org.apache.kafka.streams.processor.internals.StreamThread:220)
>> [2020-04-17 15:25:15,806] INFO stream-thread
>>
>> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
>> Shutting down
>> (org.apache.kafka.streams.processor.internals.StreamThread:1225)
>> [2020-04-17 15:25:15,816] INFO stream-thread
>>
>> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
>> State transition from PENDING_SHUTDOWN to DEAD
>> (org.apache.kafka.streams.processor.internals.StreamThread:220)
>> [2020-04-17 15:25:15,816] INFO stream-thread
>>
>> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
>> Shutdown complete
>> (org.apache.kafka.streams.processor.internals.StreamThread:1245)
>> [2020-04-17 15:25:15,830] INFO stream-thread
>>
>> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
>> Shutting down
>> (org.apache.kafka.streams.processor.internals.StreamThread:1225)
>> [2020-04-17 15:25:15,839] INFO stream-thread
>>
>> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
>> State transition from PENDING_SHUTDOWN to DEAD
>> (org.apache.kafka.streams.processor.internals.StreamThread:220)
>> *[2020-04-17 15:25:15,839] INFO stream-thread
>>
>> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
>> Shutdown complete
>> (org.apache.kafka.streams.processor.internals.StreamThread:1245)*
>>
>> So, at least this query tries to shutdown but only a single thread remains
>> alive, apparently. So:
>>
>> 1) CLI does not get any return (Query has been terminated)
>> 2) If I open another terminal, if i DESCRIBE EXTENDED the corresponding
>> stream, the query looks still attached to the stream and doesn't let me
>> drop it
>> 3) Consumer group does not get deleted
>>
>> If I restart the server:
>> 1) then, again, the query looks still attached to the stream (DESCRIBE
>> EXTENDED)
>> 3) Still, I cannot drop the stream because the query is attached
>>
>> This is a really strange behaviour, does anyone have any insight or any
>> pointers as to what I should look into?
>>
>> Thank you,
>>
>> Il giorno ven 17 apr 2020 alle ore 11:22 Federico D'Ambrosio <
>> fedexist@gmail.com> ha scritto:
>>
>>> Looking back at the ksql-stream logs:
>>>
>>> I find for a query I'm trying to terminate this line:
>>>
>>> [2020-04-17 09:17:52,502] INFO stream-client
>>>
>> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_PREV_GARANZIE_INTERNAL_AVRO_180-2625ef40-c00a-4cdf-98ea-df239aa2f29d]
>>> *Already in the pending shutdown state*, wait to complete shutdown
>>> (org.apache.kafka.streams.KafkaStreams:849)
>>>
>>> and it just stays there, without any more information.
>>> Restarting the server apparently doesn't help.
>>>
>>> Il giorno ven 17 apr 2020 alle ore 11:07 Federico D'Ambrosio <
>>> fedexist@gmail.com> ha scritto:
>>>
>>>> Hi,
>>>>
>>>> We deployed 30-something queries which convert json events to avro, but
>>>> lately I've noticed some of the queries are consuming very slowly,
>>>> accumulating quite a bit of lag.
>>>> In order to reset those streams, I tried to terminate those queries, to
>>>> then recreate the streams, but the TERMINATE command from KSQL cli just
>>>> hangs without any feedback.
>>>>
>>>> Ksql log is not much helpful, really just showing the POST request being
>>>> sent.
>>>>
>>>> How should we proceed? Is there anyway to force deletion of a
>>>> query/stream?
>>>> What could be causing this issue?
>>>>
>>>> Thank you,
>>>> Federico D'Ambrosio
>>>>
>>>
>>>
>>> --
>>> Federico D'Ambrosio
>>>
>>
>>
>> --
>> Federico D'Ambrosio
>>
>
>
> --
> -- Guozhang
Re: Cannot terminate query on KSQL
Posted by Guozhang Wang <wa...@gmail.com>.
Hi Federico,
I'd think you could re-post this question here:
https://github.com/confluentinc/ksql/issues.
Cheers,
Guozhang
On Fri, Apr 17, 2020 at 8:57 AM Federico D'Ambrosio <fe...@gmail.com>
wrote:
> So, I tried with another query which hangs and I get this, after its
> corresponding TERMINATE command:
>
> [2020-04-17 15:25:15,796] INFO stream-client
>
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43]
> State transition from RUNNING to PENDING_SHUTDOWN
> (org.apache.kafka.streams.KafkaStreams:263)
> [2020-04-17 15:25:15,801] INFO stream-thread
>
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-319]
> Informed to shut down
> (org.apache.kafka.streams.processor.internals.StreamThread:1211)
> [2020-04-17 15:25:15,801] INFO stream-thread
>
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-319]
> State transition from RUNNING to PENDING_SHUTDOWN
> (org.apache.kafka.streams.processor.internals.StreamThread:220)
> [2020-04-17 15:25:15,801] INFO stream-thread
>
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
> Informed to shut down
> (org.apache.kafka.streams.processor.internals.StreamThread:1211)
> [2020-04-17 15:25:15,801] INFO stream-thread
>
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
> State transition from RUNNING to PENDING_SHUTDOWN
> (org.apache.kafka.streams.processor.internals.StreamThread:220)
> [2020-04-17 15:25:15,801] INFO stream-thread
>
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
> Informed to shut down
> (org.apache.kafka.streams.processor.internals.StreamThread:1211)
> [2020-04-17 15:25:15,801] INFO stream-thread
>
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
> State transition from RUNNING to PENDING_SHUTDOWN
> (org.apache.kafka.streams.processor.internals.StreamThread:220)
> [2020-04-17 15:25:15,806] INFO stream-thread
>
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
> Shutting down
> (org.apache.kafka.streams.processor.internals.StreamThread:1225)
> [2020-04-17 15:25:15,816] INFO stream-thread
>
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
> State transition from PENDING_SHUTDOWN to DEAD
> (org.apache.kafka.streams.processor.internals.StreamThread:220)
> [2020-04-17 15:25:15,816] INFO stream-thread
>
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
> Shutdown complete
> (org.apache.kafka.streams.processor.internals.StreamThread:1245)
> [2020-04-17 15:25:15,830] INFO stream-thread
>
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
> Shutting down
> (org.apache.kafka.streams.processor.internals.StreamThread:1225)
> [2020-04-17 15:25:15,839] INFO stream-thread
>
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
> State transition from PENDING_SHUTDOWN to DEAD
> (org.apache.kafka.streams.processor.internals.StreamThread:220)
> *[2020-04-17 15:25:15,839] INFO stream-thread
>
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
> Shutdown complete
> (org.apache.kafka.streams.processor.internals.StreamThread:1245)*
>
> So, at least this query tries to shutdown but only a single thread remains
> alive, apparently. So:
>
> 1) CLI does not get any return (Query has been terminated)
> 2) If I open another terminal, if i DESCRIBE EXTENDED the corresponding
> stream, the query looks still attached to the stream and doesn't let me
> drop it
> 3) Consumer group does not get deleted
>
> If I restart the server:
> 1) then, again, the query looks still attached to the stream (DESCRIBE
> EXTENDED)
> 3) Still, I cannot drop the stream because the query is attached
>
> This is a really strange behaviour, does anyone have any insight or any
> pointers as to what I should look into?
>
> Thank you,
>
> Il giorno ven 17 apr 2020 alle ore 11:22 Federico D'Ambrosio <
> fedexist@gmail.com> ha scritto:
>
> > Looking back at the ksql-stream logs:
> >
> > I find for a query I'm trying to terminate this line:
> >
> > [2020-04-17 09:17:52,502] INFO stream-client
> >
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_PREV_GARANZIE_INTERNAL_AVRO_180-2625ef40-c00a-4cdf-98ea-df239aa2f29d]
> > *Already in the pending shutdown state*, wait to complete shutdown
> > (org.apache.kafka.streams.KafkaStreams:849)
> >
> > and it just stays there, without any more information.
> > Restarting the server apparently doesn't help.
> >
> > Il giorno ven 17 apr 2020 alle ore 11:07 Federico D'Ambrosio <
> > fedexist@gmail.com> ha scritto:
> >
> >> Hi,
> >>
> >> We deployed 30-something queries which convert json events to avro, but
> >> lately I've noticed some of the queries are consuming very slowly,
> >> accumulating quite a bit of lag.
> >> In order to reset those streams, I tried to terminate those queries, to
> >> then recreate the streams, but the TERMINATE command from KSQL cli just
> >> hangs without any feedback.
> >>
> >> Ksql log is not much helpful, really just showing the POST request being
> >> sent.
> >>
> >> How should we proceed? Is there anyway to force deletion of a
> >> query/stream?
> >> What could be causing this issue?
> >>
> >> Thank you,
> >> Federico D'Ambrosio
> >>
> >
> >
> > --
> > Federico D'Ambrosio
> >
>
>
> --
> Federico D'Ambrosio
>
--
-- Guozhang
Re: Cannot terminate query on KSQL
Posted by Federico D'Ambrosio <fe...@gmail.com>.
So, I tried with another query which hangs and I get this, after its
corresponding TERMINATE command:
[2020-04-17 15:25:15,796] INFO stream-client
[_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43]
State transition from RUNNING to PENDING_SHUTDOWN
(org.apache.kafka.streams.KafkaStreams:263)
[2020-04-17 15:25:15,801] INFO stream-thread
[_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-319]
Informed to shut down
(org.apache.kafka.streams.processor.internals.StreamThread:1211)
[2020-04-17 15:25:15,801] INFO stream-thread
[_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-319]
State transition from RUNNING to PENDING_SHUTDOWN
(org.apache.kafka.streams.processor.internals.StreamThread:220)
[2020-04-17 15:25:15,801] INFO stream-thread
[_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
Informed to shut down
(org.apache.kafka.streams.processor.internals.StreamThread:1211)
[2020-04-17 15:25:15,801] INFO stream-thread
[_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
State transition from RUNNING to PENDING_SHUTDOWN
(org.apache.kafka.streams.processor.internals.StreamThread:220)
[2020-04-17 15:25:15,801] INFO stream-thread
[_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
Informed to shut down
(org.apache.kafka.streams.processor.internals.StreamThread:1211)
[2020-04-17 15:25:15,801] INFO stream-thread
[_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
State transition from RUNNING to PENDING_SHUTDOWN
(org.apache.kafka.streams.processor.internals.StreamThread:220)
[2020-04-17 15:25:15,806] INFO stream-thread
[_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
Shutting down
(org.apache.kafka.streams.processor.internals.StreamThread:1225)
[2020-04-17 15:25:15,816] INFO stream-thread
[_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
State transition from PENDING_SHUTDOWN to DEAD
(org.apache.kafka.streams.processor.internals.StreamThread:220)
[2020-04-17 15:25:15,816] INFO stream-thread
[_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-320]
Shutdown complete
(org.apache.kafka.streams.processor.internals.StreamThread:1245)
[2020-04-17 15:25:15,830] INFO stream-thread
[_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
Shutting down
(org.apache.kafka.streams.processor.internals.StreamThread:1225)
[2020-04-17 15:25:15,839] INFO stream-thread
[_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
State transition from PENDING_SHUTDOWN to DEAD
(org.apache.kafka.streams.processor.internals.StreamThread:220)
*[2020-04-17 15:25:15,839] INFO stream-thread
[_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_ANAGRAFE_CARTE_CREDITO_INTERNAL_AVRO_166-1207a327-6eed-4c74-9787-95825e1bec43-StreamThread-321]
Shutdown complete
(org.apache.kafka.streams.processor.internals.StreamThread:1245)*
So, at least this query tries to shutdown but only a single thread remains
alive, apparently. So:
1) CLI does not get any return (Query has been terminated)
2) If I open another terminal, if i DESCRIBE EXTENDED the corresponding
stream, the query looks still attached to the stream and doesn't let me
drop it
3) Consumer group does not get deleted
If I restart the server:
1) then, again, the query looks still attached to the stream (DESCRIBE
EXTENDED)
3) Still, I cannot drop the stream because the query is attached
This is a really strange behaviour, does anyone have any insight or any
pointers as to what I should look into?
Thank you,
Il giorno ven 17 apr 2020 alle ore 11:22 Federico D'Ambrosio <
fedexist@gmail.com> ha scritto:
> Looking back at the ksql-stream logs:
>
> I find for a query I'm trying to terminate this line:
>
> [2020-04-17 09:17:52,502] INFO stream-client
> [_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_PREV_GARANZIE_INTERNAL_AVRO_180-2625ef40-c00a-4cdf-98ea-df239aa2f29d]
> *Already in the pending shutdown state*, wait to complete shutdown
> (org.apache.kafka.streams.KafkaStreams:849)
>
> and it just stays there, without any more information.
> Restarting the server apparently doesn't help.
>
> Il giorno ven 17 apr 2020 alle ore 11:07 Federico D'Ambrosio <
> fedexist@gmail.com> ha scritto:
>
>> Hi,
>>
>> We deployed 30-something queries which convert json events to avro, but
>> lately I've noticed some of the queries are consuming very slowly,
>> accumulating quite a bit of lag.
>> In order to reset those streams, I tried to terminate those queries, to
>> then recreate the streams, but the TERMINATE command from KSQL cli just
>> hangs without any feedback.
>>
>> Ksql log is not much helpful, really just showing the POST request being
>> sent.
>>
>> How should we proceed? Is there anyway to force deletion of a
>> query/stream?
>> What could be causing this issue?
>>
>> Thank you,
>> Federico D'Ambrosio
>>
>
>
> --
> Federico D'Ambrosio
>
--
Federico D'Ambrosio
Re: Cannot terminate query on KSQL
Posted by Federico D'Ambrosio <fe...@gmail.com>.
Looking back at the ksql-stream logs:
I find for a query I'm trying to terminate this line:
[2020-04-17 09:17:52,502] INFO stream-client
[_confluent-ksql-default_query_CSAS_PROD_GTEL_AFA_GOL_PREV_GARANZIE_INTERNAL_AVRO_180-2625ef40-c00a-4cdf-98ea-df239aa2f29d]
*Already in the pending shutdown state*, wait to complete shutdown
(org.apache.kafka.streams.KafkaStreams:849)
and it just stays there, without any more information.
Restarting the server apparently doesn't help.
Il giorno ven 17 apr 2020 alle ore 11:07 Federico D'Ambrosio <
fedexist@gmail.com> ha scritto:
> Hi,
>
> We deployed 30-something queries which convert json events to avro, but
> lately I've noticed some of the queries are consuming very slowly,
> accumulating quite a bit of lag.
> In order to reset those streams, I tried to terminate those queries, to
> then recreate the streams, but the TERMINATE command from KSQL cli just
> hangs without any feedback.
>
> Ksql log is not much helpful, really just showing the POST request being
> sent.
>
> How should we proceed? Is there anyway to force deletion of a
> query/stream?
> What could be causing this issue?
>
> Thank you,
> Federico D'Ambrosio
>
--
Federico D'Ambrosio