You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Николай Ижиков <ni...@apache.org> on 2020/01/15 18:30:11 UTC
[DISCUSSION] API to KILL any user started computation
Hello, Igniters.
As you may know, we put a lot of effort to improve Ignite metric and diagnostic API.
We have created the following API:
* Metric manager
* System view manager
As far as I know, we would have tracing capabilities soon.
I think it's time to take the next step.
We should provide to the administrator the API to cancel(stop, kill) arbitrary user started computation.
Right now we have API to do it for:
* transactions `TransactionsMXBean#getActiveTransactions`.
* SQL queries: `KILL QUERY` sql command and visor task - `VisorQueryCancelTask`
Please, note, these features are implemented via different API.
I think we should introduce uniform Cancel API for the following computations:
* service.
* specific service method execution.
* compute job(compute task).
* query(scan, continuous, text).
We should also rework kill transaction and kill SQL queries API to make them similar to each other.
I have plans to write an IEP for it and implement it.
What do you think?
Re: [DISCUSSION][IEP-39] Management API to cancel user provided tasks
and queries.
Posted by Nikolay Izhikov <ni...@apache.org>.
Hello, Igniters.
PR [1] for the IEP-39 [2] are ready to be reviewed.
Ticket [3].
Can someone, please, review my changes
I propose to introduce the following features to Ignite management APIs.
1. JMX beans
```
ComputeMXBean#cancel(String sessionId);
QueryMXBean#cacncelSQL(String id)
QueryMXBean#cacncelScan(String originNodeId, String cacheName, Long id)
QueryMXBean#cacncelContinuous(String routineId)
TransactionMXBean#cancel(String xid)
ServiceMXBean#cancel(String name)
```
2. control.sh
```
> ./control.sh --kill scan b2d221ca-ab08-4544-b8dc-8475538ed42f default 63
> ./control.sh --kill compute 58b48c3e071-b2d221ca-ab08-4544-b8dc-8475538ed42f
> ./control.sh --kill tx 30058c3e071-00000000-0bac-6d33-0000-000000000003
> ./control.sh --kill sql b2d221ca-ab08-4544-b8dc-8475538ed42f_7
> ./control.sh --kill service my-svc
> ./control.sh --kill continuous bfca668d-5df8-4879-b97f-bd1712ad01c9
```
3. SQL(KILL QUERY for SQL queries already implemented):
```
KILL SCAN_QUERY 'b2d221ca-ab08-4544-b8dc-8475538ed42f' 'default' 63
KILL TX '30058c3e071-00000000-0bac-6d33-0000-000000000003'
KILL CONTINUOUS_QUERY 'bfca668d-5df8-4879-b97f-bd1712ad01c9'
KILL COMPUTE_TASK '58b48c3e071-b2d221ca-ab08-4544-b8dc-8475538ed42f'
KILL SERVICE 'my-svc'
```
[1] https://github.com/apache/ignite/pull/7520
[2] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=145724615
[3] https://issues.apache.org/jira/browse/IGNITE-12632
> 7 февр. 2020 г., в 14:53, Nikolay Izhikov <ni...@gmail.com> написал(а):
>
>> IMHO, the control utility is a more natural way of administration/control
> of the cluster instead of JMX, for example.
>
> It’s questionable.
>
> I don’t mind to improve control.sh in this IEP.
>
> But, we should discuss to topic - what management utilities we are providing *AND* supporting and how the should work.
> As far as I know we have some REST API, but it seems abandoned and not supported.
>
>
>
>> 7 февр. 2020 г., в 14:45, Вячеслав Коптилин <sl...@gmail.com> написал(а):
>>
>> Hi Nikolay,
>>
>> Yes, I think we should add this API to the control utility as well.
>> IMHO, the control utility is a more natural way of administration/control
>> of the cluster instead of JMX, for example.
>>
>> Thanks,
>> S.
>>
>> пт, 7 февр. 2020 г. в 11:38, Nikolay Izhikov <ni...@apache.org>:
>>
>>> Hello, Vyacheslav.
>>>
>>>> It seems to me we missed API that should be introduced into control
>>> utility.
>>>
>>> Do you think we should support control.sh for cancel tasks?
>>>
>>>
>>>> 7 февр. 2020 г., в 11:04, Alexey Goncharuk <al...@gmail.com>
>>> написал(а):
>>>>
>>>> Alexei,
>>>>
>>>> I agree that there should be no principal difficulty with the unique ID
>>> for
>>>> each cancellable object, but I also see Nikolay's point about the wrong
>>>> copy-paste.
>>>>
>>>> I like minimalistic APIs, but in this case the price of a mistake may be
>>>> high. Let's let a wider circle of community members to speak out.
>>>
>>>
>
Re: [DISCUSSION][IEP-39] Management API to cancel user provided tasks
and queries.
Posted by Nikolay Izhikov <ni...@apache.org>.
> IMHO, the control utility is a more natural way of administration/control
of the cluster instead of JMX, for example.
It’s questionable.
I don’t mind to improve control.sh in this IEP.
But, we should discuss to topic - what management utilities we are providing *AND* supporting and how the should work.
As far as I know we have some REST API, but it seems abandoned and not supported.
> 7 февр. 2020 г., в 14:45, Вячеслав Коптилин <sl...@gmail.com> написал(а):
>
> Hi Nikolay,
>
> Yes, I think we should add this API to the control utility as well.
> IMHO, the control utility is a more natural way of administration/control
> of the cluster instead of JMX, for example.
>
> Thanks,
> S.
>
> пт, 7 февр. 2020 г. в 11:38, Nikolay Izhikov <ni...@apache.org>:
>
>> Hello, Vyacheslav.
>>
>>> It seems to me we missed API that should be introduced into control
>> utility.
>>
>> Do you think we should support control.sh for cancel tasks?
>>
>>
>>> 7 февр. 2020 г., в 11:04, Alexey Goncharuk <al...@gmail.com>
>> написал(а):
>>>
>>> Alexei,
>>>
>>> I agree that there should be no principal difficulty with the unique ID
>> for
>>> each cancellable object, but I also see Nikolay's point about the wrong
>>> copy-paste.
>>>
>>> I like minimalistic APIs, but in this case the price of a mistake may be
>>> high. Let's let a wider circle of community members to speak out.
>>
>>
Re: [DISCUSSION][IEP-39] Management API to cancel user provided tasks
and queries.
Posted by Вячеслав Коптилин <sl...@gmail.com>.
Hi Nikolay,
Yes, I think we should add this API to the control utility as well.
IMHO, the control utility is a more natural way of administration/control
of the cluster instead of JMX, for example.
Thanks,
S.
пт, 7 февр. 2020 г. в 11:38, Nikolay Izhikov <ni...@apache.org>:
> Hello, Vyacheslav.
>
> > It seems to me we missed API that should be introduced into control
> utility.
>
> Do you think we should support control.sh for cancel tasks?
>
>
> > 7 февр. 2020 г., в 11:04, Alexey Goncharuk <al...@gmail.com>
> написал(а):
> >
> > Alexei,
> >
> > I agree that there should be no principal difficulty with the unique ID
> for
> > each cancellable object, but I also see Nikolay's point about the wrong
> > copy-paste.
> >
> > I like minimalistic APIs, but in this case the price of a mistake may be
> > high. Let's let a wider circle of community members to speak out.
>
>
Re: [DISCUSSION][IEP-39] Management API to cancel user provided tasks
and queries.
Posted by Nikolay Izhikov <ni...@apache.org>.
Hello, Vyacheslav.
> It seems to me we missed API that should be introduced into control utility.
Do you think we should support control.sh for cancel tasks?
> 7 февр. 2020 г., в 11:04, Alexey Goncharuk <al...@gmail.com> написал(а):
>
> Alexei,
>
> I agree that there should be no principal difficulty with the unique ID for
> each cancellable object, but I also see Nikolay's point about the wrong
> copy-paste.
>
> I like minimalistic APIs, but in this case the price of a mistake may be
> high. Let's let a wider circle of community members to speak out.
Re: [DISCUSSION][IEP-39] Management API to cancel user provided tasks
and queries.
Posted by Alexey Goncharuk <al...@gmail.com>.
Alexei,
I agree that there should be no principal difficulty with the unique ID for
each cancellable object, but I also see Nikolay's point about the wrong
copy-paste.
I like minimalistic APIs, but in this case the price of a mistake may be
high. Let's let a wider circle of community members to speak out.
Re: [DISCUSSION][IEP-39] Management API to cancel user provided tasks
and queries.
Posted by Alexei Scherbakov <al...@gmail.com>.
Nick
I see no difficulty in assinging each cancellable object an IgniteUuid
(which is actually java UUID and is guaranteed to be unique per node).
We can have registry of running queries and just poking to what registry is
enough to understand query type.
чт, 6 февр. 2020 г. в 17:17, Вячеслав Коптилин <sl...@gmail.com>:
> Hello,
>
> It seems to me we missed API that should be introduced into control
> utility.
> Nikolay, could you please note this requirement on the IEP page?
>
> Thanks,
> S.
>
> чт, 6 февр. 2020 г. в 15:29, Nikolay Izhikov <ni...@apache.org>:
>
> > Ticket [1] created.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-12632
> >
> > > 5 февр. 2020 г., в 15:36, Nikolay Izhikov <ni...@gmail.com>
> > написал(а):
> > >
> > > Alexey.
> > >
> > > I’m talking the following scenario:
> > >
> > > 1. Consider we have unified bean to kill tasks:
> > >
> > > CancelMXBean {
> > > public void cancel(long id);
> > > }
> > >
> > > 2. And we have the following log:
> > >
> > > ```
> > > Transaction with ID=42 started.
> > > Compute task with ID=43 started.
> > > ```
> > >
> > > 3. We want to kill compute task and by mistake executing:
> > >
> > > cancelMxBean.cancel(42); //This will kill transaction not compute task.
> > >
> > > The user doesn’t have a chance to know, what type of object he is
> > killing.
> > > I think we should prevent this type of error by the API design.
> > >
> > >
> > >> 5 февр. 2020 г., в 14:43, Alexey Goncharuk <
> alexey.goncharuk@gmail.com>
> > написал(а):
> > >>
> > >> Nikolay,
> > >>
> > >>
> > >>> Consider copy-pasting wrong id from log to its
> > >>> parameters(killing not the buggy compute task, but *VERY* important
> > >>> transaction.
> > >>> How users even know about this type of error with the
> > >>> single method approach?
> > >>>
> > >>> I thought that the identifiers would never intersect (meaning that a
> > >> transaction and a task would never share the same ID)
> > >>
> > >> I agree that change ID types for all objects would be a hard task, so
> > >> probably it's worth discussing a single cancel entry on phase 3.
> > >
> >
> >
>
--
Best regards,
Alexei Scherbakov
Re: [DISCUSSION][IEP-39] Management API to cancel user provided tasks
and queries.
Posted by Вячеслав Коптилин <sl...@gmail.com>.
Hello,
It seems to me we missed API that should be introduced into control utility.
Nikolay, could you please note this requirement on the IEP page?
Thanks,
S.
чт, 6 февр. 2020 г. в 15:29, Nikolay Izhikov <ni...@apache.org>:
> Ticket [1] created.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12632
>
> > 5 февр. 2020 г., в 15:36, Nikolay Izhikov <ni...@gmail.com>
> написал(а):
> >
> > Alexey.
> >
> > I’m talking the following scenario:
> >
> > 1. Consider we have unified bean to kill tasks:
> >
> > CancelMXBean {
> > public void cancel(long id);
> > }
> >
> > 2. And we have the following log:
> >
> > ```
> > Transaction with ID=42 started.
> > Compute task with ID=43 started.
> > ```
> >
> > 3. We want to kill compute task and by mistake executing:
> >
> > cancelMxBean.cancel(42); //This will kill transaction not compute task.
> >
> > The user doesn’t have a chance to know, what type of object he is
> killing.
> > I think we should prevent this type of error by the API design.
> >
> >
> >> 5 февр. 2020 г., в 14:43, Alexey Goncharuk <al...@gmail.com>
> написал(а):
> >>
> >> Nikolay,
> >>
> >>
> >>> Consider copy-pasting wrong id from log to its
> >>> parameters(killing not the buggy compute task, but *VERY* important
> >>> transaction.
> >>> How users even know about this type of error with the
> >>> single method approach?
> >>>
> >>> I thought that the identifiers would never intersect (meaning that a
> >> transaction and a task would never share the same ID)
> >>
> >> I agree that change ID types for all objects would be a hard task, so
> >> probably it's worth discussing a single cancel entry on phase 3.
> >
>
>
Re: [DISCUSSION][IEP-39] Management API to cancel user provided tasks
and queries.
Posted by Nikolay Izhikov <ni...@apache.org>.
Ticket [1] created.
[1] https://issues.apache.org/jira/browse/IGNITE-12632
> 5 февр. 2020 г., в 15:36, Nikolay Izhikov <ni...@gmail.com> написал(а):
>
> Alexey.
>
> I’m talking the following scenario:
>
> 1. Consider we have unified bean to kill tasks:
>
> CancelMXBean {
> public void cancel(long id);
> }
>
> 2. And we have the following log:
>
> ```
> Transaction with ID=42 started.
> Compute task with ID=43 started.
> ```
>
> 3. We want to kill compute task and by mistake executing:
>
> cancelMxBean.cancel(42); //This will kill transaction not compute task.
>
> The user doesn’t have a chance to know, what type of object he is killing.
> I think we should prevent this type of error by the API design.
>
>
>> 5 февр. 2020 г., в 14:43, Alexey Goncharuk <al...@gmail.com> написал(а):
>>
>> Nikolay,
>>
>>
>>> Consider copy-pasting wrong id from log to its
>>> parameters(killing not the buggy compute task, but *VERY* important
>>> transaction.
>>> How users even know about this type of error with the
>>> single method approach?
>>>
>>> I thought that the identifiers would never intersect (meaning that a
>> transaction and a task would never share the same ID)
>>
>> I agree that change ID types for all objects would be a hard task, so
>> probably it's worth discussing a single cancel entry on phase 3.
>
Re: [DISCUSSION][IEP-39] Management API to cancel user provided tasks
and queries.
Posted by Nikolay Izhikov <ni...@apache.org>.
Alexey.
I’m talking the following scenario:
1. Consider we have unified bean to kill tasks:
CancelMXBean {
public void cancel(long id);
}
2. And we have the following log:
```
Transaction with ID=42 started.
Compute task with ID=43 started.
```
3. We want to kill compute task and by mistake executing:
cancelMxBean.cancel(42); //This will kill transaction not compute task.
The user doesn’t have a chance to know, what type of object he is killing.
I think we should prevent this type of error by the API design.
> 5 февр. 2020 г., в 14:43, Alexey Goncharuk <al...@gmail.com> написал(а):
>
> Nikolay,
>
>
>> Consider copy-pasting wrong id from log to its
>> parameters(killing not the buggy compute task, but *VERY* important
>> transaction.
>> How users even know about this type of error with the
>> single method approach?
>>
>> I thought that the identifiers would never intersect (meaning that a
> transaction and a task would never share the same ID)
>
> I agree that change ID types for all objects would be a hard task, so
> probably it's worth discussing a single cancel entry on phase 3.
Re: [DISCUSSION][IEP-39] Management API to cancel user provided tasks
and queries.
Posted by Alexey Goncharuk <al...@gmail.com>.
Nikolay,
> Consider copy-pasting wrong id from log to its
> parameters(killing not the buggy compute task, but *VERY* important
> transaction.
> How users even know about this type of error with the
> single method approach?
>
> I thought that the identifiers would never intersect (meaning that a
transaction and a task would never share the same ID)
I agree that change ID types for all objects would be a hard task, so
probably it's worth discussing a single cancel entry on phase 3.
Re: [DISCUSSION][IEP-39] Management API to cancel user provided tasks
and queries.
Posted by Nikolay Izhikov <ni...@apache.org>.
Igniters, thanks for the feedback.
> Can we retain only single cancel(long queryId) method in QueryMXBean ?
> I think query type can be determined by queryId.
It seems the answer is no we can’t do it for now.
Different types of queries are handled by different managers and have different sets of identifiers.
Please, take a look, into `ScanQueryView` [1], `RunningQueryManager` [2]
> Right now different types of IDs (long, UUID, IgniteUuid) look confusing and messy.
Yes.
Let’s, fix it.
> Shouldn't we unify identifiers for all ongoing tasks in Ignite and move cancel to a single method in a single bean?
I think we should:
1. create separate beans for each API - TransactionMXBean, ComputeMXBean, etc.
* The single method will be very error-prone:
Consider copy-pasting wrong id from log to its parameters(killing not the buggy compute task, but *VERY* important transaction.
How users even know about this type of error with the single method approach?
* When the user wants to kill some task(from a script or by hand) he knows the type of task for sure.
2. Implement IEP-39 with the current identifiers.
3. Improve our API with unified identifier types.
Bringing our API to the unified identifiers types approach is a very hard task to do and it will take a long time to implement.
We shouldn’t leave our users without the proposed API until this moment.
[1] https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/systemview/view/ScanQueryView.java
[2] https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/query/RunningQueryManager.java
> 5 февр. 2020 г., в 12:07, Alexey Goncharuk <al...@gmail.com> написал(а):
>
> Folks,
>
> Shouldn't we unify identifiers for all ongoing tasks in Ignite and move
> cancel to a single method in a single bean?
>
> Right now different types of IDs (long, UUID, IgniteUuid) look confusing
> and messy. I understand that such API is enforced by the IDs implementation
> in the corresponding Ignite subsystems. However, given that lately we
> developed a unified approach to metrics and system views with Nikolay's
> help, shouldn't we take one more step forward and unify this part as well?
Re: [DISCUSSION][IEP-39] Management API to cancel user provided tasks
and queries.
Posted by Alexey Goncharuk <al...@gmail.com>.
Folks,
Shouldn't we unify identifiers for all ongoing tasks in Ignite and move
cancel to a single method in a single bean?
Right now different types of IDs (long, UUID, IgniteUuid) look confusing
and messy. I understand that such API is enforced by the IDs implementation
in the corresponding Ignite subsystems. However, given that lately we
developed a unified approach to metrics and system views with Nikolay's
help, shouldn't we take one more step forward and unify this part as well?
Re: [DISCUSSION][IEP-39] Management API to cancel user provided tasks
and queries.
Posted by Alexei Scherbakov <al...@gmail.com>.
Nick
Can we retain only single cancel(long queryId) method in QueryMXBean ?
I think query type can be determined by queryId.
I also think we need similar "cancellation" API without the need to use mx
beans.
вт, 4 февр. 2020 г. в 18:07, Nikolay Izhikov <ni...@apache.org>:
> Hello, Igniters.
>
> I propose to create management API to dance user provided tasks and
> queries.
> Below my proposal in the IEP [1] form.
>
> Please, share your feedback.
>
>
> Motivation:
>
> Ignite provides many API to deploy and execute user-provided code on
> the server nodes
> inside the sam JVM as Ignite process runs.
>
> We also has a many APIs that allocate many resources on the server
> nodes:
>
> In case of some buggy code that consumes many system resources(CPU,
> RAM, flood network) or heavy query
> the whole cluster can becomes instable.
> We should provide to the cluster administator the ability to stop any
> user deployed task.
>
> Description:
> JMX beans to cancel listed tasks should be introduced:
> * Compute task
> * Service
> * Continuous query
> * Transactions
> * Queries(SQL, Scan, Text)
>
> In the scope of IEP-35 System view API introduced.
> New API should use the same identifier that is used in corresponding
> System View.
>
> Public API changes:
> * Cancel of Scan(text) queries should work similar to the SQL.
> * Cancel of compute task should invoke
> `ComputeTaskInternalFuture#cancel` for the corresponding task.
> * Cancel of service should be the same as `IgniteServices#cancel`
> * Cancel of transaction should works the same as in
> `TransactionMXBean#getActiveTransaction(kill=true)`.
> * Cancel of a continuous query should work the same as
> `QueryCursor#close` execution.
>
> ```
> QueryMXBean {
> boolean cancelSQL(Long queryId); //Already implemented in
> IGNITE-4436.
>
> boolean cancelScan(Long queryId);
>
> boolean cancelText(Long queryId);
> }
>
> ComputeMXBean {
> boolean cancel(IgniteUuid id);
> }
>
> ServiceMXBean {
> boolean cancel(IgniteUuid id);
> }
>
> TransactionMXBean {
> boolean cancel(IgniteUuid xid);
> }
>
> interface ContinuousQueryMXBean {
> boolean cancel(UUID id);
> }
> ```
>
> [1]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=145724615
>
>
>
> > 17 янв. 2020 г., в 22:43, Николай Ижиков <ni...@gmail.com>
> написал(а):
> >
> > Hello, Andrey.
> >
> > Thanks for positive feedback.
> > Appreciate it.
> >
> >> we can't cancel service or service's method
> >
> > I understand it.
> > Seems, we have several obvious options here:
> >
> > * Try to do it and if fail answer to the user: «I’ve tried and fail.
> Sorry»
> > * Kill hanging thread.
> >
> >> This invocations not register anywhere and can't be tracked and killed.
> >
> > Yes.
> > So we should invent tracking for those invocation to be able to kill it
> :)
> >
> >
> >> 17 янв. 2020 г., в 21:03, Andrey Gura <ag...@apache.org> написал(а):
> >>
> >> It would be grate.
> >>
> >> Only one comment: we can't cancel service or service's method
> >> execution because service grid API doesn't imply any requirement to
> >> service implementation. So if user do something like infinite cycle or
> >> blocking but non-interruptible operation it's impossible to interrupt
> >> it. Also service method invocation itself isn't cluster wide or local
> >> node specific operation. This invocations not register anywhere and
> >> can't be tracked and killed.
> >>
> >> Unfortunately, the same is valid for some jobs in sense of infinite or
> >> non-interruptible operations.
> >>
> >> On Wed, Jan 15, 2020 at 9:30 PM Николай Ижиков <ni...@apache.org>
> wrote:
> >>>
> >>> Hello, Igniters.
> >>>
> >>> As you may know, we put a lot of effort to improve Ignite metric and
> diagnostic API.
> >>> We have created the following API:
> >>> * Metric manager
> >>> * System view manager
> >>> As far as I know, we would have tracing capabilities soon.
> >>>
> >>> I think it's time to take the next step.
> >>> We should provide to the administrator the API to cancel(stop, kill)
> arbitrary user started computation.
> >>>
> >>> Right now we have API to do it for:
> >>> * transactions `TransactionsMXBean#getActiveTransactions`.
> >>> * SQL queries: `KILL QUERY` sql command and visor task -
> `VisorQueryCancelTask`
> >>>
> >>> Please, note, these features are implemented via different API.
> >>>
> >>> I think we should introduce uniform Cancel API for the following
> computations:
> >>>
> >>> * service.
> >>> * specific service method execution.
> >>> * compute job(compute task).
> >>> * query(scan, continuous, text).
> >>>
> >>> We should also rework kill transaction and kill SQL queries API to
> make them similar to each other.
> >>> I have plans to write an IEP for it and implement it.
> >>> What do you think?
> >>>
> >
>
>
--
Best regards,
Alexei Scherbakov
[DISCUSSION][IEP-39] Management API to cancel user provided tasks and
queries.
Posted by Nikolay Izhikov <ni...@apache.org>.
Hello, Igniters.
I propose to create management API to dance user provided tasks and queries.
Below my proposal in the IEP [1] form.
Please, share your feedback.
Motivation:
Ignite provides many API to deploy and execute user-provided code on the server nodes
inside the sam JVM as Ignite process runs.
We also has a many APIs that allocate many resources on the server nodes:
In case of some buggy code that consumes many system resources(CPU, RAM, flood network) or heavy query
the whole cluster can becomes instable.
We should provide to the cluster administator the ability to stop any user deployed task.
Description:
JMX beans to cancel listed tasks should be introduced:
* Compute task
* Service
* Continuous query
* Transactions
* Queries(SQL, Scan, Text)
In the scope of IEP-35 System view API introduced.
New API should use the same identifier that is used in corresponding System View.
Public API changes:
* Cancel of Scan(text) queries should work similar to the SQL.
* Cancel of compute task should invoke `ComputeTaskInternalFuture#cancel` for the corresponding task.
* Cancel of service should be the same as `IgniteServices#cancel`
* Cancel of transaction should works the same as in `TransactionMXBean#getActiveTransaction(kill=true)`.
* Cancel of a continuous query should work the same as `QueryCursor#close` execution.
```
QueryMXBean {
boolean cancelSQL(Long queryId); //Already implemented in IGNITE-4436.
boolean cancelScan(Long queryId);
boolean cancelText(Long queryId);
}
ComputeMXBean {
boolean cancel(IgniteUuid id);
}
ServiceMXBean {
boolean cancel(IgniteUuid id);
}
TransactionMXBean {
boolean cancel(IgniteUuid xid);
}
interface ContinuousQueryMXBean {
boolean cancel(UUID id);
}
```
[1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=145724615
> 17 янв. 2020 г., в 22:43, Николай Ижиков <ni...@gmail.com> написал(а):
>
> Hello, Andrey.
>
> Thanks for positive feedback.
> Appreciate it.
>
>> we can't cancel service or service's method
>
> I understand it.
> Seems, we have several obvious options here:
>
> * Try to do it and if fail answer to the user: «I’ve tried and fail. Sorry»
> * Kill hanging thread.
>
>> This invocations not register anywhere and can't be tracked and killed.
>
> Yes.
> So we should invent tracking for those invocation to be able to kill it :)
>
>
>> 17 янв. 2020 г., в 21:03, Andrey Gura <ag...@apache.org> написал(а):
>>
>> It would be grate.
>>
>> Only one comment: we can't cancel service or service's method
>> execution because service grid API doesn't imply any requirement to
>> service implementation. So if user do something like infinite cycle or
>> blocking but non-interruptible operation it's impossible to interrupt
>> it. Also service method invocation itself isn't cluster wide or local
>> node specific operation. This invocations not register anywhere and
>> can't be tracked and killed.
>>
>> Unfortunately, the same is valid for some jobs in sense of infinite or
>> non-interruptible operations.
>>
>> On Wed, Jan 15, 2020 at 9:30 PM Николай Ижиков <ni...@apache.org> wrote:
>>>
>>> Hello, Igniters.
>>>
>>> As you may know, we put a lot of effort to improve Ignite metric and diagnostic API.
>>> We have created the following API:
>>> * Metric manager
>>> * System view manager
>>> As far as I know, we would have tracing capabilities soon.
>>>
>>> I think it's time to take the next step.
>>> We should provide to the administrator the API to cancel(stop, kill) arbitrary user started computation.
>>>
>>> Right now we have API to do it for:
>>> * transactions `TransactionsMXBean#getActiveTransactions`.
>>> * SQL queries: `KILL QUERY` sql command and visor task - `VisorQueryCancelTask`
>>>
>>> Please, note, these features are implemented via different API.
>>>
>>> I think we should introduce uniform Cancel API for the following computations:
>>>
>>> * service.
>>> * specific service method execution.
>>> * compute job(compute task).
>>> * query(scan, continuous, text).
>>>
>>> We should also rework kill transaction and kill SQL queries API to make them similar to each other.
>>> I have plans to write an IEP for it and implement it.
>>> What do you think?
>>>
>
Re: [DISCUSSION] API to KILL any user started computation
Posted by Николай Ижиков <ni...@apache.org>.
Hello, Andrey.
Thanks for positive feedback.
Appreciate it.
> we can't cancel service or service's method
I understand it.
Seems, we have several obvious options here:
* Try to do it and if fail answer to the user: «I’ve tried and fail. Sorry»
* Kill hanging thread.
> This invocations not register anywhere and can't be tracked and killed.
Yes.
So we should invent tracking for those invocation to be able to kill it :)
> 17 янв. 2020 г., в 21:03, Andrey Gura <ag...@apache.org> написал(а):
>
> It would be grate.
>
> Only one comment: we can't cancel service or service's method
> execution because service grid API doesn't imply any requirement to
> service implementation. So if user do something like infinite cycle or
> blocking but non-interruptible operation it's impossible to interrupt
> it. Also service method invocation itself isn't cluster wide or local
> node specific operation. This invocations not register anywhere and
> can't be tracked and killed.
>
> Unfortunately, the same is valid for some jobs in sense of infinite or
> non-interruptible operations.
>
> On Wed, Jan 15, 2020 at 9:30 PM Николай Ижиков <ni...@apache.org> wrote:
>>
>> Hello, Igniters.
>>
>> As you may know, we put a lot of effort to improve Ignite metric and diagnostic API.
>> We have created the following API:
>> * Metric manager
>> * System view manager
>> As far as I know, we would have tracing capabilities soon.
>>
>> I think it's time to take the next step.
>> We should provide to the administrator the API to cancel(stop, kill) arbitrary user started computation.
>>
>> Right now we have API to do it for:
>> * transactions `TransactionsMXBean#getActiveTransactions`.
>> * SQL queries: `KILL QUERY` sql command and visor task - `VisorQueryCancelTask`
>>
>> Please, note, these features are implemented via different API.
>>
>> I think we should introduce uniform Cancel API for the following computations:
>>
>> * service.
>> * specific service method execution.
>> * compute job(compute task).
>> * query(scan, continuous, text).
>>
>> We should also rework kill transaction and kill SQL queries API to make them similar to each other.
>> I have plans to write an IEP for it and implement it.
>> What do you think?
>>
Re: [DISCUSSION] API to KILL any user started computation
Posted by Andrey Gura <ag...@apache.org>.
It would be grate.
Only one comment: we can't cancel service or service's method
execution because service grid API doesn't imply any requirement to
service implementation. So if user do something like infinite cycle or
blocking but non-interruptible operation it's impossible to interrupt
it. Also service method invocation itself isn't cluster wide or local
node specific operation. This invocations not register anywhere and
can't be tracked and killed.
Unfortunately, the same is valid for some jobs in sense of infinite or
non-interruptible operations.
On Wed, Jan 15, 2020 at 9:30 PM Николай Ижиков <ni...@apache.org> wrote:
>
> Hello, Igniters.
>
> As you may know, we put a lot of effort to improve Ignite metric and diagnostic API.
> We have created the following API:
> * Metric manager
> * System view manager
> As far as I know, we would have tracing capabilities soon.
>
> I think it's time to take the next step.
> We should provide to the administrator the API to cancel(stop, kill) arbitrary user started computation.
>
> Right now we have API to do it for:
> * transactions `TransactionsMXBean#getActiveTransactions`.
> * SQL queries: `KILL QUERY` sql command and visor task - `VisorQueryCancelTask`
>
> Please, note, these features are implemented via different API.
>
> I think we should introduce uniform Cancel API for the following computations:
>
> * service.
> * specific service method execution.
> * compute job(compute task).
> * query(scan, continuous, text).
>
> We should also rework kill transaction and kill SQL queries API to make them similar to each other.
> I have plans to write an IEP for it and implement it.
> What do you think?
>
Re: [DISCUSSION] API to KILL any user started computation
Posted by Alexei Scherbakov <al...@gmail.com>.
Sounds good.
JMX API should be very similar: accept operation type and ID.
чт, 16 янв. 2020 г. в 15:46, Николай Ижиков <ni...@apache.org>:
> Alexey.
>
> I think, yes.
> We certainly should be able to use system view data for the new KILL API.
>
> I think we should support both SQL and Java(JMX) API for this KILL command.
>
>
> > 16 янв. 2020 г., в 15:28, Alexei Scherbakov <
> alexey.scherbakoff@gmail.com> написал(а):
> >
> > Nikolaj,
> >
> > Can we use system views instead of implementing something new ?
> >
> > Each user operation has an unique ID.
> >
> > It's possible to introduce universal SQL kill something like:
> >
> > kill transaction ID
> >
> > where id is taken from system view.
> >
> >
> > чт, 16 янв. 2020 г. в 14:19, Николай Ижиков <ni...@apache.org>:
> >
> >> Hello, Alexey.
> >>
> >> I’m talking about *administrator* API.
> >>
> >> For example, the User has a cluster that is used by several
> applications.
> >> Some application starts buggy compute tasks that consume all CPU
> resources.
> >> Right now, administrator doesn’t have the ability to kill this task.
> >>
> >> This can lead to instability of the whole cluster.
> >>
> >>
> >>> 16 янв. 2020 г., в 13:42, Alexei Scherbakov <
> >> alexey.scherbakoff@gmail.com> написал(а):
> >>>
> >>> Transactions can be also rolled back using tx.close where close is
> >>> java.lang.AutoCloseable#close
> >>> It looks for me to the definition of uniform cancel API.
> >>>
> >>>
> >>>
> >>> чт, 16 янв. 2020 г. в 13:37, Alexei Scherbakov <
> >> alexey.scherbakoff@gmail.com
> >>>> :
> >>>
> >>>> Nikolaj,
> >>>>
> >>>> We already have cancellation possibilities for almost every user
> >>>> computation.
> >>>> Transactions are cancelled using tx.rollback()
> >>>> Queries are cancelled using query.close()
> >>>> Task is cancellable through ComputeTaskSession
> >>>>
> >>>> What is uniform cancel API ? Why do we need it ?
> >>>>
> >>>>
> >>>>
> >>>> ср, 15 янв. 2020 г. в 21:30, Николай Ижиков <ni...@apache.org>:
> >>>>
> >>>>> Hello, Igniters.
> >>>>>
> >>>>> As you may know, we put a lot of effort to improve Ignite metric and
> >>>>> diagnostic API.
> >>>>> We have created the following API:
> >>>>> * Metric manager
> >>>>> * System view manager
> >>>>> As far as I know, we would have tracing capabilities soon.
> >>>>>
> >>>>> I think it's time to take the next step.
> >>>>> We should provide to the administrator the API to cancel(stop, kill)
> >>>>> arbitrary user started computation.
> >>>>>
> >>>>> Right now we have API to do it for:
> >>>>> * transactions `TransactionsMXBean#getActiveTransactions`.
> >>>>> * SQL queries: `KILL QUERY` sql command and visor task -
> >>>>> `VisorQueryCancelTask`
> >>>>>
> >>>>> Please, note, these features are implemented via different API.
> >>>>>
> >>>>> I think we should introduce uniform Cancel API for the following
> >>>>> computations:
> >>>>>
> >>>>> * service.
> >>>>> * specific service method execution.
> >>>>> * compute job(compute task).
> >>>>> * query(scan, continuous, text).
> >>>>>
> >>>>> We should also rework kill transaction and kill SQL queries API to
> >> make
> >>>>> them similar to each other.
> >>>>> I have plans to write an IEP for it and implement it.
> >>>>> What do you think?
> >>>>>
> >>>>>
> >>>>
> >>>> --
> >>>>
> >>>> Best regards,
> >>>> Alexei Scherbakov
> >>>>
> >>>
> >>>
> >>> --
> >>>
> >>> Best regards,
> >>> Alexei Scherbakov
> >>
> >>
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
>
>
--
Best regards,
Alexei Scherbakov
Re: [DISCUSSION] API to KILL any user started computation
Posted by Вячеслав Коптилин <sl...@gmail.com>.
Hi Alexei,
> Not sure I've understand a question. JMX supports security via password
protection and secure channel.
That is the obvious thing I missed! :) Thank you!
Thanks,
S.
пт, 17 янв. 2020 г. в 14:45, Alexei Scherbakov <alexey.scherbakoff@gmail.com
>:
> Slava,
>
> Not sure I've understand a question. JMX supports security via password
> protection and secure channel.
>
> пт, 17 янв. 2020 г. в 14:30, Вячеслав Коптилин <sl...@gmail.com>:
>
> > Hello Nikolay,
> >
> > I'm not sure we need JMX API for this case. If I’m not mistaken, it’s
> about
> > the administrator’s ability to cancel any task/query in the cluster, and
> in
> > my opinion, such action must require administrator permission.
> > Could you please clarify how it can be done via JMX? I mean permission
> > check and etc. Perhaps, I'm missing something obvious...
> >
> > Thanks,
> > S.
> >
> > чт, 16 янв. 2020 г. в 15:46, Николай Ижиков <ni...@apache.org>:
> >
> > > Alexey.
> > >
> > > I think, yes.
> > > We certainly should be able to use system view data for the new KILL
> API.
> > >
> > > I think we should support both SQL and Java(JMX) API for this KILL
> > command.
> > >
> > >
> > > > 16 янв. 2020 г., в 15:28, Alexei Scherbakov <
> > > alexey.scherbakoff@gmail.com> написал(а):
> > > >
> > > > Nikolaj,
> > > >
> > > > Can we use system views instead of implementing something new ?
> > > >
> > > > Each user operation has an unique ID.
> > > >
> > > > It's possible to introduce universal SQL kill something like:
> > > >
> > > > kill transaction ID
> > > >
> > > > where id is taken from system view.
> > > >
> > > >
> > > > чт, 16 янв. 2020 г. в 14:19, Николай Ижиков <ni...@apache.org>:
> > > >
> > > >> Hello, Alexey.
> > > >>
> > > >> I’m talking about *administrator* API.
> > > >>
> > > >> For example, the User has a cluster that is used by several
> > > applications.
> > > >> Some application starts buggy compute tasks that consume all CPU
> > > resources.
> > > >> Right now, administrator doesn’t have the ability to kill this task.
> > > >>
> > > >> This can lead to instability of the whole cluster.
> > > >>
> > > >>
> > > >>> 16 янв. 2020 г., в 13:42, Alexei Scherbakov <
> > > >> alexey.scherbakoff@gmail.com> написал(а):
> > > >>>
> > > >>> Transactions can be also rolled back using tx.close where close is
> > > >>> java.lang.AutoCloseable#close
> > > >>> It looks for me to the definition of uniform cancel API.
> > > >>>
> > > >>>
> > > >>>
> > > >>> чт, 16 янв. 2020 г. в 13:37, Alexei Scherbakov <
> > > >> alexey.scherbakoff@gmail.com
> > > >>>> :
> > > >>>
> > > >>>> Nikolaj,
> > > >>>>
> > > >>>> We already have cancellation possibilities for almost every user
> > > >>>> computation.
> > > >>>> Transactions are cancelled using tx.rollback()
> > > >>>> Queries are cancelled using query.close()
> > > >>>> Task is cancellable through ComputeTaskSession
> > > >>>>
> > > >>>> What is uniform cancel API ? Why do we need it ?
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> ср, 15 янв. 2020 г. в 21:30, Николай Ижиков <nizhikov@apache.org
> >:
> > > >>>>
> > > >>>>> Hello, Igniters.
> > > >>>>>
> > > >>>>> As you may know, we put a lot of effort to improve Ignite metric
> > and
> > > >>>>> diagnostic API.
> > > >>>>> We have created the following API:
> > > >>>>> * Metric manager
> > > >>>>> * System view manager
> > > >>>>> As far as I know, we would have tracing capabilities soon.
> > > >>>>>
> > > >>>>> I think it's time to take the next step.
> > > >>>>> We should provide to the administrator the API to cancel(stop,
> > kill)
> > > >>>>> arbitrary user started computation.
> > > >>>>>
> > > >>>>> Right now we have API to do it for:
> > > >>>>> * transactions `TransactionsMXBean#getActiveTransactions`.
> > > >>>>> * SQL queries: `KILL QUERY` sql command and visor task -
> > > >>>>> `VisorQueryCancelTask`
> > > >>>>>
> > > >>>>> Please, note, these features are implemented via different API.
> > > >>>>>
> > > >>>>> I think we should introduce uniform Cancel API for the following
> > > >>>>> computations:
> > > >>>>>
> > > >>>>> * service.
> > > >>>>> * specific service method execution.
> > > >>>>> * compute job(compute task).
> > > >>>>> * query(scan, continuous, text).
> > > >>>>>
> > > >>>>> We should also rework kill transaction and kill SQL queries API
> to
> > > >> make
> > > >>>>> them similar to each other.
> > > >>>>> I have plans to write an IEP for it and implement it.
> > > >>>>> What do you think?
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>>> --
> > > >>>>
> > > >>>> Best regards,
> > > >>>> Alexei Scherbakov
> > > >>>>
> > > >>>
> > > >>>
> > > >>> --
> > > >>>
> > > >>> Best regards,
> > > >>> Alexei Scherbakov
> > > >>
> > > >>
> > > >
> > > > --
> > > >
> > > > Best regards,
> > > > Alexei Scherbakov
> > >
> > >
> >
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>
Re: [DISCUSSION] API to KILL any user started computation
Posted by Alexei Scherbakov <al...@gmail.com>.
Slava,
Not sure I've understand a question. JMX supports security via password
protection and secure channel.
пт, 17 янв. 2020 г. в 14:30, Вячеслав Коптилин <sl...@gmail.com>:
> Hello Nikolay,
>
> I'm not sure we need JMX API for this case. If I’m not mistaken, it’s about
> the administrator’s ability to cancel any task/query in the cluster, and in
> my opinion, such action must require administrator permission.
> Could you please clarify how it can be done via JMX? I mean permission
> check and etc. Perhaps, I'm missing something obvious...
>
> Thanks,
> S.
>
> чт, 16 янв. 2020 г. в 15:46, Николай Ижиков <ni...@apache.org>:
>
> > Alexey.
> >
> > I think, yes.
> > We certainly should be able to use system view data for the new KILL API.
> >
> > I think we should support both SQL and Java(JMX) API for this KILL
> command.
> >
> >
> > > 16 янв. 2020 г., в 15:28, Alexei Scherbakov <
> > alexey.scherbakoff@gmail.com> написал(а):
> > >
> > > Nikolaj,
> > >
> > > Can we use system views instead of implementing something new ?
> > >
> > > Each user operation has an unique ID.
> > >
> > > It's possible to introduce universal SQL kill something like:
> > >
> > > kill transaction ID
> > >
> > > where id is taken from system view.
> > >
> > >
> > > чт, 16 янв. 2020 г. в 14:19, Николай Ижиков <ni...@apache.org>:
> > >
> > >> Hello, Alexey.
> > >>
> > >> I’m talking about *administrator* API.
> > >>
> > >> For example, the User has a cluster that is used by several
> > applications.
> > >> Some application starts buggy compute tasks that consume all CPU
> > resources.
> > >> Right now, administrator doesn’t have the ability to kill this task.
> > >>
> > >> This can lead to instability of the whole cluster.
> > >>
> > >>
> > >>> 16 янв. 2020 г., в 13:42, Alexei Scherbakov <
> > >> alexey.scherbakoff@gmail.com> написал(а):
> > >>>
> > >>> Transactions can be also rolled back using tx.close where close is
> > >>> java.lang.AutoCloseable#close
> > >>> It looks for me to the definition of uniform cancel API.
> > >>>
> > >>>
> > >>>
> > >>> чт, 16 янв. 2020 г. в 13:37, Alexei Scherbakov <
> > >> alexey.scherbakoff@gmail.com
> > >>>> :
> > >>>
> > >>>> Nikolaj,
> > >>>>
> > >>>> We already have cancellation possibilities for almost every user
> > >>>> computation.
> > >>>> Transactions are cancelled using tx.rollback()
> > >>>> Queries are cancelled using query.close()
> > >>>> Task is cancellable through ComputeTaskSession
> > >>>>
> > >>>> What is uniform cancel API ? Why do we need it ?
> > >>>>
> > >>>>
> > >>>>
> > >>>> ср, 15 янв. 2020 г. в 21:30, Николай Ижиков <ni...@apache.org>:
> > >>>>
> > >>>>> Hello, Igniters.
> > >>>>>
> > >>>>> As you may know, we put a lot of effort to improve Ignite metric
> and
> > >>>>> diagnostic API.
> > >>>>> We have created the following API:
> > >>>>> * Metric manager
> > >>>>> * System view manager
> > >>>>> As far as I know, we would have tracing capabilities soon.
> > >>>>>
> > >>>>> I think it's time to take the next step.
> > >>>>> We should provide to the administrator the API to cancel(stop,
> kill)
> > >>>>> arbitrary user started computation.
> > >>>>>
> > >>>>> Right now we have API to do it for:
> > >>>>> * transactions `TransactionsMXBean#getActiveTransactions`.
> > >>>>> * SQL queries: `KILL QUERY` sql command and visor task -
> > >>>>> `VisorQueryCancelTask`
> > >>>>>
> > >>>>> Please, note, these features are implemented via different API.
> > >>>>>
> > >>>>> I think we should introduce uniform Cancel API for the following
> > >>>>> computations:
> > >>>>>
> > >>>>> * service.
> > >>>>> * specific service method execution.
> > >>>>> * compute job(compute task).
> > >>>>> * query(scan, continuous, text).
> > >>>>>
> > >>>>> We should also rework kill transaction and kill SQL queries API to
> > >> make
> > >>>>> them similar to each other.
> > >>>>> I have plans to write an IEP for it and implement it.
> > >>>>> What do you think?
> > >>>>>
> > >>>>>
> > >>>>
> > >>>> --
> > >>>>
> > >>>> Best regards,
> > >>>> Alexei Scherbakov
> > >>>>
> > >>>
> > >>>
> > >>> --
> > >>>
> > >>> Best regards,
> > >>> Alexei Scherbakov
> > >>
> > >>
> > >
> > > --
> > >
> > > Best regards,
> > > Alexei Scherbakov
> >
> >
>
--
Best regards,
Alexei Scherbakov
Re: [DISCUSSION] API to KILL any user started computation
Posted by Вячеслав Коптилин <sl...@gmail.com>.
Hello Nikolay,
I'm not sure we need JMX API for this case. If I’m not mistaken, it’s about
the administrator’s ability to cancel any task/query in the cluster, and in
my opinion, such action must require administrator permission.
Could you please clarify how it can be done via JMX? I mean permission
check and etc. Perhaps, I'm missing something obvious...
Thanks,
S.
чт, 16 янв. 2020 г. в 15:46, Николай Ижиков <ni...@apache.org>:
> Alexey.
>
> I think, yes.
> We certainly should be able to use system view data for the new KILL API.
>
> I think we should support both SQL and Java(JMX) API for this KILL command.
>
>
> > 16 янв. 2020 г., в 15:28, Alexei Scherbakov <
> alexey.scherbakoff@gmail.com> написал(а):
> >
> > Nikolaj,
> >
> > Can we use system views instead of implementing something new ?
> >
> > Each user operation has an unique ID.
> >
> > It's possible to introduce universal SQL kill something like:
> >
> > kill transaction ID
> >
> > where id is taken from system view.
> >
> >
> > чт, 16 янв. 2020 г. в 14:19, Николай Ижиков <ni...@apache.org>:
> >
> >> Hello, Alexey.
> >>
> >> I’m talking about *administrator* API.
> >>
> >> For example, the User has a cluster that is used by several
> applications.
> >> Some application starts buggy compute tasks that consume all CPU
> resources.
> >> Right now, administrator doesn’t have the ability to kill this task.
> >>
> >> This can lead to instability of the whole cluster.
> >>
> >>
> >>> 16 янв. 2020 г., в 13:42, Alexei Scherbakov <
> >> alexey.scherbakoff@gmail.com> написал(а):
> >>>
> >>> Transactions can be also rolled back using tx.close where close is
> >>> java.lang.AutoCloseable#close
> >>> It looks for me to the definition of uniform cancel API.
> >>>
> >>>
> >>>
> >>> чт, 16 янв. 2020 г. в 13:37, Alexei Scherbakov <
> >> alexey.scherbakoff@gmail.com
> >>>> :
> >>>
> >>>> Nikolaj,
> >>>>
> >>>> We already have cancellation possibilities for almost every user
> >>>> computation.
> >>>> Transactions are cancelled using tx.rollback()
> >>>> Queries are cancelled using query.close()
> >>>> Task is cancellable through ComputeTaskSession
> >>>>
> >>>> What is uniform cancel API ? Why do we need it ?
> >>>>
> >>>>
> >>>>
> >>>> ср, 15 янв. 2020 г. в 21:30, Николай Ижиков <ni...@apache.org>:
> >>>>
> >>>>> Hello, Igniters.
> >>>>>
> >>>>> As you may know, we put a lot of effort to improve Ignite metric and
> >>>>> diagnostic API.
> >>>>> We have created the following API:
> >>>>> * Metric manager
> >>>>> * System view manager
> >>>>> As far as I know, we would have tracing capabilities soon.
> >>>>>
> >>>>> I think it's time to take the next step.
> >>>>> We should provide to the administrator the API to cancel(stop, kill)
> >>>>> arbitrary user started computation.
> >>>>>
> >>>>> Right now we have API to do it for:
> >>>>> * transactions `TransactionsMXBean#getActiveTransactions`.
> >>>>> * SQL queries: `KILL QUERY` sql command and visor task -
> >>>>> `VisorQueryCancelTask`
> >>>>>
> >>>>> Please, note, these features are implemented via different API.
> >>>>>
> >>>>> I think we should introduce uniform Cancel API for the following
> >>>>> computations:
> >>>>>
> >>>>> * service.
> >>>>> * specific service method execution.
> >>>>> * compute job(compute task).
> >>>>> * query(scan, continuous, text).
> >>>>>
> >>>>> We should also rework kill transaction and kill SQL queries API to
> >> make
> >>>>> them similar to each other.
> >>>>> I have plans to write an IEP for it and implement it.
> >>>>> What do you think?
> >>>>>
> >>>>>
> >>>>
> >>>> --
> >>>>
> >>>> Best regards,
> >>>> Alexei Scherbakov
> >>>>
> >>>
> >>>
> >>> --
> >>>
> >>> Best regards,
> >>> Alexei Scherbakov
> >>
> >>
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
>
>
Re: [DISCUSSION] API to KILL any user started computation
Posted by Николай Ижиков <ni...@apache.org>.
Alexey.
I think, yes.
We certainly should be able to use system view data for the new KILL API.
I think we should support both SQL and Java(JMX) API for this KILL command.
> 16 янв. 2020 г., в 15:28, Alexei Scherbakov <al...@gmail.com> написал(а):
>
> Nikolaj,
>
> Can we use system views instead of implementing something new ?
>
> Each user operation has an unique ID.
>
> It's possible to introduce universal SQL kill something like:
>
> kill transaction ID
>
> where id is taken from system view.
>
>
> чт, 16 янв. 2020 г. в 14:19, Николай Ижиков <ni...@apache.org>:
>
>> Hello, Alexey.
>>
>> I’m talking about *administrator* API.
>>
>> For example, the User has a cluster that is used by several applications.
>> Some application starts buggy compute tasks that consume all CPU resources.
>> Right now, administrator doesn’t have the ability to kill this task.
>>
>> This can lead to instability of the whole cluster.
>>
>>
>>> 16 янв. 2020 г., в 13:42, Alexei Scherbakov <
>> alexey.scherbakoff@gmail.com> написал(а):
>>>
>>> Transactions can be also rolled back using tx.close where close is
>>> java.lang.AutoCloseable#close
>>> It looks for me to the definition of uniform cancel API.
>>>
>>>
>>>
>>> чт, 16 янв. 2020 г. в 13:37, Alexei Scherbakov <
>> alexey.scherbakoff@gmail.com
>>>> :
>>>
>>>> Nikolaj,
>>>>
>>>> We already have cancellation possibilities for almost every user
>>>> computation.
>>>> Transactions are cancelled using tx.rollback()
>>>> Queries are cancelled using query.close()
>>>> Task is cancellable through ComputeTaskSession
>>>>
>>>> What is uniform cancel API ? Why do we need it ?
>>>>
>>>>
>>>>
>>>> ср, 15 янв. 2020 г. в 21:30, Николай Ижиков <ni...@apache.org>:
>>>>
>>>>> Hello, Igniters.
>>>>>
>>>>> As you may know, we put a lot of effort to improve Ignite metric and
>>>>> diagnostic API.
>>>>> We have created the following API:
>>>>> * Metric manager
>>>>> * System view manager
>>>>> As far as I know, we would have tracing capabilities soon.
>>>>>
>>>>> I think it's time to take the next step.
>>>>> We should provide to the administrator the API to cancel(stop, kill)
>>>>> arbitrary user started computation.
>>>>>
>>>>> Right now we have API to do it for:
>>>>> * transactions `TransactionsMXBean#getActiveTransactions`.
>>>>> * SQL queries: `KILL QUERY` sql command and visor task -
>>>>> `VisorQueryCancelTask`
>>>>>
>>>>> Please, note, these features are implemented via different API.
>>>>>
>>>>> I think we should introduce uniform Cancel API for the following
>>>>> computations:
>>>>>
>>>>> * service.
>>>>> * specific service method execution.
>>>>> * compute job(compute task).
>>>>> * query(scan, continuous, text).
>>>>>
>>>>> We should also rework kill transaction and kill SQL queries API to
>> make
>>>>> them similar to each other.
>>>>> I have plans to write an IEP for it and implement it.
>>>>> What do you think?
>>>>>
>>>>>
>>>>
>>>> --
>>>>
>>>> Best regards,
>>>> Alexei Scherbakov
>>>>
>>>
>>>
>>> --
>>>
>>> Best regards,
>>> Alexei Scherbakov
>>
>>
>
> --
>
> Best regards,
> Alexei Scherbakov
Re: [DISCUSSION] API to KILL any user started computation
Posted by Alexei Scherbakov <al...@gmail.com>.
Nikolaj,
Can we use system views instead of implementing something new ?
Each user operation has an unique ID.
It's possible to introduce universal SQL kill something like:
kill transaction ID
where id is taken from system view.
чт, 16 янв. 2020 г. в 14:19, Николай Ижиков <ni...@apache.org>:
> Hello, Alexey.
>
> I’m talking about *administrator* API.
>
> For example, the User has a cluster that is used by several applications.
> Some application starts buggy compute tasks that consume all CPU resources.
> Right now, administrator doesn’t have the ability to kill this task.
>
> This can lead to instability of the whole cluster.
>
>
> > 16 янв. 2020 г., в 13:42, Alexei Scherbakov <
> alexey.scherbakoff@gmail.com> написал(а):
> >
> > Transactions can be also rolled back using tx.close where close is
> > java.lang.AutoCloseable#close
> > It looks for me to the definition of uniform cancel API.
> >
> >
> >
> > чт, 16 янв. 2020 г. в 13:37, Alexei Scherbakov <
> alexey.scherbakoff@gmail.com
> >> :
> >
> >> Nikolaj,
> >>
> >> We already have cancellation possibilities for almost every user
> >> computation.
> >> Transactions are cancelled using tx.rollback()
> >> Queries are cancelled using query.close()
> >> Task is cancellable through ComputeTaskSession
> >>
> >> What is uniform cancel API ? Why do we need it ?
> >>
> >>
> >>
> >> ср, 15 янв. 2020 г. в 21:30, Николай Ижиков <ni...@apache.org>:
> >>
> >>> Hello, Igniters.
> >>>
> >>> As you may know, we put a lot of effort to improve Ignite metric and
> >>> diagnostic API.
> >>> We have created the following API:
> >>> * Metric manager
> >>> * System view manager
> >>> As far as I know, we would have tracing capabilities soon.
> >>>
> >>> I think it's time to take the next step.
> >>> We should provide to the administrator the API to cancel(stop, kill)
> >>> arbitrary user started computation.
> >>>
> >>> Right now we have API to do it for:
> >>> * transactions `TransactionsMXBean#getActiveTransactions`.
> >>> * SQL queries: `KILL QUERY` sql command and visor task -
> >>> `VisorQueryCancelTask`
> >>>
> >>> Please, note, these features are implemented via different API.
> >>>
> >>> I think we should introduce uniform Cancel API for the following
> >>> computations:
> >>>
> >>> * service.
> >>> * specific service method execution.
> >>> * compute job(compute task).
> >>> * query(scan, continuous, text).
> >>>
> >>> We should also rework kill transaction and kill SQL queries API to
> make
> >>> them similar to each other.
> >>> I have plans to write an IEP for it and implement it.
> >>> What do you think?
> >>>
> >>>
> >>
> >> --
> >>
> >> Best regards,
> >> Alexei Scherbakov
> >>
> >
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
>
>
--
Best regards,
Alexei Scherbakov
Re: [DISCUSSION] API to KILL any user started computation
Posted by Николай Ижиков <ni...@apache.org>.
Hello, Alexey.
I’m talking about *administrator* API.
For example, the User has a cluster that is used by several applications.
Some application starts buggy compute tasks that consume all CPU resources.
Right now, administrator doesn’t have the ability to kill this task.
This can lead to instability of the whole cluster.
> 16 янв. 2020 г., в 13:42, Alexei Scherbakov <al...@gmail.com> написал(а):
>
> Transactions can be also rolled back using tx.close where close is
> java.lang.AutoCloseable#close
> It looks for me to the definition of uniform cancel API.
>
>
>
> чт, 16 янв. 2020 г. в 13:37, Alexei Scherbakov <alexey.scherbakoff@gmail.com
>> :
>
>> Nikolaj,
>>
>> We already have cancellation possibilities for almost every user
>> computation.
>> Transactions are cancelled using tx.rollback()
>> Queries are cancelled using query.close()
>> Task is cancellable through ComputeTaskSession
>>
>> What is uniform cancel API ? Why do we need it ?
>>
>>
>>
>> ср, 15 янв. 2020 г. в 21:30, Николай Ижиков <ni...@apache.org>:
>>
>>> Hello, Igniters.
>>>
>>> As you may know, we put a lot of effort to improve Ignite metric and
>>> diagnostic API.
>>> We have created the following API:
>>> * Metric manager
>>> * System view manager
>>> As far as I know, we would have tracing capabilities soon.
>>>
>>> I think it's time to take the next step.
>>> We should provide to the administrator the API to cancel(stop, kill)
>>> arbitrary user started computation.
>>>
>>> Right now we have API to do it for:
>>> * transactions `TransactionsMXBean#getActiveTransactions`.
>>> * SQL queries: `KILL QUERY` sql command and visor task -
>>> `VisorQueryCancelTask`
>>>
>>> Please, note, these features are implemented via different API.
>>>
>>> I think we should introduce uniform Cancel API for the following
>>> computations:
>>>
>>> * service.
>>> * specific service method execution.
>>> * compute job(compute task).
>>> * query(scan, continuous, text).
>>>
>>> We should also rework kill transaction and kill SQL queries API to make
>>> them similar to each other.
>>> I have plans to write an IEP for it and implement it.
>>> What do you think?
>>>
>>>
>>
>> --
>>
>> Best regards,
>> Alexei Scherbakov
>>
>
>
> --
>
> Best regards,
> Alexei Scherbakov
Re: [DISCUSSION] API to KILL any user started computation
Posted by Alexei Scherbakov <al...@gmail.com>.
Transactions can be also rolled back using tx.close where close is
java.lang.AutoCloseable#close
It looks for me to the definition of uniform cancel API.
чт, 16 янв. 2020 г. в 13:37, Alexei Scherbakov <alexey.scherbakoff@gmail.com
>:
> Nikolaj,
>
> We already have cancellation possibilities for almost every user
> computation.
> Transactions are cancelled using tx.rollback()
> Queries are cancelled using query.close()
> Task is cancellable through ComputeTaskSession
>
> What is uniform cancel API ? Why do we need it ?
>
>
>
> ср, 15 янв. 2020 г. в 21:30, Николай Ижиков <ni...@apache.org>:
>
>> Hello, Igniters.
>>
>> As you may know, we put a lot of effort to improve Ignite metric and
>> diagnostic API.
>> We have created the following API:
>> * Metric manager
>> * System view manager
>> As far as I know, we would have tracing capabilities soon.
>>
>> I think it's time to take the next step.
>> We should provide to the administrator the API to cancel(stop, kill)
>> arbitrary user started computation.
>>
>> Right now we have API to do it for:
>> * transactions `TransactionsMXBean#getActiveTransactions`.
>> * SQL queries: `KILL QUERY` sql command and visor task -
>> `VisorQueryCancelTask`
>>
>> Please, note, these features are implemented via different API.
>>
>> I think we should introduce uniform Cancel API for the following
>> computations:
>>
>> * service.
>> * specific service method execution.
>> * compute job(compute task).
>> * query(scan, continuous, text).
>>
>> We should also rework kill transaction and kill SQL queries API to make
>> them similar to each other.
>> I have plans to write an IEP for it and implement it.
>> What do you think?
>>
>>
>
> --
>
> Best regards,
> Alexei Scherbakov
>
--
Best regards,
Alexei Scherbakov
Re: [DISCUSSION] API to KILL any user started computation
Posted by Alexei Scherbakov <al...@gmail.com>.
Nikolaj,
We already have cancellation possibilities for almost every user
computation.
Transactions are cancelled using tx.rollback()
Queries are cancelled using query.close()
Task is cancellable through ComputeTaskSession
What is uniform cancel API ? Why do we need it ?
ср, 15 янв. 2020 г. в 21:30, Николай Ижиков <ni...@apache.org>:
> Hello, Igniters.
>
> As you may know, we put a lot of effort to improve Ignite metric and
> diagnostic API.
> We have created the following API:
> * Metric manager
> * System view manager
> As far as I know, we would have tracing capabilities soon.
>
> I think it's time to take the next step.
> We should provide to the administrator the API to cancel(stop, kill)
> arbitrary user started computation.
>
> Right now we have API to do it for:
> * transactions `TransactionsMXBean#getActiveTransactions`.
> * SQL queries: `KILL QUERY` sql command and visor task -
> `VisorQueryCancelTask`
>
> Please, note, these features are implemented via different API.
>
> I think we should introduce uniform Cancel API for the following
> computations:
>
> * service.
> * specific service method execution.
> * compute job(compute task).
> * query(scan, continuous, text).
>
> We should also rework kill transaction and kill SQL queries API to make
> them similar to each other.
> I have plans to write an IEP for it and implement it.
> What do you think?
>
>
--
Best regards,
Alexei Scherbakov