You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Sriveena Mattaparthi <Sr...@ekaplus.com> on 2018/08/28 14:23:58 UTC
Drop table taking more time for execution
Hi,
Execution of drop table command is taking 15 sec on client node.Any suggestions to spewed up the execution.
Thanks & Regards,
Sriveena "Confidentiality Notice: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited."
RE: Drop table taking more time for execution
Posted by Sriveena Mattaparthi <Sr...@ekaplus.com>.
Hi Denis,
We thought it would have the same performance impact as DROP TABLE. Hence we didn’t used it.
May be we will try that option and get back to you.
Thanks & Regards,
Sriveena
From: Denis Magda [mailto:dmagda@apache.org]
Sent: Thursday, August 30, 2018 11:06 AM
To: user@ignite.apache.org
Subject: Re: Drop table taking more time for execution
Next time if there is change in the data from the data source in terms of addition of new column or delete column...Then we drop the table in ignite and recreate it.
Is there any reason why you can't use ALTER TABLE command for the addition or remove of columns in runtime? It will be more efficient.
--
Denis
On Tue, Aug 28, 2018 at 9:49 AM Sriveena Mattaparthi <Sr...@ekaplus.com>> wrote:
Hi Ilya..
We load data from various data sources to ignite as tables and run data join queries on ignite and sends the response back.
Next time if there is change in the data from the data source in terms of addition of new column or delete column...Then we drop the table in ignite and recreate it.
Thanks,
Sriveena
________________________________
From: Ilya Kasnacheev <il...@gmail.com>>
Sent: 28 August 2018 20:21:20
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: Re: Drop table taking more time for execution
Hello!
First of all, what is the problem that you are facing? Why DROP TABLE taking 15s while being ran infrequently is a cause for concern?
Regards,
--
Ilya Kasnacheev
вт, 28 авг. 2018 г. в 17:42, Sriveena Mattaparthi <Sr...@ekaplus.com>>>:
Thanks Ilya for the quick reply..
We have to drop the table in few cases where the table columns and datatypes changes ...
I guess alter table to add/ drop columns may take same longer time.
Any other alternative to drop table?
Regards,
Sriveena
________________________________
From: Ilya Kasnacheev <il...@gmail.com>>>
Sent: 28 August 2018 20:04:09
To: user@ignite.apache.org<ma...@ignite.apache.org>>
Subject: Re: Drop table taking more time for execution
Hello!
As far as my understanding goes, DROP TABLE will usually destroy a cache, and that will require partition map exchange, which in turn will wait for all operations to finish. With increase of load and cluster size 15s does not seem excessive.
The suggestion is to avoid dropping/creating tables often, and especially as the course of normal operation.
Regards,
--
Ilya Kasnacheev
вт, 28 авг. 2018 г. в 17:24, Sriveena Mattaparthi <Sr...@ekaplus.com>>>:
Hi,
Execution of drop table command is taking 15 sec on client node.Any suggestions to spewed up the execution.
Thanks & Regards,
Sriveena “Confidentiality Notice: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.”
“Confidentiality Notice: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.”
“Confidentiality Notice: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.”
“Confidentiality Notice: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.”
Re: Drop table taking more time for execution
Posted by Denis Magda <dm...@apache.org>.
>
> Next time if there is change in the data from the data source in terms of
> addition of new column or delete column...Then we drop the table in ignite
> and recreate it.
Is there any reason why you can't use ALTER TABLE command for the addition
or remove of columns in runtime? It will be more efficient.
--
Denis
On Tue, Aug 28, 2018 at 9:49 AM Sriveena Mattaparthi <
Sriveena.Mattaparthi@ekaplus.com> wrote:
> Hi Ilya..
> We load data from various data sources to ignite as tables and run data
> join queries on ignite and sends the response back.
>
> Next time if there is change in the data from the data source in terms of
> addition of new column or delete column...Then we drop the table in ignite
> and recreate it.
>
> Thanks,
> Sriveena
> ________________________________
> From: Ilya Kasnacheev <il...@gmail.com>
> Sent: 28 August 2018 20:21:20
> To: user@ignite.apache.org
> Subject: Re: Drop table taking more time for execution
>
> Hello!
>
> First of all, what is the problem that you are facing? Why DROP TABLE
> taking 15s while being ran infrequently is a cause for concern?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 28 авг. 2018 г. в 17:42, Sriveena Mattaparthi <
> Sriveena.Mattaparthi@ekaplus.com<mailto:Sriveena.Mattaparthi@ekaplus.com
> >>:
> Thanks Ilya for the quick reply..
>
> We have to drop the table in few cases where the table columns and
> datatypes changes ...
> I guess alter table to add/ drop columns may take same longer time.
>
> Any other alternative to drop table?
>
> Regards,
> Sriveena
> ________________________________
> From: Ilya Kasnacheev <ilya.kasnacheev@gmail.com<mailto:
> ilya.kasnacheev@gmail.com>>
> Sent: 28 August 2018 20:04:09
> To: user@ignite.apache.org<ma...@ignite.apache.org>
> Subject: Re: Drop table taking more time for execution
>
> Hello!
>
> As far as my understanding goes, DROP TABLE will usually destroy a cache,
> and that will require partition map exchange, which in turn will wait for
> all operations to finish. With increase of load and cluster size 15s does
> not seem excessive.
>
> The suggestion is to avoid dropping/creating tables often, and especially
> as the course of normal operation.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 28 авг. 2018 г. в 17:24, Sriveena Mattaparthi <
> Sriveena.Mattaparthi@ekaplus.com<mailto:Sriveena.Mattaparthi@ekaplus.com
> >>:
> Hi,
>
> Execution of drop table command is taking 15 sec on client node.Any
> suggestions to spewed up the execution.
>
> Thanks & Regards,
> Sriveena “Confidentiality Notice: The contents of this email message and
> any attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
> “Confidentiality Notice: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
> “Confidentiality Notice: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>
Re: Drop table taking more time for execution
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
This use case doesn't sound very real-time, so I have to repeat my
question, why is it considered a problem by you?
--
Ilya Kasnacheev
вт, 28 авг. 2018 г. в 19:49, Sriveena Mattaparthi <
Sriveena.Mattaparthi@ekaplus.com>:
> Hi Ilya..
> We load data from various data sources to ignite as tables and run data
> join queries on ignite and sends the response back.
>
> Next time if there is change in the data from the data source in terms of
> addition of new column or delete column...Then we drop the table in ignite
> and recreate it.
>
> Thanks,
> Sriveena
> ________________________________
> From: Ilya Kasnacheev <il...@gmail.com>
> Sent: 28 August 2018 20:21:20
> To: user@ignite.apache.org
> Subject: Re: Drop table taking more time for execution
>
> Hello!
>
> First of all, what is the problem that you are facing? Why DROP TABLE
> taking 15s while being ran infrequently is a cause for concern?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 28 авг. 2018 г. в 17:42, Sriveena Mattaparthi <
> Sriveena.Mattaparthi@ekaplus.com<mailto:Sriveena.Mattaparthi@ekaplus.com
> >>:
> Thanks Ilya for the quick reply..
>
> We have to drop the table in few cases where the table columns and
> datatypes changes ...
> I guess alter table to add/ drop columns may take same longer time.
>
> Any other alternative to drop table?
>
> Regards,
> Sriveena
> ________________________________
> From: Ilya Kasnacheev <ilya.kasnacheev@gmail.com<mailto:
> ilya.kasnacheev@gmail.com>>
> Sent: 28 August 2018 20:04:09
> To: user@ignite.apache.org<ma...@ignite.apache.org>
> Subject: Re: Drop table taking more time for execution
>
> Hello!
>
> As far as my understanding goes, DROP TABLE will usually destroy a cache,
> and that will require partition map exchange, which in turn will wait for
> all operations to finish. With increase of load and cluster size 15s does
> not seem excessive.
>
> The suggestion is to avoid dropping/creating tables often, and especially
> as the course of normal operation.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 28 авг. 2018 г. в 17:24, Sriveena Mattaparthi <
> Sriveena.Mattaparthi@ekaplus.com<mailto:Sriveena.Mattaparthi@ekaplus.com
> >>:
> Hi,
>
> Execution of drop table command is taking 15 sec on client node.Any
> suggestions to spewed up the execution.
>
> Thanks & Regards,
> Sriveena “Confidentiality Notice: The contents of this email message and
> any attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
> “Confidentiality Notice: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
> “Confidentiality Notice: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>
Re: Drop table taking more time for execution
Posted by Sriveena Mattaparthi <Sr...@ekaplus.com>.
Hi Ilya..
We load data from various data sources to ignite as tables and run data join queries on ignite and sends the response back.
Next time if there is change in the data from the data source in terms of addition of new column or delete column...Then we drop the table in ignite and recreate it.
Thanks,
Sriveena
________________________________
From: Ilya Kasnacheev <il...@gmail.com>
Sent: 28 August 2018 20:21:20
To: user@ignite.apache.org
Subject: Re: Drop table taking more time for execution
Hello!
First of all, what is the problem that you are facing? Why DROP TABLE taking 15s while being ran infrequently is a cause for concern?
Regards,
--
Ilya Kasnacheev
вт, 28 авг. 2018 г. в 17:42, Sriveena Mattaparthi <Sr...@ekaplus.com>>:
Thanks Ilya for the quick reply..
We have to drop the table in few cases where the table columns and datatypes changes ...
I guess alter table to add/ drop columns may take same longer time.
Any other alternative to drop table?
Regards,
Sriveena
________________________________
From: Ilya Kasnacheev <il...@gmail.com>>
Sent: 28 August 2018 20:04:09
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: Re: Drop table taking more time for execution
Hello!
As far as my understanding goes, DROP TABLE will usually destroy a cache, and that will require partition map exchange, which in turn will wait for all operations to finish. With increase of load and cluster size 15s does not seem excessive.
The suggestion is to avoid dropping/creating tables often, and especially as the course of normal operation.
Regards,
--
Ilya Kasnacheev
вт, 28 авг. 2018 г. в 17:24, Sriveena Mattaparthi <Sr...@ekaplus.com>>:
Hi,
Execution of drop table command is taking 15 sec on client node.Any suggestions to spewed up the execution.
Thanks & Regards,
Sriveena “Confidentiality Notice: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.”
“Confidentiality Notice: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.”
“Confidentiality Notice: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.”
Re: Drop table taking more time for execution
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
First of all, what is the problem that you are facing? Why DROP TABLE
taking 15s while being ran infrequently is a cause for concern?
Regards,
--
Ilya Kasnacheev
вт, 28 авг. 2018 г. в 17:42, Sriveena Mattaparthi <
Sriveena.Mattaparthi@ekaplus.com>:
> Thanks Ilya for the quick reply..
>
> We have to drop the table in few cases where the table columns and
> datatypes changes ...
> I guess alter table to add/ drop columns may take same longer time.
>
> Any other alternative to drop table?
>
> Regards,
> Sriveena
> ------------------------------
> *From:* Ilya Kasnacheev <il...@gmail.com>
> *Sent:* 28 August 2018 20:04:09
> *To:* user@ignite.apache.org
> *Subject:* Re: Drop table taking more time for execution
>
> Hello!
>
> As far as my understanding goes, DROP TABLE will usually destroy a cache,
> and that will require partition map exchange, which in turn will wait for
> all operations to finish. With increase of load and cluster size 15s does
> not seem excessive.
>
> The suggestion is to avoid dropping/creating tables often, and especially
> as the course of normal operation.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 28 авг. 2018 г. в 17:24, Sriveena Mattaparthi <
> Sriveena.Mattaparthi@ekaplus.com>:
>
> Hi,
>
> Execution of drop table command is taking 15 sec on client node.Any
> suggestions to spewed up the execution.
>
> Thanks & Regards,
> Sriveena “Confidentiality Notice: The contents of this email message and
> any attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>
> “Confidentiality Notice: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>
Re: Drop table taking more time for execution
Posted by Sriveena Mattaparthi <Sr...@ekaplus.com>.
Thanks Ilya for the quick reply..
We have to drop the table in few cases where the table columns and datatypes changes ...
I guess alter table to add/ drop columns may take same longer time.
Any other alternative to drop table?
Regards,
Sriveena
________________________________
From: Ilya Kasnacheev <il...@gmail.com>
Sent: 28 August 2018 20:04:09
To: user@ignite.apache.org
Subject: Re: Drop table taking more time for execution
Hello!
As far as my understanding goes, DROP TABLE will usually destroy a cache, and that will require partition map exchange, which in turn will wait for all operations to finish. With increase of load and cluster size 15s does not seem excessive.
The suggestion is to avoid dropping/creating tables often, and especially as the course of normal operation.
Regards,
--
Ilya Kasnacheev
вт, 28 авг. 2018 г. в 17:24, Sriveena Mattaparthi <Sr...@ekaplus.com>>:
Hi,
Execution of drop table command is taking 15 sec on client node.Any suggestions to spewed up the execution.
Thanks & Regards,
Sriveena “Confidentiality Notice: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.”
“Confidentiality Notice: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.”
Re: Drop table taking more time for execution
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
As far as my understanding goes, DROP TABLE will usually destroy a cache,
and that will require partition map exchange, which in turn will wait for
all operations to finish. With increase of load and cluster size 15s does
not seem excessive.
The suggestion is to avoid dropping/creating tables often, and especially
as the course of normal operation.
Regards,
--
Ilya Kasnacheev
вт, 28 авг. 2018 г. в 17:24, Sriveena Mattaparthi <
Sriveena.Mattaparthi@ekaplus.com>:
> Hi,
>
> Execution of drop table command is taking 15 sec on client node.Any
> suggestions to spewed up the execution.
>
> Thanks & Regards,
> Sriveena “Confidentiality Notice: The contents of this email message and
> any attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>