You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Kamlesh Joshi <Ka...@ril.com> on 2019/05/07 10:00:53 UTC
RE: [External]Re: Read/query TPS is decreasing after enabling mix
load i.e. write services
Hi,
Unfortunately, I cant share the reproducer.
We are using Ignite Binary Objects for performing operations on the cluster (Get/Put). We have exposed these operations to other application (i.e. TIBCO BW services) which performs operations on the cluster.
Thanks and Regards,
Kamlesh Joshi
-----Original Message-----
From: Maxim.Pudov <pu...@gmail.com>
Sent: Wednesday, April 17, 2019 8:18 PM
To: user@ignite.apache.org
Subject: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hi, could you share a reproducer of your problem? We are missing a lot of information here. The configuration of your nodes, cache configurations, what API you use to query and update the data.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s).
are confidential and may be privileged. If you are not the intended recipient. you are hereby notified that any
review. re-transmission. conversion to hard copy. copying. circulation or other use of this message and any attachments is
strictly prohibited. If you are not the intended recipient. please notify the sender immediately by return email.
and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email.
The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
Re: [External]Re: Read/query TPS is decreasing after enabling mix
load i.e. write services
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I'm not talking about running out of disk, but about reaching maximum
throughput.
Upgrading to SSL will definitely help.Even if you must hold the bulk of
data on HDD, at least store your WAL on SSD.
Regards,
--
Ilya Kasnacheev
вт, 28 мая 2019 г. в 09:14, Kamlesh Joshi <Ka...@ril.com>:
> Thanks for your analysis on this Ilya.
>
>
>
> Have some queries/observations on this as below:
>
>
>
> 1. Disabling WAL won’t be possible as it’s the current scenario that
> we are having in Production system. Also, currently difficult to upgrade
> (may be an least wanted option).
> 2. I have seen that there is still a disk space available on the drive
> (not actually saturated).
> 3. Also, we have tried with altering WAL size to 2GB (instead of
> default of 64 MB) and default chunks of it i.e. 10 but still there was no
> visible change.
> 4. Lastly, if we replace HDD with SSDs will this problem occur or it
> will really help ?
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
> *From:* Ilya Kasnacheev <il...@gmail.com>
> *Sent:* Monday, May 27, 2019 7:49 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: [External]Re: Read/query TPS is decreasing after enabling
> mix load i.e. write services
>
>
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hello!
>
>
>
> The relevant stack trace as follows:
>
>
>
> "sys-stripe-43-#44%EDIFCustomer%" #70 prio=5 os_prio=0
> tid=0x00007fc68e274800 nid=0x7ebb runnable [0x00007fc4898d6000]
> java.lang.Thread.State: RUNNABLE
> at java.nio.MappedByteBuffer.force0(Native Method)
> at java.nio.MappedByteBuffer.force(MappedByteBuffer.java:203)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.close(FileWriteAheadLogManager.java:2798)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.access$600(FileWriteAheadLogManager.java:2438)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.rollOver(FileWriteAheadLogManager.java:1064)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.log(FileWriteAheadLogManager.java:753)
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.beforeReleaseWrite(PageMemoryImpl.java:1611)
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlockPage(PageMemoryImpl.java:1478)
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlock(PageMemoryImpl.java:449)
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlock(PageMemoryImpl.java:443)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writeUnlock(PageHandler.java:377)
> at
> org.apache.ignite.internal.processors.cache.persistence.DataStructure.writeUnlock(DataStructure.java:210)
> at
> org.apache.ignite.internal.processors.cache.persistence.DataStructure.writeUnlock(DataStructure.java:181)
> at
> org.apache.ignite.internal.processors.cache.persistence.freelist.PagesList.put(PagesList.java:684)
>
>
>
> It seems that syncing WAL file to disk takes prolonged period of time,
> which is indicative of not being able to write to disks fast enough. I
> think that you have saturated your storage capacity.
>
>
>
> Maybe it will make sense to upgrade to 2.7 and disable WAL for the period
> of data ingestion, if you can separate it. Otherwise, look int beefing up
> your storage. What might also help is fewer WAL segments of larger size.
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> пт, 24 мая 2019 г. в 09:27, Kamlesh Joshi <Ka...@ril.com>:
>
> Hi Ilya,
>
>
>
> PFA thread dumps and JFRs from the given cluster. Any observation and
> recommendation would be of great help!
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
> *From:* Ilya Kasnacheev <il...@gmail.com>
> *Sent:* Thursday, May 23, 2019 7:35 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: [External]Re: Read/query TPS is decreasing after enabling
> mix load i.e. write services
>
>
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hello!
>
>
>
> I recommend gathering thread dumps from cluster as it has performance
> issues, sharing these dumps with us.
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> чт, 23 мая 2019 г. в 10:52, Kamlesh Joshi <Ka...@ril.com>:
>
> Hi ilya,
>
>
>
> We tried with LOG_ONLY and BACKGROUND both but still behavior remains
> same. Any other parameter tweak which would help?
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
> *From:* Ilya Kasnacheev <il...@gmail.com>
> *Sent:* Wednesday, May 22, 2019 9:14 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: [External]Re: Read/query TPS is decreasing after enabling
> mix load i.e. write services
>
>
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hello!
>
>
>
> If it is 'timeout' then it is likely not the reason of your issues.
>
>
>
> What is your walMode? Is it LOG_ONLY? If it's not, try to use LOG_ONLY
> here.
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> ср, 22 мая 2019 г. в 16:30, Kamlesh Joshi <Ka...@ril.com>:
>
> Hi Ilya,
>
>
>
> Looking at the logs, every time checkpoint starts due to
> ‘timeout’ reason not because the buffer is full. So still do we need to
> increase checkpointing buffer or only changing checkpointing frequency will
> help ?
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
> *From:* Ilya Kasnacheev <il...@gmail.com>
> *Sent:* Wednesday, May 22, 2019 4:19 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: [External]Re: Read/query TPS is decreasing after enabling
> mix load i.e. write services
>
>
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hello!
>
>
>
> We recommend using SSD and not HDD with Ignite. Otherwise, try to increase
> size of Checkpoint Page Buffer - if you run out of this buffer, all
> activity will stop until checkpoint is finished. Maybe you also need to
> decrease time between checkpoints.
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> вт, 7 мая 2019 г. в 13:01, Kamlesh Joshi <Ka...@ril.com>:
>
> Hi,
>
> Unfortunately, I cant share the reproducer.
> We are using Ignite Binary Objects for performing operations on the
> cluster (Get/Put). We have exposed these operations to other application
> (i.e. TIBCO BW services) which performs operations on the cluster.
>
> Thanks and Regards,
> Kamlesh Joshi
>
> -----Original Message-----
> From: Maxim.Pudov <pu...@gmail.com>
> Sent: Wednesday, April 17, 2019 8:18 PM
> To: user@ignite.apache.org
> Subject: [External]Re: Read/query TPS is decreasing after enabling mix
> load i.e. write services
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hi, could you share a reproducer of your problem? We are missing a lot of
> information here. The configuration of your nodes, cache configurations,
> what API you use to query and update the data.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
> "Confidentiality Warning: This message and any attachments are intended
> only for the use of the intended recipient(s).
> are confidential and may be privileged. If you are not the intended
> recipient. you are hereby notified that any
> review. re-transmission. conversion to hard copy. copying. circulation or
> other use of this message and any attachments is
> strictly prohibited. If you are not the intended recipient. please notify
> the sender immediately by return email.
> and delete this message and any attachments from your system.
>
> Virus Warning: Although the company has taken reasonable precautions to
> ensure no viruses are present in this email.
> The company cannot accept responsibility for any loss or damage arising
> from the use of this email or attachment."
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> *Virus Warning:* Although the company has taken reasonable precautions to
> ensure no viruses are present in this email. The company cannot accept
> responsibility for any loss or damage arising from the use of this email or
> attachment."
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> *Virus Warning:* Although the company has taken reasonable precautions to
> ensure no viruses are present in this email. The company cannot accept
> responsibility for any loss or damage arising from the use of this email or
> attachment."
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> *Virus Warning:* Although the company has taken reasonable precautions to
> ensure no viruses are present in this email. The company cannot accept
> responsibility for any loss or damage arising from the use of this email or
> attachment."
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> *Virus Warning:* Although the company has taken reasonable precautions to
> ensure no viruses are present in this email. The company cannot accept
> responsibility for any loss or damage arising from the use of this email or
> attachment."
>
RE: [External]Re: Read/query TPS is decreasing after enabling mix
load i.e. write services
Posted by Kamlesh Joshi <Ka...@ril.com>.
Thanks for your analysis on this Ilya.
Have some queries/observations on this as below:
1. Disabling WAL won’t be possible as it’s the current scenario that we are having in Production system. Also, currently difficult to upgrade (may be an least wanted option).
2. I have seen that there is still a disk space available on the drive (not actually saturated).
3. Also, we have tried with altering WAL size to 2GB (instead of default of 64 MB) and default chunks of it i.e. 10 but still there was no visible change.
4. Lastly, if we replace HDD with SSDs will this problem occur or it will really help ?
Thanks and Regards,
Kamlesh Joshi
From: Ilya Kasnacheev <il...@gmail.com>
Sent: Monday, May 27, 2019 7:49 PM
To: user@ignite.apache.org
Subject: Re: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hello!
The relevant stack trace as follows:
"sys-stripe-43-#44%EDIFCustomer%" #70 prio=5 os_prio=0 tid=0x00007fc68e274800 nid=0x7ebb runnable [0x00007fc4898d6000]
java.lang.Thread.State: RUNNABLE
at java.nio.MappedByteBuffer.force0(Native Method)
at java.nio.MappedByteBuffer.force(MappedByteBuffer.java:203)
at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.close(FileWriteAheadLogManager.java:2798)
at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.access$600(FileWriteAheadLogManager.java:2438)
at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.rollOver(FileWriteAheadLogManager.java:1064)
at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.log(FileWriteAheadLogManager.java:753)
at org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.beforeReleaseWrite(PageMemoryImpl.java:1611)
at org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlockPage(PageMemoryImpl.java:1478)
at org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlock(PageMemoryImpl.java:449)
at org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlock(PageMemoryImpl.java:443)
at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writeUnlock(PageHandler.java:377)
at org.apache.ignite.internal.processors.cache.persistence.DataStructure.writeUnlock(DataStructure.java:210)
at org.apache.ignite.internal.processors.cache.persistence.DataStructure.writeUnlock(DataStructure.java:181)
at org.apache.ignite.internal.processors.cache.persistence.freelist.PagesList.put(PagesList.java:684)
It seems that syncing WAL file to disk takes prolonged period of time, which is indicative of not being able to write to disks fast enough. I think that you have saturated your storage capacity.
Maybe it will make sense to upgrade to 2.7 and disable WAL for the period of data ingestion, if you can separate it. Otherwise, look int beefing up your storage. What might also help is fewer WAL segments of larger size.
Regards,
--
Ilya Kasnacheev
пт, 24 мая 2019 г. в 09:27, Kamlesh Joshi <Ka...@ril.com>>:
Hi Ilya,
PFA thread dumps and JFRs from the given cluster. Any observation and recommendation would be of great help!
Thanks and Regards,
Kamlesh Joshi
From: Ilya Kasnacheev <il...@gmail.com>>
Sent: Thursday, May 23, 2019 7:35 PM
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: Re: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hello!
I recommend gathering thread dumps from cluster as it has performance issues, sharing these dumps with us.
Regards,
--
Ilya Kasnacheev
чт, 23 мая 2019 г. в 10:52, Kamlesh Joshi <Ka...@ril.com>>:
Hi ilya,
We tried with LOG_ONLY and BACKGROUND both but still behavior remains same. Any other parameter tweak which would help?
Thanks and Regards,
Kamlesh Joshi
From: Ilya Kasnacheev <il...@gmail.com>>
Sent: Wednesday, May 22, 2019 9:14 PM
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: Re: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hello!
If it is 'timeout' then it is likely not the reason of your issues.
What is your walMode? Is it LOG_ONLY? If it's not, try to use LOG_ONLY here.
Regards,
--
Ilya Kasnacheev
ср, 22 мая 2019 г. в 16:30, Kamlesh Joshi <Ka...@ril.com>>:
Hi Ilya,
Looking at the logs, every time checkpoint starts due to ‘timeout’ reason not because the buffer is full. So still do we need to increase checkpointing buffer or only changing checkpointing frequency will help ?
Thanks and Regards,
Kamlesh Joshi
From: Ilya Kasnacheev <il...@gmail.com>>
Sent: Wednesday, May 22, 2019 4:19 PM
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: Re: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hello!
We recommend using SSD and not HDD with Ignite. Otherwise, try to increase size of Checkpoint Page Buffer - if you run out of this buffer, all activity will stop until checkpoint is finished. Maybe you also need to decrease time between checkpoints.
Regards,
--
Ilya Kasnacheev
вт, 7 мая 2019 г. в 13:01, Kamlesh Joshi <Ka...@ril.com>>:
Hi,
Unfortunately, I cant share the reproducer.
We are using Ignite Binary Objects for performing operations on the cluster (Get/Put). We have exposed these operations to other application (i.e. TIBCO BW services) which performs operations on the cluster.
Thanks and Regards,
Kamlesh Joshi
-----Original Message-----
From: Maxim.Pudov <pu...@gmail.com>>
Sent: Wednesday, April 17, 2019 8:18 PM
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hi, could you share a reproducer of your problem? We are missing a lot of information here. The configuration of your nodes, cache configurations, what API you use to query and update the data.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s).
are confidential and may be privileged. If you are not the intended recipient. you are hereby notified that any
review. re-transmission. conversion to hard copy. copying. circulation or other use of this message and any attachments is
strictly prohibited. If you are not the intended recipient. please notify the sender immediately by return email.
and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email.
The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s).
are confidential and may be privileged. If you are not the intended recipient. you are hereby notified that any
review. re-transmission. conversion to hard copy. copying. circulation or other use of this message and any attachments is
strictly prohibited. If you are not the intended recipient. please notify the sender immediately by return email.
and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email.
The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
Re: [External]Re: Read/query TPS is decreasing after enabling mix
load i.e. write services
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
The relevant stack trace as follows:
"sys-stripe-43-#44%EDIFCustomer%" #70 prio=5 os_prio=0
tid=0x00007fc68e274800 nid=0x7ebb runnable [0x00007fc4898d6000]
java.lang.Thread.State: RUNNABLE
at java.nio.MappedByteBuffer.force0(Native Method)
at java.nio.MappedByteBuffer.force(MappedByteBuffer.java:203)
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.close(FileWriteAheadLogManager.java:2798)
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.access$600(FileWriteAheadLogManager.java:2438)
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.rollOver(FileWriteAheadLogManager.java:1064)
at
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.log(FileWriteAheadLogManager.java:753)
at
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.beforeReleaseWrite(PageMemoryImpl.java:1611)
at
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlockPage(PageMemoryImpl.java:1478)
at
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlock(PageMemoryImpl.java:449)
at
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlock(PageMemoryImpl.java:443)
at
org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writeUnlock(PageHandler.java:377)
at
org.apache.ignite.internal.processors.cache.persistence.DataStructure.writeUnlock(DataStructure.java:210)
at
org.apache.ignite.internal.processors.cache.persistence.DataStructure.writeUnlock(DataStructure.java:181)
at
org.apache.ignite.internal.processors.cache.persistence.freelist.PagesList.put(PagesList.java:684)
It seems that syncing WAL file to disk takes prolonged period of time,
which is indicative of not being able to write to disks fast enough. I
think that you have saturated your storage capacity.
Maybe it will make sense to upgrade to 2.7 and disable WAL for the period
of data ingestion, if you can separate it. Otherwise, look int beefing up
your storage. What might also help is fewer WAL segments of larger size.
Regards,
--
Ilya Kasnacheev
пт, 24 мая 2019 г. в 09:27, Kamlesh Joshi <Ka...@ril.com>:
> Hi Ilya,
>
>
>
> PFA thread dumps and JFRs from the given cluster. Any observation and
> recommendation would be of great help!
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
> *From:* Ilya Kasnacheev <il...@gmail.com>
> *Sent:* Thursday, May 23, 2019 7:35 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: [External]Re: Read/query TPS is decreasing after enabling
> mix load i.e. write services
>
>
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hello!
>
>
>
> I recommend gathering thread dumps from cluster as it has performance
> issues, sharing these dumps with us.
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> чт, 23 мая 2019 г. в 10:52, Kamlesh Joshi <Ka...@ril.com>:
>
> Hi ilya,
>
>
>
> We tried with LOG_ONLY and BACKGROUND both but still behavior remains
> same. Any other parameter tweak which would help?
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
> *From:* Ilya Kasnacheev <il...@gmail.com>
> *Sent:* Wednesday, May 22, 2019 9:14 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: [External]Re: Read/query TPS is decreasing after enabling
> mix load i.e. write services
>
>
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hello!
>
>
>
> If it is 'timeout' then it is likely not the reason of your issues.
>
>
>
> What is your walMode? Is it LOG_ONLY? If it's not, try to use LOG_ONLY
> here.
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> ср, 22 мая 2019 г. в 16:30, Kamlesh Joshi <Ka...@ril.com>:
>
> Hi Ilya,
>
>
>
> Looking at the logs, every time checkpoint starts due to
> ‘timeout’ reason not because the buffer is full. So still do we need to
> increase checkpointing buffer or only changing checkpointing frequency will
> help ?
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
> *From:* Ilya Kasnacheev <il...@gmail.com>
> *Sent:* Wednesday, May 22, 2019 4:19 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: [External]Re: Read/query TPS is decreasing after enabling
> mix load i.e. write services
>
>
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hello!
>
>
>
> We recommend using SSD and not HDD with Ignite. Otherwise, try to increase
> size of Checkpoint Page Buffer - if you run out of this buffer, all
> activity will stop until checkpoint is finished. Maybe you also need to
> decrease time between checkpoints.
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> вт, 7 мая 2019 г. в 13:01, Kamlesh Joshi <Ka...@ril.com>:
>
> Hi,
>
> Unfortunately, I cant share the reproducer.
> We are using Ignite Binary Objects for performing operations on the
> cluster (Get/Put). We have exposed these operations to other application
> (i.e. TIBCO BW services) which performs operations on the cluster.
>
> Thanks and Regards,
> Kamlesh Joshi
>
> -----Original Message-----
> From: Maxim.Pudov <pu...@gmail.com>
> Sent: Wednesday, April 17, 2019 8:18 PM
> To: user@ignite.apache.org
> Subject: [External]Re: Read/query TPS is decreasing after enabling mix
> load i.e. write services
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hi, could you share a reproducer of your problem? We are missing a lot of
> information here. The configuration of your nodes, cache configurations,
> what API you use to query and update the data.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
> "Confidentiality Warning: This message and any attachments are intended
> only for the use of the intended recipient(s).
> are confidential and may be privileged. If you are not the intended
> recipient. you are hereby notified that any
> review. re-transmission. conversion to hard copy. copying. circulation or
> other use of this message and any attachments is
> strictly prohibited. If you are not the intended recipient. please notify
> the sender immediately by return email.
> and delete this message and any attachments from your system.
>
> Virus Warning: Although the company has taken reasonable precautions to
> ensure no viruses are present in this email.
> The company cannot accept responsibility for any loss or damage arising
> from the use of this email or attachment."
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> *Virus Warning:* Although the company has taken reasonable precautions to
> ensure no viruses are present in this email. The company cannot accept
> responsibility for any loss or damage arising from the use of this email or
> attachment."
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> *Virus Warning:* Although the company has taken reasonable precautions to
> ensure no viruses are present in this email. The company cannot accept
> responsibility for any loss or damage arising from the use of this email or
> attachment."
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> *Virus Warning:* Although the company has taken reasonable precautions to
> ensure no viruses are present in this email. The company cannot accept
> responsibility for any loss or damage arising from the use of this email or
> attachment."
>
RE: [External]Re: Read/query TPS is decreasing after enabling mix
load i.e. write services
Posted by Kamlesh Joshi <Ka...@ril.com>.
Hi Ilya,
PFA thread dumps and JFRs from the given cluster. Any observation and recommendation would be of great help!
Thanks and Regards,
Kamlesh Joshi
From: Ilya Kasnacheev <il...@gmail.com>
Sent: Thursday, May 23, 2019 7:35 PM
To: user@ignite.apache.org
Subject: Re: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hello!
I recommend gathering thread dumps from cluster as it has performance issues, sharing these dumps with us.
Regards,
--
Ilya Kasnacheev
чт, 23 мая 2019 г. в 10:52, Kamlesh Joshi <Ka...@ril.com>>:
Hi ilya,
We tried with LOG_ONLY and BACKGROUND both but still behavior remains same. Any other parameter tweak which would help?
Thanks and Regards,
Kamlesh Joshi
From: Ilya Kasnacheev <il...@gmail.com>>
Sent: Wednesday, May 22, 2019 9:14 PM
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: Re: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hello!
If it is 'timeout' then it is likely not the reason of your issues.
What is your walMode? Is it LOG_ONLY? If it's not, try to use LOG_ONLY here.
Regards,
--
Ilya Kasnacheev
ср, 22 мая 2019 г. в 16:30, Kamlesh Joshi <Ka...@ril.com>>:
Hi Ilya,
Looking at the logs, every time checkpoint starts due to ‘timeout’ reason not because the buffer is full. So still do we need to increase checkpointing buffer or only changing checkpointing frequency will help ?
Thanks and Regards,
Kamlesh Joshi
From: Ilya Kasnacheev <il...@gmail.com>>
Sent: Wednesday, May 22, 2019 4:19 PM
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: Re: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hello!
We recommend using SSD and not HDD with Ignite. Otherwise, try to increase size of Checkpoint Page Buffer - if you run out of this buffer, all activity will stop until checkpoint is finished. Maybe you also need to decrease time between checkpoints.
Regards,
--
Ilya Kasnacheev
вт, 7 мая 2019 г. в 13:01, Kamlesh Joshi <Ka...@ril.com>>:
Hi,
Unfortunately, I cant share the reproducer.
We are using Ignite Binary Objects for performing operations on the cluster (Get/Put). We have exposed these operations to other application (i.e. TIBCO BW services) which performs operations on the cluster.
Thanks and Regards,
Kamlesh Joshi
-----Original Message-----
From: Maxim.Pudov <pu...@gmail.com>>
Sent: Wednesday, April 17, 2019 8:18 PM
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hi, could you share a reproducer of your problem? We are missing a lot of information here. The configuration of your nodes, cache configurations, what API you use to query and update the data.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s).
are confidential and may be privileged. If you are not the intended recipient. you are hereby notified that any
review. re-transmission. conversion to hard copy. copying. circulation or other use of this message and any attachments is
strictly prohibited. If you are not the intended recipient. please notify the sender immediately by return email.
and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email.
The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s).
are confidential and may be privileged. If you are not the intended recipient. you are hereby notified that any
review. re-transmission. conversion to hard copy. copying. circulation or other use of this message and any attachments is
strictly prohibited. If you are not the intended recipient. please notify the sender immediately by return email.
and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email.
The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
Re: [External]Re: Read/query TPS is decreasing after enabling mix
load i.e. write services
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I recommend gathering thread dumps from cluster as it has performance
issues, sharing these dumps with us.
Regards,
--
Ilya Kasnacheev
чт, 23 мая 2019 г. в 10:52, Kamlesh Joshi <Ka...@ril.com>:
> Hi ilya,
>
>
>
> We tried with LOG_ONLY and BACKGROUND both but still behavior remains
> same. Any other parameter tweak which would help?
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
> *From:* Ilya Kasnacheev <il...@gmail.com>
> *Sent:* Wednesday, May 22, 2019 9:14 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: [External]Re: Read/query TPS is decreasing after enabling
> mix load i.e. write services
>
>
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hello!
>
>
>
> If it is 'timeout' then it is likely not the reason of your issues.
>
>
>
> What is your walMode? Is it LOG_ONLY? If it's not, try to use LOG_ONLY
> here.
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> ср, 22 мая 2019 г. в 16:30, Kamlesh Joshi <Ka...@ril.com>:
>
> Hi Ilya,
>
>
>
> Looking at the logs, every time checkpoint starts due to
> ‘timeout’ reason not because the buffer is full. So still do we need to
> increase checkpointing buffer or only changing checkpointing frequency will
> help ?
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
> *From:* Ilya Kasnacheev <il...@gmail.com>
> *Sent:* Wednesday, May 22, 2019 4:19 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: [External]Re: Read/query TPS is decreasing after enabling
> mix load i.e. write services
>
>
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hello!
>
>
>
> We recommend using SSD and not HDD with Ignite. Otherwise, try to increase
> size of Checkpoint Page Buffer - if you run out of this buffer, all
> activity will stop until checkpoint is finished. Maybe you also need to
> decrease time between checkpoints.
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> вт, 7 мая 2019 г. в 13:01, Kamlesh Joshi <Ka...@ril.com>:
>
> Hi,
>
> Unfortunately, I cant share the reproducer.
> We are using Ignite Binary Objects for performing operations on the
> cluster (Get/Put). We have exposed these operations to other application
> (i.e. TIBCO BW services) which performs operations on the cluster.
>
> Thanks and Regards,
> Kamlesh Joshi
>
> -----Original Message-----
> From: Maxim.Pudov <pu...@gmail.com>
> Sent: Wednesday, April 17, 2019 8:18 PM
> To: user@ignite.apache.org
> Subject: [External]Re: Read/query TPS is decreasing after enabling mix
> load i.e. write services
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hi, could you share a reproducer of your problem? We are missing a lot of
> information here. The configuration of your nodes, cache configurations,
> what API you use to query and update the data.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
> "Confidentiality Warning: This message and any attachments are intended
> only for the use of the intended recipient(s).
> are confidential and may be privileged. If you are not the intended
> recipient. you are hereby notified that any
> review. re-transmission. conversion to hard copy. copying. circulation or
> other use of this message and any attachments is
> strictly prohibited. If you are not the intended recipient. please notify
> the sender immediately by return email.
> and delete this message and any attachments from your system.
>
> Virus Warning: Although the company has taken reasonable precautions to
> ensure no viruses are present in this email.
> The company cannot accept responsibility for any loss or damage arising
> from the use of this email or attachment."
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> *Virus Warning:* Although the company has taken reasonable precautions to
> ensure no viruses are present in this email. The company cannot accept
> responsibility for any loss or damage arising from the use of this email or
> attachment."
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> *Virus Warning:* Although the company has taken reasonable precautions to
> ensure no viruses are present in this email. The company cannot accept
> responsibility for any loss or damage arising from the use of this email or
> attachment."
>
RE: [External]Re: Read/query TPS is decreasing after enabling mix
load i.e. write services
Posted by Kamlesh Joshi <Ka...@ril.com>.
Hi ilya,
We tried with LOG_ONLY and BACKGROUND both but still behavior remains same. Any other parameter tweak which would help?
Thanks and Regards,
Kamlesh Joshi
From: Ilya Kasnacheev <il...@gmail.com>
Sent: Wednesday, May 22, 2019 9:14 PM
To: user@ignite.apache.org
Subject: Re: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hello!
If it is 'timeout' then it is likely not the reason of your issues.
What is your walMode? Is it LOG_ONLY? If it's not, try to use LOG_ONLY here.
Regards,
--
Ilya Kasnacheev
ср, 22 мая 2019 г. в 16:30, Kamlesh Joshi <Ka...@ril.com>>:
Hi Ilya,
Looking at the logs, every time checkpoint starts due to ‘timeout’ reason not because the buffer is full. So still do we need to increase checkpointing buffer or only changing checkpointing frequency will help ?
Thanks and Regards,
Kamlesh Joshi
From: Ilya Kasnacheev <il...@gmail.com>>
Sent: Wednesday, May 22, 2019 4:19 PM
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: Re: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hello!
We recommend using SSD and not HDD with Ignite. Otherwise, try to increase size of Checkpoint Page Buffer - if you run out of this buffer, all activity will stop until checkpoint is finished. Maybe you also need to decrease time between checkpoints.
Regards,
--
Ilya Kasnacheev
вт, 7 мая 2019 г. в 13:01, Kamlesh Joshi <Ka...@ril.com>>:
Hi,
Unfortunately, I cant share the reproducer.
We are using Ignite Binary Objects for performing operations on the cluster (Get/Put). We have exposed these operations to other application (i.e. TIBCO BW services) which performs operations on the cluster.
Thanks and Regards,
Kamlesh Joshi
-----Original Message-----
From: Maxim.Pudov <pu...@gmail.com>>
Sent: Wednesday, April 17, 2019 8:18 PM
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hi, could you share a reproducer of your problem? We are missing a lot of information here. The configuration of your nodes, cache configurations, what API you use to query and update the data.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s).
are confidential and may be privileged. If you are not the intended recipient. you are hereby notified that any
review. re-transmission. conversion to hard copy. copying. circulation or other use of this message and any attachments is
strictly prohibited. If you are not the intended recipient. please notify the sender immediately by return email.
and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email.
The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s).
are confidential and may be privileged. If you are not the intended recipient. you are hereby notified that any
review. re-transmission. conversion to hard copy. copying. circulation or other use of this message and any attachments is
strictly prohibited. If you are not the intended recipient. please notify the sender immediately by return email.
and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email.
The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
Re: [External]Re: Read/query TPS is decreasing after enabling mix
load i.e. write services
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
If it is 'timeout' then it is likely not the reason of your issues.
What is your walMode? Is it LOG_ONLY? If it's not, try to use LOG_ONLY here.
Regards,
--
Ilya Kasnacheev
ср, 22 мая 2019 г. в 16:30, Kamlesh Joshi <Ka...@ril.com>:
> Hi Ilya,
>
>
>
> Looking at the logs, every time checkpoint starts due to
> ‘timeout’ reason not because the buffer is full. So still do we need to
> increase checkpointing buffer or only changing checkpointing frequency will
> help ?
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
> *From:* Ilya Kasnacheev <il...@gmail.com>
> *Sent:* Wednesday, May 22, 2019 4:19 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: [External]Re: Read/query TPS is decreasing after enabling
> mix load i.e. write services
>
>
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hello!
>
>
>
> We recommend using SSD and not HDD with Ignite. Otherwise, try to increase
> size of Checkpoint Page Buffer - if you run out of this buffer, all
> activity will stop until checkpoint is finished. Maybe you also need to
> decrease time between checkpoints.
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> вт, 7 мая 2019 г. в 13:01, Kamlesh Joshi <Ka...@ril.com>:
>
> Hi,
>
> Unfortunately, I cant share the reproducer.
> We are using Ignite Binary Objects for performing operations on the
> cluster (Get/Put). We have exposed these operations to other application
> (i.e. TIBCO BW services) which performs operations on the cluster.
>
> Thanks and Regards,
> Kamlesh Joshi
>
> -----Original Message-----
> From: Maxim.Pudov <pu...@gmail.com>
> Sent: Wednesday, April 17, 2019 8:18 PM
> To: user@ignite.apache.org
> Subject: [External]Re: Read/query TPS is decreasing after enabling mix
> load i.e. write services
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hi, could you share a reproducer of your problem? We are missing a lot of
> information here. The configuration of your nodes, cache configurations,
> what API you use to query and update the data.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
> "Confidentiality Warning: This message and any attachments are intended
> only for the use of the intended recipient(s).
> are confidential and may be privileged. If you are not the intended
> recipient. you are hereby notified that any
> review. re-transmission. conversion to hard copy. copying. circulation or
> other use of this message and any attachments is
> strictly prohibited. If you are not the intended recipient. please notify
> the sender immediately by return email.
> and delete this message and any attachments from your system.
>
> Virus Warning: Although the company has taken reasonable precautions to
> ensure no viruses are present in this email.
> The company cannot accept responsibility for any loss or damage arising
> from the use of this email or attachment."
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> *Virus Warning:* Although the company has taken reasonable precautions to
> ensure no viruses are present in this email. The company cannot accept
> responsibility for any loss or damage arising from the use of this email or
> attachment."
>
RE: [External]Re: Read/query TPS is decreasing after enabling mix
load i.e. write services
Posted by Kamlesh Joshi <Ka...@ril.com>.
Hi Ilya,
Looking at the logs, every time checkpoint starts due to ‘timeout’ reason not because the buffer is full. So still do we need to increase checkpointing buffer or only changing checkpointing frequency will help ?
Thanks and Regards,
Kamlesh Joshi
From: Ilya Kasnacheev <il...@gmail.com>
Sent: Wednesday, May 22, 2019 4:19 PM
To: user@ignite.apache.org
Subject: Re: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hello!
We recommend using SSD and not HDD with Ignite. Otherwise, try to increase size of Checkpoint Page Buffer - if you run out of this buffer, all activity will stop until checkpoint is finished. Maybe you also need to decrease time between checkpoints.
Regards,
--
Ilya Kasnacheev
вт, 7 мая 2019 г. в 13:01, Kamlesh Joshi <Ka...@ril.com>>:
Hi,
Unfortunately, I cant share the reproducer.
We are using Ignite Binary Objects for performing operations on the cluster (Get/Put). We have exposed these operations to other application (i.e. TIBCO BW services) which performs operations on the cluster.
Thanks and Regards,
Kamlesh Joshi
-----Original Message-----
From: Maxim.Pudov <pu...@gmail.com>>
Sent: Wednesday, April 17, 2019 8:18 PM
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: [External]Re: Read/query TPS is decreasing after enabling mix load i.e. write services
The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.
Hi, could you share a reproducer of your problem? We are missing a lot of information here. The configuration of your nodes, cache configurations, what API you use to query and update the data.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s).
are confidential and may be privileged. If you are not the intended recipient. you are hereby notified that any
review. re-transmission. conversion to hard copy. copying. circulation or other use of this message and any attachments is
strictly prohibited. If you are not the intended recipient. please notify the sender immediately by return email.
and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email.
The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s).
are confidential and may be privileged. If you are not the intended recipient. you are hereby notified that any
review. re-transmission. conversion to hard copy. copying. circulation or other use of this message and any attachments is
strictly prohibited. If you are not the intended recipient. please notify the sender immediately by return email.
and delete this message and any attachments from your system.
Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email.
The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."
Re: [External]Re: Read/query TPS is decreasing after enabling mix
load i.e. write services
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
We recommend using SSD and not HDD with Ignite. Otherwise, try to increase
size of Checkpoint Page Buffer - if you run out of this buffer, all
activity will stop until checkpoint is finished. Maybe you also need to
decrease time between checkpoints.
Regards,
--
Ilya Kasnacheev
вт, 7 мая 2019 г. в 13:01, Kamlesh Joshi <Ka...@ril.com>:
> Hi,
>
> Unfortunately, I cant share the reproducer.
> We are using Ignite Binary Objects for performing operations on the
> cluster (Get/Put). We have exposed these operations to other application
> (i.e. TIBCO BW services) which performs operations on the cluster.
>
> Thanks and Regards,
> Kamlesh Joshi
>
> -----Original Message-----
> From: Maxim.Pudov <pu...@gmail.com>
> Sent: Wednesday, April 17, 2019 8:18 PM
> To: user@ignite.apache.org
> Subject: [External]Re: Read/query TPS is decreasing after enabling mix
> load i.e. write services
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hi, could you share a reproducer of your problem? We are missing a lot of
> information here. The configuration of your nodes, cache configurations,
> what API you use to query and update the data.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
> "Confidentiality Warning: This message and any attachments are intended
> only for the use of the intended recipient(s).
> are confidential and may be privileged. If you are not the intended
> recipient. you are hereby notified that any
> review. re-transmission. conversion to hard copy. copying. circulation or
> other use of this message and any attachments is
> strictly prohibited. If you are not the intended recipient. please notify
> the sender immediately by return email.
> and delete this message and any attachments from your system.
>
> Virus Warning: Although the company has taken reasonable precautions to
> ensure no viruses are present in this email.
> The company cannot accept responsibility for any loss or damage arising
> from the use of this email or attachment."
>
>