You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Mikhail <wm...@mail.ru> on 2018/10/30 13:31:34 UTC
IgniteJdbcThinDriver statements accumulation
Hello,
I need to execute a lot of SQL statements in one connection using IgniteJdbcThinDriver. I get memory leak because of accumulation of all statements in:
private final ArrayList<JdbcThinStatement> stmts = new ArrayList<>(); (IgniteJdbcThinDriver:118). All statements are added to this list. As I see it, this list is cleared only by onDisconnect() method, which is called only on error. So In case of many statements, there will be memory leaks. And possibly the same situation will occur in connection pools, because they can reuse one connection many times. Is it the desired behavior for this ignite jdbc driver?
--
Best Regards,
Mikhail
Re: time series
Posted by Denis Magda <dm...@apache.org>.
Ignite can store the whole data set on disk and X% in RAM thanks to the
native persistence. So, you decide how much data you'd like to keep in RAM:
https://ignite.apache.org/arch/memorycentric.html
As per times series, I heard that Ignite is being used for that use case.
However, you might need more. My suggestion is to start and see how it goes.
--
Denis
On Tue, Nov 6, 2018 at 4:05 AM Mikhail <wm...@mail.ru> wrote:
> Hi Igniters,
>
> Are there any best practices of storing time series
> data in Ignite? We need it for extremely high load IoT system. Cassandra is
> likely to be an appropriate solution, but slow speed of analytical SQL
> queries are not acceptable for us. We can implement Ignite over Cassandra,
> but we need to access the hole data in Ignite and the cache shouldn't be
> extremely huge (e.g. it should be cache per day).
> We want to have the similar approach, as for example in
> [1]. However, writing such functionality from scratch has a lot of
> pitfalls. Are there any out-of-the-box features for time series data in
> Ignite? Does it sound reasonable to implement rollover pattern in Ignite
> (like in ES)? Or there could be another options?
>
> [1] - https://www.elastic.co/blog/managing-time-based-indices-efficiently
>
> --
> Best Regards,
> Mikhail
time series
Posted by Mikhail <wm...@mail.ru>.
Hi Igniters,
Are there any best practices of storing time series data in Ignite? We need it for extremely high load IoT system. Cassandra is likely to be an appropriate solution, but slow speed of analytical SQL queries are not acceptable for us. We can implement Ignite over Cassandra, but we need to access the hole data in Ignite and the cache shouldn't be extremely huge (e.g. it should be cache per day).
We want to have the similar approach, as for example in [1]. However, writing such functionality from scratch has a lot of pitfalls. Are there any out-of-the-box features for time series data in Ignite? Does it sound reasonable to implement rollover pattern in Ignite (like in ES)? Or there could be another options?
[1] - https://www.elastic.co/blog/managing-time-based-indices-efficiently
--
Best Regards,
Mikhail
Re: IgniteJdbcThinDriver statements accumulation
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Looks like a bug. Track it at
https://issues.apache.org/jira/browse/IGNITE-10224
Regards,
--
Ilya Kasnacheev
вт, 30 окт. 2018 г. в 16:31, Mikhail <wm...@mail.ru>:
> Hello,
>
> I need to execute a lot of SQL statements in one connection
> using IgniteJdbcThinDriver. I get memory leak because of accumulation of
> all statements in:
> private final ArrayList<JdbcThinStatement> stmts = new ArrayList<>();
> (IgniteJdbcThinDriver:118). All statements are added to this list. As I see
> it, this list is cleared only by onDisconnect() method, which is called
> only on error. So In case of many statements, there will be memory leaks.
> And possibly the same situation will occur in connection pools, because
> they can reuse one connection many times. Is it the desired behavior for
> this ignite jdbc driver?
>
> --
> Best Regards,
> Mikhail