You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Ivan Pavlukhin <vo...@gmail.com> on 2020/05/01 04:32:18 UTC
Re: Check indexes inline size tool
This one is among other features "missing without a good reason" [1].
[1] https://issues.apache.org/jira/browse/IGNITE-11402
Best regards,
Ivan Pavlukhin
чт, 30 апр. 2020 г. в 17:18, Ilya Kasnacheev <il...@gmail.com>:
>
> Hello!
>
> I don't think it ever worked. CREATE INDEX has INLINE_SIZE clause, CREATE
> TABLE does not.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 29 апр. 2020 г. в 20:35, Denis Magda <dm...@apache.org>:
>
> > Folks,
> >
> > >
> > >
> > > Unfortunately and embarrassingly, we still do not support passing
> > > INLINE_SIZE to CREATE TABLE, at least in 2.8.0.
> >
> >
> > I'm confused about this statement. Are we talking about an
> > issue/regression that slipped into 2.8.0? I do believe the feature worked
> > before.
> >
> > -
> > Denis
> >
> >
> > On Tue, Apr 28, 2020 at 4:01 AM Ilya Kasnacheev <ilya.kasnacheev@gmail.com
> > >
> > wrote:
> >
> > > Hello!
> > >
> > > Unfortunately and embarrassingly, we still do not support passing
> > > INLINE_SIZE to CREATE TABLE, at least in 2.8.0.
> > > This means IGNITE_MAX_INDEX_PAYLOAD_SIZE is the only option to create an
> > > implicit primary key index with specified inline size.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > вт, 28 апр. 2020 г. в 02:31, Denis Magda <dm...@apache.org>:
> > >
> > > > Hi Sergey,
> > > >
> > > > Your changes look useful from the application developer perspective.
> > > > However, I'm curious why would the one change some low-level
> > > > IGNITE_MAX_INDEX_PAYLOAD_SIZE parameter when it's advised to pass
> > > > INLINE_SIZE to CREATE TABLE to change the index size cluster-wide.
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Mon, Apr 27, 2020 at 5:38 AM Sergey Antonov <
> > > antonovsergey93@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi, Igniters!
> > > > >
> > > > > I'd like to share a new small feature in AI [1].
> > > > >
> > > > > For different reasons, the cluster could have a different SQL index
> > > > inline
> > > > > size [2] on cluster nodes. For example due to
> > > > > different IGNITE_MAX_INDEX_PAYLOAD_SIZE [3] value on cluster nodes.
> > > > >
> > > > > The difference in index inline size may lead to performance
> > > degradation.
> > > > > I think we must compare inline sizes on node join and warn if
> > > difference
> > > > > found. Also, We should have the ability to check inline sizes on
> > > demand.
> > > > >
> > > > > I've implemented this check on node join and new command in
> > control.sh
> > > > >
> > > > > Look at warning message and utility command output:
> > > > >
> > > > > Warn message on a node in the cluster during new node join:
> > > > >
> > > > > [2020-04-27 15:36:21,185][WARN ][tcp-disco-msg-worker-[6ba0b823
> > > > > 127.0.0.1:47502
> > > > > crd]-#17%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > > Inline
> > > > > sizes on local node and node 5bf6ca48-34a0-4aff-8db2-c0b6df303d3f are
> > > > > different. Please drop and create again these indexes to avoid
> > > > performance
> > > > > problems with SQL queries. Problem indexes:
> > > > >
> > > > >
> > > >
> > >
> > PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> > > > >
> > > > > Warn messages on a joining node, if difference found:
> > > > > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > > > > 127.0.0.1:47501
> > > > > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > > > Inline sizes on local node and node
> > > a86e9cea-63e8-42af-a897-cec4be500001
> > > > > are different. Please drop and create again these indexes to avoid
> > > > > performance problems with SQL queries. Problem indexes:
> > > > >
> > > > >
> > > >
> > >
> > PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> > > > > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > > > > 127.0.0.1:47501
> > > > > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > > > Inline sizes on local node and node
> > > a08de16d-df05-48af-a0b9-5596d9c00002
> > > > > are different. Please drop and create again these indexes to avoid
> > > > > performance problems with SQL queries. Problem indexes:
> > > > >
> > > > >
> > > >
> > >
> > PUBLIC#TEST_TABLE#L_IDX(1,3),PUBLIC#TEST_TABLE#S1_IDX(1,3),PUBLIC#TEST_TABLE#I_IDX(1,3)
> > > > >
> > > > >
> > > > > Utility output, if a difference in inline sizes was found:
> > > > >
> > > > > Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV]
> > > > > 2020 Copyright(C) Apache Software Foundation
> > > > > User: santonov
> > > > > Time: 2020-04-27T15:32:25.759
> > > > > Command [CACHE] started
> > > > > Arguments: --cache check_index_inline_sizes --yes
> > > > >
> > > > >
> > > >
> > >
> > --------------------------------------------------------------------------------
> > > > > Found 4 secondary indexes.
> > > > > 3 index(es) have different effective inline size on nodes. It can
> > lead
> > > to
> > > > > performance degradation in SQL queries.
> > > > > Index(es):
> > > > > Full index name: PUBLIC#TEST_TABLE#L_IDX nodes:
> > > > > [ca1d2bae-89d4-4e8d-ae11-6c68f3900000] inline size: 1, nodes:
> > > > > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> > > > > Full index name: PUBLIC#TEST_TABLE#S1_IDX nodes:
> > > > > [ca1d2bae-89d4-4e8d-ae11-6c68f3900000] inline size: 1, nodes:
> > > > > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> > > > > Full index name: PUBLIC#TEST_TABLE#I_IDX nodes:
> > > > > [ca1d2bae-89d4-4e8d-ae11-6c68f3900000] inline size: 1, nodes:
> > > > > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> > > > >
> > > > > Recommendations:
> > > > > Check that value of property IGNITE_MAX_INDEX_PAYLOAD_SIZE are the
> > > same
> > > > > on all nodes.
> > > > > Recreate indexes (execute DROP INDEX, CREATE INDEX commands) with
> > > > > different inline size.
> > > > > Command [CACHE] finished with code: 0
> > > > > Control utility has completed execution at: 2020-04-27T15:32:28.025
> > > > > Execution time: 2266 ms
> > > > >
> > > > > Utility output, if all indexes have the same inline size:
> > > > >
> > > > > Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV]
> > > > > 2020 Copyright(C) Apache Software Foundation
> > > > > User: santonov
> > > > > Time: 2020-04-27T15:30:20.950
> > > > > Command [CACHE] started
> > > > > Arguments: --cache check_index_inline_sizes --yes
> > > > >
> > > > >
> > > >
> > >
> > --------------------------------------------------------------------------------
> > > > > Found 2 secondary indexes.
> > > > > All secondary indexes have the same effective inline size on all
> > > cluster
> > > > > nodes.
> > > > > Command [CACHE] finished with code: 0
> > > > > Control utility has completed execution at: 2020-04-27T15:30:23.428
> > > > > Execution time: 2478 ms
> > > > >
> > > > > Any objections?
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12942
> > > > > [2] https://apacheignite-sql.readme.io/docs/create-index
> > > > > [3]
> > > > >
> > > > >
> > > >
> > >
> > https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteSystemProperties.html#IGNITE_MAX_INDEX_PAYLOAD_SIZE
> > > > >
> > > > > --
> > > > > BR, Sergey Antonov
> > > > >
> > > >
> > >
> >