You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "oleg-vlsk (via GitHub)" <gi...@apache.org> on 2024/04/17 01:25:34 UTC

[PR] Ignite 21445 [ignite]

oleg-vlsk opened a new pull request, #11317:
URL: https://github.com/apache/ignite/pull/11317

   Thank you for submitting the pull request to the Apache Ignite.
   
   In order to streamline the review of the contribution 
   we ask you to ensure the following steps have been taken:
   
   ### The Contribution Checklist
   - [ ] There is a single JIRA ticket related to the pull request. 
   - [ ] The web-link to the pull request is attached to the JIRA ticket.
   - [ ] The JIRA ticket has the _Patch Available_ state.
   - [ ] The pull request body describes changes that have been made. 
   The description explains _WHAT_ and _WHY_ was made instead of _HOW_.
   - [ ] The pull request title is treated as the final commit message. 
   The following pattern must be used: `IGNITE-XXXX Change summary` where `XXXX` - number of JIRA issue.
   - [ ] A reviewer has been mentioned through the JIRA comments 
   (see [the Maintainers list](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers)) 
   - [ ] The pull request has been checked by the Teamcity Bot and 
   the `green visa` attached to the JIRA ticket (see [TC.Bot: Check PR](https://mtcga.gridgain.com/prs.html))
   
   ### Notes
   - [How to Contribute](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute)
   - [Coding abbreviation rules](https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules)
   - [Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines)
   - [Apache Ignite Teamcity Bot](https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot)
   
   If you need any help, please email dev@ignite.apache.org or ask anу advice on http://asf.slack.com _#ignite_ channel.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1628750663


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1499,6 +1500,59 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local index query.
+     *
+     * @param qry Query.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    public GridCloseableIterator indexQueryLocal(final GridCacheQueryAdapter qry) throws IgniteCheckedException {
+        if (!enterBusy())
+            throw new IllegalStateException("Failed to process query request (grid is stopping).");
+
+        try {
+            assert qry.type() == INDEX : "Wrong processing of query: " + qry.type();
+
+            cctx.checkSecurity(SecurityPermission.CACHE_READ);
+
+            if (qry.nodes().isEmpty())

Review Comment:
   The check is changed to `cctx.localNode().isClient()`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1622414794


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1499,6 +1502,79 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local index query.
+     *
+     * @param qry Query.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    public GridCloseableIterator indexQueryLocal(final GridCacheQueryAdapter qry) throws IgniteCheckedException {
+        if (!enterBusy())
+            throw new IllegalStateException("Failed to process query request (grid is stopping).");
+
+        try {
+            assert qry.type() == INDEX : "Wrong processing of query: " + qry.type();
+
+            cctx.checkSecurity(SecurityPermission.CACHE_READ);
+
+            GridDhtCacheAdapter<?, ?> cacheAdapter = cctx.isNear() ? cctx.near().dht() : cctx.dht();
+
+            Set<Integer> lostParts = cacheAdapter.topology().lostPartitions();
+
+            Integer part = qry.partition();
+
+            if (!lostParts.isEmpty()) {
+                if (part == null || lostParts.contains(part)) {
+                    throw new CacheException(new CacheInvalidStateException("Failed to execute query because cache partition " +
+                        "has been lost [cacheName=" + cctx.name() +
+                        ", part=" + (part == null ? lostParts.iterator().next() : part) + ']'));
+                }
+            }
+
+            if (qry.nodes().isEmpty())

Review Comment:
   IndexQueryLocalTest#testClientNodeReplicatedCache (running a local index query on a client node) will fail if we don't do this check and throw a certain exception after it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "timoninmaxim (via GitHub)" <gi...@apache.org>.
timoninmaxim commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1580903864


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryAdapter.java:
##########
@@ -607,7 +611,7 @@ private <R> CacheQueryFuture<R> execute0(@Nullable IgniteReducer<T, R> rmtReduce
                 }
             }
 
-            return new GridEmptyCloseableIterator();
+            throw new IgniteException(new ClusterGroupEmptyException());

Review Comment:
   Why do you change the behavior here?



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryAdapter.java:
##########
@@ -579,21 +580,24 @@ private <R> CacheQueryFuture<R> execute0(@Nullable IgniteReducer<T, R> rmtReduce
     }
 
     /** {@inheritDoc} */
-    @Override public GridCloseableIterator executeScanQuery() throws IgniteCheckedException {
-        assert type == SCAN : "Wrong processing of query: " + type;
+    @Override public GridCloseableIterator executeQueryWithIterator() throws IgniteCheckedException {
+        assert type == SCAN || type == INDEX : "Wrong processing of query: " + type;
 
         GridDhtCacheAdapter<?, ?> cacheAdapter = cctx.isNear() ? cctx.near().dht() : cctx.dht();
 
         Set<Integer> lostParts = cacheAdapter.topology().lostPartitions();
 
         if (!lostParts.isEmpty()) {
             if (part == null || lostParts.contains(part)) {
-                throw new CacheException(new CacheInvalidStateException("Failed to execute query because cache partition " +

Review Comment:
   Let's revert these changes



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryAdapter.java:
##########
@@ -625,7 +629,7 @@ private <R> CacheQueryFuture<R> execute0(@Nullable IgniteReducer<T, R> rmtReduce
         GridCloseableIterator it;
 
         if (loc)
-            it = qryMgr.scanQueryLocal(this, true);
+            it = qryMgr.queryLocal(this, true);

Review Comment:
   It looks strange for non-local query we call `scanQueryDistributed` while for local it calls generic method. Let's separate `scanQueryLocal` and `indexQueryLocal`.



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheProxyImpl.java:
##########
@@ -818,8 +840,8 @@ private QueryCursor<Cache.Entry<K, V>> queryContinuous(AbstractContinuousQuery q
                 return (FieldsQueryCursor<R>)ctx.kernalContext().query().querySqlFields(ctx, (SqlFieldsQuery)qry,
                     null, keepBinary, true).get(0);
 
-            if (qry instanceof ScanQuery)
-                return query((ScanQuery)qry, null, projection(qry.isLocal()));
+            if (qry instanceof ScanQuery || (qry instanceof IndexQuery && qry.isLocal()))

Review Comment:
   Looks like `if` for `IndexQuery` can be moved inside the call on the 846L. 



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheProxyImpl.java:
##########
@@ -475,23 +475,45 @@ private <T, R> QueryCursor<R> query(
 
         boolean isKeepBinary = opCtxCall != null && opCtxCall.isKeepBinary();
 
-        IgniteBiPredicate<K, V> p = scanQry.getFilter();
+        GridCacheQueryType qryType = null;
+
+        CacheQuery cacheQry = null;
+
+        if (qry instanceof ScanQuery) {
+            ScanQuery scanQry = (ScanQuery)qry;
+
+            qryType = GridCacheQueryType.SCAN;
+
+            IgniteBiPredicate<K, V> p = scanQry.getFilter();
+
+            cacheQry = ctx.queries().createScanQuery(
+                p, transformer, scanQry.getPartition(), isKeepBinary, qry.isLocal(), null);
+        }
+        else {
+            IndexQuery idxQryLoc = (IndexQuery)qry;

Review Comment:
   This logic is already specified in `IgniteCacheProxyImpl#qyery(Query, ClusterGroup)`. Let's move handle of IndexQuery in the method.



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1411,17 +1411,19 @@ else if (rmvIter)
     }
 
     /**
-     * Process local scan query.
+     * Process local query with results returned in an iterator.
      *
      * @param qry Query.
      * @param updateStatistics Update statistics flag.
      */
     @SuppressWarnings({"unchecked"})
-    protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
+    protected GridCloseableIterator queryLocal(final GridCacheQueryAdapter qry,

Review Comment:
   Let's create 2 methods - `scanQueryLocal` and `indexQueryLocal`. They should reuse some common code.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1622414794


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1499,6 +1502,79 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local index query.
+     *
+     * @param qry Query.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    public GridCloseableIterator indexQueryLocal(final GridCacheQueryAdapter qry) throws IgniteCheckedException {
+        if (!enterBusy())
+            throw new IllegalStateException("Failed to process query request (grid is stopping).");
+
+        try {
+            assert qry.type() == INDEX : "Wrong processing of query: " + qry.type();
+
+            cctx.checkSecurity(SecurityPermission.CACHE_READ);
+
+            GridDhtCacheAdapter<?, ?> cacheAdapter = cctx.isNear() ? cctx.near().dht() : cctx.dht();
+
+            Set<Integer> lostParts = cacheAdapter.topology().lostPartitions();
+
+            Integer part = qry.partition();
+
+            if (!lostParts.isEmpty()) {
+                if (part == null || lostParts.contains(part)) {
+                    throw new CacheException(new CacheInvalidStateException("Failed to execute query because cache partition " +
+                        "has been lost [cacheName=" + cctx.name() +
+                        ", part=" + (part == null ? lostParts.iterator().next() : part) + ']'));
+                }
+            }
+
+            if (qry.nodes().isEmpty())

Review Comment:
   IndexQueryLocalTest#testClientNodeReplicatedCache will fail if we don't throw a certain exception after this check.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1623822379


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheProxyImpl.java:
##########
@@ -557,6 +558,17 @@ else if (query instanceof IndexQuery) {
             if (q.getLimit() > 0)
                 qry.limit(q.getLimit());
 
+            if (query.isLocal()) {
+                final GridCloseableIterator iter = ctx.kernalContext().query().executeQuery(GridCacheQueryType.INDEX,
+                    cacheName, ctx, new IgniteOutClosureX<GridCloseableIterator>() {
+                        @Override public GridCloseableIterator applyx() throws IgniteCheckedException {
+                            return ctx.queries().indexQueryLocal((GridCacheQueryAdapter)qry);
+                        }
+                    }, true);
+
+                return new QueryCursorImpl<>(iter);

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1623822180


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -3427,4 +3503,56 @@ IgniteBiPredicate<K, V> scanFilter() {
             return scanFilter;
         }
     }
+
+    /** */
+    public static final class IndexQueryIterator<K, V> extends GridCloseableIteratorAdapter<Object> {

Review Comment:
   Changed to `private`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "timoninmaxim (via GitHub)" <gi...@apache.org>.
timoninmaxim commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1622336748


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1499,6 +1502,79 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local index query.
+     *
+     * @param qry Query.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    public GridCloseableIterator indexQueryLocal(final GridCacheQueryAdapter qry) throws IgniteCheckedException {
+        if (!enterBusy())
+            throw new IllegalStateException("Failed to process query request (grid is stopping).");
+
+        try {
+            assert qry.type() == INDEX : "Wrong processing of query: " + qry.type();
+
+            cctx.checkSecurity(SecurityPermission.CACHE_READ);
+
+            GridDhtCacheAdapter<?, ?> cacheAdapter = cctx.isNear() ? cctx.near().dht() : cctx.dht();
+
+            Set<Integer> lostParts = cacheAdapter.topology().lostPartitions();
+
+            Integer part = qry.partition();
+
+            if (!lostParts.isEmpty()) {
+                if (part == null || lostParts.contains(part)) {
+                    throw new CacheException(new CacheInvalidStateException("Failed to execute query because cache partition " +
+                        "has been lost [cacheName=" + cctx.name() +
+                        ", part=" + (part == null ? lostParts.iterator().next() : part) + ']'));
+                }
+            }
+
+            if (qry.nodes().isEmpty())
+                throw new IgniteException(new ClusterGroupEmptyException());
+
+            if (cctx.deploymentEnabled())
+                cctx.deploy().registerClasses(qry.scanFilter());

Review Comment:
   This code doesn't covered by tests. For what purpose do you need this code?



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1499,6 +1502,79 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local index query.
+     *
+     * @param qry Query.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    public GridCloseableIterator indexQueryLocal(final GridCacheQueryAdapter qry) throws IgniteCheckedException {
+        if (!enterBusy())
+            throw new IllegalStateException("Failed to process query request (grid is stopping).");
+
+        try {
+            assert qry.type() == INDEX : "Wrong processing of query: " + qry.type();
+
+            cctx.checkSecurity(SecurityPermission.CACHE_READ);
+
+            GridDhtCacheAdapter<?, ?> cacheAdapter = cctx.isNear() ? cctx.near().dht() : cctx.dht();
+
+            Set<Integer> lostParts = cacheAdapter.topology().lostPartitions();
+
+            Integer part = qry.partition();
+
+            if (!lostParts.isEmpty()) {
+                if (part == null || lostParts.contains(part)) {
+                    throw new CacheException(new CacheInvalidStateException("Failed to execute query because cache partition " +
+                        "has been lost [cacheName=" + cctx.name() +
+                        ", part=" + (part == null ? lostParts.iterator().next() : part) + ']'));
+                }
+            }
+
+            if (qry.nodes().isEmpty())

Review Comment:
   This code is already starts on local node, why do we need this check?



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1499,6 +1502,79 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local index query.
+     *
+     * @param qry Query.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    public GridCloseableIterator indexQueryLocal(final GridCacheQueryAdapter qry) throws IgniteCheckedException {
+        if (!enterBusy())
+            throw new IllegalStateException("Failed to process query request (grid is stopping).");
+
+        try {
+            assert qry.type() == INDEX : "Wrong processing of query: " + qry.type();
+
+            cctx.checkSecurity(SecurityPermission.CACHE_READ);
+
+            GridDhtCacheAdapter<?, ?> cacheAdapter = cctx.isNear() ? cctx.near().dht() : cctx.dht();
+
+            Set<Integer> lostParts = cacheAdapter.topology().lostPartitions();
+
+            Integer part = qry.partition();
+
+            if (!lostParts.isEmpty()) {
+                if (part == null || lostParts.contains(part)) {
+                    throw new CacheException(new CacheInvalidStateException("Failed to execute query because cache partition " +
+                        "has been lost [cacheName=" + cctx.name() +
+                        ", part=" + (part == null ? lostParts.iterator().next() : part) + ']'));
+                }
+            }
+
+            if (qry.nodes().isEmpty())
+                throw new IgniteException(new ClusterGroupEmptyException());
+
+            if (cctx.deploymentEnabled())
+                cctx.deploy().registerClasses(qry.scanFilter());
+
+            if (log.isDebugEnabled())
+                log.debug("Running local index query: " + qry);
+
+            if (cctx.events().isRecordable(EVT_CACHE_QUERY_EXECUTED)) {
+                cctx.gridEvents().record(new CacheQueryExecutedEvent<>(
+                    cctx.localNode(),
+                    "Index query executed.",
+                    EVT_CACHE_QUERY_EXECUTED,
+                    CacheQueryType.INDEX.name(),
+                    cctx.name(),
+                    qry.queryClassName(),
+                    null,
+                    qry.scanFilter(),
+                    null,
+                    null,
+                    securitySubjectId(cctx),
+                    cctx.kernalContext().task().resolveTaskName(qry.taskHash())));
+            }
+
+            int[] parts = null;
+
+            if (part != null)
+                parts = new int[] {part};
+
+            IndexQueryResult<K, V> idxQryRes = qryProc.queryIndex(cacheName, qry.queryClassName(), qry.idxQryDesc(),
+                qry.scanFilter(), filter(qry, parts, parts != null), qry.keepBinary(), qry.taskHash());
+
+            return new IndexQueryIterator(idxQryRes.iter());
+        }
+        catch (Exception e) {
+            throw e;

Review Comment:
   Useless catch



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheProxyImpl.java:
##########
@@ -557,6 +558,17 @@ else if (query instanceof IndexQuery) {
             if (q.getLimit() > 0)
                 qry.limit(q.getLimit());
 
+            if (query.isLocal()) {
+                final GridCloseableIterator iter = ctx.kernalContext().query().executeQuery(GridCacheQueryType.INDEX,
+                    cacheName, ctx, new IgniteOutClosureX<GridCloseableIterator>() {
+                        @Override public GridCloseableIterator applyx() throws IgniteCheckedException {
+                            return ctx.queries().indexQueryLocal((GridCacheQueryAdapter)qry);
+                        }
+                    }, true);
+
+                return new QueryCursorImpl<>(iter);

Review Comment:
   `QueryCursorImpl<>` -> `QueryCursorImpl` there is no any class to inject to the generic.



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1499,6 +1502,79 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local index query.
+     *
+     * @param qry Query.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    public GridCloseableIterator indexQueryLocal(final GridCacheQueryAdapter qry) throws IgniteCheckedException {
+        if (!enterBusy())
+            throw new IllegalStateException("Failed to process query request (grid is stopping).");
+
+        try {
+            assert qry.type() == INDEX : "Wrong processing of query: " + qry.type();
+
+            cctx.checkSecurity(SecurityPermission.CACHE_READ);
+
+            GridDhtCacheAdapter<?, ?> cacheAdapter = cctx.isNear() ? cctx.near().dht() : cctx.dht();
+
+            Set<Integer> lostParts = cacheAdapter.topology().lostPartitions();
+
+            Integer part = qry.partition();
+
+            if (!lostParts.isEmpty()) {

Review Comment:
   This code isn't covered with tests. Let's remove it and add test and code in a separate ticket



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -3427,4 +3503,56 @@ IgniteBiPredicate<K, V> scanFilter() {
             return scanFilter;
         }
     }
+
+    /** */
+    public static final class IndexQueryIterator<K, V> extends GridCloseableIteratorAdapter<Object> {

Review Comment:
   private



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1581636215


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryAdapter.java:
##########
@@ -607,7 +611,7 @@ private <R> CacheQueryFuture<R> execute0(@Nullable IgniteReducer<T, R> rmtReduce
                 }
             }
 
-            return new GridEmptyCloseableIterator();
+            throw new IgniteException(new ClusterGroupEmptyException());

Review Comment:
   If this kind of Exception is not thrown here then the IndexQueryLocalTest#testClientNodeReplicatedCache() will fail.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1581721203


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryAdapter.java:
##########
@@ -579,21 +580,24 @@ private <R> CacheQueryFuture<R> execute0(@Nullable IgniteReducer<T, R> rmtReduce
     }
 
     /** {@inheritDoc} */
-    @Override public GridCloseableIterator executeScanQuery() throws IgniteCheckedException {
-        assert type == SCAN : "Wrong processing of query: " + type;
+    @Override public GridCloseableIterator executeQueryWithIterator() throws IgniteCheckedException {
+        assert type == SCAN || type == INDEX : "Wrong processing of query: " + type;
 
         GridDhtCacheAdapter<?, ?> cacheAdapter = cctx.isNear() ? cctx.near().dht() : cctx.dht();
 
         Set<Integer> lostParts = cacheAdapter.topology().lostPartitions();
 
         if (!lostParts.isEmpty()) {
             if (part == null || lostParts.contains(part)) {
-                throw new CacheException(new CacheInvalidStateException("Failed to execute query because cache partition " +

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1591748004


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1434,22 +1439,25 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
             new InternalScanFilter<>(qry.scanFilter()) : null;
 
         try {
-            assert qry.type() == SCAN;
+            assert qry.type() == SCAN || qry.type() == INDEX;
 
             if (log.isDebugEnabled())
-                log.debug("Running local SCAN query: " + qry);
-
-            final String taskName = cctx.kernalContext().task().resolveTaskName(qry.taskHash());
-            final ClusterNode locNode = cctx.localNode();
+                log.debug("Running local " + qry.type() + " query: " + qry);
 
             if (cctx.events().isRecordable(EVT_CACHE_QUERY_EXECUTED)) {
+                final String taskName = cctx.kernalContext().task().resolveTaskName(qry.taskHash());
+
+                final ClusterNode locNode = cctx.localNode();
+
+                final String clsName = qry.queryClassName();
+
                 cctx.gridEvents().record(new CacheQueryExecutedEvent<>(
                     locNode,
-                    "Scan query executed.",
+                    qry.type() + " query executed.",
                     EVT_CACHE_QUERY_EXECUTED,
-                    CacheQueryType.SCAN.name(),
+                    qry.type() == SCAN ? CacheQueryType.SCAN.name() : CacheQueryType.INDEX.name(),
                     namex,
-                    null,
+                    clsName,

Review Comment:
   Before this code was used for scan queries only, so the class name was irrelevant for cache query events and therefore was always ‘null’. Now this code processes both scan and index queries, and with index queries we need to record the name of the queried class to the event.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1588659128


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1483,6 +1487,62 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local scan query.
+     *
+     * @param qry Query.
+     * @param updateStatistics Update statistics flag.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
+        boolean updateStatistics) throws IgniteCheckedException {
+        return queryFunc(() -> {
+            try {
+                IgniteClosure transformer = qry.transform();
+
+                injectResources(transformer);
+
+                return scanIterator(qry, transformer, true);
+            }
+            catch (Exception e) {
+                throw new RuntimeException(e);

Review Comment:
   Got rid of 'try-catch' in scanQueryLocal and indexQueryLocal by using IgniteThrowableSupplier.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1622414794


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1499,6 +1502,79 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local index query.
+     *
+     * @param qry Query.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    public GridCloseableIterator indexQueryLocal(final GridCacheQueryAdapter qry) throws IgniteCheckedException {
+        if (!enterBusy())
+            throw new IllegalStateException("Failed to process query request (grid is stopping).");
+
+        try {
+            assert qry.type() == INDEX : "Wrong processing of query: " + qry.type();
+
+            cctx.checkSecurity(SecurityPermission.CACHE_READ);
+
+            GridDhtCacheAdapter<?, ?> cacheAdapter = cctx.isNear() ? cctx.near().dht() : cctx.dht();
+
+            Set<Integer> lostParts = cacheAdapter.topology().lostPartitions();
+
+            Integer part = qry.partition();
+
+            if (!lostParts.isEmpty()) {
+                if (part == null || lostParts.contains(part)) {
+                    throw new CacheException(new CacheInvalidStateException("Failed to execute query because cache partition " +
+                        "has been lost [cacheName=" + cctx.name() +
+                        ", part=" + (part == null ? lostParts.iterator().next() : part) + ']'));
+                }
+            }
+
+            if (qry.nodes().isEmpty())

Review Comment:
   IndexQueryLocalTest#testClientNodeReplicatedCache will fail if we don't do this check and throw a certain exception after it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1623821570


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1499,6 +1502,79 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local index query.
+     *
+     * @param qry Query.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    public GridCloseableIterator indexQueryLocal(final GridCacheQueryAdapter qry) throws IgniteCheckedException {
+        if (!enterBusy())
+            throw new IllegalStateException("Failed to process query request (grid is stopping).");
+
+        try {
+            assert qry.type() == INDEX : "Wrong processing of query: " + qry.type();
+
+            cctx.checkSecurity(SecurityPermission.CACHE_READ);
+
+            GridDhtCacheAdapter<?, ?> cacheAdapter = cctx.isNear() ? cctx.near().dht() : cctx.dht();
+
+            Set<Integer> lostParts = cacheAdapter.topology().lostPartitions();
+
+            Integer part = qry.partition();
+
+            if (!lostParts.isEmpty()) {

Review Comment:
   Lost partitions check removed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1581721261


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryAdapter.java:
##########
@@ -625,7 +629,7 @@ private <R> CacheQueryFuture<R> execute0(@Nullable IgniteReducer<T, R> rmtReduce
         GridCloseableIterator it;
 
         if (loc)
-            it = qryMgr.scanQueryLocal(this, true);
+            it = qryMgr.queryLocal(this, true);

Review Comment:
   Done.



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1411,17 +1411,19 @@ else if (rmvIter)
     }
 
     /**
-     * Process local scan query.
+     * Process local query with results returned in an iterator.
      *
      * @param qry Query.
      * @param updateStatistics Update statistics flag.
      */
     @SuppressWarnings({"unchecked"})
-    protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
+    protected GridCloseableIterator queryLocal(final GridCacheQueryAdapter qry,

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1591759838


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1458,11 +1466,7 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
                     taskName));
             }
 
-            IgniteClosure transformer = qry.transform();
-
-            injectResources(transformer);
-
-            GridCloseableIterator it = scanIterator(qry, transformer, true);
+            GridCloseableIterator it = qrySupplier.get();
 
             updateStatistics = false;

Review Comment:
   Agreed to leave it as it is and revisit it later on.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1588659128


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1483,6 +1487,62 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local scan query.
+     *
+     * @param qry Query.
+     * @param updateStatistics Update statistics flag.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
+        boolean updateStatistics) throws IgniteCheckedException {
+        return queryFunc(() -> {
+            try {
+                IgniteClosure transformer = qry.transform();
+
+                injectResources(transformer);
+
+                return scanIterator(qry, transformer, true);
+            }
+            catch (Exception e) {
+                throw new RuntimeException(e);

Review Comment:
   Got rid of 'try-catch' in scanQueryLocal and indexQueryLocal by using IgniteThrowableSupplier.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1581721240


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheProxyImpl.java:
##########
@@ -818,8 +840,8 @@ private QueryCursor<Cache.Entry<K, V>> queryContinuous(AbstractContinuousQuery q
                 return (FieldsQueryCursor<R>)ctx.kernalContext().query().querySqlFields(ctx, (SqlFieldsQuery)qry,
                     null, keepBinary, true).get(0);
 
-            if (qry instanceof ScanQuery)
-                return query((ScanQuery)qry, null, projection(qry.isLocal()));
+            if (qry instanceof ScanQuery || (qry instanceof IndexQuery && qry.isLocal()))

Review Comment:
   Done.



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheProxyImpl.java:
##########
@@ -475,23 +475,45 @@ private <T, R> QueryCursor<R> query(
 
         boolean isKeepBinary = opCtxCall != null && opCtxCall.isKeepBinary();
 
-        IgniteBiPredicate<K, V> p = scanQry.getFilter();
+        GridCacheQueryType qryType = null;
+
+        CacheQuery cacheQry = null;
+
+        if (qry instanceof ScanQuery) {
+            ScanQuery scanQry = (ScanQuery)qry;
+
+            qryType = GridCacheQueryType.SCAN;
+
+            IgniteBiPredicate<K, V> p = scanQry.getFilter();
+
+            cacheQry = ctx.queries().createScanQuery(
+                p, transformer, scanQry.getPartition(), isKeepBinary, qry.isLocal(), null);
+        }
+        else {
+            IndexQuery idxQryLoc = (IndexQuery)qry;

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1622414794


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1499,6 +1502,79 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local index query.
+     *
+     * @param qry Query.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    public GridCloseableIterator indexQueryLocal(final GridCacheQueryAdapter qry) throws IgniteCheckedException {
+        if (!enterBusy())
+            throw new IllegalStateException("Failed to process query request (grid is stopping).");
+
+        try {
+            assert qry.type() == INDEX : "Wrong processing of query: " + qry.type();
+
+            cctx.checkSecurity(SecurityPermission.CACHE_READ);
+
+            GridDhtCacheAdapter<?, ?> cacheAdapter = cctx.isNear() ? cctx.near().dht() : cctx.dht();
+
+            Set<Integer> lostParts = cacheAdapter.topology().lostPartitions();
+
+            Integer part = qry.partition();
+
+            if (!lostParts.isEmpty()) {
+                if (part == null || lostParts.contains(part)) {
+                    throw new CacheException(new CacheInvalidStateException("Failed to execute query because cache partition " +
+                        "has been lost [cacheName=" + cctx.name() +
+                        ", part=" + (part == null ? lostParts.iterator().next() : part) + ']'));
+                }
+            }
+
+            if (qry.nodes().isEmpty())

Review Comment:
   The IndexQueryLocalTest#testClientNodeReplicatedCache test (running a local index query on a client node) will fail if we don't do this check and throw a certain exception after it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1623821781


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1499,6 +1502,79 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local index query.
+     *
+     * @param qry Query.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    public GridCloseableIterator indexQueryLocal(final GridCacheQueryAdapter qry) throws IgniteCheckedException {
+        if (!enterBusy())
+            throw new IllegalStateException("Failed to process query request (grid is stopping).");
+
+        try {
+            assert qry.type() == INDEX : "Wrong processing of query: " + qry.type();
+
+            cctx.checkSecurity(SecurityPermission.CACHE_READ);
+
+            GridDhtCacheAdapter<?, ?> cacheAdapter = cctx.isNear() ? cctx.near().dht() : cctx.dht();
+
+            Set<Integer> lostParts = cacheAdapter.topology().lostPartitions();
+
+            Integer part = qry.partition();
+
+            if (!lostParts.isEmpty()) {
+                if (part == null || lostParts.contains(part)) {
+                    throw new CacheException(new CacheInvalidStateException("Failed to execute query because cache partition " +
+                        "has been lost [cacheName=" + cctx.name() +
+                        ", part=" + (part == null ? lostParts.iterator().next() : part) + ']'));
+                }
+            }
+
+            if (qry.nodes().isEmpty())
+                throw new IgniteException(new ClusterGroupEmptyException());
+
+            if (cctx.deploymentEnabled())
+                cctx.deploy().registerClasses(qry.scanFilter());

Review Comment:
   Code removed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "timoninmaxim (via GitHub)" <gi...@apache.org>.
timoninmaxim merged PR #11317:
URL: https://github.com/apache/ignite/pull/11317


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1623822011


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1499,6 +1502,79 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local index query.
+     *
+     * @param qry Query.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    public GridCloseableIterator indexQueryLocal(final GridCacheQueryAdapter qry) throws IgniteCheckedException {
+        if (!enterBusy())
+            throw new IllegalStateException("Failed to process query request (grid is stopping).");
+
+        try {
+            assert qry.type() == INDEX : "Wrong processing of query: " + qry.type();
+
+            cctx.checkSecurity(SecurityPermission.CACHE_READ);
+
+            GridDhtCacheAdapter<?, ?> cacheAdapter = cctx.isNear() ? cctx.near().dht() : cctx.dht();
+
+            Set<Integer> lostParts = cacheAdapter.topology().lostPartitions();
+
+            Integer part = qry.partition();
+
+            if (!lostParts.isEmpty()) {
+                if (part == null || lostParts.contains(part)) {
+                    throw new CacheException(new CacheInvalidStateException("Failed to execute query because cache partition " +
+                        "has been lost [cacheName=" + cctx.name() +
+                        ", part=" + (part == null ? lostParts.iterator().next() : part) + ']'));
+                }
+            }
+
+            if (qry.nodes().isEmpty())
+                throw new IgniteException(new ClusterGroupEmptyException());
+
+            if (cctx.deploymentEnabled())
+                cctx.deploy().registerClasses(qry.scanFilter());
+
+            if (log.isDebugEnabled())
+                log.debug("Running local index query: " + qry);
+
+            if (cctx.events().isRecordable(EVT_CACHE_QUERY_EXECUTED)) {
+                cctx.gridEvents().record(new CacheQueryExecutedEvent<>(
+                    cctx.localNode(),
+                    "Index query executed.",
+                    EVT_CACHE_QUERY_EXECUTED,
+                    CacheQueryType.INDEX.name(),
+                    cctx.name(),
+                    qry.queryClassName(),
+                    null,
+                    qry.scanFilter(),
+                    null,
+                    null,
+                    securitySubjectId(cctx),
+                    cctx.kernalContext().task().resolveTaskName(qry.taskHash())));
+            }
+
+            int[] parts = null;
+
+            if (part != null)
+                parts = new int[] {part};
+
+            IndexQueryResult<K, V> idxQryRes = qryProc.queryIndex(cacheName, qry.queryClassName(), qry.idxQryDesc(),
+                qry.scanFilter(), filter(qry, parts, parts != null), qry.keepBinary(), qry.taskHash());
+
+            return new IndexQueryIterator(idxQryRes.iter());
+        }
+        catch (Exception e) {
+            throw e;

Review Comment:
   Catch section removed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "timoninmaxim (via GitHub)" <gi...@apache.org>.
timoninmaxim commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1627386328


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1499,6 +1500,59 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local index query.
+     *
+     * @param qry Query.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    public GridCloseableIterator indexQueryLocal(final GridCacheQueryAdapter qry) throws IgniteCheckedException {
+        if (!enterBusy())
+            throw new IllegalStateException("Failed to process query request (grid is stopping).");
+
+        try {
+            assert qry.type() == INDEX : "Wrong processing of query: " + qry.type();
+
+            cctx.checkSecurity(SecurityPermission.CACHE_READ);
+
+            if (qry.nodes().isEmpty())

Review Comment:
   If the query is local there is no need to check the nodes. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1591759391


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1434,22 +1439,25 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
             new InternalScanFilter<>(qry.scanFilter()) : null;
 
         try {
-            assert qry.type() == SCAN;
+            assert qry.type() == SCAN || qry.type() == INDEX;
 
             if (log.isDebugEnabled())
-                log.debug("Running local SCAN query: " + qry);
-
-            final String taskName = cctx.kernalContext().task().resolveTaskName(qry.taskHash());

Review Comment:
   Done.



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1434,22 +1439,25 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
             new InternalScanFilter<>(qry.scanFilter()) : null;
 
         try {
-            assert qry.type() == SCAN;
+            assert qry.type() == SCAN || qry.type() == INDEX;
 
             if (log.isDebugEnabled())
-                log.debug("Running local SCAN query: " + qry);
-
-            final String taskName = cctx.kernalContext().task().resolveTaskName(qry.taskHash());
-            final ClusterNode locNode = cctx.localNode();
+                log.debug("Running local " + qry.type() + " query: " + qry);
 
             if (cctx.events().isRecordable(EVT_CACHE_QUERY_EXECUTED)) {
+                final String taskName = cctx.kernalContext().task().resolveTaskName(qry.taskHash());
+
+                final ClusterNode locNode = cctx.localNode();
+
+                final String clsName = qry.queryClassName();
+
                 cctx.gridEvents().record(new CacheQueryExecutedEvent<>(
                     locNode,
-                    "Scan query executed.",
+                    qry.type() + " query executed.",
                     EVT_CACHE_QUERY_EXECUTED,
-                    CacheQueryType.SCAN.name(),
+                    qry.type() == SCAN ? CacheQueryType.SCAN.name() : CacheQueryType.INDEX.name(),

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1591760019


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1483,6 +1487,62 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local scan query.
+     *
+     * @param qry Query.
+     * @param updateStatistics Update statistics flag.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
+        boolean updateStatistics) throws IgniteCheckedException {
+        return queryFunc(() -> {
+            try {
+                IgniteClosure transformer = qry.transform();
+
+                injectResources(transformer);
+
+                return scanIterator(qry, transformer, true);
+            }
+            catch (Exception e) {
+                throw new RuntimeException(e);

Review Comment:
   Got rid of 'try-catch' in scanQueryLocal and indexQueryLocal by using IgniteThrowableSupplier.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1591759242


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/CacheQueryType.java:
##########
@@ -43,5 +43,8 @@ public enum CacheQueryType {
     CONTINUOUS,
 
     /** SPI query. */
-    SPI
+    SPI,
+
+    /** Index query. */
+    INDEX

Review Comment:
   Test added, new functionality for recording index query events added.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-21445 New implementation for local IndexQuery [ignite]

Posted by "timoninmaxim (via GitHub)" <gi...@apache.org>.
timoninmaxim commented on code in PR #11317:
URL: https://github.com/apache/ignite/pull/11317#discussion_r1587206258


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/CacheQueryType.java:
##########
@@ -43,5 +43,8 @@ public enum CacheQueryType {
     CONTINUOUS,
 
     /** SPI query. */
-    SPI
+    SPI,
+
+    /** Index query. */
+    INDEX

Review Comment:
   Looks like now we need a test for new QueryType



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1458,11 +1466,7 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
                     taskName));
             }
 
-            IgniteClosure transformer = qry.transform();
-
-            injectResources(transformer);
-
-            GridCloseableIterator it = scanIterator(qry, transformer, true);
+            GridCloseableIterator it = qrySupplier.get();
 
             updateStatistics = false;

Review Comment:
   This setting doesn't affect anything



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1434,22 +1439,25 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
             new InternalScanFilter<>(qry.scanFilter()) : null;
 
         try {
-            assert qry.type() == SCAN;
+            assert qry.type() == SCAN || qry.type() == INDEX;
 
             if (log.isDebugEnabled())
-                log.debug("Running local SCAN query: " + qry);
-
-            final String taskName = cctx.kernalContext().task().resolveTaskName(qry.taskHash());

Review Comment:
   Please, revert changes that actually does not change anything.



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1434,22 +1439,25 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
             new InternalScanFilter<>(qry.scanFilter()) : null;
 
         try {
-            assert qry.type() == SCAN;
+            assert qry.type() == SCAN || qry.type() == INDEX;
 
             if (log.isDebugEnabled())
-                log.debug("Running local SCAN query: " + qry);
-
-            final String taskName = cctx.kernalContext().task().resolveTaskName(qry.taskHash());
-            final ClusterNode locNode = cctx.localNode();
+                log.debug("Running local " + qry.type() + " query: " + qry);
 
             if (cctx.events().isRecordable(EVT_CACHE_QUERY_EXECUTED)) {
+                final String taskName = cctx.kernalContext().task().resolveTaskName(qry.taskHash());
+
+                final ClusterNode locNode = cctx.localNode();
+
+                final String clsName = qry.queryClassName();
+
                 cctx.gridEvents().record(new CacheQueryExecutedEvent<>(
                     locNode,
-                    "Scan query executed.",
+                    qry.type() + " query executed.",
                     EVT_CACHE_QUERY_EXECUTED,
-                    CacheQueryType.SCAN.name(),
+                    qry.type() == SCAN ? CacheQueryType.SCAN.name() : CacheQueryType.INDEX.name(),

Review Comment:
   `qry.type().name()`?



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1434,22 +1439,25 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
             new InternalScanFilter<>(qry.scanFilter()) : null;
 
         try {
-            assert qry.type() == SCAN;
+            assert qry.type() == SCAN || qry.type() == INDEX;
 
             if (log.isDebugEnabled())
-                log.debug("Running local SCAN query: " + qry);
-
-            final String taskName = cctx.kernalContext().task().resolveTaskName(qry.taskHash());
-            final ClusterNode locNode = cctx.localNode();
+                log.debug("Running local " + qry.type() + " query: " + qry);
 
             if (cctx.events().isRecordable(EVT_CACHE_QUERY_EXECUTED)) {
+                final String taskName = cctx.kernalContext().task().resolveTaskName(qry.taskHash());
+
+                final ClusterNode locNode = cctx.localNode();
+
+                final String clsName = qry.queryClassName();
+
                 cctx.gridEvents().record(new CacheQueryExecutedEvent<>(
                     locNode,
-                    "Scan query executed.",
+                    qry.type() + " query executed.",
                     EVT_CACHE_QUERY_EXECUTED,
-                    CacheQueryType.SCAN.name(),
+                    qry.type() == SCAN ? CacheQueryType.SCAN.name() : CacheQueryType.INDEX.name(),
                     namex,
-                    null,
+                    clsName,

Review Comment:
   Why do you need this change?



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/GridCacheQueryManager.java:
##########
@@ -1483,6 +1487,62 @@ protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
         }
     }
 
+    /**
+     * Process local scan query.
+     *
+     * @param qry Query.
+     * @param updateStatistics Update statistics flag.
+     * @return GridCloseableIterator.
+     */
+    @SuppressWarnings({"unchecked"})
+    protected GridCloseableIterator scanQueryLocal(final GridCacheQueryAdapter qry,
+        boolean updateStatistics) throws IgniteCheckedException {
+        return queryFunc(() -> {
+            try {
+                IgniteClosure transformer = qry.transform();
+
+                injectResources(transformer);
+
+                return scanIterator(qry, transformer, true);
+            }
+            catch (Exception e) {
+                throw new RuntimeException(e);

Review Comment:
   It looks like you change exception class.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org