You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@geode.apache.org by GitBox <gi...@apache.org> on 2020/06/19 21:44:08 UTC
[GitHub] [geode] boglesby opened a new pull request #5279: GEODE-8287: Added another condition to cause the indexed value to be …
boglesby opened a new pull request #5279:
URL: https://github.com/apache/geode/pull/5279
…used
Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?
- [ ] Has your PR been rebased against the latest commit within the target branch (typically `develop`)?
- [ ] Is your initial contribution a single, squashed commit?
- [ ] Does `gradlew build` run cleanly?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
### Note:
Please ensure that once the PR is submitted, check Concourse for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to dev@geode.apache.org.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] boglesby commented on a change in pull request #5279: GEODE-8287: Added another condition to cause the indexed value to be …
Posted by GitBox <gi...@apache.org>.
boglesby commented on a change in pull request #5279:
URL: https://github.com/apache/geode/pull/5279#discussion_r445033163
##########
File path: geode-core/src/distributedTest/java/org/apache/geode/cache/query/dunit/IndexUsageInJoinQueryDistributedTest.java
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.cache.query.dunit;
+
+import static org.apache.geode.cache.RegionShortcut.REPLICATE;
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.Serializable;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Map;
+
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+import org.apache.geode.cache.Cache;
+import org.apache.geode.cache.Region;
+import org.apache.geode.cache.client.ClientCache;
+import org.apache.geode.cache.client.ClientRegionShortcut;
+import org.apache.geode.cache.query.IndexExistsException;
+import org.apache.geode.cache.query.IndexNameConflictException;
+import org.apache.geode.cache.query.Query;
+import org.apache.geode.cache.query.QueryService;
+import org.apache.geode.cache.query.RegionNotFoundException;
+import org.apache.geode.distributed.internal.DistributionStats;
+import org.apache.geode.distributed.internal.InternalDistributedSystem;
+import org.apache.geode.pdx.PdxReader;
+import org.apache.geode.pdx.PdxSerializable;
+import org.apache.geode.pdx.PdxWriter;
+import org.apache.geode.test.dunit.rules.ClientVM;
+import org.apache.geode.test.dunit.rules.ClusterStartupRule;
+import org.apache.geode.test.dunit.rules.MemberVM;
+import org.apache.geode.test.junit.categories.OQLIndexTest;
+import org.apache.geode.test.junit.rules.serializable.SerializableTestName;
+
+@Category({OQLIndexTest.class})
+public class IndexUsageInJoinQueryDistributedTest implements Serializable {
+
+ private MemberVM locator;
+
+ private MemberVM server;
+
+ private ClientVM client;
+
+ private static final String PRODUCT_REGION_NAME = "product";
+
+ private static final String INSTRUMENT_REGION_NAME = "instrument";
+
+ @Rule
+ public ClusterStartupRule clusterRule = new ClusterStartupRule();
+
+ @Rule
+ public SerializableTestName testName = new SerializableTestName();
+
+ @Test
+ public void testSecondIndexUsedWhenExecutingJoinQuery() throws Exception {
+ // Start Locator
+ locator = clusterRule.startLocatorVM(0);
+
+ // Start server
+ server = clusterRule.startServerVM(1, locator.getPort());
+
+ // Create server regions
+ server.invoke(() -> createServerRegionsAndIndexes());
+
+ // Start client
+ client = clusterRule.startClientVM(2, c -> c.withLocatorConnection(locator.getPort()));
+
+ // Create client regions
+ client.invoke(() -> createClientRegions());
+
+ // Load regions
+ int numProducts = 1000;
+ int numInstruments = 100;
+ client.invoke(() -> loadRegions(numProducts, numInstruments));
+
+ // Get number of deserializations before query
+ long deserializationsBeforeQuery = server.invoke(() -> getDeserializations());
+
+ // Execute query
+ client.invoke(() -> executeQuery(numInstruments));
+
+ // Get number of deserializations after query
+ long deserializationsAfterQuery = server.invoke(() -> getDeserializations());
+
+ // Verify number of deserializations during query
+ // 1 for each instrument in the instrumentTypesIndex (100)
+ // For each instrument, 1 product in the productIdKeyIndex (100)
+ // 1 for the bind parameter
+ assertThat(deserializationsAfterQuery - deserializationsBeforeQuery)
+ .isEqualTo((numInstruments * 2) + 1);
+ }
+
+ private void createServerRegionsAndIndexes()
+ throws RegionNotFoundException, IndexNameConflictException, IndexExistsException {
+ Cache cache = ClusterStartupRule.getCache();
+ QueryService queryService = cache.getQueryService();
+ Region productRegion = cache.createRegionFactory(REPLICATE).create(PRODUCT_REGION_NAME);
+ queryService.createKeyIndex("productIdKeyIndex", "productId", productRegion.getFullPath());
+ Region instrumentRegion = cache.createRegionFactory(REPLICATE).create(INSTRUMENT_REGION_NAME);
+ queryService.createIndex("instrumentTypesIndex", "types['TYPE']",
+ instrumentRegion.getFullPath());
+ }
+
+ private void createClientRegions() {
+ ClientCache cache = ClusterStartupRule.getClientCache();
+ cache.createClientRegionFactory(ClientRegionShortcut.PROXY).create(PRODUCT_REGION_NAME);
+ cache.createClientRegionFactory(ClientRegionShortcut.PROXY).create(INSTRUMENT_REGION_NAME);
+ }
+
+ private void loadRegions(int numProducts, int numInstruments) {
+ ClientCache cache = ClusterStartupRule.getClientCache();
+ Region productRegion = cache.getRegion(PRODUCT_REGION_NAME);
+ Region instrumentRegion = cache.getRegion(INSTRUMENT_REGION_NAME);
+
+ // Load product region
+ for (int i = 0; i < numProducts; i++) {
+ String productId = String.valueOf(i);
+ Product product = new Product(productId);
+ productRegion.put(productId, product);
+ }
+
+ // Load instrument region
+ for (int i = 0; i < numInstruments; i++) {
+ String instrumentId = String.valueOf(i);
+ Map<String, String> types = new HashMap<>();
+ types.put("TYPE", "TYPE1");
+ Instrument instrument = new Instrument(instrumentId, "0", types);
+ instrumentRegion.put(instrumentId, instrument);
+ }
+ }
+
+ private void executeQuery(int numInstruments) throws Exception {
+ String queryString =
+ "<trace> select i,p from /instrument i, /product p where i.productId = p.productId and i.types['TYPE']=$1";
Review comment:
Done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] boglesby commented on a change in pull request #5279: GEODE-8287: Added another condition to cause the indexed value to be …
Posted by GitBox <gi...@apache.org>.
boglesby commented on a change in pull request #5279:
URL: https://github.com/apache/geode/pull/5279#discussion_r445025441
##########
File path: geode-core/src/distributedTest/java/org/apache/geode/cache/query/dunit/IndexUsageInJoinQueryDistributedTest.java
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.cache.query.dunit;
+
+import static org.apache.geode.cache.RegionShortcut.REPLICATE;
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.Serializable;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Map;
+
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+import org.apache.geode.cache.Cache;
+import org.apache.geode.cache.Region;
+import org.apache.geode.cache.client.ClientCache;
+import org.apache.geode.cache.client.ClientRegionShortcut;
+import org.apache.geode.cache.query.IndexExistsException;
+import org.apache.geode.cache.query.IndexNameConflictException;
+import org.apache.geode.cache.query.Query;
+import org.apache.geode.cache.query.QueryService;
+import org.apache.geode.cache.query.RegionNotFoundException;
+import org.apache.geode.distributed.internal.DistributionStats;
+import org.apache.geode.distributed.internal.InternalDistributedSystem;
+import org.apache.geode.pdx.PdxReader;
+import org.apache.geode.pdx.PdxSerializable;
+import org.apache.geode.pdx.PdxWriter;
+import org.apache.geode.test.dunit.rules.ClientVM;
+import org.apache.geode.test.dunit.rules.ClusterStartupRule;
+import org.apache.geode.test.dunit.rules.MemberVM;
+import org.apache.geode.test.junit.categories.OQLIndexTest;
+import org.apache.geode.test.junit.rules.serializable.SerializableTestName;
+
+@Category({OQLIndexTest.class})
+public class IndexUsageInJoinQueryDistributedTest implements Serializable {
+
+ private MemberVM locator;
+
+ private MemberVM server;
+
+ private ClientVM client;
+
+ private static final String PRODUCT_REGION_NAME = "product";
+
+ private static final String INSTRUMENT_REGION_NAME = "instrument";
+
+ @Rule
+ public ClusterStartupRule clusterRule = new ClusterStartupRule();
+
+ @Rule
+ public SerializableTestName testName = new SerializableTestName();
+
+ @Test
+ public void testSecondIndexUsedWhenExecutingJoinQuery() throws Exception {
+ // Start Locator
+ locator = clusterRule.startLocatorVM(0);
+
+ // Start server
+ server = clusterRule.startServerVM(1, locator.getPort());
+
+ // Create server regions
+ server.invoke(() -> createServerRegionsAndIndexes());
+
+ // Start client
+ client = clusterRule.startClientVM(2, c -> c.withLocatorConnection(locator.getPort()));
+
+ // Create client regions
+ client.invoke(() -> createClientRegions());
+
+ // Load regions
+ int numProducts = 1000;
+ int numInstruments = 100;
+ client.invoke(() -> loadRegions(numProducts, numInstruments));
+
+ // Get number of deserializations before query
+ long deserializationsBeforeQuery = server.invoke(() -> getDeserializations());
+
+ // Execute query
+ client.invoke(() -> executeQuery(numInstruments));
+
+ // Get number of deserializations after query
+ long deserializationsAfterQuery = server.invoke(() -> getDeserializations());
+
+ // Verify number of deserializations during query
+ // 1 for each instrument in the instrumentTypesIndex (100)
+ // For each instrument, 1 product in the productIdKeyIndex (100)
+ // 1 for the bind parameter
+ assertThat(deserializationsAfterQuery - deserializationsBeforeQuery)
+ .isEqualTo((numInstruments * 2) + 1);
+ }
+
+ private void createServerRegionsAndIndexes()
+ throws RegionNotFoundException, IndexNameConflictException, IndexExistsException {
+ Cache cache = ClusterStartupRule.getCache();
+ QueryService queryService = cache.getQueryService();
+ Region productRegion = cache.createRegionFactory(REPLICATE).create(PRODUCT_REGION_NAME);
+ queryService.createKeyIndex("productIdKeyIndex", "productId", productRegion.getFullPath());
+ Region instrumentRegion = cache.createRegionFactory(REPLICATE).create(INSTRUMENT_REGION_NAME);
+ queryService.createIndex("instrumentTypesIndex", "types['TYPE']",
+ instrumentRegion.getFullPath());
+ }
+
+ private void createClientRegions() {
+ ClientCache cache = ClusterStartupRule.getClientCache();
+ cache.createClientRegionFactory(ClientRegionShortcut.PROXY).create(PRODUCT_REGION_NAME);
+ cache.createClientRegionFactory(ClientRegionShortcut.PROXY).create(INSTRUMENT_REGION_NAME);
+ }
+
+ private void loadRegions(int numProducts, int numInstruments) {
+ ClientCache cache = ClusterStartupRule.getClientCache();
+ Region productRegion = cache.getRegion(PRODUCT_REGION_NAME);
+ Region instrumentRegion = cache.getRegion(INSTRUMENT_REGION_NAME);
+
+ // Load product region
+ for (int i = 0; i < numProducts; i++) {
+ String productId = String.valueOf(i);
+ Product product = new Product(productId);
+ productRegion.put(productId, product);
+ }
+
+ // Load instrument region
+ for (int i = 0; i < numInstruments; i++) {
+ String instrumentId = String.valueOf(i);
+ Map<String, String> types = new HashMap<>();
+ types.put("TYPE", "TYPE1");
+ Instrument instrument = new Instrument(instrumentId, "0", types);
+ instrumentRegion.put(instrumentId, instrument);
+ }
+ }
+
+ private void executeQuery(int numInstruments) throws Exception {
+ String queryString =
+ "<trace> select i,p from /instrument i, /product p where i.productId = p.productId and i.types['TYPE']=$1";
Review comment:
Yes, thanks. I'll clean that up.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] nabarunnag commented on a change in pull request #5279: GEODE-8287: Added another condition to cause the indexed value to be …
Posted by GitBox <gi...@apache.org>.
nabarunnag commented on a change in pull request #5279:
URL: https://github.com/apache/geode/pull/5279#discussion_r445014814
##########
File path: geode-core/src/distributedTest/java/org/apache/geode/cache/query/dunit/IndexUsageInJoinQueryDistributedTest.java
##########
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.cache.query.dunit;
+
+import static org.apache.geode.cache.RegionShortcut.REPLICATE;
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.Serializable;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Map;
+
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+import org.apache.geode.cache.Cache;
+import org.apache.geode.cache.Region;
+import org.apache.geode.cache.client.ClientCache;
+import org.apache.geode.cache.client.ClientRegionShortcut;
+import org.apache.geode.cache.query.IndexExistsException;
+import org.apache.geode.cache.query.IndexNameConflictException;
+import org.apache.geode.cache.query.Query;
+import org.apache.geode.cache.query.QueryService;
+import org.apache.geode.cache.query.RegionNotFoundException;
+import org.apache.geode.distributed.internal.DistributionStats;
+import org.apache.geode.distributed.internal.InternalDistributedSystem;
+import org.apache.geode.pdx.PdxReader;
+import org.apache.geode.pdx.PdxSerializable;
+import org.apache.geode.pdx.PdxWriter;
+import org.apache.geode.test.dunit.rules.ClientVM;
+import org.apache.geode.test.dunit.rules.ClusterStartupRule;
+import org.apache.geode.test.dunit.rules.MemberVM;
+import org.apache.geode.test.junit.categories.OQLIndexTest;
+import org.apache.geode.test.junit.rules.serializable.SerializableTestName;
+
+@Category({OQLIndexTest.class})
+public class IndexUsageInJoinQueryDistributedTest implements Serializable {
+
+ private MemberVM locator;
+
+ private MemberVM server;
+
+ private ClientVM client;
+
+ private static final String PRODUCT_REGION_NAME = "product";
+
+ private static final String INSTRUMENT_REGION_NAME = "instrument";
+
+ @Rule
+ public ClusterStartupRule clusterRule = new ClusterStartupRule();
+
+ @Rule
+ public SerializableTestName testName = new SerializableTestName();
+
+ @Test
+ public void testSecondIndexUsedWhenExecutingJoinQuery() throws Exception {
+ // Start Locator
+ locator = clusterRule.startLocatorVM(0);
+
+ // Start server
+ server = clusterRule.startServerVM(1, locator.getPort());
+
+ // Create server regions
+ server.invoke(() -> createServerRegionsAndIndexes());
+
+ // Start client
+ client = clusterRule.startClientVM(2, c -> c.withLocatorConnection(locator.getPort()));
+
+ // Create client regions
+ client.invoke(() -> createClientRegions());
+
+ // Load regions
+ int numProducts = 1000, numInstruments = 100;
Review comment:
There was some discussion on the dev channel to avoid single line multiple variable declarations. Maybe, can split it up
```suggestion
int numProducts = 1000;
int numInstruments = 100;
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] boglesby merged pull request #5279: GEODE-8287: Added another condition to cause the indexed value to be …
Posted by GitBox <gi...@apache.org>.
boglesby merged pull request #5279:
URL: https://github.com/apache/geode/pull/5279
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org