[GitHub] [geode] DonalEvans commented on a change in pull request #6726: GEODE-9185: Add ZPOPMAX Radish command

[GitBox <gi...@apache.org>]

DonalEvans commented on a change in pull request #6726:
URL: https://github.com/apache/geode/pull/6726#discussion_r679298996



##########
File path: geode-apis-compatible-with-redis/src/main/java/org/apache/geode/redis/internal/data/RedisSortedSet.java
##########
@@ -395,6 +395,31 @@ long zrevrank(byte[] member) {
     return null;
   }
 
+  List<byte[]> zpopmax(Region<RedisKey, RedisData> region, RedisKey key, int count) {
+    Iterator<AbstractOrderedSetEntry> scoresIterator =
+        scoreSet.getIndexRange(scoreSet.size() - 1, count, true);
+    List<byte[]> result = new ArrayList<>();
+
+    if (!scoresIterator.hasNext()) {
+      return result;
+    }
+
+    RemsDeltaInfo deltaInfo = new RemsDeltaInfo();
+    while (scoresIterator.hasNext()) {
+      AbstractOrderedSetEntry entry = scoresIterator.next();
+      scoresIterator.remove();
+      members.remove(entry.member);

Review comment:
       When removing entries from the backing collections, it's necessary to adjust the size in bytes, like we do in `memberRemove()`, otherwise we end up with an inaccurate size calculation. It might be good to introduce a method for this since it's now being done in two places (and will also need to be done in ZPOPMIN):
   ```
   private void adjustSizeInBytesForEntryRemoved(OrderedSetEntry entry) {
     sizeInBytesAdjustment -= entry.getSizeInBytes() + calculateByteArraySize(entry.member);
   }
   ```

##########
File path: geode-apis-compatible-with-redis/src/main/java/org/apache/geode/redis/internal/executor/sortedset/ZPopMaxExecutor.java
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.geode.redis.internal.executor.sortedset;
+
+import java.util.List;
+
+import org.apache.geode.redis.internal.RedisConstants;
+import org.apache.geode.redis.internal.executor.AbstractExecutor;
+import org.apache.geode.redis.internal.executor.RedisResponse;
+import org.apache.geode.redis.internal.netty.Coder;
+import org.apache.geode.redis.internal.netty.Command;
+import org.apache.geode.redis.internal.netty.ExecutionHandlerContext;
+
+public class ZPopMaxExecutor extends AbstractExecutor {
+  @Override
+  public RedisResponse executeCommand(Command command, ExecutionHandlerContext context)
+      throws Exception {
+    RedisSortedSetCommands redisSortedSetCommands = context.getSortedSetCommands();
+
+    List<byte[]> commandElements = command.getProcessedCommand();
+
+    int count = 1;
+    if (commandElements.size() > 2) {
+      try {
+        count = (int) Coder.bytesToLong(commandElements.get(2));

Review comment:
       This should instead be `count = narrowLongToInt(bytesToLong(commandElements.get(2)));` which prevents integer under/overflow if the value passed is outside the range of an int.

##########
File path: geode-apis-compatible-with-redis/src/main/java/org/apache/geode/redis/internal/executor/sortedset/ZPopMaxExecutor.java
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.geode.redis.internal.executor.sortedset;
+
+import java.util.List;
+
+import org.apache.geode.redis.internal.RedisConstants;
+import org.apache.geode.redis.internal.executor.AbstractExecutor;
+import org.apache.geode.redis.internal.executor.RedisResponse;
+import org.apache.geode.redis.internal.netty.Coder;
+import org.apache.geode.redis.internal.netty.Command;
+import org.apache.geode.redis.internal.netty.ExecutionHandlerContext;
+
+public class ZPopMaxExecutor extends AbstractExecutor {
+  @Override
+  public RedisResponse executeCommand(Command command, ExecutionHandlerContext context)
+      throws Exception {

Review comment:
       `Exception` is never thrown from this class, so this can be removed.

##########
File path: geode-apis-compatible-with-redis/src/integrationTest/java/org/apache/geode/redis/internal/executor/sortedset/AbstractZPopMaxIntegrationTest.java
##########
@@ -0,0 +1,157 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.redis.internal.executor.sortedset;
+
+import static org.apache.geode.test.dunit.rules.RedisClusterStartupRule.BIND_ADDRESS;
+import static org.apache.geode.test.dunit.rules.RedisClusterStartupRule.REDIS_CLIENT_TIMEOUT;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import redis.clients.jedis.HostAndPort;
+import redis.clients.jedis.JedisCluster;
+import redis.clients.jedis.Protocol;
+import redis.clients.jedis.Tuple;
+
+import org.apache.geode.redis.RedisIntegrationTest;
+import org.apache.geode.redis.internal.RedisConstants;
+
+public abstract class AbstractZPopMaxIntegrationTest implements RedisIntegrationTest {
+  private JedisCluster jedis;
+
+  @Before
+  public void setUp() {
+    jedis = new JedisCluster(new HostAndPort(BIND_ADDRESS, getPort()), REDIS_CLIENT_TIMEOUT);
+  }
+
+  @After
+  public void tearDown() {
+    flushAll();
+    jedis.close();
+  }
+
+  @Test
+  public void shouldError_givenWrongNumberOfArguments() {
+    assertThatThrownBy(
+        () -> jedis.sendCommand("key", Protocol.Command.ZPOPMAX, "key", "1", "2"))
+            .hasMessageContaining(RedisConstants.ERROR_SYNTAX);
+  }
+
+  @Test
+  public void shouldError_givenWrongNumberFormat() {
+    assertThatThrownBy(
+        () -> jedis.sendCommand("key", Protocol.Command.ZPOPMAX, "key", "wat"))
+            .hasMessageContaining(RedisConstants.ERROR_NOT_INTEGER);
+  }
+
+  @Test
+  public void shouldReturnEmpty_givenNonExistentSortedSet() {
+    assertThat(jedis.zpopmax("unknown", 1)).isEmpty();
+  }
+
+  @Test
+  public void shouldReturnEmpty_givenNegativeCount() {
+    jedis.zadd("key", 1, "player1");
+
+    List<?> result = (List<?>) jedis.sendCommand("key", Protocol.Command.ZPOPMAX, "key", "-1");
+    assertThat(result).isEmpty();
+    assertThat(jedis.zrange("key", 0, 10)).containsExactly("player1");
+  }
+
+  @Test
+  public void shouldReturn_highestLex_whenScoresAreEqual() {
+    jedis.zadd("key", 1, "player1");
+    jedis.zadd("key", 1, "player2");
+    jedis.zadd("key", 1, "player3");
+
+    assertThat(jedis.zpopmax("key").getElement()).isEqualTo("player3");
+    assertThat(jedis.zrange("key", 0, 10))
+        .containsExactlyInAnyOrder("player1", "player2");
+  }
+
+  @Test
+  public void shouldReturn_highestScore() {
+    jedis.zadd("key", 3, "player1");
+    jedis.zadd("key", 2, "player2");
+    jedis.zadd("key", 1, "player3");
+
+    assertThat(jedis.zpopmax("key").getElement()).isEqualTo("player1");
+    assertThat(jedis.zrange("key", 0, 10))
+        .containsExactlyInAnyOrder("player2", "player3");
+  }
+
+  @Test
+  public void withCountShouldReturn_membersInScoreOrder_whenScoresAreDifferent() {
+    List<Tuple> tuples = new ArrayList<>();
+    int count = 10;
+    // Make sure that the results are not somehow dependent on the order of insertion
+    List<Integer> shuffles = makeShuffledList(count);
+    for (int i = 0; i < count; i++) {
+      jedis.zadd("key", count - shuffles.get(i), "player" + shuffles.get(i));
+      tuples.add(new Tuple("player" + i, (double) (count - i)));
+    }
+
+    assertThat(jedis.zpopmax("key", count)).containsExactlyElementsOf(tuples);
+  }
+
+  @Test
+  public void withCountShouldReturn_membersInReverseLexicalOrder_whenScoresAreTheSame() {
+    List<Tuple> tuples = new ArrayList<>();
+    int count = 10;
+    // Make sure that the results are not somehow dependent on the order of insertion
+    List<Integer> shuffles = makeShuffledList(count);
+    for (int i = 0; i < count; i++) {
+      jedis.zadd("key", 1D, "player" + shuffles.get(i));
+      tuples.add(new Tuple("player" + (count - i - 1), 1D));
+    }
+
+    assertThat(jedis.zpopmax("key", count)).containsExactlyElementsOf(tuples);
+  }
+
+  @Test
+  public void shouldReturn_countHighestScores() {
+    for (int i = 0; i < 5; i++) {
+      jedis.zadd("key", i, "player" + i);
+    }
+
+    assertThat(jedis.zpopmax("key", 3))
+        .containsExactlyInAnyOrder(

Review comment:
       I think this should be `containsExectly()`, with the order of the `Tuples` reversed, since the order matters for the returned values.

##########
File path: geode-apis-compatible-with-redis/src/test/java/org/apache/geode/redis/internal/data/RedisSortedSetTest.java
##########
@@ -394,6 +395,69 @@ public void dummyOrderedSetEntryConstructor_setsAppropriateMemberName() {
     assertThat(entry.getMember()).isSameAs(bGREATEST_MEMBER_NAME);
   }
 
+  @Test
+  public void zpopmaxRemovesMemberWithHighestScore() {
+    int originalSize = rangeSortedSet.getSortedSetSize();
+    RedisSortedSet sortedSet = spy(rangeSortedSet);
+    Region<RedisKey, RedisData> region = uncheckedCast(mock(Region.class));
+    RedisKey key = new RedisKey();
+    int count = 1;
+
+    List<byte[]> result = sortedSet.zpopmax(region, key, count);
+    assertThat(result).containsExactly("member12".getBytes(), "2.1".getBytes());
+
+    ArgumentCaptor<RemsDeltaInfo> argumentCaptor = ArgumentCaptor.forClass(RemsDeltaInfo.class);
+    verify(sortedSet).storeChanges(eq(region), eq(key), argumentCaptor.capture());
+    assertThat(argumentCaptor.getValue().getRemoves()).containsExactly("member12".getBytes());
+    assertThat(rangeSortedSet.getSortedSetSize()).isEqualTo(originalSize - count);
+  }
+
+  @Test
+  public void zpopmaxRemovesMembersWithHighestScores_whenCountIsGreaterThanOne() {
+    int originalSize = rangeSortedSet.getSortedSetSize();
+    RedisSortedSet sortedSet = spy(rangeSortedSet);
+    Region<RedisKey, RedisData> region = uncheckedCast(mock(Region.class));
+    RedisKey key = new RedisKey();
+    int count = 3;
+
+    List<byte[]> result = sortedSet.zpopmax(region, key, count);
+    assertThat(result).containsExactlyInAnyOrder("member10".getBytes(), "1.9".getBytes(),
+        "member11".getBytes(), "2".getBytes(), "member12".getBytes(), "2.1".getBytes());
+
+    ArgumentCaptor<RemsDeltaInfo> argumentCaptor = ArgumentCaptor.forClass(RemsDeltaInfo.class);
+    verify(sortedSet).storeChanges(eq(region), eq(key), argumentCaptor.capture());
+    assertThat(argumentCaptor.getValue().getRemoves()).containsExactlyInAnyOrder(
+        "member10".getBytes(), "member11".getBytes(), "member12".getBytes());
+    assertThat(rangeSortedSet.getSortedSetSize()).isEqualTo(originalSize - count);
+  }
+
+  @Test
+  public void zpopmaxRemovesRegionEntryWhenSetBecomesEmpty() {
+    RedisSortedSet sortedSet = spy(createRedisSortedSet(score1, member1));
+    Region<RedisKey, RedisData> region = uncheckedCast(mock(Region.class));
+    RedisKey key = new RedisKey();
+
+    List<byte[]> result = sortedSet.zpopmax(region, key, 1);
+    assertThat(result).containsExactly(member1.getBytes(), score1.getBytes());
+
+    verify(sortedSet).storeChanges(eq(region), eq(key), any(RemsDeltaInfo.class));
+    verify(region).remove(key);
+  }
+
+  @Test
+  public void zpopmaxRemovesHighestLexWhenScoresAreEqual() {

Review comment:
       Could we also add a test to confirm that the size reported by `RedisSortedSet.getSizeInBytes()` is accurate after calling `zpopmax()` please? There is an existing test, `redisSortedSetGetSizeInBytes_isAccurateForAddsUpdatesAndRemoves()` that could be modified, or possibly broken up into separate test cases since it's already a bit big.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org