[GitHub] [iotdb] LeiRui opened a new issue, #7738: [UDF-SlidingTimeWindowAccessStrategy]Transformer terminates once it encounters an empty window

[GitBox <gi...@apache.org>]

LeiRui opened a new issue, #7738:
URL: https://github.com/apache/iotdb/issues/7738

   **Describe the bug**
   Given a time series `root.vehicle.d1.s1` of three points: (1,5), (2,15), (8,20),
   use the test UDF `org.apache.iotdb.db.query.udf.example.Counter` with the `SlidingTimeWindow` access strategy,
   and run the query `"select counter(s1,'access'='time','timeInterval'='3') from root.vehicle.d1"`.
   
   In this example, the sliding time windows are: [1,4), [4,7), [7,10).
   The expected result is: (1,2), (8,1). That is, `Counter` is expected to return the time-value pairs that take the timestamp of the first point in the window as `time` and the number of points in the window as `value`.
   But the running result is: (1,2), missing (8,1).
   The reason is that the transformer `terminates`, once it encounters **an empty window**, which is [4,7) in this example.
   
   
   **To Reproduce**
   I used IoTDB on the master branch with the recent commit 529711720673f79f22a7535fc0b2c1d28647ccde.
   Run the following test code to reproduce the bug:
   ```java
   package org.apache.iotdb.db.integration;
   
   import static org.junit.Assert.fail;
   
   import java.sql.Connection;
   import java.sql.ResultSet;
   import java.sql.SQLException;
   import java.sql.Statement;
   import java.util.Locale;
   import org.apache.iotdb.db.query.udf.example.ExampleUDFConstant;
   import org.apache.iotdb.integration.env.ConfigFactory;
   import org.apache.iotdb.integration.env.EnvFactory;
   import org.apache.iotdb.itbase.category.LocalStandaloneTest;
   import org.junit.AfterClass;
   import org.junit.Assert;
   import org.junit.BeforeClass;
   import org.junit.Test;
   import org.junit.experimental.categories.Category;
   
   @Category({LocalStandaloneTest.class})
   public class IoTDBSlidingTimeWindowBugTest {
   
     @BeforeClass
     public static void setUp() throws Exception {
       ConfigFactory.getConfig()
           .setUdfCollectorMemoryBudgetInMB(5)
           .setUdfTransformerMemoryBudgetInMB(5)
           .setUdfReaderMemoryBudgetInMB(5);
       EnvFactory.getEnv().initBeforeClass();
       registerUDF();
       createTimeSeries();
       generateData();
     }
   
     @AfterClass
     public static void tearDown() throws Exception {
       EnvFactory.getEnv().cleanAfterClass();
       ConfigFactory.getConfig()
           .setUdfCollectorMemoryBudgetInMB(100)
           .setUdfTransformerMemoryBudgetInMB(100)
           .setUdfReaderMemoryBudgetInMB(100);
     }
   
     @Test
     public void test1() {
       String sql = String.format("select counter(s1,'%s'='%s','%s'='%s') from root.vehicle.d1",
           ExampleUDFConstant.ACCESS_STRATEGY_KEY,
           ExampleUDFConstant.ACCESS_STRATEGY_SLIDING_TIME,
           ExampleUDFConstant.TIME_INTERVAL_KEY,
           3
       );
   
       try (Connection conn = EnvFactory.getEnv().getConnection();
           Statement statement = conn.createStatement()) {
         ResultSet resultSet = statement.executeQuery(sql);
         int count = 0;
         while (resultSet.next()) {
           System.out.println(resultSet.getString(1) + "," + resultSet.getString(2));
           count++;
         }
         /**
          * expected result:
          * (1,2)
          * (8,1)
          * but right now layerRowWindowReader.next returns false once encountering an empty window
          */
         Assert.assertEquals(2, count);
       } catch (SQLException throwable) {
         fail(throwable.getMessage());
       }
     }
   
     private static void createTimeSeries() {
       try (Connection connection = EnvFactory.getEnv().getConnection();
           Statement statement = connection.createStatement()) {
         statement.execute("SET STORAGE GROUP TO root.vehicle");
         statement.execute("CREATE TIMESERIES root.vehicle.d1.s1 with datatype=INT32,encoding=PLAIN");
         statement.execute("CREATE TIMESERIES root.vehicle.d1.s2 with datatype=INT32,encoding=PLAIN");
       } catch (SQLException throwable) {
         fail(throwable.getMessage());
       }
     }
   
     private static final String insertTemplate =
         "INSERT INTO root.vehicle.d1(timestamp,s1)" + " VALUES(%d,%d)";
   
     private static String[] creationSqls =
         new String[]{
             "SET STORAGE GROUP TO root.vehicle.d0",
             "CREATE TIMESERIES root.vehicle.d0.s0 WITH DATATYPE=INT64",
         };
   
     private static void registerUDF() {
       try (Connection connection = EnvFactory.getEnv().getConnection();
           Statement statement = connection.createStatement()) {
         statement.execute(
             "CREATE FUNCTION counter AS 'org.apache.iotdb.db.query.udf.example.Counter' using FILE 'tmp.jar'");
       } catch (SQLException throwable) {
         throwable.printStackTrace();
         fail(throwable.getMessage());
       }
     }
   
     private static void generateData() {
       try (Connection connection = EnvFactory.getEnv().getConnection();
           Statement statement = connection.createStatement()) {
         statement.execute(String.format(Locale.ENGLISH, insertTemplate, 1, 5));
         statement.execute(String.format(Locale.ENGLISH, insertTemplate, 2, 15));
         statement.execute(String.format(Locale.ENGLISH, insertTemplate, 8, 20));
         statement.execute("FLUSH");
       } catch (Exception e) {
         e.printStackTrace();
       }
     }
   
   }
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [iotdb] LeiRui closed issue #7738: [UDF-SlidingTimeWindowAccessStrategy] Transformer terminates once it encounters an empty window

[GitBox <gi...@apache.org>]

LeiRui closed issue #7738: [UDF-SlidingTimeWindowAccessStrategy] Transformer terminates once it encounters an empty window
URL: https://github.com/apache/iotdb/issues/7738


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [iotdb] lancelly commented on issue #7738: [UDF-SlidingTimeWindowAccessStrategy] Transformer terminates once it encounters an empty window

[GitBox <gi...@apache.org>]

lancelly commented on issue #7738:
URL: https://github.com/apache/iotdb/issues/7738#issuecomment-1318626138

   This issue should be fixed by https://github.com/apache/iotdb/pull/8033


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org