You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Jackie-Jiang (via GitHub)" <gi...@apache.org> on 2023/08/21 01:37:40 UTC
[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11389: create h2 tables for result comparison in TPCHQueryIntegrationTest
Jackie-Jiang commented on code in PR #11389:
URL: https://github.com/apache/pinot/pull/11389#discussion_r1299495452
##########
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/TPCHQueryIntegrationTest.java:
##########
@@ -105,28 +108,37 @@ public void setUp()
ClusterIntegrationTestUtils.buildSegmentsFromAvro(Collections.singletonList(dataFile), tableConfig, schema, 0,
tableSegmentDir, tarDir);
uploadSegments(tableName, tarDir);
+ // H2
+ ClusterIntegrationTestUtils.setUpH2TableWithAvro(Collections.singletonList(dataFile), tableName, _h2Connection);
}
}
@Test(dataProvider = "QueryDataProvider")
- public void testTPCHQueries(String query) {
- testQueriesSucceed(query);
+ public void testTPCHQueries(String[] pinotAndH2Queries) throws Exception {
+ testQueriesSucceed(pinotAndH2Queries[0], pinotAndH2Queries[1]);
}
- protected void testQueriesSucceed(String query) {
- ResultSetGroup pinotResultSetGroup = getPinotConnection().execute(query);
+ protected void testQueriesSucceed(String pinotQuery, String h2Query) throws Exception {
+ ResultSetGroup pinotResultSetGroup = getPinotConnection().execute(pinotQuery);
org.apache.pinot.client.ResultSet resultTableResultSet = pinotResultSetGroup.getResultSet(0);
-
if (CollectionUtils.isNotEmpty(pinotResultSetGroup.getExceptions())) {
Assert.fail(String.format(
- "TPC-H query raised exception: %s. query: %s", pinotResultSetGroup.getExceptions().get(0), query));
+ "TPC-H query raised exception: %s. query: %s", pinotResultSetGroup.getExceptions().get(0), pinotQuery));
}
- // TODO: Enable the following 2 assertions after fixing the data so each query returns non-zero rows
- /*
- Assert.assertTrue(resultTableResultSet.getRowCount() > 0,
- String.format("Expected non-zero rows for tpc-h query: %s", query));
- Assert.assertTrue(resultTableResultSet.getColumnCount() > 0,
- String.format("Expected non-zero columns for tpc-h query: %s", query)); */
+
+ int numRows = resultTableResultSet.getRowCount();
+ int numColumns = resultTableResultSet.getColumnCount();
+
+ // h2 response
+ Assert.assertNotNull(_h2Connection);
+ Statement h2statement = _h2Connection.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
+ h2statement.execute(h2Query);
+ ResultSet h2ResultSet = h2statement.getResultSet();
+ // TODO: fix the query parsing issue in Pinot
+ // many queries fail due to "ERROR [Connection] [main] Cannot parse table name from query"
+ // java.lang.ClassCastException:
+ // class org.apache.calcite.sql.SqlSelect cannot be cast to class org.apache.calcite.sql.SqlBasicCall
+ // (org.apache.calcite.sql.SqlSelect and org.apache.calcite.sql.SqlBasicCall are in unnamed module of loader 'app')
Review Comment:
This exception is thrown from controller when trying to get the table names from query, but won't fail the query. It will choose a random broker as the fall back path.
Just running the query in H2 doesn't provide much value. We want to compare the results from Pinot and H2 to ensure the results are expected
##########
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/TPCHQueryIntegrationTest.java:
##########
@@ -105,28 +108,37 @@ public void setUp()
ClusterIntegrationTestUtils.buildSegmentsFromAvro(Collections.singletonList(dataFile), tableConfig, schema, 0,
tableSegmentDir, tarDir);
uploadSegments(tableName, tarDir);
+ // H2
+ ClusterIntegrationTestUtils.setUpH2TableWithAvro(Collections.singletonList(dataFile), tableName, _h2Connection);
}
}
@Test(dataProvider = "QueryDataProvider")
- public void testTPCHQueries(String query) {
- testQueriesSucceed(query);
+ public void testTPCHQueries(String[] pinotAndH2Queries) throws Exception {
+ testQueriesSucceed(pinotAndH2Queries[0], pinotAndH2Queries[1]);
}
- protected void testQueriesSucceed(String query) {
- ResultSetGroup pinotResultSetGroup = getPinotConnection().execute(query);
+ protected void testQueriesSucceed(String pinotQuery, String h2Query) throws Exception {
Review Comment:
I think we should modify pinot query to be sql standard, and use the same query for pinot engine and H2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org