[GitHub] Agirish commented on a change in pull request #1489: DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs

[GitBox <gi...@apache.org>]

Agirish commented on a change in pull request #1489: DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs
URL: https://github.com/apache/drill/pull/1489#discussion_r222780156
 
 

 ##########
 File path: contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/JsonTableGroupScan.java
 ##########
 @@ -148,11 +150,16 @@ private void init() {
       table = MapRDB.getTable(scanSpec.getTableName());
       tabletInfos = table.getTabletInfos(scanSpec.getCondition());
 
-      // Calculate totalRowCount for the table from tabletInfos estimatedRowCount.
-      // This will avoid calling expensive MapRDBTableStats API to get total rowCount, avoiding
-      // duplicate work and RPCs to MapR DB server.
-      for (TabletInfo tabletInfo : tabletInfos) {
-        totalRowCount += tabletInfo.getEstimatedNumRows();
+      final MetaTable metaTable = table.getMetaTable();
+      com.mapr.db.scan.ScanStats stats = (scanSpec.getCondition() == null)
+              ? metaTable.getScanStats() : metaTable.getScanStats(scanSpec.getCondition());
+      totalRowCount = stats.getEstimatedNumRows();
+      // MapRDB client can return invalid rowCount i.e. 0, especially right after table
+      // creation. It takes 15 minutes before table stats are obtained and cached in client.
 
 Review comment:
   I think it takes 5 min now instead of 15. Need to confirm.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services