[GitHub] [drill] arina-ielchiieva commented on a change in pull request #1978: DRILL-7578: HDF5 Metadata Queries Fail with Large Files

[GitBox <gi...@apache.org>]

arina-ielchiieva commented on a change in pull request #1978: DRILL-7578: HDF5 Metadata Queries Fail with Large Files
URL: https://github.com/apache/drill/pull/1978#discussion_r383908562
 
 

 ##########
 File path: contrib/format-hdf5/src/main/java/org/apache/drill/exec/store/hdf5/HDF5BatchReader.java
 ##########
 @@ -467,6 +503,12 @@ private void projectDataset(RowSetLoader rowWriter, String datapath) {
     String fieldName = HDF5Utils.getNameFromPath(datapath);
     IHDF5Reader reader = hdf5Reader;
     HDF5DataSetInformation dsInfo = reader.object().getDataSetInformation(datapath);
+
+    // If the dataset is larger than 16MB, do not project the dataset
+    if (dsInfo.getSize() > 16777216) {
 
 Review comment:
   Use constant please instead of number, this would be more readable.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services