[jira] [Created] (SDAP-343) Fix reading of time stamp in GPM data

["Joseph C. Jacob (Jira)" <ji...@apache.org>]

Joseph C. Jacob created SDAP-343:
------------------------------------

             Summary: Fix reading of time stamp in GPM data
                 Key: SDAP-343
                 URL: https://issues.apache.org/jira/browse/SDAP-343
             Project: Apache Science Data Analytics Platform
          Issue Type: Task
          Components: granule-ingester
            Reporter: Joseph C. Jacob


The GPM IMERG Early Precipitation L3 1 day 0.1 degree x 0.1 degree V06 (GPM_3IMERGDE) dataset ([https://disc.gsfc.nasa.gov/datasets/GPM_3IMERGDE_06/summary?keywords=GPM)] gives an error upon ingest:  
{quote}{{granule_ingester.exceptions.Exceptions.TileProcessingError: Could not generate tiles from the granule because of the following error: unsupported operand type(s) for /: 'cftime._cftime.DatetimeJulian' and 'float'.}}{quote}
{{This appears to be related to how the time variable is read.  The time variable is given as: }}
       double time(time) ;
              time:units = "days since 1970-01-01 00:00:00Z" ;
              time:standard_name = "time" ;
              time:calendar = "julian" ;
              time:bounds = "time_bnds" ;
              time:origname = "time" ;
              time:fullnamepath = "/time" ;
       double time_bnds(time, nv) ;
              time_bnds:units = "days since 1970-01-01 00:00:00Z" ;
              time_bnds:coordinates = "time nv" ;
              time_bnds:origname = "time_bnds" ;
              time_bnds:fullnamepath = "/time_bnds" ;
{{The new SDAP ingester uses xarray to read the NetCDF files.  Xarray tries to force conversion to a datetime64 object if possible, but seems to have been unable to do so in this case (maybe related to the Julian calendar being used?).  }}

{{The old legacy ningester was able to read GPM in the past (during the OceanWorks project).  A notable difference is that the old ningester used the NetCDF4 module instead of xarray.}}

In this ticket we need to determine if xarray can be used correctly for this dataset, and if not, we need to revert back to using NetCDF4.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)