You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@climate.apache.org by huikyole <gi...@git.apache.org> on 2016/08/02 23:39:06 UTC
[GitHub] climate pull request #381: Climate-827
GitHub user huikyole opened a pull request:
https://github.com/apache/climate/pull/381
Climate-827
- new masking options, 'CORDEX [region_name]', 'us_states', 'countries' and 'user' have been added
- dataset_processor.subset will be updated later.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/huikyole/climate CLIMATE-827
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/climate/pull/381.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #381
----
commit 0cc4b1eac2eab8ffbc9a5ab20c6b2e8c2f34f17c
Author: huikyole <hu...@argo.jpl.nasa.gov>
Date: 2016-08-02T00:36:10Z
Initial commit
commit 02dfdec65adb81cdc659764008045e99f61bbf4d
Author: huikyole <hu...@argo.jpl.nasa.gov>
Date: 2016-08-02T23:35:28Z
CLIMATE-827 - Adding spatial masking options
- new masking options, CORDEX, us_states, countires and user have been added
- dataset_processor.subset will be updated later.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request #381: Climate-827 - Adding spatial masking options
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/climate/pull/381
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate issue #381: Climate-827 - Adding spatial masking options
Posted by huikyole <gi...@git.apache.org>.
Github user huikyole commented on the issue:
https://github.com/apache/climate/pull/381
@agoodm Could you please review this pull request?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate issue #381: Climate-827 - Adding spatial masking options
Posted by OCWJenkins <gi...@git.apache.org>.
Github user OCWJenkins commented on the issue:
https://github.com/apache/climate/pull/381
Merged build triggered. Test Failed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request #381: Climate-827 - Adding spatial masking options
Posted by agoodm <gi...@git.apache.org>.
Github user agoodm commented on a diff in the pull request:
https://github.com/apache/climate/pull/381#discussion_r73263076
--- Diff: ocw/dataset.py ---
@@ -360,20 +363,63 @@ def end(self, value):
self._end = value
- def __str__(self):
- lat_range = "({}, {})".format(self._lat_min, self._lat_max)
- lon_range = "({}, {})".format(self._lon_min, self._lon_max)
- temporal_boundaries = "({}, {})".format(self._start, self._end)
+def shapefile_boundary(boundary_type, region_names):
+ '''
+ :param boundary_type: The type of spatial subset boundary
+ :type boundary_type: :mod:'string'
- formatted_repr = (
- "<Bounds - "
- "lat-range: {}, "
- "lon-range: {}, "
- "temporal_boundaries: {}> "
- )
+ :param region_names: An array of regions for spatial subset
+ :type region_names: :mod:'list'
+ '''
- return formatted_repr.format(
- lat_range,
- lon_range,
- temporal_boundaries,
- )
+ map_read = Basemap()
+ regions =[]
+ if boundary_type == 'us_states':
+ map_read.readshapefile(ocw.__path__[0]+'/shape/usa_states','usa_states')
+ #map_read.readshapefile('/home/huikyole/climate/ocw'+'/shape/usa_states','usa_states')
+ for region_name in region_names:
+ for iregion, region_info in enumerate(map_read.usa_states_info):
+ if region_info['st'] == region_name:
+ regions.append(numpy.array(map_read.usa_states[iregion]))
+ if boundary_type == 'countries':
+ map_read.readshapefile(ocw.__path__[0]+'/shape/countries','countries')
+ #map_read.readshapefile('/home/huikyole/climate/ocw'+'/shape/countries','countries')
+ for region_name in region_names:
+ for iregion, region_info in enumerate(map_read.countries_info):
+ if region_info['COUNTRY'] == region_name:
+ regions.append(numpy.array(map_read.countries[iregion]))
+ return regions
+
--- End diff --
Please add the shapefiles directory to this PR. Without it the unit tests will fail. I will also eventually need to test if this works with conda / pypi packaging since this may require changes to the setup script.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request #381: Climate-827 - Adding spatial masking options
Posted by agoodm <gi...@git.apache.org>.
Github user agoodm commented on a diff in the pull request:
https://github.com/apache/climate/pull/381#discussion_r73261216
--- Diff: ocw/dataset.py ---
@@ -280,57 +296,44 @@ def __init__(self, lat_min, lat_max, lon_min,
else:
self._end = None
- @property
- def lat_min(self):
- return self._lat_min
-
- @lat_min.setter
- def lat_min(self, value):
- if not (-90 <= value <= 90 and value < self._lat_max):
- error = "Attempted to set lat_min to invalid value: %s" % (value)
- logger.error(error)
- raise ValueError(error)
-
- self._lat_min = value
-
- @property
- def lat_max(self):
- return self._lat_max
-
- @lat_max.setter
- def lat_max(self, value):
- if not (-90 <= value <= 90 and value > self._lat_min):
- error = "Attempted to set lat_max to invalid value: %s" % (value)
- logger.error(error)
- raise ValueError(error)
-
- self._lat_max = value
-
- @property
- def lon_min(self):
- return self._lon_min
-
- @lon_min.setter
- def lon_min(self, value):
- if not (-180 <= value <= 180 and value < self._lon_max):
- error = "Attempted to set lon_min to invalid value: %s" % (value)
- logger.error(error)
- raise ValueError(error)
-
- self._lon_min = value
-
- @property
- def lon_max(self):
- return self._lon_max
-
- @lon_max.setter
- def lon_max(self, value):
- if not (-180 <= value <= 180 and value > self._lon_min):
- error = "Attempter to set lon_max to invalid value: %s" % (value)
- logger.error(error)
- raise ValueError(error)
-
- self._lon_max = value
+ if boundary_type == 'us_states':
+ self.masked_regions = shapefile_boundary(boundary_type, us_states)
+ if boundary_type == 'countries':
+ self.masked_regions = shapefile_boundary(boundary_type, countries)
+ if boundary_type == 'user':
+ file_object = netCDF4.Dataset(user_mask_file)
+ self.mask_variable = file_object.variables[mask_variable_name][:]
+ mask_longitude = file_object.variables[longitude_name][:]
+ mask_latitude = file_object.variables[latitude_name][:]
+ if mask_longitude.ndim == 1 and mask_latitude.ndim == 1:
+ self.mask_longitude, self.mask_latitude = numpy.meshgrid(mask_longitude, mask_latitude)
+ elif mask_longitude.ndim == 2 and mask_latitude.ndim == 2:
+ self.mask_longitude = mask_longitude
+ self.mask_latitude = mask_latitude
+ if boundary_type == 'rectangular':
--- End diff --
This block of if statements will lead to an error if the default `lat_min`, `lat_max`, `lon_min`, and `lon_max` values are used (`None`). The easiest solution would be to set their default values to -90., 90., -180., and 180 respectively. Otherwise you will need another set of if-statements to check if each of them are `None`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request #381: Climate-827 - Adding spatial masking options
Posted by huikyole <gi...@git.apache.org>.
Github user huikyole commented on a diff in the pull request:
https://github.com/apache/climate/pull/381#discussion_r73282263
--- Diff: ocw/dataset.py ---
@@ -360,20 +363,63 @@ def end(self, value):
self._end = value
- def __str__(self):
- lat_range = "({}, {})".format(self._lat_min, self._lat_max)
- lon_range = "({}, {})".format(self._lon_min, self._lon_max)
- temporal_boundaries = "({}, {})".format(self._start, self._end)
+def shapefile_boundary(boundary_type, region_names):
+ '''
+ :param boundary_type: The type of spatial subset boundary
+ :type boundary_type: :mod:'string'
- formatted_repr = (
- "<Bounds - "
- "lat-range: {}, "
- "lon-range: {}, "
- "temporal_boundaries: {}> "
- )
+ :param region_names: An array of regions for spatial subset
+ :type region_names: :mod:'list'
+ '''
- return formatted_repr.format(
- lat_range,
- lon_range,
- temporal_boundaries,
- )
+ map_read = Basemap()
+ regions =[]
+ if boundary_type == 'us_states':
+ map_read.readshapefile(ocw.__path__[0]+'/shape/usa_states','usa_states')
+ #map_read.readshapefile('/home/huikyole/climate/ocw'+'/shape/usa_states','usa_states')
+ for region_name in region_names:
+ for iregion, region_info in enumerate(map_read.usa_states_info):
+ if region_info['st'] == region_name:
+ regions.append(numpy.array(map_read.usa_states[iregion]))
+ if boundary_type == 'countries':
+ map_read.readshapefile(ocw.__path__[0]+'/shape/countries','countries')
+ #map_read.readshapefile('/home/huikyole/climate/ocw'+'/shape/countries','countries')
+ for region_name in region_names:
+ for iregion, region_info in enumerate(map_read.countries_info):
+ if region_info['COUNTRY'] == region_name:
+ regions.append(numpy.array(map_read.countries[iregion]))
+ return regions
+
--- End diff --
The shape files have been added in the revised pull request. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate issue #381: Climate-827
Posted by OCWJenkins <gi...@git.apache.org>.
Github user OCWJenkins commented on the issue:
https://github.com/apache/climate/pull/381
Merged build triggered. Test Failed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request #381: Climate-827 - Adding spatial masking options
Posted by huikyole <gi...@git.apache.org>.
Github user huikyole commented on a diff in the pull request:
https://github.com/apache/climate/pull/381#discussion_r73282237
--- Diff: ocw/dataset.py ---
@@ -280,57 +296,44 @@ def __init__(self, lat_min, lat_max, lon_min,
else:
self._end = None
- @property
- def lat_min(self):
- return self._lat_min
-
- @lat_min.setter
- def lat_min(self, value):
- if not (-90 <= value <= 90 and value < self._lat_max):
- error = "Attempted to set lat_min to invalid value: %s" % (value)
- logger.error(error)
- raise ValueError(error)
-
- self._lat_min = value
-
- @property
- def lat_max(self):
- return self._lat_max
-
- @lat_max.setter
- def lat_max(self, value):
- if not (-90 <= value <= 90 and value > self._lat_min):
- error = "Attempted to set lat_max to invalid value: %s" % (value)
- logger.error(error)
- raise ValueError(error)
-
- self._lat_max = value
-
- @property
- def lon_min(self):
- return self._lon_min
-
- @lon_min.setter
- def lon_min(self, value):
- if not (-180 <= value <= 180 and value < self._lon_max):
- error = "Attempted to set lon_min to invalid value: %s" % (value)
- logger.error(error)
- raise ValueError(error)
-
- self._lon_min = value
-
- @property
- def lon_max(self):
- return self._lon_max
-
- @lon_max.setter
- def lon_max(self, value):
- if not (-180 <= value <= 180 and value > self._lon_min):
- error = "Attempter to set lon_max to invalid value: %s" % (value)
- logger.error(error)
- raise ValueError(error)
-
- self._lon_max = value
+ if boundary_type == 'us_states':
+ self.masked_regions = shapefile_boundary(boundary_type, us_states)
+ if boundary_type == 'countries':
+ self.masked_regions = shapefile_boundary(boundary_type, countries)
+ if boundary_type == 'user':
+ file_object = netCDF4.Dataset(user_mask_file)
+ self.mask_variable = file_object.variables[mask_variable_name][:]
+ mask_longitude = file_object.variables[longitude_name][:]
+ mask_latitude = file_object.variables[latitude_name][:]
+ if mask_longitude.ndim == 1 and mask_latitude.ndim == 1:
+ self.mask_longitude, self.mask_latitude = numpy.meshgrid(mask_longitude, mask_latitude)
+ elif mask_longitude.ndim == 2 and mask_latitude.ndim == 2:
+ self.mask_longitude = mask_longitude
+ self.mask_latitude = mask_latitude
+ if boundary_type == 'rectangular':
--- End diff --
This is a great suggestion!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---