You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@climate.apache.org by huikyole <gi...@git.apache.org> on 2015/05/12 10:22:48 UTC
[GitHub] climate pull request: Climate 634 please take a look at this reque...
GitHub user huikyole opened a pull request:
https://github.com/apache/climate/pull/197
Climate 634 please take a look at this request as a reference.
1. Currently, we cannot load ocw.dataset or ocw.datset_processor library in utils.py. So calc_climatology_season is moved from ocw.utils to ocw_dataset_processor
2. A new module, temporal_subset is added.
3. A new example code to test 'calc_climatology_season' is added.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/huikyole/climate CLIMATE-634
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/climate/pull/197.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #197
----
commit 30d9ff48ce1fc663ed5a4ebac8e23a1a1123cf6d
Author: huikyole <hu...@jpl.nasa.gov>
Date: 2015-05-12T08:15:17Z
To use time information of an OCW dataset object, calc_climatology_season needs to be in dataset_processor.
commit 4d8fee64a2af4280ee90ba23566418703197fc34
Author: huikyole <hu...@jpl.nasa.gov>
Date: 2015-05-12T08:20:11Z
Now the seasonal average calculation works for any OCW dataset objects
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by OCWJenkins <gi...@git.apache.org>.
Github user OCWJenkins commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-101186156
Merged build started. Test Failed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by kwhitehall <gi...@git.apache.org>.
Github user kwhitehall commented on a diff in the pull request:
https://github.com/apache/climate/pull/197#discussion_r30156559
--- Diff: examples/simple_model_to_model_bias_DJF_and_JJA.py ---
@@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import datetime
+from os import path
+import urllib
+
+import numpy as np
+
+import ocw.data_source.local as local
+import ocw.dataset_processor as dsp
+import ocw.evaluation as evaluation
+import ocw.metrics as metrics
+import ocw.plotter as plotter
+import ocw.utils as utils
+
+# File URL leader
+FILE_LEADER = "http://zipper.jpl.nasa.gov/dist/"
+# Two Local Model Files
+FILE_1 = "AFRICA_KNMI-RACMO2.2b_CTL_ERAINT_MM_50km_1989-2008_tasmax.nc"
+FILE_2 = "AFRICA_UC-WRF311_CTL_ERAINT_MM_50km-rg_1989-2008_tasmax.nc"
+# Filename for the output image/plot (without file extension)
+OUTPUT_PLOT = "wrf_bias_compared_to_knmi"
+
+FILE_1_PATH = path.join('/tmp', FILE_1)
+FILE_2_PATH = path.join('/tmp', FILE_2)
+
+if not path.exists(FILE_1_PATH):
+ urllib.urlretrieve(FILE_LEADER + FILE_1, FILE_1_PATH)
+if not path.exists(FILE_2_PATH):
+ urllib.urlretrieve(FILE_LEADER + FILE_2, FILE_2_PATH)
+
+""" Step 1: Load Local NetCDF Files into OCW Dataset Objects """
+print("Loading %s into an OCW Dataset Object" % (FILE_1_PATH,))
+knmi_dataset = local.load_file(FILE_1_PATH, "tasmax")
+print("KNMI_Dataset.values shape: (times, lats, lons) - %s \n" % (knmi_dataset.values.shape,))
+
+print("Loading %s into an OCW Dataset Object" % (FILE_2_PATH,))
+wrf_dataset = local.load_file(FILE_2_PATH, "tasmax")
+print("WRF_Dataset.values shape: (times, lats, lons) - %s \n" % (wrf_dataset.values.shape,))
+
+""" Step 2: Calculate seasonal average """
+print("Calculate seasonal average")
--- End diff --
In this example, to replicate the bug associated with the CLIMATE-634 tix, do below:
knmi_dataset.values, knmi_DJF = utils.calc_climatology_season(month_start=12, month_end=2, dataset=knmi_dataset)
wrf_dataset.values, wrf_DJF = utils.calc_climatology_season(month_start=12, month_end=2, dataset=wrf_dataset)
print("Seasonally averaged KNMI_Dataset.values shape: (times, lats, lons) - %s \n" % (knmi_dataset.values.shape,))
print("Seasonally averaged wrf_Dataset.values shape: (times, lats, lons) - %s \n" % (wrf_dataset.values.shape,))
#JJA works with the current functionality
# knmi_dataset.values, knmi_JJA = utils.calc_climatology_season(month_start=6, month_end=8, dataset=knmi_dataset)
# wrf_dataset.values,wrf_JJA = utils.calc_climatology_season(month_start=6, month_end=8, dataset=wrf_dataset)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by huikyole <gi...@git.apache.org>.
Github user huikyole commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-101436518
Now in the latest commit, utils.calc_climatology_year, utils.calc_climatology_season and utils.calc_climatology_montly can be all replaced by using dataset_processor.temporal_subset and calc_temporal_mean. Also all of the issues in the old calc_climatology_season have been addressed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: Climate 634 please take a look at this reque...
Posted by OCWJenkins <gi...@git.apache.org>.
Github user OCWJenkins commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-101186148
Merged build triggered. Test Failed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by OCWJenkins <gi...@git.apache.org>.
Github user OCWJenkins commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-101435785
Merged build triggered. Test Failed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by OCWJenkins <gi...@git.apache.org>.
Github user OCWJenkins commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-101435802
Merged build started. Test Failed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by OCWJenkins <gi...@git.apache.org>.
Github user OCWJenkins commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-121316745
Merged build triggered. Test Failed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by OCWJenkins <gi...@git.apache.org>.
Github user OCWJenkins commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-101436226
Merged build finished. Test Passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by OCWJenkins <gi...@git.apache.org>.
Github user OCWJenkins commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-121316762
Merged build started. Test Failed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by OCWJenkins <gi...@git.apache.org>.
Github user OCWJenkins commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-101438266
Merged build finished. Test Passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by OCWJenkins <gi...@git.apache.org>.
Github user OCWJenkins commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-101437915
Merged build triggered. Test Failed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by OCWJenkins <gi...@git.apache.org>.
Github user OCWJenkins commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-101186299
Merged build finished. Test Passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by OCWJenkins <gi...@git.apache.org>.
Github user OCWJenkins commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-101437947
Merged build started. Test Failed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by OCWJenkins <gi...@git.apache.org>.
Github user OCWJenkins commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-121317170
Merged build finished. Test Passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by huikyole <gi...@git.apache.org>.
Github user huikyole commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-101361410
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/climate/pull/197
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] climate pull request: CLIMATE-634 please take a look at this reque...
Posted by MJJoyce <gi...@git.apache.org>.
Github user MJJoyce commented on the pull request:
https://github.com/apache/climate/pull/197#issuecomment-101388428
The problem with the movement of calc_climatology_season in the format it was moved in is that it doesn't follow the dataset processor conventions. A dataset_processor function takes an OCW Dataset and some parameters and returns a modified OCW dataset. If you want a function that takes an OCW Dataset and returns some other information then that should be in utils.
Also I'm not seeing how this slide relates to the other comments in this thread, but I think I'm missing something?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---