You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@climate.apache.org by jo...@apache.org on 2014/06/04 23:23:24 UTC
[01/12] git commit: CLIMATE-452 - Add Unity desktop option to VM Build
Repository: climate
Updated Branches:
refs/heads/master e9437d04e -> 0f6fad6d1
CLIMATE-452 - Add Unity desktop option to VM Build
- Add Ubuntu Unity option to OCW VM Vagrant build. By default XFCE is
installed. If a user wants Unity, they should comment out the XFCE
section and uncomment the Unity section when building the machine. It
should be noted that the Unity install takes longer, but it comes with
a lot more stuff out of the box (including looking nicer in general).
Project: http://git-wip-us.apache.org/repos/asf/climate/repo
Commit: http://git-wip-us.apache.org/repos/asf/climate/commit/2dcec88c
Tree: http://git-wip-us.apache.org/repos/asf/climate/tree/2dcec88c
Diff: http://git-wip-us.apache.org/repos/asf/climate/diff/2dcec88c
Branch: refs/heads/master
Commit: 2dcec88cb026acf28d5652c95f3a142e0e900c73
Parents: e9437d0
Author: Michael Joyce <jo...@apache.org>
Authored: Tue Jun 3 12:18:38 2014 -0700
Committer: Michael Joyce <jo...@apache.org>
Committed: Tue Jun 3 12:18:38 2014 -0700
----------------------------------------------------------------------
ocw-vm/init-ocw-vm.sh | 5 +++++
1 file changed, 5 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/climate/blob/2dcec88c/ocw-vm/init-ocw-vm.sh
----------------------------------------------------------------------
diff --git a/ocw-vm/init-ocw-vm.sh b/ocw-vm/init-ocw-vm.sh
index 720775c..354d71e 100755
--- a/ocw-vm/init-ocw-vm.sh
+++ b/ocw-vm/init-ocw-vm.sh
@@ -32,10 +32,15 @@ sudo apt-get install -y vim
# GUI related installs
sudo apt-get install -y lightdm
+
+# XFCE
sudo apt-get install -y xfce4
sudo apt-get install -y xdg-utils
sudo apt-get install -y eog
+# Ubuntu Unity
+#sudo apt-get install -y ubuntu-desktop
+
# Use the Easy-OCW Ubuntu install script to get everything
# else installed!
git clone http://git-wip-us.apache.org/repos/asf/climate.git
[12/12] git commit: Resolve CLIMATE-457. Merge PR #60.
Posted by jo...@apache.org.
Resolve CLIMATE-457. Merge PR #60.
Project: http://git-wip-us.apache.org/repos/asf/climate/repo
Commit: http://git-wip-us.apache.org/repos/asf/climate/commit/0f6fad6d
Tree: http://git-wip-us.apache.org/repos/asf/climate/tree/0f6fad6d
Diff: http://git-wip-us.apache.org/repos/asf/climate/diff/0f6fad6d
Branch: refs/heads/master
Commit: 0f6fad6d18513de44fd838b92d7502649c44557a
Parents: eadf3d4 8704567
Author: Michael Joyce <jo...@apache.org>
Authored: Wed Jun 4 14:21:26 2014 -0700
Committer: Michael Joyce <jo...@apache.org>
Committed: Wed Jun 4 14:21:26 2014 -0700
----------------------------------------------------------------------
easy-ocw/install-ubuntu-12_04.sh | 10 ++++++++++
1 file changed, 10 insertions(+)
----------------------------------------------------------------------
[07/12] git commit: Resolve CLIMATE-452. Merge PR #54.
Posted by jo...@apache.org.
Resolve CLIMATE-452. Merge PR #54.
Project: http://git-wip-us.apache.org/repos/asf/climate/repo
Commit: http://git-wip-us.apache.org/repos/asf/climate/commit/e8a8589b
Tree: http://git-wip-us.apache.org/repos/asf/climate/tree/e8a8589b
Diff: http://git-wip-us.apache.org/repos/asf/climate/diff/e8a8589b
Branch: refs/heads/master
Commit: e8a8589bc0a8ffa6c479ff0cfe49aed150fec209
Parents: e9437d0 2dcec88
Author: Michael Joyce <jo...@apache.org>
Authored: Wed Jun 4 14:16:53 2014 -0700
Committer: Michael Joyce <jo...@apache.org>
Committed: Wed Jun 4 14:16:53 2014 -0700
----------------------------------------------------------------------
ocw-vm/init-ocw-vm.sh | 5 +++++
1 file changed, 5 insertions(+)
----------------------------------------------------------------------
[03/12] git commit: CLIMATE-454 - Copy over install dependencies
during VM build
Posted by jo...@apache.org.
CLIMATE-454 - Copy over install dependencies during VM build
- The requirements files for pip and conda weren't being copied over
during the VM build which was causing it to fail. The requirements
files for both pip and conda are copied over now even though the conda
one isn't used at the moment. In the future, it is planned that conda
will be used instead of downloading an Anaconda Python distribution.
Project: http://git-wip-us.apache.org/repos/asf/climate/repo
Commit: http://git-wip-us.apache.org/repos/asf/climate/commit/3fd3478a
Tree: http://git-wip-us.apache.org/repos/asf/climate/tree/3fd3478a
Diff: http://git-wip-us.apache.org/repos/asf/climate/diff/3fd3478a
Branch: refs/heads/master
Commit: 3fd3478a170d7f94166b6d1253d50608c866739b
Parents: e9437d0
Author: Michael Joyce <jo...@apache.org>
Authored: Wed Jun 4 11:13:49 2014 -0700
Committer: Michael Joyce <jo...@apache.org>
Committed: Wed Jun 4 11:13:49 2014 -0700
----------------------------------------------------------------------
ocw-vm/init-ocw-vm.sh | 5 +++++
1 file changed, 5 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/climate/blob/3fd3478a/ocw-vm/init-ocw-vm.sh
----------------------------------------------------------------------
diff --git a/ocw-vm/init-ocw-vm.sh b/ocw-vm/init-ocw-vm.sh
index 720775c..c0d6f37 100755
--- a/ocw-vm/init-ocw-vm.sh
+++ b/ocw-vm/init-ocw-vm.sh
@@ -39,5 +39,10 @@ sudo apt-get install -y eog
# Use the Easy-OCW Ubuntu install script to get everything
# else installed!
git clone http://git-wip-us.apache.org/repos/asf/climate.git
+
+# Copy the Easy-OCW install script for Ubuntu
cp climate/easy-ocw/install-ubuntu-12_04.sh .
+# Copy the requirements files for conda and pip used by Easy-OCW
+cp climate/easy-ocw/*.txt .
+
bash install-ubuntu-12_04.sh -q
[10/12] git commit: Resolve CLIMATE-455. Merge PR #58.
Posted by jo...@apache.org.
Resolve CLIMATE-455. Merge PR #58.
Project: http://git-wip-us.apache.org/repos/asf/climate/repo
Commit: http://git-wip-us.apache.org/repos/asf/climate/commit/1fb24b36
Tree: http://git-wip-us.apache.org/repos/asf/climate/tree/1fb24b36
Diff: http://git-wip-us.apache.org/repos/asf/climate/diff/1fb24b36
Branch: refs/heads/master
Commit: 1fb24b36a47598a8e6de03aa68d1a6c93484bce2
Parents: e125786 574ee97
Author: Michael Joyce <jo...@apache.org>
Authored: Wed Jun 4 14:20:03 2014 -0700
Committer: Michael Joyce <jo...@apache.org>
Committed: Wed Jun 4 14:20:03 2014 -0700
----------------------------------------------------------------------
ocw/utils.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
[02/12] git commit: CLIMATE-453 - Add Taylor Diagram creation example
Posted by jo...@apache.org.
CLIMATE-453 - Add Taylor Diagram creation example
Project: http://git-wip-us.apache.org/repos/asf/climate/repo
Commit: http://git-wip-us.apache.org/repos/asf/climate/commit/91211e2e
Tree: http://git-wip-us.apache.org/repos/asf/climate/tree/91211e2e
Diff: http://git-wip-us.apache.org/repos/asf/climate/diff/91211e2e
Branch: refs/heads/master
Commit: 91211e2e4f9a11e161efd7b030e4bf1f647ec8ae
Parents: e9437d0
Author: Michael Joyce <jo...@apache.org>
Authored: Wed Jun 4 10:39:05 2014 -0700
Committer: Michael Joyce <jo...@apache.org>
Committed: Wed Jun 4 10:39:05 2014 -0700
----------------------------------------------------------------------
examples/taylor_diagram_example.py | 98 +++++++++++++++++++++++++++++++++
1 file changed, 98 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/climate/blob/91211e2e/examples/taylor_diagram_example.py
----------------------------------------------------------------------
diff --git a/examples/taylor_diagram_example.py b/examples/taylor_diagram_example.py
new file mode 100644
index 0000000..202954a
--- /dev/null
+++ b/examples/taylor_diagram_example.py
@@ -0,0 +1,98 @@
+import datetime
+import sys
+from os import path
+import urllib
+
+import numpy
+
+from ocw.dataset import Bounds
+import ocw.data_source.local as local
+import ocw.dataset_processor as dsp
+import ocw.evaluation as evaluation
+import ocw.metrics as metrics
+import ocw.plotter as plotter
+
+FILE_LEADER = "http://zipper.jpl.nasa.gov/dist/"
+FILE_1 = "AFRICA_KNMI-RACMO2.2b_CTL_ERAINT_MM_50km_1989-2008_tasmax.nc"
+FILE_2 = "AFRICA_UC-WRF311_CTL_ERAINT_MM_50km-rg_1989-2008_tasmax.nc"
+
+# Download some example NetCDF files for the evaluation
+################################################################################
+if not path.exists(FILE_1):
+ urllib.urlretrieve(FILE_LEADER + FILE_1, FILE_1)
+
+if not path.exists(FILE_2):
+ urllib.urlretrieve(FILE_LEADER + FILE_2, FILE_2)
+
+# Load the example datasets into OCW Dataset objects. We want to load
+# the 'tasmax' variable values. We'll also name the datasets for use
+# when plotting.
+################################################################################
+knmi_dataset = local.load_file(FILE_1, "tasmax")
+wrf_dataset = local.load_file(FILE_2, "tasmax")
+
+knmi_dataset.name = "knmi"
+wrf_dataset.name = "wrf"
+
+# Date values from loaded datasets might not always fall on reasonable days.
+# With monthly data, we could have data falling on the 1st, 15th, or some other
+# day of the month. Let's fix that real quick.
+################################################################################
+knmi_dataset = dsp.normalize_dataset_datetimes(knmi_dataset, 'monthly')
+wrf_dataset = dsp.normalize_dataset_datetimes(wrf_dataset, 'monthly')
+
+# We're only going to run this evaluation over a years worth of data. We'll
+# make a Bounds object and use it to subset our datasets.
+################################################################################
+subset = Bounds(-45, 42, -24, 60, datetime.datetime(1989, 1, 1), datetime.datetime(1989, 12, 1))
+knmi_dataset = dsp.subset(subset, knmi_dataset)
+wrf_dataset = dsp.subset(subset, wrf_dataset)
+
+# Temporally re-bin the data into a monthly timestep.
+################################################################################
+knmi_dataset = dsp.temporal_rebin(knmi_dataset, datetime.timedelta(days=30))
+wrf_dataset = dsp.temporal_rebin(wrf_dataset, datetime.timedelta(days=30))
+
+# Spatially regrid the datasets onto a 1 degree grid.
+################################################################################
+# Get the bounds of the reference dataset and use it to create a new
+# set of lat/lon values on a 1 degree step
+# Using the bounds we will create a new set of lats and lons on 1 degree step
+min_lat, max_lat, min_lon, max_lon = knmi_dataset.spatial_boundaries()
+new_lons = numpy.arange(min_lon, max_lon, 1)
+new_lats = numpy.arange(min_lat, max_lat, 1)
+
+# Spatially regrid datasets using the new_lats, new_lons numpy arrays
+knmi_dataset = dsp.spatial_regrid(knmi_dataset, new_lats, new_lons)
+wrf_dataset = dsp.spatial_regrid(wrf_dataset, new_lats, new_lons)
+
+# Load the metrics that we want to use for the evaluation.
+################################################################################
+sstdr = metrics.SpatialStdDevRatio()
+pc = metrics.PatternCorrelation()
+
+# Create our new evaluation object. The knmi dataset is the evaluations
+# reference dataset. We then provide a list of 1 or more target datasets
+# to use for the evaluation. In this case, we only want to use the wrf dataset.
+# Then we pass a list of all the metrics that we want to use in the evaluation.
+################################################################################
+test_evaluation = evaluation.Evaluation(knmi_dataset, [wrf_dataset], [sstdr, pc])
+test_evaluation.run()
+
+# Pull our the evaluation results and prepare them for drawing a Taylor diagram.
+################################################################################
+spatial_stddev_ratio = test_evaluation.results[0][0]
+# Pattern correlation results are a tuple, so we need to index and grab
+# the component we care about.
+spatial_correlation = test_evaluation.results[0][1][0]
+
+taylor_data = numpy.array([[spatial_stddev_ratio], [spatial_correlation]]).transpose()
+
+# Draw our taylor diagram!
+################################################################################
+plotter.draw_taylor_diagram(taylor_data,
+ [wrf_dataset.name],
+ knmi_dataset.name,
+ fname='taylor_plot',
+ fmt='png',
+ frameon=False)
[04/12] git commit: CLIMATE-455 - Update reshape_monthly_to_annually
to use copies of data
Posted by jo...@apache.org.
CLIMATE-455 - Update reshape_monthly_to_annually to use copies of data
- utils.reshape_monthly_to_annually now uses a copy of the passed
datasets data instead of the actual datasets.values.
Project: http://git-wip-us.apache.org/repos/asf/climate/repo
Commit: http://git-wip-us.apache.org/repos/asf/climate/commit/574ee97f
Tree: http://git-wip-us.apache.org/repos/asf/climate/tree/574ee97f
Diff: http://git-wip-us.apache.org/repos/asf/climate/diff/574ee97f
Branch: refs/heads/master
Commit: 574ee97f3871a3b219f23d999957e509f0a87380
Parents: e9437d0
Author: Michael Joyce <jo...@apache.org>
Authored: Wed Jun 4 11:59:33 2014 -0700
Committer: Michael Joyce <jo...@apache.org>
Committed: Wed Jun 4 11:59:33 2014 -0700
----------------------------------------------------------------------
ocw/utils.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/climate/blob/574ee97f/ocw/utils.py
----------------------------------------------------------------------
diff --git a/ocw/utils.py b/ocw/utils.py
index 7abbbfd..4a01f78 100644
--- a/ocw/utils.py
+++ b/ocw/utils.py
@@ -231,7 +231,7 @@ def reshape_monthly_to_annually(dataset):
:rtype: Numpy array
'''
- values = dataset.values
+ values = dataset.values[:]
data_shape = values.shape
num_total_month = data_shape[0]
num_year = num_total_month / 12
[05/12] git commit: CLIMATE-456 - Move DAP test initialization to
setup_class
Posted by jo...@apache.org.
CLIMATE-456 - Move DAP test initialization to setup_class
- Move OpenDAP test initialization to setup_class method. This also
seems to fix the issue with nose-exclude not ignoring the test_dap
file when instructed to
- Rename the test class to fall inline with project naming standards.
Project: http://git-wip-us.apache.org/repos/asf/climate/repo
Commit: http://git-wip-us.apache.org/repos/asf/climate/commit/2e0cd113
Tree: http://git-wip-us.apache.org/repos/asf/climate/tree/2e0cd113
Diff: http://git-wip-us.apache.org/repos/asf/climate/diff/2e0cd113
Branch: refs/heads/master
Commit: 2e0cd1139ea46a1c5af9f776b9d8366ed040a8bf
Parents: e9437d0
Author: Michael Joyce <jo...@apache.org>
Authored: Wed Jun 4 12:06:27 2014 -0700
Committer: Michael Joyce <jo...@apache.org>
Committed: Wed Jun 4 12:06:27 2014 -0700
----------------------------------------------------------------------
ocw/tests/test_dap.py | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/climate/blob/2e0cd113/ocw/tests/test_dap.py
----------------------------------------------------------------------
diff --git a/ocw/tests/test_dap.py b/ocw/tests/test_dap.py
index ed0afaf..cdc75cd 100644
--- a/ocw/tests/test_dap.py
+++ b/ocw/tests/test_dap.py
@@ -20,8 +20,10 @@ import ocw.data_source.dap as dap
from ocw.dataset import Dataset
import datetime as dt
-class test_dap(unittest.TestCase):
- dataset = dap.load('http://test.opendap.org/dap/data/nc/sst.mnmean.nc.gz', 'sst')
+class TestDap(unittest.TestCase):
+ @classmethod
+ def setup_class(self):
+ dataset = dap.load('http://test.opendap.org/dap/data/nc/sst.mnmean.nc.gz', 'sst')
def test_dataset_is_returned(self):
self.assertTrue(isinstance(self.dataset, Dataset))
[09/12] git commit: Resolve CLIMATE-454. Merge PR #56.
Posted by jo...@apache.org.
Resolve CLIMATE-454. Merge PR #56.
Project: http://git-wip-us.apache.org/repos/asf/climate/repo
Commit: http://git-wip-us.apache.org/repos/asf/climate/commit/e1257867
Tree: http://git-wip-us.apache.org/repos/asf/climate/tree/e1257867
Diff: http://git-wip-us.apache.org/repos/asf/climate/diff/e1257867
Branch: refs/heads/master
Commit: e1257867120feeb8433ba26d474e466471de7cd2
Parents: 0f65fec 3fd3478
Author: Michael Joyce <jo...@apache.org>
Authored: Wed Jun 4 14:19:05 2014 -0700
Committer: Michael Joyce <jo...@apache.org>
Committed: Wed Jun 4 14:19:05 2014 -0700
----------------------------------------------------------------------
ocw-vm/init-ocw-vm.sh | 5 +++++
1 file changed, 5 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/climate/blob/e1257867/ocw-vm/init-ocw-vm.sh
----------------------------------------------------------------------
[11/12] git commit: Resolve CLIMATE-456. Merge PR #59.
Posted by jo...@apache.org.
Resolve CLIMATE-456. Merge PR #59.
Project: http://git-wip-us.apache.org/repos/asf/climate/repo
Commit: http://git-wip-us.apache.org/repos/asf/climate/commit/eadf3d43
Tree: http://git-wip-us.apache.org/repos/asf/climate/tree/eadf3d43
Diff: http://git-wip-us.apache.org/repos/asf/climate/diff/eadf3d43
Branch: refs/heads/master
Commit: eadf3d437d661bc709c20edd0a79322fcc325211
Parents: 1fb24b3 2e0cd11
Author: Michael Joyce <jo...@apache.org>
Authored: Wed Jun 4 14:20:47 2014 -0700
Committer: Michael Joyce <jo...@apache.org>
Committed: Wed Jun 4 14:20:47 2014 -0700
----------------------------------------------------------------------
ocw/tests/test_dap.py | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
[06/12] git commit: CLIMATE-457 - Cleanup after Easy-OCW install on
Ubuntu
Posted by jo...@apache.org.
CLIMATE-457 - Cleanup after Easy-OCW install on Ubuntu
- Cleanup Basemap and Anaconda files after installation is complete.
Project: http://git-wip-us.apache.org/repos/asf/climate/repo
Commit: http://git-wip-us.apache.org/repos/asf/climate/commit/87045672
Tree: http://git-wip-us.apache.org/repos/asf/climate/tree/87045672
Diff: http://git-wip-us.apache.org/repos/asf/climate/diff/87045672
Branch: refs/heads/master
Commit: 87045672cfd4dd26c77fba788ddbe18a8db6c57b
Parents: e9437d0
Author: Michael Joyce <jo...@apache.org>
Authored: Wed Jun 4 12:34:50 2014 -0700
Committer: Michael Joyce <jo...@apache.org>
Committed: Wed Jun 4 12:34:50 2014 -0700
----------------------------------------------------------------------
easy-ocw/install-ubuntu-12_04.sh | 10 ++++++++++
1 file changed, 10 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/climate/blob/87045672/easy-ocw/install-ubuntu-12_04.sh
----------------------------------------------------------------------
diff --git a/easy-ocw/install-ubuntu-12_04.sh b/easy-ocw/install-ubuntu-12_04.sh
index 494cdb3..adb3fbb 100755
--- a/easy-ocw/install-ubuntu-12_04.sh
+++ b/easy-ocw/install-ubuntu-12_04.sh
@@ -148,6 +148,10 @@ bash Anaconda-1.9.2-Linux-x86_64.sh
export PATH="/home/vagrant/anaconda/bin:$PATH"
subtask "done"
+task "Cleaning up Anaconda download ..."
+rm -f Anaconda-1.9.2-Linux-x86_64.sh
+subtask "done"
+
# Install Basemap. Conda cannot be used for this install since
# it fails to analyse the dependencies (at the time of writing). This
# will install it manually. At some point, this should be replaced with
@@ -175,6 +179,12 @@ cd ..
python setup.py install >> install_log
subtask "done"
+task "Cleaning up Basemap install ..."
+cd
+rm -f basemap-1.0.7.tar.gz
+rm -rf basemap-1.0.7
+subtask "done"
+
# Install miscellaneous Python packages needed for OCW. Some of these
# can be installed with Conda, but since none of them have an annoying
# compiled component we just installed them with Pip.
[08/12] git commit: Resolve CLIMATE-453. Merge PR #55.
Posted by jo...@apache.org.
Resolve CLIMATE-453. Merge PR #55.
Project: http://git-wip-us.apache.org/repos/asf/climate/repo
Commit: http://git-wip-us.apache.org/repos/asf/climate/commit/0f65fec3
Tree: http://git-wip-us.apache.org/repos/asf/climate/tree/0f65fec3
Diff: http://git-wip-us.apache.org/repos/asf/climate/diff/0f65fec3
Branch: refs/heads/master
Commit: 0f65fec3fb8253557dd294518d37263334220463
Parents: e8a8589 91211e2
Author: Michael Joyce <jo...@apache.org>
Authored: Wed Jun 4 14:17:55 2014 -0700
Committer: Michael Joyce <jo...@apache.org>
Committed: Wed Jun 4 14:17:55 2014 -0700
----------------------------------------------------------------------
examples/taylor_diagram_example.py | 98 +++++++++++++++++++++++++++++++++
1 file changed, 98 insertions(+)
----------------------------------------------------------------------