You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@climate.apache.org by "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov> on 2014/06/05 00:12:32 UTC
Model Observation Evaluation exmaple
Hi Paul and Alex,
I ran a test code to evaluate one model [1] with one observation[2]. You can find the result as attached.
This has been temporally regrided for 356 day, so we going to have 5 years of data and that is why we have 5 plots.
Do you guys thinks this is correct to have one plot for each time or it should be one plot for entire time?
Metric = Bias
parameter_id = 38
dataset_id = 10
min_lat = 0
max_lat = 20
min_lon = 0
max_lon = 20
start_time = 2000-01-01 00:00:00
end_time = 2004-12-01 00:00:00
[1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable = TAS)
[2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
[cid:03DF99EA-3331-4A42-974E-7131165A8C00]
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
Re: Model Observation Evaluation exmaple
Posted by Michael Joyce <jo...@apache.org>.
Hay Mazi,
I'm checking this out on the OCW list. It looks like your plot got dropped.
Could you upload it somewhere and provide a link for us?
Cheers!
-- Joyce
On Wed, Jun 4, 2014 at 3:12 PM, Boustani, Maziyar (398F) <
Maziyar.Boustani@jpl.nasa.gov> wrote:
> Hi Paul and Alex,
>
> I ran a test code to evaluate one model [1] with one observation[2]. You
> can find the result as attached.
> This has been temporally regrided for 356 day, so we going to have 5 years
> of data and that is why we have 5 plots.
> Do you guys thinks this is correct to have one plot for each time or it
> should be one plot for entire time?
>
> Metric = Bias
> parameter_id = 38
> dataset_id = 10
> min_lat = 0
> max_lat = 20
> min_lon = 0
> max_lon = 20
> start_time = 2000-01-01 00:00:00
> end_time = 2004-12-01 00:00:00
>
>
> [1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable =
> TAS)
> [2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
>
>
> ……………………
> Maziyar Boustani
> Software Engineer - GIS Developer
> Science Data Understanding Group
> Jet Propulsion Laboratory, California Institute of Technology
> 4800 Oak Grove Drive,
> Pasadena, CA, 91109
> maziyar.boustani@jpl.nasa.gov
>
>
Re: Model Observation Evaluation exmaple
Posted by "Loikith, Paul C (329C-Affiliate)" <Pa...@jpl.nasa.gov>.
Hi Mazi,
This looks right to me. The only thing missing is the symmetric color bar that we discussed. Please let me know if you need any help with that.
Paul
From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>>
Date: Thursday, June 5, 2014 2:40 PM
To: Paul Loikith <pa...@jpl.nasa.gov>>, "goodman@apache.org<ma...@apache.org>" <go...@apache.org>>, Kyo Lee <Hu...@jpl.nasa.gov>>
Cc: "Joyce, Michael J (398J)" <Mi...@jpl.nasa.gov>>, "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Subject: Re: Model Observation Evaluation exmaple
Hi All,
One new metric (MeanBias) has been added to list of Metrics on API and I regenerated the plot from yesterday data [1].
Please let me know it looks like.
[1]: http://oi57.tinypic.com/34te3iw.jpg
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
On Jun 4, 2014, at 3:48 PM, Boustani, Maziyar (398F) wrote:
Here is the attached [1].
[1]: http://oi58.tinypic.com/w7yo2q.jpg
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote:
Hi Mazi,
This should be one plot total. When you compute a bias, conventionally it's just one bias for the entire time period so there should be one plot. You could have multiple plots if you were computing the bias for all 4 seasons, but for annual there should only be one.
When you say you temporally regridded to 365 days, what do you mean by this? The model and obs data are monthly, correct? So shouldn't you just be computing the mean difference between the obs and the model?
Please let me know if you need further clarification.
Thanks,
Paul
From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>>
Date: Wednesday, June 4, 2014 3:12 PM
To: Paul Loikith <pa...@jpl.nasa.gov>>, "goodman@apache.org<ma...@apache.org>" <go...@apache.org>>
Cc: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Subject: Model Observation Evaluation exmaple
Hi Paul and Alex,
I ran a test code to evaluate one model [1] with one observation[2]. You can find the result as attached.
This has been temporally regrided for 356 day, so we going to have 5 years of data and that is why we have 5 plots.
Do you guys thinks this is correct to have one plot for each time or it should be one plot for entire time?
Metric = Bias
parameter_id = 38
dataset_id = 10
min_lat = 0
max_lat = 20
min_lon = 0
max_lon = 20
start_time = 2000-01-01 00:00:00
end_time = 2004-12-01 00:00:00
[1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable = TAS)
[2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
<model_vs_obs_bias.png>
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
<model_vs_obs_bias.png>
Re: Model Observation Evaluation exmaple
Posted by "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>.
Hi All,
One new metric (MeanBias) has been added to list of Metrics on API and I regenerated the plot from yesterday data [1].
Please let me know it looks like.
[1]: http://oi57.tinypic.com/34te3iw.jpg
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
On Jun 4, 2014, at 3:48 PM, Boustani, Maziyar (398F) wrote:
Here is the attached [1].
[1]: http://oi58.tinypic.com/w7yo2q.jpg
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote:
Hi Mazi,
This should be one plot total. When you compute a bias, conventionally it's just one bias for the entire time period so there should be one plot. You could have multiple plots if you were computing the bias for all 4 seasons, but for annual there should only be one.
When you say you temporally regridded to 365 days, what do you mean by this? The model and obs data are monthly, correct? So shouldn't you just be computing the mean difference between the obs and the model?
Please let me know if you need further clarification.
Thanks,
Paul
From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>>
Date: Wednesday, June 4, 2014 3:12 PM
To: Paul Loikith <pa...@jpl.nasa.gov>>, "goodman@apache.org<ma...@apache.org>" <go...@apache.org>>
Cc: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Subject: Model Observation Evaluation exmaple
Hi Paul and Alex,
I ran a test code to evaluate one model [1] with one observation[2]. You can find the result as attached.
This has been temporally regrided for 356 day, so we going to have 5 years of data and that is why we have 5 plots.
Do you guys thinks this is correct to have one plot for each time or it should be one plot for entire time?
Metric = Bias
parameter_id = 38
dataset_id = 10
min_lat = 0
max_lat = 20
min_lon = 0
max_lon = 20
start_time = 2000-01-01 00:00:00
end_time = 2004-12-01 00:00:00
[1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable = TAS)
[2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
<model_vs_obs_bias.png>
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
<model_vs_obs_bias.png>
Re: Model Observation Evaluation exmaple
Posted by "Loikith, Paul C (329C-Affiliate)" <Pa...@jpl.nasa.gov>.
I agree. Mean bias is more useful for model evaluation. This could of course be applied also to mean winter bias, or mean summer bias not just annual.
Paul
From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>>
Date: Wednesday, June 4, 2014 4:41 PM
To: Paul Loikith <pa...@jpl.nasa.gov>>, Alex Goodman <al...@colostate.edu>>
Cc: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Subject: Re: Model Observation Evaluation exmaple
I guess the best solution here is to add the new Metric "calcBiasAberageOverTime" to our Metrics and use that in this example to be able to generate one plot.
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
On Jun 4, 2014, at 4:29 PM, Loikith, Paul C (329C-Affiliate) wrote:
Yes agreed. I'm guessing the issue here is the use of "calcBias" vs. "calcBiasAveragedOverTime." The later module from metrics.py would accomplish this.
I definitely agree about the colorbar too.
Hope all is well.
Paul
From: Alex Goodman <al...@colostate.edu>>
Date: Wednesday, June 4, 2014 4:25 PM
To: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Cc: "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>>, Paul Loikith <pa...@jpl.nasa.gov>>
Subject: Re: Model Observation Evaluation exmaple
Hi Paul,
I think I wasn't sure what was exactly in Mazi plots. I thought what was being shown was the bias averaged for each year. Instead it sounds like the data was temporally regridded from monthly data to yearly data first, (eg, the annual average surface temperatures were calculated first, then the bias metric was ran on the temporally regridded data). What I was trying to say was that from visually inspecting the plots alone, a plot of the average values of all five of them would probably look similar to each individual plot. Sorry if I worded that poorly... in any case I think your suggestion is the correct one, what should be plotted is just the mean bias at each grid point for the five-year period.
Also another side-note: When plotting biases or anomalies, it is generally ideal to have the 0 contour at the center of the colorbar. Currently you must manually enter the contour levels into one of the arguments of the contour plotting function to achieve this. Finding an elegant way to do this automatically was an often requested feature of the OCW plotting interface that I unfortunately never found the time to implement. Just something to think about.
Also, thanks for the kind regards Chris, I am doing well.
Alex
On Wed, Jun 4, 2014 at 5:06 PM, Mattmann, Chris A (3980) <ch...@jpl.nasa.gov>> wrote:
THanks guys, Alex, great to see you!
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov<ma...@nasa.gov>
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Alex Goodman <al...@colostate.edu>>
Reply-To: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Date: Wednesday, June 4, 2014 3:53 PM
To: "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>>
Cc: "Loikith, Paul C (329C-Affiliate)" <Pa...@jpl.nasa.gov>>,
"dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Subject: Re: Model Observation Evaluation exmaple
>Hi Mazi,
>
>I am in agreement with Paul. It is also clear from a quick inspection of
>your plots that the model bias for each year is identical, so plotting
>just
>the 5-year average bias would give you sufficient information. You might
>want to take a quick glance at IPCC AR5 WG I, Ch. 9 [1] for some good
>examples.
>
>Thanks,
>Alex
>
>[1] -
>http://www.climatechange2013.org/images/report/WG1AR5_Chapter09_FINAL.pdf
>
>
>On Wed, Jun 4, 2014 at 4:48 PM, Boustani, Maziyar (398F) <
>Maziyar.Boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>> wrote:
>
>> Here is the attached [1].
>>
>> [1]: http://oi58.tinypic.com/w7yo2q.jpg
>>
>> ŠŠŠŠŠŠŠŠ
>> Maziyar Boustani
>> Software Engineer - GIS Developer
>> Science Data Understanding Group
>> Jet Propulsion Laboratory, California Institute of Technology
>> 4800 Oak Grove Drive,
>> Pasadena, CA, 91109
>> maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
>>
>> On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote:
>>
>> Hi Mazi,
>>
>> This should be one plot total. When you compute a bias, conventionally
>> it's just one bias for the entire time period so there should be one
>>plot.
>> You could have multiple plots if you were computing the bias for all 4
>> seasons, but for annual there should only be one.
>>
>> When you say you temporally regridded to 365 days, what do you mean by
>> this? The model and obs data are monthly, correct? So shouldn't you
>>just
>> be computing the mean difference between the obs and the model?
>>
>> Please let me know if you need further clarification.
>>
>> Thanks,
>>
>> Paul
>>
>>
>>
>> From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>>
>> Date: Wednesday, June 4, 2014 3:12 PM
>> To: Paul Loikith <pa...@jpl.nasa.gov>>, "goodman@apache.org<ma...@apache.org>" <
>> goodman@apache.org<ma...@apache.org>>
>> Cc: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
>> Subject: Model Observation Evaluation exmaple
>>
>> Hi Paul and Alex,
>>
>> I ran a test code to evaluate one model [1] with one observation[2].
>>You
>> can find the result as attached.
>> This has been temporally regrided for 356 day, so we going to have 5
>>years
>> of data and that is why we have 5 plots.
>> Do you guys thinks this is correct to have one plot for each time or it
>> should be one plot for entire time?
>>
>> Metric = Bias
>> parameter_id = 38
>> dataset_id = 10
>> min_lat = 0
>> max_lat = 20
>> min_lon = 0
>> max_lon = 20
>> start_time = 2000-01-01 00:00:00
>> end_time = 2004-12-01 00:00:00
>>
>>
>> [1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable =
>> TAS)
>> [2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
>>
>> <model_vs_obs_bias.png>
>>
>> ŠŠŠŠŠŠŠŠ
>> Maziyar Boustani
>> Software Engineer - GIS Developer
>> Science Data Understanding Group
>> Jet Propulsion Laboratory, California Institute of Technology
>> 4800 Oak Grove Drive,
>> Pasadena, CA, 91109
>> maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
>>
>> <model_vs_obs_bias.png>
>>
>>
>>
>
>
>--
>Alex Goodman
>Graduate Research Assistant
>Department of Atmospheric Science
>Colorado State University
--
Alex Goodman
Graduate Research Assistant
Department of Atmospheric Science
Colorado State University
Re: Model Observation Evaluation exmaple
Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.
I smell Review Board :)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: <Lee>, "Kyo (329C-Affiliate)" <Hu...@jpl.nasa.gov>
Reply-To: "dev@climate.apache.org" <de...@climate.apache.org>
Date: Wednesday, June 4, 2014 8:28 PM
To: "dev@climate.apache.org" <de...@climate.apache.org>
Subject: Re: Model Observation Evaluation exmaple
>Hi Mazi,
>
>We do not have to make the new metric from scratch.
>As I told you two weeks ago, someone just needs to reform the following
>function that Kim and I updated last year.
>
>--------------------------------------------------------------------------
>-
>def calcClimSeason(monthBegin, monthEnd, dataset1):
> '''
> Purpose ::
> Calculate seasonal mean montheries and climatology for both 3-D
>and point time series.
> For example, to calculate DJF mean time series, monthBegin = 12,
>monthEnd =2
> This can handle monthBegin=monthEnd i.e. for climatology of a
>specific month
>
> Input::
> monthBegin - an integer for the beginning month (Jan =1)
> monthEnd - an integer for the ending month (Jan = 1)
> dataset1 - 3d numpy array of data in (t,lat,lon) or 1d numpy
>array montheries
>
> Output::
> tSeries - if 3d numpy was entered, 3d (number of years/number of
>years -1 if monthBegin > monthEnd,lat,lon),
> if 1d data entered it is a montheries of the data of length
>number of years
> means - if 3d numpy was entered, 2d (lat,lon), if 1d data entered
> it is a floating point number representing the overall mean
> '''
> if monthBegin > monthEnd:
> # Offset the original array so that the the first month
> # becomes monthBegin, note that this cuts off the first year of
>data
> offset = slice(monthBegin - 1, monthBegin - 13)
> data = misc.reshapeMonthlyData(dataset1[offset])
> monthIndex = slice(0, 13 - monthBegin + monthEnd)
> else:
> # Since monthBegin <= monthEnd, just take a slice containing
>those months
> data = misc.reshapeMonthlyData(dataset1)
> monthIndex = slice(monthBegin - 1, monthEnd)
>
> tSeries = data[:, monthIndex].mean(axis = 1)
> means = tSeries.mean(axis = 0)
> return tSeries, means
>--------------------------------------------------------------------------
>------------------------
>
>Cheers,
>Kyo
>
>
>On Jun 4, 2014, at 4:41 PM, Boustani, Maziyar (398F)
><Ma...@jpl.nasa.gov> wrote:
>
>> I guess the best solution here is to add the new Metric
>>"calcBiasAberageOverTime" to our Metrics and use that in this example to
>>be able to generate one plot.
>>
>> ........................
>> Maziyar Boustani
>> Software Engineer - GIS Developer
>> Science Data Understanding Group
>> Jet Propulsion Laboratory, California Institute of Technology
>> 4800 Oak Grove Drive,
>> Pasadena, CA, 91109
>> maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
>>
>> On Jun 4, 2014, at 4:29 PM, Loikith, Paul C (329C-Affiliate) wrote:
>>
>> Yes agreed. I'm guessing the issue here is the use of "calcBias" vs.
>>"calcBiasAveragedOverTime." The later module from metrics.py would
>>accomplish this.
>>
>> I definitely agree about the colorbar too.
>>
>> Hope all is well.
>>
>> Paul
>>
>> From: Alex Goodman
>><al...@colostate.edu>>
>> Date: Wednesday, June 4, 2014 4:25 PM
>> To: "dev@climate.apache.org<ma...@climate.apache.org>"
>><de...@climate.apache.org>>
>> Cc: "Boustani, Maziyar (398F)"
>><Ma...@jpl.nasa.gov>>,
>>Paul Loikith
>><pa...@jpl.nasa.gov>>
>> Subject: Re: Model Observation Evaluation exmaple
>>
>> Hi Paul,
>>
>> I think I wasn't sure what was exactly in Mazi plots. I thought what
>>was being shown was the bias averaged for each year. Instead it sounds
>>like the data was temporally regridded from monthly data to yearly data
>>first, (eg, the annual average surface temperatures were calculated
>>first, then the bias metric was ran on the temporally regridded data).
>>What I was trying to say was that from visually inspecting the plots
>>alone, a plot of the average values of all five of them would probably
>>look similar to each individual plot. Sorry if I worded that poorly...
>>in any case I think your suggestion is the correct one, what should be
>>plotted is just the mean bias at each grid point for the five-year
>>period.
>>
>> Also another side-note: When plotting biases or anomalies, it is
>>generally ideal to have the 0 contour at the center of the colorbar.
>>Currently you must manually enter the contour levels into one of the
>>arguments of the contour plotting function to achieve this. Finding an
>>elegant way to do this automatically was an often requested feature of
>>the OCW plotting interface that I unfortunately never found the time to
>>implement. Just something to think about.
>>
>> Also, thanks for the kind regards Chris, I am doing well.
>>
>> Alex
>>
>>
>> On Wed, Jun 4, 2014 at 5:06 PM, Mattmann, Chris A (3980)
>><ch...@jpl.nasa.gov>>
>>wrote:
>> THanks guys, Alex, great to see you!
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Chief Architect
>> Instrument Software and Science Data Systems Section (398)
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 168-519, Mailstop: 168-527
>> Email: chris.a.mattmann@nasa.gov<ma...@nasa.gov>
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Associate Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: Alex Goodman
>><al...@colostate.edu>>
>> Reply-To: "dev@climate.apache.org<ma...@climate.apache.org>"
>><de...@climate.apache.org>>
>> Date: Wednesday, June 4, 2014 3:53 PM
>> To: "Boustani, Maziyar (398F)"
>><Ma...@jpl.nasa.gov>>
>> Cc: "Loikith, Paul C (329C-Affiliate)"
>><Pa...@jpl.nasa.gov>>,
>> "dev@climate.apache.org<ma...@climate.apache.org>"
>><de...@climate.apache.org>>
>> Subject: Re: Model Observation Evaluation exmaple
>>
>>> Hi Mazi,
>>>
>>> I am in agreement with Paul. It is also clear from a quick inspection
>>>of
>>> your plots that the model bias for each year is identical, so plotting
>>> just
>>> the 5-year average bias would give you sufficient information. You
>>>might
>>> want to take a quick glance at IPCC AR5 WG I, Ch. 9 [1] for some good
>>> examples.
>>>
>>> Thanks,
>>> Alex
>>>
>>> [1] -
>>>
>>>http://www.climatechange2013.org/images/report/WG1AR5_Chapter09_FINAL.pd
>>>f
>>>
>>>
>>> On Wed, Jun 4, 2014 at 4:48 PM, Boustani, Maziyar (398F) <
>>> Maziyar.Boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>>
>>>wrote:
>>>
>>>> Here is the attached [1].
>>>>
>>>> [1]: http://oi58.tinypic.com/w7yo2q.jpg
>>>>
>>>> ŠŠŠŠŠŠŠŠ
>>>> Maziyar Boustani
>>>> Software Engineer - GIS Developer
>>>> Science Data Understanding Group
>>>> Jet Propulsion Laboratory, California Institute of Technology
>>>> 4800 Oak Grove Drive,
>>>> Pasadena, CA, 91109
>>>> maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
>>>>
>>>> On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote:
>>>>
>>>> Hi Mazi,
>>>>
>>>> This should be one plot total. When you compute a bias,
>>>>conventionally
>>>> it's just one bias for the entire time period so there should be one
>>>> plot.
>>>> You could have multiple plots if you were computing the bias for all 4
>>>> seasons, but for annual there should only be one.
>>>>
>>>> When you say you temporally regridded to 365 days, what do you mean by
>>>> this? The model and obs data are monthly, correct? So shouldn't you
>>>> just
>>>> be computing the mean difference between the obs and the model?
>>>>
>>>> Please let me know if you need further clarification.
>>>>
>>>> Thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>>
>>>> From: <Boustani>, "Maziyar (398F)"
>>>><Ma...@jpl.nasa.gov>>
>>>> Date: Wednesday, June 4, 2014 3:12 PM
>>>> To: Paul Loikith
>>>><pa...@jpl.nasa.gov>>,
>>>>"goodman@apache.org<ma...@apache.org>" <
>>>> goodman@apache.org<ma...@apache.org>>
>>>> Cc: "dev@climate.apache.org<ma...@climate.apache.org>"
>>>><de...@climate.apache.org>>
>>>> Subject: Model Observation Evaluation exmaple
>>>>
>>>> Hi Paul and Alex,
>>>>
>>>> I ran a test code to evaluate one model [1] with one observation[2].
>>>> You
>>>> can find the result as attached.
>>>> This has been temporally regrided for 356 day, so we going to have 5
>>>> years
>>>> of data and that is why we have 5 plots.
>>>> Do you guys thinks this is correct to have one plot for each time or
>>>>it
>>>> should be one plot for entire time?
>>>>
>>>> Metric = Bias
>>>> parameter_id = 38
>>>> dataset_id = 10
>>>> min_lat = 0
>>>> max_lat = 20
>>>> min_lon = 0
>>>> max_lon = 20
>>>> start_time = 2000-01-01 00:00:00
>>>> end_time = 2004-12-01 00:00:00
>>>>
>>>>
>>>> [1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable
>>>>=
>>>> TAS)
>>>> [2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
>>>>
>>>> <model_vs_obs_bias.png>
>>>>
>>>> ŠŠŠŠŠŠŠŠ
>>>> Maziyar Boustani
>>>> Software Engineer - GIS Developer
>>>> Science Data Understanding Group
>>>> Jet Propulsion Laboratory, California Institute of Technology
>>>> 4800 Oak Grove Drive,
>>>> Pasadena, CA, 91109
>>>> maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
>>>>
>>>> <model_vs_obs_bias.png>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Alex Goodman
>>> Graduate Research Assistant
>>> Department of Atmospheric Science
>>> Colorado State University
>>
>>
>>
>>
>> --
>> Alex Goodman
>> Graduate Research Assistant
>> Department of Atmospheric Science
>> Colorado State University
>>
>
Re: Model Observation Evaluation exmaple
Posted by "Lee, Kyo (329C-Affiliate)" <Hu...@jpl.nasa.gov>.
Hi Mazi,
We do not have to make the new metric from scratch.
As I told you two weeks ago, someone just needs to reform the following function that Kim and I updated last year.
---------------------------------------------------------------------------
def calcClimSeason(monthBegin, monthEnd, dataset1):
'''
Purpose ::
Calculate seasonal mean montheries and climatology for both 3-D and point time series.
For example, to calculate DJF mean time series, monthBegin = 12, monthEnd =2
This can handle monthBegin=monthEnd i.e. for climatology of a specific month
Input::
monthBegin - an integer for the beginning month (Jan =1)
monthEnd - an integer for the ending month (Jan = 1)
dataset1 - 3d numpy array of data in (t,lat,lon) or 1d numpy array montheries
Output::
tSeries - if 3d numpy was entered, 3d (number of years/number of years -1 if monthBegin > monthEnd,lat,lon),
if 1d data entered it is a montheries of the data of length number of years
means - if 3d numpy was entered, 2d (lat,lon), if 1d data entered
it is a floating point number representing the overall mean
'''
if monthBegin > monthEnd:
# Offset the original array so that the the first month
# becomes monthBegin, note that this cuts off the first year of data
offset = slice(monthBegin - 1, monthBegin - 13)
data = misc.reshapeMonthlyData(dataset1[offset])
monthIndex = slice(0, 13 - monthBegin + monthEnd)
else:
# Since monthBegin <= monthEnd, just take a slice containing those months
data = misc.reshapeMonthlyData(dataset1)
monthIndex = slice(monthBegin - 1, monthEnd)
tSeries = data[:, monthIndex].mean(axis = 1)
means = tSeries.mean(axis = 0)
return tSeries, means
--------------------------------------------------------------------------------------------------
Cheers,
Kyo
On Jun 4, 2014, at 4:41 PM, Boustani, Maziyar (398F) <Ma...@jpl.nasa.gov> wrote:
> I guess the best solution here is to add the new Metric "calcBiasAberageOverTime" to our Metrics and use that in this example to be able to generate one plot.
>
> ……………………
> Maziyar Boustani
> Software Engineer - GIS Developer
> Science Data Understanding Group
> Jet Propulsion Laboratory, California Institute of Technology
> 4800 Oak Grove Drive,
> Pasadena, CA, 91109
> maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
>
> On Jun 4, 2014, at 4:29 PM, Loikith, Paul C (329C-Affiliate) wrote:
>
> Yes agreed. I'm guessing the issue here is the use of "calcBias" vs. "calcBiasAveragedOverTime." The later module from metrics.py would accomplish this.
>
> I definitely agree about the colorbar too.
>
> Hope all is well.
>
> Paul
>
> From: Alex Goodman <al...@colostate.edu>>
> Date: Wednesday, June 4, 2014 4:25 PM
> To: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
> Cc: "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>>, Paul Loikith <pa...@jpl.nasa.gov>>
> Subject: Re: Model Observation Evaluation exmaple
>
> Hi Paul,
>
> I think I wasn't sure what was exactly in Mazi plots. I thought what was being shown was the bias averaged for each year. Instead it sounds like the data was temporally regridded from monthly data to yearly data first, (eg, the annual average surface temperatures were calculated first, then the bias metric was ran on the temporally regridded data). What I was trying to say was that from visually inspecting the plots alone, a plot of the average values of all five of them would probably look similar to each individual plot. Sorry if I worded that poorly... in any case I think your suggestion is the correct one, what should be plotted is just the mean bias at each grid point for the five-year period.
>
> Also another side-note: When plotting biases or anomalies, it is generally ideal to have the 0 contour at the center of the colorbar. Currently you must manually enter the contour levels into one of the arguments of the contour plotting function to achieve this. Finding an elegant way to do this automatically was an often requested feature of the OCW plotting interface that I unfortunately never found the time to implement. Just something to think about.
>
> Also, thanks for the kind regards Chris, I am doing well.
>
> Alex
>
>
> On Wed, Jun 4, 2014 at 5:06 PM, Mattmann, Chris A (3980) <ch...@jpl.nasa.gov>> wrote:
> THanks guys, Alex, great to see you!
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattmann@nasa.gov<ma...@nasa.gov>
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>
>
> -----Original Message-----
> From: Alex Goodman <al...@colostate.edu>>
> Reply-To: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
> Date: Wednesday, June 4, 2014 3:53 PM
> To: "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>>
> Cc: "Loikith, Paul C (329C-Affiliate)" <Pa...@jpl.nasa.gov>>,
> "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
> Subject: Re: Model Observation Evaluation exmaple
>
>> Hi Mazi,
>>
>> I am in agreement with Paul. It is also clear from a quick inspection of
>> your plots that the model bias for each year is identical, so plotting
>> just
>> the 5-year average bias would give you sufficient information. You might
>> want to take a quick glance at IPCC AR5 WG I, Ch. 9 [1] for some good
>> examples.
>>
>> Thanks,
>> Alex
>>
>> [1] -
>> http://www.climatechange2013.org/images/report/WG1AR5_Chapter09_FINAL.pdf
>>
>>
>> On Wed, Jun 4, 2014 at 4:48 PM, Boustani, Maziyar (398F) <
>> Maziyar.Boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>> wrote:
>>
>>> Here is the attached [1].
>>>
>>> [1]: http://oi58.tinypic.com/w7yo2q.jpg
>>>
>>> ŠŠŠŠŠŠŠŠ
>>> Maziyar Boustani
>>> Software Engineer - GIS Developer
>>> Science Data Understanding Group
>>> Jet Propulsion Laboratory, California Institute of Technology
>>> 4800 Oak Grove Drive,
>>> Pasadena, CA, 91109
>>> maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
>>>
>>> On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote:
>>>
>>> Hi Mazi,
>>>
>>> This should be one plot total. When you compute a bias, conventionally
>>> it's just one bias for the entire time period so there should be one
>>> plot.
>>> You could have multiple plots if you were computing the bias for all 4
>>> seasons, but for annual there should only be one.
>>>
>>> When you say you temporally regridded to 365 days, what do you mean by
>>> this? The model and obs data are monthly, correct? So shouldn't you
>>> just
>>> be computing the mean difference between the obs and the model?
>>>
>>> Please let me know if you need further clarification.
>>>
>>> Thanks,
>>>
>>> Paul
>>>
>>>
>>>
>>> From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>>
>>> Date: Wednesday, June 4, 2014 3:12 PM
>>> To: Paul Loikith <pa...@jpl.nasa.gov>>, "goodman@apache.org<ma...@apache.org>" <
>>> goodman@apache.org<ma...@apache.org>>
>>> Cc: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
>>> Subject: Model Observation Evaluation exmaple
>>>
>>> Hi Paul and Alex,
>>>
>>> I ran a test code to evaluate one model [1] with one observation[2].
>>> You
>>> can find the result as attached.
>>> This has been temporally regrided for 356 day, so we going to have 5
>>> years
>>> of data and that is why we have 5 plots.
>>> Do you guys thinks this is correct to have one plot for each time or it
>>> should be one plot for entire time?
>>>
>>> Metric = Bias
>>> parameter_id = 38
>>> dataset_id = 10
>>> min_lat = 0
>>> max_lat = 20
>>> min_lon = 0
>>> max_lon = 20
>>> start_time = 2000-01-01 00:00:00
>>> end_time = 2004-12-01 00:00:00
>>>
>>>
>>> [1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable =
>>> TAS)
>>> [2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
>>>
>>> <model_vs_obs_bias.png>
>>>
>>> ŠŠŠŠŠŠŠŠ
>>> Maziyar Boustani
>>> Software Engineer - GIS Developer
>>> Science Data Understanding Group
>>> Jet Propulsion Laboratory, California Institute of Technology
>>> 4800 Oak Grove Drive,
>>> Pasadena, CA, 91109
>>> maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
>>>
>>> <model_vs_obs_bias.png>
>>>
>>>
>>>
>>
>>
>> --
>> Alex Goodman
>> Graduate Research Assistant
>> Department of Atmospheric Science
>> Colorado State University
>
>
>
>
> --
> Alex Goodman
> Graduate Research Assistant
> Department of Atmospheric Science
> Colorado State University
>
Re: Model Observation Evaluation exmaple
Posted by "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>.
I guess the best solution here is to add the new Metric "calcBiasAberageOverTime" to our Metrics and use that in this example to be able to generate one plot.
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
On Jun 4, 2014, at 4:29 PM, Loikith, Paul C (329C-Affiliate) wrote:
Yes agreed. I'm guessing the issue here is the use of "calcBias" vs. "calcBiasAveragedOverTime." The later module from metrics.py would accomplish this.
I definitely agree about the colorbar too.
Hope all is well.
Paul
From: Alex Goodman <al...@colostate.edu>>
Date: Wednesday, June 4, 2014 4:25 PM
To: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Cc: "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>>, Paul Loikith <pa...@jpl.nasa.gov>>
Subject: Re: Model Observation Evaluation exmaple
Hi Paul,
I think I wasn't sure what was exactly in Mazi plots. I thought what was being shown was the bias averaged for each year. Instead it sounds like the data was temporally regridded from monthly data to yearly data first, (eg, the annual average surface temperatures were calculated first, then the bias metric was ran on the temporally regridded data). What I was trying to say was that from visually inspecting the plots alone, a plot of the average values of all five of them would probably look similar to each individual plot. Sorry if I worded that poorly... in any case I think your suggestion is the correct one, what should be plotted is just the mean bias at each grid point for the five-year period.
Also another side-note: When plotting biases or anomalies, it is generally ideal to have the 0 contour at the center of the colorbar. Currently you must manually enter the contour levels into one of the arguments of the contour plotting function to achieve this. Finding an elegant way to do this automatically was an often requested feature of the OCW plotting interface that I unfortunately never found the time to implement. Just something to think about.
Also, thanks for the kind regards Chris, I am doing well.
Alex
On Wed, Jun 4, 2014 at 5:06 PM, Mattmann, Chris A (3980) <ch...@jpl.nasa.gov>> wrote:
THanks guys, Alex, great to see you!
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov<ma...@nasa.gov>
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Alex Goodman <al...@colostate.edu>>
Reply-To: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Date: Wednesday, June 4, 2014 3:53 PM
To: "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>>
Cc: "Loikith, Paul C (329C-Affiliate)" <Pa...@jpl.nasa.gov>>,
"dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Subject: Re: Model Observation Evaluation exmaple
>Hi Mazi,
>
>I am in agreement with Paul. It is also clear from a quick inspection of
>your plots that the model bias for each year is identical, so plotting
>just
>the 5-year average bias would give you sufficient information. You might
>want to take a quick glance at IPCC AR5 WG I, Ch. 9 [1] for some good
>examples.
>
>Thanks,
>Alex
>
>[1] -
>http://www.climatechange2013.org/images/report/WG1AR5_Chapter09_FINAL.pdf
>
>
>On Wed, Jun 4, 2014 at 4:48 PM, Boustani, Maziyar (398F) <
>Maziyar.Boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>> wrote:
>
>> Here is the attached [1].
>>
>> [1]: http://oi58.tinypic.com/w7yo2q.jpg
>>
>> ŠŠŠŠŠŠŠŠ
>> Maziyar Boustani
>> Software Engineer - GIS Developer
>> Science Data Understanding Group
>> Jet Propulsion Laboratory, California Institute of Technology
>> 4800 Oak Grove Drive,
>> Pasadena, CA, 91109
>> maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
>>
>> On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote:
>>
>> Hi Mazi,
>>
>> This should be one plot total. When you compute a bias, conventionally
>> it's just one bias for the entire time period so there should be one
>>plot.
>> You could have multiple plots if you were computing the bias for all 4
>> seasons, but for annual there should only be one.
>>
>> When you say you temporally regridded to 365 days, what do you mean by
>> this? The model and obs data are monthly, correct? So shouldn't you
>>just
>> be computing the mean difference between the obs and the model?
>>
>> Please let me know if you need further clarification.
>>
>> Thanks,
>>
>> Paul
>>
>>
>>
>> From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>>
>> Date: Wednesday, June 4, 2014 3:12 PM
>> To: Paul Loikith <pa...@jpl.nasa.gov>>, "goodman@apache.org<ma...@apache.org>" <
>> goodman@apache.org<ma...@apache.org>>
>> Cc: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
>> Subject: Model Observation Evaluation exmaple
>>
>> Hi Paul and Alex,
>>
>> I ran a test code to evaluate one model [1] with one observation[2].
>>You
>> can find the result as attached.
>> This has been temporally regrided for 356 day, so we going to have 5
>>years
>> of data and that is why we have 5 plots.
>> Do you guys thinks this is correct to have one plot for each time or it
>> should be one plot for entire time?
>>
>> Metric = Bias
>> parameter_id = 38
>> dataset_id = 10
>> min_lat = 0
>> max_lat = 20
>> min_lon = 0
>> max_lon = 20
>> start_time = 2000-01-01 00:00:00
>> end_time = 2004-12-01 00:00:00
>>
>>
>> [1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable =
>> TAS)
>> [2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
>>
>> <model_vs_obs_bias.png>
>>
>> ŠŠŠŠŠŠŠŠ
>> Maziyar Boustani
>> Software Engineer - GIS Developer
>> Science Data Understanding Group
>> Jet Propulsion Laboratory, California Institute of Technology
>> 4800 Oak Grove Drive,
>> Pasadena, CA, 91109
>> maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
>>
>> <model_vs_obs_bias.png>
>>
>>
>>
>
>
>--
>Alex Goodman
>Graduate Research Assistant
>Department of Atmospheric Science
>Colorado State University
--
Alex Goodman
Graduate Research Assistant
Department of Atmospheric Science
Colorado State University
Re: Model Observation Evaluation exmaple
Posted by "Loikith, Paul C (329C-Affiliate)" <Pa...@jpl.nasa.gov>.
Yes agreed. I'm guessing the issue here is the use of "calcBias" vs. "calcBiasAveragedOverTime." The later module from metrics.py would accomplish this.
I definitely agree about the colorbar too.
Hope all is well.
Paul
From: Alex Goodman <al...@colostate.edu>>
Date: Wednesday, June 4, 2014 4:25 PM
To: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Cc: "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>>, Paul Loikith <pa...@jpl.nasa.gov>>
Subject: Re: Model Observation Evaluation exmaple
Hi Paul,
I think I wasn't sure what was exactly in Mazi plots. I thought what was being shown was the bias averaged for each year. Instead it sounds like the data was temporally regridded from monthly data to yearly data first, (eg, the annual average surface temperatures were calculated first, then the bias metric was ran on the temporally regridded data). What I was trying to say was that from visually inspecting the plots alone, a plot of the average values of all five of them would probably look similar to each individual plot. Sorry if I worded that poorly... in any case I think your suggestion is the correct one, what should be plotted is just the mean bias at each grid point for the five-year period.
Also another side-note: When plotting biases or anomalies, it is generally ideal to have the 0 contour at the center of the colorbar. Currently you must manually enter the contour levels into one of the arguments of the contour plotting function to achieve this. Finding an elegant way to do this automatically was an often requested feature of the OCW plotting interface that I unfortunately never found the time to implement. Just something to think about.
Also, thanks for the kind regards Chris, I am doing well.
Alex
On Wed, Jun 4, 2014 at 5:06 PM, Mattmann, Chris A (3980) <ch...@jpl.nasa.gov>> wrote:
THanks guys, Alex, great to see you!
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov<ma...@nasa.gov>
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Alex Goodman <al...@colostate.edu>>
Reply-To: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Date: Wednesday, June 4, 2014 3:53 PM
To: "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>>
Cc: "Loikith, Paul C (329C-Affiliate)" <Pa...@jpl.nasa.gov>>,
"dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Subject: Re: Model Observation Evaluation exmaple
>Hi Mazi,
>
>I am in agreement with Paul. It is also clear from a quick inspection of
>your plots that the model bias for each year is identical, so plotting
>just
>the 5-year average bias would give you sufficient information. You might
>want to take a quick glance at IPCC AR5 WG I, Ch. 9 [1] for some good
>examples.
>
>Thanks,
>Alex
>
>[1] -
>http://www.climatechange2013.org/images/report/WG1AR5_Chapter09_FINAL.pdf
>
>
>On Wed, Jun 4, 2014 at 4:48 PM, Boustani, Maziyar (398F) <
>Maziyar.Boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>> wrote:
>
>> Here is the attached [1].
>>
>> [1]: http://oi58.tinypic.com/w7yo2q.jpg
>>
>> ŠŠŠŠŠŠŠŠ
>> Maziyar Boustani
>> Software Engineer - GIS Developer
>> Science Data Understanding Group
>> Jet Propulsion Laboratory, California Institute of Technology
>> 4800 Oak Grove Drive,
>> Pasadena, CA, 91109
>> maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
>>
>> On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote:
>>
>> Hi Mazi,
>>
>> This should be one plot total. When you compute a bias, conventionally
>> it's just one bias for the entire time period so there should be one
>>plot.
>> You could have multiple plots if you were computing the bias for all 4
>> seasons, but for annual there should only be one.
>>
>> When you say you temporally regridded to 365 days, what do you mean by
>> this? The model and obs data are monthly, correct? So shouldn't you
>>just
>> be computing the mean difference between the obs and the model?
>>
>> Please let me know if you need further clarification.
>>
>> Thanks,
>>
>> Paul
>>
>>
>>
>> From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>>
>> Date: Wednesday, June 4, 2014 3:12 PM
>> To: Paul Loikith <pa...@jpl.nasa.gov>>, "goodman@apache.org<ma...@apache.org>" <
>> goodman@apache.org<ma...@apache.org>>
>> Cc: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
>> Subject: Model Observation Evaluation exmaple
>>
>> Hi Paul and Alex,
>>
>> I ran a test code to evaluate one model [1] with one observation[2].
>>You
>> can find the result as attached.
>> This has been temporally regrided for 356 day, so we going to have 5
>>years
>> of data and that is why we have 5 plots.
>> Do you guys thinks this is correct to have one plot for each time or it
>> should be one plot for entire time?
>>
>> Metric = Bias
>> parameter_id = 38
>> dataset_id = 10
>> min_lat = 0
>> max_lat = 20
>> min_lon = 0
>> max_lon = 20
>> start_time = 2000-01-01 00:00:00
>> end_time = 2004-12-01 00:00:00
>>
>>
>> [1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable =
>> TAS)
>> [2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
>>
>> <model_vs_obs_bias.png>
>>
>> ŠŠŠŠŠŠŠŠ
>> Maziyar Boustani
>> Software Engineer - GIS Developer
>> Science Data Understanding Group
>> Jet Propulsion Laboratory, California Institute of Technology
>> 4800 Oak Grove Drive,
>> Pasadena, CA, 91109
>> maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
>>
>> <model_vs_obs_bias.png>
>>
>>
>>
>
>
>--
>Alex Goodman
>Graduate Research Assistant
>Department of Atmospheric Science
>Colorado State University
--
Alex Goodman
Graduate Research Assistant
Department of Atmospheric Science
Colorado State University
Re: Model Observation Evaluation exmaple
Posted by Alex Goodman <al...@colostate.edu>.
Hi Paul,
I think I wasn't sure what was exactly in Mazi plots. I thought what was
being shown was the bias averaged for each year. Instead it sounds like the
data was temporally regridded from monthly data to yearly data first, (eg,
the annual average surface temperatures were calculated first, then the
bias metric was ran on the temporally regridded data). What I was trying to
say was that from visually inspecting the plots alone, a plot of the
average values of all five of them would probably look similar to each
individual plot. Sorry if I worded that poorly... in any case I think your
suggestion is the correct one, what should be plotted is just the mean bias
at each grid point for the five-year period.
Also another side-note: When plotting biases or anomalies, it is generally
ideal to have the 0 contour at the center of the colorbar. Currently you
must manually enter the contour levels into one of the arguments of the
contour plotting function to achieve this. Finding an elegant way to do
this automatically was an often requested feature of the OCW plotting
interface that I unfortunately never found the time to implement. Just
something to think about.
Also, thanks for the kind regards Chris, I am doing well.
Alex
On Wed, Jun 4, 2014 at 5:06 PM, Mattmann, Chris A (3980) <
chris.a.mattmann@jpl.nasa.gov> wrote:
> THanks guys, Alex, great to see you!
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>
>
> -----Original Message-----
> From: Alex Goodman <al...@colostate.edu>
> Reply-To: "dev@climate.apache.org" <de...@climate.apache.org>
> Date: Wednesday, June 4, 2014 3:53 PM
> To: "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>
> Cc: "Loikith, Paul C (329C-Affiliate)" <Pa...@jpl.nasa.gov>,
> "dev@climate.apache.org" <de...@climate.apache.org>
> Subject: Re: Model Observation Evaluation exmaple
>
> >Hi Mazi,
> >
> >I am in agreement with Paul. It is also clear from a quick inspection of
> >your plots that the model bias for each year is identical, so plotting
> >just
> >the 5-year average bias would give you sufficient information. You might
> >want to take a quick glance at IPCC AR5 WG I, Ch. 9 [1] for some good
> >examples.
> >
> >Thanks,
> >Alex
> >
> >[1] -
> >http://www.climatechange2013.org/images/report/WG1AR5_Chapter09_FINAL.pdf
> >
> >
> >On Wed, Jun 4, 2014 at 4:48 PM, Boustani, Maziyar (398F) <
> >Maziyar.Boustani@jpl.nasa.gov> wrote:
> >
> >> Here is the attached [1].
> >>
> >> [1]: http://oi58.tinypic.com/w7yo2q.jpg
> >>
> >> ŠŠŠŠŠŠŠŠ
> >> Maziyar Boustani
> >> Software Engineer - GIS Developer
> >> Science Data Understanding Group
> >> Jet Propulsion Laboratory, California Institute of Technology
> >> 4800 Oak Grove Drive,
> >> Pasadena, CA, 91109
> >> maziyar.boustani@jpl.nasa.gov
> >>
> >> On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote:
> >>
> >> Hi Mazi,
> >>
> >> This should be one plot total. When you compute a bias, conventionally
> >> it's just one bias for the entire time period so there should be one
> >>plot.
> >> You could have multiple plots if you were computing the bias for all 4
> >> seasons, but for annual there should only be one.
> >>
> >> When you say you temporally regridded to 365 days, what do you mean by
> >> this? The model and obs data are monthly, correct? So shouldn't you
> >>just
> >> be computing the mean difference between the obs and the model?
> >>
> >> Please let me know if you need further clarification.
> >>
> >> Thanks,
> >>
> >> Paul
> >>
> >>
> >>
> >> From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>
> >> Date: Wednesday, June 4, 2014 3:12 PM
> >> To: Paul Loikith <pa...@jpl.nasa.gov>, "goodman@apache.org" <
> >> goodman@apache.org>
> >> Cc: "dev@climate.apache.org" <de...@climate.apache.org>
> >> Subject: Model Observation Evaluation exmaple
> >>
> >> Hi Paul and Alex,
> >>
> >> I ran a test code to evaluate one model [1] with one observation[2].
> >>You
> >> can find the result as attached.
> >> This has been temporally regrided for 356 day, so we going to have 5
> >>years
> >> of data and that is why we have 5 plots.
> >> Do you guys thinks this is correct to have one plot for each time or it
> >> should be one plot for entire time?
> >>
> >> Metric = Bias
> >> parameter_id = 38
> >> dataset_id = 10
> >> min_lat = 0
> >> max_lat = 20
> >> min_lon = 0
> >> max_lon = 20
> >> start_time = 2000-01-01 00:00:00
> >> end_time = 2004-12-01 00:00:00
> >>
> >>
> >> [1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable =
> >> TAS)
> >> [2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
> >>
> >> <model_vs_obs_bias.png>
> >>
> >> ŠŠŠŠŠŠŠŠ
> >> Maziyar Boustani
> >> Software Engineer - GIS Developer
> >> Science Data Understanding Group
> >> Jet Propulsion Laboratory, California Institute of Technology
> >> 4800 Oak Grove Drive,
> >> Pasadena, CA, 91109
> >> maziyar.boustani@jpl.nasa.gov
> >>
> >> <model_vs_obs_bias.png>
> >>
> >>
> >>
> >
> >
> >--
> >Alex Goodman
> >Graduate Research Assistant
> >Department of Atmospheric Science
> >Colorado State University
>
>
--
Alex Goodman
Graduate Research Assistant
Department of Atmospheric Science
Colorado State University
Re: Model Observation Evaluation exmaple
Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.
THanks guys, Alex, great to see you!
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Alex Goodman <al...@colostate.edu>
Reply-To: "dev@climate.apache.org" <de...@climate.apache.org>
Date: Wednesday, June 4, 2014 3:53 PM
To: "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>
Cc: "Loikith, Paul C (329C-Affiliate)" <Pa...@jpl.nasa.gov>,
"dev@climate.apache.org" <de...@climate.apache.org>
Subject: Re: Model Observation Evaluation exmaple
>Hi Mazi,
>
>I am in agreement with Paul. It is also clear from a quick inspection of
>your plots that the model bias for each year is identical, so plotting
>just
>the 5-year average bias would give you sufficient information. You might
>want to take a quick glance at IPCC AR5 WG I, Ch. 9 [1] for some good
>examples.
>
>Thanks,
>Alex
>
>[1] -
>http://www.climatechange2013.org/images/report/WG1AR5_Chapter09_FINAL.pdf
>
>
>On Wed, Jun 4, 2014 at 4:48 PM, Boustani, Maziyar (398F) <
>Maziyar.Boustani@jpl.nasa.gov> wrote:
>
>> Here is the attached [1].
>>
>> [1]: http://oi58.tinypic.com/w7yo2q.jpg
>>
>> ŠŠŠŠŠŠŠŠ
>> Maziyar Boustani
>> Software Engineer - GIS Developer
>> Science Data Understanding Group
>> Jet Propulsion Laboratory, California Institute of Technology
>> 4800 Oak Grove Drive,
>> Pasadena, CA, 91109
>> maziyar.boustani@jpl.nasa.gov
>>
>> On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote:
>>
>> Hi Mazi,
>>
>> This should be one plot total. When you compute a bias, conventionally
>> it's just one bias for the entire time period so there should be one
>>plot.
>> You could have multiple plots if you were computing the bias for all 4
>> seasons, but for annual there should only be one.
>>
>> When you say you temporally regridded to 365 days, what do you mean by
>> this? The model and obs data are monthly, correct? So shouldn't you
>>just
>> be computing the mean difference between the obs and the model?
>>
>> Please let me know if you need further clarification.
>>
>> Thanks,
>>
>> Paul
>>
>>
>>
>> From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>
>> Date: Wednesday, June 4, 2014 3:12 PM
>> To: Paul Loikith <pa...@jpl.nasa.gov>, "goodman@apache.org" <
>> goodman@apache.org>
>> Cc: "dev@climate.apache.org" <de...@climate.apache.org>
>> Subject: Model Observation Evaluation exmaple
>>
>> Hi Paul and Alex,
>>
>> I ran a test code to evaluate one model [1] with one observation[2].
>>You
>> can find the result as attached.
>> This has been temporally regrided for 356 day, so we going to have 5
>>years
>> of data and that is why we have 5 plots.
>> Do you guys thinks this is correct to have one plot for each time or it
>> should be one plot for entire time?
>>
>> Metric = Bias
>> parameter_id = 38
>> dataset_id = 10
>> min_lat = 0
>> max_lat = 20
>> min_lon = 0
>> max_lon = 20
>> start_time = 2000-01-01 00:00:00
>> end_time = 2004-12-01 00:00:00
>>
>>
>> [1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable =
>> TAS)
>> [2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
>>
>> <model_vs_obs_bias.png>
>>
>> ŠŠŠŠŠŠŠŠ
>> Maziyar Boustani
>> Software Engineer - GIS Developer
>> Science Data Understanding Group
>> Jet Propulsion Laboratory, California Institute of Technology
>> 4800 Oak Grove Drive,
>> Pasadena, CA, 91109
>> maziyar.boustani@jpl.nasa.gov
>>
>> <model_vs_obs_bias.png>
>>
>>
>>
>
>
>--
>Alex Goodman
>Graduate Research Assistant
>Department of Atmospheric Science
>Colorado State University
Re: Model Observation Evaluation exmaple
Posted by "Loikith, Paul C (329C-Affiliate)" <Pa...@jpl.nasa.gov>.
Hi Alex,
Just to clarify…wouldn't you always be plotting the 5-year average bias when computing a "bias" for a 5-year climatology? It looks like the bias is systematic for the 5 years, but not identical.
Paul
From: Alex Goodman <al...@colostate.edu>>
Date: Wednesday, June 4, 2014 3:53 PM
To: "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>>
Cc: Paul Loikith <pa...@jpl.nasa.gov>>, "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Subject: Re: Model Observation Evaluation exmaple
Hi Mazi,
I am in agreement with Paul. It is also clear from a quick inspection of your plots that the model bias for each year is identical, so plotting just the 5-year average bias would give you sufficient information. You might want to take a quick glance at IPCC AR5 WG I, Ch. 9 [1] for some good examples.
Thanks,
Alex
[1] - http://www.climatechange2013.org/images/report/WG1AR5_Chapter09_FINAL.pdf
On Wed, Jun 4, 2014 at 4:48 PM, Boustani, Maziyar (398F) <Ma...@jpl.nasa.gov>> wrote:
Here is the attached [1].
[1]: http://oi58.tinypic.com/w7yo2q.jpg
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote:
Hi Mazi,
This should be one plot total. When you compute a bias, conventionally it's just one bias for the entire time period so there should be one plot. You could have multiple plots if you were computing the bias for all 4 seasons, but for annual there should only be one.
When you say you temporally regridded to 365 days, what do you mean by this? The model and obs data are monthly, correct? So shouldn't you just be computing the mean difference between the obs and the model?
Please let me know if you need further clarification.
Thanks,
Paul
From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>>
Date: Wednesday, June 4, 2014 3:12 PM
To: Paul Loikith <pa...@jpl.nasa.gov>>, "goodman@apache.org<ma...@apache.org>" <go...@apache.org>>
Cc: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Subject: Model Observation Evaluation exmaple
Hi Paul and Alex,
I ran a test code to evaluate one model [1] with one observation[2]. You can find the result as attached.
This has been temporally regrided for 356 day, so we going to have 5 years of data and that is why we have 5 plots.
Do you guys thinks this is correct to have one plot for each time or it should be one plot for entire time?
Metric = Bias
parameter_id = 38
dataset_id = 10
min_lat = 0
max_lat = 20
min_lon = 0
max_lon = 20
start_time = 2000-01-01 00:00:00
end_time = 2004-12-01 00:00:00
[1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable = TAS)
[2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
<model_vs_obs_bias.png>
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
<model_vs_obs_bias.png>
--
Alex Goodman
Graduate Research Assistant
Department of Atmospheric Science
Colorado State University
Re: Model Observation Evaluation exmaple
Posted by Alex Goodman <al...@colostate.edu>.
Hi Mazi,
I am in agreement with Paul. It is also clear from a quick inspection of
your plots that the model bias for each year is identical, so plotting just
the 5-year average bias would give you sufficient information. You might
want to take a quick glance at IPCC AR5 WG I, Ch. 9 [1] for some good
examples.
Thanks,
Alex
[1] -
http://www.climatechange2013.org/images/report/WG1AR5_Chapter09_FINAL.pdf
On Wed, Jun 4, 2014 at 4:48 PM, Boustani, Maziyar (398F) <
Maziyar.Boustani@jpl.nasa.gov> wrote:
> Here is the attached [1].
>
> [1]: http://oi58.tinypic.com/w7yo2q.jpg
>
> ……………………
> Maziyar Boustani
> Software Engineer - GIS Developer
> Science Data Understanding Group
> Jet Propulsion Laboratory, California Institute of Technology
> 4800 Oak Grove Drive,
> Pasadena, CA, 91109
> maziyar.boustani@jpl.nasa.gov
>
> On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote:
>
> Hi Mazi,
>
> This should be one plot total. When you compute a bias, conventionally
> it's just one bias for the entire time period so there should be one plot.
> You could have multiple plots if you were computing the bias for all 4
> seasons, but for annual there should only be one.
>
> When you say you temporally regridded to 365 days, what do you mean by
> this? The model and obs data are monthly, correct? So shouldn't you just
> be computing the mean difference between the obs and the model?
>
> Please let me know if you need further clarification.
>
> Thanks,
>
> Paul
>
>
>
> From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>
> Date: Wednesday, June 4, 2014 3:12 PM
> To: Paul Loikith <pa...@jpl.nasa.gov>, "goodman@apache.org" <
> goodman@apache.org>
> Cc: "dev@climate.apache.org" <de...@climate.apache.org>
> Subject: Model Observation Evaluation exmaple
>
> Hi Paul and Alex,
>
> I ran a test code to evaluate one model [1] with one observation[2]. You
> can find the result as attached.
> This has been temporally regrided for 356 day, so we going to have 5 years
> of data and that is why we have 5 plots.
> Do you guys thinks this is correct to have one plot for each time or it
> should be one plot for entire time?
>
> Metric = Bias
> parameter_id = 38
> dataset_id = 10
> min_lat = 0
> max_lat = 20
> min_lon = 0
> max_lon = 20
> start_time = 2000-01-01 00:00:00
> end_time = 2004-12-01 00:00:00
>
>
> [1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable =
> TAS)
> [2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
>
> <model_vs_obs_bias.png>
>
> ……………………
> Maziyar Boustani
> Software Engineer - GIS Developer
> Science Data Understanding Group
> Jet Propulsion Laboratory, California Institute of Technology
> 4800 Oak Grove Drive,
> Pasadena, CA, 91109
> maziyar.boustani@jpl.nasa.gov
>
> <model_vs_obs_bias.png>
>
>
>
--
Alex Goodman
Graduate Research Assistant
Department of Atmospheric Science
Colorado State University
Re: Model Observation Evaluation exmaple
Posted by "Loikith, Paul C (329C-Affiliate)" <Pa...@jpl.nasa.gov>.
Perfect!
From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>>
Date: Thursday, June 5, 2014 4:12 PM
To: Kyo Lee <Hu...@jpl.nasa.gov>>, Paul Loikith <pa...@jpl.nasa.gov>>
Cc: "Joyce, Michael J (398J)" <Mi...@jpl.nasa.gov>>, "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Subject: Re: Model Observation Evaluation exmaple
Guys,
I fixed the color bar [1], does that look better?
[1]: http://oi58.tinypic.com/t50uty.jpg
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
On Jun 5, 2014, at 3:57 PM, Lee, Kyo (329C-Affiliate) wrote:
Thanks, Mazi.
The new plot looks okay except that the color bar is not symmetric with
respect to zero as Paul pointed out.
Kyo
On 6/4/14, 3:48 PM, "Boustani, Maziyar (398F)"
<Ma...@jpl.nasa.gov>> wrote:
Here is the attached [1].
[1]: http://oi58.tinypic.com/w7yo2q.jpg
ŠŠŠŠŠŠŠŠ
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote:
Hi Mazi,
This should be one plot total. When you compute a bias, conventionally
it's just one bias for the entire time period so there should be one
plot. You could have multiple plots if you were computing the bias for
all 4 seasons, but for annual there should only be one.
When you say you temporally regridded to 365 days, what do you mean by
this? The model and obs data are monthly, correct? So shouldn't you
just be computing the mean difference between the obs and the model?
Please let me know if you need further clarification.
Thanks,
Paul
From: <Boustani>, "Maziyar (398F)"
<Ma...@jpl.nasa.gov>>
Date: Wednesday, June 4, 2014 3:12 PM
To: Paul Loikith
<pa...@jpl.nasa.gov>>,
"goodman@apache.org<ma...@apache.org>"
<go...@apache.org>>
Cc: "dev@climate.apache.org<ma...@climate.apache.org>"
<de...@climate.apache.org>>
Subject: Model Observation Evaluation exmaple
Hi Paul and Alex,
I ran a test code to evaluate one model [1] with one observation[2]. You
can find the result as attached.
This has been temporally regrided for 356 day, so we going to have 5
years of data and that is why we have 5 plots.
Do you guys thinks this is correct to have one plot for each time or it
should be one plot for entire time?
Metric = Bias
parameter_id = 38
dataset_id = 10
min_lat = 0
max_lat = 20
min_lon = 0
max_lon = 20
start_time = 2000-01-01 00:00:00
end_time = 2004-12-01 00:00:00
[1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable =
TAS)
[2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
<model_vs_obs_bias.png>
ŠŠŠŠŠŠŠŠ
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
<model_vs_obs_bias.png>
Re: Model Observation Evaluation exmaple
Posted by "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>.
Guys,
I fixed the color bar [1], does that look better?
[1]: http://oi58.tinypic.com/t50uty.jpg
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
On Jun 5, 2014, at 3:57 PM, Lee, Kyo (329C-Affiliate) wrote:
Thanks, Mazi.
The new plot looks okay except that the color bar is not symmetric with
respect to zero as Paul pointed out.
Kyo
On 6/4/14, 3:48 PM, "Boustani, Maziyar (398F)"
<Ma...@jpl.nasa.gov>> wrote:
Here is the attached [1].
[1]: http://oi58.tinypic.com/w7yo2q.jpg
ŠŠŠŠŠŠŠŠ
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote:
Hi Mazi,
This should be one plot total. When you compute a bias, conventionally
it's just one bias for the entire time period so there should be one
plot. You could have multiple plots if you were computing the bias for
all 4 seasons, but for annual there should only be one.
When you say you temporally regridded to 365 days, what do you mean by
this? The model and obs data are monthly, correct? So shouldn't you
just be computing the mean difference between the obs and the model?
Please let me know if you need further clarification.
Thanks,
Paul
From: <Boustani>, "Maziyar (398F)"
<Ma...@jpl.nasa.gov>>
Date: Wednesday, June 4, 2014 3:12 PM
To: Paul Loikith
<pa...@jpl.nasa.gov>>,
"goodman@apache.org<ma...@apache.org>"
<go...@apache.org>>
Cc: "dev@climate.apache.org<ma...@climate.apache.org>"
<de...@climate.apache.org>>
Subject: Model Observation Evaluation exmaple
Hi Paul and Alex,
I ran a test code to evaluate one model [1] with one observation[2]. You
can find the result as attached.
This has been temporally regrided for 356 day, so we going to have 5
years of data and that is why we have 5 plots.
Do you guys thinks this is correct to have one plot for each time or it
should be one plot for entire time?
Metric = Bias
parameter_id = 38
dataset_id = 10
min_lat = 0
max_lat = 20
min_lon = 0
max_lon = 20
start_time = 2000-01-01 00:00:00
end_time = 2004-12-01 00:00:00
[1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable =
TAS)
[2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
<model_vs_obs_bias.png>
ŠŠŠŠŠŠŠŠ
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
<model_vs_obs_bias.png>
Re: Model Observation Evaluation exmaple
Posted by "Boustani, Maziyar (398F)" <Ma...@jpl.nasa.gov>.
Here is the attached [1].
[1]: http://oi58.tinypic.com/w7yo2q.jpg
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
On Jun 4, 2014, at 3:35 PM, Loikith, Paul C (329C-Affiliate) wrote:
Hi Mazi,
This should be one plot total. When you compute a bias, conventionally it's just one bias for the entire time period so there should be one plot. You could have multiple plots if you were computing the bias for all 4 seasons, but for annual there should only be one.
When you say you temporally regridded to 365 days, what do you mean by this? The model and obs data are monthly, correct? So shouldn't you just be computing the mean difference between the obs and the model?
Please let me know if you need further clarification.
Thanks,
Paul
From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>>
Date: Wednesday, June 4, 2014 3:12 PM
To: Paul Loikith <pa...@jpl.nasa.gov>>, "goodman@apache.org<ma...@apache.org>" <go...@apache.org>>
Cc: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Subject: Model Observation Evaluation exmaple
Hi Paul and Alex,
I ran a test code to evaluate one model [1] with one observation[2]. You can find the result as attached.
This has been temporally regrided for 356 day, so we going to have 5 years of data and that is why we have 5 plots.
Do you guys thinks this is correct to have one plot for each time or it should be one plot for entire time?
Metric = Bias
parameter_id = 38
dataset_id = 10
min_lat = 0
max_lat = 20
min_lon = 0
max_lon = 20
start_time = 2000-01-01 00:00:00
end_time = 2004-12-01 00:00:00
[1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable = TAS)
[2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
<model_vs_obs_bias.png>
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>
<model_vs_obs_bias.png>
Re: Model Observation Evaluation exmaple
Posted by "Loikith, Paul C (329C-Affiliate)" <Pa...@jpl.nasa.gov>.
Hi Mazi,
This should be one plot total. When you compute a bias, conventionally it's just one bias for the entire time period so there should be one plot. You could have multiple plots if you were computing the bias for all 4 seasons, but for annual there should only be one.
When you say you temporally regridded to 365 days, what do you mean by this? The model and obs data are monthly, correct? So shouldn't you just be computing the mean difference between the obs and the model?
Please let me know if you need further clarification.
Thanks,
Paul
From: <Boustani>, "Maziyar (398F)" <Ma...@jpl.nasa.gov>>
Date: Wednesday, June 4, 2014 3:12 PM
To: Paul Loikith <pa...@jpl.nasa.gov>>, "goodman@apache.org<ma...@apache.org>" <go...@apache.org>>
Cc: "dev@climate.apache.org<ma...@climate.apache.org>" <de...@climate.apache.org>>
Subject: Model Observation Evaluation exmaple
Hi Paul and Alex,
I ran a test code to evaluate one model [1] with one observation[2]. You can find the result as attached.
This has been temporally regrided for 356 day, so we going to have 5 years of data and that is why we have 5 plots.
Do you guys thinks this is correct to have one plot for each time or it should be one plot for entire time?
Metric = Bias
parameter_id = 38
dataset_id = 10
min_lat = 0
max_lat = 20
min_lon = 0
max_lon = 20
start_time = 2000-01-01 00:00:00
end_time = 2004-12-01 00:00:00
[1] : AFRICA_UQAM-CRCM5_CTL_ERAINT_MM_50km_1989-2008_tas.nc (variable = TAS)
[2] : CRU3.1 Daily-Mean Temperature (parmeter_id = 38)
[cid:03DF99EA-3331-4A42-974E-7131165A8C00]
……………………
Maziyar Boustani
Software Engineer - GIS Developer
Science Data Understanding Group
Jet Propulsion Laboratory, California Institute of Technology
4800 Oak Grove Drive,
Pasadena, CA, 91109
maziyar.boustani@jpl.nasa.gov<ma...@jpl.nasa.gov>