You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov> on 2015/01/15 03:08:45 UTC
SciSpark: NASA AIST14 proposal
Hi Spark Devs,
Just wanted to FYI that I was funded on a 2 year NASA proposal
to build out the concept of a scientific RDD (create by space/time,
and other operations) for use in some neat climate related NASA
use cases.
http://esto.nasa.gov/files/solicitations/AIST_14/ROSES2014_AIST_A41_awards.
html
I will keep everyone posted and plan on interacting with the list
over here to get it done. I expect that we’ll start work in March.
In the meanwhile you guys can scope the abstract at the link provided.
Happy
to chat about it if you have any questions too.
Cheers!
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: SciSpark: NASA AIST14 proposal
Posted by andy petrella <an...@gmail.com>.
Hey Chris,
This sounds amazing!
You might have to check also with the Geotrellis
<https://github.com/geotrellis/geotrellis> team (Rob and Eugene for
instance) who have already covered quite interesting ground dealing with
tiles as RDD element.
Some algebra operations are there, but also thingies like Shortest Path
(within rasters).
A small, I've a student who is working on a implementation of LU/LC using
Spark (first using CA, then I hope extension using stochastic methods like
local random forest).
If you consider implementing a R-Tree (or perhaps the SD version) for OGIs
operation, I thought that IndexedRDD
<https://github.com/amplab/spark-indexedrdd> could be interesting to
consider (I've been asked to look at options to implement this kind of
distributed and resilient R-Tree, so I'll be happy to see how it'd perform
^^).
cheers and have fun!
andy
On Thu Jan 15 2015 at 5:53:27 AM RJ Nowling <rn...@gmail.com> wrote:
> Congratulations, Chris!
>
> I created a JIRA for "dimensional" RDDs that might be relevant:
> https://issues.apache.org/jira/browse/SPARK-4727
>
> Jeremy Freeman pointed me to his lab's work on for neuroscience that have
> some related functionality :
> http://thefreemanlab.com/thunder/
>
> On Wed, Jan 14, 2015 at 11:07 PM, Aniket <an...@gmail.com>
> wrote:
>
> > Hi Chris
> >
> > This is super cool. I was wondering if this would be an open source
> project
> > so that people can contribute or reuse?
> >
> > Thanks,
> > Aniket
> >
> > On Thu Jan 15 2015 at 07:39:29 Mattmann, Chris A (3980) [via Apache Spark
> > Developers List] <ml...@n3.nabble.com> wrote:
> >
> > > Hi Spark Devs,
> > >
> > > Just wanted to FYI that I was funded on a 2 year NASA proposal
> > > to build out the concept of a scientific RDD (create by space/time,
> > > and other operations) for use in some neat climate related NASA
> > > use cases.
> > >
> > >
> > http://esto.nasa.gov/files/solicitations/AIST_14/
> ROSES2014_AIST_A41_awards
> > .
> > >
> > > html
> > >
> > >
> > > I will keep everyone posted and plan on interacting with the list
> > > over here to get it done. I expect that we’ll start work in March.
> > > In the meanwhile you guys can scope the abstract at the link provided.
> > > Happy
> > > to chat about it if you have any questions too.
> > >
> > > Cheers!
> > >
> > > Chris
> > >
> > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > Chris Mattmann, Ph.D.
> > > Chief Architect
> > > Instrument Software and Science Data Systems Section (398)
> > > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> > > Office: 168-519, Mailstop: 168-527
> > > Email: [hidden email]
> > > <http:///user/SendEmail.jtp?type=node&node=10115&i=0>
> > > WWW: http://sunset.usc.edu/~mattmann/
> > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > Adjunct Associate Professor, Computer Science Department
> > > University of Southern California, Los Angeles, CA 90089 USA
> > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > >
> > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > <http:///user/SendEmail.jtp?type=node&node=10115&i=1>
> > > For additional commands, e-mail: [hidden email]
> > > <http:///user/SendEmail.jtp?type=node&node=10115&i=2>
> > >
> > >
> > > ------------------------------
> > > If you reply to this email, your message will be added to the
> discussion
> > > below:
> > >
> > >
> > http://apache-spark-developers-list.1001551.n3.nabble.com/SciSpark-NASA-
> AIST14-proposal-tp10115.html
> > > To start a new topic under Apache Spark Developers List, email
> > > ml-node+s1001551n1h76@n3.nabble.com
> > > To unsubscribe from Apache Spark Developers List, click here
> > > <
> > http://apache-spark-developers-list.1001551.n3.nabble.com/template/
> NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=
> YW5pa2V0LmJoYXRuYWdhckBnbWFpbC5jb218MXwxMzE3NTAzMzQz
> > >
> > > .
> > > NAML
> > > <
> > http://apache-spark-developers-list.1001551.n3.nabble.com/template/
> NamlServlet.jtp?macro=macro_viewer&id=instant_html%
> 21nabble%3Aemail.naml&base=nabble.naml.namespaces.
> BasicNamespace-nabble.view.web.template.NabbleNamespace-
> nabble.naml.namespaces.BasicNamespace-nabble.view.
> web.template.NabbleNamespace-nabble.view.web.template.
> NodeNamespace&breadcrumbs=notify_subscribers%21nabble%
> 3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_
> instant_email%21nabble%3Aemail.naml
> > >
> > >
> >
> >
> >
> >
> > --
> > View this message in context:
> > http://apache-spark-developers-list.1001551.n3.nabble.com/SciSpark-NASA-
> AIST14-proposal-tp10115p10118.html
> > Sent from the Apache Spark Developers List mailing list archive at
> > Nabble.com.
>
Re: SciSpark: NASA AIST14 proposal
Posted by RJ Nowling <rn...@gmail.com>.
Congratulations, Chris!
I created a JIRA for "dimensional" RDDs that might be relevant:
https://issues.apache.org/jira/browse/SPARK-4727
Jeremy Freeman pointed me to his lab's work on for neuroscience that have
some related functionality :
http://thefreemanlab.com/thunder/
On Wed, Jan 14, 2015 at 11:07 PM, Aniket <an...@gmail.com> wrote:
> Hi Chris
>
> This is super cool. I was wondering if this would be an open source project
> so that people can contribute or reuse?
>
> Thanks,
> Aniket
>
> On Thu Jan 15 2015 at 07:39:29 Mattmann, Chris A (3980) [via Apache Spark
> Developers List] <ml...@n3.nabble.com> wrote:
>
> > Hi Spark Devs,
> >
> > Just wanted to FYI that I was funded on a 2 year NASA proposal
> > to build out the concept of a scientific RDD (create by space/time,
> > and other operations) for use in some neat climate related NASA
> > use cases.
> >
> >
> http://esto.nasa.gov/files/solicitations/AIST_14/ROSES2014_AIST_A41_awards
> .
> >
> > html
> >
> >
> > I will keep everyone posted and plan on interacting with the list
> > over here to get it done. I expect that we’ll start work in March.
> > In the meanwhile you guys can scope the abstract at the link provided.
> > Happy
> > to chat about it if you have any questions too.
> >
> > Cheers!
> >
> > Chris
> >
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > Chris Mattmann, Ph.D.
> > Chief Architect
> > Instrument Software and Science Data Systems Section (398)
> > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> > Office: 168-519, Mailstop: 168-527
> > Email: [hidden email]
> > <http:///user/SendEmail.jtp?type=node&node=10115&i=0>
> > WWW: http://sunset.usc.edu/~mattmann/
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > Adjunct Associate Professor, Computer Science Department
> > University of Southern California, Los Angeles, CA 90089 USA
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > <http:///user/SendEmail.jtp?type=node&node=10115&i=1>
> > For additional commands, e-mail: [hidden email]
> > <http:///user/SendEmail.jtp?type=node&node=10115&i=2>
> >
> >
> > ------------------------------
> > If you reply to this email, your message will be added to the discussion
> > below:
> >
> >
> http://apache-spark-developers-list.1001551.n3.nabble.com/SciSpark-NASA-AIST14-proposal-tp10115.html
> > To start a new topic under Apache Spark Developers List, email
> > ml-node+s1001551n1h76@n3.nabble.com
> > To unsubscribe from Apache Spark Developers List, click here
> > <
> http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=YW5pa2V0LmJoYXRuYWdhckBnbWFpbC5jb218MXwxMzE3NTAzMzQz
> >
> > .
> > NAML
> > <
> http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
> >
>
>
>
>
> --
> View this message in context:
> http://apache-spark-developers-list.1001551.n3.nabble.com/SciSpark-NASA-AIST14-proposal-tp10115p10118.html
> Sent from the Apache Spark Developers List mailing list archive at
> Nabble.com.
Re: SciSpark: NASA AIST14 proposal
Posted by Aniket <an...@gmail.com>.
Hi Chris
This is super cool. I was wondering if this would be an open source project
so that people can contribute or reuse?
Thanks,
Aniket
On Thu Jan 15 2015 at 07:39:29 Mattmann, Chris A (3980) [via Apache Spark
Developers List] <ml...@n3.nabble.com> wrote:
> Hi Spark Devs,
>
> Just wanted to FYI that I was funded on a 2 year NASA proposal
> to build out the concept of a scientific RDD (create by space/time,
> and other operations) for use in some neat climate related NASA
> use cases.
>
> http://esto.nasa.gov/files/solicitations/AIST_14/ROSES2014_AIST_A41_awards.
>
> html
>
>
> I will keep everyone posted and plan on interacting with the list
> over here to get it done. I expect that we’ll start work in March.
> In the meanwhile you guys can scope the abstract at the link provided.
> Happy
> to chat about it if you have any questions too.
>
> Cheers!
>
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: [hidden email]
> <http:///user/SendEmail.jtp?type=node&node=10115&i=0>
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> <http:///user/SendEmail.jtp?type=node&node=10115&i=1>
> For additional commands, e-mail: [hidden email]
> <http:///user/SendEmail.jtp?type=node&node=10115&i=2>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-spark-developers-list.1001551.n3.nabble.com/SciSpark-NASA-AIST14-proposal-tp10115.html
> To start a new topic under Apache Spark Developers List, email
> ml-node+s1001551n1h76@n3.nabble.com
> To unsubscribe from Apache Spark Developers List, click here
> <http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=YW5pa2V0LmJoYXRuYWdhckBnbWFpbC5jb218MXwxMzE3NTAzMzQz>
> .
> NAML
> <http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/SciSpark-NASA-AIST14-proposal-tp10115p10118.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
Re: SciSpark: NASA AIST14 proposal
Posted by Matei Zaharia <ma...@gmail.com>.
Yeah, very cool! You may also want to check out https://issues.apache.org/jira/browse/SPARK-5097 as something to build upon for these operations.
Matei
> On Jan 14, 2015, at 6:18 PM, Reynold Xin <rx...@databricks.com> wrote:
>
> Chris,
>
> This is really cool. Congratulations and thanks for sharing the news.
>
>
> On Wed, Jan 14, 2015 at 6:08 PM, Mattmann, Chris A (3980) <
> chris.a.mattmann@jpl.nasa.gov> wrote:
>
>> Hi Spark Devs,
>>
>> Just wanted to FYI that I was funded on a 2 year NASA proposal
>> to build out the concept of a scientific RDD (create by space/time,
>> and other operations) for use in some neat climate related NASA
>> use cases.
>>
>> http://esto.nasa.gov/files/solicitations/AIST_14/ROSES2014_AIST_A41_awards.
>> html
>>
>>
>> I will keep everyone posted and plan on interacting with the list
>> over here to get it done. I expect that we’ll start work in March.
>> In the meanwhile you guys can scope the abstract at the link provided.
>> Happy
>> to chat about it if you have any questions too.
>>
>> Cheers!
>>
>> Chris
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Chief Architect
>> Instrument Software and Science Data Systems Section (398)
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 168-519, Mailstop: 168-527
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Associate Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: SciSpark: NASA AIST14 proposal
Posted by Reynold Xin <rx...@databricks.com>.
Chris,
This is really cool. Congratulations and thanks for sharing the news.
On Wed, Jan 14, 2015 at 6:08 PM, Mattmann, Chris A (3980) <
chris.a.mattmann@jpl.nasa.gov> wrote:
> Hi Spark Devs,
>
> Just wanted to FYI that I was funded on a 2 year NASA proposal
> to build out the concept of a scientific RDD (create by space/time,
> and other operations) for use in some neat climate related NASA
> use cases.
>
> http://esto.nasa.gov/files/solicitations/AIST_14/ROSES2014_AIST_A41_awards.
> html
>
>
> I will keep everyone posted and plan on interacting with the list
> over here to get it done. I expect that we’ll start work in March.
> In the meanwhile you guys can scope the abstract at the link provided.
> Happy
> to chat about it if you have any questions too.
>
> Cheers!
>
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>