You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Imran Rajjad <ra...@gmail.com> on 2017/10/10 09:21:56 UTC
best spark spatial lib?
I need to have a location column inside my Dataframe so that I can do
spatial queries and geometry operations. Are there any third-party packages
that perform this kind of operations. I have seen a few like Geospark and
megalan but they don't support operations where spatial and logical
operators can be combined.
regards,
Imran
--
I.R
Re: best spark spatial lib?
Posted by Silvio Fiorito <si...@granturing.com>.
There’s a number of packages for geospatial analysis, depends on the features you need. Here are a few I know of and/or have used:
Magellan: https://github.com/harsha2010/magellan
MrGeo: https://github.com/ngageoint/mrgeo
GeoMesa: http://www.geomesa.org/documentation/tutorials/spark.html
GeoSpark (just heard of this): http://geospark.datasyslab.org/
Thanks,
Silvio
From: Imran Rajjad <ra...@gmail.com>
Date: Tuesday, October 10, 2017 at 5:22 AM
To: "user @spark" <us...@spark.apache.org>
Subject: best spark spatial lib?
I need to have a location column inside my Dataframe so that I can do spatial queries and geometry operations. Are there any third-party packages that perform this kind of operations. I have seen a few like Geospark and megalan but they don't support operations where spatial and logical operators can be combined.
regards,
Imran
--
I.R
Re: best spark spatial lib?
Posted by Imran Rajjad <ra...@gmail.com>.
Thanks guy for the response.
Basically I am migrating an oracle pl/sql procedure to spark-java. In
oracle I have a table with geometry column, on which I am able to do a
"where col = 1 and geom.within(another_geom)"
I am looking for a less complicated port in to spark for which queries. I
will give these libraries a shot.
@Ram .. Magellan I gues does not support Java
regards,
Imran
On Wed, Oct 11, 2017 at 12:07 AM, Ram Sriharsha <sr...@gmail.com>
wrote:
> why can't you do this in Magellan?
> Can you post a sample query that you are trying to run that has spatial
> and logical operators combined? Maybe I am not understanding the issue
> properly
>
> Ram
>
> On Tue, Oct 10, 2017 at 2:21 AM, Imran Rajjad <ra...@gmail.com> wrote:
>
>> I need to have a location column inside my Dataframe so that I can do
>> spatial queries and geometry operations. Are there any third-party packages
>> that perform this kind of operations. I have seen a few like Geospark and
>> megalan but they don't support operations where spatial and logical
>> operators can be combined.
>>
>> regards,
>> Imran
>>
>> --
>> I.R
>>
>
>
--
I.R
Re: best spark spatial lib?
Posted by Ram Sriharsha <sr...@gmail.com>.
why can't you do this in Magellan?
Can you post a sample query that you are trying to run that has spatial and
logical operators combined? Maybe I am not understanding the issue properly
Ram
On Tue, Oct 10, 2017 at 2:21 AM, Imran Rajjad <ra...@gmail.com> wrote:
> I need to have a location column inside my Dataframe so that I can do
> spatial queries and geometry operations. Are there any third-party packages
> that perform this kind of operations. I have seen a few like Geospark and
> megalan but they don't support operations where spatial and logical
> operators can be combined.
>
> regards,
> Imran
>
> --
> I.R
>
Re: best spark spatial lib?
Posted by Jim Hughes <jn...@ccri.com>.
Hi all,
GeoMesa integrates with Spark SQL and allows for queries like:
select * from chicago where case_number = 1 and st_intersects(geom,
st_makeBox2d(st_point(-77, 38), st_point(-76, 39)))
GeoMesa does this by calling package protected Spark methods to
implement geospatial user defined types and functions.
Cheers,
Jim
On 10/10/2017 11:30 AM, Georg Heiler wrote:
> What about someting like gromesa?
> Anastasios Zouzias <zouzias@gmail.com <ma...@gmail.com>>
> schrieb am Di. 10. Okt. 2017 um 15:29:
>
> Hi,
>
> Which spatial operations do you require exactly? Also, I don't
> follow what you mean by combining logical operators?
>
> I have created a library that wraps Lucene's spatial functionality
> here: https://github.com/zouzias/spark-lucenerdd/wiki/Spatial-search
>
> You could give a try to the library, it supports intersections /
> within / etc. Ideally, I try to push all spatial Lucene features
> in the library.
>
> See also,
> https://github.com/zouzias/spark-lucenerdd/wiki/Related-Work for
> related libraries.
>
> Best,
> Anastasios
>
>
> On Tue, Oct 10, 2017 at 11:21 AM, Imran Rajjad <rajjad@gmail.com
> <ma...@gmail.com>> wrote:
>
> I need to have a location column inside my Dataframe so that I
> can do spatial queries and geometry operations. Are there any
> third-party packages that perform this kind of operations. I
> have seen a few like Geospark and megalan but they don't
> support operations where spatial and logical operators can be
> combined.
>
> regards,
> Imran
>
> --
> I.R
>
>
>
>
> --
> -- Anastasios Zouzias
>
Re: best spark spatial lib?
Posted by Georg Heiler <ge...@gmail.com>.
What about someting like gromesa?
Anastasios Zouzias <zo...@gmail.com> schrieb am Di. 10. Okt. 2017 um
15:29:
> Hi,
>
> Which spatial operations do you require exactly? Also, I don't follow what
> you mean by combining logical operators?
>
> I have created a library that wraps Lucene's spatial functionality here:
> https://github.com/zouzias/spark-lucenerdd/wiki/Spatial-search
>
> You could give a try to the library, it supports intersections / within /
> etc. Ideally, I try to push all spatial Lucene features in the library.
>
> See also, https://github.com/zouzias/spark-lucenerdd/wiki/Related-Work
> for related libraries.
>
> Best,
> Anastasios
>
>
> On Tue, Oct 10, 2017 at 11:21 AM, Imran Rajjad <ra...@gmail.com> wrote:
>
>> I need to have a location column inside my Dataframe so that I can do
>> spatial queries and geometry operations. Are there any third-party packages
>> that perform this kind of operations. I have seen a few like Geospark and
>> megalan but they don't support operations where spatial and logical
>> operators can be combined.
>>
>> regards,
>> Imran
>>
>> --
>> I.R
>>
>
>
>
> --
> -- Anastasios Zouzias
> <az...@zurich.ibm.com>
>
Re: best spark spatial lib?
Posted by Anastasios Zouzias <zo...@gmail.com>.
Hi,
Which spatial operations do you require exactly? Also, I don't follow what
you mean by combining logical operators?
I have created a library that wraps Lucene's spatial functionality here:
https://github.com/zouzias/spark-lucenerdd/wiki/Spatial-search
You could give a try to the library, it supports intersections / within /
etc. Ideally, I try to push all spatial Lucene features in the library.
See also, https://github.com/zouzias/spark-lucenerdd/wiki/Related-Work for
related libraries.
Best,
Anastasios
On Tue, Oct 10, 2017 at 11:21 AM, Imran Rajjad <ra...@gmail.com> wrote:
> I need to have a location column inside my Dataframe so that I can do
> spatial queries and geometry operations. Are there any third-party packages
> that perform this kind of operations. I have seen a few like Geospark and
> megalan but they don't support operations where spatial and logical
> operators can be combined.
>
> regards,
> Imran
>
> --
> I.R
>
--
-- Anastasios Zouzias
<az...@zurich.ibm.com>