You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sedona.apache.org by Netanel Malka <ma...@apache.org> on 2021/02/17 13:35:12 UTC
Develop new features/fix bugs
Hi Jia,
Are there any features that we want to develop? or fix some bugs?
My free time now allows me to make some contributions to the project.
Thanks,
Re: Develop new features/fix bugs
Posted by Jia Yu <ji...@apache.org>.
That sounds good to me.
I always use Intellij to develop Sedona. It looks like Intellij has a bug
when handling inherited POM. Please try to comment out this "parent" part:
https://github.com/apache/incubator-sedona/blob/master/pom.xml#L21 Then
probably the problem will be gone.
But don't commit this change in Git. Otherwise we won't be able to publish
ASF releases.
Thanks,
Jia
On Sat, Feb 20, 2021 at 11:15 AM Netanel Malka <ne...@gmail.com> wrote:
> Thanks, Jia.
> I think I will start with the first issue.
> I will create a JIRA Ticket.
>
> By the way, I tried to work on Intellij IDEA and I can't make the project
> work. There are dependencies issues that only appear on Intellij and not
> when using maven. Hence, I am working with VS code which appears to work
> well.
>
> Do you work with Intellij as a code editor on the Sedona project?
>
> Thanks.
>
> On Thu, 18 Feb 2021 at 23:02, Paweł Kociński <pa...@gmail.com>
> wrote:
>
>> Hi Jia,
>> Do we have some estimates about next release ? I think it is good idea to
>> create JIRA tasks from that link
>>
>> https://lists.apache.org/thread.html/r4ebf7c253fff77900cac8b6ad478c8f4808b707062dd8c6b40853a6f%40%3Cdev.sedona.apache.org%3E
>> .
>> I can focus on point Implement loading geospatial data sources (geojson,
>> osm) and postgis direct support.
>> Regards,
>> Paweł
>>
>> czw., 18 lut 2021 o 06:38 Jia Yu <ji...@gmail.com> napisał(a):
>>
>> > H Netanel,
>> >
>> > Thanks for asking. I think there are a couple of issues to be improved,
>> as
>> > listed here:
>> >
>> https://lists.apache.org/thread.html/r4ebf7c253fff77900cac8b6ad478c8f4808b707062dd8c6b40853a6f%40%3Cdev.sedona.apache.org%3E
>> >
>> > On the list given by Pawel and my thoughts, I think the most important
>> > thing to be fixed is that (sorted by importance):
>> >
>> > 1. Replace the object serializer in RDD API with the WKB serializer you
>> > wrote before, currently only DataFrame API uses WKB based serializer. In
>> > Sedona-core, there is a small benchmark that can test the performance of
>> > the WKB serializer.
>> > 2. Allow broadcast join for big+small data frames, or at least give
>> users
>> > a way to force the broadcast join. Currently, once the optimized Sedona
>> > join is enabled, the broadcast join is automatically disabled.
>> > 3. Allow null spatial objects in SpatialRDD and DataFrame. Or at least
>> > give users a meaningful exception:
>> > https://issues.apache.org/jira/browse/SEDONA-4
>> >
>> > Pawel was working on some improvements for GeoJSON loading. So we can
>> > leave it for him.
>> >
>> > Do you have any other issues you want to work on?
>> >
>> > Thanks,
>> > Jia
>> >
>> > On Wed, Feb 17, 2021 at 5:35 AM Netanel Malka <ma...@apache.org> wrote:
>> >
>> >> Hi Jia,
>> >> Are there any features that we want to develop? or fix some bugs?
>> >>
>> >> My free time now allows me to make some contributions to the project.
>> >>
>> >> Thanks,
>> >>
>> >>
>>
>
>
> --
> Best regards,
> Netanel Malka.
>
Re: Develop new features/fix bugs
Posted by Netanel Malka <ne...@gmail.com>.
Thanks, Jia.
I think I will start with the first issue.
I will create a JIRA Ticket.
By the way, I tried to work on Intellij IDEA and I can't make the project
work. There are dependencies issues that only appear on Intellij and not
when using maven. Hence, I am working with VS code which appears to work
well.
Do you work with Intellij as a code editor on the Sedona project?
Thanks.
On Thu, 18 Feb 2021 at 23:02, Paweł Kociński <pa...@gmail.com>
wrote:
> Hi Jia,
> Do we have some estimates about next release ? I think it is good idea to
> create JIRA tasks from that link
>
> https://lists.apache.org/thread.html/r4ebf7c253fff77900cac8b6ad478c8f4808b707062dd8c6b40853a6f%40%3Cdev.sedona.apache.org%3E
> .
> I can focus on point Implement loading geospatial data sources (geojson,
> osm) and postgis direct support.
> Regards,
> Paweł
>
> czw., 18 lut 2021 o 06:38 Jia Yu <ji...@gmail.com> napisał(a):
>
> > H Netanel,
> >
> > Thanks for asking. I think there are a couple of issues to be improved,
> as
> > listed here:
> >
> https://lists.apache.org/thread.html/r4ebf7c253fff77900cac8b6ad478c8f4808b707062dd8c6b40853a6f%40%3Cdev.sedona.apache.org%3E
> >
> > On the list given by Pawel and my thoughts, I think the most important
> > thing to be fixed is that (sorted by importance):
> >
> > 1. Replace the object serializer in RDD API with the WKB serializer you
> > wrote before, currently only DataFrame API uses WKB based serializer. In
> > Sedona-core, there is a small benchmark that can test the performance of
> > the WKB serializer.
> > 2. Allow broadcast join for big+small data frames, or at least give users
> > a way to force the broadcast join. Currently, once the optimized Sedona
> > join is enabled, the broadcast join is automatically disabled.
> > 3. Allow null spatial objects in SpatialRDD and DataFrame. Or at least
> > give users a meaningful exception:
> > https://issues.apache.org/jira/browse/SEDONA-4
> >
> > Pawel was working on some improvements for GeoJSON loading. So we can
> > leave it for him.
> >
> > Do you have any other issues you want to work on?
> >
> > Thanks,
> > Jia
> >
> > On Wed, Feb 17, 2021 at 5:35 AM Netanel Malka <ma...@apache.org> wrote:
> >
> >> Hi Jia,
> >> Are there any features that we want to develop? or fix some bugs?
> >>
> >> My free time now allows me to make some contributions to the project.
> >>
> >> Thanks,
> >>
> >>
>
--
Best regards,
Netanel Malka.
Re: Develop new features/fix bugs
Posted by Paweł Kociński <pa...@gmail.com>.
Hi Jia,
Do we have some estimates about next release ? I think it is good idea to
create JIRA tasks from that link
https://lists.apache.org/thread.html/r4ebf7c253fff77900cac8b6ad478c8f4808b707062dd8c6b40853a6f%40%3Cdev.sedona.apache.org%3E.
I can focus on point Implement loading geospatial data sources (geojson,
osm) and postgis direct support.
Regards,
Paweł
czw., 18 lut 2021 o 06:38 Jia Yu <ji...@gmail.com> napisał(a):
> H Netanel,
>
> Thanks for asking. I think there are a couple of issues to be improved, as
> listed here:
> https://lists.apache.org/thread.html/r4ebf7c253fff77900cac8b6ad478c8f4808b707062dd8c6b40853a6f%40%3Cdev.sedona.apache.org%3E
>
> On the list given by Pawel and my thoughts, I think the most important
> thing to be fixed is that (sorted by importance):
>
> 1. Replace the object serializer in RDD API with the WKB serializer you
> wrote before, currently only DataFrame API uses WKB based serializer. In
> Sedona-core, there is a small benchmark that can test the performance of
> the WKB serializer.
> 2. Allow broadcast join for big+small data frames, or at least give users
> a way to force the broadcast join. Currently, once the optimized Sedona
> join is enabled, the broadcast join is automatically disabled.
> 3. Allow null spatial objects in SpatialRDD and DataFrame. Or at least
> give users a meaningful exception:
> https://issues.apache.org/jira/browse/SEDONA-4
>
> Pawel was working on some improvements for GeoJSON loading. So we can
> leave it for him.
>
> Do you have any other issues you want to work on?
>
> Thanks,
> Jia
>
> On Wed, Feb 17, 2021 at 5:35 AM Netanel Malka <ma...@apache.org> wrote:
>
>> Hi Jia,
>> Are there any features that we want to develop? or fix some bugs?
>>
>> My free time now allows me to make some contributions to the project.
>>
>> Thanks,
>>
>>
Re: Develop new features/fix bugs
Posted by Jia Yu <ji...@gmail.com>.
H Netanel,
Thanks for asking. I think there are a couple of issues to be improved, as
listed here:
https://lists.apache.org/thread.html/r4ebf7c253fff77900cac8b6ad478c8f4808b707062dd8c6b40853a6f%40%3Cdev.sedona.apache.org%3E
On the list given by Pawel and my thoughts, I think the most important
thing to be fixed is that (sorted by importance):
1. Replace the object serializer in RDD API with the WKB serializer you
wrote before, currently only DataFrame API uses WKB based serializer. In
Sedona-core, there is a small benchmark that can test the performance of
the WKB serializer.
2. Allow broadcast join for big+small data frames, or at least give users a
way to force the broadcast join. Currently, once the optimized Sedona join
is enabled, the broadcast join is automatically disabled.
3. Allow null spatial objects in SpatialRDD and DataFrame. Or at least give
users a meaningful exception: https://issues.apache.org/jira/browse/SEDONA-4
Pawel was working on some improvements for GeoJSON loading. So we can leave
it for him.
Do you have any other issues you want to work on?
Thanks,
Jia
On Wed, Feb 17, 2021 at 5:35 AM Netanel Malka <ma...@apache.org> wrote:
> Hi Jia,
> Are there any features that we want to develop? or fix some bugs?
>
> My free time now allows me to make some contributions to the project.
>
> Thanks,
>
>