You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hawq.apache.org by kdunn926 <gi...@git.apache.org> on 2016/06/09 17:12:19 UTC
[GitHub] incubator-hawq pull request #700: HAWQ-799 - Initial commmit for HAWQ Postgi...
GitHub user kdunn926 opened a pull request:
https://github.com/apache/incubator-hawq/pull/700
HAWQ-799 - Initial commmit for HAWQ Postgis support
I was able to successfully compile and complete the entire OGC test suite against HAWQ master branch. This is the code for that work. Looking for feedback / comments on whether or not this is a good idea.
Thanks,
Kyle
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kdunn926/incubator-hawq master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-hawq/pull/700.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #700
----
commit d785712519121cc2149a9404f472540a645ebe00
Author: Kyle Dunn <kd...@gmail.com>
Date: 2016-06-09T17:01:39Z
Initial commmit for HAWQ Postgis support
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Posted by kdunn926 <gi...@git.apache.org>.
Github user kdunn926 commented on the issue:
https://github.com/apache/incubator-hawq/pull/700
@liming01, agreed, performance will not be optimal without indexes. I believe there is value in this PR regardless, until [HAWQ-303](https://issues.apache.org/jira/browse/HAWQ-303) is implemented.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Posted by liming01 <gi...@git.apache.org>.
Github user liming01 commented on the issue:
https://github.com/apache/incubator-hawq/pull/700
@kdunn926, if spatial index can not support, then the query performance will be very slow. What we can offer will be far away from user's expectation. Need to think a workaround.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Posted by liming01 <gi...@git.apache.org>.
Github user liming01 commented on the issue:
https://github.com/apache/incubator-hawq/pull/700
@kdunn926, one question:
Because hawq doesn't support index at present, how can you create spatial index? Where to store the index data? Did you test it(the statement CREATE INDEX is disabled in hawq currently)?
Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Posted by changleicn <gi...@git.apache.org>.
Github user changleicn commented on the issue:
https://github.com/apache/incubator-hawq/pull/700
Great to have this feature in hawq.
I just checked the license (http://postgis.net/), it is GPLv2 which is not compatible with apache license. This is similar to PL/R. so I think we can do postgis with similar method for PL/R. That is, put the code in another repo (for PL/R, it is currently in pivotal github repo), and add a "make" target to build the package.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #700: HAWQ-799. Initial commmit for HAWQ Postgis...
Posted by kdunn926 <gi...@git.apache.org>.
Github user kdunn926 closed the pull request at:
https://github.com/apache/incubator-hawq/pull/700
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Posted by edespino <gi...@git.apache.org>.
Github user edespino commented on the issue:
https://github.com/apache/incubator-hawq/pull/700
@kdunn926 - As this point since the code is GPLv2, this PR is not compatible with the Apache HAWQ (incubating) code base. It makes sense to close this PR and find an alternate (non-Apache) location for this code base.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Posted by kdunn926 <gi...@git.apache.org>.
Github user kdunn926 commented on the issue:
https://github.com/apache/incubator-hawq/pull/700
Well, @ivannovick - looks like we can either merge this code in the [GPDB PR](https://github.com/greenplum-db/postgis/pull/1) or leave yet another differentiating product feature on the table.
CC: @tpednekar-pivotal
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Posted by changleicn <gi...@git.apache.org>.
Github user changleicn commented on the issue:
https://github.com/apache/incubator-hawq/pull/700
It can be planned after the first release.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Posted by kdunn926 <gi...@git.apache.org>.
Github user kdunn926 commented on the issue:
https://github.com/apache/incubator-hawq/pull/700
Hey @xunzhang -
1. I'm validating the build procedure right now. I'll provide it once I've replicated it again.
2. To install in my HAWQ sandbox, I actually used a modified GPPKG from Greenplum's PostGIS then replaced the installed libraries with the correct ones compiled against HAWQ 2.0 GA release. Per my understanding of how PostGIS works, the parallel logic should be handled transparently as long as the correct libraries are in place on every segment server; similar to PL/Python, R, etc.
3. I targeted contrib as I felt depends/thirdparty implied some HAWQ build dependency on PostGIS, rather than some independent functionality contributed to HAWQ. Ultimately, the core team should decide where it is best place. Maybe @changleicn has some insight here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Posted by vVineet <gi...@git.apache.org>.
Github user vVineet commented on the issue:
https://github.com/apache/incubator-hawq/pull/700
@changleicn - what do you suggest as the next steps on this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Posted by xunzhang <gi...@git.apache.org>.
Github user xunzhang commented on the issue:
https://github.com/apache/incubator-hawq/pull/700
Hi @kdunn926 ,
1. Does this pull request contain the logic of configuration and building `Postgis` when compiling hawq? If so, can you separate this part into another commit for code review convenience?
2. Where's parallel logic of supporting `Postgis` inside HAWQ?
3. I advice you to import `Postgis` inside `depends/thirdparty` folder instead `contrib`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Posted by kdunn926 <gi...@git.apache.org>.
Github user kdunn926 commented on the issue:
https://github.com/apache/incubator-hawq/pull/700
@xunzhang - Below is the build and test process I've used within [this](https://s3.amazonaws.com/hdbonhdp/HDB-Sandbox-2.0-vmware-2016060805245816.zip) sandbox:
Please see [this](https://github.com/greenplum-db/postgis/pull/1) PR for the majority of the HAWQ-specific code changes relative to that PostGIS branch.
```
# PostGIS build dependencies
yum install gdal gdal-devel geos geos-devel proj proj-devel libxml2-devel byacc bison libxslt ImageMagick -y
git clone ...
cd incubator-hawq/contrib/postgis
# Make sure residual objects are gone
for o in o so a la ; do find ./ -type f -name *.${o} | xargs -exec rm ; done ;
make distclean
# Some build-time requirements
export CFLAGS="-I/usr/local/hawq-2.0.0.0/include -I/usr/local/hawq-2.0.0.0/include/postgresql/server"
export CXXFLAGS=${CFLAGS}
export PG_CFLAGS=${CFLAGS}
export PG_CPPFLAGS=${CFLAGS}
export LD_LIBRARY_PATH=/lib64:/usr/lib64:/usr/local/hawq-2.0.0.0/lib
./configure --prefix=/temp --with-pgconfig=/usr/local/hawq/bin/pg_config
# This is a build-time work-around for the stale references to
# /opt/gcc-4.4.2/bin/gcc provided by HAWQ's pg_config output
sudo mkdir -p /opt/gcc-4.4.2/bin
cd /opt/gcc-4.4.2/bin
sudo ln -s /usr/bin/gcc gcc
make -j9
sudo make install
export PGPORT=10432
cd extras/ogc_test_suite
make
file contents of RPM in the PostGIS GPPKG:
/temp
/temp/bin
/temp/bin/pgsql2shp
/temp/bin/shp2pgsql
/temp/include
/temp/include/liblwgeom.h
/temp/lib
/temp/lib/liblwgeom-2.0.3.so
/temp/lib/liblwgeom.a
/temp/lib/liblwgeom.la
/temp/lib/liblwgeom.so
/temp/lib/postgresql
/temp/lib/postgresql/postgis-2.0.so
/temp/share
/temp/share/postgresql
/temp/share/postgresql/contrib
/temp/share/postgresql/contrib/postgis-2.0
/temp/share/postgresql/contrib/postgis-2.0/legacy.sql
/temp/share/postgresql/contrib/postgis-2.0/legacy_gist.sql
/temp/share/postgresql/contrib/postgis-2.0/legacy_minimal.sql
/temp/share/postgresql/contrib/postgis-2.0/postgis.sql
/temp/share/postgresql/contrib/postgis-2.0/postgis_comments.sql
/temp/share/postgresql/contrib/postgis-2.0/postgis_restore.pl
/temp/share/postgresql/contrib/postgis-2.0/postgis_upgrade_20_minor.sql
/temp/share/postgresql/contrib/postgis-2.0/raster_comments.sql
/temp/share/postgresql/contrib/postgis-2.0/spatial_ref_sys.sql
/temp/share/postgresql/contrib/postgis-2.0/topology.sql
/temp/share/postgresql/contrib/postgis-2.0/topology_comments.sql
/temp/share/postgresql/contrib/postgis-2.0/topology_upgrade_20_minor.sql
/temp/share/postgresql/contrib/postgis-2.0/uninstall_legacy.sql
/temp/share/postgresql/contrib/postgis-2.0/uninstall_postgis.sql
/temp/share/postgresql/contrib/postgis-2.0/uninstall_topology.sql
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Posted by kdunn926 <gi...@git.apache.org>.
Github user kdunn926 commented on the issue:
https://github.com/apache/incubator-hawq/pull/700
@liming01 - you are correct, because HAWQ doesn't support indexes, this work has the same limitation. All the spatial operation tests completed successfully, with the exception of spatial indexes and UPDATEs.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Posted by kdunn926 <gi...@git.apache.org>.
Github user kdunn926 commented on the issue:
https://github.com/apache/incubator-hawq/pull/700
That makes sense. I think it would be great to target the same code base as the one GPDB uses (original PR) so both projects benefit from features without having to keep two repos in sync that have significant overlap in code.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---