You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudberry.apache.org by dj...@apache.org on 2026/10/19 21:01:04 UTC
(cloudberry) branch main updated: Remove gpcontrib/pxf_fdw from the core
This is an automated email from the ASF dual-hosted git repository.
tuhaihe pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
The following commit(s) were added to refs/heads/main by this push:
new 064553d4758 Remove gpcontrib/pxf_fdw from the core
064553d4758 is described below
commit 064553d4758bd7d504debc6ccfe46730f039265b
Author: Dianjin Wang <wa...@gmail.com>
AuthorDate: Mon Oct 19 10:20:19 2026 +0800
Remove gpcontrib/pxf_fdw from the core
pxf_fdw is maintained in apache/cloudberry-pxf and should be built
from there. Greenplum already dropped its in-tree copy in Nov 2022
(greenplum-db/gpdb-archive@913de8e), and our own CI has built with
--disable-pxf for over ten months, so the copy here is not built or
tested anywhere and nobody maintains it.
Remove the source along with everything that only existed for it:
- the --enable-pxf/--disable-pxf configure option and the libcurl
check tied to it
- the pxf targets in gpcontrib/Makefile and enable_pxf in
Makefile.global.in
- the pxf flag in the Coverity, SonarQube and devops build scripts
- the pom.xml RAT excludes for gpcontrib/pxf_fdw
- the HAWQ entries in LICENSE and NOTICE; pxf_fdw was the only code
in the tree derived from Apache HAWQ
Users who need PXF should build pxf_fdw from apache/cloudberry-pxf.
Passing --enable-pxf to configure now only prints an unrecognized
option warning.
See: https://lists.apache.org/thread/y128rjbk5xm23xps2tml917x2g9nqsd8
Assisted-by: Claude Code
---
.github/workflows/coverity.yml | 1 -
.github/workflows/sonarqube.yml | 1 -
LICENSE | 15 -
NOTICE | 7 -
configure | 37 +-
configure.ac | 13 +-
.../cloudberry/scripts/configure-cloudberry.sh | 2 -
gpcontrib/Makefile | 15 +-
gpcontrib/gp_exttable_fdw/Makefile | 2 -
gpcontrib/pxf_fdw/.gitignore | 1 -
gpcontrib/pxf_fdw/Makefile | 22 -
gpcontrib/pxf_fdw/README.md | 19 -
.../pxf_fdw/expected/pxf_fdw_foreign_table.out | 328 -----
gpcontrib/pxf_fdw/expected/pxf_fdw_server.out | 244 ---
.../pxf_fdw/expected/pxf_fdw_user_mapping.out | 266 ----
gpcontrib/pxf_fdw/expected/pxf_fdw_wrapper.out | 274 ----
gpcontrib/pxf_fdw/libchurl.c | 1182 ---------------
gpcontrib/pxf_fdw/libchurl.h | 150 --
gpcontrib/pxf_fdw/pxf_bridge.c | 242 ---
gpcontrib/pxf_fdw/pxf_bridge.h | 85 --
gpcontrib/pxf_fdw/pxf_deparse.c | 77 -
gpcontrib/pxf_fdw/pxf_fdw--1.0.sql | 59 -
gpcontrib/pxf_fdw/pxf_fdw.c | 935 ------------
gpcontrib/pxf_fdw/pxf_fdw.control | 5 -
gpcontrib/pxf_fdw/pxf_fdw.h | 65 -
gpcontrib/pxf_fdw/pxf_filter.c | 1555 --------------------
gpcontrib/pxf_fdw/pxf_filter.h | 151 --
gpcontrib/pxf_fdw/pxf_fragment.c | 589 --------
gpcontrib/pxf_fdw/pxf_fragment.h | 108 --
gpcontrib/pxf_fdw/pxf_header.c | 352 -----
gpcontrib/pxf_fdw/pxf_header.h | 43 -
gpcontrib/pxf_fdw/pxf_option.c | 551 -------
gpcontrib/pxf_fdw/sql/pxf_fdw_foreign_table.sql | 356 -----
gpcontrib/pxf_fdw/sql/pxf_fdw_server.sql | 230 ---
gpcontrib/pxf_fdw/sql/pxf_fdw_user_mapping.sql | 248 ----
gpcontrib/pxf_fdw/sql/pxf_fdw_wrapper.sql | 257 ----
pom.xml | 9 -
src/Makefile.global.in | 1 -
38 files changed, 4 insertions(+), 8493 deletions(-)
diff --git a/.github/workflows/coverity.yml b/.github/workflows/coverity.yml
index c6535407484..146fc47c397 100644
--- a/.github/workflows/coverity.yml
+++ b/.github/workflows/coverity.yml
@@ -99,7 +99,6 @@ jobs:
--enable-orafce \
--enable-orca \
--enable-pax \
- --enable-pxf \
--enable-tap-tests \
--with-gssapi \
--with-ldap \
diff --git a/.github/workflows/sonarqube.yml b/.github/workflows/sonarqube.yml
index a4229c483b5..9fe3c2436c4 100644
--- a/.github/workflows/sonarqube.yml
+++ b/.github/workflows/sonarqube.yml
@@ -102,7 +102,6 @@ jobs:
--enable-orafce \
--enable-orca \
--enable-pax \
- --enable-pxf \
--enable-tap-tests \
--with-gssapi \
--with-ldap \
diff --git a/LICENSE b/LICENSE
index c8d476eb2b1..5fe72074a81 100644
--- a/LICENSE
+++ b/LICENSE
@@ -509,21 +509,6 @@ The Greenplum Database software includes:
----------------------------
Apache License - Version 2.0
- PXF foreign data wrapper client code derived from Apache HAWQ,
- copyright The Apache Software Foundation:
- gpcontrib/pxf_fdw/libchurl.c
- gpcontrib/pxf_fdw/libchurl.h
- gpcontrib/pxf_fdw/pxf_filter.c
- gpcontrib/pxf_fdw/pxf_filter.h
- gpcontrib/pxf_fdw/pxf_fragment.h
- gpcontrib/pxf_fdw/pxf_header.c
- (see the Apache License above, and NOTICE)
- Greenplum Database took this code from HAWQ's PXF external table
- client (src/backend/access/external and src/include/access in
- Apache HAWQ) and later reworked it into a foreign data wrapper.
- pxf_bridge.c, pxf_bridge.h and pxf_header.h in the same directory
- carry the same Apache License header but share no code with HAWQ.
-
cmockery unit testing framework, copyright 2008 Google Inc.:
src/test/unit/cmockery
(see the Apache License above; also shipped in-tree as
diff --git a/NOTICE b/NOTICE
index 475ffb11541..0226ded13da 100644
--- a/NOTICE
+++ b/NOTICE
@@ -10,10 +10,3 @@ This product includes software originally developed by VMware.
Greenplum Database
Copyright (c) 2004-2020 VMware, Inc. or its affiliates. All Rights Reserved.
-
----------------------------------------------------------------------------
-
-This product includes software originally developed by Apache HAWQ.
-
-Apache HAWQ
-Copyright 2017 The Apache Software Foundation
diff --git a/configure b/configure
index d6ae6a0ab4a..1cb53e09cdd 100755
--- a/configure
+++ b/configure
@@ -820,7 +820,6 @@ enable_debug
enable_orafce
enable_faultinjector
enable_debug_extensions
-enable_pxf
enable_gpfdist
enable_strong_random
enable_rpath
@@ -890,7 +889,6 @@ enable_spinlocks
enable_atomics
enable_strong_random
enable_gpfdist
-enable_pxf
enable_debug_extensions
enable_faultinjector
enable_orafce
@@ -1617,7 +1615,6 @@ Optional Features:
--disable-atomics do not use atomic operations
--disable-strong-random do not use a strong random number source
--disable-gpfdist do not use gpfdist
- --disable-pxf do not build pxf
--disable-debug-extensions
exclude debug extensions in gpcontrib
--disable-faultinjector disable fault injector
@@ -3575,34 +3572,6 @@ fi
-#
-# pxf
-#
-
-
-# Check whether --enable-pxf was given.
-if test "${enable_pxf+set}" = set; then :
- enableval=$enable_pxf;
- case $enableval in
- yes)
- :
- ;;
- no)
- :
- ;;
- *)
- as_fn_error $? "no argument expected for --enable-pxf option" "$LINENO" 5
- ;;
- esac
-
-else
- enable_pxf=yes
-
-fi
-
-
-
-
#
# include debug extensions in gpcontrib
#
@@ -11868,7 +11837,7 @@ fi
#
-# libcurl. Used for external table support and the PXF extension
+# libcurl. Used for external table support and gpcloud
#
@@ -11896,10 +11865,6 @@ fi
-if test "$with_libcurl" = "no" && test "$enable_pxf" = "yes"; then
- as_fn_error $? "libcurl is required by PXF" "$LINENO" 5
-fi
-
if test "$with_libcurl" = "no" && test "$enable_gpcloud" = "yes"; then
as_fn_error $? "libcurl is required by gpcloud" "$LINENO" 5
fi
diff --git a/configure.ac b/configure.ac
index 264f59e4b51..207f350f7df 100644
--- a/configure.ac
+++ b/configure.ac
@@ -227,13 +227,6 @@ PGAC_ARG_BOOL(enable, gpfdist, yes,
[do not use gpfdist])
AC_SUBST(enable_gpfdist)
-#
-# pxf
-#
-PGAC_ARG_BOOL(enable, pxf, yes,
- [do not build pxf])
-AC_SUBST(enable_pxf)
-
#
# include debug extensions in gpcontrib
#
@@ -1515,16 +1508,12 @@ PGAC_ARG_BOOL(with, rt, yes,
AC_SUBST(with_rt)
#
-# libcurl. Used for external table support and the PXF extension
+# libcurl. Used for external table support and gpcloud
#
PGAC_ARG_BOOL(with, libcurl, yes,
[do not use libcurl])
AC_SUBST(with_libcurl)
-if test "$with_libcurl" = "no" && test "$enable_pxf" = "yes"; then
- AC_MSG_ERROR([libcurl is required by PXF])
-fi
-
if test "$with_libcurl" = "no" && test "$enable_gpcloud" = "yes"; then
AC_MSG_ERROR([libcurl is required by gpcloud])
fi
diff --git a/devops/build/automation/cloudberry/scripts/configure-cloudberry.sh b/devops/build/automation/cloudberry/scripts/configure-cloudberry.sh
index eda9586c28f..3efcb6443d0 100755
--- a/devops/build/automation/cloudberry/scripts/configure-cloudberry.sh
+++ b/devops/build/automation/cloudberry/scripts/configure-cloudberry.sh
@@ -34,7 +34,6 @@
# - Oracle Compatibility (orafce)
# - ORCA Query Optimizer
# - PAX Access Method
-# - PXF External Table Access
# - Test Automation Support (tap-tests)
#
# System Integration:
@@ -159,7 +158,6 @@ execute_cmd ./configure --prefix=${BUILD_DESTINATION} \
--enable-orafce \
--enable-orca \
--enable-pax \
- --disable-pxf \
--enable-tap-tests \
${CONFIGURE_DEBUG_OPTS} \
--with-diskquota \
diff --git a/gpcontrib/Makefile b/gpcontrib/Makefile
index a03c881a92f..d37015b2914 100755
--- a/gpcontrib/Makefile
+++ b/gpcontrib/Makefile
@@ -5,7 +5,7 @@ default: all
top_builddir = ..
include $(top_builddir)/src/Makefile.global
-.PHONY: all install clean distclean gpcloud pxf mapreduce
+.PHONY: all install clean distclean gpcloud mapreduce
#
# targets built via top_builddir/GNUmakefile
@@ -53,7 +53,7 @@ ifeq "$(with_gp_stats_collector)" "yes"
endif
$(call recurse,all install clean distclean, $(recurse_targets))
-all: gpcloud pxf mapreduce orafce
+all: gpcloud mapreduce orafce
gpcloud:
@if [ "$(enable_gpcloud)" = "yes" ]; then \
@@ -61,11 +61,6 @@ gpcloud:
$(MAKE) -C gpcloud/bin/gpcheckcloud; \
fi
-pxf:
- @if [ "$(enable_pxf)" = "yes" ]; then \
- $(MAKE) -C pxf_fdw; \
- fi
-
mapreduce:
@if [ "$(enable_mapreduce)" = "yes" ]; then \
$(MAKE) -C gpmapreduce; \
@@ -84,9 +79,6 @@ install:
@if [ "$(enable_mapreduce)" = "yes" ]; then \
$(MAKE) -C gpmapreduce install; \
fi
- @if [ "$(enable_pxf)" = "yes" ]; then \
- $(MAKE) -C pxf_fdw install; \
- fi
@if [ "$(enable_orafce)" = "yes" ]; then \
$(MAKE) -C orafce NO_PGXS=true install; \
fi
@@ -94,20 +86,17 @@ install:
clean:
if [ "$(enable_mapreduce)" = "yes" ]; then $(MAKE) -C gpmapreduce clean; fi
if [ "$(enable_gpcloud)" = "yes" ]; then $(MAKE) -C gpcloud cleanall; fi
- if [ "$(enable_pxf)" = "yes" ]; then $(MAKE) -C pxf_fdw clean; fi
if [ "${enable_orafce}" = "yes" ]; then $(MAKE) -C orafce NO_PGXS=true clean; fi
distclean:
if [ "$(enable_mapreduce)" = "yes" ]; then $(MAKE) -C gpmapreduce distclean; fi
if [ "$(enable_gpcloud)" = "yes" ]; then $(MAKE) -C gpcloud distclean; fi
- if [ "$(enable_pxf)" = "yes" ]; then $(MAKE) -C pxf_fdw distclean; fi
if [ "${enable_orafce}" = "yes" ]; then $(MAKE) -C orafce NO_PGXS=true distclean; fi
installcheck:
$(MAKE) -C gp_internal_tools installcheck
if [ "$(enable_orafce)" = "yes" ]; then $(MAKE) -C orafce installcheck; fi
- if [ "$(enable_pxf)" = "yes" ]; then $(MAKE) -C pxf_fdw installcheck; fi
if [ "$(with_zstd)" = "yes" ]; then $(MAKE) -C zstd installcheck; fi
$(MAKE) -C gp_sparse_vector installcheck
$(MAKE) -C gp_toolkit installcheck
diff --git a/gpcontrib/gp_exttable_fdw/Makefile b/gpcontrib/gp_exttable_fdw/Makefile
index a2b5cf9fb4f..c6d2ba125a3 100644
--- a/gpcontrib/gp_exttable_fdw/Makefile
+++ b/gpcontrib/gp_exttable_fdw/Makefile
@@ -1,5 +1,3 @@
-# contrib/pxf_fdw/Makefile
-
MODULE_big = gp_exttable_fdw
OBJS = gp_exttable_fdw.o extaccess.o option.o
diff --git a/gpcontrib/pxf_fdw/.gitignore b/gpcontrib/pxf_fdw/.gitignore
deleted file mode 100644
index 872aa273a4e..00000000000
--- a/gpcontrib/pxf_fdw/.gitignore
+++ /dev/null
@@ -1 +0,0 @@
-results
\ No newline at end of file
diff --git a/gpcontrib/pxf_fdw/Makefile b/gpcontrib/pxf_fdw/Makefile
deleted file mode 100644
index afde4727f4f..00000000000
--- a/gpcontrib/pxf_fdw/Makefile
+++ /dev/null
@@ -1,22 +0,0 @@
-# contrib/pxf_fdw/Makefile
-
-MODULE_big = pxf_fdw
-OBJS = pxf_fdw.o pxf_bridge.o pxf_deparse.o pxf_filter.o pxf_fragment.o pxf_header.o pxf_option.o libchurl.o
-
-SHLIB_LINK += -lcurl
-
-EXTENSION = pxf_fdw
-DATA = pxf_fdw--1.0.sql
-
-REGRESS = pxf_fdw_wrapper pxf_fdw_server pxf_fdw_user_mapping pxf_fdw_foreign_table
-
-ifdef USE_PGXS
-PG_CONFIG = pg_config
-PGXS := $(shell $(PG_CONFIG) --pgxs)
-include $(PGXS)
-else
-subdir = gpcontrib/pxf_fdw
-top_builddir = ../..
-include $(top_builddir)/src/Makefile.global
-include $(top_srcdir)/contrib/contrib-global.mk
-endif
diff --git a/gpcontrib/pxf_fdw/README.md b/gpcontrib/pxf_fdw/README.md
deleted file mode 100644
index ba6f8b89aed..00000000000
--- a/gpcontrib/pxf_fdw/README.md
+++ /dev/null
@@ -1,19 +0,0 @@
-# PXF Foreign Data Wrapper for Cloudberry and PostgreSQL
-
-This Cloudberry and PostgreSQL extension implements a Foreign Data Wrapper (FDW)
-for [PXF](https://github.com/greenplum-db/pxf).
-
-PXF is a query federation engine that accesses data residing in external systems
-such as Hadoop, Hive, HBase, relational databases, S3, Google Cloud Storage,
-among other external systems.
-
-### Development
-
-## Compile
-make
-
-## Install
-make install
-
-## Test
-make installcheck
\ No newline at end of file
diff --git a/gpcontrib/pxf_fdw/expected/pxf_fdw_foreign_table.out b/gpcontrib/pxf_fdw/expected/pxf_fdw_foreign_table.out
deleted file mode 100644
index fb7d7cacbd1..00000000000
--- a/gpcontrib/pxf_fdw/expected/pxf_fdw_foreign_table.out
+++ /dev/null
@@ -1,328 +0,0 @@
--- start_matchsubs
--- m/ERROR: COPY quote available only in CSV mode/
--- s/ERROR: COPY quote available only in CSV mode/ERROR: quote available only in CSV mode/
--- end_matchsubs
--- ===================================================================
--- Validation for TABLE options
--- ===================================================================
---
--- Table creation fails if protocol option is provided
---
-CREATE FOREIGN TABLE pxf_fdw_test_table (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( protocol 'pxf_fdw_test2' );
-ERROR: the protocol option can only be defined at the pg_foreign_data_wrapper level
---
--- Table creation fails if resource is not provided
---
-CREATE FOREIGN TABLE pxf_fdw_test_table (id int, name text)
- SERVER pxf_fdw_test_server;
-ERROR: the resource option must be defined at the foreign table level
---
--- Table creation fails if resource is provided as an empty string
---
-CREATE FOREIGN TABLE pxf_fdw_test_table (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '' );
-ERROR: the resource option must be defined at the foreign table level
---
--- Table creation fails if config option is provided
---
-CREATE FOREIGN TABLE pxf_fdw_test_table ()
- SERVER pxf_fdw_test_server
- OPTIONS ( config '/foo/bar' );
-ERROR: the config option can only be defined at the pg_foreign_server level
---
--- Table creation succeeds if delimiter is provided with a single one-byte char
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_delim (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', delimiter '|' );
---
--- Table creation succeeds if delimiter is provided with a single one-byte char
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_quote (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', format 'csv', quote '`' );
---
--- Table creation succeeds if escape is provided with a single one-byte char
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_escape (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', format 'csv', escape '\' );
---
--- Table creation succeeds if null is provided with a single one-byte char
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_null (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', null '' );
---
--- Table creation succeeds if encoding is provided with a single one-byte char
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_encoding (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', encoding 'UTF-8' );
---
--- Table creation succeeds if newline is valid
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_newline (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', newline 'CRLF' );
---
--- Table creation succeeds if resource and fill_missing_fields are provided
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_fill_missing_fields (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', fill_missing_fields 'true');
---
--- Table creation succeeds if force_null is provided on column definition
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_force_null (
- id int OPTIONS (force_null 'true'),
- name text OPTIONS (force_null 'false')
- ) SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource' );
---
--- Table creation succeeds if force_not_null is provided on column definition
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_force_not_null (
- id int OPTIONS (force_not_null 'true'),
- name text OPTIONS (force_not_null 'false')
- ) SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource' );
---
--- Table creation succeeds if resource is provided and header is valid
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_header (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', header 'TRUE' );
---
--- Table creation succeeds if resource is provided and protocol is not provided
---
-CREATE FOREIGN TABLE pxf_fdw_test_table (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', extra 'extra_value' );
---
--- Table creation fails if reject_limit is < 1
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( reject_limit '-4' );
-ERROR: invalid reject_limit value '-4', should be a positive integer
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( reject_limit '0' );
-ERROR: invalid reject_limit value '0', should be a positive integer
---
--- Table creation fails if reject_limit_type is not 'percent' or 'rows'
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit_type (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( reject_limit_type 'not-percent-or-rows' );
-ERROR: invalid reject_limit_type value, only 'rows' and 'percent' are supported
---
--- Table creation fails if reject_limit is non-numeric
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit 'no-number-here' );
-ERROR: invalid reject_limit value 'no-number-here', should be a positive integer
---
--- Table creation fails if reject_limit_type is 'rows' and reject_limit < 2
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit_type (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit '1', reject_limit_type 'rows' );
-ERROR: invalid (ROWS) reject_limit value '1', valid values are 2 or larger
---
--- Table creation fails if reject_limit_type is 'percent' and not between 1-100
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit_type (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit '-1', reject_limit_type 'percent' );
-ERROR: invalid reject_limit value '-1', should be a positive integer
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit_type (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit '101', reject_limit_type 'percent' );
-ERROR: invalid (PERCENT) reject_limit value '101', valid values are 1 to 100
---
--- Table creation fails if only resource and log errors are set
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_log_errors (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', log_errors 'true' );
-ERROR: the log_errors option cannot be set without reject_limit
---
--- Table creation fails if only log_errors is non-boolean
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_log_errors (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit '4', log_errors 'yes' );
-ERROR: log_errors requires a Boolean value
---
--- Table creation fails if quote is provided (CSV-only option)
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_csv_only ()
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/foo', quote '9' );
-ERROR: quote available only in CSV mode
---
--- Table creation succeeds if resource is provided and reject_limit is provided correctly
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit '4' );
---
--- Table creation succeeds if resource is provided and reject_limit_type is provided correctly
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit_type (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit_type 'rows', reject_limit '4' );
---
--- Table create succeeds if resource, reject_limit, and log_errors are provided correctly
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_log_errors (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit '4', log_errors 'true' );
---
--- Table alteration fails if protocol option is added
---
-ALTER FOREIGN TABLE pxf_fdw_test_table
- OPTIONS ( ADD protocol 'table_protocol' );
-ERROR: the protocol option can only be defined at the pg_foreign_data_wrapper level
---
--- Table alteration fails if resource option is dropped
---
-ALTER FOREIGN TABLE pxf_fdw_test_table
- OPTIONS ( DROP resource );
-ERROR: the resource option must be defined at the foreign table level
---
--- Table alteration fails if resource is provided as an empty string
---
-ALTER FOREIGN TABLE pxf_fdw_test_table
- OPTIONS ( SET resource '' );
-ERROR: the resource option must be defined at the foreign table level
---
--- Table alteration succeeds if resource option is set
---
-ALTER FOREIGN TABLE pxf_fdw_test_table
- OPTIONS ( SET resource '/new/path/to/resource' );
---
--- Table alteration succeeds if header is provided correctly
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_header
- OPTIONS ( SET header 'FALSE' );
---
--- Table alteration succeeds if delimiter is a single one-byte char
---
-ALTER FOREIGN TABLE pxf_fdw_test_table
- OPTIONS ( ADD delimiter ',' );
---
--- Table alteration succeeds if quote is a single one-byte char
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_quote
- OPTIONS ( SET quote '"' );
---
--- Table alteration succeeds if escape is a single one-byte char
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_escape
- OPTIONS ( SET escape '+' );
---
--- Table alteration succeeds if null is a single one-byte char
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_null
- OPTIONS ( SET null 'null' );
---
--- Table alteration succeeds if encoding is a single one-byte char
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_encoding
- OPTIONS ( SET encoding 'ISO-8859-1' );
---
--- Table alteration succeeds if newline is valid
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_newline
- OPTIONS ( SET newline 'CRLF' );
---
--- Table alteration succeeds when fill_missing_fields is provided
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_fill_missing_fields
- OPTIONS ( SET fill_missing_fields 'true' );
---
--- Table alteration succeeds when force_null is provided on column definition
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_force_null
- ALTER COLUMN id OPTIONS (DROP force_null);
---
--- Table alteration succeeds when force_not_null is provided on column definition
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_force_not_null
- ALTER COLUMN id OPTIONS (DROP force_not_null);
---
--- Table alteration fails if reject_limit is less than 1
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit
- OPTIONS ( SET reject_limit '-4' );
-ERROR: invalid reject_limit value '-4', should be a positive integer
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit
- OPTIONS ( SET reject_limit '0' );
-ERROR: invalid reject_limit value '0', should be a positive integer
---
--- Table alteration fails if reject_limit_type is not 'percent' or 'rows'
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit_type
- OPTIONS ( SET reject_limit_type 'not-percent-or-rows' );
-ERROR: invalid reject_limit_type value, only 'rows' and 'percent' are supported
---
--- Table alteration fails if reject_limit is non-numeric
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit
- OPTIONS ( SET reject_limit 'no-number-here' );
-ERROR: invalid reject_limit value 'no-number-here', should be a positive integer
---
--- Table alteration fails if reject_limit_type is rows and reject_limit is < 2
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit_type
- OPTIONS ( SET reject_limit '1', SET reject_limit_type 'rows' );
-ERROR: invalid (ROWS) reject_limit value '1', valid values are 2 or larger
---
--- Table alteration fails if reject_limit_type is percent and reject_limit is not within 1-100
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit_type
- OPTIONS ( SET reject_limit '-1', SET reject_limit_type 'percent' );
-ERROR: invalid reject_limit value '-1', should be a positive integer
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit_type
- OPTIONS ( SET reject_limit '101', SET reject_limit_type 'percent' );
-ERROR: invalid (PERCENT) reject_limit value '101', valid values are 1 to 100
---
--- Table alteration succeeds if resource is provided and reject_limit is provided correctly
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit
- OPTIONS ( SET reject_limit '5' );
---
--- Table alteration succeeds if resource is provided and reject_limit_type is provided correctly
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit_type
- OPTIONS ( SET reject_limit_type 'percent' );
---
--- Table alteration succeeds for log_errors if table has valid resource and reject_limit
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_log_errors
- OPTIONS ( SET log_errors 'false' );
---
--- Table alteration fails for log_errors if table has valid resource but no reject_limit
---
-ALTER FOREIGN TABLE pxf_fdw_test_table
- OPTIONS ( ADD log_errors 'false' );
-ERROR: the log_errors option cannot be set without reject_limit
---
--- Table alteration fails for log_errors if log_errors is non-boolean
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_log_errors
- OPTIONS ( SET log_errors 'no' );
-ERROR: log_errors requires a Boolean value
---
--- Table alteration fails if config option is provided
---
-ALTER FOREIGN TABLE pxf_fdw_test_table
- OPTIONS ( ADD config '/foo/bar' );
-ERROR: the config option can only be defined at the pg_foreign_server level
diff --git a/gpcontrib/pxf_fdw/expected/pxf_fdw_server.out b/gpcontrib/pxf_fdw/expected/pxf_fdw_server.out
deleted file mode 100644
index 3c0f8e853ad..00000000000
--- a/gpcontrib/pxf_fdw/expected/pxf_fdw_server.out
+++ /dev/null
@@ -1,244 +0,0 @@
--- ===================================================================
--- Validation for SERVER options
--- ===================================================================
---
--- Server creation fails if protocol option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( protocol 'pxf_fdw_test2' );
-ERROR: the protocol option can only be defined at the pg_foreign_data_wrapper level
---
--- Server creation fails if resource option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( resource '/invalid/option/for/server' );
-ERROR: the resource option can only be defined at the pg_foreign_table level
---
--- Server creation fails if delimiter option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( delimiter ' ' );
-ERROR: invalid option "delimiter"
-HINT: There are no valid options in this context.
---
--- Server creation fails if header option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( header 'TRUE' );
-ERROR: invalid option "header"
-HINT: There are no valid options in this context.
---
--- Server creation fails if quote option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( quote '`' );
-ERROR: invalid option "quote"
-HINT: There are no valid options in this context.
---
--- Server creation fails if escape option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( escape '\' );
-ERROR: invalid option "escape"
-HINT: There are no valid options in this context.
---
--- Server creation fails if null option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( null '' );
-ERROR: invalid option "null"
-HINT: There are no valid options in this context.
---
--- Server creation fails if encoding option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( encoding 'UTF-8' );
-ERROR: invalid option "encoding"
-HINT: There are no valid options in this context.
---
--- Server creation fails if newline option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( newline 'CRLF' );
-ERROR: invalid option "newline"
-HINT: There are no valid options in this context.
---
--- Server creation fails if fill_missing_fields option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( fill_missing_fields '' );
-ERROR: invalid option "fill_missing_fields"
-HINT: There are no valid options in this context.
---
--- Server creation fails if force_null option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( force_null 'true' );
-ERROR: invalid option "force_null"
-HINT: There are no valid options in this context.
---
--- Server creation fails if force_not_null option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( force_not_null 'true' );
-ERROR: invalid option "force_not_null"
-HINT: There are no valid options in this context.
---
--- Server creation fails if reject_limit option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( reject_limit '5' );
-ERROR: the reject_limit option can only be defined at the pg_foreign_table level
---
--- Server creation fails if reject_limit_type option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( reject_limit_type 'rows' );
-ERROR: the reject_limit_type option can only be defined at the pg_foreign_table level
---
--- Server creation fails if log_errors option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( log_errors 'true' );
-ERROR: the log_errors option can only be defined at the pg_foreign_table level
---
--- Server creation succeeds if protocol option is not provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw;
---
--- Server creation succeeds if config option is provided
---
-CREATE SERVER pxf_fdw_test_server_with_config
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( config '/foo/bar' );
---
--- Server alteration fails if protocol option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD protocol 'pxf_fdw_test2' );
-ERROR: the protocol option can only be defined at the pg_foreign_data_wrapper level
---
--- Server alteration fails if resource option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD resource '/invalid/option/for/server' );
-ERROR: the resource option can only be defined at the pg_foreign_table level
---
--- Server alteration fails if header option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD header 'TRUE' );
-ERROR: invalid option "header"
-HINT: There are no valid options in this context.
---
--- Server alteration fails if delimiter option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD delimiter ' ' );
-ERROR: invalid option "delimiter"
-HINT: There are no valid options in this context.
---
--- Server alteration fails if quote option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD quote '`' );
-ERROR: invalid option "quote"
-HINT: There are no valid options in this context.
---
--- Server alteration fails if escape option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD escape '\' );
-ERROR: invalid option "escape"
-HINT: There are no valid options in this context.
---
--- Server alteration fails if null option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD null '' );
-ERROR: invalid option "null"
-HINT: There are no valid options in this context.
---
--- Server alteration fails if encoding option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD encoding 'UTF-8' );
-ERROR: invalid option "encoding"
-HINT: There are no valid options in this context.
---
--- Server alteration fails if newline option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD newline 'CRLF' );
-ERROR: invalid option "newline"
-HINT: There are no valid options in this context.
---
--- Server alteration fails if fill_missing_fields option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD fill_missing_fields '' );
-ERROR: invalid option "fill_missing_fields"
-HINT: There are no valid options in this context.
---
--- Server alteration fails if force_null option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD force_null 'true' );
-ERROR: invalid option "force_null"
-HINT: There are no valid options in this context.
---
--- Server alteration fails if force_not_null option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD force_not_null 'true' );
-ERROR: invalid option "force_not_null"
-HINT: There are no valid options in this context.
---
--- Server alteration fails if reject_limit option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD reject_limit '5' );
-ERROR: the reject_limit option can only be defined at the pg_foreign_table level
---
--- Server alteration fails if reject_limit_type option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD reject_limit_type 'rows' );
-ERROR: the reject_limit_type option can only be defined at the pg_foreign_table level
---
--- Server alteration fails if log_errors option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD log_errors 'true' );
-ERROR: the log_errors option can only be defined at the pg_foreign_table level
---
--- Server alteration succeeds if config option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD config '/foo/bar' );
---
--- Server alteration succeeds if config option is modified
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( SET config '/foo/bar' );
---
--- Server alteration succeeds if config option is dropped
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( DROP config );
diff --git a/gpcontrib/pxf_fdw/expected/pxf_fdw_user_mapping.out b/gpcontrib/pxf_fdw/expected/pxf_fdw_user_mapping.out
deleted file mode 100644
index b6f3a9155a7..00000000000
--- a/gpcontrib/pxf_fdw/expected/pxf_fdw_user_mapping.out
+++ /dev/null
@@ -1,266 +0,0 @@
--- ===================================================================
--- Validation for USER MAPPING options
--- ===================================================================
---
--- User mapping creation fails if protocol option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( protocol 'usermappingprotocol' );
-ERROR: the protocol option can only be defined at the pg_foreign_data_wrapper level
---
--- User mapping creation fails if resource option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/invalid/option/for/user/mapping' );
-ERROR: the resource option can only be defined at the pg_foreign_table level
---
--- User mapping creation fails if header option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( header 'TRUE' );
-ERROR: invalid option "header"
-HINT: There are no valid options in this context.
---
--- User mapping creation fails if delimiter option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( delimiter ' ' );
-ERROR: invalid option "delimiter"
-HINT: There are no valid options in this context.
---
--- User mapping creation fails if quote option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( quote '`' );
-ERROR: invalid option "quote"
-HINT: There are no valid options in this context.
---
--- User mapping creation fails if escape option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( escape '\' );
-ERROR: invalid option "escape"
-HINT: There are no valid options in this context.
---
--- User mapping creation fails if null option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( null '' );
-ERROR: invalid option "null"
-HINT: There are no valid options in this context.
---
--- User mapping creation fails if encoding option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( encoding 'UTF-8' );
-ERROR: invalid option "encoding"
-HINT: There are no valid options in this context.
---
--- User mapping creation fails if newline option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( newline 'CRLF' );
-ERROR: invalid option "newline"
-HINT: There are no valid options in this context.
---
--- User mapping creation fails if fill_missing_fields option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( fill_missing_fields '' );
-ERROR: invalid option "fill_missing_fields"
-HINT: There are no valid options in this context.
---
--- User mapping creation fails if force_null option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( force_null 'true' );
-ERROR: invalid option "force_null"
-HINT: There are no valid options in this context.
---
--- User mapping creation fails if force_not_null option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( force_not_null 'true' );
-ERROR: invalid option "force_not_null"
-HINT: There are no valid options in this context.
---
--- User mapping creation fails if reject_limit option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( reject_limit '5' );
-ERROR: the reject_limit option can only be defined at the pg_foreign_table level
---
--- User mapping creation fails if reject_limit_type option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( reject_limit_type 'rows' );
-ERROR: the reject_limit_type option can only be defined at the pg_foreign_table level
---
--- User mapping creation fails if log_errors option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( log_errors 'true' );
-ERROR: the log_errors option can only be defined at the pg_foreign_table level
---
--- User mapping creation fails if mpp_execute option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( mpp_execute 'any' );
-ERROR: the mpp_execute option cannot be defined at the user mapping level
---
--- User mapping creation fails if config option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( config '/foo/bar' );
-ERROR: the config option can only be defined at the pg_foreign_server level
---
--- User mapping creation succeeds if protocol option is not provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server;
---
--- User mapping alteration fails if protocol option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD protocol 'usermappingprotocol' );
-ERROR: the protocol option can only be defined at the pg_foreign_data_wrapper level
---
--- User mapping alteration fails if resource option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD resource '/invalid/option/for/user/mapping' );
-ERROR: the resource option can only be defined at the pg_foreign_table level
---
--- User mapping alteration fails if header option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD header 'TRUE' );
-ERROR: invalid option "header"
-HINT: There are no valid options in this context.
---
--- User mapping alteration fails if delimiter option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD delimiter ' ' );
-ERROR: invalid option "delimiter"
-HINT: There are no valid options in this context.
---
--- User mapping alteration fails if quote option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD quote '`' );
-ERROR: invalid option "quote"
-HINT: There are no valid options in this context.
---
--- User mapping alteration fails if escape option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD escape '\' );
-ERROR: invalid option "escape"
-HINT: There are no valid options in this context.
---
--- User mapping alteration fails if null option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD null '' );
-ERROR: invalid option "null"
-HINT: There are no valid options in this context.
---
--- User mapping alteration fails if encoding option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD encoding 'UTF-8' );
-ERROR: invalid option "encoding"
-HINT: There are no valid options in this context.
---
--- User mapping alteration fails if newline option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD newline 'CRLF' );
-ERROR: invalid option "newline"
-HINT: There are no valid options in this context.
---
--- User mapping alteration fails if fill_missing_fields option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD fill_missing_fields '' );
-ERROR: invalid option "fill_missing_fields"
-HINT: There are no valid options in this context.
---
--- User mapping alteration fails if force_null option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD force_null 'true' );
-ERROR: invalid option "force_null"
-HINT: There are no valid options in this context.
---
--- User mapping alteration fails if force_not_null option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD force_not_null 'true' );
-ERROR: invalid option "force_not_null"
-HINT: There are no valid options in this context.
---
--- User mapping alteration fails if reject_limit option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD reject_limit '5' );
-ERROR: the reject_limit option can only be defined at the pg_foreign_table level
---
--- User mapping alteration fails if reject_limit_type option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD reject_limit_type 'rows' );
-ERROR: the reject_limit_type option can only be defined at the pg_foreign_table level
---
--- User mapping alteration fails if log_errors option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD log_errors 'true' );
-ERROR: the log_errors option can only be defined at the pg_foreign_table level
---
--- User mapping alteration fails if mpp_execute option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD mpp_execute 'any' );
-ERROR: the mpp_execute option cannot be defined at the user mapping level
---
--- User mapping alteration fails if config option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD config '/foo/bar' );
-ERROR: the config option can only be defined at the pg_foreign_server level
diff --git a/gpcontrib/pxf_fdw/expected/pxf_fdw_wrapper.out b/gpcontrib/pxf_fdw/expected/pxf_fdw_wrapper.out
deleted file mode 100644
index e4f7ad97911..00000000000
--- a/gpcontrib/pxf_fdw/expected/pxf_fdw_wrapper.out
+++ /dev/null
@@ -1,274 +0,0 @@
--- ===================================================================
--- create FDW objects
--- ===================================================================
-CREATE EXTENSION pxf_fdw;
-DROP ROLE IF EXISTS pxf_fdw_user;
-CREATE ROLE pxf_fdw_user;
-NOTICE: resource queue required -- using default resource queue "pg_default"
--- ===================================================================
--- Validation for WRAPPER options
--- ===================================================================
---
--- Foreign-data wrapper creation fails if protocol option is not provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator;
-ERROR: the protocol option must be defined for PXF foreign-data wrappers
---
--- Foreign-data wrapper creation fails if protocol option is empty
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol '' );
-ERROR: the protocol option must be defined for PXF foreign-data wrappers
---
--- Foreign-data wrapper creation fails if resource option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( resource '/invalid/option/for/wrapper' );
-ERROR: the resource option can only be defined at the pg_foreign_table level
---
--- Foreign-data wrapper creation fails if header option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', header 'TRUE' );
-ERROR: invalid option "header"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper creation fails if delimiter option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', delimiter ' ' );
-ERROR: invalid option "delimiter"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper creation fails if quote option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', quote '`' );
-ERROR: invalid option "quote"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper creation fails if escape option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', escape '\' );
-ERROR: invalid option "escape"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper creation fails if null option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', null '' );
-ERROR: invalid option "null"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper creation fails if encoding option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', encoding 'UTF-8' );
-ERROR: invalid option "encoding"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper creation fails if newline option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', newline 'CRLF' );
-ERROR: invalid option "newline"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper creation fails if fill_missing_fields option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', fill_missing_fields '' );
-ERROR: invalid option "fill_missing_fields"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper creation fails if force_null option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', force_null 'true' );
-ERROR: invalid option "force_null"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper creation fails if force_not_null option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', force_not_null 'true' );
-ERROR: invalid option "force_not_null"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper creation fails if reject_limit option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', reject_limit '5' );
-ERROR: the reject_limit option can only be defined at the pg_foreign_table level
---
--- Foreign-data wrapper creation fails if reject_limit_type option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', reject_limit_type 'rows' );
-ERROR: the reject_limit_type option can only be defined at the pg_foreign_table level
---
--- Foreign-data wrapper creation fails if log_errors option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', log_errors 'true' );
-ERROR: the log_errors option can only be defined at the pg_foreign_table level
---
--- Foreign-data wrapper creation fails if config option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', config '/foo/bar' );
-ERROR: the config option can only be defined at the pg_foreign_server level
---
--- Foreign-data wrapper succeeds when protocol is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', mpp_execute 'all segments' );
---
--- Foreign-data wrapper alteration fails when protocol is dropped
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( DROP protocol );
-ERROR: the protocol option must be defined for PXF foreign-data wrappers
---
--- Foreign-data wrapper alteration fails if protocol option is empty
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( SET protocol '' );
-ERROR: the protocol option must be defined for PXF foreign-data wrappers
---
--- Foreign-data wrapper alteration fails if resource option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD resource '/invalid/option/for/wrapper' );
-ERROR: the resource option can only be defined at the pg_foreign_table level
---
--- Foreign-data wrapper alteration fails if header option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD header 'TRUE' );
-ERROR: invalid option "header"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper alteration fails if delimiter option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD delimiter ' ' );
-ERROR: invalid option "delimiter"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper alteration fails if quote option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD quote '`' );
-ERROR: invalid option "quote"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper alteration fails if escape option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD escape '\' );
-ERROR: invalid option "escape"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper alteration fails if null option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD null '' );
-ERROR: invalid option "null"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper alteration fails if encoding option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD encoding 'UTF-8' );
-ERROR: invalid option "encoding"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper alteration fails if newline option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD newline 'CRLF' );
-ERROR: invalid option "newline"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper alteration fails if fill_missing_fields option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD fill_missing_fields '' );
-ERROR: invalid option "fill_missing_fields"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper alteration fails if force_null option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD force_null 'true' );
-ERROR: invalid option "force_null"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper alteration fails if force_not_null option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD force_not_null 'true' );
-ERROR: invalid option "force_not_null"
-HINT: There are no valid options in this context.
---
--- Foreign-data wrapper alteration fails if reject_limit option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD reject_limit '5' );
-ERROR: the reject_limit option can only be defined at the pg_foreign_table level
---
--- Foreign-data wrapper alteration fails if reject_limit_type option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD reject_limit_type 'rows' );
-ERROR: the reject_limit_type option can only be defined at the pg_foreign_table level
---
--- Foreign-data wrapper alteration fails if log_errors option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD log_errors 'true' );
-ERROR: the log_errors option can only be defined at the pg_foreign_table level
---
--- Foreign-data wrapper alteration fails if config option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD config '/foo/bar' );
-ERROR: the config option can only be defined at the pg_foreign_server level
diff --git a/gpcontrib/pxf_fdw/libchurl.c b/gpcontrib/pxf_fdw/libchurl.c
deleted file mode 100644
index 173dcd00d52..00000000000
--- a/gpcontrib/pxf_fdw/libchurl.c
+++ /dev/null
@@ -1,1182 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-#include "postgres.h"
-
-#include "libchurl.h"
-#include "lib/stringinfo.h"
-#include "miscadmin.h"
-#include "utils/guc.h"
-
-/* include libcurl without typecheck.
- * This allows wrapping curl_easy_setopt to be wrapped
- * for readability. O/w an error is generated when anything
- * other than the expected type is given as parameter
- */
-#define CURL_DISABLE_TYPECHECK
-#include <curl/curl.h>
-#undef CURL_DISABLE_TYPECHECK
-
-/*
- * internal buffer for libchurl internal context
- */
-typedef struct churl_buffer
-{
- char *ptr;
- int max;
- int bot,
- top;
-
-} churl_buffer;
-
-/*
- * internal context of libchurl
- */
-typedef struct churl_handle
-{
- /* curl easy API handle */
- CURL *curl_handle;
-
- /*
- * curl multi API handle used to allow non-blocking callbacks
- */
- CURLM *multi_handle;
-
- /*
- * curl API puts internal errors in this buffer used for error reporting
- */
- char curl_error_buffer[CURL_ERROR_SIZE];
-
- /*
- * perform() (libcurl API) lets us know if the session is over using this
- * int
- */
- int curl_still_running;
-
- /* internal buffer for download */
- churl_buffer *download_buffer;
-
- /* internal buffer for upload */
- churl_buffer *upload_buffer;
-
- /*
- * holds http error code returned from remote server
- */
- char *last_http_reponse;
-
- /* true on upload, false on download */
- bool upload;
-} churl_context;
-
-/*
- * holds http header properties
- */
-typedef struct churl_settings
-{
- struct curl_slist *headers;
-} churl_settings;
-
-
-churl_context *churl_new_context(void);
-void create_curl_handle(churl_context *context);
-void set_curl_option(churl_context *context, CURLoption option, const void *data);
-size_t read_callback(void *ptr, size_t size, size_t nmemb, void *userdata);
-void setup_multi_handle(churl_context *context);
-void multi_perform(churl_context *context);
-bool internal_buffer_large_enough(churl_buffer *buffer, size_t required);
-void flush_internal_buffer(churl_context *context);
-char *get_dest_address(CURL * curl_handle);
-void enlarge_internal_buffer(churl_buffer *buffer, size_t required);
-void finish_upload(churl_context *context);
-void cleanup_curl_handle(churl_context *context);
-void multi_remove_handle(churl_context *context);
-void cleanup_internal_buffer(churl_buffer *buffer);
-void churl_cleanup_context(churl_context *context);
-size_t write_callback(char *buffer, size_t size, size_t nitems, void *userp);
-void fill_internal_buffer(churl_context *context, int want);
-void churl_headers_set(churl_context *context, CHURL_HEADERS settings);
-void check_response_status(churl_context *context);
-void check_response_code(churl_context *context);
-void check_response(churl_context *context);
-void clear_error_buffer(churl_context *context);
-size_t header_callback(char *buffer, size_t size, size_t nitems, void *userp);
-void free_http_response(churl_context *context);
-void compact_internal_buffer(churl_buffer *buffer);
-void realloc_internal_buffer(churl_buffer *buffer, size_t required);
-bool handle_special_error(long response, StringInfo err);
-char *get_http_error_msg(long http_ret_code, char *msg, char *curl_error_buffer);
-char *build_header_str(const char *format, const char *key, const char *value);
-
-
-/*
- * Debug function - print the http headers
- */
-void
-print_http_headers(CHURL_HEADERS headers)
-{
- if ((DEBUG2 >= log_min_messages) || (DEBUG2 >= client_min_messages))
- {
- churl_settings *settings = (churl_settings *) headers;
- struct curl_slist *header_cell = settings->headers;
- char *header_data;
- int count = 0;
-
- while (header_cell != NULL)
- {
- header_data = header_cell->data;
- elog(DEBUG2, "churl http header: cell #%d: %s",
- count, header_data ? header_data : "NONE");
- header_cell = header_cell->next;
- ++count;
- }
- }
-}
-
-CHURL_HEADERS
-churl_headers_init(void)
-{
- churl_settings *settings = (churl_settings *) palloc0(sizeof(churl_settings));
-
- return (CHURL_HEADERS) settings;
-}
-
-/*
- * Build a header string, in the form of given format (e.g. "%s: %s"),
- * and populate <key> and <value> in it.
- * If value is empty, return <key>.
- */
-char *
-build_header_str(const char *format, const char *key, const char *value)
-{
- char *header_option = NULL;
-
- if (value == NULL) /* the option is just a "key" */
- header_option = pstrdup(key);
- else /* the option is a "key: value" */
- {
- StringInfoData formatter;
-
- initStringInfo(&formatter);
- appendStringInfo(&formatter, format, key, value);
- header_option = formatter.data;
- }
- return header_option;
-}
-
-void
-churl_headers_append(CHURL_HEADERS headers, const char *key, const char *value)
-{
- churl_settings *settings = (churl_settings *) headers;
- char *header_option = NULL;
-
- header_option = build_header_str("%s: %s", key, value);
-
- settings->headers = curl_slist_append(settings->headers,
- header_option);
- pfree(header_option);
-}
-
-void
-churl_headers_override(CHURL_HEADERS headers, const char *key, const char *value)
-{
-
- churl_settings *settings = (churl_settings *) headers;
- struct curl_slist *header_cell = settings->headers;
- char *key_option = NULL;
- char *header_data = NULL;
-
- /* key must not be empty */
- Assert(key != NULL);
-
- /* key to compare with in the headers */
- key_option = build_header_str("%s:%s", key, value ? "" : NULL);
-
- /* find key in headers list */
- while (header_cell != NULL)
- {
- header_data = header_cell->data;
-
- if (strncmp(key_option, header_data, strlen(key_option)) == 0)
- {
- elog(DEBUG2, "churl_headers_override: Found existing header %s with key %s (for new value %s)",
- header_data, key_option, value);
- break;
- }
- header_cell = header_cell->next;
- }
-
- if (header_cell != NULL) /* found key */
- {
- char *new_data = build_header_str("%s: %s", key, value);
- char *old_data = header_cell->data;
-
- header_cell->data = strdup(new_data);
- elog(DEBUG4, "churl_headers_override: new data: %s, old data: %s", new_data, old_data);
- free(old_data);
- pfree(new_data);
- }
- else
- {
- churl_headers_append(headers, key, value);
- }
-
- pfree(key_option);
-}
-
-void
-churl_headers_remove(CHURL_HEADERS headers, const char *key, bool has_value)
-{
-
- churl_settings *settings = (churl_settings *) headers;
- struct curl_slist *to_del_cell = settings->headers;
- struct curl_slist *prev_cell = NULL;
- char *key_option = NULL;
- char *header_data = NULL;
-
- /* key must not be empty */
- Assert(key != NULL);
-
- /* key to compare with in the headers */
- key_option = build_header_str("%s:%s", key, has_value ? "" : NULL);
-
- /* find key in headers list */
- while (to_del_cell != NULL)
- {
-
- header_data = to_del_cell->data;
-
- if (strncmp(key_option, header_data, strlen(key_option)) == 0)
- {
- elog(DEBUG2, "churl_headers_remove: Found existing header %s with key %s",
- header_data, key_option);
- break;
- }
- prev_cell = to_del_cell;
- to_del_cell = to_del_cell->next;
- }
-
- if (to_del_cell != NULL) /* found key */
- {
- /* skip this cell */
- if (prev_cell != NULL)
- {
- /* not the header */
- prev_cell->next = to_del_cell->next;
- }
- else
- {
- /* remove header - make the next cell header now */
- settings->headers = to_del_cell->next;
- }
-
- /* remove header data and cell */
- if (to_del_cell->data)
- free(to_del_cell->data);
- free(to_del_cell);
- }
- else
- {
- elog(DEBUG2, "churl_headers_remove: No header with key %s to remove",
- key_option);
- }
-
- pfree(key_option);
-}
-
-void
-churl_headers_cleanup(CHURL_HEADERS headers)
-{
- churl_settings *settings = (churl_settings *) headers;
-
- if (!settings)
- return;
-
- if (settings->headers)
- curl_slist_free_all(settings->headers);
-
- pfree(settings);
-}
-
-static CHURL_HANDLE
-churl_init(const char *url, CHURL_HEADERS headers)
-{
- churl_context *context = churl_new_context();
-
- create_curl_handle(context);
- clear_error_buffer(context);
-
-/* Required for resolving localhost on some docker environments that
- * had intermittent networking issues when using pxf on HAWQ
- * However, CURLOPT_RESOLVE is only available in curl versions 7.21 and above */
-#ifdef CURLOPT_RESOLVE
- if (strstr(url, LocalhostIpV4) != NULL)
- {
- struct curl_slist *resolve_hosts = NULL;
- char *pxf_host_entry = (char *) palloc0(strlen(PxfServiceAddress) + strlen(LocalhostIpV4Entry) + 1);
-
- strcat(pxf_host_entry, PxfServiceAddress);
- strcat(pxf_host_entry, LocalhostIpV4Entry);
- resolve_hosts = curl_slist_append(NULL, pxf_host_entry);
- set_curl_option(context, CURLOPT_RESOLVE, resolve_hosts);
- pfree(pxf_host_entry);
- }
-#endif
-
- set_curl_option(context, CURLOPT_URL, url);
- set_curl_option(context, CURLOPT_VERBOSE, (const void *) false);
- set_curl_option(context, CURLOPT_ERRORBUFFER, context->curl_error_buffer);
- set_curl_option(context, CURLOPT_IPRESOLVE, (const void *) CURL_IPRESOLVE_V4);
- set_curl_option(context, CURLOPT_WRITEFUNCTION, write_callback);
- set_curl_option(context, CURLOPT_WRITEDATA, context);
- set_curl_option(context, CURLOPT_HEADERFUNCTION, header_callback);
- set_curl_option(context, CURLOPT_HEADERDATA, context);
- churl_headers_set(context, headers);
-
- return (CHURL_HANDLE) context;
-}
-
-CHURL_HANDLE
-churl_init_upload(const char *url, CHURL_HEADERS headers)
-{
- churl_context *context = churl_init(url, headers);
-
- context->upload = true;
-
- set_curl_option(context, CURLOPT_POST, (const void *) true);
- set_curl_option(context, CURLOPT_READFUNCTION, read_callback);
- set_curl_option(context, CURLOPT_READDATA, context);
- churl_headers_append(headers, "Content-Type", "application/octet-stream");
- churl_headers_append(headers, "Transfer-Encoding", "chunked");
- churl_headers_append(headers, "Expect", "100-continue");
-
- print_http_headers(headers);
- setup_multi_handle(context);
- return (CHURL_HANDLE) context;
-}
-
-CHURL_HANDLE
-churl_init_download(const char *url, CHURL_HEADERS headers)
-{
- churl_context *context = churl_init(url, headers);
-
- context->upload = false;
-
- print_http_headers(headers);
- setup_multi_handle(context);
- return (CHURL_HANDLE) context;
-}
-
-void
-churl_download_restart(CHURL_HANDLE handle, const char *url, CHURL_HEADERS headers)
-{
- churl_context *context = (churl_context *) handle;
-
- Assert(!context->upload);
-
- /* halt current transfer */
- multi_remove_handle(context);
-
- /* set a new url */
- set_curl_option(context, CURLOPT_URL, url);
-
- /* set headers again */
- if (headers)
- churl_headers_set(context, headers);
-
- /* restart */
- setup_multi_handle(context);
-}
-
-/*
- * upload
- */
-size_t
-churl_write(CHURL_HANDLE handle, const char *buf, size_t bufsize)
-{
- churl_context *context = (churl_context *) handle;
- churl_buffer *context_buffer = context->upload_buffer;
-
- Assert(context->upload);
-
- if (!internal_buffer_large_enough(context_buffer, bufsize))
- {
- flush_internal_buffer(context);
- if (!internal_buffer_large_enough(context_buffer, bufsize))
- enlarge_internal_buffer(context_buffer, bufsize);
- }
-
- memcpy(context_buffer->ptr + context_buffer->top, buf, bufsize);
- context_buffer->top += bufsize;
-
- return bufsize;
-}
-
-/*
- * check that connection is ok, read a few bytes and check response.
- */
-void
-churl_read_check_connectivity(CHURL_HANDLE handle)
-{
- churl_context *context = (churl_context *) handle;
-
- Assert(!context->upload);
-
- fill_internal_buffer(context, 1);
- check_response(context);
-}
-
-/*
- * download
- */
-size_t
-churl_read(CHURL_HANDLE handle, char *buf, size_t max_size)
-{
- int n = 0;
- churl_context *context = (churl_context *) handle;
- churl_buffer *context_buffer = context->download_buffer;
-
- Assert(!context->upload);
-
- fill_internal_buffer(context, max_size);
-
- n = context_buffer->top - context_buffer->bot;
-
- /*------
- * TODO: this means we are done. Should we do something with it?
- * if (n == 0 && !context->curl_still_running)
- * context->eof = true;
- *------
- */
-
- if (n > max_size)
- n = max_size;
-
- memcpy(buf, context_buffer->ptr + context_buffer->bot, n);
- context_buffer->bot += n;
-
- return n;
-}
-
-void
-churl_cleanup(CHURL_HANDLE handle, bool after_error)
-{
- churl_context *context = (churl_context *) handle;
-
- if (!context)
- return;
-
- /* don't try to read/write data after an error */
- if (!after_error)
- {
- if (context->upload)
- finish_upload(context);
- else
- churl_read_check_connectivity(handle);
- }
-
- cleanup_curl_handle(context);
- cleanup_internal_buffer(context->download_buffer);
- cleanup_internal_buffer(context->upload_buffer);
- churl_cleanup_context(context);
-}
-
-churl_context *
-churl_new_context()
-{
- churl_context *context = palloc0(sizeof(churl_context));
-
- context->download_buffer = palloc0(sizeof(churl_buffer));
- context->upload_buffer = palloc0(sizeof(churl_buffer));
- return context;
-}
-
-void
-clear_error_buffer(churl_context *context)
-{
- if (!context)
- return;
- context->curl_error_buffer[0] = 0;
-}
-
-void
-create_curl_handle(churl_context *context)
-{
- context->curl_handle = curl_easy_init();
- if (!context->curl_handle)
- elog(ERROR, "internal error: curl_easy_init failed");
-}
-
-void
-set_curl_option(churl_context *context, CURLoption option, const void *data)
-{
- int curl_error;
-
- if (CURLE_OK != (curl_error = curl_easy_setopt(context->curl_handle, option, data)))
- elog(ERROR, "internal error: curl_easy_setopt %d error (%d - %s)",
- option, curl_error, curl_easy_strerror(curl_error));
-}
-
-/*
- * Called by libcurl perform during an upload.
- * Copies data from internal buffer to libcurl's buffer.
- * Once zero is returned, libcurl knows upload is over
- */
-size_t
-read_callback(void *ptr, size_t size, size_t nmemb, void *userdata)
-{
- churl_context *context = (churl_context *) userdata;
- churl_buffer *context_buffer = context->upload_buffer;
-
- int written = Min(size * nmemb, context_buffer->top - context_buffer->bot);
-
- memcpy(ptr, context_buffer->ptr + context_buffer->bot, written);
- context_buffer->bot += written;
-
- return written;
-}
-
-/*
- * Setups the libcurl multi API
- */
-void
-setup_multi_handle(churl_context *context)
-{
- int curl_error;
-
- /* Create multi handle on first use */
- if (!context->multi_handle)
- if (!(context->multi_handle = curl_multi_init()))
- elog(ERROR, "internal error: curl_multi_init failed");
-
- /* add the easy handle to the multi handle */
- /* don't blame me, blame libcurl */
- if (CURLM_OK != (curl_error = curl_multi_add_handle(context->multi_handle, context->curl_handle)))
- if (CURLM_CALL_MULTI_PERFORM != curl_error)
- elog(ERROR, "internal error: curl_multi_add_handle failed (%d - %s)",
- curl_error, curl_easy_strerror(curl_error));
-
- multi_perform(context);
-}
-
-/*
- * Does the real work. Causes libcurl to do
- * as little work as possible and return.
- * During this functions execution,
- * callbacks are called.
- */
-void
-multi_perform(churl_context *context)
-{
- int curl_error;
-
- while (CURLM_CALL_MULTI_PERFORM ==
- (curl_error = curl_multi_perform(context->multi_handle, &context->curl_still_running)));
-
- if (curl_error != CURLM_OK)
- elog(ERROR, "internal error: curl_multi_perform failed (%d - %s)",
- curl_error, curl_easy_strerror(curl_error));
-}
-
-bool
-internal_buffer_large_enough(churl_buffer *buffer, size_t required)
-{
- return ((buffer->top + required) <= buffer->max);
-}
-
-void
-flush_internal_buffer(churl_context *context)
-{
- churl_buffer *context_buffer = context->upload_buffer;
-
- if (context_buffer->top == 0)
- return;
-
- while ((context->curl_still_running != 0) &&
- ((context_buffer->top - context_buffer->bot) > 0))
- {
- /*
- * Allow canceling a query while waiting for input from remote service
- */
- CHECK_FOR_INTERRUPTS();
-
- multi_perform(context);
- }
-
- if ((context->curl_still_running == 0) &&
- ((context_buffer->top - context_buffer->bot) > 0))
- elog(ERROR, "failed sending to remote component %s", get_dest_address(context->curl_handle));
-
- check_response(context);
-
- context_buffer->top = 0;
- context_buffer->bot = 0;
-}
-
-/*
- * Returns the remote ip and port of the curl response.
- * If it's not available, returns an empty string.
- * The returned value should be free'd.
- */
-char *
-get_dest_address(CURL * curl_handle)
-{
- char *dest_url = NULL;
-
- /* add dest url, if any, and curl was nice to tell us */
- if (CURLE_OK == curl_easy_getinfo(curl_handle, CURLINFO_PRIMARY_IP, &dest_url) && dest_url)
- {
- /* TODO: do not hardcode the port here */
- return psprintf("'%s:%d'", dest_url, 5888);
- }
- return dest_url;
-}
-
-void
-enlarge_internal_buffer(churl_buffer *buffer, size_t required)
-{
- if (buffer->ptr != NULL)
- pfree(buffer->ptr);
-
- buffer->max = required + 1024;
- buffer->ptr = palloc(buffer->max);
-}
-
-/*
- * Let libcurl finish the upload by
- * calling perform repeatedly
- */
-void
-finish_upload(churl_context *context)
-{
- if (!context->multi_handle)
- return;
-
- flush_internal_buffer(context);
-
- /*
- * allow read_callback to say 'all done' by returning a zero thus ending
- * the connection
- */
- while (context->curl_still_running != 0)
- multi_perform(context);
-
- check_response(context);
-}
-
-void
-cleanup_curl_handle(churl_context *context)
-{
- if (!context->curl_handle)
- return;
- if (context->multi_handle)
- multi_remove_handle(context);
- curl_easy_cleanup(context->curl_handle);
- context->curl_handle = NULL;
- curl_multi_cleanup(context->multi_handle);
- context->multi_handle = NULL;
-}
-
-void
-multi_remove_handle(churl_context *context)
-{
- int curl_error;
-
- Assert(context->curl_handle && context->multi_handle);
-
- if (CURLM_OK !=
- (curl_error = curl_multi_remove_handle(context->multi_handle, context->curl_handle)))
- elog(ERROR, "internal error: curl_multi_remove_handle failed (%d - %s)",
- curl_error, curl_easy_strerror(curl_error));
-}
-
-void
-cleanup_internal_buffer(churl_buffer *buffer)
-{
- if ((buffer) && (buffer->ptr))
- {
- pfree(buffer->ptr);
- buffer->ptr = NULL;
- buffer->bot = 0;
- buffer->top = 0;
- buffer->max = 0;
- }
-}
-
-void
-churl_cleanup_context(churl_context *context)
-{
- if (context)
- {
- if (context->download_buffer)
- pfree(context->download_buffer);
- if (context->upload_buffer)
- pfree(context->upload_buffer);
-
- pfree(context);
- }
-}
-
-/*
- * Called by libcurl perform during a download.
- * Stores data from libcurl's buffer into the internal buffer.
- * If internal buffer is not large enough, increases it.
- */
-size_t
-write_callback(char *buffer, size_t size, size_t nitems, void *userp)
-{
- churl_context *context = (churl_context *) userp;
- churl_buffer *context_buffer = context->download_buffer;
- const int nbytes = size * nitems;
-
- if (!internal_buffer_large_enough(context_buffer, nbytes))
- {
- compact_internal_buffer(context_buffer);
- if (!internal_buffer_large_enough(context_buffer, nbytes))
- realloc_internal_buffer(context_buffer, nbytes);
- }
-
- /* enough space. copy buffer into curl->buf */
- memcpy(context_buffer->ptr + context_buffer->top, buffer, nbytes);
- context_buffer->top += nbytes;
-
- return nbytes;
-}
-
-/*
- * Fills internal buffer up to want bytes.
- * returns when size reached or transfer ended
- */
-void
-fill_internal_buffer(churl_context *context, int want)
-{
- fd_set fdread;
- fd_set fdwrite;
- fd_set fdexcep;
- int maxfd;
- int curl_error;
-
- /* attempt to fill buffer */
- while (context->curl_still_running &&
- ((context->download_buffer->top - context->download_buffer->bot) < want))
- {
- FD_ZERO(&fdread);
- FD_ZERO(&fdwrite);
- FD_ZERO(&fdexcep);
-
- /* allow canceling a query while waiting for input from remote service */
- CHECK_FOR_INTERRUPTS();
-
- /* set a suitable timeout to fail on */
- long curl_timeo = -1;
- struct timeval timeout;
-
- timeout.tv_sec = 1;
- timeout.tv_usec = 0;
-
- curl_multi_timeout(context->multi_handle, &curl_timeo);
- if (curl_timeo >= 0)
- {
- timeout.tv_sec = curl_timeo / 1000;
- if (timeout.tv_sec > 1)
- timeout.tv_sec = 1;
- else
- timeout.tv_usec = (curl_timeo % 1000) * 1000;
- }
-
- /* get file descriptors from the transfers */
- curl_error = curl_multi_fdset(context->multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);
- if (CURLE_OK != curl_error)
- elog(ERROR, "internal error: curl_multi_fdset failed (%d - %s)",
- curl_error, curl_easy_strerror(curl_error));
-
- /* curl is not ready if maxfd -1 is returned */
- if (maxfd == -1)
- pg_usleep(100);
- else if (-1 == select(maxfd + 1, &fdread, &fdwrite, &fdexcep, &timeout))
- {
- int save_errno = errno;
-
- if (errno == EINTR || errno == EAGAIN)
- continue;
- elog(ERROR, "internal error: select failed on curl_multi_fdset (maxfd %d) (%d - %m)",
- maxfd, save_errno);
- }
- multi_perform(context);
- }
-}
-
-void
-churl_headers_set(churl_context *context, CHURL_HEADERS headers)
-{
- churl_settings *settings = (churl_settings *) headers;
-
- set_curl_option(context, CURLOPT_HTTPHEADER, settings->headers);
-}
-
-/*
- * Checks that the response finished successfully
- * with a valid response status and code.
- */
-void
-check_response(churl_context *context)
-{
- check_response_code(context);
- check_response_status(context);
-}
-
-/*
- * Checks that libcurl transfers completed successfully.
- * This is different than the response code (HTTP code) -
- * a message can have a response code 200 (OK), but end prematurely
- * and so have an error status.
- */
-void
-check_response_status(churl_context *context)
-{
- CURLMsg *msg; /* for picking up messages with the transfer
- * status */
- int msgs_left; /* how many messages are left */
- long status;
-
- while ((msg = curl_multi_info_read(context->multi_handle, &msgs_left)))
- {
- int i = 0;
-
- /* CURLMSG_DONE is the only possible status. */
- if (msg->msg != CURLMSG_DONE)
- continue;
- if (CURLE_OK != (status = msg->data.result))
- {
- char *addr = get_dest_address(msg->easy_handle);
- StringInfoData err;
-
- initStringInfo(&err);
-
- appendStringInfo(&err, "transfer error (%ld): %s",
- status, curl_easy_strerror(status));
-
- if (strlen(addr) != 0)
- appendStringInfo(&err, " from %s", addr);
- pfree(addr);
- elog(ERROR, "%s", err.data);
- }
- elog(DEBUG2, "check_response_status: msg %d done with status OK", i++);
- }
-}
-
-/*
- * Parses return code from libcurl operation and
- * reports if different than 200 and 100
- */
-void
-check_response_code(churl_context *context)
-{
- long response_code;
- char *response_text = NULL;
- int curl_error;
-
- if (CURLE_OK != (curl_error = curl_easy_getinfo(context->curl_handle, CURLINFO_RESPONSE_CODE, &response_code)))
- elog(ERROR, "internal error: curl_easy_getinfo failed(%d - %s)",
- curl_error, curl_easy_strerror(curl_error));
-
- elog(DEBUG2, "http response code: %ld", response_code);
- if ((response_code == 0) && (context->curl_still_running > 0))
- {
- elog(DEBUG2, "check_response_code: curl is still running, but no data was received.");
- }
- else if (response_code != 200 && response_code != 100)
- {
- StringInfoData err;
- char *http_error_msg;
- char *addr;
-
- initStringInfo(&err);
-
- /* prepare response text if any */
- if (context->download_buffer->ptr)
- {
- context->download_buffer->ptr[context->download_buffer->top] = '\0';
- response_text = context->download_buffer->ptr + context->download_buffer->bot;
- }
-
- /* add remote http error code */
- appendStringInfo(&err, "remote component error (%ld)", response_code);
-
- addr = get_dest_address(context->curl_handle);
- if (strlen(addr) != 0)
- {
- appendStringInfo(&err, " from %s", addr);
- }
- pfree(addr);
-
- if (!handle_special_error(response_code, &err))
- {
- /*
- * add detailed error message from the http response.
- * response_text could be NULL in some cases. get_http_error_msg
- * checks for that.
- */
- http_error_msg = get_http_error_msg(response_code, response_text, context->curl_error_buffer);
-
- /*
- * check for a specific confusing error, and replace with a
- * clearer one
- */
- if (strstr(http_error_msg, "instance does not contain any root resource classes") != NULL)
- {
- appendStringInfo(&err, " : PXF not correctly installed in CLASSPATH");
- }
- else
- {
- appendStringInfo(&err, ": %s", http_error_msg);
- }
- }
-
- elog(ERROR, "%s", err.data);
-
- }
-
- free_http_response(context);
-}
-
-/*
- * Extracts the error message from the full HTTP response
- * We test for several conditions in the http_ret_code and the HTTP response message.
- * The first condition that matches, defines the final message string and ends the function.
- * The layout of the HTTP response message is:
-
- <html>
- <head>
- <meta meta_data_attributes />
- <title> title_content_which_has_a_brief_description_of_the_error </title>
- </head>
- <body>
- <h2> heading_containing_the_error_code </h2>
- <p>
- main_body_paragraph_with_a_detailed_description_of_the_error_on_the_rest_server_side
- <pre> the_error_in_original_format_not_HTML_ususally_the_title_of_the_java_exception</pre>
- </p>
- <h3>Caused by:</h3>
- <pre>
- the_full_java_exception_with_the_stack_output
- </pre>
- <hr /><i><small>Powered by Jetty://</small></i>
- <br/>
- <br/>
- </body>
- </html>
-
- * Our first priority is to get the paragraph <p> inside <body>, and in case we don't find it, then we try to get
- * the <title>.
- */
-char *
-get_http_error_msg(long http_ret_code, char *msg, char *curl_error_buffer)
-{
- char *start,
- *end,
- *ret;
- StringInfoData errMsg;
-
- initStringInfo(&errMsg);
-
- /*
- * 1. The server not listening on the port specified in the <create
- * external...> statement" In this case there is no Response from the
- * server, so we issue our own message
- */
-
- if (http_ret_code == 0)
- {
- if (curl_error_buffer == NULL)
- return "There is no pxf servlet listening on the host and port specified in the external table url";
- else
- {
- return curl_error_buffer;
- }
- }
-
- /*
- * 2. There is a response from the server since the http_ret_code is not
- * 0, but there is no response message. This is an abnormal situation that
- * could be the result of a bug, libraries incompatibility or versioning
- * issue in the Rest server or our curl client. In this case we again
- * issue our own message.
- */
- if (!msg || (msg && strlen(msg) == 0))
- {
- appendStringInfo(&errMsg, "HTTP status code is %ld but HTTP response string is empty", http_ret_code);
- ret = pstrdup(errMsg.data);
- pfree(errMsg.data);
- return ret;
- }
-
- /*
- * 3. The "normal" case - There is an HTTP response and the response has a
- * <body> section inside where there is a paragraph contained by the <p>
- * tag.
- */
- start = strstr(msg, "<body>");
- if (start != NULL)
- {
- start = strstr(start, "<p>");
- if (start != NULL)
- {
- char *tmp;
- bool skip = false;
-
- start += 3;
- end = strstr(start, "</p>"); /* assuming where is a <p>, there
- * is a </p> */
- if (end != NULL)
- {
- /* Take one more line after the </p> */
- tmp = strchr(end, '\n');
- if (tmp != NULL)
- end = tmp;
-
- tmp = start;
-
- /*
- * Right now we have the full paragraph inside the <body>. We
- * need to extract from it the <pre> tags, the '\n' and the
- * '\r'.
- */
- while (tmp != end)
- {
- if (*tmp == '>') /* skipping the <pre> tags */
- skip = false;
- else if (*tmp == '<') /* skipping the <pre> tags */
- {
- skip = true;
- appendStringInfoChar(&errMsg, ' ');
- }
- else if (*tmp != '\n' && *tmp != '\r' && skip == false)
- appendStringInfoChar(&errMsg, *tmp);
- tmp++;
- }
-
- ret = pstrdup(errMsg.data);
- pfree(errMsg.data);
- return ret;
- }
- }
- }
-
- /*
- * 4. We did not find the <body>. So we try to print the <title>.
- */
- start = strstr(msg, "<title>");
- if (start != NULL)
- {
- start += 7;
-
- /*
- * no need to check if end is null, if <title> exists then also
- * </title> exists
- */
- end = strstr(start, "</title>");
- if (end != NULL)
- {
- ret = pnstrdup(start, end - start);
- return ret;
- }
- }
-
- /*
- * 5. This is an unexpected situation. We received an error message from
- * the server but it does not have neither a <body> nor a <title>. In this
- * case we return the error message we received as-is.
- */
- return msg;
-}
-
-void
-free_http_response(churl_context *context)
-{
- if (!context->last_http_reponse)
- return;
-
- pfree(context->last_http_reponse);
- context->last_http_reponse = NULL;
-}
-
-/*
- * Called during a perform by libcurl on either download or an upload.
- * Stores the first line of the header for error reporting
- */
-size_t
-header_callback(char *buffer, size_t size, size_t nitems, void *userp)
-{
- const int nbytes = size * nitems;
- churl_context *context = (churl_context *) userp;
-
- if (context->last_http_reponse)
- return nbytes;
-
- char *p = palloc(nbytes + 1);
-
- memcpy(p, buffer, nbytes);
- p[nbytes] = 0;
- context->last_http_reponse = p;
-
- return nbytes;
-}
-
-void
-compact_internal_buffer(churl_buffer *buffer)
-{
- int n;
-
- /* no compaction required */
- if (buffer->bot == 0)
- return;
-
- n = buffer->top - buffer->bot;
- memmove(buffer->ptr, buffer->ptr + buffer->bot, n);
- buffer->bot = 0;
- buffer->top = n;
-}
-
-void
-realloc_internal_buffer(churl_buffer *buffer, size_t required)
-{
- int n;
-
- n = buffer->top - buffer->bot + required + 1024;
- if (buffer->ptr == NULL)
- buffer->ptr = palloc(n);
- else
- /* repalloc does not support NULL ptr */
- buffer->ptr = repalloc(buffer->ptr, n);
-
- buffer->max = n;
-}
-
-bool
-handle_special_error(long response, StringInfo err)
-{
- switch (response)
- {
- case 404:
- appendStringInfo(err, ": PXF service could not be reached. PXF is not running in the tomcat container");
- break;
- default:
- return false;
- }
- return true;
-}
diff --git a/gpcontrib/pxf_fdw/libchurl.h b/gpcontrib/pxf_fdw/libchurl.h
deleted file mode 100644
index d9a006b6332..00000000000
--- a/gpcontrib/pxf_fdw/libchurl.h
+++ /dev/null
@@ -1,150 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-#ifndef _LIBCHURL_H_
-#define _LIBCHURL_H_
-
-#include "postgres.h"
-
-/*
- * CHunked cURL API
- * NOTES:
- * 1) Does not multi thread
- * 2) Does not talk IPv6
- */
-typedef void *CHURL_HEADERS;
-typedef void *CHURL_HANDLE;
-
-/*
- * PUT example
- * -----------
- *
- * CHURL_HEADERS http_headers = churl_headers_init();
- * churl_headers_append(http_headers, "a", "b");
- * churl_headers_append(http_headers, "c", "d");
- *
- * CHURL_HANDLE churl = churl_init_upload("http://127.0.0.1:12345", http_headers);
- * while(have_stuff_to_write())
- * {
- * churl_write(churl);
- * }
- *
- * churl_cleanup(churl);
- * churl_headers_cleanup(http_headers);
- *
- * GET example
- * -----------
- *
- * CHURL_HEADERS http_headers = churl_headers_init();
- * churl_headers_append(http_headers, "a", "b");
- * churl_headers_append(http_headers, "c", "d");
- *
- * CHURL_HANDLE churl = churl_init_download("http://127.0.0.1:12345", http_headers);
- *
- * char buf[64 * 1024];
- * size_t n = 0;
- * while ((n = churl_read(churl, buf, sizeof(buf))) != 0)
- * {
- * do_something(buf, n);
- * }
- *
- * churl_cleanup(churl);
- * churl_headers_cleanup(http_headers);
- */
-
-/*
- * Create a handle for adding headers
- */
-CHURL_HEADERS churl_headers_init(void);
-
-/*
- * Add a new header
- * Headers are added in the form 'key: value'
- */
-void churl_headers_append(CHURL_HEADERS headers, const char *key, const char *value);
-
-/*
- * Override header with given 'key'.
- * If header doesn't exist, create new one (using churl_headers_append).
- * Headers are added in the form 'key: value'
- */
-void churl_headers_override(CHURL_HEADERS headers, const char *key, const char *value);
-
-/*
- * Remove header with given 'key'.
- * has_value specifies if the header has a value or only a key.
- * If the header doesn't exist, do nothing.
- * If the header is the first one on the list,
- * point the headers list to the next element.
- */
-void churl_headers_remove(CHURL_HEADERS headers, const char *key, bool has_value);
-
-/*
- * Cleanup handle for headers
- */
-void churl_headers_cleanup(CHURL_HEADERS headers);
-
-/*
- * Start an upload to url
- * returns a handle to churl transfer
- */
-CHURL_HANDLE churl_init_upload(const char *url, CHURL_HEADERS headers);
-
-/*
- * Start a download to url
- * returns a handle to churl transfer
- */
-CHURL_HANDLE churl_init_download(const char *url, CHURL_HEADERS headers);
-
-/*
- * Restart a session to a new URL
- * This will use the same headers
- */
-void churl_download_restart(CHURL_HANDLE, const char *url, CHURL_HEADERS headers);
-
-/*
- * Send buf of bufsize
- */
-size_t churl_write(CHURL_HANDLE handle, const char *buf, size_t bufsize);
-
-/*
- * Receive up to max_size into buf
- */
-size_t churl_read(CHURL_HANDLE handle, char *buf, size_t max_size);
-
-/*
- * Check connectivity by reading some bytes and checking response
- */
-void churl_read_check_connectivity(CHURL_HANDLE handle);
-
-/*
- * Cleanup churl resources
- */
-void churl_cleanup(CHURL_HANDLE handle, bool after_error);
-
-/*
- * Debug function - print the http headers
- */
-void print_http_headers(CHURL_HEADERS headers);
-
-#define LocalhostIpV4Entry ":127.0.0.1"
-#define LocalhostIpV4 "localhost"
-#define REST_HEADER_JSON_RESPONSE "Accept: application/json"
-
-#endif /* _LIBCHURL_H_ */
diff --git a/gpcontrib/pxf_fdw/pxf_bridge.c b/gpcontrib/pxf_fdw/pxf_bridge.c
deleted file mode 100644
index 196f45fccc2..00000000000
--- a/gpcontrib/pxf_fdw/pxf_bridge.c
+++ /dev/null
@@ -1,242 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- *
- */
-
-#include "pxf_bridge.h"
-#include "cdb/cdbtm.h"
-#include "cdb/cdbvars.h"
-
-/* helper function declarations */
-static void BuildUriForRead(PxfFdwScanState *pxfsstate);
-static void BuildUriForWrite(PxfFdwModifyState *pxfmstate);
-static void SetCurrentFragmentHeaders(PxfFdwScanState *pxfsstate);
-static size_t FillBuffer(PxfFdwScanState *pxfsstate, char *start, int minlen, int maxlen);
-
-/*
- * Clean up churl related data structures from the PXF FDW modify state.
- */
-void
-PxfBridgeCleanup(PxfFdwModifyState *pxfmstate)
-{
- if (pxfmstate == NULL)
- return;
-
- churl_cleanup(pxfmstate->churl_handle, false);
- pxfmstate->churl_handle = NULL;
-
- churl_headers_cleanup(pxfmstate->churl_headers);
- pxfmstate->churl_headers = NULL;
-
- /* TODO: do we need to cleanup filter_str for foreign scan? */
-/* if (pxfmstate->filter_str != NULL)
- {
- pfree(pxfmstate->filter_str);
- pxfmstate->filter_str = NULL;
- }*/
-}
-
-/*
- * Sets up data before starting import
- */
-void
-PxfBridgeImportStart(PxfFdwScanState *pxfsstate)
-{
- if (!pxfsstate->fragments)
- return;
-
- pxfsstate->current_fragment = list_head(pxfsstate->fragments);
- pxfsstate->churl_headers = churl_headers_init();
-
- BuildUriForRead(pxfsstate);
- BuildHttpHeaders(pxfsstate->churl_headers,
- pxfsstate->options,
- pxfsstate->relation,
- pxfsstate->filter_str,
- pxfsstate->retrieved_attrs);
- SetCurrentFragmentHeaders(pxfsstate);
-
- pxfsstate->churl_handle = churl_init_download(pxfsstate->uri.data, pxfsstate->churl_headers);
-
- /* read some bytes to make sure the connection is established */
- churl_read_check_connectivity(pxfsstate->churl_handle);
-}
-
-/*
- * Sets up data before starting export
- */
-void
-PxfBridgeExportStart(PxfFdwModifyState *pxfmstate)
-{
- BuildUriForWrite(pxfmstate);
- pxfmstate->churl_headers = churl_headers_init();
- BuildHttpHeaders(pxfmstate->churl_headers,
- pxfmstate->options,
- pxfmstate->relation,
- NULL,
- NULL);
- pxfmstate->churl_handle = churl_init_upload(pxfmstate->uri.data, pxfmstate->churl_headers);
-}
-
-/*
- * Reads data from the PXF server into the given buffer of a given size
- */
-int
-PxfBridgeRead(void *outbuf, int minlen, int maxlen, void *extra)
-{
- size_t n = 0;
- PxfFdwScanState *pxfsstate = (PxfFdwScanState *) extra;
-
- if (!pxfsstate->fragments)
- return (int) n;
-
- while ((n = FillBuffer(pxfsstate, outbuf, minlen, maxlen)) == 0)
- {
- /*
- * done processing all data for current fragment - check if the
- * connection terminated with an error
- */
- churl_read_check_connectivity(pxfsstate->churl_handle);
-
- /* start processing next fragment */
- pxfsstate->current_fragment = lnext(pxfsstate->fragments, pxfsstate->current_fragment);
- if (pxfsstate->current_fragment == NULL)
- return 0;
-
- SetCurrentFragmentHeaders(pxfsstate);
- churl_download_restart(pxfsstate->churl_handle, pxfsstate->uri.data, pxfsstate->churl_headers);
-
- /* read some bytes to make sure the connection is established */
- churl_read_check_connectivity(pxfsstate->churl_handle);
- }
-
- return (int) n;
-}
-
-/*
- * Writes data from the given buffer of a given size to the PXF server
- */
-int
-PxfBridgeWrite(PxfFdwModifyState *pxfmstate, char *databuf, int datalen)
-{
- size_t n = 0;
-
- if (datalen > 0)
- {
- n = churl_write(pxfmstate->churl_handle, databuf, datalen);
- elog(DEBUG5, "pxf PxfBridgeWrite: segment %d wrote %zu bytes to %s", PXF_SEGMENT_ID, n, pxfmstate->options->resource);
- }
-
- return (int) n;
-}
-
-/*
- * Format the URI for reading by adding PXF service endpoint details
- */
-static void
-BuildUriForRead(PxfFdwScanState *pxfsstate)
-{
- FragmentData *data = (FragmentData *) lfirst(pxfsstate->current_fragment);
-
- resetStringInfo(&pxfsstate->uri);
- appendStringInfo(&pxfsstate->uri, "http://%s/%s/%s/Bridge", data->authority, PXF_SERVICE_PREFIX, PXF_VERSION);
- elog(DEBUG2, "pxf_fdw: uri %s for read", pxfsstate->uri.data);
-}
-
-/*
- * Format the URI for writing by adding PXF service endpoint details
- */
-static void
-BuildUriForWrite(PxfFdwModifyState *pxfmstate)
-{
- PxfOptions *options = pxfmstate->options;
-
- resetStringInfo(&pxfmstate->uri);
- appendStringInfo(&pxfmstate->uri, "http://%s:%d/%s/%s/Writable/stream", options->pxf_host, options->pxf_port, PXF_SERVICE_PREFIX, PXF_VERSION);
- elog(DEBUG2, "pxf_fdw: uri %s with file name for write: %s", pxfmstate->uri.data, options->resource);
-}
-
-/*
- * Change the headers with current fragment information:
- * 1. X-GP-DATA-DIR header is changed to the source name of the current fragment.
- * We reuse the same http header to send all requests for specific fragments.
- * The original header's value contains the name of the general path of the query
- * (can be with wildcard or just a directory name), and this value is changed here
- * to the specific source name of each fragment name.
- * 2. X-GP-FRAGMENT-USER-DATA header is changed to the current fragment's user data.
- * If the fragment doesn't have user data, the header will be removed.
- */
-static void
-SetCurrentFragmentHeaders(PxfFdwScanState *pxfsstate)
-{
- FragmentData *frag_data = (FragmentData *) lfirst(pxfsstate->current_fragment);
- int fragment_count = list_length(pxfsstate->fragments);
-
- elog(DEBUG2, "pxf_fdw: set_current_fragment_source_name: source_name %s, index %s, has user data: %s ",
- frag_data->source_name, frag_data->index, frag_data->user_data ? "TRUE" : "FALSE");
-
- churl_headers_override(pxfsstate->churl_headers, "X-GP-DATA-DIR", frag_data->source_name);
- churl_headers_override(pxfsstate->churl_headers, "X-GP-DATA-FRAGMENT", frag_data->index);
- churl_headers_override(pxfsstate->churl_headers, "X-GP-FRAGMENT-METADATA", frag_data->fragment_md);
- churl_headers_override(pxfsstate->churl_headers, "X-GP-FRAGMENT-INDEX", frag_data->index);
-
- if (frag_data->fragment_idx == fragment_count)
- {
- churl_headers_override(pxfsstate->churl_headers, "X-GP-LAST-FRAGMENT", "true");
- }
-
- if (frag_data->user_data)
- {
- churl_headers_override(pxfsstate->churl_headers, "X-GP-FRAGMENT-USER-DATA", frag_data->user_data);
- }
- else
- {
- churl_headers_remove(pxfsstate->churl_headers, "X-GP-FRAGMENT-USER-DATA", true);
- }
-
- if (frag_data->profile)
- {
- /* if current fragment has optimal profile set it */
- churl_headers_override(pxfsstate->churl_headers, "X-GP-PROFILE", frag_data->profile);
- elog(DEBUG2, "pxf_fdw: SetCurrentFragmentHeaders: using profile: %s", frag_data->profile);
-
- }
-}
-
-/*
- * Read data from churl until the buffer is full or there is no more data to be read
- */
-static size_t
-FillBuffer(PxfFdwScanState *pxfsstate, char *start, int minlen, int maxlen)
-{
- size_t n = 0;
- char *ptr = start;
- char *minend = ptr + minlen;
- char *maxend = ptr + maxlen;
-
- while (ptr < minend)
- {
- n = churl_read(pxfsstate->churl_handle, ptr, maxend - ptr);
- if (n == 0)
- break;
-
- ptr += n;
- }
-
- return ptr - start;
-}
diff --git a/gpcontrib/pxf_fdw/pxf_bridge.h b/gpcontrib/pxf_fdw/pxf_bridge.h
deleted file mode 100644
index 1ef7d8c906f..00000000000
--- a/gpcontrib/pxf_fdw/pxf_bridge.h
+++ /dev/null
@@ -1,85 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- *
- */
-
-#ifndef _PXFBRIDGE_H
-#define _PXFBRIDGE_H
-
-#include "libchurl.h"
-#include "pxf_fragment.h"
-
-#include "cdb/cdbvars.h"
-#include "nodes/pg_list.h"
-
-/*
- * Execution state of a foreign scan using pxf_fdw.
- */
-typedef struct PxfFdwScanState
-{
- CHURL_HEADERS churl_headers;
- CHURL_HANDLE churl_handle;
- StringInfoData uri;
- ListCell *current_fragment;
- Relation relation;
- char *filter_str;
- ExprState *quals;
- List *fragments;
- List *retrieved_attrs;
- PxfOptions *options;
-#if PG_VERSION_NUM < 140000
- CopyState cstate;
-#else
- CopyFromState cstate;
-#endif
-} PxfFdwScanState;
-
-/*
- * Execution state of a foreign insert operation.
- */
-typedef struct PxfFdwModifyState
-{
-#if PG_VERSION_NUM < 140000
- CopyState cstate; /* state of writing to PXF */
-#else
- CopyToState cstate; /* state of writing to PXF */
-#endif
-
- CHURL_HANDLE churl_handle; /* curl handle */
- CHURL_HEADERS churl_headers; /* curl headers */
- StringInfoData uri; /* rest endpoint URI for modify */
- Relation relation;
- PxfOptions *options; /* FDW options */
-} PxfFdwModifyState;
-
-/* Clean up churl related data structures from the context */
-void PxfBridgeCleanup(PxfFdwModifyState *context);
-
-/* Sets up data before starting import */
-void PxfBridgeImportStart(PxfFdwScanState *pxfsstate);
-
-/* Sets up data before starting export */
-void PxfBridgeExportStart(PxfFdwModifyState *pxfmstate);
-
-/* Reads data from the PXF server into the given buffer of a given size */
-int PxfBridgeRead(void *outbuf, int minlen, int maxlen, void *extra);
-
-/* Writes data from the given buffer of a given size to the PXF server */
-int PxfBridgeWrite(PxfFdwModifyState *context, char *databuf, int datalen);
-
-#endif /* _PXFBRIDGE_H */
diff --git a/gpcontrib/pxf_fdw/pxf_deparse.c b/gpcontrib/pxf_fdw/pxf_deparse.c
deleted file mode 100644
index 33214dd969e..00000000000
--- a/gpcontrib/pxf_fdw/pxf_deparse.c
+++ /dev/null
@@ -1,77 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * pxf_deparse.c
- * Query deparser for pxf_fdw
- *
- * This file includes functions that examine query WHERE clauses to see
- * whether they're safe to send to the remote server for execution.
- *
- * IDENTIFICATION
- * contrib/pxf_fdw/pxf_deparse.c
- *
- *-------------------------------------------------------------------------
- */
-#include "postgres.h"
-
-#include "pxf_fdw.h"
-
-/*
- * We create an integer List of the columns being retrieved, which is
- * returned to *retrieved_attrs.
- */
-void
-deparseTargetList(Relation rel, Bitmapset *attrs_used, List **retrieved_attrs)
-{
- TupleDesc tupdesc = RelationGetDescr(rel);
- bool have_wholerow;
- int i;
-
- *retrieved_attrs = NIL;
-
- /* If there's a whole-row reference, we'll need all the columns. */
- have_wholerow = bms_is_member(0 - FirstLowInvalidHeapAttributeNumber, attrs_used);
-
- if (have_wholerow)
- return;
-
- for (i = 1; i <= tupdesc->natts; i++)
- {
- Form_pg_attribute attr = TupleDescAttr(tupdesc, i - 1);
-
- /* Ignore dropped attributes. */
- if (attr->attisdropped)
- continue;
-
- if (bms_is_member(i - FirstLowInvalidHeapAttributeNumber, attrs_used))
- {
- *retrieved_attrs = lappend_int(*retrieved_attrs, i);
- }
- }
-}
-
-/*
- * Examine each qual clause in input_conds, and classify them into two groups,
- * which are returned as two lists:
- * - remote_conds contains expressions that can be evaluated remotely
- * - local_conds contains expressions that can't be evaluated remotely
- */
-void
-classifyConditions(PlannerInfo *root,
- RelOptInfo *baserel,
- List *input_conds,
- List **remote_conds,
- List **local_conds)
-{
- ListCell *lc;
-
- *remote_conds = NIL;
- *local_conds = NIL;
-
- foreach(lc, input_conds)
- {
- RestrictInfo *ri = (RestrictInfo *) lfirst(lc);
-
- /* for now, just assume that all WHERE clauses are OK on remote */
- *remote_conds = lappend(*remote_conds, ri);
- }
-}
diff --git a/gpcontrib/pxf_fdw/pxf_fdw--1.0.sql b/gpcontrib/pxf_fdw/pxf_fdw--1.0.sql
deleted file mode 100644
index ca9fa73b64d..00000000000
--- a/gpcontrib/pxf_fdw/pxf_fdw--1.0.sql
+++ /dev/null
@@ -1,59 +0,0 @@
-/* contrib/pxf_fdw/pxf_fdw--1.0.sql */
-
--- complain if script is sourced in psql, rather than via CREATE EXTENSION
-\echo Use "CREATE EXTENSION pxf_fdw" to load this file. \quit
-
-CREATE FUNCTION pxf_fdw_handler()
-RETURNS fdw_handler
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT;
-
-CREATE FUNCTION pxf_fdw_validator(text[], oid)
-RETURNS void
-AS 'MODULE_PATHNAME'
-LANGUAGE C STRICT;
-
-CREATE FOREIGN DATA WRAPPER jdbc_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'jdbc', mpp_execute 'all segments' );
-
-CREATE FOREIGN DATA WRAPPER hdfs_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'hdfs', mpp_execute 'all segments' );
-
-CREATE FOREIGN DATA WRAPPER hive_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'hive', mpp_execute 'all segments' );
-
-CREATE FOREIGN DATA WRAPPER hbase_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'hbase', mpp_execute 'all segments' );
-
-CREATE FOREIGN DATA WRAPPER s3_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 's3', mpp_execute 'all segments' );
-
-CREATE FOREIGN DATA WRAPPER gs_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'gs', mpp_execute 'all segments' );
-
-CREATE FOREIGN DATA WRAPPER adl_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'adl', mpp_execute 'all segments' );
-
-CREATE FOREIGN DATA WRAPPER wasbs_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'wasbs', mpp_execute 'all segments' );
-
-CREATE FOREIGN DATA WRAPPER file_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'localfile', mpp_execute 'all segments' );
\ No newline at end of file
diff --git a/gpcontrib/pxf_fdw/pxf_fdw.c b/gpcontrib/pxf_fdw/pxf_fdw.c
deleted file mode 100644
index 0ce9a82c1b4..00000000000
--- a/gpcontrib/pxf_fdw/pxf_fdw.c
+++ /dev/null
@@ -1,935 +0,0 @@
-/*
- * pxf_fdw.c
- * Foreign-data wrapper for PXF (Platform Extension Framework)
- *
- * IDENTIFICATION
- * contrib/pxf_fdw/pxf_fdw.c
- */
-
-#include "postgres.h"
-
-#include "pxf_fdw.h"
-#include "pxf_bridge.h"
-#include "pxf_filter.h"
-#include "pxf_fragment.h"
-
-#include "access/reloptions.h"
-#include "access/table.h"
-#include "cdb/cdbsreh.h"
-#include "cdb/cdbvars.h"
-#include "commands/copy.h"
-#if (PG_VERSION_NUM >= 140000)
-#include "commands/copyfrom_internal.h"
-#include "commands/copyto_internal.h"
-#endif
-#include "commands/defrem.h"
-#include "commands/explain.h"
-#include "foreign/fdwapi.h"
-#include "foreign/foreign.h"
-#include "nodes/pg_list.h"
-#include "optimizer/optimizer.h"
-#include "optimizer/paths.h"
-#include "optimizer/pathnode.h"
-#include "optimizer/planmain.h"
-#include "optimizer/restrictinfo.h"
-#include "parser/parsetree.h"
-#if (PG_VERSION_NUM >= 140000)
-#include "utils/backend_progress.h"
-#endif
-#include "utils/lsyscache.h"
-#include "utils/memutils.h"
-
-PG_MODULE_MAGIC;
-
-#define DEFAULT_PXF_FDW_STARTUP_COST 50000
-
-extern Datum pxf_fdw_handler(PG_FUNCTION_ARGS);
-
-/*
- * SQL functions
- */
-PG_FUNCTION_INFO_V1(pxf_fdw_handler);
-
-/*
- * FDW functions declarations
- */
-static void pxfGetForeignRelSize(PlannerInfo *root, RelOptInfo *baserel, Oid foreigntableid);
-
-static void pxfGetForeignPaths(PlannerInfo *root, RelOptInfo *baserel, Oid foreigntableid);
-
-#if (PG_VERSION_NUM <= 90500)
-
-static ForeignScan *pxfGetForeignPlan(PlannerInfo *root, RelOptInfo *baserel, Oid foreigntableid, ForeignPath *best_path, List *tlist,
- List *scan_clauses);
-
-#else
-static ForeignScan *pxfGetForeignPlan(PlannerInfo *root, RelOptInfo *baserel, Oid foreigntableid, ForeignPath *best_path, List *tlist, List *scan_clauses, Plan *outer_plan);
-#endif
-
-static void pxfExplainForeignScan(ForeignScanState *node, ExplainState *es);
-
-static void pxfBeginForeignScan(ForeignScanState *node, int eflags);
-
-static TupleTableSlot *pxfIterateForeignScan(ForeignScanState *node);
-
-static void pxfReScanForeignScan(ForeignScanState *node);
-
-static void pxfEndForeignScan(ForeignScanState *node);
-
-/* Foreign updates */
-static void pxfBeginForeignModify(ModifyTableState *mtstate, ResultRelInfo *resultRelInfo, List *fdw_private, int subplan_index, int eflags);
-static TupleTableSlot *pxfExecForeignInsert(EState *estate, ResultRelInfo *resultRelInfo, TupleTableSlot *slot, TupleTableSlot *planSlot);
-
-static void pxfEndForeignModify(EState *estate, ResultRelInfo *resultRelInfo);
-
-static int pxfIsForeignRelUpdatable(Relation rel);
-
-/*
- * Helper functions
- */
-static void InitCopyState(PxfFdwScanState *pxfsstate);
-static void InitCopyStateForModify(PxfFdwModifyState *pxfmstate);
-
-#if (PG_VERSION_NUM < 140000)
-static CopyState BeginCopyTo(Relation forrel, List *options);
-#else
-static CopyToState BeginCopyToModify(Relation forrel, List *options);
-#endif
-
-#if (PG_VERSION_NUM >= 140000)
-static void EndCopyScan(CopyFromState cstate);
-static void EndCopyModify(CopyToState cstate);
-#endif
-
-/*
- * Foreign-data wrapper handler functions:
- * returns a struct with pointers to the
- * pxf_fdw callback routines.
- */
-Datum
-pxf_fdw_handler(PG_FUNCTION_ARGS)
-{
- FdwRoutine *fdw_routine = makeNode(FdwRoutine);
-
- /*
- * foreign table scan support
- */
-
- /* master - only */
- fdw_routine->GetForeignRelSize = pxfGetForeignRelSize;
- fdw_routine->GetForeignPaths = pxfGetForeignPaths;
- fdw_routine->GetForeignPlan = pxfGetForeignPlan;
- fdw_routine->ExplainForeignScan = pxfExplainForeignScan;
-
- /* segment - only when mpp_execute = segments */
- fdw_routine->BeginForeignScan = pxfBeginForeignScan;
- fdw_routine->IterateForeignScan = pxfIterateForeignScan;
- fdw_routine->ReScanForeignScan = pxfReScanForeignScan;
- fdw_routine->EndForeignScan = pxfEndForeignScan;
-
- /*
- * foreign table insert support
- */
-
- /*
- * AddForeignUpdateTargets set to NULL, no extra target expressions are
- * added
- */
- fdw_routine->AddForeignUpdateTargets = NULL;
-
- /*
- * PlanForeignModify set to NULL, no additional plan-time actions are
- * taken
- */
- fdw_routine->PlanForeignModify = NULL;
- fdw_routine->BeginForeignModify = pxfBeginForeignModify;
- fdw_routine->ExecForeignInsert = pxfExecForeignInsert;
-
- /*
- * ExecForeignUpdate and ExecForeignDelete set to NULL since updates and
- * deletes are not supported
- */
- fdw_routine->ExecForeignUpdate = NULL;
- fdw_routine->ExecForeignDelete = NULL;
- fdw_routine->EndForeignModify = pxfEndForeignModify;
- fdw_routine->IsForeignRelUpdatable = pxfIsForeignRelUpdatable;
-
- PG_RETURN_POINTER(fdw_routine);
-}
-
-typedef struct PxfFdwRelationInfo
-{
- /* baserestrictinfo clauses, broken down into safe and unsafe subsets. */
- List *remote_conds;
- List *local_conds;
-
- /* List of attributes (columns) that we need to get */
- List *retrieved_attrs;
-
- /* Bitmap of attr numbers we need to fetch from the remote server. */
- Bitmapset *attrs_used;
-} PxfFdwRelationInfo;
-
-/*
- * Indexes of FDW-private information stored in fdw_private lists.
- *
- * We store various information in ForeignScan.fdw_private to pass it from
- * planner to executor. Currently we store:
- *
- * 1) WHERE clause text to be sent to the remote server
- * 2) Integer list of attribute numbers retrieved by the SELECT
- *
- * These items are indexed with the enum FdwScanPrivateIndex, so an item
- * can be fetched with list_nth(). For example, to get the WHERE clauses:
- * sql = strVal(list_nth(fdw_private, FdwScanPrivateWhereClauses));
- */
-enum FdwScanPrivateIndex
-{
- /* WHERE clauses to be sent to PXF (as a String node) */
- FdwScanPrivateWhereClauses,
- /* Integer list of attribute numbers retrieved by the SELECT */
- FdwScanPrivateRetrievedAttrs,
- /* List of fragments to be processed by the segments */
- FdwScanPrivateFragmentList
-};
-
-/*
- * GetForeignRelSize
- * set relation size estimates for a foreign table
- */
-static void
-pxfGetForeignRelSize(PlannerInfo *root, RelOptInfo *baserel, Oid foreigntableid)
-{
- elog(DEBUG5, "pxf_fdw: pxfGetForeignRelSize starts on segment: %d", PXF_SEGMENT_ID);
- Relation rel;
- ListCell *lc;
-
- PxfFdwRelationInfo *fpinfo = (PxfFdwRelationInfo *) palloc(sizeof(PxfFdwRelationInfo));
-
- baserel->fdw_private = (void *) fpinfo;
-
- fpinfo->attrs_used = NULL;
-
- /*
- * Core code already has some lock on each rel being planned, so we can
- * use NoLock here.
- */
- RangeTblEntry *rte = planner_rt_fetch(baserel->relid, root);
-
- rel = table_open(rte->relid, NoLock);
-
- /*
- * Identify which baserestrictinfo clauses can be sent to the remote
- * server and which can't.
- */
- classifyConditions(root, baserel, baserel->baserestrictinfo,
- &fpinfo->remote_conds, &fpinfo->local_conds);
-
- /*
- * Identify which attributes will need to be retrieved from the remote
- * server
- */
-#if (PG_VERSION_NUM <= 90500)
- pull_varattnos((Node *) baserel->reltargetlist, baserel->relid, &fpinfo->attrs_used);
-#else
- pull_varattnos((Node *) baserel->reltarget->exprs, baserel->relid, &fpinfo->attrs_used);
-#endif
- foreach(lc, fpinfo->local_conds)
- {
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
-
- pull_varattnos((Node *) rinfo->clause, baserel->relid, &fpinfo->attrs_used);
- }
-
- deparseTargetList(rel, fpinfo->attrs_used, &fpinfo->retrieved_attrs);
-
- heap_close(rel, NoLock);
-
- /* Use an artificial number of estimated rows */
- baserel->rows = 1000;
-
- elog(DEBUG5, "pxf_fdw: pxfGetForeignRelSize ends on segment: %d", PXF_SEGMENT_ID);
-}
-
-/*
- * GetForeignPaths
- * create access path for a scan on the foreign table
- */
-static void
-pxfGetForeignPaths(PlannerInfo *root,
- RelOptInfo *baserel,
- Oid foreigntableid)
-{
- ForeignPath *path = NULL;
- int total_cost = DEFAULT_PXF_FDW_STARTUP_COST;
- PxfFdwRelationInfo *fpinfo = (PxfFdwRelationInfo *) baserel->fdw_private;
-
-
- elog(DEBUG5, "pxf_fdw: pxfGetForeignPaths starts on segment: %d", PXF_SEGMENT_ID);
-
-
- path = create_foreignscan_path(root, baserel,
-#if PG_VERSION_NUM >= 90600
- NULL, /* default pathtarget */
-#endif
- baserel->rows,
- DEFAULT_PXF_FDW_STARTUP_COST,
- total_cost,
- NIL, /* no pathkeys */
- NULL, /* no outer rel either */
-#if PG_VERSION_NUM >= 90500
- NULL, /* no extra plan */
-#endif
- fpinfo->retrieved_attrs);
-
-
-
- /*
- * Create a ForeignPath node and add it as only possible path.
- */
- add_path(baserel, (Path *) path, root);
-
- elog(DEBUG5, "pxf_fdw: pxfGetForeignPaths ends on segment: %d", PXF_SEGMENT_ID);
-}
-
-/*
- * GetForeignPlan
- * create a ForeignScan plan node
- */
-#if PG_VERSION_NUM >= 90500
-static ForeignScan *
-pxfGetForeignPlan(PlannerInfo *root,
- RelOptInfo *baserel,
- Oid foreigntableid,
- ForeignPath *best_path,
- List *tlist,
- List *scan_clauses,
- Plan *outer_plan)
-#else
-static ForeignScan *
-pxfGetForeignPlan(PlannerInfo *root,
- RelOptInfo *baserel,
- Oid foreigntableid,
- ForeignPath *best_path,
- List *tlist, /* target list */
- List *scan_clauses)
-#endif
-{
- Index scan_relid = baserel->relid;
- PxfFdwRelationInfo *fpinfo = (PxfFdwRelationInfo *) baserel->fdw_private;
- List *fdw_private;
-
- elog(DEBUG5, "pxf_fdw: pxfGetForeignPlan starts on segment: %d", PXF_SEGMENT_ID);
-
- /*
- * We have no native ability to evaluate restriction clauses, so we just
- * put all the scan_clauses into the plan node's qual list for the
- * executor to check. So all we have to do here is strip RestrictInfo
- * nodes from the clauses and ignore pseudoconstants (which will be
- * handled elsewhere).
- */
- scan_clauses = extract_actual_clauses(scan_clauses, false);
-
- /*
- * Build the fdw_private list that will be available to the executor.
- * Items in the list must match enum FdwScanPrivateIndex, above.
- */
-
- /* here we serialize the WHERE clauses */
- char *where_clauses_str = SerializePxfFilterQuals(fpinfo->remote_conds);
-
- fdw_private = list_make2(makeString(where_clauses_str), fpinfo->retrieved_attrs);
-
- elog(DEBUG5, "pxf_fdw: pxfGetForeignPlan ends on segment: %d", PXF_SEGMENT_ID);
-
- return make_foreignscan(
- tlist,
- scan_clauses,
- scan_relid,
- NIL, /* no expressions to evaluate */
- fdw_private
-#if PG_VERSION_NUM >= 90500
- ,NIL
- ,NIL
- ,outer_plan
-#endif
- );
-
-}
-
-/*
- * pxfExplainForeignScan
- * Produce extra output for EXPLAIN of a ForeignScan on a foreign table
- */
-static void
-pxfExplainForeignScan(ForeignScanState *node, ExplainState *es)
-{
- elog(DEBUG5, "pxf_fdw: pxfExplainForeignScan starts on segment: %d", PXF_SEGMENT_ID);
-
- /* TODO: make this a meaningful callback */
-
- elog(DEBUG5, "pxf_fdw: pxfExplainForeignScan ends on segment: %d", PXF_SEGMENT_ID);
-}
-
-/*
- * BeginForeignScan
- * called during executor startup. perform any initialization
- * needed, but not start the actual scan.
- */
-static void
-pxfBeginForeignScan(ForeignScanState *node, int eflags)
-{
- elog(DEBUG5, "pxf_fdw: pxfBeginForeignScan starts on segment: %d", PXF_SEGMENT_ID);
-
- /*
- * Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
- */
- if (eflags & EXEC_FLAG_EXPLAIN_ONLY)
- return;
-
- List* serializedFragmentList;
- List* fragments;
- ForeignTable *rel = GetForeignTable(RelationGetRelid(node->ss.ss_currentRelation));
-
- ExprState *quals = node->ss.ps.qual;
- Oid foreigntableid = RelationGetRelid(node->ss.ss_currentRelation);
- PxfFdwScanState *pxfsstate = NULL;
- Relation relation = node->ss.ss_currentRelation;
- ForeignScan *foreignScan = (ForeignScan *) node->ss.ps.plan;
- PxfOptions *options = PxfGetOptions(foreigntableid);
-
- /* retrieve fdw-private information from pxfGetForeignPlan() */
- char *filter_str = strVal(list_nth(foreignScan->fdw_private, FdwScanPrivateWhereClauses));
- List *retrieved_attrs = (List *) list_nth(foreignScan->fdw_private, FdwScanPrivateRetrievedAttrs);
-
- /*
- * When running queries on all segments, master makes the fragmenter call
- * and segments receive a List of FragmentData from master. When the query
- * only runs on coordinator or any, the fragment list is retrieved by the
- * executing process.
- */
- if (rel->exec_location != FTEXECLOCATION_ALL_SEGMENTS || Gp_role == GP_ROLE_DISPATCH)
- {
- fragments = GetFragmentList(options,
- relation,
- filter_str,
- retrieved_attrs);
-
- if (Gp_role == GP_ROLE_DISPATCH)
- {
- /*
- * serialize fragment list and pass it down to segments
- * by appending it to foreignScan->fdw_private
- */
- serializedFragmentList = SerializeFragmentList(fragments);
- foreignScan->fdw_private = lappend(foreignScan->fdw_private, serializedFragmentList);
-
- /* master does not process any fragments */
- return;
- }
- }
- else
- {
- serializedFragmentList = (List *) list_nth(foreignScan->fdw_private, FdwScanPrivateFragmentList);
- fragments = DeserializeFragmentList(serializedFragmentList);
-
- /*
- * Call the work allocation algorithm when execution happens on all
- * segments
- */
- fragments = FilterFragmentsForSegment(fragments);
- }
-
- /* Assign PXF location for the allocated fragments */
- AssignPxfLocationToFragments(options, fragments);
-
- /*
- * Save state in node->fdw_state. We must save enough information to call
- * BeginCopyFrom() again.
- */
- pxfsstate = (PxfFdwScanState *) palloc(sizeof(PxfFdwScanState));
- initStringInfo(&pxfsstate->uri);
-
- pxfsstate->filter_str = filter_str;
- pxfsstate->fragments = fragments;
- pxfsstate->options = options;
- pxfsstate->quals = quals;
- pxfsstate->relation = relation;
- pxfsstate->retrieved_attrs = retrieved_attrs;
-
- InitCopyState(pxfsstate);
- node->fdw_state = (void *) pxfsstate;
-
- elog(DEBUG5, "pxf_fdw: pxfBeginForeignScan ends on segment: %d", PXF_SEGMENT_ID);
-}
-
-/*
- * IterateForeignScan
- * Retrieve next row from the result set, or clear tuple slot to indicate
- * EOF.
- * Fetch one row from the foreign source, returning it in a tuple table slot
- * (the node's ScanTupleSlot should be used for this purpose).
- * Return NULL if no more rows are available.
- */
-static TupleTableSlot *
-pxfIterateForeignScan(ForeignScanState *node)
-{
- elog(DEBUG5, "pxf_fdw: pxfIterateForeignScan Executing on segment: %d", PXF_SEGMENT_ID);
-
- PxfFdwScanState *pxfsstate = (PxfFdwScanState *) node->fdw_state;
- TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
- ErrorContextCallback errcallback;
- bool found;
-
- /* Set up callback to identify error line number. */
- errcallback.callback = CopyFromErrorCallback;
- errcallback.arg = (void *) pxfsstate->cstate;
- errcallback.previous = error_context_stack;
- error_context_stack = &errcallback;
-
- /*
- * The protocol for loading a virtual tuple into a slot is first
- * ExecClearTuple, then fill the values/isnull arrays, then
- * ExecStoreVirtualTuple. If we don't find another row in the file, we
- * just skip the last step, leaving the slot empty as required.
- *
- * We can pass ExprContext = NULL because we read all columns from the
- * file, so no need to evaluate default expressions.
- */
- ExecClearTuple(slot);
-
- found = NextCopyFrom(pxfsstate->cstate,
- NULL,
- slot->tts_values,
- slot->tts_isnull);
- if (found)
- {
- if (pxfsstate->cstate->cdbsreh)
- {
- /*
- * If NextCopyFrom failed, the processed row count will have
- * already been updated, but we need to update it in a successful
- * case.
- *
- * GPDB_91_MERGE_FIXME: this is almost certainly not the right
- * place for this, but row counts are currently scattered all over
- * the place. Consolidate.
- */
- pxfsstate->cstate->cdbsreh->processed++;
- }
-
- ExecStoreVirtualTuple(slot);
- }
-
- /* Remove error callback. */
- error_context_stack = errcallback.previous;
-
- return slot;
-}
-
-/*
- * ReScanForeignScan
- * Restart the scan from the beginning
- */
-static void
-pxfReScanForeignScan(ForeignScanState *node)
-{
- elog(DEBUG5, "pxf_fdw: pxfReScanForeignScan starts on segment: %d", PXF_SEGMENT_ID);
-
- PxfFdwScanState *pxfsstate = (PxfFdwScanState *) node->fdw_state;
-
-#if (PG_VERSION_NUM < 140000)
- EndCopyFrom(pxfsstate->cstate);
-#else
- EndCopyScan(pxfsstate->cstate);
-#endif
- InitCopyState(pxfsstate);
-
- elog(DEBUG5, "pxf_fdw: pxfReScanForeignScan ends on segment: %d", PXF_SEGMENT_ID);
-}
-
-/*
- * EndForeignScan
- * End the scan and release resources.
- */
-static void
-pxfEndForeignScan(ForeignScanState *node)
-{
- elog(DEBUG5, "pxf_fdw: pxfEndForeignScan starts on segment: %d", PXF_SEGMENT_ID);
-
- ForeignScan *foreignScan = (ForeignScan *) node->ss.ps.plan;
- PxfFdwScanState *pxfsstate = (PxfFdwScanState *) node->fdw_state;
-
- /* Release resources */
- if (foreignScan->fdw_private)
- {
- elog(DEBUG5, "Freeing fdw_private");
- pfree(foreignScan->fdw_private);
- }
-
- /* if pxfsstate is NULL, we are in EXPLAIN; nothing to do */
- if (pxfsstate)
-#if (PG_VERSION_NUM < 140000)
- EndCopyFrom(pxfsstate->cstate);
-#else
- EndCopyScan(pxfsstate->cstate);
-#endif
-
- elog(DEBUG5, "pxf_fdw: pxfEndForeignScan ends on segment: %d", PXF_SEGMENT_ID);
-}
-
-/*
- * pxfBeginForeignModify
- * Begin an insert/update/delete operation on a foreign table
- */
-static void
-pxfBeginForeignModify(ModifyTableState *mtstate,
- ResultRelInfo *resultRelInfo,
- List *fdw_private,
- int subplan_index,
- int eflags)
-{
- elog(DEBUG5, "pxf_fdw: pxfBeginForeignModify starts on segment: %d", PXF_SEGMENT_ID);
-
- PxfOptions *options = NULL;
- PxfFdwModifyState *pxfmstate = NULL;
- Relation relation = resultRelInfo->ri_RelationDesc;
- TupleDesc tupDesc;
-
- if (eflags & EXEC_FLAG_EXPLAIN_ONLY)
- return;
-
- tupDesc = RelationGetDescr(relation);
- options = PxfGetOptions(RelationGetRelid(relation));
- pxfmstate = palloc(sizeof(PxfFdwModifyState));
-
- initStringInfo(&pxfmstate->uri);
- pxfmstate->relation = relation;
- pxfmstate->options = options;
-
- InitCopyStateForModify(pxfmstate);
-
- resultRelInfo->ri_FdwState = pxfmstate;
-
- elog(DEBUG5, "pxf_fdw: pxfBeginForeignModify ends on segment: %d", PXF_SEGMENT_ID);
-}
-
-/*
- * pxfExecForeignInsert
- * Insert one row into a foreign table
- */
-static TupleTableSlot *
-pxfExecForeignInsert(EState *estate,
- ResultRelInfo *resultRelInfo,
- TupleTableSlot *slot,
- TupleTableSlot *planSlot)
-{
- elog(DEBUG5, "pxf_fdw: pxfExecForeignInsert starts on segment: %d", PXF_SEGMENT_ID);
-
- PxfFdwModifyState *pxfmstate = (PxfFdwModifyState *) resultRelInfo->ri_FdwState;
-#if (PG_VERSION_NUM < 140000)
- CopyState cstate = pxfmstate->cstate;
-#else
- CopyToState cstate = pxfmstate->cstate;
-#endif
-
- /* TEXT or CSV */
- slot_getallattrs(slot);
- CopyOneRowTo(cstate, slot);
- CopySendEndOfRow(cstate);
-
- StringInfo fe_msgbuf = cstate->fe_msgbuf;
-
- int bytes_written = PxfBridgeWrite(pxfmstate, fe_msgbuf->data, fe_msgbuf->len);
-
- if (bytes_written == -1)
- {
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not write to foreign resource: %m")));
- }
-
- elog(DEBUG3, "pxf_fdw %d bytes written", bytes_written);
-
- /* Reset our buffer to start clean next round */
- cstate->fe_msgbuf->len = 0;
- cstate->fe_msgbuf->data[0] = '\0';
-
- elog(DEBUG5, "pxf_fdw: pxfExecForeignInsert ends on segment: %d", PXF_SEGMENT_ID);
- return slot;
-}
-
-/*
- * pxfEndForeignModify
- * Finish an insert/update/delete operation on a foreign table
- */
-static void
-pxfEndForeignModify(EState *estate,
- ResultRelInfo *resultRelInfo)
-{
- elog(DEBUG5, "pxf_fdw: pxfEndForeignModify starts on segment: %d", PXF_SEGMENT_ID);
-
- PxfFdwModifyState *pxfmstate = (PxfFdwModifyState *) resultRelInfo->ri_FdwState;
-
- /* If pxfmstate is NULL, we are in EXPLAIN; nothing to do */
- if (pxfmstate == NULL)
- return;
-
-#if (PG_VERSION_NUM < 140000)
- EndCopyFrom(pxfmstate->cstate);
-#else
- EndCopyModify(pxfmstate->cstate);
-#endif
- PxfBridgeCleanup(pxfmstate);
-
- elog(DEBUG5, "pxf_fdw: pxfEndForeignModify ends on segment: %d", PXF_SEGMENT_ID);
-}
-
-/*
- * pxfIsForeignRelUpdatable
- * Assume table is updatable regardless of settings.
- * Determine whether a foreign table supports INSERT, UPDATE and/or
- * DELETE.
- */
-static int
-pxfIsForeignRelUpdatable(Relation rel)
-{
- elog(DEBUG5, "pxf_fdw: pxfIsForeignRelUpdatable starts on segment: %d", PXF_SEGMENT_ID);
- elog(DEBUG5, "pxf_fdw: pxfIsForeignRelUpdatable ends on segment: %d", PXF_SEGMENT_ID);
- /* Only INSERTs are allowed at the moment */
- return 1u << (unsigned int) CMD_INSERT | 0u << (unsigned int) CMD_UPDATE | 0u << (unsigned int) CMD_DELETE;
-}
-
-/*
- * Initiates a copy state for pxfBeginForeignScan() and pxfReScanForeignScan()
- */
-static void
-InitCopyState(PxfFdwScanState *pxfsstate)
-{
-#if (PG_VERSION_NUM < 140000)
- CopyState cstate;
-#else
- CopyFromState cstate;
-#endif
-
- PxfBridgeImportStart(pxfsstate);
-
- /*
- * Create CopyState from FDW options. We always acquire all columns, so
- * as to match the expected ScanTupleSlot signature.
- */
- cstate = BeginCopyFrom(NULL,
- pxfsstate->relation,
-#if (PG_VERSION_NUM >= 140000)
- NULL,
-#endif
- NULL,
- false, /* is_program */
- &PxfBridgeRead, /* data_source_cb */
- pxfsstate, /* data_source_cb_extra */
- NIL, /* attnamelist */
- pxfsstate->options->copy_options); /* copy options */
-
-
- if (pxfsstate->options->reject_limit == -1)
- {
- /* Default error handling - "all-or-nothing" */
- cstate->cdbsreh = NULL; /* no SREH */
- cstate->errMode = ALL_OR_NOTHING;
- }
- else
- {
- /* no error log by default */
- cstate->errMode = SREH_IGNORE;
-
- /* select the SREH mode */
- if (pxfsstate->options->log_errors)
- cstate->errMode = SREH_LOG; /* errors into file */
-
- cstate->cdbsreh = makeCdbSreh(pxfsstate->options->reject_limit,
- pxfsstate->options->is_reject_limit_rows,
- pxfsstate->options->resource,
- (char *) cstate->cur_relname,
- pxfsstate->options->log_errors ? LOG_ERRORS_ENABLE : LOG_ERRORS_DISABLE);
-
- cstate->cdbsreh->relid = RelationGetRelid(pxfsstate->relation);
- }
-
- /* and 'fe_mgbuf' */
- cstate->fe_msgbuf = makeStringInfo();
-
- /*
- * Create a temporary memory context that we can reset once per row to
- * recover palloc'd memory. This avoids any problems with leaks inside
- * datatype input or output routines, and should be faster than retail
- * pfree's anyway.
- */
- cstate->rowcontext = AllocSetContextCreate(CurrentMemoryContext,
- "PxfFdwMemCxt",
- ALLOCSET_DEFAULT_MINSIZE,
- ALLOCSET_DEFAULT_INITSIZE,
- ALLOCSET_DEFAULT_MAXSIZE);
-
- pxfsstate->cstate = cstate;
-}
-
-/*
- * Initiates a copy state for pxfBeginForeignModify()
- */
-static void
-InitCopyStateForModify(PxfFdwModifyState *pxfmstate)
-{
- List *copy_options;
-#if (PG_VERSION_NUM < 140000)
- CopyState cstate;
-#else
- CopyToState cstate;
-#endif
-
- copy_options = pxfmstate->options->copy_options;
-
- PxfBridgeExportStart(pxfmstate);
-
- /*
- * Create CopyState from FDW options. We always acquire all columns, so
- * as to match the expected ScanTupleSlot signature.
- */
-#if (PG_VERSION_NUM < 140000)
- cstate = BeginCopyTo(pxfmstate->relation, copy_options);
-#else
- cstate = BeginCopyToModify(pxfmstate->relation, copy_options);
-#endif
-
- /* Initialize 'out_functions', like CopyTo() would. */
-
- TupleDesc tupDesc = RelationGetDescr(cstate->rel);
- Form_pg_attribute attr = tupDesc->attrs;
- int num_phys_attrs = tupDesc->natts;
-
- cstate->out_functions = (FmgrInfo *) palloc(num_phys_attrs * sizeof(FmgrInfo));
- ListCell *cur;
-
- foreach(cur, cstate->attnumlist)
- {
- int attnum = lfirst_int(cur);
- Oid out_func_oid;
- bool isvarlena;
-
- getTypeOutputInfo(attr[attnum - 1].atttypid,
- &out_func_oid,
- &isvarlena);
- fmgr_info(out_func_oid, &cstate->out_functions[attnum - 1]);
- }
-
- /* and 'fe_mgbuf' */
- cstate->fe_msgbuf = makeStringInfo();
-
- /*
- * Create a temporary memory context that we can reset once per row to
- * recover palloc'd memory. This avoids any problems with leaks inside
- * datatype input or output routines, and should be faster than retail
- * pfree's anyway.
- */
- cstate->rowcontext = AllocSetContextCreate(CurrentMemoryContext,
- "PxfFdwMemCxt",
- ALLOCSET_DEFAULT_MINSIZE,
- ALLOCSET_DEFAULT_INITSIZE,
- ALLOCSET_DEFAULT_MAXSIZE);
-
- pxfmstate->cstate = cstate;
-}
-
-/*
- * Set up CopyState for writing to an foreign table.
- */
-#if (PG_VERSION_NUM < 140000)
-static CopyState
-BeginCopyTo(Relation forrel, List *options)
-#else
-static CopyToState
-BeginCopyToModify(Relation forrel, List *options)
-#endif
-{
-#if (PG_VERSION_NUM < 140000)
- CopyState cstate;
-#else
- CopyToState cstate;
-#endif
-
- Assert(forrel->rd_rel->relkind == RELKIND_FOREIGN_TABLE);
-
-#if (PG_VERSION_NUM <= 90500)
- cstate = BeginCopy(false, forrel, NULL, NULL, NIL, options, NULL);
-#elif (PG_VERSION_NUM < 120000)
- cstate = BeginCopy(false, forrel, NULL, NULL, forrel->rd_id, NIL, options, NULL);
-#elif (PG_VERSION_NUM < 140000)
- cstate = BeginCopy(NULL, false, forrel, NULL, forrel->rd_id, NIL, options, NULL);
-#else
- cstate = BeginCopy(NULL, forrel, NULL, forrel->rd_id, NIL, options, NULL);
-#endif
- cstate->dispatch_mode = COPY_DIRECT;
-
- /*
- * We use COPY_CALLBACK to mean that the each line should be left in
- * fe_msgbuf. There is no actual callback!
- */
- cstate->copy_dest = COPY_CALLBACK;
-
- /*
- * Some more initialization, that in the normal COPY TO codepath, is done
- * in CopyTo() itself.
- */
-#if (PG_VERSION_NUM < 140000)
- cstate->null_print_client = cstate->null_print; /* default */
- if (cstate->need_transcoding)
- cstate->null_print_client = pg_server_to_custom(cstate->null_print,
- cstate->null_print_len,
- cstate->file_encoding,
- cstate->enc_conversion_proc);
-#else
- cstate->opts.null_print_client = cstate->opts.null_print; /* default */
- if (cstate->need_transcoding)
- cstate->opts.null_print_client = pg_server_to_any(cstate->opts.null_print,
- cstate->opts.null_print_len,
- cstate->opts.file_encoding);
-#endif
- return cstate;
-}
-
-#if (PG_VERSION_NUM >= 140000)
-/*
- * Clean up storage and release resources for COPY FROM.
- */
-static void
-EndCopyScan(CopyFromState cstate)
-{
- /* No COPY FROM related resources except memory. */
- Assert(!cstate->is_program);
- Assert(cstate->filename == NULL);
-
- /* Clean up single row error handling related memory */
- if (cstate->cdbsreh)
- destroyCdbSreh(cstate->cdbsreh);
-
- pgstat_progress_end_command();
-
- MemoryContextDelete(cstate->copycontext);
- pfree(cstate);
-}
-
-/*
- * Clean up storage and release resources for COPY TO.
- */
-static void
-EndCopyModify(CopyToState cstate)
-{
- /* No COPY FROM related resources except memory. */
- Assert(!cstate->is_program);
- Assert(cstate->filename == NULL);
-
- pgstat_progress_end_command();
-
- MemoryContextDelete(cstate->copycontext);
- pfree(cstate);
-}
-#endif
diff --git a/gpcontrib/pxf_fdw/pxf_fdw.control b/gpcontrib/pxf_fdw/pxf_fdw.control
deleted file mode 100644
index 82409463254..00000000000
--- a/gpcontrib/pxf_fdw/pxf_fdw.control
+++ /dev/null
@@ -1,5 +0,0 @@
-# pxf_fdw extension
-comment = 'PXF Foreign Data Wrapper for Apache Cloudberry'
-default_version = '1.0'
-module_pathname = '$libdir/pxf_fdw'
-relocatable = true
\ No newline at end of file
diff --git a/gpcontrib/pxf_fdw/pxf_fdw.h b/gpcontrib/pxf_fdw/pxf_fdw.h
deleted file mode 100644
index f1b51457f6d..00000000000
--- a/gpcontrib/pxf_fdw/pxf_fdw.h
+++ /dev/null
@@ -1,65 +0,0 @@
-/*
- * pxf_fdw.h
- * Foreign-data wrapper for PXF (Platform Extension Framework)
- *
- * IDENTIFICATION
- * contrib/pxf_fdw/pxf_fdw.h
- */
-
-#include "postgres.h"
-
-#include "access/formatter.h"
-#include "commands/copy.h"
-#include "nodes/pathnodes.h"
-#include "nodes/pg_list.h"
-#include "utils/rel.h"
-
-#ifndef PXF_FDW_H
-#define PXF_FDW_H
-
-#define PXF_FDW_DEFAULT_PROTOCOL "http"
-#define PXF_FDW_DEFAULT_HOST "localhost"
-#define PXF_FDW_DEFAULT_PORT 5888
-
-/*
- * Structure to store the PXF options */
-typedef struct PxfOptions
-{
- /* PXF service options */
- int pxf_port; /* port number for the PXF Service */
- char *pxf_host; /* hostname for the PXF Service */
- char *pxf_protocol; /* protocol for the PXF Service (i.e HTTP or
- * HTTPS) */
-
- /* Server doesn't come from options, it is the actual SERVER name */
- char *server; /* the name of the external server */
-
- /* Defined at options, but it is not visible to FDWs */
- char exec_location; /* execute on COORDINATOR, ANY or ALL SEGMENTS,
- * Cloudberry MPP specific */
-
- /* Single Row Error Handling */
- int reject_limit;
- bool is_reject_limit_rows;
- bool log_errors;
-
- /* FDW options */
- char *protocol; /* PXF protocol */
- char *resource; /* PXF resource */
- char *format; /* PXF resource format */
- char *profile; /* protocol[:format] */
-
- List *copy_options; /* merged options for COPY */
- List *options; /* merged options, excluding COPY, protocol,
- * resource, format, wire_format, pxf_port,
- * pxf_host, and pxf_protocol */
-} PxfOptions;
-
-/* Functions prototypes for pxf_option.c file */
-PxfOptions *PxfGetOptions(Oid foreigntableid);
-
-/* in pxf_deparse.c */
-extern void deparseTargetList(Relation rel, Bitmapset *attrs_used, List **retrieved_attrs);
-extern void classifyConditions(PlannerInfo *root, RelOptInfo *baserel, List *input_conds, List **remote_conds, List **local_conds);
-
-#endif /* _PXF_FDW_H_ */
diff --git a/gpcontrib/pxf_fdw/pxf_filter.c b/gpcontrib/pxf_fdw/pxf_filter.c
deleted file mode 100644
index 403bde8e355..00000000000
--- a/gpcontrib/pxf_fdw/pxf_filter.c
+++ /dev/null
@@ -1,1555 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership. The
- * ASF licenses this file to you under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- * implied. See the License for the specific language governing
- * permissions and limitations under the License.
- */
-
-/*
- * pxf_filter.c
- *
- * Functions for handling push down of supported scan level filters to PXF.
- */
-#include "pxf_filter.h"
-
-#include "catalog/pg_operator_d.h"
-#include "optimizer/clauses.h"
-#include "parser/parse_expr.h"
-#include "utils/builtins.h"
-#include "utils/guc.h"
-#include "utils/lsyscache.h"
-
-static List *PxfMakeExpressionItemsList(List *quals, Node *parent);
-static void PxfFreeFilter(PxfFilterDesc *filter);
-static char *PxfSerializeFilterList(List *filters);
-static bool OpExprToPxfFilter(OpExpr *expr, PxfFilterDesc *filter);
-static bool
- ScalarArrayOpExprToPxfFilter(ScalarArrayOpExpr *expr, PxfFilterDesc *filter);
-static bool VarToPxfFilter(Var *var, PxfFilterDesc *filter);
-static bool SupportedFilterType(Oid type);
-static bool SupportedOperatorTypeOpExpr(Oid type, PxfFilterDesc *filter);
-static bool SupportedOperatorTypeScalarArrayOpExpr(Oid type,
- PxfFilterDesc *filter,
- bool useOr);
-static void ScalarConstToStr(Const *constval, StringInfo buf);
-static void ListConstToStr(Const *constval, StringInfo buf);
-static List *AppendAttrFromVar(Var *var, List *attrs);
-static void AddExtraAndExpressionItems(List *expressionItems,
- int extraAndOperatorsNum);
-static List *GetAttrsFromExpr(Expr *expr, bool *expressionIsSupported);
-
-/*
- * All supported operators and their PXF operator codes.
- * Note that it is OK to use hardcoded OIDs, since these are all pinned
- * down system catalog operators.
- * See pg_operator.h
- */
-dbop_pxfop_map pxf_supported_opr_op_expr[] =
-{
- /* int2 */
- {Int2EqualOperator /* int2eq */ , PXFOP_EQ},
- {95 /* int2lt */ , PXFOP_LT},
- {520 /* int2gt */ , PXFOP_GT},
- {522 /* int2le */ , PXFOP_LE},
- {524 /* int2ge */ , PXFOP_GE},
- {519 /* int2ne */ , PXFOP_NE},
-
- /* int4 */
- {Int4EqualOperator /* int4eq */ , PXFOP_EQ},
- {97 /* int4lt */ , PXFOP_LT},
- {521 /* int4gt */ , PXFOP_GT},
- {523 /* int4le */ , PXFOP_LE},
- {525 /* int4ge */ , PXFOP_GE},
- {518 /* int4lt */ , PXFOP_NE},
-
- /* int8 */
- {Int8EqualOperator /* int8eq */ , PXFOP_EQ},
- {412 /* int8lt */ , PXFOP_LT},
- {413 /* int8gt */ , PXFOP_GT},
- {414 /* int8le */ , PXFOP_LE},
- {415 /* int8ge */ , PXFOP_GE},
- {411 /* int8lt */ , PXFOP_NE},
-
- /* text */
- {TextEqualOperator /* texteq */ , PXFOP_EQ},
- {664 /* text_lt */ , PXFOP_LT},
- {666 /* text_gt */ , PXFOP_GT},
- {665 /* text_le */ , PXFOP_LE},
- {667 /* text_ge */ , PXFOP_GE},
- {531 /* textlt */ , PXFOP_NE},
- {1209 /* textlike */ , PXFOP_LIKE},
-
- /* int2 to int4 */
- {Int24EqualOperator /* int24eq */ , PXFOP_EQ},
- {534 /* int24lt */ , PXFOP_LT},
- {536 /* int24gt */ , PXFOP_GT},
- {540 /* int24le */ , PXFOP_LE},
- {542 /* int24ge */ , PXFOP_GE},
- {538 /* int24ne */ , PXFOP_NE},
-
- /* int4 to int2 */
- {Int42EqualOperator /* int42eq */ , PXFOP_EQ},
- {535 /* int42lt */ , PXFOP_LT},
- {537 /* int42gt */ , PXFOP_GT},
- {541 /* int42le */ , PXFOP_LE},
- {543 /* int42ge */ , PXFOP_GE},
- {539 /* int42ne */ , PXFOP_NE},
-
- /* int8 to int4 */
- {Int84EqualOperator /* int84eq */ , PXFOP_EQ},
- {418 /* int84lt */ , PXFOP_LT},
- {419 /* int84gt */ , PXFOP_GT},
- {420 /* int84le */ , PXFOP_LE},
- {430 /* int84ge */ , PXFOP_GE},
- {417 /* int84ne */ , PXFOP_NE},
-
- /* int4 to int8 */
- {Int48EqualOperator /* int48eq */ , PXFOP_EQ},
- {37 /* int48lt */ , PXFOP_LT},
- {76 /* int48gt */ , PXFOP_GT},
- {80 /* int48le */ , PXFOP_LE},
- {82 /* int48ge */ , PXFOP_GE},
- {36 /* int48ne */ , PXFOP_NE},
-
- /* int2 to int8 */
- {Int28EqualOperator /* int28eq */ , PXFOP_EQ},
- {1864 /* int28lt */ , PXFOP_LT},
- {1865 /* int28gt */ , PXFOP_GT},
- {1866 /* int28le */ , PXFOP_LE},
- {1867 /* int28ge */ , PXFOP_GE},
- {1863 /* int28ne */ , PXFOP_NE},
-
- /* int8 to int2 */
- {Int82EqualOperator /* int82eq */ , PXFOP_EQ},
- {1870 /* int82lt */ , PXFOP_LT},
- {1871 /* int82gt */ , PXFOP_GT},
- {1872 /* int82le */ , PXFOP_LE},
- {1873 /* int82ge */ , PXFOP_GE},
- {1869 /* int82ne */ , PXFOP_NE},
-
- /* date */
- {DateEqualOperator /* date_eq */ , PXFOP_EQ},
- {1095 /* date_lt */ , PXFOP_LT},
- {1097 /* date_gt */ , PXFOP_GT},
- {1096 /* date_le */ , PXFOP_LE},
- {1098 /* date_ge */ , PXFOP_GE},
- {1094 /* date_ne */ , PXFOP_NE},
-
- /* timestamp */
- {TimestampEqualOperator /* timestamp_eq */ , PXFOP_EQ},
- {2062 /* timestamp_lt */ , PXFOP_LT},
- {2064 /* timestamp_gt */ , PXFOP_GT},
- {2063 /* timestamp_le */ , PXFOP_LE},
- {2065 /* timestamp_ge */ , PXFOP_GE},
- {2061 /* timestamp_ne */ , PXFOP_NE},
-
- /* float8 */
- {Float8EqualOperator /* float8eq */ , PXFOP_EQ},
- {672 /* float8lt */ , PXFOP_LT},
- {674 /* float8gt */ , PXFOP_GT},
- {673 /* float8le */ , PXFOP_LE},
- {675 /* float8ge */ , PXFOP_GE},
- {671 /* float8ne */ , PXFOP_NE},
-
- /* float48 */
- {1120 /* float48eq */ , PXFOP_EQ},
- {1122 /* float48lt */ , PXFOP_LT},
- {1123 /* float48gt */ , PXFOP_GT},
- {1124 /* float48le */ , PXFOP_LE},
- {1125 /* float48ge */ , PXFOP_GE},
- {1121 /* float48ne */ , PXFOP_NE},
-
- /* boolean */
- {BooleanEqualOperator /* booleq */ , PXFOP_EQ},
- {58 /* boollt */ , PXFOP_LT},
- {59 /* boolgt */ , PXFOP_GT},
- {1694 /* boolle */ , PXFOP_LE},
- {1695 /* boolge */ , PXFOP_GE},
- {85 /* boolne */ , PXFOP_NE},
-
- /* bpchar */
- {BpcharEqualOperator /* bpchareq */ , PXFOP_EQ},
- {1058 /* bpcharlt */ , PXFOP_LT},
- {1060 /* bpchargt */ , PXFOP_GT},
- {1059 /* bpcharle */ , PXFOP_LE},
- {1061 /* bpcharge */ , PXFOP_GE},
- {1057 /* bpcharne */ , PXFOP_NE},
-};
-
-dbop_pxfop_array_map pxf_supported_opr_scalar_array_op_expr[] =
-{
- /* int2 */
- {Int2EqualOperator /* int2eq */ , PXFOP_IN, true},
-
- /* int4 */
- {Int4EqualOperator /* int4eq */ , PXFOP_IN, true},
-
- /* int8 */
- {Int8EqualOperator /* int8eq */ , PXFOP_IN, true},
-
- /* text */
- {TextEqualOperator /* texteq */ , PXFOP_IN, true},
-
- /* int2 to int4 */
- {Int24EqualOperator /* int24eq */ , PXFOP_IN,
- true},
-
- /* int4 to int2 */
- {Int42EqualOperator /* int42eq */ , PXFOP_IN,
- true},
-
- /* int8 to int4 */
- {Int84EqualOperator /* int84eq */ , PXFOP_IN,
- true},
-
- /* int4 to int8 */
- {Int48EqualOperator /* int48eq */ , PXFOP_IN,
- true},
-
- /* int2 to int8 */
- {Int28EqualOperator /* int28eq */ , PXFOP_IN,
- true},
-
- /* int8 to int2 */
- {Int82EqualOperator /* int82eq */ , PXFOP_IN,
- true},
-
- /* date */
- {DateEqualOperator /* date_eq */ , PXFOP_IN, true},
-
- /* timestamp */
- {TimestampEqualOperator /* timestamp_eq */ ,
- PXFOP_IN, true},
-
- /* float8 */
- {Float8EqualOperator /* float8eq */ , PXFOP_IN,
- true},
-
- /* float48 */
- {1120 /* float48eq */ , PXFOP_IN, true},
-
- /* bpchar */
- {BpcharEqualOperator /* bpchareq */ , PXFOP_IN,
- true},
-};
-
-Oid pxf_supported_types[] =
-{
- INT2OID,
- INT4OID,
- INT8OID,
- FLOAT4OID,
- FLOAT8OID,
- NUMERICOID,
- BOOLOID,
- TEXTOID,
- VARCHAROID,
- BPCHAROID,
- CHAROID,
- DATEOID,
- TIMESTAMPOID,
- /* complex datatypes */
- INT2ARRAYOID,
- INT4ARRAYOID,
- INT8ARRAYOID,
- TEXTARRAYOID
-};
-
-static void
-PxfFreeExpressionItemsList(List *expressionItems)
-{
- ExpressionItem *expressionItem = NULL;
-
- while (list_length(expressionItems) > 0)
- {
- expressionItem = (ExpressionItem *) linitial(expressionItems);
- pfree(expressionItem);
-
- /*
- * to avoid freeing already freed items - delete all occurrences of
- * current expression
- */
- int previousLength = expressionItems->length + 1;
-
- while (expressionItems != NULL &&
- previousLength > expressionItems->length)
- {
- previousLength = expressionItems->length;
- expressionItems = list_delete_ptr(expressionItems, expressionItem);
- }
- }
-}
-
-/*
- * PxfMakeExpressionItemsList
- *
- * Given a scan node qual list, find the filters that are eligible to be used
- * by PXF, construct an expressions list, which consists of OpExpr or BoolExpr nodes
- * and return it to the caller.
- *
- * Basically this function just transforms expression tree to Reversed Polish Notation list.
- *
- *
- */
-static List *
-PxfMakeExpressionItemsList(List *quals, Node *parent)
-{
- ExpressionItem *expressionItem = NULL;
- List *result = NIL;
- ListCell *lc = NULL;
- ListCell *ilc = NULL;
-
- int quals_size = list_length(quals);
-
- if (quals_size == 0)
- return NIL;
-
- foreach(lc, quals)
- {
- expressionItem = (ExpressionItem *) palloc0(sizeof(ExpressionItem));
- Node *node = (Node *) lfirst(lc);
-
- expressionItem->node = node;
- expressionItem->parent = parent;
- expressionItem->processed = false;
-
- NodeTag tag = nodeTag(node);
-
- elog(DEBUG1, "PxfMakeExpressionItemsList: found NodeTag %d", tag);
- switch (tag)
- {
- case T_Var:
- /* IN(single_value) */
- case T_OpExpr:
- /* Comparison operators >,>=,=,etc */
- case T_ScalarArrayOpExpr:
- case T_NullTest:
- {
- result = lappend(result, expressionItem);
- break;
- }
- case T_BoolExpr:
- /* Logical operators AND, OR, NOT */
- {
- BoolExpr *expr = (BoolExpr *) node;
-
- elog(DEBUG1,
- "PxfMakeExpressionItemsList: found T_BoolExpr; make recursive call");
- List *inner_result =
- PxfMakeExpressionItemsList(expr->args, node);
-
- elog(DEBUG1,
- "PxfMakeExpressionItemsList: recursive call end");
-
- result = list_concat(result, inner_result);
-
- int childNodesNum = 0;
-
- /* Find number of child nodes on first level */
- foreach(ilc, inner_result)
- {
- ExpressionItem *ei = (ExpressionItem *) lfirst(ilc);
-
- if (!ei->processed && ei->parent == node)
- {
- ei->processed = true;
- childNodesNum++;
- }
- }
-
- if (expr->boolop == NOT_EXPR)
- {
- for (int i = 0; i < childNodesNum; i++)
- {
- result = lappend(result, expressionItem);
- }
- }
- else if (expr->boolop == AND_EXPR || expr->boolop == OR_EXPR)
- {
- for (int i = 0; i < childNodesNum - 1; i++)
- {
- result = lappend(result, expressionItem);
- }
- }
- else
- {
- elog(ERROR,
- "internal error in pxffilters.c:PxfMakeExpressionItemsList. "
- "Found unknown boolean expression type");
- }
- break;
- }
- default:
- elog(DEBUG1,
- "PxfMakeExpressionItemsList: unsupported node tag %d",
- tag);
- break;
- }
- }
- if (quals_size > 1 && parent == NULL)
- {
- /*
- * Planner (but not ORCA) will omit AND operators at the root level,
- * so if we find more than 1 qualifier,
- */
- /*
- * it means we are looking at 2 or more expressions that are
- * implicitly AND-ed by the planner.
- */
- /*
- * Here, to make it explicit, we will need to add additional AND
- * operators to compensate for the missing ones.
- */
- AddExtraAndExpressionItems(result, quals_size - 1);
- }
-
- return result;
-}
-
-static void
-PxfFreeFilter(PxfFilterDesc *filter)
-{
- if (!filter)
- return;
-
- if (filter->l.conststr)
- {
- if (filter->l.conststr->data)
- pfree(filter->l.conststr->data);
- pfree(filter->l.conststr);
- }
- if (filter->r.conststr)
- {
- if (filter->r.conststr->data)
- pfree(filter->r.conststr->data);
- pfree(filter->r.conststr);
- }
-
- pfree(filter);
-}
-
-/*
-* PxfSerializeFilterList
-*
-* Takes expression items list in RPN notation, produce a
-* serialized string representation in order to communicate this list
-* over the wire.
-*
-* The serialized string is in a RPN (Reversed Polish Notation) format
-* as flattened tree. Operands and operators are represented with their
-* respective codes. Each filter is serialized as follows:
-*
-* <attcode><attnum><constcode><constval><constsizecode><constsize><constdata><constvalue><opercode><opernum>
-*
-* Example filter list:
-*
-* Column(0) > 1 AND Column(0) < 5 AND Column(2) == "third"
-*
-* Yields the following serialized string:
-*
-* a0c23s1d1o2a1c23s1d5o1a2c25s5dthirdo5l0l0
-*
-* Where:
-*
-* a0 - first column of table
-* c23 - scalar constant with type oid 23(INT4)
-* s1 - size of constant in bytes
-* d1 - serialized constant value
-* o2 - greater than operation
-* a1 - second column of table
-* c23 - scalar constant with type oid 23(INT4)
-* s1 - size of constant in bytes
-* d5 - serialized constant value
-* o1 - less than operation
-* a2 - third column of table
-* c25 - scalar constant with type oid 25(TEXT)
-* s5 - size of constant in bytes
-* dthird - serialized constant value
-* o5 - equals operation
-* l0 - AND operator
-* l0 - AND operator
-*
-*/
-static char *
-PxfSerializeFilterList(List *filters)
-{
- StringInfo resbuf;
- ListCell *lc = NULL;
-
- if (list_length(filters) == 0)
- return NULL;
-
- resbuf = makeStringInfo();
-
- /*
- * Iterate through the expression items in the list and serialize them one
- * after the other.
- */
- foreach(lc, filters)
- {
- ExpressionItem *expressionItem = (ExpressionItem *) lfirst(lc);
- Node *node = expressionItem->node;
- NodeTag tag = nodeTag(node);
-
- switch (tag)
- {
- case T_Var:
- {
- elog(DEBUG1,
- "PxfSerializeFilterList: node tag %d (T_Var)",
- tag);
- PxfFilterDesc
- *filter = (PxfFilterDesc *) palloc0(sizeof(PxfFilterDesc));
- Var *var = (Var *) node;
-
- if (VarToPxfFilter(var, filter))
- {
- PxfOperand l = filter->l;
- PxfOperand r = filter->r;
- PxfOperatorCode o = filter->op;
-
- if (pxfoperand_is_attr(l) && pxfoperand_is_scalar_const(r))
- {
- appendStringInfo(resbuf,
- "%c%d%c%d%c%lu%c%s",
- PXF_ATTR_CODE,
- l.attnum - 1, /* Java attrs are
- * 0-based */
- PXF_SCALAR_CONST_CODE,
- r.consttype,
- PXF_SIZE_BYTES,
- strlen(r.conststr->data),
- PXF_CONST_DATA,
- (r.conststr)->data);
- }
- else
- {
- /*
- * VarToPxfFilter() should have never let this
- * happen
- */
- elog(ERROR,
- "internal error in pxf_filter.c:PxfSerializeFilterList"
- ". Found a non const+attr filter");
- }
- appendStringInfo(resbuf, "%c%d", PXF_OPERATOR_CODE, o);
- PxfFreeFilter(filter);
-
- }
- else
- {
- /*
- * if at least one expression item is not supported,
- * whole filter doesn't make sense
- */
- elog(DEBUG1,
- "Query will not be optimized to use filter push-down.");
- pfree(filter);
- pfree(resbuf->data);
- return NULL;
- }
- break;
- }
- case T_OpExpr:
- {
- elog(DEBUG1,
- "PxfSerializeFilterList: node tag %d (T_OpExpr)",
- tag);
- PxfFilterDesc
- *filter =
- (PxfFilterDesc *) palloc0(sizeof(PxfFilterDesc));
- OpExpr *expr = (OpExpr *) node;
-
- if (OpExprToPxfFilter(expr, filter))
- {
- PxfOperand l = filter->l;
- PxfOperand r = filter->r;
- PxfOperatorCode o = filter->op;
-
- if (pxfoperand_is_attr(l) && pxfoperand_is_scalar_const(r))
- {
- appendStringInfo(resbuf,
- "%c%d%c%d%c%lu%c%s",
- PXF_ATTR_CODE,
- l.attnum - 1, /* Java attrs are
- * 0-based */
- PXF_SCALAR_CONST_CODE,
- r.consttype,
- PXF_SIZE_BYTES,
- strlen(r.conststr->data),
- PXF_CONST_DATA,
- (r.conststr)->data);
- }
- else if (pxfoperand_is_scalar_const(l) &&
- pxfoperand_is_attr(r))
- {
- appendStringInfo(resbuf,
- "%c%d%c%lu%c%s%c%d",
- PXF_SCALAR_CONST_CODE,
- l.consttype,
- PXF_SIZE_BYTES,
- strlen(l.conststr->data),
- PXF_CONST_DATA,
- (l.conststr)->data,
- PXF_ATTR_CODE,
- r.attnum - 1); /* Java attrs are
- * 0-based */
- }
- else
- {
- /*
- * OpExprToPxfFilter() should have never let this
- * happen
- */
- elog(ERROR,
- "internal error in pxf_filter.c:PxfSerializeFilterList"
- ". Found a non const+attr filter");
- }
- appendStringInfo(resbuf, "%c%d", PXF_OPERATOR_CODE, o);
- PxfFreeFilter(filter);
- }
- else
- {
- /*
- * if at least one expression item is not supported,
- * whole filter doesn't make sense
- */
- elog(DEBUG1,
- "Query will not be optimized to use filter push-down.");
- pfree(filter);
- pfree(resbuf->data);
- return NULL;
- }
- break;
- }
- case T_ScalarArrayOpExpr:
- {
- elog(DEBUG1,
- "PxfSerializeFilterList: node tag %d (T_ScalarArrayOpExpr)",
- tag);
- ScalarArrayOpExpr *expr = (ScalarArrayOpExpr *) node;
- PxfFilterDesc *filter =
- (PxfFilterDesc *) palloc0(sizeof(PxfFilterDesc));
-
- if (ScalarArrayOpExprToPxfFilter(expr, filter))
- {
- PxfOperand l = filter->l;
- PxfOperand r = filter->r;
- PxfOperatorCode o = filter->op;
-
- if (pxfoperand_is_attr(l) && pxfoperand_is_list_const(r))
- {
- appendStringInfo(resbuf, "%c%d%c%d%s",
- PXF_ATTR_CODE, l.attnum - 1, /* Java attrs are
- * 0-based */
- PXF_LIST_CONST_CODE, r.consttype,
- r.conststr->data);
- }
- else if (pxfoperand_is_list_const(l) &&
- pxfoperand_is_attr(r))
- {
- appendStringInfo(resbuf, "%c%d%s%c%d",
- PXF_SCALAR_CONST_CODE, l.consttype,
- l.conststr->data,
- PXF_ATTR_CODE, r.attnum - 1); /* Java attrs are
- * 0-based */
- }
- else
- {
- /*
- * scalararrayopexpr_to_pxffilter() should have
- * never let this happen
- */
- elog(ERROR,
- "internal error in pxf_filter.c:PxfSerializeFilterList"
- ". Found a non const+attr filter");
- }
- appendStringInfo(resbuf, "%c%d", PXF_OPERATOR_CODE, o);
- PxfFreeFilter(filter);
- }
- else
- {
- /*
- * if at least one expression item is not supported,
- * whole filter doesn't make sense
- */
- elog(DEBUG1,
- "Query will not be optimized to use filter push-down.");
- pfree(filter);
- pfree(resbuf->data);
- return NULL;
- }
- break;
- }
- case T_BoolExpr:
- {
- BoolExpr *expr = (BoolExpr *) node;
- BoolExprType boolType = expr->boolop;
-
- elog(DEBUG1,
- "PxfSerializeFilterList: node tag %d (T_BoolExpr), bool node type %d",
- tag,
- boolType);
- appendStringInfo(resbuf,
- "%c%d",
- PXF_LOGICAL_OPERATOR_CODE,
- boolType);
- break;
- }
- case T_NullTest:
- {
- elog(DEBUG1,
- "PxfSerializeFilterList: node tag %d (T_NullTest)",
- tag);
- NullTest *expr = (NullTest *) node;
- Var *var = (Var *) expr->arg;
-
- /* TODO: add check for supported operation */
- if (!SupportedFilterType(var->vartype))
- {
- elog(DEBUG1,
- "Query will not be optimized to use filter push-down.");
- return NULL;
- }
-
- /*
- * filter expression for T_NullTest will not have any
- * constant value
- */
- if (expr->nulltesttype == IS_NULL)
- {
- appendStringInfo(resbuf,
- "%c%d%c%d",
- PXF_ATTR_CODE,
- var->varattno - 1,
- PXF_OPERATOR_CODE,
- PXFOP_IS_NULL);
- }
- else if (expr->nulltesttype == IS_NOT_NULL)
- {
- appendStringInfo(resbuf,
- "%c%d%c%d",
- PXF_ATTR_CODE,
- var->varattno - 1,
- PXF_OPERATOR_CODE,
- PXFOP_IS_NOTNULL);
- }
- else
- {
- elog(ERROR,
- "internal error in pxf_filter.c:PxfSerializeFilterList.:"
- " Found a NullTest filter with incorrect NullTestType");
- }
- break;
- }
- default:
- {
- elog(DEBUG5, "Skipping tag: %d", tag);
- }
- }
- }
-
- if (resbuf->len == 0)
- {
- pfree(resbuf->data);
- return NULL;
- }
-
- return resbuf->data;
-}
-
-/*
-* OpExprToPxfFilter
-*
-* check if an OpExpr qualifies to be pushed-down to PXF.
-* if it is - create it and return a success code.
-*/
-static bool
-OpExprToPxfFilter(OpExpr *expr, PxfFilterDesc *filter)
-{
- Node *leftop = NULL;
- Node *rightop = NULL;
- Oid rightop_type = InvalidOid;
- Oid leftop_type = InvalidOid;
-
- if ((!expr) || (!filter))
- return false;
-
- leftop = get_leftop((Expr *) expr);
- rightop = get_rightop((Expr *) expr);
- leftop_type = exprType(leftop);
- rightop_type = exprType(rightop);
-
- /* only binary oprs supported currently */
- if (!rightop)
- {
- elog(DEBUG1, "OpExprToPxfFilter: unary op! leftop_type: %d, op: %d",
- leftop_type, expr->opno);
- return false;
- }
-
- elog(DEBUG1, "OpExprToPxfFilter: leftop (expr type: %d, arg type: %d), "
- "rightop_type (expr type: %d, arg type %d), op: %d",
- leftop_type, nodeTag(leftop),
- rightop_type, nodeTag(rightop),
- expr->opno);
-
- /*
- * check if supported type -
- */
- if (!SupportedFilterType(rightop_type) || !SupportedFilterType(leftop_type))
- return false;
-
- /*
- * check if supported operator -
- */
- if (!SupportedOperatorTypeOpExpr(expr->opno, filter))
- return false;
-
- /* arguments must be VAR and CONST */
- if (IsA(leftop, Var) &&IsA(rightop, Const))
- {
- filter->l.opcode = PXF_ATTR_CODE;
- filter->l.attnum = ((Var *) leftop)->varattno;
- filter->l.consttype = InvalidOid;
- if (filter->l.attnum <= InvalidAttrNumber)
- return false; /* system attr not supported */
-
- filter->r.opcode = PXF_SCALAR_CONST_CODE;
- filter->r.attnum = InvalidAttrNumber;
- filter->r.conststr = makeStringInfo();
- ScalarConstToStr((Const *) rightop, filter->r.conststr);
- filter->r.consttype = ((Const *) rightop)->consttype;
- }
- else if (IsA(leftop, Const) &&IsA(rightop, Var))
- {
- filter->l.opcode = PXF_SCALAR_CONST_CODE;
- filter->l.attnum = InvalidAttrNumber;
- filter->l.conststr = makeStringInfo();
- ScalarConstToStr((Const *) leftop, filter->l.conststr);
- filter->l.consttype = ((Const *) leftop)->consttype;
-
- filter->r.opcode = PXF_ATTR_CODE;
- filter->r.attnum = ((Var *) rightop)->varattno;
- filter->r.consttype = InvalidOid;
- if (filter->r.attnum <= InvalidAttrNumber)
- return false; /* system attr not supported */
- }
- else
- {
- elog(DEBUG1, "OpExprToPxfFilter: expression is not a Var+Const");
- return false;
- }
-
- return true;
-}
-
-static bool
-ScalarArrayOpExprToPxfFilter(ScalarArrayOpExpr *expr, PxfFilterDesc *filter)
-{
-
- Node *leftop = NULL;
- Node *rightop = NULL;
-
- leftop = (Node *) linitial(expr->args);
- rightop = (Node *) lsecond(expr->args);
- Oid leftop_type = exprType(leftop);
- Oid rightop_type = exprType(rightop);
-
- /*
- * check if supported type -
- */
- if (!SupportedFilterType(rightop_type) || !SupportedFilterType(leftop_type))
- return false;
-
- /*
- * check if supported operator -
- */
- if (!SupportedOperatorTypeScalarArrayOpExpr(expr->opno,
- filter,
- expr->useOr))
- return false;
-
- if (IsA(leftop, Var) &&IsA(rightop, Const))
- {
- filter->l.opcode = PXF_ATTR_CODE;
- filter->l.attnum = ((Var *) leftop)->varattno;
- filter->l.consttype = InvalidOid;
- if (filter->l.attnum <= InvalidAttrNumber)
- return false; /* system attr not supported */
-
- filter->r.opcode = PXF_LIST_CONST_CODE;
- filter->r.attnum = InvalidAttrNumber;
- filter->r.conststr = makeStringInfo();
- ListConstToStr((Const *) rightop, filter->r.conststr);
- filter->r.consttype = ((Const *) rightop)->consttype;
- }
- else if (IsA(leftop, Const) &&IsA(rightop, Var))
- {
- filter->l.opcode = PXF_LIST_CONST_CODE;
- filter->l.attnum = InvalidAttrNumber;
- filter->l.conststr = makeStringInfo();
- ListConstToStr((Const *) leftop, filter->l.conststr);
- filter->l.consttype = ((Const *) leftop)->consttype;
-
- filter->r.opcode = PXF_ATTR_CODE;
- filter->r.attnum = ((Var *) rightop)->varattno;
- filter->r.consttype = InvalidOid;
- if (filter->r.attnum <= InvalidAttrNumber)
- return false; /* system attr not supported */
- }
- else
- {
- elog(DEBUG1,
- "PxfSerializeFilterList: expression is not a Var+Const");
- return false;
- }
-
- return true;
-}
-
-static bool
-VarToPxfFilter(Var *var, PxfFilterDesc *filter)
-{
- Oid var_type = InvalidOid;
-
- if ((!var) || (!filter))
- return false;
-
- var_type = exprType((Node *) var);
-
- /*
- * check if supported type -
- */
- if (!SupportedFilterType(var_type))
- return false;
-
- /* arguments must be VAR and CONST */
- if (IsA(var, Var))
- {
- filter->l.opcode = PXF_ATTR_CODE;
- filter->l.attnum = var->varattno;
- filter->l.consttype = InvalidOid;
- if (filter->l.attnum <= InvalidAttrNumber)
- return false; /* system attr not supported */
-
- filter->r.opcode = PXF_SCALAR_CONST_CODE;
- filter->r.attnum = InvalidAttrNumber;
- filter->r.conststr = makeStringInfo();
- appendStringInfo(filter->r.conststr, TrueConstValue);
- filter->r.consttype = BOOLOID;
- }
- else
- {
- elog(DEBUG1, "VarToPxfFilter: expression is not a Var");
- return false;
- }
-
- return true;
-}
-
-static List *
-AppendAttrFromVar(Var *var, List *attrs)
-{
- AttrNumber varattno = var->varattno;
-
- /* system attr not supported */
- if (varattno > InvalidAttrNumber)
- return lappend_int(attrs, varattno - 1);
-
- return attrs;
-}
-
-/*
-* append_attr_from_func_args
-*
-* extracts all columns from FuncExpr into attrs
-* assigns false to expressionIsSupported if at least one of items is not supported
-*/
-static List *
-append_attr_from_func_args(FuncExpr *expr,
- List *attrs,
- bool *expressionIsSupported)
-{
- if (!expressionIsSupported)
- {
- return NIL;
- }
- ListCell *lc = NULL;
-
- foreach(lc, expr->args)
- {
- Node *node = (Node *) lfirst(lc);
-
- if (IsA(node, FuncExpr))
- {
- attrs = append_attr_from_func_args((FuncExpr *) node,
- attrs,
- expressionIsSupported);
- }
- else if (IsA(node, Var))
- {
- attrs = AppendAttrFromVar((Var *) node, attrs);
- }
- else if (IsA(node, OpExpr))
- {
- attrs = GetAttrsFromExpr((Expr *) node, expressionIsSupported);
- }
- else
- {
- *expressionIsSupported = false;
- return NIL;
- }
- }
-
- return attrs;
-
-}
-
-/*
-* GetAttrsFromExpr
-*
-* extracts and returns list of all columns from Expr
-* assigns false to expressionIsSupported if at least one of items is not supported
-*/
-static List *
-GetAttrsFromExpr(Expr *expr, bool *expressionIsSupported)
-{
- Node *leftop = NULL;
- Node *rightop = NULL;
- List *attrs = NIL;
-
- *expressionIsSupported = true;
-
- if ((!expr))
- return attrs;
-
- if (IsA(expr, OpExpr))
- {
- leftop = get_leftop(expr);
- rightop = get_rightop(expr);
- }
- else if (IsA(expr, ScalarArrayOpExpr))
- {
- ScalarArrayOpExpr *saop = (ScalarArrayOpExpr *) expr;
-
- leftop = (Node *) linitial(saop->args);
- rightop = (Node *) lsecond(saop->args);
- }
- else
- {
- /* If expression type is not known, report that it's not supported */
- *expressionIsSupported = false;
- return NIL;
- }
-
- /* We support following combinations of operands: */
- /* Var, Const */
- /* Relabel, Const */
- /* FuncExpr, Const */
- /* Const, Var */
- /* Const, Relabel */
- /* Const, FuncExpr */
- /* For most of datatypes column is represented by Var node */
- /* For varchar column is represented by RelabelType node */
- if (IsA(leftop, Var) &&IsA(rightop, Const))
- {
- attrs = AppendAttrFromVar((Var *) leftop, attrs);
- }
- else if (IsA(leftop, RelabelType) &&IsA(rightop, Const))
- {
- attrs = AppendAttrFromVar((Var *) ((RelabelType *) leftop)->arg, attrs);
- }
- else if (IsA(leftop, FuncExpr) &&IsA(rightop, Const))
- {
- FuncExpr *expr = (FuncExpr *) leftop;
-
- attrs = append_attr_from_func_args(expr, attrs, expressionIsSupported);
- }
- else if (IsA(rightop, Var) &&IsA(leftop, Const))
- {
- attrs = AppendAttrFromVar((Var *) rightop, attrs);
- }
- else if (IsA(rightop, RelabelType) &&IsA(leftop, Const))
- {
- attrs =
- AppendAttrFromVar((Var *) ((RelabelType *) rightop)->arg, attrs);
- }
- else if (IsA(rightop, FuncExpr) &&IsA(leftop, Const))
- {
- FuncExpr *expr = (FuncExpr *) rightop;
-
- attrs = append_attr_from_func_args(expr, attrs, expressionIsSupported);
- }
- else
- {
- /*
- * If operand type or combination is not known, report that it's not
- * supported
- */
- /* to avoid partially extracted attributes from expression */
- *expressionIsSupported = false;
- return NIL;
- }
-
- return attrs;
-
-}
-
-/*
-* SupportedFilterType
-*
-* Return true if the type is supported by pxf_filter.
-* Supported defines are defined in pxf_supported_types.
-*/
-static bool
-SupportedFilterType(Oid type)
-{
- int nargs = sizeof(pxf_supported_types) / sizeof(Oid);
- int i;
-
- /* is type supported? */
- for (i = 0; i < nargs; i++)
- {
- if (type == pxf_supported_types[i])
- return true;
- }
-
- elog(DEBUG1,
- "SupportedFilterType: filter pushdown is not supported for datatype oid: %d",
- type);
-
- return false;
-}
-
-static bool
-SupportedOperatorTypeOpExpr(Oid type, PxfFilterDesc *filter)
-{
-
- int nargs = sizeof(pxf_supported_opr_op_expr) / sizeof(dbop_pxfop_map);
- int i;
-
- /* is operator supported? if so, set the corresponding PXFOP */
- for (i = 0; i < nargs; i++)
- {
- /* NOTE: switch to hash table lookup if */
- /* array grows. for now it's cheap enough */
- if (type == pxf_supported_opr_op_expr[i].dbop)
- {
- filter->op = pxf_supported_opr_op_expr[i].pxfop;
- return true; /* filter qualifies! */
- }
- }
-
- elog(DEBUG1,
- "OpExprToPxfFilter: operator is not supported, operator code: %d",
- type);
-
- return false;
-}
-
-static bool
-SupportedOperatorTypeScalarArrayOpExpr(Oid type,
- PxfFilterDesc *filter,
- bool useOr)
-{
-
- int nargs = sizeof(pxf_supported_opr_scalar_array_op_expr) /
- sizeof(dbop_pxfop_array_map);
- int i;
-
- /* is operator supported? if so, set the corresponding PXFOP */
- for (i = 0; i < nargs; i++)
- {
- /* NOTE: switch to hash table lookup if */
- /* array grows. for now it's cheap enough */
- if (useOr == pxf_supported_opr_scalar_array_op_expr[i].useOr &&
- type == pxf_supported_opr_scalar_array_op_expr[i].dbop)
- {
- filter->op = pxf_supported_opr_scalar_array_op_expr[i].pxfop;
- return true; /* filter qualifies! */
- }
- }
-
- elog(DEBUG1,
- "SupportedOperatorTypeScalarArrayOpExpr: operator is not supported, operator code: %d",
- type);
-
- return false;
-}
-
-/*
-* const_to_str
-*
-* Extract the value stored in a const operand into a string. If the operand
-* type is text based, make sure to escape the value with surrounding quotes.
-*/
-static void
-ScalarConstToStr(Const *constval, StringInfo buf)
-{
- Oid typoutput;
- bool typIsVarlena;
- char *extval;
-
- if (constval->constisnull)
- {
- /* TODO: test this edge case and its consequences */
- appendStringInfo(buf, NullConstValue);
- return;
- }
-
- getTypeOutputInfo(constval->consttype,
- &typoutput, &typIsVarlena);
-
- extval = OidOutputFunctionCall(typoutput, constval->constvalue);
-
- switch (constval->consttype)
- {
- case INT2OID:
- case INT4OID:
- case INT8OID:
- case FLOAT4OID:
- case FLOAT8OID:
- case NUMERICOID:
- case TEXTOID:
- case VARCHAROID:
- case BPCHAROID:
- case CHAROID:
- case BYTEAOID:
- case DATEOID:
- case TIMESTAMPOID:
- appendStringInfo(buf, "%s", extval);
- break;
- default:
- /* should never happen. we filter on types earlier */
- elog(ERROR,
- "internal error in pxf_filter.c:ScalarConstToStr. "
- "Using unsupported data type (%d) (value %s)",
- constval->consttype, extval);
-
- }
-}
-
-/*
-* ListConstToStr
-*
-* Extracts the value stored in a list constant to a string.
-* Currently supported data types: int2[], int4[], int8[], text[]
-* Example:
-* Input: ['abc', 'xyz']
-* Output: s3dabcs3dxyz
-*
-*/
-static void
-ListConstToStr(Const *constval, StringInfo buf)
-{
- StringInfo interm_buf;
- Datum *dats;
- ArrayType *arr;
- int len;
-
- if (constval->constisnull)
- {
- elog(DEBUG1, "Null constant is not expected in this context.");
- return;
- }
-
- if (constval->constbyval)
- {
- elog(DEBUG1,
- "Constant passed by value is not expected in this context.");
- return;
- }
-
- arr = DatumGetArrayTypeP(constval->constvalue);
-
- interm_buf = makeStringInfo();
-
- switch (constval->consttype)
- {
- case INT2ARRAYOID:
- {
- int16 value;
-
- deconstruct_array(arr,
- INT2OID,
- sizeof(value),
- true,
- 's',
- &dats,
- NULL,
- &len);
-
- for (int i = 0; i < len; i++)
- {
- value = DatumGetInt16(dats[i]);
-
- appendStringInfo(interm_buf, "%hd", value);
-
- appendStringInfo(buf, "%c%d%c%s",
- PXF_SIZE_BYTES, interm_buf->len,
- PXF_CONST_DATA, interm_buf->data);
- resetStringInfo(interm_buf);
- }
- break;
- }
- case INT4ARRAYOID:
- {
- int32 value;
-
- deconstruct_array(arr,
- INT4OID,
- sizeof(value),
- true,
- 'i',
- &dats,
- NULL,
- &len);
-
- for (int i = 0; i < len; i++)
- {
- value = DatumGetInt32(dats[i]);
-
- appendStringInfo(interm_buf, "%d", value);
-
- appendStringInfo(buf, "%c%d%c%s",
- PXF_SIZE_BYTES, interm_buf->len,
- PXF_CONST_DATA, interm_buf->data);
- resetStringInfo(interm_buf);
- }
- break;
- }
- case INT8ARRAYOID:
- {
- int64 value;
-
- deconstruct_array(arr,
- INT8OID,
- sizeof(value),
- true,
- 'd',
- &dats,
- NULL,
- &len);
-
- for (int i = 0; i < len; i++)
- {
- value = DatumGetInt64(dats[i]);
-
- appendStringInfo(interm_buf, "%ld", value);
-
- appendStringInfo(buf, "%c%d%c%s",
- PXF_SIZE_BYTES, interm_buf->len,
- PXF_CONST_DATA, interm_buf->data);
- resetStringInfo(interm_buf);
- }
- break;
- }
- case TEXTARRAYOID:
- {
- char *value;
-
- deconstruct_array(arr, TEXTOID, -1, false, 'i', &dats, NULL, &len);
-
- for (int i = 0; i < len; i++)
- {
- value = DatumGetCString(DirectFunctionCall1(textout, dats[i]));
-
- appendStringInfo(interm_buf, "%s", value);
-
- appendStringInfo(buf, "%c%d%c%s",
- PXF_SIZE_BYTES, interm_buf->len,
- PXF_CONST_DATA, interm_buf->data);
- resetStringInfo(interm_buf);
- }
- break;
- }
- default:
- /* should never happen. we filter on types earlier */
- elog(ERROR,
- "internal error in pxf_filter.c:ListConstToStr. "
- "Using unsupported data type (%d)",
- constval->consttype);
-
- }
-
- pfree(interm_buf->data);
-}
-
-/*
-* SerializePxfFilterQuals
-*
-* Wrapper around pxf_make_filter_list -> PxfSerializeFilterList.
-*
-* The function accepts the scan qual list and produces a serialized
-* string that represents the push down filters (See called functions
-* headers for more information).
-*/
-char *
-SerializePxfFilterQuals(List *quals)
-{
- char *result = NULL;
-
- if (quals == NULL)
- return result;
-
- List *clauses = NULL;
- ListCell *lc;
-
- foreach(lc, quals)
- {
- RestrictInfo *ri = (RestrictInfo *) lfirst(lc);
-
- clauses = lappend(clauses, ri->clause);
- }
-
- /*
- * expressionItems will contain all the expressions including comparator
- * and logical operators in postfix order
- */
- List *expressionItems = PxfMakeExpressionItemsList(clauses, NULL);
-
- /*
- * result will contain serialized version of the above postfix ordered
- * expressions list
- */
- result = PxfSerializeFilterList(expressionItems);
-
- PxfFreeExpressionItemsList(expressionItems);
-
- elog(DEBUG1,
- "serializePxfFilterQuals: resulting filter string is '%s'",
- (result == NULL) ? "" : result);
-
- return result;
-}
-
-/*
-* Adds a given number of AND expression items to an existing list of expression items
-*/
-void
-AddExtraAndExpressionItems(List *expressionItems, int extraAndOperatorsNum)
-{
- if ((!expressionItems) || (extraAndOperatorsNum < 1))
- {
- return;
- }
-
- ExpressionItem
- *andExpressionItem = (ExpressionItem *) palloc0(sizeof(ExpressionItem));
-
- BoolExpr *andExpr = makeNode(BoolExpr);
-
- andExpr->boolop = AND_EXPR;
- andExpressionItem->node = (Node *) andExpr;
- andExpressionItem->parent = NULL;
- andExpressionItem->processed = false;
-
- for (int i = 0; i < extraAndOperatorsNum; i++)
- {
- expressionItems = lappend(expressionItems, andExpressionItem);
- }
-}
-
-/*
-* Returns a list of attributes, extracted from quals.
-* Returns NIL if any error occurred.
-* Supports AND, OR, NOT operations.
-* Supports =, <, <=, >, >=, IS NULL, IS NOT NULL, BETWEEN, IN operators.
-* List might contain duplicates.
-* Caller should release memory once result is not needed.
-*/
-List *
-extractPxfAttributes(List *quals, bool *qualsAreSupported)
-{
- if (!(*qualsAreSupported))
- return NIL;
- ListCell *lc = NULL;
- List *attributes = NIL;
- bool expressionIsSupported;
-
- *qualsAreSupported = true;
-
- if (list_length(quals) == 0)
- return NIL;
-
- foreach(lc, quals)
- {
- Node *node = (Node *) lfirst(lc);
- NodeTag tag = nodeTag(node);
-
- switch (tag)
- {
- case T_OpExpr:
- case T_ScalarArrayOpExpr:
- {
- Expr *expr = (Expr *) node;
- List *attrs = GetAttrsFromExpr(expr,
- &expressionIsSupported);
-
- if (!expressionIsSupported)
- {
- *qualsAreSupported = false;
- return NIL;
- }
- attributes = list_concat(attributes, attrs);
- break;
- }
- case T_BoolExpr:
- {
- BoolExpr *expr = (BoolExpr *) node;
- bool innerBoolQualsAreSupported = true;
- List *inner_result =
- extractPxfAttributes(expr->args,
- &innerBoolQualsAreSupported);
-
- if (!innerBoolQualsAreSupported)
- {
- *qualsAreSupported = false;
- return NIL;
- }
- attributes = list_concat(attributes, inner_result);
- break;
- }
- case T_NullTest:
- {
- NullTest *expr = (NullTest *) node;
-
- attributes =
- AppendAttrFromVar((Var *) expr->arg, attributes);
- break;
- }
- case T_BooleanTest:
- {
- BooleanTest *expr = (BooleanTest *) node;
-
- attributes =
- AppendAttrFromVar((Var *) expr->arg, attributes);
- break;
- }
- default:
- {
- /*
- * tag is not supported, it's risk of having: 1)
- * false-positive tuples 2) unable to join tables 3) etc
- */
- elog(INFO,
- "extractPxfAttributes: unsupported node tag %d, unable to extract attribute from qualifier",
- tag);
- return NIL;
- }
- }
- }
-
- return attributes;
-}
diff --git a/gpcontrib/pxf_fdw/pxf_filter.h b/gpcontrib/pxf_fdw/pxf_filter.h
deleted file mode 100644
index cc4c07a85d7..00000000000
--- a/gpcontrib/pxf_fdw/pxf_filter.h
+++ /dev/null
@@ -1,151 +0,0 @@
-/*
- * pxf_filter.h
- *
- * Header for handling push down of supported scan level filters to PXF.
- *
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- *
- */
-#ifndef _PXF_FILTERS_H_
-#define _PXF_FILTERS_H_
-
-#include "postgres.h"
-#include "access/attnum.h"
-#include "executor/executor.h"
-#include "nodes/pg_list.h"
-
-
-/*
- * each supported operator has a code that will describe the operator
- * type in the final serialized string that gets pushed down.
- *
- * NOTE: the codes could be forced into a single byte, but list will
- * grow larger in the future, so why bother.
- */
-typedef enum PxfOperatorCode
-{
- PXFOP_LT = 1,
- PXFOP_GT,
- PXFOP_LE,
- PXFOP_GE,
- PXFOP_EQ,
- PXFOP_NE,
- PXFOP_LIKE,
- PXFOP_IS_NULL,
- PXFOP_IS_NOTNULL,
- PXFOP_IN
-
-} PxfOperatorCode;
-
-/*
- * each supported operand from both sides of the operator is represented
- * by a code that will describe the operator type in the final serialized
- * string that gets pushed down.
- */
-#define PXF_ATTR_CODE 'a'
-#define PXF_SCALAR_CONST_CODE 'c'
-#define PXF_LIST_CONST_CODE 'm'
-#define PXF_SIZE_BYTES 's'
-#define PXF_CONST_DATA 'd'
-#define PXF_OPERATOR_CODE 'o'
-#define PXF_LOGICAL_OPERATOR_CODE 'l'
-
-#define NullConstValue "NULL"
-#define TrueConstValue "true"
-#define FalseConstValue "false"
-
-/*
- * An Operand has any of the above codes, and the information specific to
- * its type. This could be compacted but filter structures are expected to
- * be very few and very small so it's ok as is.
- */
-typedef struct PxfOperand
-{
- char opcode; /* PXF_ATTR_CODE, PXF_SCALAR_CONST_CODE,
- * PXF_LIST_CONST_CODE */
- AttrNumber attnum; /* used when opcode is PXF_ATTR_CODE */
- StringInfo conststr; /* used when opcode is PXF_SCALAR_CONST_CODE
- * or PXF_LIST_CONST_CODE */
- Oid consttype; /* used when opcode is PXF_SCALAR_CONST_CODE
- * or PXF_LIST_CONST_CODE */
-
-} PxfOperand;
-
-/*
- * A PXF filter has a left and right operands, and an operator.
- */
-typedef struct PxfFilterDesc
-{
- PxfOperand l; /* left operand */
- PxfOperand r; /* right operand or InvalidAttrNumber if none */
- PxfOperatorCode op; /* operator code */
-
-} PxfFilterDesc;
-
-/*
- * A mapping of GPDB operator OID to PXF operator code.
- * Used for Node of type 'T_OpExpr'
- */
-typedef struct dbop_pxfop_map
-{
- Oid dbop;
- PxfOperatorCode pxfop;
-
-} dbop_pxfop_map;
-
-/*
- * A mapping of GPDB operator OID to PXF operator code ('PXFOP_IN').
- * Used for Node of type 'T_ScalarArrayOpExpr'
- */
-typedef struct dbop_pxfop_array_map
-{
- Oid dbop;
- PxfOperatorCode pxfop;
- bool useOr;
-
-} dbop_pxfop_array_map;
-
-typedef struct ExpressionItem
-{
- Node *node;
- Node *parent;
- bool processed;
-} ExpressionItem;
-
-static inline bool
-pxfoperand_is_attr(PxfOperand x)
-{
- return (x.opcode == PXF_ATTR_CODE);
-}
-
-static inline bool
-pxfoperand_is_scalar_const(PxfOperand x)
-{
- return (x.opcode == PXF_SCALAR_CONST_CODE);
-}
-
-static inline bool
-pxfoperand_is_list_const(PxfOperand x)
-{
- return (x.opcode == PXF_LIST_CONST_CODE);
-}
-
-char *SerializePxfFilterQuals(List *quals);
-List *extractPxfAttributes(List *quals, bool *qualsAreSupported);
-
-#endif /* // _PXF_FILTERS_H_ */
diff --git a/gpcontrib/pxf_fdw/pxf_fragment.c b/gpcontrib/pxf_fdw/pxf_fragment.c
deleted file mode 100644
index 3104a32391f..00000000000
--- a/gpcontrib/pxf_fdw/pxf_fragment.c
+++ /dev/null
@@ -1,589 +0,0 @@
-#include "postgres.h"
-
-#include "pxf_fragment.h"
-
-#include "cdb/cdbtm.h"
-#include "cdb/cdbvars.h"
-#include "commands/copy.h"
-#if (PG_VERSION_NUM >= 140000)
-#include "commands/copyfrom_internal.h"
-#endif
-#include "common/jsonapi.h"
-#include "foreign/foreign.h"
-#include "utils/builtins.h"
-#include "utils/guc.h"
-
-#define LOG_DEBUG (FRAGDEBUG >= log_min_messages) || (FRAGDEBUG >= client_min_messages)
-
-static List *GetDataFragmentList(PxfOptions *options, ClientContext * client_context);
-static void RestRequest(PxfOptions *options, ClientContext * client_context, char *rest_msg);
-static List *ParseGetFragmentsResponse(StringInfo rest_buf);
-static void Init(ClientContext * client_context);
-static void LogFragmentList(const char *debugHeader, List *fragments);
-static void InitClientContext(ClientContext * client_context);
-static void CallRest(PxfOptions *options, ClientContext * client_context, char *rest_msg);
-static void ProcessRequest(ClientContext * client_context, char *uri);
-static JsonParseErrorType PxfFragmentScalar(void *state, char *token, JsonTokenType type);
-static JsonParseErrorType PxfFragmentObjectStart(void *state, char *name, bool isnull);
-static JsonParseErrorType PxfArrayElementStart(void *state, bool isnull);
-static JsonParseErrorType PxfArrayElementEnd(void *state, bool isnull);
-
-/* Get List of fragments using PXF
- * Returns selected fragments that have been allocated to the current segment
- */
-List *
-GetFragmentList(PxfOptions *options,
- Relation relation,
- char *filter_string,
- List *retrieved_attrs)
-{
- List *data_fragments;
-
- /* Context for the Fragmenter API */
- ClientContext client_context;
-
- Assert(options != NULL);
-
- /* 1. Initialize curl headers */
- Init(&client_context);
-
- /* Enrich the curl HTTP header */
- BuildHttpHeaders(client_context.http_headers,
- options,
- relation,
- filter_string,
- retrieved_attrs);
-
- /*
- * Get the fragments data from the PXF service
- */
- data_fragments = GetDataFragmentList(options, &client_context);
- if (data_fragments == NIL)
- return NIL;
-
- if (LOG_DEBUG)
- LogFragmentList("Available Data fragments", data_fragments);
-
- return data_fragments;
-}
-
-/*
- * Assign PXF Host for each Data Fragment
- * Will use the same host as the existing segment as the PXF host.
- */
-void
-AssignPxfLocationToFragments(PxfOptions *options, List *fragments)
-{
- ListCell *frag_c = NULL;
-
- foreach(frag_c, fragments)
- {
- FragmentData *fragment = (FragmentData *) lfirst(frag_c);
- fragment->authority = psprintf("%s:%d", options->pxf_host, options->pxf_port);
- }
-}
-
-/*
- * GetDataFragmentList
- *
- * 1. Request a list of fragments from the PXF Fragmenter class
- * that was specified in the pxf URL.
- *
- * 2. Process the returned list and create a list of DataFragment with it.
- */
-static List *
-GetDataFragmentList(PxfOptions *options,
- ClientContext * client_context)
-{
- List *data_fragments;
- char *restMsg = "http://%s:%d/%s/%s/Fragmenter/getFragments";
-
- RestRequest(options, client_context, restMsg);
-
- /* parse the JSON response and form a fragments list to return */
- data_fragments = ParseGetFragmentsResponse(&(client_context->the_rest_buf));
-
- return data_fragments;
-}
-
-/*
- * Wrap the REST call.
- * TODO: Add logic later to handle HDFS HA related exception
- * to failover with a retry for the HA HDFS scenario.
- */
-static void
-RestRequest(PxfOptions *options, ClientContext * client_context, char *rest_msg)
-{
- Assert(options->pxf_host != NULL && options->pxf_port > 0);
-
- /* construct the request */
- CallRest(options, client_context, rest_msg);
-}
-
-/*
- * parse the response of the PXF Fragments call. An example:
- *
- * Request:
- * curl --header "X-GP-FRAGMENTER: HdfsDataFragmenter" "http://goldsa1mac.local:50070/pxf/v2/Fragmenter/getFragments?path=demo" (demo is a directory)
- *
- * Response (left as a single line purposefully):
- * {"PXFFragments":[{"index":0,"userData":null,"sourceName":"demo/text2.csv","metadata":"rO0ABXcQAAAAAAAAAAAAAAAAAAAABXVyABNbTGphdmEubGFuZy5TdHJpbmc7rdJW5+kde0cCAAB4cAAAAAN0ABxhZXZjZWZlcm5hczdtYnAuY29ycC5lbWMuY29tdAAcYWV2Y2VmZXJuYXM3bWJwLmNvcnAuZW1jLmNvbXQAHGFldmNlZmVybmFzN21icC5jb3JwLmVtYy5jb20=","replicas":["10.207.4.23","10.207.4.23","10.207.4.23"]},{"index":0,"userData":null,"sourceName":"demo/text_csv.csv","metadata":"rO0ABXcQAAAAAAAAAAAAAAAAAAAABnVyABNbTGphdmEubGFuZy5TdHJpbmc7rdJW5 [...]
- */
-
-typedef enum pxf_fragment_object
-{
- PXF_PARSE_START,
- PXF_PARSE_INDEX,
- PXF_PARSE_USERDATA,
- PXF_PARSE_PROFILE,
- PXF_PARSE_SOURCENAME,
- PXF_PARSE_METADATA,
- PXF_PARSE_REPLICAS
-} pxf_fragment_object;
-
-typedef struct FragmentState
-{
- JsonLexContext *lex;
- pxf_fragment_object object;
- List *fragments;
- bool has_replicas;
- int arraydepth;
-} FragmentState;
-
-static JsonParseErrorType
-PxfFragmentObjectStart(void *state, char *name, bool isnull)
-{
- FragmentState *s = (FragmentState *) state;
-
- if (s->lex->token_type == JSON_TOKEN_NUMBER)
- {
- if (pg_strcasecmp(name, "index") == 0)
- s->object = PXF_PARSE_INDEX;
- }
- else if (s->lex->token_type == JSON_TOKEN_STRING || s->lex->token_type == JSON_TOKEN_NULL)
- {
- if (pg_strcasecmp(name, "userData") == 0)
- s->object = PXF_PARSE_USERDATA;
- else if (pg_strcasecmp(name, "sourceName") == 0)
- s->object = PXF_PARSE_SOURCENAME;
- else if (pg_strcasecmp(name, "metadata") == 0)
- s->object = PXF_PARSE_METADATA;
- else if (pg_strcasecmp(name, "profile") == 0)
- s->object = PXF_PARSE_PROFILE;
- else
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("unrecognized object in PXF fragment: \"%s\"",
- name)));
- }
- else if (s->lex->token_type == JSON_TOKEN_ARRAY_START)
- {
- if (pg_strcasecmp(name, "PXFFragments") == 0)
- {
- if (s->object != PXF_PARSE_START)
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("malformed PXF fragment")));
- }
- else if (pg_strcasecmp(name, "replicas") == 0)
- {
- s->object = PXF_PARSE_REPLICAS;
- s->has_replicas = false;
- }
- else
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("unrecognized array in PXF fragment: \"%s\"",
- name)));
- }
- return JSON_SUCCESS;
-}
-
-static void
-CheckAndAssign(char **field, JsonTokenType type, char *token,
- JsonTokenType expected_type, bool nullable)
-{
- if (type == JSON_TOKEN_NULL && nullable)
- return;
-
- if (type == expected_type)
- *field = pstrdup(token);
- else
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("unexpected value \"%s\" for attribute", token)));
-}
-
-static JsonParseErrorType
-PxfFragmentScalar(void *state, char *token, JsonTokenType type)
-{
- FragmentState *s = (FragmentState *) state;
- FragmentData *d = (FragmentData *) llast(s->fragments);
-
- /*
- * Populate the fragment depending on the type of the scalar and the
- * current object
- */
- switch (s->object)
- {
- case PXF_PARSE_USERDATA:
- CheckAndAssign(&(d->user_data), type, token, JSON_TOKEN_STRING, true);
- break;
- case PXF_PARSE_METADATA:
- CheckAndAssign(&(d->fragment_md), type, token, JSON_TOKEN_STRING, true);
- break;
- case PXF_PARSE_PROFILE:
- CheckAndAssign(&(d->profile), type, token, JSON_TOKEN_STRING, true);
- break;
- case PXF_PARSE_SOURCENAME:
- CheckAndAssign(&(d->source_name), type, token, JSON_TOKEN_STRING, false);
- break;
- case PXF_PARSE_INDEX:
- CheckAndAssign(&(d->index), type, token, JSON_TOKEN_NUMBER, true);
- break;
- case PXF_PARSE_REPLICAS:
- if (type == JSON_TOKEN_STRING)
- s->has_replicas = true;
- break;
- case PXF_PARSE_START:
- break;
- default:
- elog(ERROR, "Unexpected PXF object state: %d", s->object);
- break;
- }
-
- return JSON_SUCCESS;
-}
-
-static JsonParseErrorType
-PxfArrayElementStart(void *state, bool isnull)
-{
- FragmentState *s = (FragmentState *) state;
- FragmentData *data;
-
- /*
- * If we are entering a nested array, an array inside the object in the
- * main PXFFragments array, then we are still inside the current fragment
- * and there is nothing but book keeping to do
- */
- if (++s->arraydepth > 1)
- return JSON_SUCCESS;
-
- /*
- * Reaching here means we are entering a new fragment in the PXFFragments
- * array, allocate a new fragment on the list to populate during parsing.
- */
- data = palloc0(sizeof(FragmentData));
- s->fragments = lappend(s->fragments, data);
- s->has_replicas = false;
- s->object = PXF_PARSE_START;
-
- return JSON_SUCCESS;
-}
-
-static JsonParseErrorType
-PxfArrayElementEnd(void *state, bool isnull)
-{
- FragmentState *s = (FragmentState *) state;
-
- if (--s->arraydepth == 0)
- {
- if (!s->has_replicas)
- s->fragments = list_truncate(s->fragments,
- list_length(s->fragments) - 1);
- }
-
- return JSON_SUCCESS;
-}
-
-static List *
-ParseGetFragmentsResponse(StringInfo rest_buf)
-{
- JsonSemAction *sem;
- FragmentState *state;
- text *json;
-
- sem = palloc0(sizeof(JsonSemAction));
- state = palloc0(sizeof(FragmentState));
-
- state->fragments = NIL;
- json = cstring_to_text(rest_buf->data);
- state->lex = makeJsonLexContextCstringLen(VARDATA_ANY(json), VARSIZE_ANY_EXHDR(json), GetDatabaseEncoding(), true);
- state->object = PXF_PARSE_START;
- state->arraydepth = 0;
- state->has_replicas = false;
-
- sem->semstate = state;
- sem->scalar = PxfFragmentScalar;
- sem->object_field_start = PxfFragmentObjectStart;
- sem->array_element_start = PxfArrayElementStart;
- sem->array_element_end = PxfArrayElementEnd;
-
- pg_parse_json(state->lex, sem);
-
- pfree(state->lex);
-
- return state->fragments;
-}
-
-/*
- * Takes a list of fragments and determines which ones need to be processes by the given segment based on MOD function.
- * Removes the elements which will not be processed from the list and frees up their memory.
- * Returns the resulting list, or NIL if no elements satisfy the condition.
- */
-List *
-FilterFragmentsForSegment(List *list)
-{
- if (!list)
- elog(ERROR, "Parameter list is null in FilterFragmentsForSegment");
-
- /*
- * to determine which segment S should process an element at a given index
- * I, use a randomized MOD function
- *
- * S = MOD(I + MOD(gp_session_id, N) + gp_command_count, N)
- *
- * which ensures more fair work distribution for small lists of just a few
- * elements across N segments global session ID and command count is used
- * as a randomizer, as it is different for every query while being the same
- * across all segments for a given query
- */
-
- ListCell *current;
-
- int index = 0;
- int frag_index = 1;
- int numsegments = PXF_SEGMENT_COUNT;
- int32 shift = gp_session_id % numsegments;
-
- foreach(current, list)
- {
- if (PXF_SEGMENT_ID == (index + shift + gp_command_count) % numsegments)
- {
- /*
- * current segment is the one that should process, keep the
- * element, adjust cursor pointers
- */
- FragmentData *frag = (FragmentData *) current->ptr_value;
-
- frag->fragment_idx = frag_index++;
- }
- else
- {
- /*
- * current segment should not process this element, another will,
- * drop the element from the list
- */
-
- if (current->ptr_value)
- FreeFragment((FragmentData *) current->ptr_value);
- list = foreach_delete_current(list, current);
- }
- index++;
- }
- return list;
-}
-
-/*
- * Serializes a List of FragmentData such that it can be transmitted
- * via the interconnect, and the list can be passed from master to
- * segments
- */
-List *
-SerializeFragmentList(List *fragments)
-{
- ListCell *frag_c = NULL;
- List *serializedFragmentList = NIL;
- List *serializedFragment;
-
- foreach(frag_c, fragments)
- {
- FragmentData *fragment = (FragmentData *) lfirst(frag_c);
-
- serializedFragment = NIL;
- serializedFragment = lappend(serializedFragment, makeString(fragment->index));
- serializedFragment = lappend(serializedFragment, makeString(fragment->source_name));
- serializedFragment = lappend(serializedFragment, makeString(fragment->fragment_md));
- serializedFragment = lappend(serializedFragment, makeString(fragment->user_data));
- serializedFragment = lappend(serializedFragment, makeString(fragment->profile));
- serializedFragment = lappend(serializedFragment, makeString(psprintf("%d", fragment->fragment_idx)));
-
- serializedFragmentList = lappend(serializedFragmentList, serializedFragment);
- }
-
- return serializedFragmentList;
-}
-
-/*
- * Deserializes a List of FragmentData received from the interconnect
- */
-List *
-DeserializeFragmentList(List *serializedFragmentList)
-{
- ListCell *frag_c = NULL;
- List *fragments = NIL;
-
- foreach(frag_c, serializedFragmentList)
- {
- List *serializedFragment = (List *) lfirst(frag_c);
-
- FragmentData *fragment = (FragmentData *) palloc0(sizeof(FragmentData));
- fragment->index = strVal(list_nth(serializedFragment, 0));
- fragment->source_name = strVal(list_nth(serializedFragment, 1));
- fragment->fragment_md = strVal(list_nth(serializedFragment, 2));
- fragment->user_data = strVal(list_nth(serializedFragment, 3));
- fragment->profile = strVal(list_nth(serializedFragment, 4));
- fragment->fragment_idx = atoi(strVal(list_nth(serializedFragment, 5)));
-
- fragments = lappend(fragments, fragment);
- }
-
- return fragments;
-}
-
-/*
- * Preliminary curl initializations for the REST communication
- */
-static void
-Init(ClientContext * client_context)
-{
- InitClientContext(client_context);
- /* Communication with back-end, initialize churl client context and header */
- client_context->http_headers = churl_headers_init();
-
- /* set HTTP header that guarantees response in JSON format */
- churl_headers_append(client_context->http_headers, REST_HEADER_JSON_RESPONSE, NULL);
-}
-
-/*
- * Free fragment data
- */
-void
-FreeFragment(FragmentData *data)
-{
- if (data->authority)
- pfree(data->authority);
- if (data->fragment_md)
- pfree(data->fragment_md);
- if (data->index)
- pfree(data->index);
- if (data->profile)
- pfree(data->profile);
- if (data->source_name)
- pfree(data->source_name);
- if (data->user_data)
- pfree(data->user_data);
- pfree(data);
-}
-
-/*
- * Debug function - print the splits data structure obtained from the namenode
- * response to <GET_BLOCK_LOCATIONS> request
- */
-static void
-LogFragmentList(const char *debugHeader, List *fragments)
-{
- ListCell *fragment_cell = NULL;
- StringInfoData log_str;
-
- initStringInfo(&log_str);
-
- appendStringInfo(&log_str, "---%s---\nFragment list: (%d elements)\n",
- debugHeader, fragments ? fragments->length : 0);
-
- foreach(fragment_cell, fragments)
- {
- FragmentData *frag = (FragmentData *) lfirst(fragment_cell);
-
- appendStringInfo(&log_str, "Fragment index: %s\n", frag->index);
- appendStringInfo(&log_str, "authority: %s\n", frag->authority);
- appendStringInfo(&log_str, "source: %s\n", frag->source_name);
- appendStringInfo(&log_str, "metadata: %s\n", frag->fragment_md ? frag->fragment_md : "NULL");
- appendStringInfo(&log_str, "user data: %s\n", frag->user_data ? frag->user_data : "NULL");
- appendStringInfo(&log_str, "profile: %s\n", frag->profile ? frag->profile : "NULL");
- }
- appendStringInfo(&log_str, "------\n");
-
- elog(FRAGDEBUG, "%s", log_str.data);
- pfree(log_str.data);
-}
-
-/*
- * Initializes the client context
- */
-static void
-InitClientContext(ClientContext * client_context)
-{
- client_context->http_headers = NULL;
- client_context->handle = NULL;
- initStringInfo(&(client_context->the_rest_buf));
-}
-
-
-/*
- * call_rest
- *
- * Creates the REST message and sends it to the PXF service located on
- * <hadoop_uri->host>:<hadoop_uri->port>
- */
-static void
-CallRest(PxfOptions *options, ClientContext * client_context, char *rest_msg)
-{
- StringInfoData request;
-
- initStringInfo(&request);
-
- appendStringInfo(&request, rest_msg,
- options->pxf_host,
- options->pxf_port,
- PXF_SERVICE_PREFIX,
- PXF_VERSION);
-
- /* send the request. The response will exist in rest_buf.data */
- ProcessRequest(client_context, request.data);
- pfree(request.data);
-}
-
-/*
- * Reads from churl in chunks of 64K and copies data to the context's buffer
- */
-static void
-ProcessRequest(ClientContext * client_context, char *uri)
-{
- size_t n = 0;
- char buffer[RAW_BUF_SIZE];
-
- print_http_headers(client_context->http_headers);
- client_context->handle = churl_init_download(uri, client_context->http_headers);
- if (client_context->handle == NULL)
- ereport(ERROR,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("unsuccessful connection to uri: \"%s\"", uri)));
- memset(buffer, 0, RAW_BUF_SIZE);
- resetStringInfo(&(client_context->the_rest_buf));
-
- /*
- * This try-catch ensures that in case of an exception during the
- * "communication with PXF and the accumulation of PXF data in
- * client_context->the_rest_buf", we still get to terminate the libcurl
- * connection nicely and avoid leaving the PXF server connection hung.
- */
- PG_TRY();
- {
- /* read some bytes to make sure the connection is established */
- churl_read_check_connectivity(client_context->handle);
- while ((n = churl_read(client_context->handle, buffer, sizeof(buffer))) != 0)
- {
- appendBinaryStringInfo(&(client_context->the_rest_buf), buffer, n);
- memset(buffer, 0, RAW_BUF_SIZE);
- }
- churl_cleanup(client_context->handle, false);
- }
- PG_CATCH();
- {
- if (client_context->handle)
- churl_cleanup(client_context->handle, true);
- PG_RE_THROW();
- }
- PG_END_TRY();
-}
diff --git a/gpcontrib/pxf_fdw/pxf_fragment.h b/gpcontrib/pxf_fdw/pxf_fragment.h
deleted file mode 100644
index 642806b9855..00000000000
--- a/gpcontrib/pxf_fdw/pxf_fragment.h
+++ /dev/null
@@ -1,108 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- *
- */
-
-#ifndef GPDB_PXFFRAGMENT_H
-#define GPDB_PXFFRAGMENT_H
-
-#include "libchurl.h"
-#include "pxf_header.h"
-
-#include "lib/stringinfo.h"
-#include "utils/rel.h"
-
-#define PXF_SEGMENT_ID GpIdentity.segindex
-#define PXF_SEGMENT_COUNT getgpsegmentCount()
-
-/*
- * Context for the Fragmenter
- */
-typedef struct sClientContext
-{
- CHURL_HEADERS http_headers;
- CHURL_HANDLE handle;
- /* part of the HTTP response - received */
- /* from one call to churl_read */
- /* contains the complete HTTP response */
- StringInfoData the_rest_buf;
-} ClientContext;
-
-/*
- * FragmentData - describes a single Hadoop file split / HBase table region
- * in means of location (ip, port), the source name of the specific file/table that is being accessed,
- * and the index of a of list of fragments (splits/regions) for that source name.
- * The index refers to the list of the fragments of that source name.
- * user_data is optional.
- */
-typedef struct FragmentData
-{
- char *authority;
- char *index;
- char *source_name;
- char *fragment_md;
- char *user_data;
- char *profile;
- int fragment_idx;
-} FragmentData;
-
-/*
- * One debug level for all log messages from the data allocation algorithm
- */
-#define FRAGDEBUG DEBUG2
-
-/*
- * Gets the fragments for the given uri location
- */
-extern List *GetFragmentList(PxfOptions *options,
- Relation relation,
- char *filter_string,
- List *retrieved_attrs);
-
-/*
- * Takes a list of fragments and determines which ones need to be processed
- * by the given segment based on MOD function. Removes the elements which will
- * not be processed from the list and frees up their memory.
- * Returns the resulting list, or NIL if no elements satisfy the condition.
- */
-extern List *FilterFragmentsForSegment(List *list);
-
-/*
- * Serializes a List of FragmentData such that it can be transmitted
- * via the interconnect, and the list can be passed from master to
- * segments
- */
-extern List * SerializeFragmentList(List *fragments);
-
-/*
- * Deserializes a List of FragmentData received from the interconnect
- */
-extern List * DeserializeFragmentList(List *serializedFragmentList);
-
-/*
- * Assign PXF Host for each Data Fragment
- * Will use the same host as the existing segment as the PXF host.
- */
-extern void AssignPxfLocationToFragments(PxfOptions *options, List *fragments);
-
-/*
- * Frees the given fragment
- */
-extern void FreeFragment(FragmentData *data);
-
-#endif /* GPDB_PXFFRAGMENT_H */
diff --git a/gpcontrib/pxf_fdw/pxf_header.c b/gpcontrib/pxf_fdw/pxf_header.c
deleted file mode 100644
index b90afac3e7b..00000000000
--- a/gpcontrib/pxf_fdw/pxf_header.c
+++ /dev/null
@@ -1,352 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-#include "pxf_filter.h"
-#include "pxf_header.h"
-
-#include "access/external.h"
-#include "access/url.h"
-#include "utils/builtins.h"
-#include "commands/defrem.h"
-#include "utils/formatting.h"
-#include "utils/timestamp.h"
-#include "utils/syscache.h"
-
-/* helper function declarations */
-static void AddAlignmentSizeHttpHeader(CHURL_HEADERS headers);
-static void AddTupleDescriptionToHttpHeader(CHURL_HEADERS headers, Relation rel);
-static void AddOptionsToHttpHeader(CHURL_HEADERS headers, List *options);
-static void AddProjectionDescHttpHeader(CHURL_HEADERS headers, List *retrieved_attrs);
-static void AddProjectionIndexHeader(CHURL_HEADERS headers, int attno, char *long_number);
-static char *NormalizeKeyName(const char *key);
-static char *TypeOidGetTypename(Oid typid);
-
-/*
- * Add key/value pairs to connection header.
- * These values are the context of the query and used
- * by the remote component.
- */
-void
-BuildHttpHeaders(CHURL_HEADERS headers,
- PxfOptions *options,
- Relation relation,
- char *filter_string,
- List *retrieved_attrs)
-{
- extvar_t ev;
- char pxfPortString[sizeof(int32) * 8];
-
- if (relation != NULL)
- {
- /* Record fields - name and type of each field */
- AddTupleDescriptionToHttpHeader(headers, relation);
- }
-
- if (retrieved_attrs != NULL)
- {
- /* add the list of attrs to the projection desc http headers */
- AddProjectionDescHttpHeader(headers, retrieved_attrs);
- }
-
- /* GP cluster configuration */
- external_set_env_vars(&ev, "pxf_fdw", false, NULL, NULL, false, 0);
-
- /*
- * make sure that user identity is known and set, otherwise impersonation
- * by PXF will be impossible
- */
- if (!ev.GP_USER || !ev.GP_USER[0])
- ereport(ERROR,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("user identity is unknown")));
- churl_headers_append(headers, "X-GP-USER", ev.GP_USER);
-
- churl_headers_append(headers, "X-GP-SEGMENT-ID", ev.GP_SEGMENT_ID);
- churl_headers_append(headers, "X-GP-SEGMENT-COUNT", ev.GP_SEGMENT_COUNT);
- churl_headers_append(headers, "X-GP-XID", ev.GP_XID);
-
- AddAlignmentSizeHttpHeader(headers);
-
- /* Convert the number of attributes to a string */
- pg_ltoa(options->pxf_port, pxfPortString);
-
- /* headers for uri data */
- churl_headers_append(headers, "X-GP-URL-HOST", options->pxf_host);
- churl_headers_append(headers, "X-GP-URL-PORT", pxfPortString);
-
- churl_headers_append(headers, "X-GP-OPTIONS-PROFILE", options->profile);
- /* only text format is supported for FDW */
- churl_headers_append(headers, "X-GP-FORMAT", "TEXT");
- churl_headers_append(headers, "X-GP-DATA-DIR", options->resource);
- churl_headers_append(headers, "X-GP-OPTIONS-SERVER", options->server);
-
- /* extra options */
- AddOptionsToHttpHeader(headers, options->options);
-
- /* copy options */
- AddOptionsToHttpHeader(headers, options->copy_options);
-
- /* filters */
- if (filter_string && strcmp(filter_string, "") != 0)
- {
- churl_headers_append(headers, "X-GP-FILTER", filter_string);
- churl_headers_append(headers, "X-GP-HAS-FILTER", "1");
- }
- else
- churl_headers_append(headers, "X-GP-HAS-FILTER", "0");
-}
-
-/* Report alignment size to remote component
- * GPDBWritable uses alignment that has to be the same as
- * in the C code.
- * Since the C code can be compiled for both 32 and 64 bits,
- * the alignment can be either 4 or 8.
- */
-static void
-AddAlignmentSizeHttpHeader(CHURL_HEADERS headers)
-{
- char tmp[sizeof(char *)];
-
- pg_ltoa(sizeof(char *), tmp);
- churl_headers_append(headers, "X-GP-ALIGNMENT", tmp);
-}
-
-/*
- * Report tuple description to remote component
- * Currently, number of attributes, attributes names, types and types modifiers
- * Each attribute has a pair of key/value
- * where X is the number of the attribute
- * X-GP-ATTR-NAMEX - attribute X's name
- * X-GP-ATTR-TYPECODEX - attribute X's type OID (e.g, 16)
- * X-GP-ATTR-TYPENAMEX - attribute X's type name (e.g, "boolean")
- * optional - X-GP-ATTR-TYPEMODX-COUNT - total number of modifier for attribute X
- * optional - X-GP-ATTR-TYPEMODX-Y - attribute X's modifiers Y (types which have precision info, like numeric(p,s))
- */
-static void
-AddTupleDescriptionToHttpHeader(CHURL_HEADERS headers, Relation rel)
-{
- char long_number[sizeof(int32) * 8];
- StringInfoData formatter;
- TupleDesc tuple;
-
- initStringInfo(&formatter);
-
- /* Get tuple description itself */
- tuple = RelationGetDescr(rel);
-
- /* Convert the number of attributes to a string */
- pg_ltoa(tuple->natts, long_number);
- churl_headers_append(headers, "X-GP-ATTRS", long_number);
-
- /* Iterate attributes */
- for (int i = 0; i < tuple->natts; ++i)
- {
- Form_pg_attribute attr = TupleDescAttr(tuple, i);
-
- /* Add a key/value pair for attribute name */
- resetStringInfo(&formatter);
- appendStringInfo(&formatter, "X-GP-ATTR-NAME%u", i);
- churl_headers_append(headers, formatter.data, attr->attname.data);
-
- /* Add a key/value pair for attribute type */
- resetStringInfo(&formatter);
- appendStringInfo(&formatter, "X-GP-ATTR-TYPECODE%u", i);
- pg_ltoa(attr->atttypid, long_number);
- churl_headers_append(headers, formatter.data, long_number);
-
- /* Add a key/value pair for attribute type name */
- resetStringInfo(&formatter);
- appendStringInfo(&formatter, "X-GP-ATTR-TYPENAME%u", i);
- churl_headers_append(headers, formatter.data, TypeOidGetTypename(attr->atttypid));
-
- /* Add attribute type modifiers if any */
- if (attr->atttypmod > -1)
- {
- switch (attr->atttypid)
- {
- case NUMERICOID:
- {
- resetStringInfo(&formatter);
- appendStringInfo(&formatter, "X-GP-ATTR-TYPEMOD%u-COUNT", i);
- pg_ltoa(2, long_number);
- churl_headers_append(headers, formatter.data, long_number);
-
-
- /* precision */
- resetStringInfo(&formatter);
- appendStringInfo(&formatter, "X-GP-ATTR-TYPEMOD%u-%u", i, 0);
- pg_ltoa((attr->atttypmod >> 16) & 0xffff, long_number);
- churl_headers_append(headers, formatter.data, long_number);
-
- /* scale */
- resetStringInfo(&formatter);
- appendStringInfo(&formatter, "X-GP-ATTR-TYPEMOD%u-%u", i, 1);
- pg_ltoa((attr->atttypmod - VARHDRSZ) & 0xffff, long_number);
- churl_headers_append(headers, formatter.data, long_number);
- break;
- }
- case CHAROID:
- case BPCHAROID:
- case VARCHAROID:
- {
- resetStringInfo(&formatter);
- appendStringInfo(&formatter, "X-GP-ATTR-TYPEMOD%u-COUNT", i);
- pg_ltoa(1, long_number);
- churl_headers_append(headers, formatter.data, long_number);
-
- resetStringInfo(&formatter);
- appendStringInfo(&formatter, "X-GP-ATTR-TYPEMOD%u-%u", i, 0);
- pg_ltoa((attr->atttypmod - VARHDRSZ), long_number);
- churl_headers_append(headers, formatter.data, long_number);
- break;
- }
- case VARBITOID:
- case BITOID:
- case TIMESTAMPOID:
- case TIMESTAMPTZOID:
- case TIMEOID:
- case TIMETZOID:
- {
- resetStringInfo(&formatter);
- appendStringInfo(&formatter, "X-GP-ATTR-TYPEMOD%u-COUNT", i);
- pg_ltoa(1, long_number);
- churl_headers_append(headers, formatter.data, long_number);
-
- resetStringInfo(&formatter);
- appendStringInfo(&formatter, "X-GP-ATTR-TYPEMOD%u-%u", i, 0);
- pg_ltoa((attr->atttypmod), long_number);
- churl_headers_append(headers, formatter.data, long_number);
- break;
- }
- case INTERVALOID:
- {
- resetStringInfo(&formatter);
- appendStringInfo(&formatter, "X-GP-ATTR-TYPEMOD%u-COUNT", i);
- pg_ltoa(1, long_number);
- churl_headers_append(headers, formatter.data, long_number);
-
- resetStringInfo(&formatter);
- appendStringInfo(&formatter, "X-GP-ATTR-TYPEMOD%u-%u", i, 0);
- pg_ltoa(INTERVAL_PRECISION(attr->atttypmod), long_number);
- churl_headers_append(headers, formatter.data, long_number);
- break;
- }
- default:
- elog(DEBUG5, "addTupleDescriptionToHttpHeader: unsupported type %d ", attr->atttypid);
- break;
- }
- }
- }
-
- pfree(formatter.data);
-}
-
-/*
- * Report projection description to the remote component
- */
-static void
-AddProjectionDescHttpHeader(CHURL_HEADERS headers, List *retrieved_attrs)
-{
- ListCell *lc1 = NULL;
- char long_number[sizeof(int32) * 8];
-
- foreach(lc1, retrieved_attrs)
- {
- int attno = lfirst_int(lc1);
-
- /* zero-based index in the server side */
- AddProjectionIndexHeader(headers, attno - 1, long_number);
- }
-
- if (retrieved_attrs->length == 0)
- return;
-
- /* Convert the number of projection columns to a string */
- pg_ltoa(retrieved_attrs->length, long_number);
- churl_headers_append(headers, "X-GP-ATTRS-PROJ", long_number);
-}
-
-/*
- * Adds the projection index header for the given attno
- */
-static void
-AddProjectionIndexHeader(CHURL_HEADERS headers,
- int attno,
- char *long_number)
-{
- pg_ltoa(attno, long_number);
- churl_headers_append(headers, "X-GP-ATTRS-PROJ-IDX", long_number);
-}
-
-/*
- * Add all the FDW options in the list to the curl headers
- */
-static void
-AddOptionsToHttpHeader(CHURL_HEADERS headers, List *options)
-{
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = (DefElem *) lfirst(cell);
- char *x_gp_key = NormalizeKeyName(def->defname);
-
- churl_headers_append(headers, x_gp_key, defGetString(def));
- pfree(x_gp_key);
- }
-}
-
-/*
- * Full name of the HEADER KEY expected by the PXF service
- * Converts input string to upper case and prepends "X-GP-OPTIONS-" string
- * This will be used for all user defined parameters to be isolate from internal parameters
- */
-static char *
-NormalizeKeyName(const char *key)
-{
- if (!key || strlen(key) == 0)
- elog(ERROR, "internal error in pxfutils.c:normalize_key_name, parameter key is null or empty");
-
- return psprintf("X-GP-OPTIONS-%s", asc_toupper(pstrdup(key), strlen(key)));
-}
-
-/*
- * TypeOidGetTypename
- * Get the name of the type, given the OID
- */
-static char *
-TypeOidGetTypename(Oid typid)
-{
-
- Assert(OidIsValid(typid));
-
- HeapTuple typtup = SearchSysCache(TYPEOID,
- ObjectIdGetDatum(typid),
- 0, 0, 0);
-
- if (!HeapTupleIsValid(typtup))
- elog(ERROR, "cache lookup failed for type %u", typid);
-
- Form_pg_type typform = (Form_pg_type) GETSTRUCT(typtup);
- char *typname = psprintf("%s", NameStr(typform->typname));
-
- ReleaseSysCache(typtup);
-
- return typname;
-}
diff --git a/gpcontrib/pxf_fdw/pxf_header.h b/gpcontrib/pxf_fdw/pxf_header.h
deleted file mode 100644
index 3b8d0574eb3..00000000000
--- a/gpcontrib/pxf_fdw/pxf_header.h
+++ /dev/null
@@ -1,43 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-#ifndef _PXFHEADERS_H_
-#define _PXFHEADERS_H_
-
-#define PXF_SERVICE_PREFIX "pxf"
-#define PXF_VERSION "v15"
-
-#include "libchurl.h"
-
-#include "pxf_fdw.h"
-
-#include "nodes/execnodes.h"
-#include "nodes/nodeFuncs.h"
-#include "utils/rel.h"
-
-/*
- * Adds the headers necessary for PXF service call
- */
-extern void BuildHttpHeaders(CHURL_HEADERS headers,
- PxfOptions *options,
- Relation relation,
- char *filter_string,
- List *retrieved_attrs);
-
-#endif /* _PXFHEADERS_H_ */
diff --git a/gpcontrib/pxf_fdw/pxf_option.c b/gpcontrib/pxf_fdw/pxf_option.c
deleted file mode 100644
index 60386c2e521..00000000000
--- a/gpcontrib/pxf_fdw/pxf_option.c
+++ /dev/null
@@ -1,551 +0,0 @@
-/*
- * pxf_option.c
- * Foreign-data wrapper option handling for PXF (Platform Extension Framework)
- *
- * IDENTIFICATION
- * contrib/pxf_fdw/pxf_option.c
- */
-
-#include "postgres.h"
-
-#include "pxf_fdw.h"
-
-#include "access/reloptions.h"
-#include "catalog/pg_foreign_data_wrapper.h"
-#include "catalog/pg_foreign_server.h"
-#include "catalog/pg_foreign_table.h"
-#include "catalog/pg_user_mapping.h"
-#include "commands/copy.h"
-#include "commands/defrem.h"
-#include "nodes/makefuncs.h"
-#include "foreign/foreign.h"
-
-#define FDW_OPTION_WIRE_FORMAT_TEXT "text"
-#define FDW_OPTION_WIRE_FORMAT_CSV "csv"
-
-#define FDW_OPTION_REJECT_LIMIT_ROWS "rows"
-#define FDW_OPTION_REJECT_LIMIT_PERCENT "percent"
-
-#define FDW_OPTION_CONFIG "config"
-#define FDW_OPTION_FORMAT "format"
-#define FDW_OPTION_LOG_ERRORS "log_errors"
-#define FDW_OPTION_MPP_EXECUTE "mpp_execute"
-#define FDW_OPTION_PROTOCOL "protocol"
-#define FDW_OPTION_PXF_HOST "pxf_host"
-#define FDW_OPTION_PXF_PORT "pxf_port"
-#define FDW_OPTION_PXF_PROTOCOL "pxf_protocol"
-#define FDW_OPTION_REJECT_LIMIT "reject_limit"
-#define FDW_OPTION_REJECT_LIMIT_TYPE "reject_limit_type"
-#define FDW_OPTION_RESOURCE "resource"
-
-#define FDW_COPY_OPTION_FORMAT "format"
-#define FDW_COPY_OPTION_HEADER "header"
-#define FDW_COPY_OPTION_DELIMITER "delimiter"
-#define FDW_COPY_OPTION_QUOTE "quote"
-#define FDW_COPY_OPTION_ESCAPE "escape"
-#define FDW_COPY_OPTION_NULL "null"
-#define FDW_COPY_OPTION_ENCODING "encoding"
-#define FDW_COPY_OPTION_NEWLINE "newline"
-#define FDW_COPY_OPTION_FILL_MISSING_FIELDS "fill_missing_fields"
-#define FDW_COPY_OPTION_FORCE_NOT_NULL "force_not_null"
-#define FDW_COPY_OPTION_FORCE_NULL "force_null"
-
-/*
- * Describes the valid copy options for objects that use this wrapper.
- */
-struct PxfFdwOption
-{
- const char *optname;
- Oid optcontext; /* Oid of catalog in which option may appear */
-};
-
-static const struct PxfFdwOption valid_options[] = {
- {FDW_OPTION_PROTOCOL, ForeignDataWrapperRelationId},
- {FDW_OPTION_RESOURCE, ForeignTableRelationId},
- {FDW_OPTION_FORMAT, ForeignTableRelationId},
- {FDW_OPTION_CONFIG, ForeignServerRelationId},
-
- /* Error handling */
- {FDW_OPTION_REJECT_LIMIT, ForeignTableRelationId},
- {FDW_OPTION_REJECT_LIMIT_TYPE, ForeignTableRelationId},
- {FDW_OPTION_LOG_ERRORS, ForeignTableRelationId},
-
- /* Sentinel */
- {NULL, InvalidOid}
-};
-
-/*
- * Valid COPY options for *_pxf_fdw.
- * These options are based on the options for the COPY FROM command.
- * But note that force_not_null and force_null are handled as boolean options
- * attached to a column, not as table options.
- *
- * Note: If you are adding new option for user mapping, you need to modify
- * fileGetOptions(), which currently doesn't bother to look at user mappings.
- */
-static const struct PxfFdwOption valid_copy_options[] = {
- /* Format options */
- /* oids option is not supported */
- /* freeze option is not supported */
- {FDW_OPTION_FORMAT, ForeignTableRelationId},
- {FDW_COPY_OPTION_HEADER, ForeignTableRelationId},
- {FDW_COPY_OPTION_DELIMITER, ForeignTableRelationId},
- {FDW_COPY_OPTION_QUOTE, ForeignTableRelationId},
- {FDW_COPY_OPTION_ESCAPE, ForeignTableRelationId},
- {FDW_COPY_OPTION_NULL, ForeignTableRelationId},
- {FDW_COPY_OPTION_ENCODING, ForeignTableRelationId},
- {FDW_COPY_OPTION_NEWLINE, ForeignTableRelationId},
- {FDW_COPY_OPTION_FILL_MISSING_FIELDS, ForeignTableRelationId},
- {FDW_COPY_OPTION_FORCE_NOT_NULL, AttributeRelationId},
- {FDW_COPY_OPTION_FORCE_NULL, AttributeRelationId},
-
- /* Sentinel */
- {NULL, InvalidOid}
-};
-
-extern Datum pxf_fdw_validator(PG_FUNCTION_ARGS);
-
-/*
- * SQL functions
- */
-PG_FUNCTION_INFO_V1(pxf_fdw_validator);
-
-/*
- * Helper functions
- */
-static Datum ValidateCopyOptions(List *options_list, Oid catalog);
-
-static bool IsCopyOption(const char *option);
-
-static bool IsValidCopyOption(const char *option, Oid context);
-
-static void ValidateOption(char *, Oid);
-
-/*
- * Validate the generic options given to a FOREIGN DATA WRAPPER, SERVER,
- * USER MAPPING or FOREIGN TABLE that uses file_fdw.
- *
- * Raise an ERROR if the option or its value is considered invalid.
- *
- */
-Datum
-pxf_fdw_validator(PG_FUNCTION_ARGS)
-{
- char *protocol = NULL;
- char *resource = NULL;
- char *reject_limit_type = FDW_OPTION_REJECT_LIMIT_ROWS;
- bool log_errors_set = false;
- List *options_list = untransformRelOptions(PG_GETARG_DATUM(0));
- Oid catalog = PG_GETARG_OID(1);
- List *copy_options = NIL;
- ListCell *cell;
- int reject_limit = -1,
- pxf_port;
-
- foreach(cell, options_list)
- {
- DefElem *def = (DefElem *) lfirst(cell);
-
- /*
- * check whether option is valid at it's catalog level, if not valid,
- * error out
- */
- ValidateOption(def->defname, catalog);
-
- if (strcmp(def->defname, FDW_OPTION_PROTOCOL) == 0)
- protocol = defGetString(def);
- else if (strcmp(def->defname, FDW_OPTION_RESOURCE) == 0)
- resource = defGetString(def);
- else if (strcmp(def->defname, FDW_OPTION_MPP_EXECUTE) == 0)
- {
- if (catalog == UserMappingRelationId)
- ereport(ERROR,
- (errcode(ERRCODE_FDW_INVALID_OPTION_NAME),
- errmsg("the %s option cannot be defined at the user mapping level",
- FDW_OPTION_MPP_EXECUTE)));
- }
- else if (strcmp(def->defname, FDW_OPTION_FORMAT) == 0)
- {
- /*
- * Format option in PXF is different from the COPY format option.
- * In PXF, format refers to the file format on the external
- * system, for example Parquet, Avro, Text, CSV.
- *
- * For COPY, the format can only be text, csv, or binary. pxf_fdw
- * leverages the csv and text formats in COPY.
- */
- char *value = defGetString(def);
-
- if (pg_strcasecmp(value, FDW_OPTION_WIRE_FORMAT_TEXT) == 0 ||
- pg_strcasecmp(value, FDW_OPTION_WIRE_FORMAT_CSV) == 0)
- copy_options = lappend(copy_options, def);
- }
- else if (strcmp(def->defname, FDW_OPTION_PXF_PORT) == 0)
- {
- pxf_port = atoi(defGetString(def));
-
- /*
- * TODO: a PXF service can be running on port 80 behind a load-
- * balancer we need to remove this restriction (i.e. kubernetes)
- */
- if (pxf_port < 1024 || pxf_port > 65535)
- ereport(ERROR,
- (errcode(ERRCODE_FDW_INVALID_STRING_FORMAT),
- errmsg("invalid port number: %d. valid port numbers are 1024 to 65535", pxf_port)));
- }
- else if (strcmp(def->defname, FDW_OPTION_REJECT_LIMIT) == 0)
- {
- char *endptr = NULL;
- char *pStr = defGetString(def);
-
- reject_limit = (int) strtol(pStr, &endptr, 10);
-
- if (pStr == endptr || reject_limit < 1)
- ereport(ERROR,
- (errcode(ERRCODE_FDW_INVALID_STRING_FORMAT),
- errmsg("invalid %s value '%s', should be a positive integer", FDW_OPTION_REJECT_LIMIT, pStr)));
- }
- else if (strcmp(def->defname, FDW_OPTION_REJECT_LIMIT_TYPE) == 0)
- {
- reject_limit_type = defGetString(def);
- if (pg_strcasecmp(reject_limit_type,
- FDW_OPTION_REJECT_LIMIT_ROWS) != 0 &&
- pg_strcasecmp(reject_limit_type,
- FDW_OPTION_REJECT_LIMIT_PERCENT) != 0)
- ereport(ERROR,
- (errcode(ERRCODE_FDW_INVALID_STRING_FORMAT),
- errmsg("invalid %s value, only '%s' and '%s' are supported",
- FDW_OPTION_REJECT_LIMIT_TYPE,
- FDW_OPTION_REJECT_LIMIT_ROWS,
- FDW_OPTION_REJECT_LIMIT_PERCENT)));
- }
- else if (strcmp(def->defname, FDW_OPTION_LOG_ERRORS) == 0)
- {
- (void) defGetBoolean(def); /* call is required for validation */
- log_errors_set = true;
- }
- else if (IsCopyOption(def->defname))
- copy_options = lappend(copy_options, def);
- }
-
- if (catalog == ForeignDataWrapperRelationId &&
- (protocol == NULL || strcmp(protocol, "") == 0))
- {
- ereport(ERROR,
- (errcode(ERRCODE_FDW_DYNAMIC_PARAMETER_VALUE_NEEDED),
- errmsg("the %s option must be defined for PXF foreign-data wrappers", FDW_OPTION_PROTOCOL)));
- }
-
- if (catalog == ForeignTableRelationId &&
- (resource == NULL || strcmp(resource, "") == 0))
- {
- ereport(ERROR,
- (errcode(ERRCODE_FDW_DYNAMIC_PARAMETER_VALUE_NEEDED),
- errmsg("the %s option must be defined at the foreign table level", FDW_OPTION_RESOURCE)));
- }
-
- /* Validate reject limit */
- if (reject_limit != -1)
- {
- if (pg_strcasecmp(reject_limit_type, FDW_OPTION_REJECT_LIMIT_ROWS) == 0)
- {
- if (reject_limit < 2)
- ereport(ERROR,
- (errcode(ERRCODE_FDW_INVALID_STRING_FORMAT),
- errmsg("invalid (ROWS) %s value '%d', valid values are 2 or larger",
- FDW_OPTION_REJECT_LIMIT,
- reject_limit)));
- }
- else
- {
- if (reject_limit < 1 || reject_limit > 100)
- ereport(ERROR,
- (errcode(ERRCODE_FDW_INVALID_STRING_FORMAT),
- errmsg("invalid (PERCENT) %s value '%d', valid values are 1 to 100",
- FDW_OPTION_REJECT_LIMIT,
- reject_limit)));
- }
- }
- else
- {
- if (log_errors_set)
- ereport(ERROR,
- (errcode(ERRCODE_FDW_INVALID_STRING_FORMAT),
- errmsg("the %s option cannot be set without reject_limit", FDW_OPTION_LOG_ERRORS)));
- }
-
- /*
- * Additional validations for Copy options
- */
- ValidateCopyOptions(copy_options, catalog);
-
- PG_RETURN_VOID();
-}
-
-/*
- * Validate options for the Copy command. Postgresql has its own validation
- * for the copy options. We only do special validation for force_not_null
- * and force_null, because they are set at the attribute level.
- */
-Datum
-ValidateCopyOptions(List *options_list, Oid catalog)
-{
- DefElem *force_not_null = NULL;
- DefElem *force_null = NULL;
- List *copy_options = NIL;
- ListCell *cell;
-
- /*
- * Check that only options supported by copy, and allowed for the current
- * object type, are given.
- */
- foreach(cell, options_list)
- {
- DefElem *def = (DefElem *) lfirst(cell);
-
- if (!IsValidCopyOption(def->defname, catalog))
- {
- const struct PxfFdwOption *opt;
- StringInfoData buf;
-
- /*
- * Unknown option specified, complain about it. Provide a hint
- * with list of valid options for the object.
- */
- initStringInfo(&buf);
- for (opt = valid_copy_options; opt->optname; opt++)
- {
- if (catalog == opt->optcontext)
- appendStringInfo(&buf, "%s%s", (buf.len > 0) ? ", " : "",
- opt->optname);
- }
-
- ereport(ERROR,
- (errcode(ERRCODE_FDW_INVALID_OPTION_NAME),
- errmsg("invalid option \"%s\"", def->defname),
- buf.len > 0
- ? errhint("Valid options in this context are: %s", buf.data)
- : errhint("There are no valid options in this context.")));
- }
-
- /*
- * force_not_null is a boolean option; after validation we can discard
- * it - it will be retrieved later in PxfGetOptions()
- */
- if (strcmp(def->defname, FDW_COPY_OPTION_FORCE_NOT_NULL) == 0)
- {
- if (force_not_null)
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("conflicting or redundant options"),
- errhint("option \"%s\" supplied more than once for a column", FDW_COPY_OPTION_FORCE_NOT_NULL)));
- force_not_null = def;
- /* Don't care what the value is, as long as it's a legal boolean */
- (void) defGetBoolean(def);
- }
- /* See comments for force_not_null above */
- else if (strcmp(def->defname, FDW_COPY_OPTION_FORCE_NULL) == 0)
- {
- if (force_null)
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("conflicting or redundant options"),
- errhint("option \"%s\" supplied more than once for a column", FDW_COPY_OPTION_FORCE_NULL)));
- force_null = def;
- (void) defGetBoolean(def);
- }
- else
- copy_options = lappend(copy_options, def);
- }
-
- /*
- * Apply the core COPY code's validation logic for more checks.
- */
-#if (PG_VERSION_NUM < 140000)
- ProcessCopyOptions(NULL, NULL, true, copy_options);
-#else
- ProcessCopyOptions(NULL, NULL, true, copy_options, InvalidOid);
-#endif
-
- PG_RETURN_VOID();
-}
-
-/*
- * Fetch the options for a pxf_fdw foreign table.
- */
-PxfOptions *
-PxfGetOptions(Oid foreigntableid)
-{
- Node *wireFormat;
- UserMapping *user;
- ForeignTable *table;
- ForeignServer *server;
- ForeignDataWrapper *wrapper;
- List *options;
- PxfOptions *opt;
- ListCell *lc;
- List *copy_options,
- *other_options,
- *other_option_name_strings = NULL;
-
- opt = (PxfOptions *) palloc(sizeof(PxfOptions));
- memset(opt, 0, sizeof(PxfOptions));
-
- copy_options = NIL;
- other_options = NIL;
-
- opt->reject_limit = -1;
- opt->is_reject_limit_rows = true;
- opt->log_errors = false;
-
- /*
- * Extract options from FDW objects.
- */
- table = GetForeignTable(foreigntableid);
- server = GetForeignServer(table->serverid);
- user = GetUserMapping(GetUserId(), server->serverid);
- wrapper = GetForeignDataWrapper(server->fdwid);
-
- options = NIL;
- /* order matters here for precedence enforcement */
- options = list_concat(options, table->options);
- options = list_concat(options, user->options);
- options = list_concat(options, server->options);
- options = list_concat(options, wrapper->options);
-
- /* Loop through the options, and get the server/port */
- foreach(lc, options)
- {
- DefElem *def = (DefElem *) lfirst(lc);
-
- if (strcmp(def->defname, FDW_OPTION_PXF_HOST) == 0)
- opt->pxf_host = defGetString(def);
-
- else if (strcmp(def->defname, FDW_OPTION_PXF_PORT) == 0)
- opt->pxf_port = atoi(defGetString(def));
- else if (strcmp(def->defname, FDW_OPTION_PXF_PROTOCOL) == 0)
- opt->pxf_protocol = defGetString(def);
- else if (strcmp(def->defname, FDW_OPTION_PROTOCOL) == 0)
- opt->protocol = defGetString(def);
- else if (strcmp(def->defname, FDW_OPTION_RESOURCE) == 0)
- opt->resource = defGetString(def);
- else if (strcmp(def->defname, FDW_OPTION_REJECT_LIMIT) == 0)
- opt->reject_limit = atoi(defGetString(def));
- else if (strcmp(def->defname, FDW_OPTION_REJECT_LIMIT_TYPE) == 0)
- opt->is_reject_limit_rows = pg_strcasecmp(FDW_OPTION_REJECT_LIMIT_ROWS, defGetString(def)) == 0;
- else if (strcmp(def->defname, FDW_OPTION_LOG_ERRORS) == 0)
- opt->log_errors = defGetBoolean(def);
- else if (strcmp(def->defname, FDW_OPTION_FORMAT) == 0)
- {
- opt->format = defGetString(def);
- }
- else if (IsCopyOption(def->defname))
- copy_options = lappend(copy_options, def);
- else
- {
- String *val = makeString(def->defname);
-
- /*
- * if we have already seen this option before disregard the new
- * value. We only take the first value that we see. And the
- * precedence is table -> user mapping -> server -> wrapper
- */
- if (list_member(other_option_name_strings, val))
- continue;
- other_options = lappend(other_options, def);
- other_option_name_strings = lappend(other_option_name_strings, val);
- }
- } /* foreach */
-
- /* The profile corresponds to protocol[:format] */
- opt->profile = opt->protocol;
-
- if (opt->format)
- opt->profile = psprintf("%s:%s", opt->protocol, opt->format);
-
- if (opt->format && pg_strcasecmp(opt->format, FDW_OPTION_WIRE_FORMAT_TEXT) == 0)
- wireFormat = (Node *)makeString(FDW_OPTION_WIRE_FORMAT_TEXT);
- else
- /* default wire_format is CSV */
- wireFormat = (Node *)makeString(FDW_OPTION_WIRE_FORMAT_CSV);
-
- copy_options = lappend(copy_options, makeDefElem(FDW_COPY_OPTION_FORMAT, wireFormat, -1));
-
- opt->copy_options = copy_options;
- opt->options = other_options;
-
- opt->server = server->servername;
-
- /* Follows precedence rules table > server > wrapper */
- opt->exec_location = table->exec_location;
-
- /* Set defaults when not provided */
- if (!opt->pxf_host)
- opt->pxf_host = PXF_FDW_DEFAULT_HOST;
-
- if (!opt->pxf_port)
- opt->pxf_port = PXF_FDW_DEFAULT_PORT;
-
- if (!opt->pxf_protocol)
- opt->pxf_protocol = PXF_FDW_DEFAULT_PROTOCOL;
-
- return opt;
-}
-
-/*
- * Check if the provided option is one of the valid options.
- * context is the Oid of the catalog holding the object the option is for.
- */
-static bool
-IsValidCopyOption(const char *option, Oid context)
-{
- const struct PxfFdwOption *entry;
-
- for (entry = valid_copy_options; entry->optname; entry++)
- {
- if (context == entry->optcontext && strcmp(entry->optname, option) == 0)
- return true;
- }
- return false;
-}
-
-/*
- * Check if the option is a COPY option
- */
-static bool
-IsCopyOption(const char *option)
-{
- const struct PxfFdwOption *entry;
-
- for (entry = valid_copy_options; entry->optname; entry++)
- {
- if (strcmp(entry->optname, option) == 0)
- return true;
- }
- return false;
-}
-
-/*
- * Goes through standard list of options to make sure option is defined at the correct catalog level
- */
-static void
-ValidateOption(char *option, Oid catalog)
-{
- const struct PxfFdwOption *entry;
-
- for (entry = valid_options; entry->optname; entry++)
- {
- /* option can only be defined at its catalog level */
- if (strcmp(entry->optname, option) == 0 && catalog != entry->optcontext)
- {
- Relation rel = RelationIdGetRelation(entry->optcontext);
-
- ereport(ERROR,
- (errcode(ERRCODE_FDW_INVALID_OPTION_NAME),
- errmsg(
- "the %s option can only be defined at the %s level",
- option,
- RelationGetRelationName(rel))));
- }
- }
-}
diff --git a/gpcontrib/pxf_fdw/sql/pxf_fdw_foreign_table.sql b/gpcontrib/pxf_fdw/sql/pxf_fdw_foreign_table.sql
deleted file mode 100644
index c477040640b..00000000000
--- a/gpcontrib/pxf_fdw/sql/pxf_fdw_foreign_table.sql
+++ /dev/null
@@ -1,356 +0,0 @@
--- start_matchsubs
--- m/ERROR: COPY quote available only in CSV mode/
--- s/ERROR: COPY quote available only in CSV mode/ERROR: quote available only in CSV mode/
--- end_matchsubs
--- ===================================================================
--- Validation for TABLE options
--- ===================================================================
-
---
--- Table creation fails if protocol option is provided
---
-CREATE FOREIGN TABLE pxf_fdw_test_table (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( protocol 'pxf_fdw_test2' );
-
---
--- Table creation fails if resource is not provided
---
-CREATE FOREIGN TABLE pxf_fdw_test_table (id int, name text)
- SERVER pxf_fdw_test_server;
-
---
--- Table creation fails if resource is provided as an empty string
---
-CREATE FOREIGN TABLE pxf_fdw_test_table (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '' );
-
---
--- Table creation fails if config option is provided
---
-CREATE FOREIGN TABLE pxf_fdw_test_table ()
- SERVER pxf_fdw_test_server
- OPTIONS ( config '/foo/bar' );
-
---
--- Table creation succeeds if delimiter is provided with a single one-byte char
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_delim (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', delimiter '|' );
-
---
--- Table creation succeeds if delimiter is provided with a single one-byte char
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_quote (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', format 'csv', quote '`' );
-
---
--- Table creation succeeds if escape is provided with a single one-byte char
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_escape (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', format 'csv', escape '\' );
-
---
--- Table creation succeeds if null is provided with a single one-byte char
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_null (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', null '' );
-
---
--- Table creation succeeds if encoding is provided with a single one-byte char
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_encoding (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', encoding 'UTF-8' );
-
---
--- Table creation succeeds if newline is valid
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_newline (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', newline 'CRLF' );
-
---
--- Table creation succeeds if resource and fill_missing_fields are provided
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_fill_missing_fields (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', fill_missing_fields 'true');
-
---
--- Table creation succeeds if force_null is provided on column definition
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_force_null (
- id int OPTIONS (force_null 'true'),
- name text OPTIONS (force_null 'false')
- ) SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource' );
-
---
--- Table creation succeeds if force_not_null is provided on column definition
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_force_not_null (
- id int OPTIONS (force_not_null 'true'),
- name text OPTIONS (force_not_null 'false')
- ) SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource' );
-
---
--- Table creation succeeds if resource is provided and header is valid
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_with_header (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', header 'TRUE' );
-
---
--- Table creation succeeds if resource is provided and protocol is not provided
---
-CREATE FOREIGN TABLE pxf_fdw_test_table (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', extra 'extra_value' );
-
---
--- Table creation fails if reject_limit is < 1
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( reject_limit '-4' );
-
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( reject_limit '0' );
-
---
--- Table creation fails if reject_limit_type is not 'percent' or 'rows'
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit_type (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( reject_limit_type 'not-percent-or-rows' );
-
---
--- Table creation fails if reject_limit is non-numeric
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit 'no-number-here' );
-
---
--- Table creation fails if reject_limit_type is 'rows' and reject_limit < 2
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit_type (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit '1', reject_limit_type 'rows' );
-
---
--- Table creation fails if reject_limit_type is 'percent' and not between 1-100
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit_type (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit '-1', reject_limit_type 'percent' );
-
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit_type (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit '101', reject_limit_type 'percent' );
-
---
--- Table creation fails if only resource and log errors are set
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_log_errors (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', log_errors 'true' );
-
---
--- Table creation fails if only log_errors is non-boolean
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_log_errors (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit '4', log_errors 'yes' );
-
---
--- Table creation fails if quote is provided (CSV-only option)
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_csv_only ()
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/foo', quote '9' );
-
---
--- Table creation succeeds if resource is provided and reject_limit is provided correctly
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit '4' );
-
---
--- Table creation succeeds if resource is provided and reject_limit_type is provided correctly
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_reject_limit_type (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit_type 'rows', reject_limit '4' );
-
---
--- Table create succeeds if resource, reject_limit, and log_errors are provided correctly
---
-CREATE FOREIGN TABLE pxf_fdw_test_table_log_errors (id int, name text)
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/path/to/resource', reject_limit '4', log_errors 'true' );
-
---
--- Table alteration fails if protocol option is added
---
-ALTER FOREIGN TABLE pxf_fdw_test_table
- OPTIONS ( ADD protocol 'table_protocol' );
-
---
--- Table alteration fails if resource option is dropped
---
-ALTER FOREIGN TABLE pxf_fdw_test_table
- OPTIONS ( DROP resource );
-
---
--- Table alteration fails if resource is provided as an empty string
---
-ALTER FOREIGN TABLE pxf_fdw_test_table
- OPTIONS ( SET resource '' );
-
---
--- Table alteration succeeds if resource option is set
---
-ALTER FOREIGN TABLE pxf_fdw_test_table
- OPTIONS ( SET resource '/new/path/to/resource' );
-
---
--- Table alteration succeeds if header is provided correctly
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_header
- OPTIONS ( SET header 'FALSE' );
-
---
--- Table alteration succeeds if delimiter is a single one-byte char
---
-ALTER FOREIGN TABLE pxf_fdw_test_table
- OPTIONS ( ADD delimiter ',' );
-
---
--- Table alteration succeeds if quote is a single one-byte char
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_quote
- OPTIONS ( SET quote '"' );
-
---
--- Table alteration succeeds if escape is a single one-byte char
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_escape
- OPTIONS ( SET escape '+' );
-
---
--- Table alteration succeeds if null is a single one-byte char
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_null
- OPTIONS ( SET null 'null' );
-
---
--- Table alteration succeeds if encoding is a single one-byte char
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_encoding
- OPTIONS ( SET encoding 'ISO-8859-1' );
-
---
--- Table alteration succeeds if newline is valid
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_newline
- OPTIONS ( SET newline 'CRLF' );
-
---
--- Table alteration succeeds when fill_missing_fields is provided
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_fill_missing_fields
- OPTIONS ( SET fill_missing_fields 'true' );
-
---
--- Table alteration succeeds when force_null is provided on column definition
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_force_null
- ALTER COLUMN id OPTIONS (DROP force_null);
-
---
--- Table alteration succeeds when force_not_null is provided on column definition
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_with_force_not_null
- ALTER COLUMN id OPTIONS (DROP force_not_null);
-
---
--- Table alteration fails if reject_limit is less than 1
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit
- OPTIONS ( SET reject_limit '-4' );
-
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit
- OPTIONS ( SET reject_limit '0' );
-
---
--- Table alteration fails if reject_limit_type is not 'percent' or 'rows'
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit_type
- OPTIONS ( SET reject_limit_type 'not-percent-or-rows' );
-
---
--- Table alteration fails if reject_limit is non-numeric
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit
- OPTIONS ( SET reject_limit 'no-number-here' );
-
---
--- Table alteration fails if reject_limit_type is rows and reject_limit is < 2
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit_type
- OPTIONS ( SET reject_limit '1', SET reject_limit_type 'rows' );
-
---
--- Table alteration fails if reject_limit_type is percent and reject_limit is not within 1-100
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit_type
- OPTIONS ( SET reject_limit '-1', SET reject_limit_type 'percent' );
-
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit_type
- OPTIONS ( SET reject_limit '101', SET reject_limit_type 'percent' );
-
---
--- Table alteration succeeds if resource is provided and reject_limit is provided correctly
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit
- OPTIONS ( SET reject_limit '5' );
-
---
--- Table alteration succeeds if resource is provided and reject_limit_type is provided correctly
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_reject_limit_type
- OPTIONS ( SET reject_limit_type 'percent' );
-
---
--- Table alteration succeeds for log_errors if table has valid resource and reject_limit
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_log_errors
- OPTIONS ( SET log_errors 'false' );
-
---
--- Table alteration fails for log_errors if table has valid resource but no reject_limit
---
-ALTER FOREIGN TABLE pxf_fdw_test_table
- OPTIONS ( ADD log_errors 'false' );
-
---
--- Table alteration fails for log_errors if log_errors is non-boolean
---
-ALTER FOREIGN TABLE pxf_fdw_test_table_log_errors
- OPTIONS ( SET log_errors 'no' );
-
---
--- Table alteration fails if config option is provided
---
-ALTER FOREIGN TABLE pxf_fdw_test_table
- OPTIONS ( ADD config '/foo/bar' );
diff --git a/gpcontrib/pxf_fdw/sql/pxf_fdw_server.sql b/gpcontrib/pxf_fdw/sql/pxf_fdw_server.sql
deleted file mode 100644
index 748a262a961..00000000000
--- a/gpcontrib/pxf_fdw/sql/pxf_fdw_server.sql
+++ /dev/null
@@ -1,230 +0,0 @@
--- ===================================================================
--- Validation for SERVER options
--- ===================================================================
-
---
--- Server creation fails if protocol option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( protocol 'pxf_fdw_test2' );
-
---
--- Server creation fails if resource option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( resource '/invalid/option/for/server' );
-
---
--- Server creation fails if delimiter option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( delimiter ' ' );
-
---
--- Server creation fails if header option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( header 'TRUE' );
-
---
--- Server creation fails if quote option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( quote '`' );
-
---
--- Server creation fails if escape option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( escape '\' );
-
---
--- Server creation fails if null option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( null '' );
-
---
--- Server creation fails if encoding option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( encoding 'UTF-8' );
-
---
--- Server creation fails if newline option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( newline 'CRLF' );
-
---
--- Server creation fails if fill_missing_fields option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( fill_missing_fields '' );
-
---
--- Server creation fails if force_null option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( force_null 'true' );
-
---
--- Server creation fails if force_not_null option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( force_not_null 'true' );
-
---
--- Server creation fails if reject_limit option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( reject_limit '5' );
-
---
--- Server creation fails if reject_limit_type option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( reject_limit_type 'rows' );
-
---
--- Server creation fails if log_errors option is provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( log_errors 'true' );
-
---
--- Server creation succeeds if protocol option is not provided
---
-CREATE SERVER pxf_fdw_test_server
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw;
-
---
--- Server creation succeeds if config option is provided
---
-CREATE SERVER pxf_fdw_test_server_with_config
- FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( config '/foo/bar' );
-
---
--- Server alteration fails if protocol option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD protocol 'pxf_fdw_test2' );
-
---
--- Server alteration fails if resource option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD resource '/invalid/option/for/server' );
-
---
--- Server alteration fails if header option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD header 'TRUE' );
-
---
--- Server alteration fails if delimiter option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD delimiter ' ' );
-
---
--- Server alteration fails if quote option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD quote '`' );
-
---
--- Server alteration fails if escape option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD escape '\' );
-
---
--- Server alteration fails if null option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD null '' );
-
---
--- Server alteration fails if encoding option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD encoding 'UTF-8' );
-
---
--- Server alteration fails if newline option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD newline 'CRLF' );
-
---
--- Server alteration fails if fill_missing_fields option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD fill_missing_fields '' );
-
---
--- Server alteration fails if force_null option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD force_null 'true' );
-
---
--- Server alteration fails if force_not_null option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD force_not_null 'true' );
-
---
--- Server alteration fails if reject_limit option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD reject_limit '5' );
-
---
--- Server alteration fails if reject_limit_type option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD reject_limit_type 'rows' );
-
---
--- Server alteration fails if log_errors option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD log_errors 'true' );
-
---
--- Server alteration succeeds if config option is added
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( ADD config '/foo/bar' );
-
---
--- Server alteration succeeds if config option is modified
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( SET config '/foo/bar' );
-
---
--- Server alteration succeeds if config option is dropped
---
-ALTER SERVER pxf_fdw_test_server
- OPTIONS ( DROP config );
-
diff --git a/gpcontrib/pxf_fdw/sql/pxf_fdw_user_mapping.sql b/gpcontrib/pxf_fdw/sql/pxf_fdw_user_mapping.sql
deleted file mode 100644
index 4f2081cbcd0..00000000000
--- a/gpcontrib/pxf_fdw/sql/pxf_fdw_user_mapping.sql
+++ /dev/null
@@ -1,248 +0,0 @@
--- ===================================================================
--- Validation for USER MAPPING options
--- ===================================================================
-
---
--- User mapping creation fails if protocol option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( protocol 'usermappingprotocol' );
-
---
--- User mapping creation fails if resource option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( resource '/invalid/option/for/user/mapping' );
-
---
--- User mapping creation fails if header option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( header 'TRUE' );
-
---
--- User mapping creation fails if delimiter option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( delimiter ' ' );
-
---
--- User mapping creation fails if quote option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( quote '`' );
-
---
--- User mapping creation fails if escape option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( escape '\' );
-
---
--- User mapping creation fails if null option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( null '' );
-
---
--- User mapping creation fails if encoding option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( encoding 'UTF-8' );
-
---
--- User mapping creation fails if newline option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( newline 'CRLF' );
-
---
--- User mapping creation fails if fill_missing_fields option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( fill_missing_fields '' );
-
---
--- User mapping creation fails if force_null option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( force_null 'true' );
-
---
--- User mapping creation fails if force_not_null option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( force_not_null 'true' );
-
---
--- User mapping creation fails if reject_limit option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( reject_limit '5' );
-
---
--- User mapping creation fails if reject_limit_type option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( reject_limit_type 'rows' );
-
---
--- User mapping creation fails if log_errors option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( log_errors 'true' );
-
---
--- User mapping creation fails if mpp_execute option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( mpp_execute 'any' );
-
---
--- User mapping creation fails if config option is provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( config '/foo/bar' );
-
---
--- User mapping creation succeeds if protocol option is not provided
---
-CREATE USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server;
-
---
--- User mapping alteration fails if protocol option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD protocol 'usermappingprotocol' );
-
---
--- User mapping alteration fails if resource option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD resource '/invalid/option/for/user/mapping' );
-
---
--- User mapping alteration fails if header option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD header 'TRUE' );
-
---
--- User mapping alteration fails if delimiter option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD delimiter ' ' );
-
---
--- User mapping alteration fails if quote option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD quote '`' );
-
---
--- User mapping alteration fails if escape option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD escape '\' );
-
---
--- User mapping alteration fails if null option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD null '' );
-
---
--- User mapping alteration fails if encoding option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD encoding 'UTF-8' );
-
---
--- User mapping alteration fails if newline option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD newline 'CRLF' );
-
---
--- User mapping alteration fails if fill_missing_fields option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD fill_missing_fields '' );
-
---
--- User mapping alteration fails if force_null option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD force_null 'true' );
-
---
--- User mapping alteration fails if force_not_null option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD force_not_null 'true' );
-
---
--- User mapping alteration fails if reject_limit option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD reject_limit '5' );
-
---
--- User mapping alteration fails if reject_limit_type option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD reject_limit_type 'rows' );
-
---
--- User mapping alteration fails if log_errors option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD log_errors 'true' );
-
---
--- User mapping alteration fails if mpp_execute option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD mpp_execute 'any' );
-
---
--- User mapping alteration fails if config option is added
---
-ALTER USER MAPPING FOR pxf_fdw_user
- SERVER pxf_fdw_test_server
- OPTIONS ( ADD config '/foo/bar' );
-
diff --git a/gpcontrib/pxf_fdw/sql/pxf_fdw_wrapper.sql b/gpcontrib/pxf_fdw/sql/pxf_fdw_wrapper.sql
deleted file mode 100644
index e6fa98882ce..00000000000
--- a/gpcontrib/pxf_fdw/sql/pxf_fdw_wrapper.sql
+++ /dev/null
@@ -1,257 +0,0 @@
--- ===================================================================
--- create FDW objects
--- ===================================================================
-
-CREATE EXTENSION pxf_fdw;
-
-DROP ROLE IF EXISTS pxf_fdw_user;
-CREATE ROLE pxf_fdw_user;
-
--- ===================================================================
--- Validation for WRAPPER options
--- ===================================================================
-
---
--- Foreign-data wrapper creation fails if protocol option is not provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator;
-
---
--- Foreign-data wrapper creation fails if protocol option is empty
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol '' );
-
---
--- Foreign-data wrapper creation fails if resource option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( resource '/invalid/option/for/wrapper' );
-
---
--- Foreign-data wrapper creation fails if header option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', header 'TRUE' );
-
---
--- Foreign-data wrapper creation fails if delimiter option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', delimiter ' ' );
-
---
--- Foreign-data wrapper creation fails if quote option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', quote '`' );
-
---
--- Foreign-data wrapper creation fails if escape option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', escape '\' );
-
---
--- Foreign-data wrapper creation fails if null option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', null '' );
-
---
--- Foreign-data wrapper creation fails if encoding option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', encoding 'UTF-8' );
-
---
--- Foreign-data wrapper creation fails if newline option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', newline 'CRLF' );
-
---
--- Foreign-data wrapper creation fails if fill_missing_fields option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', fill_missing_fields '' );
-
---
--- Foreign-data wrapper creation fails if force_null option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', force_null 'true' );
-
---
--- Foreign-data wrapper creation fails if force_not_null option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', force_not_null 'true' );
-
---
--- Foreign-data wrapper creation fails if reject_limit option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', reject_limit '5' );
-
---
--- Foreign-data wrapper creation fails if reject_limit_type option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', reject_limit_type 'rows' );
-
---
--- Foreign-data wrapper creation fails if log_errors option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', log_errors 'true' );
-
---
--- Foreign-data wrapper creation fails if config option is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', config '/foo/bar' );
-
---
--- Foreign-data wrapper succeeds when protocol is provided
---
-CREATE FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- HANDLER pxf_fdw_handler
- VALIDATOR pxf_fdw_validator
- OPTIONS ( protocol 'pxf_fdw_test', mpp_execute 'all segments' );
-
---
--- Foreign-data wrapper alteration fails when protocol is dropped
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( DROP protocol );
-
---
--- Foreign-data wrapper alteration fails if protocol option is empty
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( SET protocol '' );
-
---
--- Foreign-data wrapper alteration fails if resource option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD resource '/invalid/option/for/wrapper' );
-
---
--- Foreign-data wrapper alteration fails if header option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD header 'TRUE' );
-
---
--- Foreign-data wrapper alteration fails if delimiter option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD delimiter ' ' );
-
---
--- Foreign-data wrapper alteration fails if quote option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD quote '`' );
-
---
--- Foreign-data wrapper alteration fails if escape option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD escape '\' );
-
---
--- Foreign-data wrapper alteration fails if null option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD null '' );
-
---
--- Foreign-data wrapper alteration fails if encoding option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD encoding 'UTF-8' );
-
---
--- Foreign-data wrapper alteration fails if newline option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD newline 'CRLF' );
-
---
--- Foreign-data wrapper alteration fails if fill_missing_fields option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD fill_missing_fields '' );
-
---
--- Foreign-data wrapper alteration fails if force_null option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD force_null 'true' );
-
---
--- Foreign-data wrapper alteration fails if force_not_null option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD force_not_null 'true' );
-
---
--- Foreign-data wrapper alteration fails if reject_limit option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD reject_limit '5' );
-
---
--- Foreign-data wrapper alteration fails if reject_limit_type option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD reject_limit_type 'rows' );
-
---
--- Foreign-data wrapper alteration fails if log_errors option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD log_errors 'true' );
-
---
--- Foreign-data wrapper alteration fails if config option is added
---
-ALTER FOREIGN DATA WRAPPER pxf_fdw_test_pxf_fdw
- OPTIONS ( ADD config '/foo/bar' );
diff --git a/pom.xml b/pom.xml
index 85e5fbb8074..e1d3ed08633 100644
--- a/pom.xml
+++ b/pom.xml
@@ -87,13 +87,6 @@ code or new licensing patterns.
<exclude>gpcontrib/orafce/parallel_schedule</exclude>
<exclude>gpcontrib/orafce/.travis.yml</exclude>
- <exclude>gpcontrib/pxf_fdw/pxf_option.c</exclude>
- <exclude>gpcontrib/pxf_fdw/Makefile</exclude>
- <exclude>gpcontrib/pxf_fdw/pxf_fdw.h</exclude>
- <exclude>gpcontrib/pxf_fdw/pxf_fragment.c</exclude>
- <exclude>gpcontrib/pxf_fdw/pxf_fdw.control</exclude>
- <exclude>gpcontrib/pxf_fdw/pxf_deparse.c</exclude>
- <exclude>gpcontrib/pxf_fdw/pxf_fdw.c</exclude>
<exclude>gpcontrib/gp_inject_fault/Makefile</exclude>
<exclude>gpcontrib/gp_inject_fault/gp_inject_fault.control</exclude>
@@ -1378,8 +1371,6 @@ code or new licensing patterns.
<exclude>gpcontrib/pg_hint_plan/**/*.sql</exclude>
<exclude>gpcontrib/pg_hint_plan/**/*.out</exclude>
<exclude>gpcontrib/pg_hint_plan/sql/**/*.sql</exclude>
- <exclude>gpcontrib/pxf_fdw/**/*.sql</exclude>
- <exclude>gpcontrib/pxf_fdw/**/*.out</exclude>
<exclude>gpcontrib/zstd/**/*.sql</exclude>
<exclude>gpcontrib/zstd/**/*.out</exclude>
<exclude>src/bin/**/*.sql</exclude>
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index ee1c76e0276..5a05bc1a7e8 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -226,7 +226,6 @@ enable_orca = @enable_orca@
enable_catalog_ext = @enable_catalog_ext@
enable_serverless = @enable_serverless@
enable_gpfdist = @enable_gpfdist@
-enable_pxf = @enable_pxf@
enable_debug_extensions = @enable_debug_extensions@
enable_orafce = @enable_orafce@
enable_mapreduce = @enable_mapreduce@
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org