You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hawq.apache.org by Bhanu Joshi <Bh...@lntinfotech.com> on 2017/04/20 13:16:06 UTC
Query on Data Extraction from HAWQ
Hi.
I am new to HAWQ. I have a question.
Can you please tell me different ways I can extract the data from HAWQ to flat files?
I understand HAWQ also starts the data files in HDFS.
Regards,
Bhanu Joshi
RE: Query on Data Extraction from HAWQ
Posted by Jim Campbell <ja...@pivotal.io>.
You place a copy of gpfdist on the system that you want to write the file.
You will need to start the service and use the options as explained in the
documentation. On the server, you need to create a writable external table
with the correct options for your application. From there, it is an
insert/select statement.
James Campbell
Data Engineer
Pivotal Software
P: 571-247-6511
E: jacampbell@pivotal.io
On April 20, 2017 at 10:15:24 AM, Joshi Bhanu (bhanu.joshi@lntinfotech.com)
wrote:
Hi,
I need to move the data to another file system. Extract from HAWQ to
downstream 3rd party system.
So I can use gpfdist and it will extract the data and can be moved to an
sftp server.
Regards,
Bhanu
*From:* Jim Campbell [mailto:jacampbell@pivotal.io]
*Sent:* Thursday, April 20, 2017 7:17 PM
*To:* user@hawq.incubator.apache.org; Bhanu Joshi <
Bhanu.Joshi@lntinfotech.com>
*Subject:* Re: Query on Data Extraction from HAWQ
Do you want the flat file placed on HDFS, or do you want it on another file
system?
In both cases you can use writeable external tables. The documentation
provides examples of doing this multiple ways. Once you create an external
table, you do an insert-select to place the data into the external table.
If you want the file on HDFS, you will use one of the PXF options. On
another file system, you could use gpfdist.
Of course, you can always use the command line tool psql and select the
data into a file.
James Campbell
Data Engineer
Pivotal Software
P: 571-247-6511
E: jacampbell@pivotal.io
On April 20, 2017 at 9:16:21 AM, Joshi Bhanu (bhanu.joshi@lntinfotech.com)
wrote:
Hi.
I am new to HAWQ. I have a question.
Can you please tell me different ways I can extract the data from HAWQ to
flat files?
I understand HAWQ also starts the data files in HDFS.
Regards,
Bhanu Joshi
RE: Query on Data Extraction from HAWQ
Posted by Bhanu Joshi <Bh...@lntinfotech.com>.
Hi,
I need to move the data to another file system. Extract from HAWQ to downstream 3rd party system.
So I can use gpfdist and it will extract the data and can be moved to an sftp server.
Regards,
Bhanu
From: Jim Campbell [mailto:jacampbell@pivotal.io]
Sent: Thursday, April 20, 2017 7:17 PM
To: user@hawq.incubator.apache.org; Bhanu Joshi <Bh...@lntinfotech.com>
Subject: Re: Query on Data Extraction from HAWQ
Do you want the flat file placed on HDFS, or do you want it on another file system?
In both cases you can use writeable external tables. The documentation provides examples of doing this multiple ways. Once you create an external table, you do an insert-select to place the data into the external table. If you want the file on HDFS, you will use one of the PXF options. On another file system, you could use gpfdist.
Of course, you can always use the command line tool psql and select the data into a file.
James Campbell
Data Engineer
Pivotal Software
P: 571-247-6511
E: jacampbell@pivotal.io<ma...@pivotal.io>
On April 20, 2017 at 9:16:21 AM, Joshi Bhanu (bhanu.joshi@lntinfotech.com<ma...@lntinfotech.com>) wrote:
Hi.
I am new to HAWQ. I have a question.
Can you please tell me different ways I can extract the data from HAWQ to flat files?
I understand HAWQ also starts the data files in HDFS.
Regards,
Bhanu Joshi
Re: Query on Data Extraction from HAWQ
Posted by Jim Campbell <ja...@pivotal.io>.
Do you want the flat file placed on HDFS, or do you want it on another file
system?
In both cases you can use writeable external tables. The documentation
provides examples of doing this multiple ways. Once you create an external
table, you do an insert-select to place the data into the external table.
If you want the file on HDFS, you will use one of the PXF options. On
another file system, you could use gpfdist.
Of course, you can always use the command line tool psql and select the
data into a file.
James Campbell
Data Engineer
Pivotal Software
P: 571-247-6511
E: jacampbell@pivotal.io
On April 20, 2017 at 9:16:21 AM, Joshi Bhanu (bhanu.joshi@lntinfotech.com)
wrote:
Hi.
I am new to HAWQ. I have a question.
Can you please tell me different ways I can extract the data from HAWQ to
flat files?
I understand HAWQ also starts the data files in HDFS.
Regards,
Bhanu Joshi