You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Sriram Nookala <sr...@firstfuel.com> on 2017/09/05 18:05:39 UTC
Phoenix CSV Bulk Load Tool Date format for TIMESTAMP
I'm trying to bulkload data using the CsvBulkLoadTool, one of the columns
is a data in the format MMMMYYDD for example 20160912. I don't get an
error, but the parsing is wrong and when I use sqlline I see the date show
up as 20160912-01-01 00:00:00.000. I had assumed as per the fix for
https://issues.apache.org/jira/browse/PHOENIX-1127 all data values would be
parsed correctly.
Re: Phoenix CSV Bulk Load Tool Date format for TIMESTAMP
Posted by Ankit Singhal <an...@gmail.com>.
Yes, you can write your own custom mapper to do conversions (look at
CsvToKeyValueMapper, CsvUpsertExecutor#createConversionFunction) or
consider using chaining of jobs(where the first Job with multiple inputs
standardizing the date format followed by CSVBulkLoadTool) or writing a
custom TextInputFormat for standardizing the date format input to
CSVBulkLoadTool or etc.
On Thu, Sep 7, 2017 at 1:37 AM, Sriram Nookala <sr...@firstfuel.com> wrote:
> I'm still trying to set those up in Amazon EMR. However, setting the `
> phoenix.query.dateFormatTimeZone` wouldn't fix the issue for all files
> since we could receive a different date format in some other type of files.
> Is there an option to write a custom mapper to transform the date?
>
> On Tue, Sep 5, 2017 at 2:50 PM, Josh Elser <el...@apache.org> wrote:
>
>> Sriram,
>>
>> Did you set the timezone and date-format configuration properties
>> correctly for your environment?
>>
>> See `phoenix.query.dateFormatTimeZone` and `phoenix.query.dateFormat` as
>> described http://phoenix.apache.org/tuning.html
>>
>>
>> On 9/5/17 2:05 PM, Sriram Nookala wrote:
>>
>>> I'm trying to bulkload data using the CsvBulkLoadTool, one of the
>>> columns is a data in the format MMMMYYDD for example 20160912. I don't get
>>> an error, but the parsing is wrong and when I use sqlline I see the date
>>> show up as 20160912-01-01 00:00:00.000. I had assumed as per the fix for
>>> https://issues.apache.org/jira/browse/PHOENIX-1127 all data values
>>> would be parsed correctly.
>>>
>>
>
Re: Phoenix CSV Bulk Load Tool Date format for TIMESTAMP
Posted by Sriram Nookala <sr...@firstfuel.com>.
I'm still trying to set those up in Amazon EMR. However, setting the `
phoenix.query.dateFormatTimeZone` wouldn't fix the issue for all files
since we could receive a different date format in some other type of files.
Is there an option to write a custom mapper to transform the date?
On Tue, Sep 5, 2017 at 2:50 PM, Josh Elser <el...@apache.org> wrote:
> Sriram,
>
> Did you set the timezone and date-format configuration properties
> correctly for your environment?
>
> See `phoenix.query.dateFormatTimeZone` and `phoenix.query.dateFormat` as
> described http://phoenix.apache.org/tuning.html
>
>
> On 9/5/17 2:05 PM, Sriram Nookala wrote:
>
>> I'm trying to bulkload data using the CsvBulkLoadTool, one of the columns
>> is a data in the format MMMMYYDD for example 20160912. I don't get an
>> error, but the parsing is wrong and when I use sqlline I see the date show
>> up as 20160912-01-01 00:00:00.000. I had assumed as per the fix for
>> https://issues.apache.org/jira/browse/PHOENIX-1127 all data values would
>> be parsed correctly.
>>
>
Re: Support of OFFSET in Phoenix 4.7
Posted by rafa <ra...@gmail.com>.
Hi Sumanta,
Here you have the answer. You already asked the same question some months
ago :)
https://mail-archives.apache.org/mod_mbox/phoenix-user/201705.mbox/browser
From 4.8
regards,
rafa
On Wed, Sep 6, 2017 at 9:19 AM, Sumanta Gh <su...@tcs.com> wrote:
> Hi,
> From which version of Phoenix pagination with OFFSET is supported. It
> seems this is not supported in 4.7
>
> https://phoenix.apache.org/paged.html
>
> regards,
> Sumanta
>
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
Support of OFFSET in Phoenix 4.7
Posted by Sumanta Gh <su...@tcs.com>.
Hi,
From which version of Phoenix pagination with OFFSET is supported. It seems this is not supported in 4.7
https://phoenix.apache.org/paged.html
regards,
Sumanta
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
Re: Phoenix CSV Bulk Load Tool Date format for TIMESTAMP
Posted by Josh Elser <el...@apache.org>.
Sriram,
Did you set the timezone and date-format configuration properties
correctly for your environment?
See `phoenix.query.dateFormatTimeZone` and `phoenix.query.dateFormat` as
described http://phoenix.apache.org/tuning.html
On 9/5/17 2:05 PM, Sriram Nookala wrote:
> I'm trying to bulkload data using the CsvBulkLoadTool, one of the
> columns is a data in the format MMMMYYDD for example 20160912. I don't
> get an error, but the parsing is wrong and when I use sqlline I see the
> date show up as 20160912-01-01 00:00:00.000. I had assumed as per the
> fix for https://issues.apache.org/jira/browse/PHOENIX-1127 all data
> values would be parsed correctly.