You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Nick Dimiduk <nd...@gmail.com> on 2015/05/15 03:01:32 UTC
Date parsing error, or user error.
Heya,
Giving the RC a spin, and also investigating the impact of HBASE-13604, I'm
having a spot of trouble with bulk load. I'm executing CsvBulkLoadToolIT
manually. Seems like date string parsing is not working, but I may well be
doing something wrong.
Thanks.
$ cat /tmp/input1.csv
1,Name 1,1970/01/01
2,Name 2,1970/01/02
0: jdbc:phoenix:localhost> select COLUMN_NAME,DATA_TYPE from SYSTEM.CATALOG
where TABLE_NAME='TABLE1';
+------------------------------------------+-----------------------------------+
| COLUMN_NAME | DATA_TYPE
|
+------------------------------------------+-----------------------------------+
| | null
|
| ID | 4
|
| NAME | 12
|
| T | 91
|
+------------------------------------------+-----------------------------------+
4 rows selected (0.027 seconds)
$ HADOOP_CLASSPATH=/tmp/hbase-1.0.1/lib/hbase-protocol-1.0.1.jar hadoop jar
phoenix-4.4.0-HBase-1.0-client.jar
org.apache.phoenix.mapreduce.CsvBulkLoadTool --table TABLE1 --input
/tmp/input1.csv --zookeeper localhost:2181
...
Caused by: java.lang.RuntimeException: Error on record,
java.lang.IllegalArgumentException: Invalid format: "1970/01/01" is
malformed at "/01/01", record =[1, Name 1, 1970/01/01]
at
org.apache.phoenix.mapreduce.CsvToKeyValueMapper$MapperUpsertListener.errorOnRecord(CsvToKeyValueMapper.java:277)
at
org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:168)
at
org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:136)
at
org.apache.phoenix.mapreduce.CsvToKeyValueMapper.map(CsvToKeyValueMapper.java:160)
... 10 more
On Mon, May 11, 2015 at 5:41 PM, rajeshbabu@apache.org <
chrajeshbabu32@gmail.com> wrote:
> Hi Everyone,
>
> This is a call for a vote on Apache Phoenix 4.4.0-HBase-1.0 RC1. This is
> the
> next minor release of Phoenix 4, compatible with the 1.0(branch of
> Apache HBase(1.0.1+) . The release includes both a source-only release and
> a
> convenience binary release.
>
> The 4.4.0-HBase-1.0 release has feature parity with our pending
> 4.4.0-HBase-0.98 release. New features include:
> - Support HBase HA Query(timeline-consistent region replica read)[1]
> - Alter session query support(at present changing query consistency level.
> Can be used for changing connection properties.)
>
> The source tarball, including signatures, digests, etc can be found at:
>
> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/src/
>
> The binary artifacts can be found at:
>
> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/bin/
>
> Release artifacts are signed with the following key:
> http://people.apache.org/~rajeshbabu/E3A65DBC.asc
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/phoenix/KEYS
>
> The hash and tag to be voted upon:
>
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=09d1840876d7b55e32d753b6666541eb7df22b85
>
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.4.0-HBase-1.0-rc1
>
> Vote will be open for at least 72 hours. Please vote:
>
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>
> Thanks,
> The Apache Phoenix Team
>
> [1] https://issues.apache.org/jira/browse/PHOENIX-1683
>
Re: Date parsing error, or user error.
Posted by Nick Dimiduk <nd...@gmail.com>.
Ah! Thanks Gabriel.
On Tue, May 26, 2015 at 1:09 PM, Gabriel Reid <ga...@gmail.com>
wrote:
> It works in the integration test because the date format is specifically
> set (as an extra bit of testing)
>
>
> https://github.com/apache/phoenix/blob/master/phoenix-core/src/it/java/org/apache/phoenix/mapreduce/CsvBulkLoadToolIT.java#L105
>
>
> On Tue, May 26, 2015 at 9:33 PM Nick Dimiduk <nd...@gmail.com> wrote:
>
>> This is working for me as expected, thanks Gabriel. I find it odd that we
>> don't have an issue in our IT test though
>>
>> printWriter.println("1,Name 1,1970/01/01");
>> printWriter.println("2,Name 2,1970/01/02");
>>
>>
>> https://github.com/apache/phoenix/blob/master/phoenix-core/src/it/java/org/apache/phoenix/mapreduce/CsvBulkLoadToolIT.java#L99-L100
>>
>> On Sat, May 16, 2015 at 9:14 AM, Gabriel Reid <ga...@gmail.com>
>> wrote:
>>
>>> Hi Nick,
>>>
>>> The date format is (if I'm not mistaken) ISO-8601, so I think you'll
>>> have to format your date values as 1970-01-01.
>>>
>>> - Gabriel
>>>
>>>
>>> On Fri, May 15, 2015 at 02:02 Nick Dimiduk <nd...@gmail.com> wrote:
>>>
>>>> Heya,
>>>>
>>>> Giving the RC a spin, and also investigating the impact of HBASE-13604,
>>>> I'm
>>>> having a spot of trouble with bulk load. I'm executing CsvBulkLoadToolIT
>>>> manually. Seems like date string parsing is not working, but I may well
>>>> be
>>>> doing something wrong.
>>>>
>>>> Thanks.
>>>>
>>>> $ cat /tmp/input1.csv
>>>> 1,Name 1,1970/01/01
>>>> 2,Name 2,1970/01/02
>>>>
>>>> 0: jdbc:phoenix:localhost> select COLUMN_NAME,DATA_TYPE from
>>>> SYSTEM.CATALOG
>>>> where TABLE_NAME='TABLE1';
>>>>
>>>> +------------------------------------------+-----------------------------------+
>>>> | COLUMN_NAME | DATA_TYPE
>>>> |
>>>>
>>>> +------------------------------------------+-----------------------------------+
>>>> | | null
>>>> |
>>>> | ID | 4
>>>> |
>>>> | NAME | 12
>>>> |
>>>> | T | 91
>>>> |
>>>>
>>>> +------------------------------------------+-----------------------------------+
>>>> 4 rows selected (0.027 seconds)
>>>>
>>>> $ HADOOP_CLASSPATH=/tmp/hbase-1.0.1/lib/hbase-protocol-1.0.1.jar hadoop
>>>> jar
>>>> phoenix-4.4.0-HBase-1.0-client.jar
>>>> org.apache.phoenix.mapreduce.CsvBulkLoadTool --table TABLE1 --input
>>>> /tmp/input1.csv --zookeeper localhost:2181
>>>> ...
>>>> Caused by: java.lang.RuntimeException: Error on record,
>>>> java.lang.IllegalArgumentException: Invalid format: "1970/01/01" is
>>>> malformed at "/01/01", record =[1, Name 1, 1970/01/01]
>>>> at
>>>>
>>>> org.apache.phoenix.mapreduce.CsvToKeyValueMapper$MapperUpsertListener.errorOnRecord(CsvToKeyValueMapper.java:277)
>>>> at
>>>>
>>>> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:168)
>>>> at
>>>>
>>>> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:136)
>>>> at
>>>>
>>>> org.apache.phoenix.mapreduce.CsvToKeyValueMapper.map(CsvToKeyValueMapper.java:160)
>>>> ... 10 more
>>>>
>>>> On Mon, May 11, 2015 at 5:41 PM, rajeshbabu@apache.org <
>>>> chrajeshbabu32@gmail.com> wrote:
>>>>
>>>> > Hi Everyone,
>>>> >
>>>> > This is a call for a vote on Apache Phoenix 4.4.0-HBase-1.0 RC1. This
>>>> is
>>>> > the
>>>> > next minor release of Phoenix 4, compatible with the 1.0(branch of
>>>> > Apache HBase(1.0.1+) . The release includes both a source-only
>>>> release and
>>>> > a
>>>> > convenience binary release.
>>>> >
>>>> > The 4.4.0-HBase-1.0 release has feature parity with our pending
>>>> > 4.4.0-HBase-0.98 release. New features include:
>>>> > - Support HBase HA Query(timeline-consistent region replica read)[1]
>>>> > - Alter session query support(at present changing query consistency
>>>> level.
>>>> > Can be used for changing connection properties.)
>>>> >
>>>> > The source tarball, including signatures, digests, etc can be found
>>>> at:
>>>> >
>>>> >
>>>> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/src/
>>>> >
>>>> > The binary artifacts can be found at:
>>>> >
>>>> >
>>>> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/bin/
>>>> >
>>>> > Release artifacts are signed with the following key:
>>>> > http://people.apache.org/~rajeshbabu/E3A65DBC.asc
>>>> >
>>>> > KEYS file available here:
>>>> > https://dist.apache.org/repos/dist/release/phoenix/KEYS
>>>> >
>>>> > The hash and tag to be voted upon:
>>>> >
>>>> >
>>>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=09d1840876d7b55e32d753b6666541eb7df22b85
>>>> >
>>>> >
>>>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.4.0-HBase-1.0-rc1
>>>> >
>>>> > Vote will be open for at least 72 hours. Please vote:
>>>> >
>>>> > [ ] +1 approve
>>>> > [ ] +0 no opinion
>>>> > [ ] -1 disapprove (and reason why)
>>>> >
>>>> > Thanks,
>>>> > The Apache Phoenix Team
>>>> >
>>>> > [1] https://issues.apache.org/jira/browse/PHOENIX-1683
>>>> >
>>>>
>>>
>>
Re: Date parsing error, or user error.
Posted by Nick Dimiduk <nd...@gmail.com>.
Ah! Thanks Gabriel.
On Tue, May 26, 2015 at 1:09 PM, Gabriel Reid <ga...@gmail.com>
wrote:
> It works in the integration test because the date format is specifically
> set (as an extra bit of testing)
>
>
> https://github.com/apache/phoenix/blob/master/phoenix-core/src/it/java/org/apache/phoenix/mapreduce/CsvBulkLoadToolIT.java#L105
>
>
> On Tue, May 26, 2015 at 9:33 PM Nick Dimiduk <nd...@gmail.com> wrote:
>
>> This is working for me as expected, thanks Gabriel. I find it odd that we
>> don't have an issue in our IT test though
>>
>> printWriter.println("1,Name 1,1970/01/01");
>> printWriter.println("2,Name 2,1970/01/02");
>>
>>
>> https://github.com/apache/phoenix/blob/master/phoenix-core/src/it/java/org/apache/phoenix/mapreduce/CsvBulkLoadToolIT.java#L99-L100
>>
>> On Sat, May 16, 2015 at 9:14 AM, Gabriel Reid <ga...@gmail.com>
>> wrote:
>>
>>> Hi Nick,
>>>
>>> The date format is (if I'm not mistaken) ISO-8601, so I think you'll
>>> have to format your date values as 1970-01-01.
>>>
>>> - Gabriel
>>>
>>>
>>> On Fri, May 15, 2015 at 02:02 Nick Dimiduk <nd...@gmail.com> wrote:
>>>
>>>> Heya,
>>>>
>>>> Giving the RC a spin, and also investigating the impact of HBASE-13604,
>>>> I'm
>>>> having a spot of trouble with bulk load. I'm executing CsvBulkLoadToolIT
>>>> manually. Seems like date string parsing is not working, but I may well
>>>> be
>>>> doing something wrong.
>>>>
>>>> Thanks.
>>>>
>>>> $ cat /tmp/input1.csv
>>>> 1,Name 1,1970/01/01
>>>> 2,Name 2,1970/01/02
>>>>
>>>> 0: jdbc:phoenix:localhost> select COLUMN_NAME,DATA_TYPE from
>>>> SYSTEM.CATALOG
>>>> where TABLE_NAME='TABLE1';
>>>>
>>>> +------------------------------------------+-----------------------------------+
>>>> | COLUMN_NAME | DATA_TYPE
>>>> |
>>>>
>>>> +------------------------------------------+-----------------------------------+
>>>> | | null
>>>> |
>>>> | ID | 4
>>>> |
>>>> | NAME | 12
>>>> |
>>>> | T | 91
>>>> |
>>>>
>>>> +------------------------------------------+-----------------------------------+
>>>> 4 rows selected (0.027 seconds)
>>>>
>>>> $ HADOOP_CLASSPATH=/tmp/hbase-1.0.1/lib/hbase-protocol-1.0.1.jar hadoop
>>>> jar
>>>> phoenix-4.4.0-HBase-1.0-client.jar
>>>> org.apache.phoenix.mapreduce.CsvBulkLoadTool --table TABLE1 --input
>>>> /tmp/input1.csv --zookeeper localhost:2181
>>>> ...
>>>> Caused by: java.lang.RuntimeException: Error on record,
>>>> java.lang.IllegalArgumentException: Invalid format: "1970/01/01" is
>>>> malformed at "/01/01", record =[1, Name 1, 1970/01/01]
>>>> at
>>>>
>>>> org.apache.phoenix.mapreduce.CsvToKeyValueMapper$MapperUpsertListener.errorOnRecord(CsvToKeyValueMapper.java:277)
>>>> at
>>>>
>>>> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:168)
>>>> at
>>>>
>>>> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:136)
>>>> at
>>>>
>>>> org.apache.phoenix.mapreduce.CsvToKeyValueMapper.map(CsvToKeyValueMapper.java:160)
>>>> ... 10 more
>>>>
>>>> On Mon, May 11, 2015 at 5:41 PM, rajeshbabu@apache.org <
>>>> chrajeshbabu32@gmail.com> wrote:
>>>>
>>>> > Hi Everyone,
>>>> >
>>>> > This is a call for a vote on Apache Phoenix 4.4.0-HBase-1.0 RC1. This
>>>> is
>>>> > the
>>>> > next minor release of Phoenix 4, compatible with the 1.0(branch of
>>>> > Apache HBase(1.0.1+) . The release includes both a source-only
>>>> release and
>>>> > a
>>>> > convenience binary release.
>>>> >
>>>> > The 4.4.0-HBase-1.0 release has feature parity with our pending
>>>> > 4.4.0-HBase-0.98 release. New features include:
>>>> > - Support HBase HA Query(timeline-consistent region replica read)[1]
>>>> > - Alter session query support(at present changing query consistency
>>>> level.
>>>> > Can be used for changing connection properties.)
>>>> >
>>>> > The source tarball, including signatures, digests, etc can be found
>>>> at:
>>>> >
>>>> >
>>>> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/src/
>>>> >
>>>> > The binary artifacts can be found at:
>>>> >
>>>> >
>>>> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/bin/
>>>> >
>>>> > Release artifacts are signed with the following key:
>>>> > http://people.apache.org/~rajeshbabu/E3A65DBC.asc
>>>> >
>>>> > KEYS file available here:
>>>> > https://dist.apache.org/repos/dist/release/phoenix/KEYS
>>>> >
>>>> > The hash and tag to be voted upon:
>>>> >
>>>> >
>>>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=09d1840876d7b55e32d753b6666541eb7df22b85
>>>> >
>>>> >
>>>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.4.0-HBase-1.0-rc1
>>>> >
>>>> > Vote will be open for at least 72 hours. Please vote:
>>>> >
>>>> > [ ] +1 approve
>>>> > [ ] +0 no opinion
>>>> > [ ] -1 disapprove (and reason why)
>>>> >
>>>> > Thanks,
>>>> > The Apache Phoenix Team
>>>> >
>>>> > [1] https://issues.apache.org/jira/browse/PHOENIX-1683
>>>> >
>>>>
>>>
>>
Re: Date parsing error, or user error.
Posted by Gabriel Reid <ga...@gmail.com>.
It works in the integration test because the date format is specifically
set (as an extra bit of testing)
https://github.com/apache/phoenix/blob/master/phoenix-core/src/it/java/org/apache/phoenix/mapreduce/CsvBulkLoadToolIT.java#L105
On Tue, May 26, 2015 at 9:33 PM Nick Dimiduk <nd...@gmail.com> wrote:
> This is working for me as expected, thanks Gabriel. I find it odd that we
> don't have an issue in our IT test though
>
> printWriter.println("1,Name 1,1970/01/01");
> printWriter.println("2,Name 2,1970/01/02");
>
>
> https://github.com/apache/phoenix/blob/master/phoenix-core/src/it/java/org/apache/phoenix/mapreduce/CsvBulkLoadToolIT.java#L99-L100
>
> On Sat, May 16, 2015 at 9:14 AM, Gabriel Reid <ga...@gmail.com>
> wrote:
>
>> Hi Nick,
>>
>> The date format is (if I'm not mistaken) ISO-8601, so I think you'll have
>> to format your date values as 1970-01-01.
>>
>> - Gabriel
>>
>>
>> On Fri, May 15, 2015 at 02:02 Nick Dimiduk <nd...@gmail.com> wrote:
>>
>>> Heya,
>>>
>>> Giving the RC a spin, and also investigating the impact of HBASE-13604,
>>> I'm
>>> having a spot of trouble with bulk load. I'm executing CsvBulkLoadToolIT
>>> manually. Seems like date string parsing is not working, but I may well
>>> be
>>> doing something wrong.
>>>
>>> Thanks.
>>>
>>> $ cat /tmp/input1.csv
>>> 1,Name 1,1970/01/01
>>> 2,Name 2,1970/01/02
>>>
>>> 0: jdbc:phoenix:localhost> select COLUMN_NAME,DATA_TYPE from
>>> SYSTEM.CATALOG
>>> where TABLE_NAME='TABLE1';
>>>
>>> +------------------------------------------+-----------------------------------+
>>> | COLUMN_NAME | DATA_TYPE
>>> |
>>>
>>> +------------------------------------------+-----------------------------------+
>>> | | null
>>> |
>>> | ID | 4
>>> |
>>> | NAME | 12
>>> |
>>> | T | 91
>>> |
>>>
>>> +------------------------------------------+-----------------------------------+
>>> 4 rows selected (0.027 seconds)
>>>
>>> $ HADOOP_CLASSPATH=/tmp/hbase-1.0.1/lib/hbase-protocol-1.0.1.jar hadoop
>>> jar
>>> phoenix-4.4.0-HBase-1.0-client.jar
>>> org.apache.phoenix.mapreduce.CsvBulkLoadTool --table TABLE1 --input
>>> /tmp/input1.csv --zookeeper localhost:2181
>>> ...
>>> Caused by: java.lang.RuntimeException: Error on record,
>>> java.lang.IllegalArgumentException: Invalid format: "1970/01/01" is
>>> malformed at "/01/01", record =[1, Name 1, 1970/01/01]
>>> at
>>>
>>> org.apache.phoenix.mapreduce.CsvToKeyValueMapper$MapperUpsertListener.errorOnRecord(CsvToKeyValueMapper.java:277)
>>> at
>>>
>>> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:168)
>>> at
>>>
>>> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:136)
>>> at
>>>
>>> org.apache.phoenix.mapreduce.CsvToKeyValueMapper.map(CsvToKeyValueMapper.java:160)
>>> ... 10 more
>>>
>>> On Mon, May 11, 2015 at 5:41 PM, rajeshbabu@apache.org <
>>> chrajeshbabu32@gmail.com> wrote:
>>>
>>> > Hi Everyone,
>>> >
>>> > This is a call for a vote on Apache Phoenix 4.4.0-HBase-1.0 RC1. This
>>> is
>>> > the
>>> > next minor release of Phoenix 4, compatible with the 1.0(branch of
>>> > Apache HBase(1.0.1+) . The release includes both a source-only release
>>> and
>>> > a
>>> > convenience binary release.
>>> >
>>> > The 4.4.0-HBase-1.0 release has feature parity with our pending
>>> > 4.4.0-HBase-0.98 release. New features include:
>>> > - Support HBase HA Query(timeline-consistent region replica read)[1]
>>> > - Alter session query support(at present changing query consistency
>>> level.
>>> > Can be used for changing connection properties.)
>>> >
>>> > The source tarball, including signatures, digests, etc can be found at:
>>> >
>>> >
>>> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/src/
>>> >
>>> > The binary artifacts can be found at:
>>> >
>>> >
>>> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/bin/
>>> >
>>> > Release artifacts are signed with the following key:
>>> > http://people.apache.org/~rajeshbabu/E3A65DBC.asc
>>> >
>>> > KEYS file available here:
>>> > https://dist.apache.org/repos/dist/release/phoenix/KEYS
>>> >
>>> > The hash and tag to be voted upon:
>>> >
>>> >
>>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=09d1840876d7b55e32d753b6666541eb7df22b85
>>> >
>>> >
>>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.4.0-HBase-1.0-rc1
>>> >
>>> > Vote will be open for at least 72 hours. Please vote:
>>> >
>>> > [ ] +1 approve
>>> > [ ] +0 no opinion
>>> > [ ] -1 disapprove (and reason why)
>>> >
>>> > Thanks,
>>> > The Apache Phoenix Team
>>> >
>>> > [1] https://issues.apache.org/jira/browse/PHOENIX-1683
>>> >
>>>
>>
>
Re: Date parsing error, or user error.
Posted by Gabriel Reid <ga...@gmail.com>.
It works in the integration test because the date format is specifically
set (as an extra bit of testing)
https://github.com/apache/phoenix/blob/master/phoenix-core/src/it/java/org/apache/phoenix/mapreduce/CsvBulkLoadToolIT.java#L105
On Tue, May 26, 2015 at 9:33 PM Nick Dimiduk <nd...@gmail.com> wrote:
> This is working for me as expected, thanks Gabriel. I find it odd that we
> don't have an issue in our IT test though
>
> printWriter.println("1,Name 1,1970/01/01");
> printWriter.println("2,Name 2,1970/01/02");
>
>
> https://github.com/apache/phoenix/blob/master/phoenix-core/src/it/java/org/apache/phoenix/mapreduce/CsvBulkLoadToolIT.java#L99-L100
>
> On Sat, May 16, 2015 at 9:14 AM, Gabriel Reid <ga...@gmail.com>
> wrote:
>
>> Hi Nick,
>>
>> The date format is (if I'm not mistaken) ISO-8601, so I think you'll have
>> to format your date values as 1970-01-01.
>>
>> - Gabriel
>>
>>
>> On Fri, May 15, 2015 at 02:02 Nick Dimiduk <nd...@gmail.com> wrote:
>>
>>> Heya,
>>>
>>> Giving the RC a spin, and also investigating the impact of HBASE-13604,
>>> I'm
>>> having a spot of trouble with bulk load. I'm executing CsvBulkLoadToolIT
>>> manually. Seems like date string parsing is not working, but I may well
>>> be
>>> doing something wrong.
>>>
>>> Thanks.
>>>
>>> $ cat /tmp/input1.csv
>>> 1,Name 1,1970/01/01
>>> 2,Name 2,1970/01/02
>>>
>>> 0: jdbc:phoenix:localhost> select COLUMN_NAME,DATA_TYPE from
>>> SYSTEM.CATALOG
>>> where TABLE_NAME='TABLE1';
>>>
>>> +------------------------------------------+-----------------------------------+
>>> | COLUMN_NAME | DATA_TYPE
>>> |
>>>
>>> +------------------------------------------+-----------------------------------+
>>> | | null
>>> |
>>> | ID | 4
>>> |
>>> | NAME | 12
>>> |
>>> | T | 91
>>> |
>>>
>>> +------------------------------------------+-----------------------------------+
>>> 4 rows selected (0.027 seconds)
>>>
>>> $ HADOOP_CLASSPATH=/tmp/hbase-1.0.1/lib/hbase-protocol-1.0.1.jar hadoop
>>> jar
>>> phoenix-4.4.0-HBase-1.0-client.jar
>>> org.apache.phoenix.mapreduce.CsvBulkLoadTool --table TABLE1 --input
>>> /tmp/input1.csv --zookeeper localhost:2181
>>> ...
>>> Caused by: java.lang.RuntimeException: Error on record,
>>> java.lang.IllegalArgumentException: Invalid format: "1970/01/01" is
>>> malformed at "/01/01", record =[1, Name 1, 1970/01/01]
>>> at
>>>
>>> org.apache.phoenix.mapreduce.CsvToKeyValueMapper$MapperUpsertListener.errorOnRecord(CsvToKeyValueMapper.java:277)
>>> at
>>>
>>> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:168)
>>> at
>>>
>>> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:136)
>>> at
>>>
>>> org.apache.phoenix.mapreduce.CsvToKeyValueMapper.map(CsvToKeyValueMapper.java:160)
>>> ... 10 more
>>>
>>> On Mon, May 11, 2015 at 5:41 PM, rajeshbabu@apache.org <
>>> chrajeshbabu32@gmail.com> wrote:
>>>
>>> > Hi Everyone,
>>> >
>>> > This is a call for a vote on Apache Phoenix 4.4.0-HBase-1.0 RC1. This
>>> is
>>> > the
>>> > next minor release of Phoenix 4, compatible with the 1.0(branch of
>>> > Apache HBase(1.0.1+) . The release includes both a source-only release
>>> and
>>> > a
>>> > convenience binary release.
>>> >
>>> > The 4.4.0-HBase-1.0 release has feature parity with our pending
>>> > 4.4.0-HBase-0.98 release. New features include:
>>> > - Support HBase HA Query(timeline-consistent region replica read)[1]
>>> > - Alter session query support(at present changing query consistency
>>> level.
>>> > Can be used for changing connection properties.)
>>> >
>>> > The source tarball, including signatures, digests, etc can be found at:
>>> >
>>> >
>>> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/src/
>>> >
>>> > The binary artifacts can be found at:
>>> >
>>> >
>>> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/bin/
>>> >
>>> > Release artifacts are signed with the following key:
>>> > http://people.apache.org/~rajeshbabu/E3A65DBC.asc
>>> >
>>> > KEYS file available here:
>>> > https://dist.apache.org/repos/dist/release/phoenix/KEYS
>>> >
>>> > The hash and tag to be voted upon:
>>> >
>>> >
>>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=09d1840876d7b55e32d753b6666541eb7df22b85
>>> >
>>> >
>>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.4.0-HBase-1.0-rc1
>>> >
>>> > Vote will be open for at least 72 hours. Please vote:
>>> >
>>> > [ ] +1 approve
>>> > [ ] +0 no opinion
>>> > [ ] -1 disapprove (and reason why)
>>> >
>>> > Thanks,
>>> > The Apache Phoenix Team
>>> >
>>> > [1] https://issues.apache.org/jira/browse/PHOENIX-1683
>>> >
>>>
>>
>
Re: Date parsing error, or user error.
Posted by Nick Dimiduk <nd...@gmail.com>.
This is working for me as expected, thanks Gabriel. I find it odd that we
don't have an issue in our IT test though
printWriter.println("1,Name 1,1970/01/01");
printWriter.println("2,Name 2,1970/01/02");
https://github.com/apache/phoenix/blob/master/phoenix-core/src/it/java/org/apache/phoenix/mapreduce/CsvBulkLoadToolIT.java#L99-L100
On Sat, May 16, 2015 at 9:14 AM, Gabriel Reid <ga...@gmail.com>
wrote:
> Hi Nick,
>
> The date format is (if I'm not mistaken) ISO-8601, so I think you'll have
> to format your date values as 1970-01-01.
>
> - Gabriel
>
>
> On Fri, May 15, 2015 at 02:02 Nick Dimiduk <nd...@gmail.com> wrote:
>
>> Heya,
>>
>> Giving the RC a spin, and also investigating the impact of HBASE-13604,
>> I'm
>> having a spot of trouble with bulk load. I'm executing CsvBulkLoadToolIT
>> manually. Seems like date string parsing is not working, but I may well be
>> doing something wrong.
>>
>> Thanks.
>>
>> $ cat /tmp/input1.csv
>> 1,Name 1,1970/01/01
>> 2,Name 2,1970/01/02
>>
>> 0: jdbc:phoenix:localhost> select COLUMN_NAME,DATA_TYPE from
>> SYSTEM.CATALOG
>> where TABLE_NAME='TABLE1';
>>
>> +------------------------------------------+-----------------------------------+
>> | COLUMN_NAME | DATA_TYPE
>> |
>>
>> +------------------------------------------+-----------------------------------+
>> | | null
>> |
>> | ID | 4
>> |
>> | NAME | 12
>> |
>> | T | 91
>> |
>>
>> +------------------------------------------+-----------------------------------+
>> 4 rows selected (0.027 seconds)
>>
>> $ HADOOP_CLASSPATH=/tmp/hbase-1.0.1/lib/hbase-protocol-1.0.1.jar hadoop
>> jar
>> phoenix-4.4.0-HBase-1.0-client.jar
>> org.apache.phoenix.mapreduce.CsvBulkLoadTool --table TABLE1 --input
>> /tmp/input1.csv --zookeeper localhost:2181
>> ...
>> Caused by: java.lang.RuntimeException: Error on record,
>> java.lang.IllegalArgumentException: Invalid format: "1970/01/01" is
>> malformed at "/01/01", record =[1, Name 1, 1970/01/01]
>> at
>>
>> org.apache.phoenix.mapreduce.CsvToKeyValueMapper$MapperUpsertListener.errorOnRecord(CsvToKeyValueMapper.java:277)
>> at
>>
>> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:168)
>> at
>>
>> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:136)
>> at
>>
>> org.apache.phoenix.mapreduce.CsvToKeyValueMapper.map(CsvToKeyValueMapper.java:160)
>> ... 10 more
>>
>> On Mon, May 11, 2015 at 5:41 PM, rajeshbabu@apache.org <
>> chrajeshbabu32@gmail.com> wrote:
>>
>> > Hi Everyone,
>> >
>> > This is a call for a vote on Apache Phoenix 4.4.0-HBase-1.0 RC1. This is
>> > the
>> > next minor release of Phoenix 4, compatible with the 1.0(branch of
>> > Apache HBase(1.0.1+) . The release includes both a source-only release
>> and
>> > a
>> > convenience binary release.
>> >
>> > The 4.4.0-HBase-1.0 release has feature parity with our pending
>> > 4.4.0-HBase-0.98 release. New features include:
>> > - Support HBase HA Query(timeline-consistent region replica read)[1]
>> > - Alter session query support(at present changing query consistency
>> level.
>> > Can be used for changing connection properties.)
>> >
>> > The source tarball, including signatures, digests, etc can be found at:
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/src/
>> >
>> > The binary artifacts can be found at:
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/bin/
>> >
>> > Release artifacts are signed with the following key:
>> > http://people.apache.org/~rajeshbabu/E3A65DBC.asc
>> >
>> > KEYS file available here:
>> > https://dist.apache.org/repos/dist/release/phoenix/KEYS
>> >
>> > The hash and tag to be voted upon:
>> >
>> >
>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=09d1840876d7b55e32d753b6666541eb7df22b85
>> >
>> >
>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.4.0-HBase-1.0-rc1
>> >
>> > Vote will be open for at least 72 hours. Please vote:
>> >
>> > [ ] +1 approve
>> > [ ] +0 no opinion
>> > [ ] -1 disapprove (and reason why)
>> >
>> > Thanks,
>> > The Apache Phoenix Team
>> >
>> > [1] https://issues.apache.org/jira/browse/PHOENIX-1683
>> >
>>
>
Re: Date parsing error, or user error.
Posted by Nick Dimiduk <nd...@gmail.com>.
This is working for me as expected, thanks Gabriel. I find it odd that we
don't have an issue in our IT test though
printWriter.println("1,Name 1,1970/01/01");
printWriter.println("2,Name 2,1970/01/02");
https://github.com/apache/phoenix/blob/master/phoenix-core/src/it/java/org/apache/phoenix/mapreduce/CsvBulkLoadToolIT.java#L99-L100
On Sat, May 16, 2015 at 9:14 AM, Gabriel Reid <ga...@gmail.com>
wrote:
> Hi Nick,
>
> The date format is (if I'm not mistaken) ISO-8601, so I think you'll have
> to format your date values as 1970-01-01.
>
> - Gabriel
>
>
> On Fri, May 15, 2015 at 02:02 Nick Dimiduk <nd...@gmail.com> wrote:
>
>> Heya,
>>
>> Giving the RC a spin, and also investigating the impact of HBASE-13604,
>> I'm
>> having a spot of trouble with bulk load. I'm executing CsvBulkLoadToolIT
>> manually. Seems like date string parsing is not working, but I may well be
>> doing something wrong.
>>
>> Thanks.
>>
>> $ cat /tmp/input1.csv
>> 1,Name 1,1970/01/01
>> 2,Name 2,1970/01/02
>>
>> 0: jdbc:phoenix:localhost> select COLUMN_NAME,DATA_TYPE from
>> SYSTEM.CATALOG
>> where TABLE_NAME='TABLE1';
>>
>> +------------------------------------------+-----------------------------------+
>> | COLUMN_NAME | DATA_TYPE
>> |
>>
>> +------------------------------------------+-----------------------------------+
>> | | null
>> |
>> | ID | 4
>> |
>> | NAME | 12
>> |
>> | T | 91
>> |
>>
>> +------------------------------------------+-----------------------------------+
>> 4 rows selected (0.027 seconds)
>>
>> $ HADOOP_CLASSPATH=/tmp/hbase-1.0.1/lib/hbase-protocol-1.0.1.jar hadoop
>> jar
>> phoenix-4.4.0-HBase-1.0-client.jar
>> org.apache.phoenix.mapreduce.CsvBulkLoadTool --table TABLE1 --input
>> /tmp/input1.csv --zookeeper localhost:2181
>> ...
>> Caused by: java.lang.RuntimeException: Error on record,
>> java.lang.IllegalArgumentException: Invalid format: "1970/01/01" is
>> malformed at "/01/01", record =[1, Name 1, 1970/01/01]
>> at
>>
>> org.apache.phoenix.mapreduce.CsvToKeyValueMapper$MapperUpsertListener.errorOnRecord(CsvToKeyValueMapper.java:277)
>> at
>>
>> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:168)
>> at
>>
>> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:136)
>> at
>>
>> org.apache.phoenix.mapreduce.CsvToKeyValueMapper.map(CsvToKeyValueMapper.java:160)
>> ... 10 more
>>
>> On Mon, May 11, 2015 at 5:41 PM, rajeshbabu@apache.org <
>> chrajeshbabu32@gmail.com> wrote:
>>
>> > Hi Everyone,
>> >
>> > This is a call for a vote on Apache Phoenix 4.4.0-HBase-1.0 RC1. This is
>> > the
>> > next minor release of Phoenix 4, compatible with the 1.0(branch of
>> > Apache HBase(1.0.1+) . The release includes both a source-only release
>> and
>> > a
>> > convenience binary release.
>> >
>> > The 4.4.0-HBase-1.0 release has feature parity with our pending
>> > 4.4.0-HBase-0.98 release. New features include:
>> > - Support HBase HA Query(timeline-consistent region replica read)[1]
>> > - Alter session query support(at present changing query consistency
>> level.
>> > Can be used for changing connection properties.)
>> >
>> > The source tarball, including signatures, digests, etc can be found at:
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/src/
>> >
>> > The binary artifacts can be found at:
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/bin/
>> >
>> > Release artifacts are signed with the following key:
>> > http://people.apache.org/~rajeshbabu/E3A65DBC.asc
>> >
>> > KEYS file available here:
>> > https://dist.apache.org/repos/dist/release/phoenix/KEYS
>> >
>> > The hash and tag to be voted upon:
>> >
>> >
>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=09d1840876d7b55e32d753b6666541eb7df22b85
>> >
>> >
>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.4.0-HBase-1.0-rc1
>> >
>> > Vote will be open for at least 72 hours. Please vote:
>> >
>> > [ ] +1 approve
>> > [ ] +0 no opinion
>> > [ ] -1 disapprove (and reason why)
>> >
>> > Thanks,
>> > The Apache Phoenix Team
>> >
>> > [1] https://issues.apache.org/jira/browse/PHOENIX-1683
>> >
>>
>
Re: Date parsing error, or user error.
Posted by Gabriel Reid <ga...@gmail.com>.
Hi Nick,
The date format is (if I'm not mistaken) ISO-8601, so I think you'll have
to format your date values as 1970-01-01.
- Gabriel
On Fri, May 15, 2015 at 02:02 Nick Dimiduk <nd...@gmail.com> wrote:
> Heya,
>
> Giving the RC a spin, and also investigating the impact of HBASE-13604, I'm
> having a spot of trouble with bulk load. I'm executing CsvBulkLoadToolIT
> manually. Seems like date string parsing is not working, but I may well be
> doing something wrong.
>
> Thanks.
>
> $ cat /tmp/input1.csv
> 1,Name 1,1970/01/01
> 2,Name 2,1970/01/02
>
> 0: jdbc:phoenix:localhost> select COLUMN_NAME,DATA_TYPE from SYSTEM.CATALOG
> where TABLE_NAME='TABLE1';
>
> +------------------------------------------+-----------------------------------+
> | COLUMN_NAME | DATA_TYPE
> |
>
> +------------------------------------------+-----------------------------------+
> | | null
> |
> | ID | 4
> |
> | NAME | 12
> |
> | T | 91
> |
>
> +------------------------------------------+-----------------------------------+
> 4 rows selected (0.027 seconds)
>
> $ HADOOP_CLASSPATH=/tmp/hbase-1.0.1/lib/hbase-protocol-1.0.1.jar hadoop jar
> phoenix-4.4.0-HBase-1.0-client.jar
> org.apache.phoenix.mapreduce.CsvBulkLoadTool --table TABLE1 --input
> /tmp/input1.csv --zookeeper localhost:2181
> ...
> Caused by: java.lang.RuntimeException: Error on record,
> java.lang.IllegalArgumentException: Invalid format: "1970/01/01" is
> malformed at "/01/01", record =[1, Name 1, 1970/01/01]
> at
>
> org.apache.phoenix.mapreduce.CsvToKeyValueMapper$MapperUpsertListener.errorOnRecord(CsvToKeyValueMapper.java:277)
> at
>
> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:168)
> at
>
> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:136)
> at
>
> org.apache.phoenix.mapreduce.CsvToKeyValueMapper.map(CsvToKeyValueMapper.java:160)
> ... 10 more
>
> On Mon, May 11, 2015 at 5:41 PM, rajeshbabu@apache.org <
> chrajeshbabu32@gmail.com> wrote:
>
> > Hi Everyone,
> >
> > This is a call for a vote on Apache Phoenix 4.4.0-HBase-1.0 RC1. This is
> > the
> > next minor release of Phoenix 4, compatible with the 1.0(branch of
> > Apache HBase(1.0.1+) . The release includes both a source-only release
> and
> > a
> > convenience binary release.
> >
> > The 4.4.0-HBase-1.0 release has feature parity with our pending
> > 4.4.0-HBase-0.98 release. New features include:
> > - Support HBase HA Query(timeline-consistent region replica read)[1]
> > - Alter session query support(at present changing query consistency
> level.
> > Can be used for changing connection properties.)
> >
> > The source tarball, including signatures, digests, etc can be found at:
> >
> >
> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/src/
> >
> > The binary artifacts can be found at:
> >
> >
> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/bin/
> >
> > Release artifacts are signed with the following key:
> > http://people.apache.org/~rajeshbabu/E3A65DBC.asc
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/phoenix/KEYS
> >
> > The hash and tag to be voted upon:
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=09d1840876d7b55e32d753b6666541eb7df22b85
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.4.0-HBase-1.0-rc1
> >
> > Vote will be open for at least 72 hours. Please vote:
> >
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove (and reason why)
> >
> > Thanks,
> > The Apache Phoenix Team
> >
> > [1] https://issues.apache.org/jira/browse/PHOENIX-1683
> >
>
Re: Date parsing error, or user error.
Posted by Gabriel Reid <ga...@gmail.com>.
Hi Nick,
The date format is (if I'm not mistaken) ISO-8601, so I think you'll have
to format your date values as 1970-01-01.
- Gabriel
On Fri, May 15, 2015 at 02:02 Nick Dimiduk <nd...@gmail.com> wrote:
> Heya,
>
> Giving the RC a spin, and also investigating the impact of HBASE-13604, I'm
> having a spot of trouble with bulk load. I'm executing CsvBulkLoadToolIT
> manually. Seems like date string parsing is not working, but I may well be
> doing something wrong.
>
> Thanks.
>
> $ cat /tmp/input1.csv
> 1,Name 1,1970/01/01
> 2,Name 2,1970/01/02
>
> 0: jdbc:phoenix:localhost> select COLUMN_NAME,DATA_TYPE from SYSTEM.CATALOG
> where TABLE_NAME='TABLE1';
>
> +------------------------------------------+-----------------------------------+
> | COLUMN_NAME | DATA_TYPE
> |
>
> +------------------------------------------+-----------------------------------+
> | | null
> |
> | ID | 4
> |
> | NAME | 12
> |
> | T | 91
> |
>
> +------------------------------------------+-----------------------------------+
> 4 rows selected (0.027 seconds)
>
> $ HADOOP_CLASSPATH=/tmp/hbase-1.0.1/lib/hbase-protocol-1.0.1.jar hadoop jar
> phoenix-4.4.0-HBase-1.0-client.jar
> org.apache.phoenix.mapreduce.CsvBulkLoadTool --table TABLE1 --input
> /tmp/input1.csv --zookeeper localhost:2181
> ...
> Caused by: java.lang.RuntimeException: Error on record,
> java.lang.IllegalArgumentException: Invalid format: "1970/01/01" is
> malformed at "/01/01", record =[1, Name 1, 1970/01/01]
> at
>
> org.apache.phoenix.mapreduce.CsvToKeyValueMapper$MapperUpsertListener.errorOnRecord(CsvToKeyValueMapper.java:277)
> at
>
> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:168)
> at
>
> org.apache.phoenix.util.csv.CsvUpsertExecutor.execute(CsvUpsertExecutor.java:136)
> at
>
> org.apache.phoenix.mapreduce.CsvToKeyValueMapper.map(CsvToKeyValueMapper.java:160)
> ... 10 more
>
> On Mon, May 11, 2015 at 5:41 PM, rajeshbabu@apache.org <
> chrajeshbabu32@gmail.com> wrote:
>
> > Hi Everyone,
> >
> > This is a call for a vote on Apache Phoenix 4.4.0-HBase-1.0 RC1. This is
> > the
> > next minor release of Phoenix 4, compatible with the 1.0(branch of
> > Apache HBase(1.0.1+) . The release includes both a source-only release
> and
> > a
> > convenience binary release.
> >
> > The 4.4.0-HBase-1.0 release has feature parity with our pending
> > 4.4.0-HBase-0.98 release. New features include:
> > - Support HBase HA Query(timeline-consistent region replica read)[1]
> > - Alter session query support(at present changing query consistency
> level.
> > Can be used for changing connection properties.)
> >
> > The source tarball, including signatures, digests, etc can be found at:
> >
> >
> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/src/
> >
> > The binary artifacts can be found at:
> >
> >
> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.4.0-HBase-1.0-rc1/bin/
> >
> > Release artifacts are signed with the following key:
> > http://people.apache.org/~rajeshbabu/E3A65DBC.asc
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/phoenix/KEYS
> >
> > The hash and tag to be voted upon:
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=09d1840876d7b55e32d753b6666541eb7df22b85
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.4.0-HBase-1.0-rc1
> >
> > Vote will be open for at least 72 hours. Please vote:
> >
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove (and reason why)
> >
> > Thanks,
> > The Apache Phoenix Team
> >
> > [1] https://issues.apache.org/jira/browse/PHOENIX-1683
> >
>