You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Mich Talebzadeh <mi...@gmail.com> on 2020/05/11 20:45:10 UTC
How to bulk load a csv file into a phoenix table created in lowercase
Hi,
I have a phoenix table *created in lowercase *as follows:
CREATE TABLE "MICHTEST"."StagingNotificationPreferencesRT"
(
ROWKEY VARCHAR NOT NULL PRIMARY KEY
, "cf"."partyId" VARCHAR
, "cf"."childNotificationId" VARCHAR
, "cf"."brand" VARCHAR
, "cf"."accountReference" VARCHAR
, "cf"."expiredDate" VARCHAR
, "cf"."parentNotificationId" VARCHAR
...
This table creates OK. When I try to bulk load from a csv file at Linux
bash command line as follows
*psql -t MICHTEST."StagingNotificationPreferencesRT"
./StagingNotificationPreferences.csv -s*
20/05/11 21:35:07 ERROR util.CSVCommonsLoader: Error upserting record
[3c7953b3-0c69-42ee-9abf-fe7a77d29c87, 876543914, 1, LTB, , , 50, 76543210,
, 2017-13-18 18:29:23:345, 19, 1, , , , , , , 9876543210123456]
java.lang.RuntimeException:
org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03): Table
undefined. tableName=MICHTEST.STAGINGNOTIFICATIONPREFERENCESRT
Note that the table name shown in error is uppercase! Now this works if I
go back and create table all in UPPERCASE!
Some documentation in below link
https://phoenix.apache.org/bulk_dataload.html
talks about using backslash with table name etc but that does not work/
Any ideas how to bulk load into table created in mixed case!
Regards,
Mich
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
Re: How to bulk load a csv file into a phoenix table created in lowercase
Posted by Mich Talebzadeh <mi...@gmail.com>.
Thanks.
Done,
https://issues.apache.org/jira/browse/PHOENIX-5900
<https://issues.apache.org/jira/browse/PHOENIX-5900>Thanks
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Thu, 14 May 2020 at 20:48, Ankit Singhal <an...@apache.org> wrote:
> Can you please try using CSVBulkLoad (as a workaround, we have it recently
> fixed) and raise a bug for the Psql not handling table-name case
> sensitivity.
> https://issues.apache.org/jira/browse/PHOENIX-3541
> https://issues.apache.org/jira/browse/PHOENIX-5319
>
> Regards,
> Ankit Singhal
>
>
>
> On Thu, May 14, 2020 at 12:15 PM Mich Talebzadeh <
> mich.talebzadeh@gmail.com> wrote:
>
>> Well.
>>
>> This was happening at work
>>
>> I think the question is not the shell but the way Phoenix interpret the
>> mix case. This works through once you are inside phoenix sql shell.
>> However, the way psql interprets it is not working.
>>
>> You can easily replicate it by creating the csv file and the mixed case
>> table yourself. any shell should do I believe.
>>
>> Thanks,
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Thu, 14 May 2020 at 19:00, Josh Elser <el...@apache.org> wrote:
>>
>>> Quoting can be really annoying.
>>>
>>> Remember that when you are executing commands, you have to deal with
>>> what your shell will do to quoting.
>>>
>>> You say that escaping the slashes doesn't work? e.g. `pqsl -t
>>> MICHTEST.\"StagingNotificationPreferencesRT\" ...`
>>>
>>> What else did you try? Have you checked to see if there are any known
>>> issues around quoting/naming?
>>>
>>> On 5/11/20 4:45 PM, Mich Talebzadeh wrote:
>>> > Hi,
>>> >
>>> > I have a phoenix table *created in lowercase *as follows:
>>> >
>>> > CREATE TABLE "MICHTEST"."StagingNotificationPreferencesRT"
>>> > (
>>> > ROWKEY VARCHAR NOT NULL PRIMARY KEY
>>> > , "cf"."partyId" VARCHAR
>>> > , "cf"."childNotificationId" VARCHAR
>>> > , "cf"."brand" VARCHAR
>>> > , "cf"."accountReference" VARCHAR
>>> > , "cf"."expiredDate" VARCHAR
>>> > , "cf"."parentNotificationId" VARCHAR
>>> > ...
>>> >
>>> > This table creates OK. When I try to bulk load from a csv file at
>>> Linux
>>> > bash command line as follows
>>> >
>>> > *psql -t MICHTEST."StagingNotificationPreferencesRT"
>>> > ./StagingNotificationPreferences.csv -s*
>>> >
>>> > 20/05/11 21:35:07 ERROR util.CSVCommonsLoader: Error upserting record
>>> > [3c7953b3-0c69-42ee-9abf-fe7a77d29c87, 876543914, 1, LTB, , , 50,
>>> > 76543210, , 2017-13-18 18:29:23:345, 19, 1, , , , , , ,
>>> 9876543210123456]
>>> >
>>> > java.lang.RuntimeException:
>>> > org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03):
>>> > Table undefined. tableName=MICHTEST.STAGINGNOTIFICATIONPREFERENCESRT
>>> >
>>> >
>>> > Note that the table name shown in error is uppercase! Now this works
>>> if
>>> > I go back and create table all in UPPERCASE!
>>> >
>>> >
>>> > Some documentation in below link
>>> >
>>> >
>>> > https://phoenix.apache.org/bulk_dataload.html
>>> >
>>> >
>>> > talks about using backslash with table name etc but that does not work/
>>> >
>>> >
>>> > Any ideas how to bulk load into table created in mixed case!
>>> >
>>> >
>>> > Regards,
>>> >
>>> >
>>> > Mich
>>> >
>>> >
>>> > Dr Mich Talebzadeh
>>> >
>>> > LinkedIn
>>> > /
>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw/
>>> >
>>> > http://talebzadehmich.wordpress.com
>>> >
>>> >
>>> > *Disclaimer:* Use it at your own risk.Any and all responsibility for
>>> any
>>> > loss, damage or destruction of data or any other property which may
>>> > arise from relying on this email's technical content is explicitly
>>> > disclaimed. The author will in no case be liable for any monetary
>>> > damages arising from such loss, damage or destruction.
>>> >
>>>
>>
Re: How to bulk load a csv file into a phoenix table created in lowercase
Posted by Ankit Singhal <an...@apache.org>.
Can you please try using CSVBulkLoad (as a workaround, we have it recently
fixed) and raise a bug for the Psql not handling table-name case
sensitivity.
https://issues.apache.org/jira/browse/PHOENIX-3541
https://issues.apache.org/jira/browse/PHOENIX-5319
Regards,
Ankit Singhal
On Thu, May 14, 2020 at 12:15 PM Mich Talebzadeh <mi...@gmail.com>
wrote:
> Well.
>
> This was happening at work
>
> I think the question is not the shell but the way Phoenix interpret the
> mix case. This works through once you are inside phoenix sql shell.
> However, the way psql interprets it is not working.
>
> You can easily replicate it by creating the csv file and the mixed case
> table yourself. any shell should do I believe.
>
> Thanks,
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Thu, 14 May 2020 at 19:00, Josh Elser <el...@apache.org> wrote:
>
>> Quoting can be really annoying.
>>
>> Remember that when you are executing commands, you have to deal with
>> what your shell will do to quoting.
>>
>> You say that escaping the slashes doesn't work? e.g. `pqsl -t
>> MICHTEST.\"StagingNotificationPreferencesRT\" ...`
>>
>> What else did you try? Have you checked to see if there are any known
>> issues around quoting/naming?
>>
>> On 5/11/20 4:45 PM, Mich Talebzadeh wrote:
>> > Hi,
>> >
>> > I have a phoenix table *created in lowercase *as follows:
>> >
>> > CREATE TABLE "MICHTEST"."StagingNotificationPreferencesRT"
>> > (
>> > ROWKEY VARCHAR NOT NULL PRIMARY KEY
>> > , "cf"."partyId" VARCHAR
>> > , "cf"."childNotificationId" VARCHAR
>> > , "cf"."brand" VARCHAR
>> > , "cf"."accountReference" VARCHAR
>> > , "cf"."expiredDate" VARCHAR
>> > , "cf"."parentNotificationId" VARCHAR
>> > ...
>> >
>> > This table creates OK. When I try to bulk load from a csv file at Linux
>> > bash command line as follows
>> >
>> > *psql -t MICHTEST."StagingNotificationPreferencesRT"
>> > ./StagingNotificationPreferences.csv -s*
>> >
>> > 20/05/11 21:35:07 ERROR util.CSVCommonsLoader: Error upserting record
>> > [3c7953b3-0c69-42ee-9abf-fe7a77d29c87, 876543914, 1, LTB, , , 50,
>> > 76543210, , 2017-13-18 18:29:23:345, 19, 1, , , , , , ,
>> 9876543210123456]
>> >
>> > java.lang.RuntimeException:
>> > org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03):
>> > Table undefined. tableName=MICHTEST.STAGINGNOTIFICATIONPREFERENCESRT
>> >
>> >
>> > Note that the table name shown in error is uppercase! Now this works if
>> > I go back and create table all in UPPERCASE!
>> >
>> >
>> > Some documentation in below link
>> >
>> >
>> > https://phoenix.apache.org/bulk_dataload.html
>> >
>> >
>> > talks about using backslash with table name etc but that does not work/
>> >
>> >
>> > Any ideas how to bulk load into table created in mixed case!
>> >
>> >
>> > Regards,
>> >
>> >
>> > Mich
>> >
>> >
>> > Dr Mich Talebzadeh
>> >
>> > LinkedIn
>> > /
>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw/
>> >
>> > http://talebzadehmich.wordpress.com
>> >
>> >
>> > *Disclaimer:* Use it at your own risk.Any and all responsibility for
>> any
>> > loss, damage or destruction of data or any other property which may
>> > arise from relying on this email's technical content is explicitly
>> > disclaimed. The author will in no case be liable for any monetary
>> > damages arising from such loss, damage or destruction.
>> >
>>
>
Re: How to bulk load a csv file into a phoenix table created in lowercase
Posted by Mich Talebzadeh <mi...@gmail.com>.
Well.
This was happening at work
I think the question is not the shell but the way Phoenix interpret the mix
case. This works through once you are inside phoenix sql shell. However,
the way psql interprets it is not working.
You can easily replicate it by creating the csv file and the mixed case
table yourself. any shell should do I believe.
Thanks,
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Thu, 14 May 2020 at 19:00, Josh Elser <el...@apache.org> wrote:
> Quoting can be really annoying.
>
> Remember that when you are executing commands, you have to deal with
> what your shell will do to quoting.
>
> You say that escaping the slashes doesn't work? e.g. `pqsl -t
> MICHTEST.\"StagingNotificationPreferencesRT\" ...`
>
> What else did you try? Have you checked to see if there are any known
> issues around quoting/naming?
>
> On 5/11/20 4:45 PM, Mich Talebzadeh wrote:
> > Hi,
> >
> > I have a phoenix table *created in lowercase *as follows:
> >
> > CREATE TABLE "MICHTEST"."StagingNotificationPreferencesRT"
> > (
> > ROWKEY VARCHAR NOT NULL PRIMARY KEY
> > , "cf"."partyId" VARCHAR
> > , "cf"."childNotificationId" VARCHAR
> > , "cf"."brand" VARCHAR
> > , "cf"."accountReference" VARCHAR
> > , "cf"."expiredDate" VARCHAR
> > , "cf"."parentNotificationId" VARCHAR
> > ...
> >
> > This table creates OK. When I try to bulk load from a csv file at Linux
> > bash command line as follows
> >
> > *psql -t MICHTEST."StagingNotificationPreferencesRT"
> > ./StagingNotificationPreferences.csv -s*
> >
> > 20/05/11 21:35:07 ERROR util.CSVCommonsLoader: Error upserting record
> > [3c7953b3-0c69-42ee-9abf-fe7a77d29c87, 876543914, 1, LTB, , , 50,
> > 76543210, , 2017-13-18 18:29:23:345, 19, 1, , , , , , , 9876543210123456]
> >
> > java.lang.RuntimeException:
> > org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03):
> > Table undefined. tableName=MICHTEST.STAGINGNOTIFICATIONPREFERENCESRT
> >
> >
> > Note that the table name shown in error is uppercase! Now this works if
> > I go back and create table all in UPPERCASE!
> >
> >
> > Some documentation in below link
> >
> >
> > https://phoenix.apache.org/bulk_dataload.html
> >
> >
> > talks about using backslash with table name etc but that does not work/
> >
> >
> > Any ideas how to bulk load into table created in mixed case!
> >
> >
> > Regards,
> >
> >
> > Mich
> >
> >
> > Dr Mich Talebzadeh
> >
> > LinkedIn
> > /
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw/
> >
> > http://talebzadehmich.wordpress.com
> >
> >
> > *Disclaimer:* Use it at your own risk.Any and all responsibility for any
> > loss, damage or destruction of data or any other property which may
> > arise from relying on this email's technical content is explicitly
> > disclaimed. The author will in no case be liable for any monetary
> > damages arising from such loss, damage or destruction.
> >
>
Re: How to bulk load a csv file into a phoenix table created in
lowercase
Posted by Josh Elser <el...@apache.org>.
Quoting can be really annoying.
Remember that when you are executing commands, you have to deal with
what your shell will do to quoting.
You say that escaping the slashes doesn't work? e.g. `pqsl -t
MICHTEST.\"StagingNotificationPreferencesRT\" ...`
What else did you try? Have you checked to see if there are any known
issues around quoting/naming?
On 5/11/20 4:45 PM, Mich Talebzadeh wrote:
> Hi,
>
> I have a phoenix table *created in lowercase *as follows:
>
> CREATE TABLE "MICHTEST"."StagingNotificationPreferencesRT"
> (
> ROWKEY VARCHAR NOT NULL PRIMARY KEY
> , "cf"."partyId" VARCHAR
> , "cf"."childNotificationId" VARCHAR
> , "cf"."brand" VARCHAR
> , "cf"."accountReference" VARCHAR
> , "cf"."expiredDate" VARCHAR
> , "cf"."parentNotificationId" VARCHAR
> ...
>
> This table creates OK. When I try to bulk load from a csv file at Linux
> bash command line as follows
>
> *psql -t MICHTEST."StagingNotificationPreferencesRT"
> ./StagingNotificationPreferences.csv -s*
>
> 20/05/11 21:35:07 ERROR util.CSVCommonsLoader: Error upserting record
> [3c7953b3-0c69-42ee-9abf-fe7a77d29c87, 876543914, 1, LTB, , , 50,
> 76543210, , 2017-13-18 18:29:23:345, 19, 1, , , , , , , 9876543210123456]
>
> java.lang.RuntimeException:
> org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03):
> Table undefined. tableName=MICHTEST.STAGINGNOTIFICATIONPREFERENCESRT
>
>
> Note that the table name shown in error is uppercase! Now this works if
> I go back and create table all in UPPERCASE!
>
>
> Some documentation in below link
>
>
> https://phoenix.apache.org/bulk_dataload.html
>
>
> talks about using backslash with table name etc but that does not work/
>
>
> Any ideas how to bulk load into table created in mixed case!
>
>
> Regards,
>
>
> Mich
>
>
> Dr Mich Talebzadeh
>
> LinkedIn
> /https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw/
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk.Any and all responsibility for any
> loss, damage or destruction of data or any other property which may
> arise from relying on this email's technical content is explicitly
> disclaimed. The author will in no case be liable for any monetary
> damages arising from such loss, damage or destruction.
>