You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by "kubilay.tsilkara" <ku...@gmail.com> on 2015/09/21 17:39:26 UTC
Is there a Pentaho connector for Phoenix
I've tried to connect to Phoenix using Pentaho PDI (aka Kettle)
<http://community.pentaho.com/projects/data-integration/>with no success.
Pentaho is an ETL tool which can do parallel loads to loads of endpoints,
including HBase, Hive etc... using JDBC/ODBC connectors. Is there one for
Phoenix? Does anybody know?
Thank you!
Kubilay
Re: Is there a Pentaho connector for Phoenix
Posted by "kubilay.tsilkara" <ku...@gmail.com>.
The phoenix drivers do let *Pentaho to connect to Phoenix via a Generic
Database pure JDBC* connection as described in
http://talat.uyarer.com/post/121179803796/how-to-connect-hbase-using-apache-phoenix-from
I can see mappings, fields from both sides, source and target!
But I can't do an UPSERT into Phoenix from RDBMS as what Pentaho generates
on the Target is INSERT SQL statements and the Phoenix driver does not
accept that.
Pentaho transformation log says
*2015/09/22 15:32:37 - Phoenix Table.0 - Caused by:
org.apache.phoenix.exception.PhoenixParserException: ERROR 601 (42P00):
Syntax error. Encountered "INSERT" at line 1, column 1.*
I guess I can write to Pentaho Forums and request *a connector for Phoenix
to be build*. They do provide exclusive UPSERT connectors to *Salesforce,
MongoDB* etc...
This missing Pentaho UPSERT connector for Phoenix (just changing the INSERT
to UPSERT) would ease parallel loads into *Phoenix via Pentaho ETL* very
much as anybody who uses Pentaho, or even other tools like Informatica
would be able to UPSERT directly into Phoenix from RDBMS sources.
Best Regards
Kubilay Tsil Kara
On 22 September 2015 at 11:56, kubilay.tsilkara <ku...@gmail.com>
wrote:
> Hi James
>
> Thank you!
>
> I have also found now a way to connect Pentaho PDI (aka Kettle) to
> Phoenix. I used steps from this blog
> http://talat.uyarer.com/post/121179803796/how-to-connect-hbase-using-apache-phoenix-from
>
> It works with generic Phoenix drivers, the use case is to create an ETL
> transformation from MySQL -> Phoenix to load couple of millions of rows.
> The transformation is valid I see the tables on both source and target.
>
> The hurdle I hit now, is that Pentaho source MySQL Sends INSERTs to
> Phoenix, whereas Phoenix expects UPSERTs so I have to do some sort of
> transformation in the middle to translate those INSERTs to UPSERTs with a
> 'decode' like transformation I suppose.
>
> Things look very promising so far.
>
> Cheers.
>
> Kubilay Tsil Kara
>
> Kubilay
>
> On 21 September 2015 at 19:05, James Taylor <ja...@apache.org>
> wrote:
>
>> Have you seen this blog post, as it details how to connect Phoenix
>> to Saiku through Pentaho?
>> https://blogs.apache.org/phoenix/entry/olap_with_apache_phoenix_and
>>
>> HTH. Thanks,
>>
>> James
>>
>> On Mon, Sep 21, 2015 at 8:39 AM, kubilay.tsilkara <
>> kubilay.tsilkara@gmail.com> wrote:
>>
>>> I've tried to connect to Phoenix using Pentaho PDI (aka Kettle)
>>> <http://community.pentaho.com/projects/data-integration/>with no
>>> success.
>>>
>>> Pentaho is an ETL tool which can do parallel loads to loads of
>>> endpoints, including HBase, Hive etc... using JDBC/ODBC connectors. Is
>>> there one for Phoenix? Does anybody know?
>>>
>>> Thank you!
>>>
>>> Kubilay
>>>
>>
>>
>
Re: Is there a Pentaho connector for Phoenix
Posted by "kubilay.tsilkara" <ku...@gmail.com>.
Hi James
Thank you!
I have also found now a way to connect Pentaho PDI (aka Kettle) to Phoenix.
I used steps from this blog
http://talat.uyarer.com/post/121179803796/how-to-connect-hbase-using-apache-phoenix-from
It works with generic Phoenix drivers, the use case is to create an ETL
transformation from MySQL -> Phoenix to load couple of millions of rows.
The transformation is valid I see the tables on both source and target.
The hurdle I hit now, is that Pentaho source MySQL Sends INSERTs to
Phoenix, whereas Phoenix expects UPSERTs so I have to do some sort of
transformation in the middle to translate those INSERTs to UPSERTs with a
'decode' like transformation I suppose.
Things look very promising so far.
Cheers.
Kubilay Tsil Kara
Kubilay
On 21 September 2015 at 19:05, James Taylor <ja...@apache.org> wrote:
> Have you seen this blog post, as it details how to connect Phoenix
> to Saiku through Pentaho?
> https://blogs.apache.org/phoenix/entry/olap_with_apache_phoenix_and
>
> HTH. Thanks,
>
> James
>
> On Mon, Sep 21, 2015 at 8:39 AM, kubilay.tsilkara <
> kubilay.tsilkara@gmail.com> wrote:
>
>> I've tried to connect to Phoenix using Pentaho PDI (aka Kettle)
>> <http://community.pentaho.com/projects/data-integration/>with no
>> success.
>>
>> Pentaho is an ETL tool which can do parallel loads to loads of endpoints,
>> including HBase, Hive etc... using JDBC/ODBC connectors. Is there one for
>> Phoenix? Does anybody know?
>>
>> Thank you!
>>
>> Kubilay
>>
>
>
Re: Is there a Pentaho connector for Phoenix
Posted by James Taylor <ja...@apache.org>.
Have you seen this blog post, as it details how to connect Phoenix to Saiku
through Pentaho?
https://blogs.apache.org/phoenix/entry/olap_with_apache_phoenix_and
HTH. Thanks,
James
On Mon, Sep 21, 2015 at 8:39 AM, kubilay.tsilkara <
kubilay.tsilkara@gmail.com> wrote:
> I've tried to connect to Phoenix using Pentaho PDI (aka Kettle)
> <http://community.pentaho.com/projects/data-integration/>with no success.
>
> Pentaho is an ETL tool which can do parallel loads to loads of endpoints,
> including HBase, Hive etc... using JDBC/ODBC connectors. Is there one for
> Phoenix? Does anybody know?
>
> Thank you!
>
> Kubilay
>