You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Siddharth Ubale <si...@syncoms.com> on 2015/01/02 08:21:08 UTC

Phoenix in production

Hi Guys,

We are seriously thinking of phoenix in Production environment , however, we have no much data of how Phoenix is behaving in production.
Can anyone let us know if anyone is using Phoenix in Production and any challenges which they have experienced.

Thanks,
Siddharth Ubale,
Synchronized Communications
#43, Velankani Tech Park, Block No. II,
3rd Floor, Electronic City Phase I,
Bangalore – 560 100
Tel : +91 80 3202 4060
Web: www.syncoms.com<http://www.syncoms.com/>
[LogoNEWmohLARGE]
London|Bangalore|Orlando

we innovate, plan, execute, and transform the business​


Re: Phoenix in production

Posted by anil gupta <an...@gmail.com>.
Hi Siddharth,

I havent used Phoenix with Storm but  I have used HBase with Storm.
IMHO, it should be fairly simple to write your own connector(code in
Spout/Bolt) to talk with Phoenix. It should mostly be same code that you
use in standalone Phoenix java app.

Thanks,
Anil Gupta

On Tue, Jan 6, 2015 at 11:26 PM, Siddharth Ubale <
siddharth.ubale@syncoms.com> wrote:

>  Hi Anil,
>
>
>
> Thanks a lot for sharing ur thoughts.
>
> Have you tried using Phoenix API along with Apache Storm?
>
> There is a storm RDBMS connector implementation , but it was last updated
> in 2012 and I am having problems with it.
>
> Any thoughts on Storm-Phoenix integration?
>
>
>
> Thanks,
>
> Siddharth ubale
>
>
>
> *From:* anil gupta [mailto:anilgupta84@gmail.com]
> *Sent:* Wednesday, January 07, 2015 12:49 PM
> *To:* user@phoenix.apache.org
> *Cc:* James Taylor
> *Subject:* Re: Phoenix in production
>
>
>
> Hi Siddharth,
>
> I have used Phoenix in production for more than a year at Intuit.(i no
> longer work there.) I was pretty happy with Phoenix because it provided us
> capability to use sql-like(JDBC compliant) querying on a NoSql DB. With
> help of Phoenix, the adoption rate of HBase in my team was much more faster
> and it makes it very easy to read data from HBase Table.(without Phoenix, i
> personally used to write java programs for queries with complex filters.or
> non-string columns).
>
> Following are the some of the challenges i faced while using Phoenix:
>
> 1. Connecting to secure HBase cluster.
>
> 2. Phoenix creating big files under /tmp folder when a "select *" type
> query is done.
>
> 3. Phoenix only used to work with SQuirreL(GUI based DB client)
> <http://squirrel-sql.sourceforge.net/>
>
> I know for sure that #1 and #3 are addressed. There was jira for #2, i m
> not sure whats the status of it right now.
>
> Obviously,i haven't used all the features of Phoenix. So, i would suggest
> you to do your own evaluation(POC) and take guidance from the Phoenix
> community if you are planning to use any specific feature. This community
> is quite active.
>
>
>
> Hope This Helps,
>
> Anil Gupta
>
>
>
>
>
> On Thu, Jan 1, 2015 at 11:21 PM, Siddharth Ubale <
> siddharth.ubale@syncoms.com> wrote:
>
>  Hi Guys,
>
>
>
> We are seriously thinking of phoenix in Production environment , however,
> we have no much data of how Phoenix is behaving in production.
>
> Can anyone let us know if anyone is using Phoenix in Production and any
> challenges which they have experienced.
>
>
>
> Thanks,
>
> Siddharth Ubale,
>
> *Synchronized Communications *
>
> *#43, Velankani Tech Park, Block No. II, *
>
> *3rd Floor, Electronic City Phase I,*
>
> *Bangalore – 560 100*
>
> *Tel : +91 80 3202 4060 <%2B91%2080%203202%204060>*
>
> *Web:* *www.syncoms.com* <http://www.syncoms.com/>
>
> *[image: LogoNEWmohLARGE]*
>
> *London*|*Bangalore*|*Orlando*
>
>
>
> *we innovate, plan, execute, and transform the business​*
>
>
>
>
>
>
> --
>
> Thanks & Regards,
> Anil Gupta
>



-- 
Thanks & Regards,
Anil Gupta

Re: Re: Phoenix in production

Posted by "sunfl@certusnet.com.cn" <su...@certusnet.com.cn>.
Hi, 

Glad to share our experience of using Phoenix in Production. I believe that Siddharth had done 
sufficient tests and practices about Phoenix performance. Here are some tips about how we are using
Phoenix for our projects:
1. We facilitate Phoenix to give convinience for both RD and QA engineers, as they are glad to use 
standard sql to operate hbase with no much loss of query performance. 
2. In Production environment, we mainly integrate Apache Spark with Phoenix to optimize data loading to
Phoenix tables with or withour secondary indexes. Glad that current performance of writing had worked 
smoothly with both compared to previously used MySQL InfoBright and other sql schema. We both had tested 
a lot for secondary indexes and query optimization for Phoenix before moving Phoenix to Production 
environment. Now we can get most of the features worked for Phoenix in our job. 
3. Challenges had a lot too, such as bulkload performance with wal enabled, query optimization, statistical data collection with Phoenix
full table scan, and so on. However, we believe Phoenix be a sufficient schema for sql query over HBase and we are glad 
that even more our projects are considering using Phoenix.

Thanks,
Sun.






CertusNet 

From: Justin Workman
Date: 2015-01-07 15:54
To: user@phoenix.apache.org
Subject: Re: Phoenix in production
I am also using Phoenix in production and have been now for roughly 6 months. We adopted Phoenix for most of the same reasons Anil mentions. 

We are connection to a secure cluster without issue. We have also implemented our own storm persistence bolt, and a lookup bolt that are reading and writing to our Phoenix/Hbase cluster. 

I have still had problems getting a secure connection from client tools for adhoc analysis other than sqlline. Still fighting though connections from SquirrelSQL, although I haven't dedicated a whole lot of time trying to figure it out. 

Thanks
Justin

Sent from my iPhone

On Jan 7, 2015, at 12:26 AM, Siddharth Ubale <si...@syncoms.com> wrote:

Hi Anil,
 
Thanks a lot for sharing ur thoughts.
Have you tried using Phoenix API along with Apache Storm?
There is a storm RDBMS connector implementation , but it was last updated in 2012 and I am having problems with it.
Any thoughts on Storm-Phoenix integration?
 
Thanks,
Siddharth ubale
 
From: anil gupta [mailto:anilgupta84@gmail.com] 
Sent: Wednesday, January 07, 2015 12:49 PM
To: user@phoenix.apache.org
Cc: James Taylor
Subject: Re: Phoenix in production
 
Hi Siddharth,
I have used Phoenix in production for more than a year at Intuit.(i no longer work there.) I was pretty happy with Phoenix because it provided us capability to use sql-like(JDBC compliant) querying on a NoSql DB. With help of Phoenix, the adoption rate of HBase in my team was much more faster and it makes it very easy to read data from HBase Table.(without Phoenix, i personally used to write java programs for queries with complex filters.or non-string columns).
Following are the some of the challenges i faced while using Phoenix:
1. Connecting to secure HBase cluster.
2. Phoenix creating big files under /tmp folder when a "select *" type query is done.
3. Phoenix only used to work with SQuirreL(GUI based DB client)
I know for sure that #1 and #3 are addressed. There was jira for #2, i m not sure whats the status of it right now.
Obviously,i haven't used all the features of Phoenix. So, i would suggest you to do your own evaluation(POC) and take guidance from the Phoenix community if you are planning to use any specific feature. This community is quite active. 
 
Hope This Helps,
Anil Gupta
 
 
On Thu, Jan 1, 2015 at 11:21 PM, Siddharth Ubale <si...@syncoms.com> wrote:
Hi Guys,
 
We are seriously thinking of phoenix in Production environment , however, we have no much data of how Phoenix is behaving in production.
Can anyone let us know if anyone is using Phoenix in Production and any challenges which they have experienced.
 
Thanks,
Siddharth Ubale,
Synchronized Communications 
#43, Velankani Tech Park, Block No. II, 
3rd Floor, Electronic City Phase I,
Bangalore – 560 100
Tel : +91 80 3202 4060
Web: www.syncoms.com
<image001.jpg>
London|Bangalore|Orlando
 
we innovate, plan, execute, and transform the business​
 



-- 
Thanks & Regards,
Anil Gupta

Re: Phoenix in production

Posted by anil gupta <an...@gmail.com>.
Inline.

On Tue, Jan 6, 2015 at 11:54 PM, Justin Workman <ju...@gmail.com>
wrote:

> I am also using Phoenix in production and have been now for roughly 6
> months. We adopted Phoenix for most of the same reasons Anil mentions.
>
> We are connection to a secure cluster without issue. We have also
> implemented our own storm persistence bolt, and a lookup bolt that are
> reading and writing to our Phoenix/Hbase cluster.
>
> I have still had problems getting a secure connection from client tools
> for adhoc analysis other than sqlline. Still fighting though connections
> from SquirrelSQL, although I haven't dedicated a whole lot of time trying
> to figure it out.
>
Anil: We were successfully able to connect to secure HBase cluster via
SquirellSQL. What version of Phoenix you were using? What steps you follow
to connect to secure cluster? Can you share the error message?

>
> Thanks
> Justin
>
> Sent from my iPhone
>
> On Jan 7, 2015, at 12:26 AM, Siddharth Ubale <si...@syncoms.com>
> wrote:
>
>  Hi Anil,
>
>
>
> Thanks a lot for sharing ur thoughts.
>
> Have you tried using Phoenix API along with Apache Storm?
>
> There is a storm RDBMS connector implementation , but it was last updated
> in 2012 and I am having problems with it.
>
> Any thoughts on Storm-Phoenix integration?
>
>
>
> Thanks,
>
> Siddharth ubale
>
>
>
> *From:* anil gupta [mailto:anilgupta84@gmail.com <an...@gmail.com>]
> *Sent:* Wednesday, January 07, 2015 12:49 PM
> *To:* user@phoenix.apache.org
> *Cc:* James Taylor
> *Subject:* Re: Phoenix in production
>
>
>
> Hi Siddharth,
>
> I have used Phoenix in production for more than a year at Intuit.(i no
> longer work there.) I was pretty happy with Phoenix because it provided us
> capability to use sql-like(JDBC compliant) querying on a NoSql DB. With
> help of Phoenix, the adoption rate of HBase in my team was much more faster
> and it makes it very easy to read data from HBase Table.(without Phoenix, i
> personally used to write java programs for queries with complex filters.or
> non-string columns).
>
> Following are the some of the challenges i faced while using Phoenix:
>
> 1. Connecting to secure HBase cluster.
>
> 2. Phoenix creating big files under /tmp folder when a "select *" type
> query is done.
>
> 3. Phoenix only used to work with SQuirreL(GUI based DB client)
> <http://squirrel-sql.sourceforge.net/>
>
> I know for sure that #1 and #3 are addressed. There was jira for #2, i m
> not sure whats the status of it right now.
>
> Obviously,i haven't used all the features of Phoenix. So, i would suggest
> you to do your own evaluation(POC) and take guidance from the Phoenix
> community if you are planning to use any specific feature. This community
> is quite active.
>
>
>
> Hope This Helps,
>
> Anil Gupta
>
>
>
>
>
> On Thu, Jan 1, 2015 at 11:21 PM, Siddharth Ubale <
> siddharth.ubale@syncoms.com> wrote:
>
>  Hi Guys,
>
>
>
> We are seriously thinking of phoenix in Production environment , however,
> we have no much data of how Phoenix is behaving in production.
>
> Can anyone let us know if anyone is using Phoenix in Production and any
> challenges which they have experienced.
>
>
>
> Thanks,
>
> Siddharth Ubale,
>
> *Synchronized Communications *
>
> *#43, Velankani Tech Park, Block No. II, *
>
> *3rd Floor, Electronic City Phase I,*
>
> *Bangalore – 560 100*
>
> *Tel : +91 80 3202 4060 <%2B91%2080%203202%204060>*
>
> *Web:* *www.syncoms.com* <http://www.syncoms.com/>
>
> *<image001.jpg>*
>
> *London*|*Bangalore*|*Orlando*
>
>
>
> *we innovate, plan, execute, and transform the business​*
>
>
>
>
>
>
> --
>
> Thanks & Regards,
> Anil Gupta
>
>


-- 
Thanks & Regards,
Anil Gupta

Re: Phoenix in production

Posted by Justin Workman <ju...@gmail.com>.
I am also using Phoenix in production and have been now for roughly 6 months. We adopted Phoenix for most of the same reasons Anil mentions. 

We are connection to a secure cluster without issue. We have also implemented our own storm persistence bolt, and a lookup bolt that are reading and writing to our Phoenix/Hbase cluster. 

I have still had problems getting a secure connection from client tools for adhoc analysis other than sqlline. Still fighting though connections from SquirrelSQL, although I haven't dedicated a whole lot of time trying to figure it out. 

Thanks
Justin

Sent from my iPhone

> On Jan 7, 2015, at 12:26 AM, Siddharth Ubale <si...@syncoms.com> wrote:
> 
> Hi Anil,
>  
> Thanks a lot for sharing ur thoughts.
> Have you tried using Phoenix API along with Apache Storm?
> There is a storm RDBMS connector implementation , but it was last updated in 2012 and I am having problems with it.
> Any thoughts on Storm-Phoenix integration?
>  
> Thanks,
> Siddharth ubale
>  
> From: anil gupta [mailto:anilgupta84@gmail.com] 
> Sent: Wednesday, January 07, 2015 12:49 PM
> To: user@phoenix.apache.org
> Cc: James Taylor
> Subject: Re: Phoenix in production
>  
> Hi Siddharth,
> 
> I have used Phoenix in production for more than a year at Intuit.(i no longer work there.) I was pretty happy with Phoenix because it provided us capability to use sql-like(JDBC compliant) querying on a NoSql DB. With help of Phoenix, the adoption rate of HBase in my team was much more faster and it makes it very easy to read data from HBase Table.(without Phoenix, i personally used to write java programs for queries with complex filters.or non-string columns).
> 
> Following are the some of the challenges i faced while using Phoenix:
> 1. Connecting to secure HBase cluster.
> 2. Phoenix creating big files under /tmp folder when a "select *" type query is done.
> 3. Phoenix only used to work with SQuirreL(GUI based DB client)
> 
> I know for sure that #1 and #3 are addressed. There was jira for #2, i m not sure whats the status of it right now.
> 
> Obviously,i haven't used all the features of Phoenix. So, i would suggest you to do your own evaluation(POC) and take guidance from the Phoenix community if you are planning to use any specific feature. This community is quite active.
>  
> Hope This Helps,
> Anil Gupta
>  
>  
> On Thu, Jan 1, 2015 at 11:21 PM, Siddharth Ubale <si...@syncoms.com> wrote:
> Hi Guys,
>  
> We are seriously thinking of phoenix in Production environment , however, we have no much data of how Phoenix is behaving in production.
> Can anyone let us know if anyone is using Phoenix in Production and any challenges which they have experienced.
>  
> Thanks,
> Siddharth Ubale,
> Synchronized Communications
> #43, Velankani Tech Park, Block No. II,
> 3rd Floor, Electronic City Phase I,
> Bangalore – 560 100
> Tel : +91 80 3202 4060
> Web: www.syncoms.com
> <image001.jpg>
> London|Bangalore|Orlando
>  
> we innovate, plan, execute, and transform the business​
>  
> 
> 
> 
> --
> Thanks & Regards,
> Anil Gupta

RE: Phoenix in production

Posted by Siddharth Ubale <si...@syncoms.com>.
Hi Anil,

Thanks a lot for sharing ur thoughts.
Have you tried using Phoenix API along with Apache Storm?
There is a storm RDBMS connector implementation , but it was last updated in 2012 and I am having problems with it.
Any thoughts on Storm-Phoenix integration?

Thanks,
Siddharth ubale

From: anil gupta [mailto:anilgupta84@gmail.com]
Sent: Wednesday, January 07, 2015 12:49 PM
To: user@phoenix.apache.org
Cc: James Taylor
Subject: Re: Phoenix in production

Hi Siddharth,
I have used Phoenix in production for more than a year at Intuit.(i no longer work there.) I was pretty happy with Phoenix because it provided us capability to use sql-like(JDBC compliant) querying on a NoSql DB. With help of Phoenix, the adoption rate of HBase in my team was much more faster and it makes it very easy to read data from HBase Table.(without Phoenix, i personally used to write java programs for queries with complex filters.or non-string columns).
Following are the some of the challenges i faced while using Phoenix:
1. Connecting to secure HBase cluster.
2. Phoenix creating big files under /tmp folder when a "select *" type query is done.
3. Phoenix only used to work with SQuirreL(GUI based DB client)<http://squirrel-sql.sourceforge.net/>
I know for sure that #1 and #3 are addressed. There was jira for #2, i m not sure whats the status of it right now.
Obviously,i haven't used all the features of Phoenix. So, i would suggest you to do your own evaluation(POC) and take guidance from the Phoenix community if you are planning to use any specific feature. This community is quite active.

Hope This Helps,
Anil Gupta


On Thu, Jan 1, 2015 at 11:21 PM, Siddharth Ubale <si...@syncoms.com>> wrote:
Hi Guys,

We are seriously thinking of phoenix in Production environment , however, we have no much data of how Phoenix is behaving in production.
Can anyone let us know if anyone is using Phoenix in Production and any challenges which they have experienced.

Thanks,
Siddharth Ubale,
Synchronized Communications
#43, Velankani Tech Park, Block No. II,
3rd Floor, Electronic City Phase I,
Bangalore – 560 100
Tel : +91 80 3202 4060<tel:%2B91%2080%203202%204060>
Web: www.syncoms.com<http://www.syncoms.com/>
[LogoNEWmohLARGE]
London|Bangalore|Orlando

we innovate, plan, execute, and transform the business​




--
Thanks & Regards,
Anil Gupta

Re: Phoenix in production

Posted by anil gupta <an...@gmail.com>.
Hi Siddharth,

I have used Phoenix in production for more than a year at Intuit.(i no
longer work there.) I was pretty happy with Phoenix because it provided us
capability to use sql-like(JDBC compliant) querying on a NoSql DB. With
help of Phoenix, the adoption rate of HBase in my team was much more faster
and it makes it very easy to read data from HBase Table.(without Phoenix, i
personally used to write java programs for queries with complex filters.or
non-string columns).

Following are the some of the challenges i faced while using Phoenix:
1. Connecting to secure HBase cluster.
2. Phoenix creating big files under /tmp folder when a "select *" type
query is done.
3. Phoenix only used to work with SQuirreL(GUI based DB client)
<http://squirrel-sql.sourceforge.net/>

I know for sure that #1 and #3 are addressed. There was jira for #2, i m
not sure whats the status of it right now.

Obviously,i haven't used all the features of Phoenix. So, i would suggest
you to do your own evaluation(POC) and take guidance from the Phoenix
community if you are planning to use any specific feature. This community
is quite active.

Hope This Helps,
Anil Gupta
<http://squirrel-sql.sourceforge.net/>


On Thu, Jan 1, 2015 at 11:21 PM, Siddharth Ubale <
siddharth.ubale@syncoms.com> wrote:

>  Hi Guys,
>
>
>
> We are seriously thinking of phoenix in Production environment , however,
> we have no much data of how Phoenix is behaving in production.
>
> Can anyone let us know if anyone is using Phoenix in Production and any
> challenges which they have experienced.
>
>
>
> Thanks,
>
> Siddharth Ubale,
>
> *Synchronized Communications *
>
> *#43, Velankani Tech Park, Block No. II, *
>
> *3rd Floor, Electronic City Phase I,*
>
> *Bangalore – 560 100*
>
> *Tel : +91 80 3202 4060 <%2B91%2080%203202%204060>*
>
> *Web:* *www.syncoms.com* <http://www.syncoms.com/>
>
> *[image: LogoNEWmohLARGE]*
>
> *London*|*Bangalore*|*Orlando*
>
>
>
> *we innovate, plan, execute, and transform the business​*
>
>
>



-- 
Thanks & Regards,
Anil Gupta

Re: Re: Fwd: Phoenix in production

Posted by Kristoffer Sjögren <st...@gmail.com>.
We have been using Phoenix 2.2.3 in production for about a year and I agree
with the previous comments.

- Mainly a storage for temporal OLAP-like data in single tables without
secondary indexes.
- Data ingested via pig on hourly basis.
- Heavy usage of composite primary keys using skip-scans whenever possible.
- Use salting.
- Jooq's MySQL dialect works quite well with Phoenix.
- The only real downtime we had was when we mistakenly connected to Phoenix
using a newer client which automatically upgraded the cluster. The forced
us to upgrade all region servers with new phoenix jar files.
- We are going to upgrade to Phoenix 3.0 sometime soon.

All and all happy with Phoenix. Just make sure to read the documentation
and follow advice found on http://phoenix.apache.org and you'll be fine.

Cheers,
-Kristoffer



On Wed, Jan 7, 2015 at 9:17 AM, sunfl@certusnet.com.cn <
sunfl@certusnet.com.cn> wrote:

> Hi,
> spark-phoenix integration would be great as Spark community is greately
> active now and more
> and more developers are using Apache Spark.
>
> Thanks,
> Sun.
>
> ------------------------------
> ------------------------------
>
>
> *From:* James Taylor <ja...@apache.org>
> *Date:* 2015-01-07 16:10
> *To:* sunfl@certusnet.com.cn
> *Subject:* Re: Fwd: Phoenix in production
> This is great, Sun! Thank you so much. Would you mind posting this on our
> user list in response to Siddharth's email? I think other Phoenix users
> would find it interesting as well.
>
> On a side note, not sure how general what you developed is, but it would
> be interesting to pursue a general Spark integration in Phoenix as an open
> source contribution.
>
> Thanks,
> James
>
> On Tue, Jan 6, 2015 at 5:41 PM, sunfl@certusnet.com.cn <
> sunfl@certusnet.com.cn> wrote:
>
>> Hi, James & Siddharth
>>
>> Glad to share our experience of using Phoenix in Production. I believe
>> that Siddharth had done
>> sufficient tests and practices about Phoenix performance. Here are some
>> tips about how we are using
>> Phoenix for our projects:
>> 1. We facilitate Phoenix to give convinience for both RD and QA
>> engineers, as they are glad to use
>> standard sql to operate hbase with no much loss of query performance.
>> 2. In Production environment, we mainly integrate Apache Spark with
>> Phoenix to optimize data loading to
>> Phoenix tables with or withour secondary indexes. Glad that current
>> performance of writing had worked
>> smoothly with both compared to previously used MySQL InfoBright and other
>> sql schema. We both had tested
>> a lot for secondary indexes and query optimization for Phoenix before
>> moving Phoenix to Production
>> environment. Now we can get most of the features worked for Phoenix in
>> our job.
>> 3. Challenges had a lot too, such as bulkload performance with wal
>> enabled, query optimization, statistical data collection with Phoenix
>> full table scan, and so on. However, we believe Phoenix be a sufficient
>> schema for sql query over HBase and we are glad
>> that even more our projects are considering using Phoenix.
>>
>> Thanks,
>> Sun.
>>
>> ------------------------------
>> ------------------------------
>>
>> CertusNet
>>
>>
>> *From:* James Taylor <ja...@apache.org>
>> *Date:* 2015-01-07 09:10
>> *To:* sunfl@certusnet.com.cn
>> *Subject:* Fwd: Phoenix in production
>> Hi Sun,
>> Any experiences you can share with Siddharth?
>> Thanks,
>> James
>>
>> ---------- Forwarded message ----------
>> From: Siddharth Ubale <si...@syncoms.com>
>> Date: Thu, Jan 1, 2015 at 11:21 PM
>> Subject: Phoenix in production
>> To: "user@phoenix.apache.org" <us...@phoenix.apache.org>
>>
>>
>>  Hi Guys,
>>
>>
>>
>> We are seriously thinking of phoenix in Production environment , however,
>> we have no much data of how Phoenix is behaving in production.
>>
>> Can anyone let us know if anyone is using Phoenix in Production and any
>> challenges which they have experienced.
>>
>>
>>
>> Thanks,
>>
>> Siddharth Ubale,
>>
>> *Synchronized Communications *
>>
>> *#43, Velankani Tech Park, Block No. II, *
>>
>> *3rd Floor, Electronic City Phase I,*
>>
>> *Bangalore – 560 100*
>>
>> *Tel : +91 80 3202 4060 <%2B91%2080%203202%204060>*
>>
>> *Web:* *www.syncoms.com* <http://www.syncoms.com/>
>>
>> *[image: LogoNEWmohLARGE]*
>>
>> *London*|*Bangalore*|*Orlando*
>>
>>
>>
>> *we innovate, plan, execute, and transform the business​*
>>
>>
>>
>>
>

Re: Re: Fwd: Phoenix in production

Posted by Kristoffer Sjögren <st...@gmail.com>.
We have been using Phoenix 2.2.3 in production for about a year and I agree
with the previous comments.

- Mainly a storage for temporal OLAP-like data in single tables without
secondary indexes.
- Data ingested via pig on hourly basis.
- Heavy usage of composite primary keys using skip-scans whenever possible.
- Use salting.
- Jooq's MySQL dialect works quite well with Phoenix.
- The only real downtime we had was when we mistakenly connected to Phoenix
using a newer client which automatically upgraded the cluster. The forced
us to upgrade all region servers with new phoenix jar files.
- We are going to upgrade to Phoenix 3.0 sometime soon.

All and all happy with Phoenix. Just make sure to read the documentation
and follow advice found on http://phoenix.apache.org and you'll be fine.

Cheers,
-Kristoffer



On Wed, Jan 7, 2015 at 9:17 AM, sunfl@certusnet.com.cn <
sunfl@certusnet.com.cn> wrote:

> Hi,
> spark-phoenix integration would be great as Spark community is greately
> active now and more
> and more developers are using Apache Spark.
>
> Thanks,
> Sun.
>
> ------------------------------
> ------------------------------
>
>
> *From:* James Taylor <ja...@apache.org>
> *Date:* 2015-01-07 16:10
> *To:* sunfl@certusnet.com.cn
> *Subject:* Re: Fwd: Phoenix in production
> This is great, Sun! Thank you so much. Would you mind posting this on our
> user list in response to Siddharth's email? I think other Phoenix users
> would find it interesting as well.
>
> On a side note, not sure how general what you developed is, but it would
> be interesting to pursue a general Spark integration in Phoenix as an open
> source contribution.
>
> Thanks,
> James
>
> On Tue, Jan 6, 2015 at 5:41 PM, sunfl@certusnet.com.cn <
> sunfl@certusnet.com.cn> wrote:
>
>> Hi, James & Siddharth
>>
>> Glad to share our experience of using Phoenix in Production. I believe
>> that Siddharth had done
>> sufficient tests and practices about Phoenix performance. Here are some
>> tips about how we are using
>> Phoenix for our projects:
>> 1. We facilitate Phoenix to give convinience for both RD and QA
>> engineers, as they are glad to use
>> standard sql to operate hbase with no much loss of query performance.
>> 2. In Production environment, we mainly integrate Apache Spark with
>> Phoenix to optimize data loading to
>> Phoenix tables with or withour secondary indexes. Glad that current
>> performance of writing had worked
>> smoothly with both compared to previously used MySQL InfoBright and other
>> sql schema. We both had tested
>> a lot for secondary indexes and query optimization for Phoenix before
>> moving Phoenix to Production
>> environment. Now we can get most of the features worked for Phoenix in
>> our job.
>> 3. Challenges had a lot too, such as bulkload performance with wal
>> enabled, query optimization, statistical data collection with Phoenix
>> full table scan, and so on. However, we believe Phoenix be a sufficient
>> schema for sql query over HBase and we are glad
>> that even more our projects are considering using Phoenix.
>>
>> Thanks,
>> Sun.
>>
>> ------------------------------
>> ------------------------------
>>
>> CertusNet
>>
>>
>> *From:* James Taylor <ja...@apache.org>
>> *Date:* 2015-01-07 09:10
>> *To:* sunfl@certusnet.com.cn
>> *Subject:* Fwd: Phoenix in production
>> Hi Sun,
>> Any experiences you can share with Siddharth?
>> Thanks,
>> James
>>
>> ---------- Forwarded message ----------
>> From: Siddharth Ubale <si...@syncoms.com>
>> Date: Thu, Jan 1, 2015 at 11:21 PM
>> Subject: Phoenix in production
>> To: "user@phoenix.apache.org" <us...@phoenix.apache.org>
>>
>>
>>  Hi Guys,
>>
>>
>>
>> We are seriously thinking of phoenix in Production environment , however,
>> we have no much data of how Phoenix is behaving in production.
>>
>> Can anyone let us know if anyone is using Phoenix in Production and any
>> challenges which they have experienced.
>>
>>
>>
>> Thanks,
>>
>> Siddharth Ubale,
>>
>> *Synchronized Communications *
>>
>> *#43, Velankani Tech Park, Block No. II, *
>>
>> *3rd Floor, Electronic City Phase I,*
>>
>> *Bangalore – 560 100*
>>
>> *Tel : +91 80 3202 4060 <%2B91%2080%203202%204060>*
>>
>> *Web:* *www.syncoms.com* <http://www.syncoms.com/>
>>
>> *[image: LogoNEWmohLARGE]*
>>
>> *London*|*Bangalore*|*Orlando*
>>
>>
>>
>> *we innovate, plan, execute, and transform the business​*
>>
>>
>>
>>
>

Re: Re: Fwd: Phoenix in production

Posted by Josh Mahonin <jm...@interset.com>.
On Wed, Jan 7, 2015 at 1:43 PM, anil gupta <an...@gmail.com> wrote:

> Yup, I am aware of Spark HBase integration. Phoenix-Spark integration
> would be more sweet. :)


Hi Anil,

I'm using Spark and Phoenix in production fairly successfully. There's very
little required for integration, since Phoenix has Hadoop Input and Output
formats that Spark can use natively.

As well, there is anther project which aims to bring the full Spark SQL
integration to Phoenix:
https://github.com/simplymeasured/phoenix-spark

Josh

Re: Re: Fwd: Phoenix in production

Posted by anil gupta <an...@gmail.com>.
Yup, I am aware of Spark HBase integration. Phoenix-Spark integration would
be more sweet. :)

On Wed, Jan 7, 2015 at 12:40 AM, sunfl@certusnet.com.cn <
sunfl@certusnet.com.cn> wrote:

> Hi Anil,
> Well, there are already good opensouce project on github for Spark on
> HBase, like the following:
> https://github.com/cloudera-labs/SparkOnHBase
>
> Phoenix integration shall be more convenient based on that. Considering to
> share our code for using
> that schema.
>
> Thanks,
> Sun.
>
> ------------------------------
> ------------------------------
>
> CertusNet
>
>
>
> *From:* anil gupta <an...@gmail.com>
> *Date:* 2015-01-07 16:28
> *To:* user@phoenix.apache.org
> *CC:* James Taylor <ja...@apache.org>; dev <de...@phoenix.apache.org>
> *Subject:* Re: Re: Fwd: Phoenix in production
> Hi Sun,
>
> Phoenix-Spark would be a nice addon if you can open source it. I am
> planning/thinking to using Spark on HBase for one of my project.
>
> ~Anil
>
> On Wed, Jan 7, 2015 at 12:17 AM, sunfl@certusnet.com.cn <
> sunfl@certusnet.com.cn> wrote:
>
>> Hi,
>> spark-phoenix integration would be great as Spark community is greately
>> active now and more
>> and more developers are using Apache Spark.
>>
>> Thanks,
>> Sun.
>>
>> ------------------------------
>> ------------------------------
>>
>>
>> *From:* James Taylor <ja...@apache.org>
>> *Date:* 2015-01-07 16:10
>> *To:* sunfl@certusnet.com.cn
>> *Subject:* Re: Fwd: Phoenix in production
>> This is great, Sun! Thank you so much. Would you mind posting this on our
>> user list in response to Siddharth's email? I think other Phoenix users
>> would find it interesting as well.
>>
>> On a side note, not sure how general what you developed is, but it would
>> be interesting to pursue a general Spark integration in Phoenix as an open
>> source contribution.
>>
>> Thanks,
>> James
>>
>> On Tue, Jan 6, 2015 at 5:41 PM, sunfl@certusnet.com.cn <
>> sunfl@certusnet.com.cn> wrote:
>>
>>> Hi, James & Siddharth
>>>
>>> Glad to share our experience of using Phoenix in Production. I believe
>>> that Siddharth had done
>>> sufficient tests and practices about Phoenix performance. Here are some
>>> tips about how we are using
>>> Phoenix for our projects:
>>> 1. We facilitate Phoenix to give convinience for both RD and QA
>>> engineers, as they are glad to use
>>> standard sql to operate hbase with no much loss of query performance.
>>> 2. In Production environment, we mainly integrate Apache Spark with
>>> Phoenix to optimize data loading to
>>> Phoenix tables with or withour secondary indexes. Glad that current
>>> performance of writing had worked
>>> smoothly with both compared to previously used MySQL InfoBright and
>>> other sql schema. We both had tested
>>> a lot for secondary indexes and query optimization for Phoenix before
>>> moving Phoenix to Production
>>> environment. Now we can get most of the features worked for Phoenix in
>>> our job.
>>> 3. Challenges had a lot too, such as bulkload performance with wal
>>> enabled, query optimization, statistical data collection with Phoenix
>>> full table scan, and so on. However, we believe Phoenix be a sufficient
>>> schema for sql query over HBase and we are glad
>>> that even more our projects are considering using Phoenix.
>>>
>>> Thanks,
>>> Sun.
>>>
>>> ------------------------------
>>> ------------------------------
>>>
>>> CertusNet
>>>
>>>
>>> *From:* James Taylor <ja...@apache.org>
>>> *Date:* 2015-01-07 09:10
>>> *To:* sunfl@certusnet.com.cn
>>> *Subject:* Fwd: Phoenix in production
>>> Hi Sun,
>>> Any experiences you can share with Siddharth?
>>> Thanks,
>>> James
>>>
>>> ---------- Forwarded message ----------
>>> From: Siddharth Ubale <si...@syncoms.com>
>>> Date: Thu, Jan 1, 2015 at 11:21 PM
>>> Subject: Phoenix in production
>>> To: "user@phoenix.apache.org" <us...@phoenix.apache.org>
>>>
>>>
>>>  Hi Guys,
>>>
>>>
>>>
>>> We are seriously thinking of phoenix in Production environment ,
>>> however, we have no much data of how Phoenix is behaving in production.
>>>
>>> Can anyone let us know if anyone is using Phoenix in Production and any
>>> challenges which they have experienced.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Siddharth Ubale,
>>>
>>> *Synchronized Communications *
>>>
>>> *#43, Velankani Tech Park, Block No. II, *
>>>
>>> *3rd Floor, Electronic City Phase I,*
>>>
>>> *Bangalore – 560 100*
>>>
>>> *Tel : +91 80 3202 4060 <%2B91%2080%203202%204060>*
>>>
>>> *Web:* *www.syncoms.com* <http://www.syncoms.com/>
>>>
>>> *[image: LogoNEWmohLARGE]*
>>>
>>> *London*|*Bangalore*|*Orlando*
>>>
>>>
>>>
>>> *we innovate, plan, execute, and transform the business​*
>>>
>>>
>>>
>>>
>>
>
>
> --
> Thanks & Regards,
> Anil Gupta
>
>


-- 
Thanks & Regards,
Anil Gupta

Re: Re: Fwd: Phoenix in production

Posted by anil gupta <an...@gmail.com>.
Yup, I am aware of Spark HBase integration. Phoenix-Spark integration would
be more sweet. :)

On Wed, Jan 7, 2015 at 12:40 AM, sunfl@certusnet.com.cn <
sunfl@certusnet.com.cn> wrote:

> Hi Anil,
> Well, there are already good opensouce project on github for Spark on
> HBase, like the following:
> https://github.com/cloudera-labs/SparkOnHBase
>
> Phoenix integration shall be more convenient based on that. Considering to
> share our code for using
> that schema.
>
> Thanks,
> Sun.
>
> ------------------------------
> ------------------------------
>
> CertusNet
>
>
>
> *From:* anil gupta <an...@gmail.com>
> *Date:* 2015-01-07 16:28
> *To:* user@phoenix.apache.org
> *CC:* James Taylor <ja...@apache.org>; dev <de...@phoenix.apache.org>
> *Subject:* Re: Re: Fwd: Phoenix in production
> Hi Sun,
>
> Phoenix-Spark would be a nice addon if you can open source it. I am
> planning/thinking to using Spark on HBase for one of my project.
>
> ~Anil
>
> On Wed, Jan 7, 2015 at 12:17 AM, sunfl@certusnet.com.cn <
> sunfl@certusnet.com.cn> wrote:
>
>> Hi,
>> spark-phoenix integration would be great as Spark community is greately
>> active now and more
>> and more developers are using Apache Spark.
>>
>> Thanks,
>> Sun.
>>
>> ------------------------------
>> ------------------------------
>>
>>
>> *From:* James Taylor <ja...@apache.org>
>> *Date:* 2015-01-07 16:10
>> *To:* sunfl@certusnet.com.cn
>> *Subject:* Re: Fwd: Phoenix in production
>> This is great, Sun! Thank you so much. Would you mind posting this on our
>> user list in response to Siddharth's email? I think other Phoenix users
>> would find it interesting as well.
>>
>> On a side note, not sure how general what you developed is, but it would
>> be interesting to pursue a general Spark integration in Phoenix as an open
>> source contribution.
>>
>> Thanks,
>> James
>>
>> On Tue, Jan 6, 2015 at 5:41 PM, sunfl@certusnet.com.cn <
>> sunfl@certusnet.com.cn> wrote:
>>
>>> Hi, James & Siddharth
>>>
>>> Glad to share our experience of using Phoenix in Production. I believe
>>> that Siddharth had done
>>> sufficient tests and practices about Phoenix performance. Here are some
>>> tips about how we are using
>>> Phoenix for our projects:
>>> 1. We facilitate Phoenix to give convinience for both RD and QA
>>> engineers, as they are glad to use
>>> standard sql to operate hbase with no much loss of query performance.
>>> 2. In Production environment, we mainly integrate Apache Spark with
>>> Phoenix to optimize data loading to
>>> Phoenix tables with or withour secondary indexes. Glad that current
>>> performance of writing had worked
>>> smoothly with both compared to previously used MySQL InfoBright and
>>> other sql schema. We both had tested
>>> a lot for secondary indexes and query optimization for Phoenix before
>>> moving Phoenix to Production
>>> environment. Now we can get most of the features worked for Phoenix in
>>> our job.
>>> 3. Challenges had a lot too, such as bulkload performance with wal
>>> enabled, query optimization, statistical data collection with Phoenix
>>> full table scan, and so on. However, we believe Phoenix be a sufficient
>>> schema for sql query over HBase and we are glad
>>> that even more our projects are considering using Phoenix.
>>>
>>> Thanks,
>>> Sun.
>>>
>>> ------------------------------
>>> ------------------------------
>>>
>>> CertusNet
>>>
>>>
>>> *From:* James Taylor <ja...@apache.org>
>>> *Date:* 2015-01-07 09:10
>>> *To:* sunfl@certusnet.com.cn
>>> *Subject:* Fwd: Phoenix in production
>>> Hi Sun,
>>> Any experiences you can share with Siddharth?
>>> Thanks,
>>> James
>>>
>>> ---------- Forwarded message ----------
>>> From: Siddharth Ubale <si...@syncoms.com>
>>> Date: Thu, Jan 1, 2015 at 11:21 PM
>>> Subject: Phoenix in production
>>> To: "user@phoenix.apache.org" <us...@phoenix.apache.org>
>>>
>>>
>>>  Hi Guys,
>>>
>>>
>>>
>>> We are seriously thinking of phoenix in Production environment ,
>>> however, we have no much data of how Phoenix is behaving in production.
>>>
>>> Can anyone let us know if anyone is using Phoenix in Production and any
>>> challenges which they have experienced.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Siddharth Ubale,
>>>
>>> *Synchronized Communications *
>>>
>>> *#43, Velankani Tech Park, Block No. II, *
>>>
>>> *3rd Floor, Electronic City Phase I,*
>>>
>>> *Bangalore – 560 100*
>>>
>>> *Tel : +91 80 3202 4060 <%2B91%2080%203202%204060>*
>>>
>>> *Web:* *www.syncoms.com* <http://www.syncoms.com/>
>>>
>>> *[image: LogoNEWmohLARGE]*
>>>
>>> *London*|*Bangalore*|*Orlando*
>>>
>>>
>>>
>>> *we innovate, plan, execute, and transform the business​*
>>>
>>>
>>>
>>>
>>
>
>
> --
> Thanks & Regards,
> Anil Gupta
>
>


-- 
Thanks & Regards,
Anil Gupta

Re: Re: Fwd: Phoenix in production

Posted by "sunfl@certusnet.com.cn" <su...@certusnet.com.cn>.
Hi Anil,
Well, there are already good opensouce project on github for Spark on HBase, like the following:
https://github.com/cloudera-labs/SparkOnHBase 

Phoenix integration shall be more convenient based on that. Considering to share our code for using
that schema.

Thanks,
Sun.





CertusNet 

 
From: anil gupta
Date: 2015-01-07 16:28
To: user@phoenix.apache.org
CC: James Taylor; dev
Subject: Re: Re: Fwd: Phoenix in production
Hi Sun,

Phoenix-Spark would be a nice addon if you can open source it. I am planning/thinking to using Spark on HBase for one of my project.

~Anil

On Wed, Jan 7, 2015 at 12:17 AM, sunfl@certusnet.com.cn <su...@certusnet.com.cn> wrote:
Hi,
spark-phoenix integration would be great as Spark community is greately active now and more 
and more developers are using Apache Spark.

Thanks,
Sun.






From: James Taylor
Date: 2015-01-07 16:10
To: sunfl@certusnet.com.cn
Subject: Re: Fwd: Phoenix in production
This is great, Sun! Thank you so much. Would you mind posting this on our user list in response to Siddharth's email? I think other Phoenix users would find it interesting as well.

On a side note, not sure how general what you developed is, but it would be interesting to pursue a general Spark integration in Phoenix as an open source contribution.

Thanks,
James

On Tue, Jan 6, 2015 at 5:41 PM, sunfl@certusnet.com.cn <su...@certusnet.com.cn> wrote:
Hi, James & Siddharth

Glad to share our experience of using Phoenix in Production. I believe that Siddharth had done 
sufficient tests and practices about Phoenix performance. Here are some tips about how we are using
Phoenix for our projects:
1. We facilitate Phoenix to give convinience for both RD and QA engineers, as they are glad to use 
standard sql to operate hbase with no much loss of query performance. 
2. In Production environment, we mainly integrate Apache Spark with Phoenix to optimize data loading to
Phoenix tables with or withour secondary indexes. Glad that current performance of writing had worked 
smoothly with both compared to previously used MySQL InfoBright and other sql schema. We both had tested 
a lot for secondary indexes and query optimization for Phoenix before moving Phoenix to Production 
environment. Now we can get most of the features worked for Phoenix in our job. 
3. Challenges had a lot too, such as bulkload performance with wal enabled, query optimization, statistical data collection with Phoenix
full table scan, and so on. However, we believe Phoenix be a sufficient schema for sql query over HBase and we are glad 
that even more our projects are considering using Phoenix.

Thanks,
Sun.





CertusNet 

From: James Taylor
Date: 2015-01-07 09:10
To: sunfl@certusnet.com.cn
Subject: Fwd: Phoenix in production
Hi Sun,
Any experiences you can share with Siddharth?
Thanks,
James

---------- Forwarded message ----------
From: Siddharth Ubale <si...@syncoms.com>
Date: Thu, Jan 1, 2015 at 11:21 PM
Subject: Phoenix in production
To: "user@phoenix.apache.org" <us...@phoenix.apache.org>


Hi Guys,
 
We are seriously thinking of phoenix in Production environment , however, we have no much data of how Phoenix is behaving in production.
Can anyone let us know if anyone is using Phoenix in Production and any challenges which they have experienced.
 
Thanks,
Siddharth Ubale,
Synchronized Communications 
#43, Velankani Tech Park, Block No. II, 
3rd Floor, Electronic City Phase I,
Bangalore – 560 100
Tel : +91 80 3202 4060
Web: www.syncoms.com
London|Bangalore|Orlando
 
we innovate, plan, execute, and transform the business​
 





-- 
Thanks & Regards,
Anil Gupta

Re: Re: Fwd: Phoenix in production

Posted by "sunfl@certusnet.com.cn" <su...@certusnet.com.cn>.
Hi Anil,
Well, there are already good opensouce project on github for Spark on HBase, like the following:
https://github.com/cloudera-labs/SparkOnHBase 

Phoenix integration shall be more convenient based on that. Considering to share our code for using
that schema.

Thanks,
Sun.





CertusNet 

 
From: anil gupta
Date: 2015-01-07 16:28
To: user@phoenix.apache.org
CC: James Taylor; dev
Subject: Re: Re: Fwd: Phoenix in production
Hi Sun,

Phoenix-Spark would be a nice addon if you can open source it. I am planning/thinking to using Spark on HBase for one of my project.

~Anil

On Wed, Jan 7, 2015 at 12:17 AM, sunfl@certusnet.com.cn <su...@certusnet.com.cn> wrote:
Hi,
spark-phoenix integration would be great as Spark community is greately active now and more 
and more developers are using Apache Spark.

Thanks,
Sun.






From: James Taylor
Date: 2015-01-07 16:10
To: sunfl@certusnet.com.cn
Subject: Re: Fwd: Phoenix in production
This is great, Sun! Thank you so much. Would you mind posting this on our user list in response to Siddharth's email? I think other Phoenix users would find it interesting as well.

On a side note, not sure how general what you developed is, but it would be interesting to pursue a general Spark integration in Phoenix as an open source contribution.

Thanks,
James

On Tue, Jan 6, 2015 at 5:41 PM, sunfl@certusnet.com.cn <su...@certusnet.com.cn> wrote:
Hi, James & Siddharth

Glad to share our experience of using Phoenix in Production. I believe that Siddharth had done 
sufficient tests and practices about Phoenix performance. Here are some tips about how we are using
Phoenix for our projects:
1. We facilitate Phoenix to give convinience for both RD and QA engineers, as they are glad to use 
standard sql to operate hbase with no much loss of query performance. 
2. In Production environment, we mainly integrate Apache Spark with Phoenix to optimize data loading to
Phoenix tables with or withour secondary indexes. Glad that current performance of writing had worked 
smoothly with both compared to previously used MySQL InfoBright and other sql schema. We both had tested 
a lot for secondary indexes and query optimization for Phoenix before moving Phoenix to Production 
environment. Now we can get most of the features worked for Phoenix in our job. 
3. Challenges had a lot too, such as bulkload performance with wal enabled, query optimization, statistical data collection with Phoenix
full table scan, and so on. However, we believe Phoenix be a sufficient schema for sql query over HBase and we are glad 
that even more our projects are considering using Phoenix.

Thanks,
Sun.





CertusNet 

From: James Taylor
Date: 2015-01-07 09:10
To: sunfl@certusnet.com.cn
Subject: Fwd: Phoenix in production
Hi Sun,
Any experiences you can share with Siddharth?
Thanks,
James

---------- Forwarded message ----------
From: Siddharth Ubale <si...@syncoms.com>
Date: Thu, Jan 1, 2015 at 11:21 PM
Subject: Phoenix in production
To: "user@phoenix.apache.org" <us...@phoenix.apache.org>


Hi Guys,
 
We are seriously thinking of phoenix in Production environment , however, we have no much data of how Phoenix is behaving in production.
Can anyone let us know if anyone is using Phoenix in Production and any challenges which they have experienced.
 
Thanks,
Siddharth Ubale,
Synchronized Communications 
#43, Velankani Tech Park, Block No. II, 
3rd Floor, Electronic City Phase I,
Bangalore – 560 100
Tel : +91 80 3202 4060
Web: www.syncoms.com
London|Bangalore|Orlando
 
we innovate, plan, execute, and transform the business​
 





-- 
Thanks & Regards,
Anil Gupta

Re: Re: Fwd: Phoenix in production

Posted by anil gupta <an...@gmail.com>.
Hi Sun,

Phoenix-Spark would be a nice addon if you can open source it. I am
planning/thinking to using Spark on HBase for one of my project.

~Anil

On Wed, Jan 7, 2015 at 12:17 AM, sunfl@certusnet.com.cn <
sunfl@certusnet.com.cn> wrote:

> Hi,
> spark-phoenix integration would be great as Spark community is greately
> active now and more
> and more developers are using Apache Spark.
>
> Thanks,
> Sun.
>
> ------------------------------
> ------------------------------
>
>
> *From:* James Taylor <ja...@apache.org>
> *Date:* 2015-01-07 16:10
> *To:* sunfl@certusnet.com.cn
> *Subject:* Re: Fwd: Phoenix in production
> This is great, Sun! Thank you so much. Would you mind posting this on our
> user list in response to Siddharth's email? I think other Phoenix users
> would find it interesting as well.
>
> On a side note, not sure how general what you developed is, but it would
> be interesting to pursue a general Spark integration in Phoenix as an open
> source contribution.
>
> Thanks,
> James
>
> On Tue, Jan 6, 2015 at 5:41 PM, sunfl@certusnet.com.cn <
> sunfl@certusnet.com.cn> wrote:
>
>> Hi, James & Siddharth
>>
>> Glad to share our experience of using Phoenix in Production. I believe
>> that Siddharth had done
>> sufficient tests and practices about Phoenix performance. Here are some
>> tips about how we are using
>> Phoenix for our projects:
>> 1. We facilitate Phoenix to give convinience for both RD and QA
>> engineers, as they are glad to use
>> standard sql to operate hbase with no much loss of query performance.
>> 2. In Production environment, we mainly integrate Apache Spark with
>> Phoenix to optimize data loading to
>> Phoenix tables with or withour secondary indexes. Glad that current
>> performance of writing had worked
>> smoothly with both compared to previously used MySQL InfoBright and other
>> sql schema. We both had tested
>> a lot for secondary indexes and query optimization for Phoenix before
>> moving Phoenix to Production
>> environment. Now we can get most of the features worked for Phoenix in
>> our job.
>> 3. Challenges had a lot too, such as bulkload performance with wal
>> enabled, query optimization, statistical data collection with Phoenix
>> full table scan, and so on. However, we believe Phoenix be a sufficient
>> schema for sql query over HBase and we are glad
>> that even more our projects are considering using Phoenix.
>>
>> Thanks,
>> Sun.
>>
>> ------------------------------
>> ------------------------------
>>
>> CertusNet
>>
>>
>> *From:* James Taylor <ja...@apache.org>
>> *Date:* 2015-01-07 09:10
>> *To:* sunfl@certusnet.com.cn
>> *Subject:* Fwd: Phoenix in production
>> Hi Sun,
>> Any experiences you can share with Siddharth?
>> Thanks,
>> James
>>
>> ---------- Forwarded message ----------
>> From: Siddharth Ubale <si...@syncoms.com>
>> Date: Thu, Jan 1, 2015 at 11:21 PM
>> Subject: Phoenix in production
>> To: "user@phoenix.apache.org" <us...@phoenix.apache.org>
>>
>>
>>  Hi Guys,
>>
>>
>>
>> We are seriously thinking of phoenix in Production environment , however,
>> we have no much data of how Phoenix is behaving in production.
>>
>> Can anyone let us know if anyone is using Phoenix in Production and any
>> challenges which they have experienced.
>>
>>
>>
>> Thanks,
>>
>> Siddharth Ubale,
>>
>> *Synchronized Communications *
>>
>> *#43, Velankani Tech Park, Block No. II, *
>>
>> *3rd Floor, Electronic City Phase I,*
>>
>> *Bangalore – 560 100*
>>
>> *Tel : +91 80 3202 4060 <%2B91%2080%203202%204060>*
>>
>> *Web:* *www.syncoms.com* <http://www.syncoms.com/>
>>
>> *[image: LogoNEWmohLARGE]*
>>
>> *London*|*Bangalore*|*Orlando*
>>
>>
>>
>> *we innovate, plan, execute, and transform the business​*
>>
>>
>>
>>
>


-- 
Thanks & Regards,
Anil Gupta

Re: Re: Fwd: Phoenix in production

Posted by anil gupta <an...@gmail.com>.
Hi Sun,

Phoenix-Spark would be a nice addon if you can open source it. I am
planning/thinking to using Spark on HBase for one of my project.

~Anil

On Wed, Jan 7, 2015 at 12:17 AM, sunfl@certusnet.com.cn <
sunfl@certusnet.com.cn> wrote:

> Hi,
> spark-phoenix integration would be great as Spark community is greately
> active now and more
> and more developers are using Apache Spark.
>
> Thanks,
> Sun.
>
> ------------------------------
> ------------------------------
>
>
> *From:* James Taylor <ja...@apache.org>
> *Date:* 2015-01-07 16:10
> *To:* sunfl@certusnet.com.cn
> *Subject:* Re: Fwd: Phoenix in production
> This is great, Sun! Thank you so much. Would you mind posting this on our
> user list in response to Siddharth's email? I think other Phoenix users
> would find it interesting as well.
>
> On a side note, not sure how general what you developed is, but it would
> be interesting to pursue a general Spark integration in Phoenix as an open
> source contribution.
>
> Thanks,
> James
>
> On Tue, Jan 6, 2015 at 5:41 PM, sunfl@certusnet.com.cn <
> sunfl@certusnet.com.cn> wrote:
>
>> Hi, James & Siddharth
>>
>> Glad to share our experience of using Phoenix in Production. I believe
>> that Siddharth had done
>> sufficient tests and practices about Phoenix performance. Here are some
>> tips about how we are using
>> Phoenix for our projects:
>> 1. We facilitate Phoenix to give convinience for both RD and QA
>> engineers, as they are glad to use
>> standard sql to operate hbase with no much loss of query performance.
>> 2. In Production environment, we mainly integrate Apache Spark with
>> Phoenix to optimize data loading to
>> Phoenix tables with or withour secondary indexes. Glad that current
>> performance of writing had worked
>> smoothly with both compared to previously used MySQL InfoBright and other
>> sql schema. We both had tested
>> a lot for secondary indexes and query optimization for Phoenix before
>> moving Phoenix to Production
>> environment. Now we can get most of the features worked for Phoenix in
>> our job.
>> 3. Challenges had a lot too, such as bulkload performance with wal
>> enabled, query optimization, statistical data collection with Phoenix
>> full table scan, and so on. However, we believe Phoenix be a sufficient
>> schema for sql query over HBase and we are glad
>> that even more our projects are considering using Phoenix.
>>
>> Thanks,
>> Sun.
>>
>> ------------------------------
>> ------------------------------
>>
>> CertusNet
>>
>>
>> *From:* James Taylor <ja...@apache.org>
>> *Date:* 2015-01-07 09:10
>> *To:* sunfl@certusnet.com.cn
>> *Subject:* Fwd: Phoenix in production
>> Hi Sun,
>> Any experiences you can share with Siddharth?
>> Thanks,
>> James
>>
>> ---------- Forwarded message ----------
>> From: Siddharth Ubale <si...@syncoms.com>
>> Date: Thu, Jan 1, 2015 at 11:21 PM
>> Subject: Phoenix in production
>> To: "user@phoenix.apache.org" <us...@phoenix.apache.org>
>>
>>
>>  Hi Guys,
>>
>>
>>
>> We are seriously thinking of phoenix in Production environment , however,
>> we have no much data of how Phoenix is behaving in production.
>>
>> Can anyone let us know if anyone is using Phoenix in Production and any
>> challenges which they have experienced.
>>
>>
>>
>> Thanks,
>>
>> Siddharth Ubale,
>>
>> *Synchronized Communications *
>>
>> *#43, Velankani Tech Park, Block No. II, *
>>
>> *3rd Floor, Electronic City Phase I,*
>>
>> *Bangalore – 560 100*
>>
>> *Tel : +91 80 3202 4060 <%2B91%2080%203202%204060>*
>>
>> *Web:* *www.syncoms.com* <http://www.syncoms.com/>
>>
>> *[image: LogoNEWmohLARGE]*
>>
>> *London*|*Bangalore*|*Orlando*
>>
>>
>>
>> *we innovate, plan, execute, and transform the business​*
>>
>>
>>
>>
>


-- 
Thanks & Regards,
Anil Gupta

Re: Re: Fwd: Phoenix in production

Posted by "sunfl@certusnet.com.cn" <su...@certusnet.com.cn>.
Hi,
spark-phoenix integration would be great as Spark community is greately active now and more 
and more developers are using Apache Spark.

Thanks,
Sun.






From: James Taylor
Date: 2015-01-07 16:10
To: sunfl@certusnet.com.cn
Subject: Re: Fwd: Phoenix in production
This is great, Sun! Thank you so much. Would you mind posting this on our user list in response to Siddharth's email? I think other Phoenix users would find it interesting as well.

On a side note, not sure how general what you developed is, but it would be interesting to pursue a general Spark integration in Phoenix as an open source contribution.

Thanks,
James

On Tue, Jan 6, 2015 at 5:41 PM, sunfl@certusnet.com.cn <su...@certusnet.com.cn> wrote:
Hi, James & Siddharth

Glad to share our experience of using Phoenix in Production. I believe that Siddharth had done 
sufficient tests and practices about Phoenix performance. Here are some tips about how we are using
Phoenix for our projects:
1. We facilitate Phoenix to give convinience for both RD and QA engineers, as they are glad to use 
standard sql to operate hbase with no much loss of query performance. 
2. In Production environment, we mainly integrate Apache Spark with Phoenix to optimize data loading to
Phoenix tables with or withour secondary indexes. Glad that current performance of writing had worked 
smoothly with both compared to previously used MySQL InfoBright and other sql schema. We both had tested 
a lot for secondary indexes and query optimization for Phoenix before moving Phoenix to Production 
environment. Now we can get most of the features worked for Phoenix in our job. 
3. Challenges had a lot too, such as bulkload performance with wal enabled, query optimization, statistical data collection with Phoenix
full table scan, and so on. However, we believe Phoenix be a sufficient schema for sql query over HBase and we are glad 
that even more our projects are considering using Phoenix.

Thanks,
Sun.





CertusNet 

From: James Taylor
Date: 2015-01-07 09:10
To: sunfl@certusnet.com.cn
Subject: Fwd: Phoenix in production
Hi Sun,
Any experiences you can share with Siddharth?
Thanks,
James

---------- Forwarded message ----------
From: Siddharth Ubale <si...@syncoms.com>
Date: Thu, Jan 1, 2015 at 11:21 PM
Subject: Phoenix in production
To: "user@phoenix.apache.org" <us...@phoenix.apache.org>


Hi Guys,
 
We are seriously thinking of phoenix in Production environment , however, we have no much data of how Phoenix is behaving in production.
Can anyone let us know if anyone is using Phoenix in Production and any challenges which they have experienced.
 
Thanks,
Siddharth Ubale,
Synchronized Communications 
#43, Velankani Tech Park, Block No. II, 
3rd Floor, Electronic City Phase I,
Bangalore – 560 100
Tel : +91 80 3202 4060
Web: www.syncoms.com
London|Bangalore|Orlando
 
we innovate, plan, execute, and transform the business​
 



Re: Re: Fwd: Phoenix in production

Posted by "sunfl@certusnet.com.cn" <su...@certusnet.com.cn>.
Hi,
spark-phoenix integration would be great as Spark community is greately active now and more 
and more developers are using Apache Spark.

Thanks,
Sun.






From: James Taylor
Date: 2015-01-07 16:10
To: sunfl@certusnet.com.cn
Subject: Re: Fwd: Phoenix in production
This is great, Sun! Thank you so much. Would you mind posting this on our user list in response to Siddharth's email? I think other Phoenix users would find it interesting as well.

On a side note, not sure how general what you developed is, but it would be interesting to pursue a general Spark integration in Phoenix as an open source contribution.

Thanks,
James

On Tue, Jan 6, 2015 at 5:41 PM, sunfl@certusnet.com.cn <su...@certusnet.com.cn> wrote:
Hi, James & Siddharth

Glad to share our experience of using Phoenix in Production. I believe that Siddharth had done 
sufficient tests and practices about Phoenix performance. Here are some tips about how we are using
Phoenix for our projects:
1. We facilitate Phoenix to give convinience for both RD and QA engineers, as they are glad to use 
standard sql to operate hbase with no much loss of query performance. 
2. In Production environment, we mainly integrate Apache Spark with Phoenix to optimize data loading to
Phoenix tables with or withour secondary indexes. Glad that current performance of writing had worked 
smoothly with both compared to previously used MySQL InfoBright and other sql schema. We both had tested 
a lot for secondary indexes and query optimization for Phoenix before moving Phoenix to Production 
environment. Now we can get most of the features worked for Phoenix in our job. 
3. Challenges had a lot too, such as bulkload performance with wal enabled, query optimization, statistical data collection with Phoenix
full table scan, and so on. However, we believe Phoenix be a sufficient schema for sql query over HBase and we are glad 
that even more our projects are considering using Phoenix.

Thanks,
Sun.





CertusNet 

From: James Taylor
Date: 2015-01-07 09:10
To: sunfl@certusnet.com.cn
Subject: Fwd: Phoenix in production
Hi Sun,
Any experiences you can share with Siddharth?
Thanks,
James

---------- Forwarded message ----------
From: Siddharth Ubale <si...@syncoms.com>
Date: Thu, Jan 1, 2015 at 11:21 PM
Subject: Phoenix in production
To: "user@phoenix.apache.org" <us...@phoenix.apache.org>


Hi Guys,
 
We are seriously thinking of phoenix in Production environment , however, we have no much data of how Phoenix is behaving in production.
Can anyone let us know if anyone is using Phoenix in Production and any challenges which they have experienced.
 
Thanks,
Siddharth Ubale,
Synchronized Communications 
#43, Velankani Tech Park, Block No. II, 
3rd Floor, Electronic City Phase I,
Bangalore – 560 100
Tel : +91 80 3202 4060
Web: www.syncoms.com
London|Bangalore|Orlando
 
we innovate, plan, execute, and transform the business​