You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by "Gupta, Atul" <at...@lowes.com> on 2021/08/22 10:34:28 UTC
HBASE Queries & Slack Channel Support
Hi HBASE PMC Members.
Greeting!
We are one of the active users of HBASE and Phoenix. There are number of HBASE RT/Batch use cases are running in Lowes. I’m owning platform team, so my team is responsible for maintaining and support the HBASE business use cases.
In last few months we are facing number of challenges due to read/write latencies. I have reached out directly to PMC members, and everyone has suggested me to write to user mail group.
We are writing data into HBASE using Phoenix which is I guess not a recommended method if someone is looking for high throughput. Do we have any document regarding this? Also, what are Phoenix limitations if any?
Do we have SLACK channel which I can join to clarify my some of doubts quickly?
Looking for your quick response on this.
Thanks,
Atul Gupta
Sr. Director, Data Engineering
Lowes
________________________________
NOTICE: All information in and attached to the e-mails below may be proprietary, confidential, privileged and otherwise protected from improper or erroneous disclosure. If you are not the sender's intended recipient, you are not authorized to intercept, read, print, retain, copy, forward, or disseminate this message. If you have erroneously received this communication, please notify the sender immediately by phone (704-758-1000) or by e-mail and destroy all copies of this message electronic, paper, or otherwise. By transmitting documents via this email: Users, Customers, Suppliers and Vendors collectively acknowledge and agree the transmittal of information via email is voluntary, is offered as a convenience, and is not a secured method of communication; Not to transmit any payment information E.G. credit card, debit card, checking account, wire transfer information, passwords, or sensitive and personal information E.G. Driver's license, DOB, social security, or any other information the user wishes to remain confidential; To transmit only non-confidential information such as plans, pictures and drawings and to assume all risk and liability for and indemnify Lowe's from any claims, losses or damages that may arise from the transmittal of documents or including non-confidential information in the body of an email transmittal. Thank you.
Re: HBASE Queries & Slack Channel Support
Posted by Jignesh Patel <ji...@gmail.com>.
This is an interesting discussion about latency vs throughput.
We are using an older version of apache hadoop 2.6 and HBase and recently
we faced latency issues. The way we solved temporarily is avoiding joins
wherever possible and bringing lookup data in cache(instead of reading from
HBase). For the secondary indexing purpose we are using Solr. Probably it
solved the problem temporarily but I know eventually we will have the same
problem of handling latency vs throughput.
So happy to get this discussion going.
On Mon, Aug 23, 2021 at 4:39 PM Josh Elser <el...@apache.org> wrote:
> +1 for following up in Phoenix for Phoenix-specific question, but I
> thought it was worth mentioning that there's no reason that you can't do
> "high throughput" access to HBase via Phoenix. Phoenix has parity for
> most high-throughput approaches that you would have access to in HBase.
>
> There is no one answer to which method you should use, because the
> reality is "it depends". To set clear expectations, high latencies are
> often the tradeoff you have to make for high throughput (latency and
> throughput are often inversely proportional).
>
> Usually, the first round of performance issues boil down to data
> modeling. It's a good thought exercise for you to think through what
> you're requirements are and what the "average" latency for HBase is on
> your hardware (decoupled from your real-life data), and then compare
> that to your actual workload. This helps frame your current performance
> against a "potential" performance.
>
> On 8/23/21 2:29 PM, Daniel Wong wrote:
> > Hi Atul questions about Phoenix support is better done through the
> apache phoenix mailing lists. Depending on you use patterns Phoenix may or
> may not perform better than only Hbase. I'm happy to invite you to the
> apache phoenix slack as well if you reach out to me though you are more
> likely to hit a wider audience in user@phoenix.
> >
> > Daniel Wong
> >
> > On 2021/08/22 10:34:28, "Gupta, Atul" <at...@lowes.com> wrote:
> >> Hi HBASE PMC Members.
> >>
> >> Greeting!
> >>
> >> We are one of the active users of HBASE and Phoenix. There are number
> of HBASE RT/Batch use cases are running in Lowes. I’m owning platform team,
> so my team is responsible for maintaining and support the HBASE business
> use cases.
> >> In last few months we are facing number of challenges due to read/write
> latencies. I have reached out directly to PMC members, and everyone has
> suggested me to write to user mail group.
> >>
> >> We are writing data into HBASE using Phoenix which is I guess not a
> recommended method if someone is looking for high throughput. Do we have
> any document regarding this? Also, what are Phoenix limitations if any?
> >>
> >> Do we have SLACK channel which I can join to clarify my some of doubts
> quickly?
> >>
> >> Looking for your quick response on this.
> >>
> >> Thanks,
> >> Atul Gupta
> >> Sr. Director, Data Engineering
> >> Lowes
> >> ________________________________
> >> NOTICE: All information in and attached to the e-mails below may be
> proprietary, confidential, privileged and otherwise protected from improper
> or erroneous disclosure. If you are not the sender's intended recipient,
> you are not authorized to intercept, read, print, retain, copy, forward, or
> disseminate this message. If you have erroneously received this
> communication, please notify the sender immediately by phone (704-758-1000)
> or by e-mail and destroy all copies of this message electronic, paper, or
> otherwise. By transmitting documents via this email: Users, Customers,
> Suppliers and Vendors collectively acknowledge and agree the transmittal of
> information via email is voluntary, is offered as a convenience, and is not
> a secured method of communication; Not to transmit any payment information
> E.G. credit card, debit card, checking account, wire transfer information,
> passwords, or sensitive and personal information E.G. Driver's license,
> DOB, social security, or any other informati
> > on the user wishes to remain confidential; To transmit only
> non-confidential information such as plans, pictures and drawings and to
> assume all risk and liability for and indemnify Lowe's from any claims,
> losses or damages that may arise from the transmittal of documents or
> including non-confidential information in the body of an email transmittal.
> Thank you.
> >>
>
Re: HBASE Queries & Slack Channel Support
Posted by Josh Elser <el...@apache.org>.
+1 for following up in Phoenix for Phoenix-specific question, but I
thought it was worth mentioning that there's no reason that you can't do
"high throughput" access to HBase via Phoenix. Phoenix has parity for
most high-throughput approaches that you would have access to in HBase.
There is no one answer to which method you should use, because the
reality is "it depends". To set clear expectations, high latencies are
often the tradeoff you have to make for high throughput (latency and
throughput are often inversely proportional).
Usually, the first round of performance issues boil down to data
modeling. It's a good thought exercise for you to think through what
you're requirements are and what the "average" latency for HBase is on
your hardware (decoupled from your real-life data), and then compare
that to your actual workload. This helps frame your current performance
against a "potential" performance.
On 8/23/21 2:29 PM, Daniel Wong wrote:
> Hi Atul questions about Phoenix support is better done through the apache phoenix mailing lists. Depending on you use patterns Phoenix may or may not perform better than only Hbase. I'm happy to invite you to the apache phoenix slack as well if you reach out to me though you are more likely to hit a wider audience in user@phoenix.
>
> Daniel Wong
>
> On 2021/08/22 10:34:28, "Gupta, Atul" <at...@lowes.com> wrote:
>> Hi HBASE PMC Members.
>>
>> Greeting!
>>
>> We are one of the active users of HBASE and Phoenix. There are number of HBASE RT/Batch use cases are running in Lowes. I’m owning platform team, so my team is responsible for maintaining and support the HBASE business use cases.
>> In last few months we are facing number of challenges due to read/write latencies. I have reached out directly to PMC members, and everyone has suggested me to write to user mail group.
>>
>> We are writing data into HBASE using Phoenix which is I guess not a recommended method if someone is looking for high throughput. Do we have any document regarding this? Also, what are Phoenix limitations if any?
>>
>> Do we have SLACK channel which I can join to clarify my some of doubts quickly?
>>
>> Looking for your quick response on this.
>>
>> Thanks,
>> Atul Gupta
>> Sr. Director, Data Engineering
>> Lowes
>> ________________________________
>> NOTICE: All information in and attached to the e-mails below may be proprietary, confidential, privileged and otherwise protected from improper or erroneous disclosure. If you are not the sender's intended recipient, you are not authorized to intercept, read, print, retain, copy, forward, or disseminate this message. If you have erroneously received this communication, please notify the sender immediately by phone (704-758-1000) or by e-mail and destroy all copies of this message electronic, paper, or otherwise. By transmitting documents via this email: Users, Customers, Suppliers and Vendors collectively acknowledge and agree the transmittal of information via email is voluntary, is offered as a convenience, and is not a secured method of communication; Not to transmit any payment information E.G. credit card, debit card, checking account, wire transfer information, passwords, or sensitive and personal information E.G. Driver's license, DOB, social security, or any other informati
> on the user wishes to remain confidential; To transmit only non-confidential information such as plans, pictures and drawings and to assume all risk and liability for and indemnify Lowe's from any claims, losses or damages that may arise from the transmittal of documents or including non-confidential information in the body of an email transmittal. Thank you.
>>
Re: HBASE Queries & Slack Channel Support
Posted by Daniel Wong <db...@apache.org>.
Hi Atul questions about Phoenix support is better done through the apache phoenix mailing lists. Depending on you use patterns Phoenix may or may not perform better than only Hbase. I'm happy to invite you to the apache phoenix slack as well if you reach out to me though you are more likely to hit a wider audience in user@phoenix.
Daniel Wong
On 2021/08/22 10:34:28, "Gupta, Atul" <at...@lowes.com> wrote:
> Hi HBASE PMC Members.
>
> Greeting!
>
> We are one of the active users of HBASE and Phoenix. There are number of HBASE RT/Batch use cases are running in Lowes. I’m owning platform team, so my team is responsible for maintaining and support the HBASE business use cases.
> In last few months we are facing number of challenges due to read/write latencies. I have reached out directly to PMC members, and everyone has suggested me to write to user mail group.
>
> We are writing data into HBASE using Phoenix which is I guess not a recommended method if someone is looking for high throughput. Do we have any document regarding this? Also, what are Phoenix limitations if any?
>
> Do we have SLACK channel which I can join to clarify my some of doubts quickly?
>
> Looking for your quick response on this.
>
> Thanks,
> Atul Gupta
> Sr. Director, Data Engineering
> Lowes
> ________________________________
> NOTICE: All information in and attached to the e-mails below may be proprietary, confidential, privileged and otherwise protected from improper or erroneous disclosure. If you are not the sender's intended recipient, you are not authorized to intercept, read, print, retain, copy, forward, or disseminate this message. If you have erroneously received this communication, please notify the sender immediately by phone (704-758-1000) or by e-mail and destroy all copies of this message electronic, paper, or otherwise. By transmitting documents via this email: Users, Customers, Suppliers and Vendors collectively acknowledge and agree the transmittal of information via email is voluntary, is offered as a convenience, and is not a secured method of communication; Not to transmit any payment information E.G. credit card, debit card, checking account, wire transfer information, passwords, or sensitive and personal information E.G. Driver's license, DOB, social security, or any other informati
on the user wishes to remain confidential; To transmit only non-confidential information such as plans, pictures and drawings and to assume all risk and liability for and indemnify Lowe's from any claims, losses or damages that may arise from the transmittal of documents or including non-confidential information in the body of an email transmittal. Thank you.
>