You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Andrzej <bo...@wp.pl> on 2017/09/28 12:58:06 UTC
How avoid error when variant column not exists ?
I get:
return result->Value(family, qualifier).value()
how can I check if (family,qulifier) exists? or is possible return empty
string instead error?
Re: If possible read families from tables and (more important) qualifiers?
Posted by Stack <st...@duboce.net>.
On Thu, Oct 5, 2017 at 8:57 AM, Ted Yu <yu...@gmail.com> wrote:
> See Enis' comment:
>
> https://issues.apache.org/jira/browse/HBASE-14850?
> focusedCommentId=15840799&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-15840799
>
> though it was half year old.
>
>
That is not a status by my reading. That is an overview of how the client
is implemented.
A state of C++ cliient would be appreciated; what is to done, what works,
when we think it will be done.
Thanks,
S
> On Thu, Oct 5, 2017 at 8:53 AM, Sean Busbey <bu...@apache.org> wrote:
>
> > I believe there is an umbrella jira tracking the progress of the
> > feature on its branch. I haven't kept up with development enough to know
> if
> > it's accurate and/or if things have reached the stage of explicitly
> > documenting things.
> >
> > @Ted Y might be able to provide a pointer?
> >
> > On Thu, Oct 5, 2017 at 2:03 AM, Andrzej <bo...@wp.pl> wrote:
> > > W dniu 05.10.2017 o 08:01, Sean Busbey pisze:
> > >>
> > >> Presuming this is the using the C++ client that is under active
> > >> development and not yet in a release, please limit discussion of it to
> > >> the dev@hbase list.
> > >>
> > >> (I have sent this to the dev list and moved the user@ list to bcc)
> > >
> > >
> > > Is anywhere features of native client?, what is working, what yet no.
> > > I want use this client because is very fast, whereas in my tests,
> Thrift
> > is
> > > slow.
> >
>
Re: If possible read families from tables and (more important) qualifiers?
Posted by Ted Yu <yu...@gmail.com>.
See Enis' comment:
https://issues.apache.org/jira/browse/HBASE-14850?focusedCommentId=15840799&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15840799
though it was half year old.
On Thu, Oct 5, 2017 at 8:53 AM, Sean Busbey <bu...@apache.org> wrote:
> I believe there is an umbrella jira tracking the progress of the
> feature on its branch. I haven't kept up with development enough to know if
> it's accurate and/or if things have reached the stage of explicitly
> documenting things.
>
> @Ted Y might be able to provide a pointer?
>
> On Thu, Oct 5, 2017 at 2:03 AM, Andrzej <bo...@wp.pl> wrote:
> > W dniu 05.10.2017 o 08:01, Sean Busbey pisze:
> >>
> >> Presuming this is the using the C++ client that is under active
> >> development and not yet in a release, please limit discussion of it to
> >> the dev@hbase list.
> >>
> >> (I have sent this to the dev list and moved the user@ list to bcc)
> >
> >
> > Is anywhere features of native client?, what is working, what yet no.
> > I want use this client because is very fast, whereas in my tests, Thrift
> is
> > slow.
>
Re: If possible read families from tables and (more important) qualifiers?
Posted by Sean Busbey <bu...@apache.org>.
I believe there is an umbrella jira tracking the progress of the
feature on its branch. I haven't kept up with development enough to know if
it's accurate and/or if things have reached the stage of explicitly
documenting things.
@Ted Y might be able to provide a pointer?
On Thu, Oct 5, 2017 at 2:03 AM, Andrzej <bo...@wp.pl> wrote:
> W dniu 05.10.2017 o 08:01, Sean Busbey pisze:
>>
>> Presuming this is the using the C++ client that is under active
>> development and not yet in a release, please limit discussion of it to
>> the dev@hbase list.
>>
>> (I have sent this to the dev list and moved the user@ list to bcc)
>
>
> Is anywhere features of native client?, what is working, what yet no.
> I want use this client because is very fast, whereas in my tests, Thrift is
> slow.
Re: If possible read families from tables and (more important)
qualifiers?
Posted by Andrzej <bo...@wp.pl>.
W dniu 05.10.2017 o 08:01, Sean Busbey pisze:
> Presuming this is the using the C++ client that is under active
> development and not yet in a release, please limit discussion of it to
> the dev@hbase list.
>
> (I have sent this to the dev list and moved the user@ list to bcc)
Is anywhere features of native client?, what is working, what yet no.
I want use this client because is very fast, whereas in my tests, Thrift
is slow.
Re: If possible read families from tables and (more important) qualifiers?
Posted by Sean Busbey <bu...@apache.org>.
Presuming this is the using the C++ client that is under active
development and not yet in a release, please limit discussion of it to
the dev@hbase list.
(I have sent this to the dev list and moved the user@ list to bcc)
On Thu, Oct 5, 2017 at 12:41 AM, Andrzej <bo...@wp.pl> wrote:
> I get:
> return result->Value(family, qualifier).value()
>
> result is optional, OK - it works.
> But sometimes I must read unknown structure of table, or more often, I know
> families but I don't know qualifiers.
>
> P.S. At long last I succeed building HBase client out of Docker, probably I
> can even write shared library encapsulated HBase client to external program.
>
>
Re: If possible read families from tables and (more important) qualifiers?
Posted by Sean Busbey <bu...@apache.org>.
Presuming this is the using the C++ client that is under active
development and not yet in a release, please limit discussion of it to
the dev@hbase list.
(I have sent this to the dev list and moved the user@ list to bcc)
On Thu, Oct 5, 2017 at 12:41 AM, Andrzej <bo...@wp.pl> wrote:
> I get:
> return result->Value(family, qualifier).value()
>
> result is optional, OK - it works.
> But sometimes I must read unknown structure of table, or more often, I know
> families but I don't know qualifiers.
>
> P.S. At long last I succeed building HBase client out of Docker, probably I
> can even write shared library encapsulated HBase client to external program.
>
>
If possible read families from tables and (more important)
qualifiers?
Posted by Andrzej <bo...@wp.pl>.
I get:
return result->Value(family, qualifier).value()
result is optional, OK - it works.
But sometimes I must read unknown structure of table, or more often, I
know families but I don't know qualifiers.
P.S. At long last I succeed building HBase client out of Docker,
probably I can even write shared library encapsulated HBase client to
external program.
Re: Is possible make shared object from HBase sources?
Posted by Sambhaji Sawant <sa...@gmail.com>.
Hello guys
Can you please suggest how can I use Multitenancy in HBase.I was searching
it but cannot get proper information. Please post some information related
to the multitenancy in hbase
On Sat, Sep 30, 2017 at 12:02 AM, Andrzej <bo...@wp.pl> wrote:
> W dniu 29.09.2017 o 20:15, Ted Yu pisze:
>
>> I didn't get the 'final link failed: Bad value' error when building on
>> docker.
>>
>> Consider rebuilding docker VM.
>>
>
> I start time consuming process rebuilding docker by command "build
> --no-cache" I must also change dockerfile to change PIC of libevent?
>
Re: Is possible make shared object from HBase sources?
Posted by Ted Yu <yu...@gmail.com>.
From https://buckbuild.com/rule/cxx_library.html , it seems to
support shared PIC-enabled build on Android.
bq. it is main, it is not shared library?
I added main() because you started with cxx_binary in BUCK which requires
main().
I did a quick search but haven't found build target in BUCK which
generates shared
PIC-enabled output.
Consider using make / CMake.
On Sat, Sep 30, 2017 at 12:17 AM, Andrzej <bo...@wp.pl> wrote:
> W dniu 30.09.2017 o 06:46, Ted Yu pisze:
>
>> https://pastebin.com/PPGkuxGS
>>
>
> > int main(int argc, char *argv[]) {
>
> I see, it is main, it is not shared library?
>
>
>> In BUCK, there is no need for compiler_flags and linker_flags
>>
>
> Compiler flags and linker flags to compilation .so file instead executable
> binary.
> Code pastebin.com/PPGkuxGS creates executable or shared library?
>
Re: Is possible make shared object from HBase sources?
Posted by Andrzej <bo...@wp.pl>.
W dniu 30.09.2017 o 06:46, Ted Yu pisze:
> https://pastebin.com/PPGkuxGS
> int main(int argc, char *argv[]) {
I see, it is main, it is not shared library?
>
> In BUCK, there is no need for compiler_flags and linker_flags
Compiler flags and linker flags to compilation .so file instead
executable binary.
Code pastebin.com/PPGkuxGS creates executable or shared library?
Re: Is possible make shared object from HBase sources?
Posted by Ted Yu <yu...@gmail.com>.
Was there particular reason you used the following ?
extern "C" {
I was able to build the following:
https://pastebin.com/PPGkuxGS
In BUCK, there is no need for compiler_flags and linker_flags
See if the above formation works for you.
On Fri, Sep 29, 2017 at 8:28 PM, Andrzej <bo...@wp.pl> wrote:
> W dniu 30.09.2017 o 01:07, Ted Yu pisze:
>
>> If you can generate a (minimal) patch (without revealing your IP) which
>> fails (buck) building reliably, I will surely investigate.
>>
>> Minimal example with only one method which uses Configuration and Client:
> https://gist.github.com/borneq/9cdb03dba4ba29c971bf5bea73857446
>
> Thanks,
>
Re: Is possible make shared object from HBase sources?
Posted by Andrzej <bo...@wp.pl>.
W dniu 30.09.2017 o 01:07, Ted Yu pisze:
> If you can generate a (minimal) patch (without revealing your IP) which
> fails (buck) building reliably, I will surely investigate.
>
Minimal example with only one method which uses Configuration and Client:
https://gist.github.com/borneq/9cdb03dba4ba29c971bf5bea73857446
Thanks,
Re: Is possible make shared object from HBase sources?
Posted by Ted Yu <yu...@gmail.com>.
If you can generate a (minimal) patch (without revealing your IP) which
fails (buck) building reliably, I will surely investigate.
Cheers
On Fri, Sep 29, 2017 at 2:33 PM, Andrzej <bo...@wp.pl> wrote:
> W dniu 29.09.2017 o 21:40, Andrzej pisze:
>
>> This is gist of my shared object:
>> https://gist.github.com/borneq/7148a6f13af736869fd9b1038b85e690
>> It compiling but is linking error - libevent.a is not PIC
>>
>>
> My changed BUCK file HBASE-14850/hbase-native-client/third-party/BUCK
> https://pastebin.com/7jDZTTPN
> Note: I have add compiler_flags=['-fPIC'] and changed force_static to
> False but not helps...
>
Re: Is possible make shared object from HBase sources?
Posted by Andrzej <bo...@wp.pl>.
W dniu 29.09.2017 o 21:40, Andrzej pisze:
> This is gist of my shared object:
> https://gist.github.com/borneq/7148a6f13af736869fd9b1038b85e690
> It compiling but is linking error - libevent.a is not PIC
>
My changed BUCK file HBASE-14850/hbase-native-client/third-party/BUCK
https://pastebin.com/7jDZTTPN
Note: I have add compiler_flags=['-fPIC'] and changed force_static to
False but not helps...
Re: Is possible make shared object from HBase sources?
Posted by Andrzej <bo...@wp.pl>.
W dniu 29.09.2017 o 21:32, Andrzej pisze:
> I have changed HBASE-14850/hbase-native-client/third-party/BUCK
>
> def add_system_libs(names=[],
> lib_dir="/usr/lib/x86_64-linux-gnu",
> deps=[],
> exported_deps=[],
> exported_linker_flags=[],
> compiler_flags=['-fPIC']):<-------------HERE
>
> and force_static=False,
>
> But still error
This is gist of my shared object:
https://gist.github.com/borneq/7148a6f13af736869fd9b1038b85e690
It compiling but is linking error - libevent.a is not PIC
Re: Is possible make shared object from HBase sources?
Posted by Andrzej <bo...@wp.pl>.
W dniu 29.09.2017 o 20:32, Andrzej pisze:
> I start time consuming process rebuilding docker by command "build
> --no-cache" I must also change dockerfile to change PIC of libevent?
I have changed HBASE-14850/hbase-native-client/third-party/BUCK
def add_system_libs(names=[],
lib_dir="/usr/lib/x86_64-linux-gnu",
deps=[],
exported_deps=[],
exported_linker_flags=[],
compiler_flags=['-fPIC']):<-------------HERE
and force_static=False,
But still error
Re: Is possible make shared object from HBase sources?
Posted by Andrzej <bo...@wp.pl>.
W dniu 29.09.2017 o 20:15, Ted Yu pisze:
> I didn't get the 'final link failed: Bad value' error when building on
> docker.
>
> Consider rebuilding docker VM.
I start time consuming process rebuilding docker by command "build
--no-cache" I must also change dockerfile to change PIC of libevent?
Re: Is possible make shared object from HBase sources?
Posted by Ted Yu <yu...@gmail.com>.
I didn't get the 'final link failed: Bad value' error when building on
docker.
Consider rebuilding docker VM.
FYI
On Fri, Sep 29, 2017 at 11:01 AM, Andrzej <bo...@wp.pl> wrote:
> W dniu 29.09.2017 o 19:12, Ted Yu pisze:
>
>> bq. Problem is with libevent.a library
>>
>> Can you share more details on what you found ?
>>
>> I did a quick search but didn't find how libevent.a is involved.
>>
>>
> Compiling with Buck and Docker:
> [+] BUILDING...12.4s [16%] (89/91 JOBS, 2 UPDATED, 2 [2.2%] CACHE MISS)
> |=> //NativeLib:NativeLib#binary... 2.0s (running c++ link[1.9s])
> /usr/bin/ld: /usr/src/hbase/hbase-native-client/buck-out/gen/third-party/
> gen_libevent/gen_libevent/libevent.a(event.o): relocation R_X86_64_PC32
> against symbol `event_global_current_base_' can not be used when making a
> shared object; recompile with -fPIC
> /usr/bin/ld: final link failed: Bad value
> clang: error: linker command failed with exit code 1 (use -v to see
> invocation)
>
>
Re: Is possible make shared object from HBase sources?
Posted by Andrzej <bo...@wp.pl>.
W dniu 29.09.2017 o 19:12, Ted Yu pisze:
> bq. Problem is with libevent.a library
>
> Can you share more details on what you found ?
>
> I did a quick search but didn't find how libevent.a is involved.
>
Compiling with Buck and Docker:
[+] BUILDING...12.4s [16%] (89/91 JOBS, 2 UPDATED, 2 [2.2%] CACHE MISS)
|=> //NativeLib:NativeLib#binary... 2.0s (running c++ link[1.9s])
/usr/bin/ld:
/usr/src/hbase/hbase-native-client/buck-out/gen/third-party/gen_libevent/gen_libevent/libevent.a(event.o):
relocation R_X86_64_PC32 against symbol `event_global_current_base_' can
not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
clang: error: linker command failed with exit code 1 (use -v to see
invocation)
Re: Is possible make shared object from HBase sources?
Posted by Ted Yu <yu...@gmail.com>.
bq. Problem is with libevent.a library
Can you share more details on what you found ?
I did a quick search but didn't find how libevent.a is involved.
On Fri, Sep 29, 2017 at 10:08 AM, Andrzej <bo...@wp.pl> wrote:
> I don't know why program hangs at:
> hTable->Put(*std::move(put))
> is I have written standalone test program, it was not errors.
>
> Secondly: standalone program puts column more, *more faster* than two
> programs communicate by tcp and even udp.
> Is one solution: shared object, but I have tried it a month ago. Problems
> was with Position Independent Code (PIC).
> Problem is with libevent.a library.
> It is possible change it , maybe by modify dockerfile?
>
>
Is possible make shared object from HBase sources?
Posted by Andrzej <bo...@wp.pl>.
I don't know why program hangs at:
hTable->Put(*std::move(put))
is I have written standalone test program, it was not errors.
Secondly: standalone program puts column more, *more faster* than two
programs communicate by tcp and even udp.
Is one solution: shared object, but I have tried it a month ago.
Problems was with Position Independent Code (PIC).
Problem is with libevent.a library.
It is possible change it , maybe by modify dockerfile?
Re: Program hangs at hTable->Put(*std::move(put))
Posted by Ted Yu <yu...@gmail.com>.
Are you running the client on docker environment for a real cluster ?
Can you list the output of 'uname -a' from one of the cluster nodes ?
Is it possible to narrow the scenario in a unit test ?
For the row you're writing to, locate the region hosting it. On the region
server for this region, was there active connection from the client ?
Thanks
On Fri, Sep 29, 2017 at 8:45 AM, Andrzej <bo...@wp.pl> wrote:
> W dniu 29.09.2017 o 17:14, Ted Yu pisze:
>
>> From client side, you can specify any column you want to write.
>>>
>> If the column happens to exist for the row, new value would be written.
>>
>> For the code snippet, there is getTable() and Put().
>> Can you separate into two statements to narrow down which call hangs ?
>>
>> Which commit are you using in the branch ?
>>
>> Did you build with buck ?
>>
>
> I build with buck, I use commit e2bb10a8708e7a675558ca0aad9e340823573ff6
> from 2017-09-22
>
> messages:
> getTable hTable = 0x10000
> getTable hEnv=1 hTable & 0xffff=0
> getTable, after get()
> after getTable, before Put, table=1800d300
>
> code:
> my_printf("hbPutColumns before put\n");
> Table *table = getTable(hTable);
> my_printf("after getTable, before Put, table=%x\n",table);
> table->Put(*put);
> my_printf("end hbPutColumns\n");
>
> table is not null, it is raw pointer from unique pointer
>
Re: Program hangs at hTable->Put(*std::move(put))
Posted by Andrzej <bo...@wp.pl>.
W dniu 29.09.2017 o 17:14, Ted Yu pisze:
>>From client side, you can specify any column you want to write.
> If the column happens to exist for the row, new value would be written.
>
> For the code snippet, there is getTable() and Put().
> Can you separate into two statements to narrow down which call hangs ?
>
> Which commit are you using in the branch ?
>
> Did you build with buck ?
I build with buck, I use commit e2bb10a8708e7a675558ca0aad9e340823573ff6
from 2017-09-22
messages:
getTable hTable = 0x10000
getTable hEnv=1 hTable & 0xffff=0
getTable, after get()
after getTable, before Put, table=1800d300
code:
my_printf("hbPutColumns before put\n");
Table *table = getTable(hTable);
my_printf("after getTable, before Put, table=%x\n",table);
table->Put(*put);
my_printf("end hbPutColumns\n");
table is not null, it is raw pointer from unique pointer
Re: Program hangs at hTable->Put(*std::move(put))
Posted by Ted Yu <yu...@gmail.com>.
From client side, you can specify any column you want to write.
If the column happens to exist for the row, new value would be written.
For the code snippet, there is getTable() and Put().
Can you separate into two statements to narrow down which call hangs ?
Which commit are you using in the branch ?
Did you build with buck ?
Cheers
On Fri, Sep 29, 2017 at 7:33 AM, Andrzej <bo...@wp.pl> wrote:
> W dniu 29.09.2017 o 16:25, Andrzej pisze:
>
>> Maybe problem with unique_ptr?
>>
>
> Changing to shared_ptr not helps.
> More info:
> first I add less column, without variant column and is OK.
> But second pass I update row - for ths same row I add new column and maybe
> put existing columns? What is error when put existing columns and how
> replace columns?
>
Re: Program hangs at hTable->Put(*std::move(put))
Posted by Andrzej <bo...@wp.pl>.
W dniu 29.09.2017 o 16:25, Andrzej pisze:
> Maybe problem with unique_ptr?
Changing to shared_ptr not helps.
More info:
first I add less column, without variant column and is OK.
But second pass I update row - for ths same row I add new column and
maybe put existing columns? What is error when put existing columns and
how replace columns?
Program hangs at hTable->Put(*std::move(put))
Posted by Andrzej <bo...@wp.pl>.
void HBaseSession::hbPutColumns(uint32_t hTable, string row,
vector<string>& families, vector<string>& qualifiers, vector<string>&
values)
{
auto put = std::make_unique<Put>(row);
for (int i = 0; i<qualifiers.size(); i++)
put->AddColumn(families[i], qualifiers[i], values[i]);
my_printf("hbPutColumns before put\n");
getTable(hTable)->Put(*std::move(put)); //<--------------hangs!
my_printf("end hbPutColumns\n");
}
Program go into procedure getTable, but not do my_printf("end
hbPutColumns\n")
Maybe problem with unique_ptr?
Many times program do this Put, but sometimes hangs
Re: How avoid error when variant column not exists ?
Posted by Ted Yu <yu...@gmail.com>.
The return type of Value is optional:
optional<std::string> Value(const std::string &family, const std::string
&qualifier) const;
You can use the following expression:
result->Value("d", "extra") == none
FYI
On Thu, Sep 28, 2017 at 5:58 AM, Andrzej <bo...@wp.pl> wrote:
> I get:
> return result->Value(family, qualifier).value()
> how can I check if (family,qulifier) exists? or is possible return empty
> string instead error?
>