You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Hilmi Yildirim <hi...@neofonie.de> on 2015/05/28 10:14:33 UTC
Read HBase release-0.8.1
Hi,
I used release-0.8.1for Hadoop 2 to read from HBase. For that I placed
my hbase-site.xml with the correct configurations into the resource
folder. When I start the job locally then the job uses the config inside
the hbase-site.xml and it works great. When I want to run the job on a
cluster then the job uses the default hbase config instead of the config
defined in the hbase-site.xml. The hbase-site.xml is included in the fat
jar. To start the job I use the web interace.
Best Regards,
--
--
Hilmi Yildirim
Software Developer R&D
T: +49 30 24627-281
Hilmi.Yildirim@neofonie.de
http://www.neofonie.de
Besuchen Sie den Neo Tech Blog für Anwender:
http://blog.neofonie.de/
Folgen Sie uns:
https://plus.google.com/+neofonie
http://www.linkedin.com/company/neofonie-gmbh
https://www.xing.com/companies/neofoniegmbh
Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
Handelsregister Berlin-Charlottenburg: HRB 67460
Geschäftsführung: Thomas Kitlitschko
Re: Read HBase release-0.8.1
Posted by Hilmi Yildirim <hi...@neofonie.de>.
I solved the problem by overwriting the method "configure". There, I
used the "addResource" method of the HBaseConfiguration to add the
correct hbase-site.xml
Best Regards,
Hilmi
Am 28.05.2015 um 11:21 schrieb Hilmi Yildirim:
> When I execute both job locally then the result of both methods is the
> same. But when I execute the job on the cluster then the first method
> returns a path to the hbase-site.xml but the second method returns null.
>
> Best Regards,
> Hilmi
>
> Am 28.05.2015 um 10:51 schrieb Stephan Ewen:
>> I am not an HBase expert. How does HBase find the hbase-site.xml?
>> Does it simply search through the classpath?
>>
>> If it uses the classpath, can you see what hbase-site.xml is used?
>> - In your user code, you can do a
>> getClass().getClassloader().gerResource("hbase-site.xml") and see
>> where it is?
>> - Can you do the same with
>> ClassLoader.getSystemClassLoader().getResource("hbase-site.xml") and
>> see if that returns something different?
>>
>>
>> Thanks,
>> Stephan
>>
>>
>> On Thu, May 28, 2015 at 10:14 AM, Hilmi Yildirim
>> <hilmi.yildirim@neofonie.de <ma...@neofonie.de>> wrote:
>>
>> Hi,
>> I used release-0.8.1for Hadoop 2 to read from HBase. For that I
>> placed my hbase-site.xml with the correct configurations into the
>> resource folder. When I start the job locally then the job uses
>> the config inside the hbase-site.xml and it works great. When I
>> want to run the job on a cluster then the job uses the default
>> hbase config instead of the config defined in the hbase-site.xml.
>> The hbase-site.xml is included in the fat jar. To start the job I
>> use the web interace.
>>
>> Best Regards,
>>
>> --
>> --
>> Hilmi Yildirim
>> Software Developer R&D
>>
>> T: +49 30 24627-281 <tel:%2B49%2030%2024627-281>
>> Hilmi.Yildirim@neofonie.de <ma...@neofonie.de>
>>
>> http://www.neofonie.de
>>
>> Besuchen Sie den Neo Tech Blog für Anwender:
>> http://blog.neofonie.de/
>>
>> Folgen Sie uns:
>> https://plus.google.com/+neofonie
>> http://www.linkedin.com/company/neofonie-gmbh
>> https://www.xing.com/companies/neofoniegmbh
>>
>> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
>> Handelsregister Berlin-Charlottenburg: HRB 67460
>> Geschäftsführung: Thomas Kitlitschko
>>
>>
>
> --
> --
> Hilmi Yildirim
> Software Developer R&D
>
> T: +49 30 24627-281
> Hilmi.Yildirim@neofonie.de
>
> http://www.neofonie.de
>
> Besuchen Sie den Neo Tech Blog für Anwender:
> http://blog.neofonie.de/
>
> Folgen Sie uns:
> https://plus.google.com/+neofonie
> http://www.linkedin.com/company/neofonie-gmbh
> https://www.xing.com/companies/neofoniegmbh
>
> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
> Handelsregister Berlin-Charlottenburg: HRB 67460
> Geschäftsführung: Thomas Kitlitschko
--
--
Hilmi Yildirim
Software Developer R&D
T: +49 30 24627-281
Hilmi.Yildirim@neofonie.de
http://www.neofonie.de
Besuchen Sie den Neo Tech Blog für Anwender:
http://blog.neofonie.de/
Folgen Sie uns:
https://plus.google.com/+neofonie
http://www.linkedin.com/company/neofonie-gmbh
https://www.xing.com/companies/neofoniegmbh
Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
Handelsregister Berlin-Charlottenburg: HRB 67460
Geschäftsführung: Thomas Kitlitschko
Re: Read HBase release-0.8.1
Posted by Hilmi Yildirim <hi...@neofonie.de>.
When I execute both job locally then the result of both methods is the
same. But when I execute the job on the cluster then the first method
returns a path to the hbase-site.xml but the second method returns null.
Best Regards,
Hilmi
Am 28.05.2015 um 10:51 schrieb Stephan Ewen:
> I am not an HBase expert. How does HBase find the hbase-site.xml? Does
> it simply search through the classpath?
>
> If it uses the classpath, can you see what hbase-site.xml is used?
> - In your user code, you can do a
> getClass().getClassloader().gerResource("hbase-site.xml") and see
> where it is?
> - Can you do the same with
> ClassLoader.getSystemClassLoader().getResource("hbase-site.xml") and
> see if that returns something different?
>
>
> Thanks,
> Stephan
>
>
> On Thu, May 28, 2015 at 10:14 AM, Hilmi Yildirim
> <hilmi.yildirim@neofonie.de <ma...@neofonie.de>> wrote:
>
> Hi,
> I used release-0.8.1for Hadoop 2 to read from HBase. For that I
> placed my hbase-site.xml with the correct configurations into the
> resource folder. When I start the job locally then the job uses
> the config inside the hbase-site.xml and it works great. When I
> want to run the job on a cluster then the job uses the default
> hbase config instead of the config defined in the hbase-site.xml.
> The hbase-site.xml is included in the fat jar. To start the job I
> use the web interace.
>
> Best Regards,
>
> --
> --
> Hilmi Yildirim
> Software Developer R&D
>
> T: +49 30 24627-281 <tel:%2B49%2030%2024627-281>
> Hilmi.Yildirim@neofonie.de <ma...@neofonie.de>
>
> http://www.neofonie.de
>
> Besuchen Sie den Neo Tech Blog für Anwender:
> http://blog.neofonie.de/
>
> Folgen Sie uns:
> https://plus.google.com/+neofonie
> http://www.linkedin.com/company/neofonie-gmbh
> https://www.xing.com/companies/neofoniegmbh
>
> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
> Handelsregister Berlin-Charlottenburg: HRB 67460
> Geschäftsführung: Thomas Kitlitschko
>
>
--
--
Hilmi Yildirim
Software Developer R&D
T: +49 30 24627-281
Hilmi.Yildirim@neofonie.de
http://www.neofonie.de
Besuchen Sie den Neo Tech Blog für Anwender:
http://blog.neofonie.de/
Folgen Sie uns:
https://plus.google.com/+neofonie
http://www.linkedin.com/company/neofonie-gmbh
https://www.xing.com/companies/neofoniegmbh
Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
Handelsregister Berlin-Charlottenburg: HRB 67460
Geschäftsführung: Thomas Kitlitschko
Re: Read HBase release-0.8.1
Posted by Stephan Ewen <se...@apache.org>.
I am not an HBase expert. How does HBase find the hbase-site.xml? Does it
simply search through the classpath?
If it uses the classpath, can you see what hbase-site.xml is used?
- In your user code, you can do a
getClass().getClassloader().gerResource("hbase-site.xml") and see where it
is?
- Can you do the same with
ClassLoader.getSystemClassLoader().getResource("hbase-site.xml") and see if
that returns something different?
Thanks,
Stephan
On Thu, May 28, 2015 at 10:14 AM, Hilmi Yildirim <hilmi.yildirim@neofonie.de
> wrote:
> Hi,
> I used release-0.8.1for Hadoop 2 to read from HBase. For that I placed my
> hbase-site.xml with the correct configurations into the resource folder.
> When I start the job locally then the job uses the config inside the
> hbase-site.xml and it works great. When I want to run the job on a cluster
> then the job uses the default hbase config instead of the config defined in
> the hbase-site.xml. The hbase-site.xml is included in the fat jar. To start
> the job I use the web interace.
>
> Best Regards,
>
> --
> --
> Hilmi Yildirim
> Software Developer R&D
>
> T: +49 30 24627-281
> Hilmi.Yildirim@neofonie.de
>
> http://www.neofonie.de
>
> Besuchen Sie den Neo Tech Blog für Anwender:
> http://blog.neofonie.de/
>
> Folgen Sie uns:
> https://plus.google.com/+neofonie
> http://www.linkedin.com/company/neofonie-gmbh
> https://www.xing.com/companies/neofoniegmbh
>
> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
> Handelsregister Berlin-Charlottenburg: HRB 67460
> Geschäftsführung: Thomas Kitlitschko
>
>