You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Hilmi Yildirim <hi...@neofonie.de> on 2015/05/28 10:24:14 UTC
Read from HBase flink-0.9.0-milestone-1-bin-hadoop2
Hi,
I also tried to read from HBase using the
flink-0.9.0-milestone-1-bin-hadoop2 version. It works great and it uses
the config declared in the hbase-site.xml.
I configured a flink cluster with 2 machines. For that I deployed the
flink folder with the corresponding configurations on each machine. To
start the cluster I use the start script ./start-cluster.sh. Then, I run
the job and it works. Now, I configured a flink cluster with 13
machines. All machines read from hbase correctly besides 1 or 2
machines. These 1 or 2 machines get following exception ||
"org.apache.hadoop.hbase.UnknownScannerException:
org.apache.hadoop.hbase.UnknownScannerException: Name 2423, already closed?"
When I restart the job then the exception occurs for other 1 or 2 machines.
Does anyone know why this exception occurs?
Best Regards,
--
--
Hilmi Yildirim
Software Developer R&D
T: +49 30 24627-281
Hilmi.Yildirim@neofonie.de
http://www.neofonie.de
Besuchen Sie den Neo Tech Blog für Anwender:
http://blog.neofonie.de/
Folgen Sie uns:
https://plus.google.com/+neofonie
http://www.linkedin.com/company/neofonie-gmbh
https://www.xing.com/companies/neofoniegmbh
Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
Handelsregister Berlin-Charlottenburg: HRB 67460
Geschäftsführung: Thomas Kitlitschko
Re: Read from HBase flink-0.9.0-milestone-1-bin-hadoop2
Posted by Stephan Ewen <se...@apache.org>.
Can you look into the Flink and HBase logs if it locks some exception there?
On Thu, May 28, 2015 at 10:24 AM, Hilmi Yildirim <hilmi.yildirim@neofonie.de
> wrote:
> Hi,
> I also tried to read from HBase using the
> flink-0.9.0-milestone-1-bin-hadoop2 version. It works great and it uses the
> config declared in the hbase-site.xml.
>
> I configured a flink cluster with 2 machines. For that I deployed the
> flink folder with the corresponding configurations on each machine. To
> start the cluster I use the start script ./start-cluster.sh. Then, I run
> the job and it works. Now, I configured a flink cluster with 13 machines.
> All machines read from hbase correctly besides 1 or 2 machines. These 1 or
> 2 machines get following exception
> "org.apache.hadoop.hbase.UnknownScannerException:
> org.apache.hadoop.hbase.UnknownScannerException: Name 2423, already closed?"
>
> When I restart the job then the exception occurs for other 1 or 2 machines.
>
> Does anyone know why this exception occurs?
>
> Best Regards,
>
> --
> --
> Hilmi Yildirim
> Software Developer R&D
>
> T: +49 30 24627-281Hilmi.Yildirim@neofonie.de
> http://www.neofonie.de
>
> Besuchen Sie den Neo Tech Blog für Anwender:http://blog.neofonie.de/
>
> Folgen Sie uns:https://plus.google.com/+neofoniehttp://www.linkedin.com/company/neofonie-gmbhhttps://www.xing.com/companies/neofoniegmbh
>
> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
> Handelsregister Berlin-Charlottenburg: HRB 67460
> Geschäftsführung: Thomas Kitlitschko
>
>
Re: Read from HBase flink-0.9.0-milestone-1-bin-hadoop2
Posted by Hilmi Yildirim <hi...@neofonie.de>.
I solved the problem. The problem was that I created an HBase table only
on 1 machine. 13 machines requested data from this sigle machine which
lead to a bottleneck. Now, I have split the table on 13 machines and it
is working now. It continues to throw UnknownScannerExceptions but it is
working. Maybe it is beneficially to create a task to catch this
exception or to handle it correctly.
Best Regards,
Hilmi
Am 28.05.2015 um 11:34 schrieb Flavio Pompermaier:
>
> The unknown scanner exception is just a warning due to the fact
> that the table scan is not eager but lazy..so between two consecutive
> getNextRecord() call the Habase server timeouts the scanner
> resource's..In the house extension I just recreate the scanner from
> the last read key. This is a workaround to avoid too much tuning with
> caching and timeouts hbase params...
>
> On 28 May 2015 10:24, "Hilmi Yildirim" <hilmi.yildirim@neofonie.de
> <ma...@neofonie.de>> wrote:
>
> Hi,
> I also tried to read from HBase using the
> flink-0.9.0-milestone-1-bin-hadoop2 version. It works great and it
> uses the config declared in the hbase-site.xml.
>
> I configured a flink cluster with 2 machines. For that I deployed
> the flink folder with the corresponding configurations on each
> machine. To start the cluster I use the start script
> ./start-cluster.sh. Then, I run the job and it works. Now, I
> configured a flink cluster with 13 machines. All machines read
> from hbase correctly besides 1 or 2 machines. These 1 or 2
> machines get following exception ||
> "org.apache.hadoop.hbase.UnknownScannerException:
> org.apache.hadoop.hbase.UnknownScannerException: Name 2423,
> already closed?"
>
> When I restart the job then the exception occurs for other 1 or 2
> machines.
>
> Does anyone know why this exception occurs?
>
> Best Regards,
>
> --
> --
> Hilmi Yildirim
> Software Developer R&D
>
> T:+49 30 24627-281 <tel:%2B49%2030%2024627-281>
> Hilmi.Yildirim@neofonie.de <ma...@neofonie.de>
>
> http://www.neofonie.de
>
> Besuchen Sie den Neo Tech Blog für Anwender:
> http://blog.neofonie.de/
>
> Folgen Sie uns:
> https://plus.google.com/+neofonie
> http://www.linkedin.com/company/neofonie-gmbh
> https://www.xing.com/companies/neofoniegmbh
>
> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
> Handelsregister Berlin-Charlottenburg: HRB 67460
> Geschäftsführung: Thomas Kitlitschko
>
--
--
Hilmi Yildirim
Software Developer R&D
T: +49 30 24627-281
Hilmi.Yildirim@neofonie.de
http://www.neofonie.de
Besuchen Sie den Neo Tech Blog für Anwender:
http://blog.neofonie.de/
Folgen Sie uns:
https://plus.google.com/+neofonie
http://www.linkedin.com/company/neofonie-gmbh
https://www.xing.com/companies/neofoniegmbh
Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
Handelsregister Berlin-Charlottenburg: HRB 67460
Geschäftsführung: Thomas Kitlitschko
Re: Read from HBase flink-0.9.0-milestone-1-bin-hadoop2
Posted by Flavio Pompermaier <po...@okkam.it>.
The unknown scanner exception is just a warning due to the fact that the
table scan is not eager but lazy..so between two consecutive
getNextRecord() call the Habase server timeouts the scanner resource's..In
the house extension I just recreate the scanner from the last read key.
This is a workaround to avoid too much tuning with caching and timeouts
hbase params...
On 28 May 2015 10:24, "Hilmi Yildirim" <hi...@neofonie.de> wrote:
> Hi,
> I also tried to read from HBase using the
> flink-0.9.0-milestone-1-bin-hadoop2 version. It works great and it uses the
> config declared in the hbase-site.xml.
>
> I configured a flink cluster with 2 machines. For that I deployed the
> flink folder with the corresponding configurations on each machine. To
> start the cluster I use the start script ./start-cluster.sh. Then, I run
> the job and it works. Now, I configured a flink cluster with 13 machines.
> All machines read from hbase correctly besides 1 or 2 machines. These 1 or
> 2 machines get following exception
> "org.apache.hadoop.hbase.UnknownScannerException:
> org.apache.hadoop.hbase.UnknownScannerException: Name 2423, already closed?"
>
> When I restart the job then the exception occurs for other 1 or 2 machines.
>
> Does anyone know why this exception occurs?
>
> Best Regards,
>
> --
> --
> Hilmi Yildirim
> Software Developer R&D
>
> T: +49 30 24627-281Hilmi.Yildirim@neofonie.de
> http://www.neofonie.de
>
> Besuchen Sie den Neo Tech Blog für Anwender:http://blog.neofonie.de/
>
> Folgen Sie uns:https://plus.google.com/+neofoniehttp://www.linkedin.com/company/neofonie-gmbhhttps://www.xing.com/companies/neofoniegmbh
>
> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
> Handelsregister Berlin-Charlottenburg: HRB 67460
> Geschäftsführung: Thomas Kitlitschko
>
>