Apache Drill - Unable to start Drill in distributed mode (In GCP Dataproc)

[Vigneswaran S <vi...@gmail.com>]

Dear Apache Drill Team,

I am trying to run Apache Drill in distributed mode on Google Cloud
Dataproc, but unable to start drillbit on each node in the cluster.

I have created a basic cluster (1 master, 2 worker) with GCP Dataproc
service, using the initialization scripts and instructions provided in the
Apache Drill website.

https://drill.apache.org/docs/installing-drill-in-distributed-mode-with-gcp-dataproc/


Apache Drill 1.19.0 and Apache Zookeeper 3.6.3 versions were configured in
the setup script. The cluster provisioning in Dataproc was successful and I
am able to connect with each node using SSH. When I tried to check the
status of Zookeeper using telnet localhost 2181 and entering stats, it is
showing the following

[image: zookeeper.png]

Then, I try to start drillbit service on each node using the command
bin/drillbit.sh start as mentioned here

https://drill.apache.org/docs/starting-drill-in-distributed-mode/

then it shows

Starting drillbit, logging to /opt/drill/log/drillbit.out

When I check the status of drill using bin/drillbit.sh status, it displays

/opt/drill/drillbit.pid file is present but drillbit is not running.

When I try to access Drill web UI public_ip_addr:8047 using public ip
address of any node, it gives "can’t establish a connection to the server".
So it is unclear whether drill is running or not. Note: I have opened port
8047 under firewall rules

Kindly provide help on how to resolve the issue and set up Apache Drill in
distributed mode on GCP.

Regards,
Vigneswaran S
vigneswaran.raw@gmail.com

Re: Apache Drill - Unable to start Drill in distributed mode (In GCP Dataproc)

[James Turton <dz...@apache.org>]

Hi!

Please see my comment on your StackOverflow post. After looking at the 
provisioning scripts that you're using, I don't think you should be 
trying to start Drillbits manually.

Regards
James

On 2023/05/29 16:28, Vigneswaran S wrote:
> Dear Apache Drill Team,
>
> I am trying to run Apache Drill in distributed mode on Google Cloud 
> Dataproc, but unable to start drillbit on each node in the cluster.
>
> I have created a basic cluster (1 master, 2 worker) with GCP Dataproc 
> service, using the initialization scripts and instructions provided in 
> the Apache Drill website.
>
> https://drill.apache.org/docs/installing-drill-in-distributed-mode-with-gcp-dataproc/ 
>
>
> Apache Drill 1.19.0 and Apache Zookeeper 3.6.3 versions were 
> configured in the setup script. The cluster provisioning in Dataproc 
> was successful and I am able to connect with each node using SSH. When 
> I tried to check the status of Zookeeper using telnet localhost 2181 
> and entering stats, it is showing the following
>
> zookeeper.png
>
> Then, I try to start drillbit service on each node using the command 
> bin/drillbit.sh start as mentioned here
>
> https://drill.apache.org/docs/starting-drill-in-distributed-mode/
>
> then it shows
>
> Starting drillbit, logging to /opt/drill/log/drillbit.out
>
> When I check the status of drill using bin/drillbit.sh status, it displays
>
> /opt/drill/drillbit.pid file is present but drillbit is not running.
>
> When I try to access Drill web UI public_ip_addr:8047 using public ip 
> address of any node, it gives "can’t establish a connection to the 
> server". So it is unclear whether drill is running or not. Note: I 
> have opened port 8047 under firewall rules
>
> Kindly provide help on how to resolve the issue and set up Apache 
> Drill in distributed mode on GCP.
>
> Regards,
> Vigneswaran S
> vigneswaran.raw@gmail.com
>