Any pointers to launch drill bit programmatically

[Hema T <he...@gmail.com>]

Hello,

I am working on a use-case to use drill jdbc features. The ask is on
launching a drillbit programmatically within a Java prog execution without
having it run separately in embedded/distributed mode. Any Java API
references/examples /suggestions will be helpful.
Thanks

Re: Any pointers to launch drill bit programmatically

[Paul Rogers <pa...@yahoo.com.INVALID>]

Hi Hema,


Welcome to Drill! Drill's primary use case is to run distributed queries across multiple nodes. For that, Drill generally runs as a stand-alone process, perhaps launched by YARN, Docker, K8s or your own scripts. Although Drill runs in embedded mode, this is mostly to provide a "quick start" when playing with sample and local data in Sqlline.


Take a look at the file $DRILL_HOME/bin/embedded. Just use the following JDBC connection string: jdbc:drill:zk=local. All of Drill's jars must be on your class path. Drill also must have visibility to its config files.

You may find, however, that Drill is quite heavy-weight when used as a library, since this is not the main use case. For example:

* Drill will run in your root class loader. This means that Drill's many dependencies may conflict with the versions you use. 


* Drill uses storage plugin configurations to access data. In server mode, Drill stores this information in ZK so it is available across runs. In embedded mode, the data is stored in /tmp (which can be changed.)

* Drill will use a large amount of memory to run queries: the default is 4 GB. Queries will run across multiple threads and make heavy use of CPU (if you have large amounts of data.)


* If you plan to run more than one of your applications on a single node, you may end up with port conflicts as Drill will bind to its four ports, even in embedded mode.


While you can make an embedded Drill work, you might find you are better off running a Drill process separately from your application.

Thanks,
- Paul

 

    On Tuesday, February 4, 2020, 10:22:41 PM PST, Hema T <he...@gmail.com> wrote:  
 
 Hello,

I am working on a use-case to use drill jdbc features. The ask is on
launching a drillbit programmatically within a Java prog execution without
having it run separately in embedded/distributed mode. Any Java API
references/examples /suggestions will be helpful.
Thanks