You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by James Taylor <ja...@apache.org> on 2018/02/02 19:13:07 UTC
Re: Apache Phoenix integration
There's also a much deeper integration between Phoenix + Drill (code named
Drillix) underway that should be possible to complete now that Apache Drill
is off of it's fork and on a later version of Apache Calcite. I'm hoping
that the output of this will be a Phoenix adapter in Drill. See
presentation here[1] and WIP code here[2].
Thanks,
James
[1] http://phoenix.apache.org/presentations/Drillix.pdf
[2] https://github.com/jacques-n/drill/tree/phoenix_plugin
On Fri, Feb 2, 2018 at 10:21 AM, Kunal Khatua <kk...@mapr.com> wrote:
> That's great, Flavio!
>
> You can create a Google doc for review and share it on the user list.
>
> @Bridget handles the documentation on the Apache website, so she can do
> the final touches and help it find a home on the website.
>
> -----Original Message-----
> From: Flavio Pompermaier [mailto:pompermaier@okkam.it]
> Sent: Friday, February 02, 2018 9:04 AM
> To: user@drill.apache.org
> Cc: James Taylor <ja...@apache.org>
> Subject: Re: Apache Phoenix integration
>
> Eventually I made it to integrate Phoenix with Drill! I debugged remotely
> the drill-embedded via Eclipse and I discovered that the problem was that
> you need some extra jars to make it work!
> Where can I write some documentation about debugging remotely Drill from
> Eclipse and integration with Drill?
>
> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier <po...@okkam.it>
> wrote:
>
> > What is the fastest way to debug the JDBC plugin from Eclipse? I don't
> > see anything in the logs that could help...
> > Is it possible to connect directly to the external embedded drill
> > running on my machine if I enable jmx?
> > it seems that the JDBC connection is established correctly but Drill
> > throws an Exception (that is not well unwrapped by Jersey):
> >
> > 2018-02-02 16:54:04,520 [qtp159619134-56] INFO
> > o.a.p.q.ConnectionQueryServicesImpl
> > - HConnection established. Stacktrace for informational purposes:
> > hconnection-0x1b9fe9f8
> > java.lang.Thread.getStackTrace(Thread.java:1552)
> > org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(
> > ConnectionQueryServicesImpl.java:410)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(
> > ConnectionQueryServicesImpl.java:256)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> > ConnectionQueryServicesImpl.java:2408)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> > ConnectionQueryServicesImpl.java:2384)
> > org.apache.phoenix.util.PhoenixContextExecutor.call(
> > PhoenixContextExecutor.java:76)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
> > ConnectionQueryServicesImpl.java:2384)
> > org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(
> > PhoenixDriver.java:255)
> > org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(
> > PhoenixEmbeddedDriver.java:150)
> > org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
> > org.apache.commons.dbcp.DriverConnectionFactory.createConnection(
> > DriverConnectionFactory.java:38)
> > org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(
> > PoolableConnectionFactory.java:582)
> > org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(
> > BasicDataSource.java:1556)
> > org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactor
> > y(BasicDataSource.java:1545)
> > org.apache.commons.dbcp.BasicDataSource.createDataSource(
> > BasicDataSource.java:1388)
> > org.apache.commons.dbcp.BasicDataSource.getConnection(
> > BasicDataSource.java:1044)
> > org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.
> > get(JdbcUtils.java:73)
> > org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(
> > JdbcSchema.java:138)
> > org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.<init>(
> > JdbcStoragePlugin.java:103)
> > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> > sun.reflect.NativeConstructorAccessorImpl.newInstance(
> > NativeConstructorAccessorImpl.java:62)
> > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> > DelegatingConstructorAccessorImpl.java:45)
> > java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> > org.apache.drill.exec.store.StoragePluginRegistryImpl.create(
> > StoragePluginRegistryImpl.java:346)
> > org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(
> > StoragePluginRegistryImpl.java:239)
> > org.apache.drill.exec.server.rest.PluginConfigWrapper.
> > createOrUpdateInStorage(PluginConfigWrapper.java:57)
> > org.apache.drill.exec.server.rest.StorageResources.
> > createOrUpdatePluginJSON(StorageResources.java:162)
> > org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlugi
> > n(
> > StorageResources.java:177)
> > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:
> > 62)
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:43)
> > java.lang.reflect.Method.invoke(Method.java:498)
> > org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHan
> > dle
> > rFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
> > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD
> > isp
> > atcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
> > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD
> > isp
> > atcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
> > org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatche
> > rPr ovider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherPr
> > ovider.java:195)
> > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD
> > isp
> > atcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104)
> > org.glassfish.jersey.server.model.ResourceMethodInvoker.
> > invoke(ResourceMethodInvoker.java:387)
> > org.glassfish.jersey.server.model.ResourceMethodInvoker.
> > apply(ResourceMethodInvoker.java:331)
> > org.glassfish.jersey.server.model.ResourceMethodInvoker.
> > apply(ResourceMethodInvoker.java:103)
> > org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269
> > )
> > org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
> > org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
> > org.glassfish.jersey.internal.Errors.process(Errors.java:315)
> > org.glassfish.jersey.internal.Errors.process(Errors.java:297)
> > org.glassfish.jersey.internal.Errors.process(Errors.java:267)
> > org.glassfish.jersey.process.internal.RequestScope.
> > runInScope(RequestScope.java:297)
> > org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:2
> > 52) org.glassfish.jersey.server.ApplicationHandler.handle(
> > ApplicationHandler.java:1023)
> > org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:37
> > 2) org.glassfish.jersey.servlet.ServletContainer.service(
> > ServletContainer.java:382)
> > org.glassfish.jersey.servlet.ServletContainer.service(
> > ServletContainer.java:345)
> > org.glassfish.jersey.servlet.ServletContainer.service(
> > ServletContainer.java:220)
> > org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)
> > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:
> > 551) org.eclipse.jetty.server.session.SessionHandler.
> > doHandle(SessionHandler.java:219)
> > org.eclipse.jetty.server.handler.ContextHandler.
> > doHandle(ContextHandler.java:1111)
> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:4
> > 78) org.eclipse.jetty.server.session.SessionHandler.
> > doScope(SessionHandler.java:183)
> > org.eclipse.jetty.server.handler.ContextHandler.
> > doScope(ContextHandler.java:1045)
> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
> > ScopedHandler.java:141)
> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> > HandlerWrapper.java:97)
> > org.eclipse.jetty.server.Server.handle(Server.java:462)
> > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
> > org.eclipse.jetty.server.HttpConnection.onFillable(
> > HttpConnection.java:232)
> > org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:
> > 534) org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> > QueuedThreadPool.java:607)
> > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(
> > QueuedThreadPool.java:536)
> > java.lang.Thread.run(Thread.java:745)
> >
> >
> > On Fri, Feb 2, 2018 at 10:21 AM, Flavio Pompermaier
> > <po...@okkam.it>
> > wrote:
> >
> >> Hi Saurabh,
> >> I'd like to use Drill because in my use case I have multiple sources
> >> (HDFS, HBase, Phoenix, Oracle, Mysql) and I'd like to allow the user
> >> to see the data.
> >> Without writing a plugin for each storage system I'd like to leverage
> >> Apache Drill as a broker towards all of them.
> >>
> >> Right now I've tried to puth the phoenix-core.jar into the
> >> jar/3rdparty folder but when I try to create the Phoenix storage I
> >> get the following
> >> error:
> >> Please retry: error (unable to create/ update storage)
> >>
> >> I'm trying to debug but I don't see anything wrong..
> >>
> >>
> >> On Fri, Feb 2, 2018 at 1:42 AM, Saurabh Mahapatra <
> >> saurabhmahapatra94@gmail.com> wrote:
> >>
> >>> Hi Flavio,
> >>>
> >>> I was curious what your use case is. Phoenix provides SQL access
> >>> with secondary indexes on HBase. Why would you want to use Drill?
> >>>
> >>> Thanks,
> >>> Saurabh
> >>>
> >>> On Thu, Feb 1, 2018 at 3:08 PM, Kunal Khatua <kk...@mapr.com> wrote:
> >>>
> >>> > The JDBC storage plugin allows Drill to leverage any SQL system
> >>> > that
> >>> has
> >>> > JDBC drivers, so it should work.
> >>> >
> >>> > That said, the JDBC storage plugin is a community developed
> >>> > storage plugin, so it might not be fully tested.
> >>> >
> >>> > If you are looking to simply have the Phoenix JDBC driver bundled
> >>> > into
> >>> the
> >>> > Drill 3rd party directory, I (think) the reason it is not bundled
> >>> > is because we're not actively tracking compatibility of different
> >>> > JDBC
> >>> drivers
> >>> > with Drill. There could also be license issues, which is why we
> >>> > can't bundle drivers for platforms like Oracle, etc.
> >>> >
> >>> > ~ Kunal
> >>> >
> >>> > -----Original Message-----
> >>> > From: Flavio Pompermaier [mailto:pompermaier@okkam.it]
> >>> > Sent: Thursday, February 01, 2018 9:39 AM
> >>> > To: user@drill.apache.org
> >>> > Subject: Apache Phoenix integration
> >>> >
> >>> > Hi all,
> >>> > looking at https://urldefense.proofpoint.
> com/v2/url?u=https-3A__drill.
> >>> > apache.org_docs_apache-2Ddrill-2Dcontribution-2Dideas_&d=DwIBaQ&c=
> >>> > cskdkSMqhcnjZxdQVpwTXg&r=-cT6otg6lpT_XkmYy7yg3A&m=daBPP_
> >>> > PaenGs721v8erkPwwO543kYzW-gKUv-c1jVWM&s=KMq_T0hogIOIv_
> >>> > ttrCLCxBeEt-AUKKxpcQt3PpeNSbs&e=
> >>> > it seems that Apache Phoenix is not integrated with Drill right
> >>> > now. In theory the integration should be quite straightforward
> >>> > using the JDBC Phoenix connector, isn't it?
> >>> > Right now there are just a few problem of class loading when
> >>> > putting
> >>> the
> >>> > phoenix jar into jars/3rdparty but it shouldn't be too hard to
> >>> > shade conflicting classes away.
> >>> >
> >>> > Has anyone attempted to connect to to Apache before?
> >>> >
> >>> > Best,
> >>> > Flavio
> >>> >
> >>>
> >>
> >>
> >>
> >> --
> >> Flavio Pompermaier
> >> Development Department
> >>
> >> OKKAM S.r.l.
> >> Tel. +(39) 0461 041809 <+39%200461%20041809>
> >>
> >
> >
> >
>
>
> --
> Flavio Pompermaier
> Development Department
>
> OKKAM S.r.l.
> Tel. +(39) 0461 041809
>
Re: Apache Phoenix integration
Posted by Flavio Pompermaier <po...@okkam.it>.
This is a very great news! But what do you mean exactly with "Apache Drill
is off of it's fork"? When is this deeper integration going to be
released/integrated?
Will Phoenix (eventually) become just a Drill storage plugin, with huge
benefits for both?
On 2 Feb 2018 20:20, "James Taylor" <ja...@apache.org> wrote:
> Good idea - a joint meetup would be great.
>
> On Fri, Feb 2, 2018 at 11:15 AM, Saurabh Mahapatra <
> saurabhmahapatra94@gmail.com> wrote:
>
>> This is absolutely great news! Thanks for sharing, James!
>>
>> We should have this as a presentation in one of our weekly Drill hangout
>> sessions.
>>
>> It’s about time for us to do a meetup. A joint meetup perhaps?
>>
>> Saurabh
>>
>> Sent from my iPhone
>>
>>
>>
>> > On Feb 2, 2018, at 11:13 AM, James Taylor <ja...@apache.org>
>> wrote:
>> >
>> > There's also a much deeper integration between Phoenix + Drill (code
>> named
>> > Drillix) underway that should be possible to complete now that Apache
>> Drill
>> > is off of it's fork and on a later version of Apache Calcite. I'm hoping
>> > that the output of this will be a Phoenix adapter in Drill. See
>> > presentation here[1] and WIP code here[2].
>> >
>> > Thanks,
>> > James
>> >
>> > [1] http://phoenix.apache.org/presentations/Drillix.pdf
>> > [2] https://github.com/jacques-n/drill/tree/phoenix_plugin
>> >
>> >> On Fri, Feb 2, 2018 at 10:21 AM, Kunal Khatua <kk...@mapr.com>
>> wrote:
>> >>
>> >> That's great, Flavio!
>> >>
>> >> You can create a Google doc for review and share it on the user list.
>> >>
>> >> @Bridget handles the documentation on the Apache website, so she can do
>> >> the final touches and help it find a home on the website.
>> >>
>> >> -----Original Message-----
>> >> From: Flavio Pompermaier [mailto:pompermaier@okkam.it]
>> >> Sent: Friday, February 02, 2018 9:04 AM
>> >> To: user@drill.apache.org
>> >> Cc: James Taylor <ja...@apache.org>
>> >> Subject: Re: Apache Phoenix integration
>> >>
>> >> Eventually I made it to integrate Phoenix with Drill! I debugged
>> remotely
>> >> the drill-embedded via Eclipse and I discovered that the problem was
>> that
>> >> you need some extra jars to make it work!
>> >> Where can I write some documentation about debugging remotely Drill
>> from
>> >> Eclipse and integration with Drill?
>> >>
>> >> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier <
>> pompermaier@okkam.it>
>> >> wrote:
>> >>
>> >>> What is the fastest way to debug the JDBC plugin from Eclipse? I don't
>> >>> see anything in the logs that could help...
>> >>> Is it possible to connect directly to the external embedded drill
>> >>> running on my machine if I enable jmx?
>> >>> it seems that the JDBC connection is established correctly but Drill
>> >>> throws an Exception (that is not well unwrapped by Jersey):
>> >>>
>> >>> 2018-02-02 16:54:04,520 [qtp159619134-56] INFO
>> >>> o.a.p.q.ConnectionQueryServicesImpl
>> >>> - HConnection established. Stacktrace for informational purposes:
>> >>> hconnection-0x1b9fe9f8
>> >>> java.lang.Thread.getStackTrace(Thread.java:1552)
>> >>> org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(
>> >>> ConnectionQueryServicesImpl.java:410)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(
>> >>> ConnectionQueryServicesImpl.java:256)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
>> >>> ConnectionQueryServicesImpl.java:2408)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
>> >>> ConnectionQueryServicesImpl.java:2384)
>> >>> org.apache.phoenix.util.PhoenixContextExecutor.call(
>> >>> PhoenixContextExecutor.java:76)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
>> >>> ConnectionQueryServicesImpl.java:2384)
>> >>> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(
>> >>> PhoenixDriver.java:255)
>> >>> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(
>> >>> PhoenixEmbeddedDriver.java:150)
>> >>> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
>> >>> org.apache.commons.dbcp.DriverConnectionFactory.createConnection(
>> >>> DriverConnectionFactory.java:38)
>> >>> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(
>> >>> PoolableConnectionFactory.java:582)
>> >>> org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(
>> >>> BasicDataSource.java:1556)
>> >>> org.apache.commons.dbcp.BasicDataSource.createPoolableConnec
>> tionFactor
>> >>> y(BasicDataSource.java:1545)
>> >>> org.apache.commons.dbcp.BasicDataSource.createDataSource(
>> >>> BasicDataSource.java:1388)
>> >>> org.apache.commons.dbcp.BasicDataSource.getConnection(
>> >>> BasicDataSource.java:1044)
>> >>> org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.
>> >>> get(JdbcUtils.java:73)
>> >>> org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(
>> >>> JdbcSchema.java:138)
>> >>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.<init>(
>> >>> JdbcStoragePlugin.java:103)
>> >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>> >>> sun.reflect.NativeConstructorAccessorImpl.newInstance(
>> >>> NativeConstructorAccessorImpl.java:62)
>> >>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
>> >>> DelegatingConstructorAccessorImpl.java:45)
>> >>> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>> >>> org.apache.drill.exec.store.StoragePluginRegistryImpl.create(
>> >>> StoragePluginRegistryImpl.java:346)
>> >>> org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(
>> >>> StoragePluginRegistryImpl.java:239)
>> >>> org.apache.drill.exec.server.rest.PluginConfigWrapper.
>> >>> createOrUpdateInStorage(PluginConfigWrapper.java:57)
>> >>> org.apache.drill.exec.server.rest.StorageResources.
>> >>> createOrUpdatePluginJSON(StorageResources.java:162)
>> >>> org.apache.drill.exec.server.rest.StorageResources.createOrU
>> pdatePlugi
>> >>> n(
>> >>> StorageResources.java:177)
>> >>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >>> sun.reflect.NativeMethodAccessorImpl.invoke(
>> >> NativeMethodAccessorImpl.java:
>> >>> 62)
>> >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> >>> DelegatingMethodAccessorImpl.java:43)
>> >>> java.lang.reflect.Method.invoke(Method.java:498)
>> >>> org.glassfish.jersey.server.model.internal.ResourceMethodInv
>> ocationHan
>> >>> dle
>> >>> rFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
>> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResou
>> rceMethodD
>> >>> isp
>> >>> atcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
>> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResou
>> rceMethodD
>> >>> isp
>> >>> atcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
>> >>> org.glassfish.jersey.server.model.internal.JavaResourceMetho
>> dDispatche
>> >>> rPr ovider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherPr
>> >>> ovider.java:195)
>> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResou
>> rceMethodD
>> >>> isp
>> >>> atcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104)
>> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
>> >>> invoke(ResourceMethodInvoker.java:387)
>> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
>> >>> apply(ResourceMethodInvoker.java:331)
>> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
>> >>> apply(ResourceMethodInvoker.java:103)
>> >>> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntim
>> e.java:269
>> >>> )
>> >>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
>> >>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
>> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:315)
>> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:297)
>> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:267)
>> >>> org.glassfish.jersey.process.internal.RequestScope.
>> >>> runInScope(RequestScope.java:297)
>> >>> org.glassfish.jersey.server.ServerRuntime.process(ServerRunt
>> ime.java:2
>> >>> 52) org.glassfish.jersey.server.ApplicationHandler.handle(
>> >>> ApplicationHandler.java:1023)
>> >>> org.glassfish.jersey.servlet.WebComponent.service(WebCompone
>> nt.java:37
>> >>> 2) org.glassfish.jersey.servlet.ServletContainer.service(
>> >>> ServletContainer.java:382)
>> >>> org.glassfish.jersey.servlet.ServletContainer.service(
>> >>> ServletContainer.java:345)
>> >>> org.glassfish.jersey.servlet.ServletContainer.service(
>> >>> ServletContainer.java:220)
>> >>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder
>> .java:738)
>> >>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHan
>> dler.java:
>> >>> 551) org.eclipse.jetty.server.session.SessionHandler.
>> >>> doHandle(SessionHandler.java:219)
>> >>> org.eclipse.jetty.server.handler.ContextHandler.
>> >>> doHandle(ContextHandler.java:1111)
>> >>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHand
>> ler.java:4
>> >>> 78) org.eclipse.jetty.server.session.SessionHandler.
>> >>> doScope(SessionHandler.java:183)
>> >>> org.eclipse.jetty.server.handler.ContextHandler.
>> >>> doScope(ContextHandler.java:1045)
>> >>> org.eclipse.jetty.server.handler.ScopedHandler.handle(
>> >>> ScopedHandler.java:141)
>> >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(
>> >>> HandlerWrapper.java:97)
>> >>> org.eclipse.jetty.server.Server.handle(Server.java:462)
>> >>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
>> >>> org.eclipse.jetty.server.HttpConnection.onFillable(
>> >>> HttpConnection.java:232)
>> >>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnec
>> tion.java:
>> >>> 534) org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
>> >>> QueuedThreadPool.java:607)
>> >>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(
>> >>> QueuedThreadPool.java:536)
>> >>> java.lang.Thread.run(Thread.java:745)
>> >>>
>> >>>
>> >>> On Fri, Feb 2, 2018 at 10:21 AM, Flavio Pompermaier
>> >>> <po...@okkam.it>
>> >>> wrote:
>> >>>
>> >>>> Hi Saurabh,
>> >>>> I'd like to use Drill because in my use case I have multiple sources
>> >>>> (HDFS, HBase, Phoenix, Oracle, Mysql) and I'd like to allow the user
>> >>>> to see the data.
>> >>>> Without writing a plugin for each storage system I'd like to leverage
>> >>>> Apache Drill as a broker towards all of them.
>> >>>>
>> >>>> Right now I've tried to puth the phoenix-core.jar into the
>> >>>> jar/3rdparty folder but when I try to create the Phoenix storage I
>> >>>> get the following
>> >>>> error:
>> >>>> Please retry: error (unable to create/ update storage)
>> >>>>
>> >>>> I'm trying to debug but I don't see anything wrong..
>> >>>>
>> >>>>
>> >>>> On Fri, Feb 2, 2018 at 1:42 AM, Saurabh Mahapatra <
>> >>>> saurabhmahapatra94@gmail.com> wrote:
>> >>>>
>> >>>>> Hi Flavio,
>> >>>>>
>> >>>>> I was curious what your use case is. Phoenix provides SQL access
>> >>>>> with secondary indexes on HBase. Why would you want to use Drill?
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Saurabh
>> >>>>>
>> >>>>>> On Thu, Feb 1, 2018 at 3:08 PM, Kunal Khatua <kk...@mapr.com>
>> wrote:
>> >>>>>>
>> >>>>>> The JDBC storage plugin allows Drill to leverage any SQL system
>> >>>>>> that
>> >>>>> has
>> >>>>>> JDBC drivers, so it should work.
>> >>>>>>
>> >>>>>> That said, the JDBC storage plugin is a community developed
>> >>>>>> storage plugin, so it might not be fully tested.
>> >>>>>>
>> >>>>>> If you are looking to simply have the Phoenix JDBC driver bundled
>> >>>>>> into
>> >>>>> the
>> >>>>>> Drill 3rd party directory, I (think) the reason it is not bundled
>> >>>>>> is because we're not actively tracking compatibility of different
>> >>>>>> JDBC
>> >>>>> drivers
>> >>>>>> with Drill. There could also be license issues, which is why we
>> >>>>>> can't bundle drivers for platforms like Oracle, etc.
>> >>>>>>
>> >>>>>> ~ Kunal
>> >>>>>>
>> >>>>>> -----Original Message-----
>> >>>>>> From: Flavio Pompermaier [mailto:pompermaier@okkam.it]
>> >>>>>> Sent: Thursday, February 01, 2018 9:39 AM
>> >>>>>> To: user@drill.apache.org
>> >>>>>> Subject: Apache Phoenix integration
>> >>>>>>
>> >>>>>> Hi all,
>> >>>>>> looking at https://urldefense.proofpoint.
>> >> com/v2/url?u=https-3A__drill.
>> >>>>>> apache.org_docs_apache-2Ddrill-2Dcontribution-2Dideas_&d=DwIBaQ&c=
>> >>>>>> cskdkSMqhcnjZxdQVpwTXg&r=-cT6otg6lpT_XkmYy7yg3A&m=daBPP_
>> >>>>>> PaenGs721v8erkPwwO543kYzW-gKUv-c1jVWM&s=KMq_T0hogIOIv_
>> >>>>>> ttrCLCxBeEt-AUKKxpcQt3PpeNSbs&e=
>> >>>>>> it seems that Apache Phoenix is not integrated with Drill right
>> >>>>>> now. In theory the integration should be quite straightforward
>> >>>>>> using the JDBC Phoenix connector, isn't it?
>> >>>>>> Right now there are just a few problem of class loading when
>> >>>>>> putting
>> >>>>> the
>> >>>>>> phoenix jar into jars/3rdparty but it shouldn't be too hard to
>> >>>>>> shade conflicting classes away.
>> >>>>>>
>> >>>>>> Has anyone attempted to connect to to Apache before?
>> >>>>>>
>> >>>>>> Best,
>> >>>>>> Flavio
>> >>>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Flavio Pompermaier
>> >>>> Development Department
>> >>>>
>> >>>> OKKAM S.r.l.
>> >>>> Tel. +(39) 0461 041809 <+39%200461%20041809> <+39%200461%20041809>
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >> Flavio Pompermaier
>> >> Development Department
>> >>
>> >> OKKAM S.r.l.
>> >> Tel. +(39) 0461 041809 <+39%200461%20041809>
>> >>
>>
>
>
Re: Apache Phoenix integration
Posted by Flavio Pompermaier <po...@okkam.it>.
This is a very great news! But what do you mean exactly with "Apache Drill
is off of it's fork"? When is this deeper integration going to be
released/integrated?
Will Phoenix (eventually) become just a Drill storage plugin, with huge
benefits for both?
On 2 Feb 2018 20:20, "James Taylor" <ja...@apache.org> wrote:
> Good idea - a joint meetup would be great.
>
> On Fri, Feb 2, 2018 at 11:15 AM, Saurabh Mahapatra <
> saurabhmahapatra94@gmail.com> wrote:
>
>> This is absolutely great news! Thanks for sharing, James!
>>
>> We should have this as a presentation in one of our weekly Drill hangout
>> sessions.
>>
>> It’s about time for us to do a meetup. A joint meetup perhaps?
>>
>> Saurabh
>>
>> Sent from my iPhone
>>
>>
>>
>> > On Feb 2, 2018, at 11:13 AM, James Taylor <ja...@apache.org>
>> wrote:
>> >
>> > There's also a much deeper integration between Phoenix + Drill (code
>> named
>> > Drillix) underway that should be possible to complete now that Apache
>> Drill
>> > is off of it's fork and on a later version of Apache Calcite. I'm hoping
>> > that the output of this will be a Phoenix adapter in Drill. See
>> > presentation here[1] and WIP code here[2].
>> >
>> > Thanks,
>> > James
>> >
>> > [1] http://phoenix.apache.org/presentations/Drillix.pdf
>> > [2] https://github.com/jacques-n/drill/tree/phoenix_plugin
>> >
>> >> On Fri, Feb 2, 2018 at 10:21 AM, Kunal Khatua <kk...@mapr.com>
>> wrote:
>> >>
>> >> That's great, Flavio!
>> >>
>> >> You can create a Google doc for review and share it on the user list.
>> >>
>> >> @Bridget handles the documentation on the Apache website, so she can do
>> >> the final touches and help it find a home on the website.
>> >>
>> >> -----Original Message-----
>> >> From: Flavio Pompermaier [mailto:pompermaier@okkam.it]
>> >> Sent: Friday, February 02, 2018 9:04 AM
>> >> To: user@drill.apache.org
>> >> Cc: James Taylor <ja...@apache.org>
>> >> Subject: Re: Apache Phoenix integration
>> >>
>> >> Eventually I made it to integrate Phoenix with Drill! I debugged
>> remotely
>> >> the drill-embedded via Eclipse and I discovered that the problem was
>> that
>> >> you need some extra jars to make it work!
>> >> Where can I write some documentation about debugging remotely Drill
>> from
>> >> Eclipse and integration with Drill?
>> >>
>> >> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier <
>> pompermaier@okkam.it>
>> >> wrote:
>> >>
>> >>> What is the fastest way to debug the JDBC plugin from Eclipse? I don't
>> >>> see anything in the logs that could help...
>> >>> Is it possible to connect directly to the external embedded drill
>> >>> running on my machine if I enable jmx?
>> >>> it seems that the JDBC connection is established correctly but Drill
>> >>> throws an Exception (that is not well unwrapped by Jersey):
>> >>>
>> >>> 2018-02-02 16:54:04,520 [qtp159619134-56] INFO
>> >>> o.a.p.q.ConnectionQueryServicesImpl
>> >>> - HConnection established. Stacktrace for informational purposes:
>> >>> hconnection-0x1b9fe9f8
>> >>> java.lang.Thread.getStackTrace(Thread.java:1552)
>> >>> org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(
>> >>> ConnectionQueryServicesImpl.java:410)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(
>> >>> ConnectionQueryServicesImpl.java:256)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
>> >>> ConnectionQueryServicesImpl.java:2408)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
>> >>> ConnectionQueryServicesImpl.java:2384)
>> >>> org.apache.phoenix.util.PhoenixContextExecutor.call(
>> >>> PhoenixContextExecutor.java:76)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
>> >>> ConnectionQueryServicesImpl.java:2384)
>> >>> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(
>> >>> PhoenixDriver.java:255)
>> >>> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(
>> >>> PhoenixEmbeddedDriver.java:150)
>> >>> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
>> >>> org.apache.commons.dbcp.DriverConnectionFactory.createConnection(
>> >>> DriverConnectionFactory.java:38)
>> >>> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(
>> >>> PoolableConnectionFactory.java:582)
>> >>> org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(
>> >>> BasicDataSource.java:1556)
>> >>> org.apache.commons.dbcp.BasicDataSource.createPoolableConnec
>> tionFactor
>> >>> y(BasicDataSource.java:1545)
>> >>> org.apache.commons.dbcp.BasicDataSource.createDataSource(
>> >>> BasicDataSource.java:1388)
>> >>> org.apache.commons.dbcp.BasicDataSource.getConnection(
>> >>> BasicDataSource.java:1044)
>> >>> org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.
>> >>> get(JdbcUtils.java:73)
>> >>> org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(
>> >>> JdbcSchema.java:138)
>> >>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.<init>(
>> >>> JdbcStoragePlugin.java:103)
>> >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>> >>> sun.reflect.NativeConstructorAccessorImpl.newInstance(
>> >>> NativeConstructorAccessorImpl.java:62)
>> >>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
>> >>> DelegatingConstructorAccessorImpl.java:45)
>> >>> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>> >>> org.apache.drill.exec.store.StoragePluginRegistryImpl.create(
>> >>> StoragePluginRegistryImpl.java:346)
>> >>> org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(
>> >>> StoragePluginRegistryImpl.java:239)
>> >>> org.apache.drill.exec.server.rest.PluginConfigWrapper.
>> >>> createOrUpdateInStorage(PluginConfigWrapper.java:57)
>> >>> org.apache.drill.exec.server.rest.StorageResources.
>> >>> createOrUpdatePluginJSON(StorageResources.java:162)
>> >>> org.apache.drill.exec.server.rest.StorageResources.createOrU
>> pdatePlugi
>> >>> n(
>> >>> StorageResources.java:177)
>> >>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >>> sun.reflect.NativeMethodAccessorImpl.invoke(
>> >> NativeMethodAccessorImpl.java:
>> >>> 62)
>> >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> >>> DelegatingMethodAccessorImpl.java:43)
>> >>> java.lang.reflect.Method.invoke(Method.java:498)
>> >>> org.glassfish.jersey.server.model.internal.ResourceMethodInv
>> ocationHan
>> >>> dle
>> >>> rFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
>> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResou
>> rceMethodD
>> >>> isp
>> >>> atcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
>> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResou
>> rceMethodD
>> >>> isp
>> >>> atcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
>> >>> org.glassfish.jersey.server.model.internal.JavaResourceMetho
>> dDispatche
>> >>> rPr ovider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherPr
>> >>> ovider.java:195)
>> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResou
>> rceMethodD
>> >>> isp
>> >>> atcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104)
>> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
>> >>> invoke(ResourceMethodInvoker.java:387)
>> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
>> >>> apply(ResourceMethodInvoker.java:331)
>> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
>> >>> apply(ResourceMethodInvoker.java:103)
>> >>> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntim
>> e.java:269
>> >>> )
>> >>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
>> >>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
>> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:315)
>> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:297)
>> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:267)
>> >>> org.glassfish.jersey.process.internal.RequestScope.
>> >>> runInScope(RequestScope.java:297)
>> >>> org.glassfish.jersey.server.ServerRuntime.process(ServerRunt
>> ime.java:2
>> >>> 52) org.glassfish.jersey.server.ApplicationHandler.handle(
>> >>> ApplicationHandler.java:1023)
>> >>> org.glassfish.jersey.servlet.WebComponent.service(WebCompone
>> nt.java:37
>> >>> 2) org.glassfish.jersey.servlet.ServletContainer.service(
>> >>> ServletContainer.java:382)
>> >>> org.glassfish.jersey.servlet.ServletContainer.service(
>> >>> ServletContainer.java:345)
>> >>> org.glassfish.jersey.servlet.ServletContainer.service(
>> >>> ServletContainer.java:220)
>> >>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder
>> .java:738)
>> >>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHan
>> dler.java:
>> >>> 551) org.eclipse.jetty.server.session.SessionHandler.
>> >>> doHandle(SessionHandler.java:219)
>> >>> org.eclipse.jetty.server.handler.ContextHandler.
>> >>> doHandle(ContextHandler.java:1111)
>> >>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHand
>> ler.java:4
>> >>> 78) org.eclipse.jetty.server.session.SessionHandler.
>> >>> doScope(SessionHandler.java:183)
>> >>> org.eclipse.jetty.server.handler.ContextHandler.
>> >>> doScope(ContextHandler.java:1045)
>> >>> org.eclipse.jetty.server.handler.ScopedHandler.handle(
>> >>> ScopedHandler.java:141)
>> >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(
>> >>> HandlerWrapper.java:97)
>> >>> org.eclipse.jetty.server.Server.handle(Server.java:462)
>> >>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
>> >>> org.eclipse.jetty.server.HttpConnection.onFillable(
>> >>> HttpConnection.java:232)
>> >>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnec
>> tion.java:
>> >>> 534) org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
>> >>> QueuedThreadPool.java:607)
>> >>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(
>> >>> QueuedThreadPool.java:536)
>> >>> java.lang.Thread.run(Thread.java:745)
>> >>>
>> >>>
>> >>> On Fri, Feb 2, 2018 at 10:21 AM, Flavio Pompermaier
>> >>> <po...@okkam.it>
>> >>> wrote:
>> >>>
>> >>>> Hi Saurabh,
>> >>>> I'd like to use Drill because in my use case I have multiple sources
>> >>>> (HDFS, HBase, Phoenix, Oracle, Mysql) and I'd like to allow the user
>> >>>> to see the data.
>> >>>> Without writing a plugin for each storage system I'd like to leverage
>> >>>> Apache Drill as a broker towards all of them.
>> >>>>
>> >>>> Right now I've tried to puth the phoenix-core.jar into the
>> >>>> jar/3rdparty folder but when I try to create the Phoenix storage I
>> >>>> get the following
>> >>>> error:
>> >>>> Please retry: error (unable to create/ update storage)
>> >>>>
>> >>>> I'm trying to debug but I don't see anything wrong..
>> >>>>
>> >>>>
>> >>>> On Fri, Feb 2, 2018 at 1:42 AM, Saurabh Mahapatra <
>> >>>> saurabhmahapatra94@gmail.com> wrote:
>> >>>>
>> >>>>> Hi Flavio,
>> >>>>>
>> >>>>> I was curious what your use case is. Phoenix provides SQL access
>> >>>>> with secondary indexes on HBase. Why would you want to use Drill?
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Saurabh
>> >>>>>
>> >>>>>> On Thu, Feb 1, 2018 at 3:08 PM, Kunal Khatua <kk...@mapr.com>
>> wrote:
>> >>>>>>
>> >>>>>> The JDBC storage plugin allows Drill to leverage any SQL system
>> >>>>>> that
>> >>>>> has
>> >>>>>> JDBC drivers, so it should work.
>> >>>>>>
>> >>>>>> That said, the JDBC storage plugin is a community developed
>> >>>>>> storage plugin, so it might not be fully tested.
>> >>>>>>
>> >>>>>> If you are looking to simply have the Phoenix JDBC driver bundled
>> >>>>>> into
>> >>>>> the
>> >>>>>> Drill 3rd party directory, I (think) the reason it is not bundled
>> >>>>>> is because we're not actively tracking compatibility of different
>> >>>>>> JDBC
>> >>>>> drivers
>> >>>>>> with Drill. There could also be license issues, which is why we
>> >>>>>> can't bundle drivers for platforms like Oracle, etc.
>> >>>>>>
>> >>>>>> ~ Kunal
>> >>>>>>
>> >>>>>> -----Original Message-----
>> >>>>>> From: Flavio Pompermaier [mailto:pompermaier@okkam.it]
>> >>>>>> Sent: Thursday, February 01, 2018 9:39 AM
>> >>>>>> To: user@drill.apache.org
>> >>>>>> Subject: Apache Phoenix integration
>> >>>>>>
>> >>>>>> Hi all,
>> >>>>>> looking at https://urldefense.proofpoint.
>> >> com/v2/url?u=https-3A__drill.
>> >>>>>> apache.org_docs_apache-2Ddrill-2Dcontribution-2Dideas_&d=DwIBaQ&c=
>> >>>>>> cskdkSMqhcnjZxdQVpwTXg&r=-cT6otg6lpT_XkmYy7yg3A&m=daBPP_
>> >>>>>> PaenGs721v8erkPwwO543kYzW-gKUv-c1jVWM&s=KMq_T0hogIOIv_
>> >>>>>> ttrCLCxBeEt-AUKKxpcQt3PpeNSbs&e=
>> >>>>>> it seems that Apache Phoenix is not integrated with Drill right
>> >>>>>> now. In theory the integration should be quite straightforward
>> >>>>>> using the JDBC Phoenix connector, isn't it?
>> >>>>>> Right now there are just a few problem of class loading when
>> >>>>>> putting
>> >>>>> the
>> >>>>>> phoenix jar into jars/3rdparty but it shouldn't be too hard to
>> >>>>>> shade conflicting classes away.
>> >>>>>>
>> >>>>>> Has anyone attempted to connect to to Apache before?
>> >>>>>>
>> >>>>>> Best,
>> >>>>>> Flavio
>> >>>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Flavio Pompermaier
>> >>>> Development Department
>> >>>>
>> >>>> OKKAM S.r.l.
>> >>>> Tel. +(39) 0461 041809 <+39%200461%20041809> <+39%200461%20041809>
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >> Flavio Pompermaier
>> >> Development Department
>> >>
>> >> OKKAM S.r.l.
>> >> Tel. +(39) 0461 041809 <+39%200461%20041809>
>> >>
>>
>
>
Re: Apache Phoenix integration
Posted by James Taylor <ja...@apache.org>.
Good idea - a joint meetup would be great.
On Fri, Feb 2, 2018 at 11:15 AM, Saurabh Mahapatra <
saurabhmahapatra94@gmail.com> wrote:
> This is absolutely great news! Thanks for sharing, James!
>
> We should have this as a presentation in one of our weekly Drill hangout
> sessions.
>
> It’s about time for us to do a meetup. A joint meetup perhaps?
>
> Saurabh
>
> Sent from my iPhone
>
>
>
> > On Feb 2, 2018, at 11:13 AM, James Taylor <ja...@apache.org>
> wrote:
> >
> > There's also a much deeper integration between Phoenix + Drill (code
> named
> > Drillix) underway that should be possible to complete now that Apache
> Drill
> > is off of it's fork and on a later version of Apache Calcite. I'm hoping
> > that the output of this will be a Phoenix adapter in Drill. See
> > presentation here[1] and WIP code here[2].
> >
> > Thanks,
> > James
> >
> > [1] http://phoenix.apache.org/presentations/Drillix.pdf
> > [2] https://github.com/jacques-n/drill/tree/phoenix_plugin
> >
> >> On Fri, Feb 2, 2018 at 10:21 AM, Kunal Khatua <kk...@mapr.com> wrote:
> >>
> >> That's great, Flavio!
> >>
> >> You can create a Google doc for review and share it on the user list.
> >>
> >> @Bridget handles the documentation on the Apache website, so she can do
> >> the final touches and help it find a home on the website.
> >>
> >> -----Original Message-----
> >> From: Flavio Pompermaier [mailto:pompermaier@okkam.it]
> >> Sent: Friday, February 02, 2018 9:04 AM
> >> To: user@drill.apache.org
> >> Cc: James Taylor <ja...@apache.org>
> >> Subject: Re: Apache Phoenix integration
> >>
> >> Eventually I made it to integrate Phoenix with Drill! I debugged
> remotely
> >> the drill-embedded via Eclipse and I discovered that the problem was
> that
> >> you need some extra jars to make it work!
> >> Where can I write some documentation about debugging remotely Drill from
> >> Eclipse and integration with Drill?
> >>
> >> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier <
> pompermaier@okkam.it>
> >> wrote:
> >>
> >>> What is the fastest way to debug the JDBC plugin from Eclipse? I don't
> >>> see anything in the logs that could help...
> >>> Is it possible to connect directly to the external embedded drill
> >>> running on my machine if I enable jmx?
> >>> it seems that the JDBC connection is established correctly but Drill
> >>> throws an Exception (that is not well unwrapped by Jersey):
> >>>
> >>> 2018-02-02 16:54:04,520 [qtp159619134-56] INFO
> >>> o.a.p.q.ConnectionQueryServicesImpl
> >>> - HConnection established. Stacktrace for informational purposes:
> >>> hconnection-0x1b9fe9f8
> >>> java.lang.Thread.getStackTrace(Thread.java:1552)
> >>> org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(
> >>> ConnectionQueryServicesImpl.java:410)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(
> >>> ConnectionQueryServicesImpl.java:256)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> >>> ConnectionQueryServicesImpl.java:2408)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> >>> ConnectionQueryServicesImpl.java:2384)
> >>> org.apache.phoenix.util.PhoenixContextExecutor.call(
> >>> PhoenixContextExecutor.java:76)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
> >>> ConnectionQueryServicesImpl.java:2384)
> >>> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(
> >>> PhoenixDriver.java:255)
> >>> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(
> >>> PhoenixEmbeddedDriver.java:150)
> >>> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
> >>> org.apache.commons.dbcp.DriverConnectionFactory.createConnection(
> >>> DriverConnectionFactory.java:38)
> >>> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(
> >>> PoolableConnectionFactory.java:582)
> >>> org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(
> >>> BasicDataSource.java:1556)
> >>> org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactor
> >>> y(BasicDataSource.java:1545)
> >>> org.apache.commons.dbcp.BasicDataSource.createDataSource(
> >>> BasicDataSource.java:1388)
> >>> org.apache.commons.dbcp.BasicDataSource.getConnection(
> >>> BasicDataSource.java:1044)
> >>> org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.
> >>> get(JdbcUtils.java:73)
> >>> org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(
> >>> JdbcSchema.java:138)
> >>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.<init>(
> >>> JdbcStoragePlugin.java:103)
> >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> >>> sun.reflect.NativeConstructorAccessorImpl.newInstance(
> >>> NativeConstructorAccessorImpl.java:62)
> >>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> >>> DelegatingConstructorAccessorImpl.java:45)
> >>> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> >>> org.apache.drill.exec.store.StoragePluginRegistryImpl.create(
> >>> StoragePluginRegistryImpl.java:346)
> >>> org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(
> >>> StoragePluginRegistryImpl.java:239)
> >>> org.apache.drill.exec.server.rest.PluginConfigWrapper.
> >>> createOrUpdateInStorage(PluginConfigWrapper.java:57)
> >>> org.apache.drill.exec.server.rest.StorageResources.
> >>> createOrUpdatePluginJSON(StorageResources.java:162)
> >>> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlugi
> >>> n(
> >>> StorageResources.java:177)
> >>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>> sun.reflect.NativeMethodAccessorImpl.invoke(
> >> NativeMethodAccessorImpl.java:
> >>> 62)
> >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> >>> DelegatingMethodAccessorImpl.java:43)
> >>> java.lang.reflect.Method.invoke(Method.java:498)
> >>> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHan
> >>> dle
> >>> rFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD
> >>> isp
> >>> atcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD
> >>> isp
> >>> atcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
> >>> org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatche
> >>> rPr ovider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherPr
> >>> ovider.java:195)
> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD
> >>> isp
> >>> atcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104)
> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
> >>> invoke(ResourceMethodInvoker.java:387)
> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
> >>> apply(ResourceMethodInvoker.java:331)
> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
> >>> apply(ResourceMethodInvoker.java:103)
> >>> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269
> >>> )
> >>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
> >>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:315)
> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:297)
> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:267)
> >>> org.glassfish.jersey.process.internal.RequestScope.
> >>> runInScope(RequestScope.java:297)
> >>> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:2
> >>> 52) org.glassfish.jersey.server.ApplicationHandler.handle(
> >>> ApplicationHandler.java:1023)
> >>> org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:37
> >>> 2) org.glassfish.jersey.servlet.ServletContainer.service(
> >>> ServletContainer.java:382)
> >>> org.glassfish.jersey.servlet.ServletContainer.service(
> >>> ServletContainer.java:345)
> >>> org.glassfish.jersey.servlet.ServletContainer.service(
> >>> ServletContainer.java:220)
> >>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)
> >>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:
> >>> 551) org.eclipse.jetty.server.session.SessionHandler.
> >>> doHandle(SessionHandler.java:219)
> >>> org.eclipse.jetty.server.handler.ContextHandler.
> >>> doHandle(ContextHandler.java:1111)
> >>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:4
> >>> 78) org.eclipse.jetty.server.session.SessionHandler.
> >>> doScope(SessionHandler.java:183)
> >>> org.eclipse.jetty.server.handler.ContextHandler.
> >>> doScope(ContextHandler.java:1045)
> >>> org.eclipse.jetty.server.handler.ScopedHandler.handle(
> >>> ScopedHandler.java:141)
> >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> >>> HandlerWrapper.java:97)
> >>> org.eclipse.jetty.server.Server.handle(Server.java:462)
> >>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
> >>> org.eclipse.jetty.server.HttpConnection.onFillable(
> >>> HttpConnection.java:232)
> >>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:
> >>> 534) org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> >>> QueuedThreadPool.java:607)
> >>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(
> >>> QueuedThreadPool.java:536)
> >>> java.lang.Thread.run(Thread.java:745)
> >>>
> >>>
> >>> On Fri, Feb 2, 2018 at 10:21 AM, Flavio Pompermaier
> >>> <po...@okkam.it>
> >>> wrote:
> >>>
> >>>> Hi Saurabh,
> >>>> I'd like to use Drill because in my use case I have multiple sources
> >>>> (HDFS, HBase, Phoenix, Oracle, Mysql) and I'd like to allow the user
> >>>> to see the data.
> >>>> Without writing a plugin for each storage system I'd like to leverage
> >>>> Apache Drill as a broker towards all of them.
> >>>>
> >>>> Right now I've tried to puth the phoenix-core.jar into the
> >>>> jar/3rdparty folder but when I try to create the Phoenix storage I
> >>>> get the following
> >>>> error:
> >>>> Please retry: error (unable to create/ update storage)
> >>>>
> >>>> I'm trying to debug but I don't see anything wrong..
> >>>>
> >>>>
> >>>> On Fri, Feb 2, 2018 at 1:42 AM, Saurabh Mahapatra <
> >>>> saurabhmahapatra94@gmail.com> wrote:
> >>>>
> >>>>> Hi Flavio,
> >>>>>
> >>>>> I was curious what your use case is. Phoenix provides SQL access
> >>>>> with secondary indexes on HBase. Why would you want to use Drill?
> >>>>>
> >>>>> Thanks,
> >>>>> Saurabh
> >>>>>
> >>>>>> On Thu, Feb 1, 2018 at 3:08 PM, Kunal Khatua <kk...@mapr.com>
> wrote:
> >>>>>>
> >>>>>> The JDBC storage plugin allows Drill to leverage any SQL system
> >>>>>> that
> >>>>> has
> >>>>>> JDBC drivers, so it should work.
> >>>>>>
> >>>>>> That said, the JDBC storage plugin is a community developed
> >>>>>> storage plugin, so it might not be fully tested.
> >>>>>>
> >>>>>> If you are looking to simply have the Phoenix JDBC driver bundled
> >>>>>> into
> >>>>> the
> >>>>>> Drill 3rd party directory, I (think) the reason it is not bundled
> >>>>>> is because we're not actively tracking compatibility of different
> >>>>>> JDBC
> >>>>> drivers
> >>>>>> with Drill. There could also be license issues, which is why we
> >>>>>> can't bundle drivers for platforms like Oracle, etc.
> >>>>>>
> >>>>>> ~ Kunal
> >>>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Flavio Pompermaier [mailto:pompermaier@okkam.it]
> >>>>>> Sent: Thursday, February 01, 2018 9:39 AM
> >>>>>> To: user@drill.apache.org
> >>>>>> Subject: Apache Phoenix integration
> >>>>>>
> >>>>>> Hi all,
> >>>>>> looking at https://urldefense.proofpoint.
> >> com/v2/url?u=https-3A__drill.
> >>>>>> apache.org_docs_apache-2Ddrill-2Dcontribution-2Dideas_&d=DwIBaQ&c=
> >>>>>> cskdkSMqhcnjZxdQVpwTXg&r=-cT6otg6lpT_XkmYy7yg3A&m=daBPP_
> >>>>>> PaenGs721v8erkPwwO543kYzW-gKUv-c1jVWM&s=KMq_T0hogIOIv_
> >>>>>> ttrCLCxBeEt-AUKKxpcQt3PpeNSbs&e=
> >>>>>> it seems that Apache Phoenix is not integrated with Drill right
> >>>>>> now. In theory the integration should be quite straightforward
> >>>>>> using the JDBC Phoenix connector, isn't it?
> >>>>>> Right now there are just a few problem of class loading when
> >>>>>> putting
> >>>>> the
> >>>>>> phoenix jar into jars/3rdparty but it shouldn't be too hard to
> >>>>>> shade conflicting classes away.
> >>>>>>
> >>>>>> Has anyone attempted to connect to to Apache before?
> >>>>>>
> >>>>>> Best,
> >>>>>> Flavio
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Flavio Pompermaier
> >>>> Development Department
> >>>>
> >>>> OKKAM S.r.l.
> >>>> Tel. +(39) 0461 041809 <+39%200461%20041809>
> >>>>
> >>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Flavio Pompermaier
> >> Development Department
> >>
> >> OKKAM S.r.l.
> >> Tel. +(39) 0461 041809
> >>
>
Re: Apache Phoenix integration
Posted by James Taylor <ja...@apache.org>.
Good idea - a joint meetup would be great.
On Fri, Feb 2, 2018 at 11:15 AM, Saurabh Mahapatra <
saurabhmahapatra94@gmail.com> wrote:
> This is absolutely great news! Thanks for sharing, James!
>
> We should have this as a presentation in one of our weekly Drill hangout
> sessions.
>
> It’s about time for us to do a meetup. A joint meetup perhaps?
>
> Saurabh
>
> Sent from my iPhone
>
>
>
> > On Feb 2, 2018, at 11:13 AM, James Taylor <ja...@apache.org>
> wrote:
> >
> > There's also a much deeper integration between Phoenix + Drill (code
> named
> > Drillix) underway that should be possible to complete now that Apache
> Drill
> > is off of it's fork and on a later version of Apache Calcite. I'm hoping
> > that the output of this will be a Phoenix adapter in Drill. See
> > presentation here[1] and WIP code here[2].
> >
> > Thanks,
> > James
> >
> > [1] http://phoenix.apache.org/presentations/Drillix.pdf
> > [2] https://github.com/jacques-n/drill/tree/phoenix_plugin
> >
> >> On Fri, Feb 2, 2018 at 10:21 AM, Kunal Khatua <kk...@mapr.com> wrote:
> >>
> >> That's great, Flavio!
> >>
> >> You can create a Google doc for review and share it on the user list.
> >>
> >> @Bridget handles the documentation on the Apache website, so she can do
> >> the final touches and help it find a home on the website.
> >>
> >> -----Original Message-----
> >> From: Flavio Pompermaier [mailto:pompermaier@okkam.it]
> >> Sent: Friday, February 02, 2018 9:04 AM
> >> To: user@drill.apache.org
> >> Cc: James Taylor <ja...@apache.org>
> >> Subject: Re: Apache Phoenix integration
> >>
> >> Eventually I made it to integrate Phoenix with Drill! I debugged
> remotely
> >> the drill-embedded via Eclipse and I discovered that the problem was
> that
> >> you need some extra jars to make it work!
> >> Where can I write some documentation about debugging remotely Drill from
> >> Eclipse and integration with Drill?
> >>
> >> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier <
> pompermaier@okkam.it>
> >> wrote:
> >>
> >>> What is the fastest way to debug the JDBC plugin from Eclipse? I don't
> >>> see anything in the logs that could help...
> >>> Is it possible to connect directly to the external embedded drill
> >>> running on my machine if I enable jmx?
> >>> it seems that the JDBC connection is established correctly but Drill
> >>> throws an Exception (that is not well unwrapped by Jersey):
> >>>
> >>> 2018-02-02 16:54:04,520 [qtp159619134-56] INFO
> >>> o.a.p.q.ConnectionQueryServicesImpl
> >>> - HConnection established. Stacktrace for informational purposes:
> >>> hconnection-0x1b9fe9f8
> >>> java.lang.Thread.getStackTrace(Thread.java:1552)
> >>> org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(
> >>> ConnectionQueryServicesImpl.java:410)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(
> >>> ConnectionQueryServicesImpl.java:256)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> >>> ConnectionQueryServicesImpl.java:2408)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> >>> ConnectionQueryServicesImpl.java:2384)
> >>> org.apache.phoenix.util.PhoenixContextExecutor.call(
> >>> PhoenixContextExecutor.java:76)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
> >>> ConnectionQueryServicesImpl.java:2384)
> >>> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(
> >>> PhoenixDriver.java:255)
> >>> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(
> >>> PhoenixEmbeddedDriver.java:150)
> >>> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
> >>> org.apache.commons.dbcp.DriverConnectionFactory.createConnection(
> >>> DriverConnectionFactory.java:38)
> >>> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(
> >>> PoolableConnectionFactory.java:582)
> >>> org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(
> >>> BasicDataSource.java:1556)
> >>> org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactor
> >>> y(BasicDataSource.java:1545)
> >>> org.apache.commons.dbcp.BasicDataSource.createDataSource(
> >>> BasicDataSource.java:1388)
> >>> org.apache.commons.dbcp.BasicDataSource.getConnection(
> >>> BasicDataSource.java:1044)
> >>> org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.
> >>> get(JdbcUtils.java:73)
> >>> org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(
> >>> JdbcSchema.java:138)
> >>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.<init>(
> >>> JdbcStoragePlugin.java:103)
> >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> >>> sun.reflect.NativeConstructorAccessorImpl.newInstance(
> >>> NativeConstructorAccessorImpl.java:62)
> >>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> >>> DelegatingConstructorAccessorImpl.java:45)
> >>> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> >>> org.apache.drill.exec.store.StoragePluginRegistryImpl.create(
> >>> StoragePluginRegistryImpl.java:346)
> >>> org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(
> >>> StoragePluginRegistryImpl.java:239)
> >>> org.apache.drill.exec.server.rest.PluginConfigWrapper.
> >>> createOrUpdateInStorage(PluginConfigWrapper.java:57)
> >>> org.apache.drill.exec.server.rest.StorageResources.
> >>> createOrUpdatePluginJSON(StorageResources.java:162)
> >>> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlugi
> >>> n(
> >>> StorageResources.java:177)
> >>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>> sun.reflect.NativeMethodAccessorImpl.invoke(
> >> NativeMethodAccessorImpl.java:
> >>> 62)
> >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> >>> DelegatingMethodAccessorImpl.java:43)
> >>> java.lang.reflect.Method.invoke(Method.java:498)
> >>> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHan
> >>> dle
> >>> rFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD
> >>> isp
> >>> atcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD
> >>> isp
> >>> atcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
> >>> org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatche
> >>> rPr ovider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherPr
> >>> ovider.java:195)
> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD
> >>> isp
> >>> atcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104)
> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
> >>> invoke(ResourceMethodInvoker.java:387)
> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
> >>> apply(ResourceMethodInvoker.java:331)
> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
> >>> apply(ResourceMethodInvoker.java:103)
> >>> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269
> >>> )
> >>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
> >>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:315)
> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:297)
> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:267)
> >>> org.glassfish.jersey.process.internal.RequestScope.
> >>> runInScope(RequestScope.java:297)
> >>> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:2
> >>> 52) org.glassfish.jersey.server.ApplicationHandler.handle(
> >>> ApplicationHandler.java:1023)
> >>> org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:37
> >>> 2) org.glassfish.jersey.servlet.ServletContainer.service(
> >>> ServletContainer.java:382)
> >>> org.glassfish.jersey.servlet.ServletContainer.service(
> >>> ServletContainer.java:345)
> >>> org.glassfish.jersey.servlet.ServletContainer.service(
> >>> ServletContainer.java:220)
> >>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)
> >>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:
> >>> 551) org.eclipse.jetty.server.session.SessionHandler.
> >>> doHandle(SessionHandler.java:219)
> >>> org.eclipse.jetty.server.handler.ContextHandler.
> >>> doHandle(ContextHandler.java:1111)
> >>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:4
> >>> 78) org.eclipse.jetty.server.session.SessionHandler.
> >>> doScope(SessionHandler.java:183)
> >>> org.eclipse.jetty.server.handler.ContextHandler.
> >>> doScope(ContextHandler.java:1045)
> >>> org.eclipse.jetty.server.handler.ScopedHandler.handle(
> >>> ScopedHandler.java:141)
> >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> >>> HandlerWrapper.java:97)
> >>> org.eclipse.jetty.server.Server.handle(Server.java:462)
> >>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
> >>> org.eclipse.jetty.server.HttpConnection.onFillable(
> >>> HttpConnection.java:232)
> >>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:
> >>> 534) org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> >>> QueuedThreadPool.java:607)
> >>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(
> >>> QueuedThreadPool.java:536)
> >>> java.lang.Thread.run(Thread.java:745)
> >>>
> >>>
> >>> On Fri, Feb 2, 2018 at 10:21 AM, Flavio Pompermaier
> >>> <po...@okkam.it>
> >>> wrote:
> >>>
> >>>> Hi Saurabh,
> >>>> I'd like to use Drill because in my use case I have multiple sources
> >>>> (HDFS, HBase, Phoenix, Oracle, Mysql) and I'd like to allow the user
> >>>> to see the data.
> >>>> Without writing a plugin for each storage system I'd like to leverage
> >>>> Apache Drill as a broker towards all of them.
> >>>>
> >>>> Right now I've tried to puth the phoenix-core.jar into the
> >>>> jar/3rdparty folder but when I try to create the Phoenix storage I
> >>>> get the following
> >>>> error:
> >>>> Please retry: error (unable to create/ update storage)
> >>>>
> >>>> I'm trying to debug but I don't see anything wrong..
> >>>>
> >>>>
> >>>> On Fri, Feb 2, 2018 at 1:42 AM, Saurabh Mahapatra <
> >>>> saurabhmahapatra94@gmail.com> wrote:
> >>>>
> >>>>> Hi Flavio,
> >>>>>
> >>>>> I was curious what your use case is. Phoenix provides SQL access
> >>>>> with secondary indexes on HBase. Why would you want to use Drill?
> >>>>>
> >>>>> Thanks,
> >>>>> Saurabh
> >>>>>
> >>>>>> On Thu, Feb 1, 2018 at 3:08 PM, Kunal Khatua <kk...@mapr.com>
> wrote:
> >>>>>>
> >>>>>> The JDBC storage plugin allows Drill to leverage any SQL system
> >>>>>> that
> >>>>> has
> >>>>>> JDBC drivers, so it should work.
> >>>>>>
> >>>>>> That said, the JDBC storage plugin is a community developed
> >>>>>> storage plugin, so it might not be fully tested.
> >>>>>>
> >>>>>> If you are looking to simply have the Phoenix JDBC driver bundled
> >>>>>> into
> >>>>> the
> >>>>>> Drill 3rd party directory, I (think) the reason it is not bundled
> >>>>>> is because we're not actively tracking compatibility of different
> >>>>>> JDBC
> >>>>> drivers
> >>>>>> with Drill. There could also be license issues, which is why we
> >>>>>> can't bundle drivers for platforms like Oracle, etc.
> >>>>>>
> >>>>>> ~ Kunal
> >>>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Flavio Pompermaier [mailto:pompermaier@okkam.it]
> >>>>>> Sent: Thursday, February 01, 2018 9:39 AM
> >>>>>> To: user@drill.apache.org
> >>>>>> Subject: Apache Phoenix integration
> >>>>>>
> >>>>>> Hi all,
> >>>>>> looking at https://urldefense.proofpoint.
> >> com/v2/url?u=https-3A__drill.
> >>>>>> apache.org_docs_apache-2Ddrill-2Dcontribution-2Dideas_&d=DwIBaQ&c=
> >>>>>> cskdkSMqhcnjZxdQVpwTXg&r=-cT6otg6lpT_XkmYy7yg3A&m=daBPP_
> >>>>>> PaenGs721v8erkPwwO543kYzW-gKUv-c1jVWM&s=KMq_T0hogIOIv_
> >>>>>> ttrCLCxBeEt-AUKKxpcQt3PpeNSbs&e=
> >>>>>> it seems that Apache Phoenix is not integrated with Drill right
> >>>>>> now. In theory the integration should be quite straightforward
> >>>>>> using the JDBC Phoenix connector, isn't it?
> >>>>>> Right now there are just a few problem of class loading when
> >>>>>> putting
> >>>>> the
> >>>>>> phoenix jar into jars/3rdparty but it shouldn't be too hard to
> >>>>>> shade conflicting classes away.
> >>>>>>
> >>>>>> Has anyone attempted to connect to to Apache before?
> >>>>>>
> >>>>>> Best,
> >>>>>> Flavio
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Flavio Pompermaier
> >>>> Development Department
> >>>>
> >>>> OKKAM S.r.l.
> >>>> Tel. +(39) 0461 041809 <+39%200461%20041809>
> >>>>
> >>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Flavio Pompermaier
> >> Development Department
> >>
> >> OKKAM S.r.l.
> >> Tel. +(39) 0461 041809
> >>
>
Re: Apache Phoenix integration
Posted by Saurabh Mahapatra <sa...@gmail.com>.
This is absolutely great news! Thanks for sharing, James!
We should have this as a presentation in one of our weekly Drill hangout sessions.
It’s about time for us to do a meetup. A joint meetup perhaps?
Saurabh
Sent from my iPhone
> On Feb 2, 2018, at 11:13 AM, James Taylor <ja...@apache.org> wrote:
>
> There's also a much deeper integration between Phoenix + Drill (code named
> Drillix) underway that should be possible to complete now that Apache Drill
> is off of it's fork and on a later version of Apache Calcite. I'm hoping
> that the output of this will be a Phoenix adapter in Drill. See
> presentation here[1] and WIP code here[2].
>
> Thanks,
> James
>
> [1] http://phoenix.apache.org/presentations/Drillix.pdf
> [2] https://github.com/jacques-n/drill/tree/phoenix_plugin
>
>> On Fri, Feb 2, 2018 at 10:21 AM, Kunal Khatua <kk...@mapr.com> wrote:
>>
>> That's great, Flavio!
>>
>> You can create a Google doc for review and share it on the user list.
>>
>> @Bridget handles the documentation on the Apache website, so she can do
>> the final touches and help it find a home on the website.
>>
>> -----Original Message-----
>> From: Flavio Pompermaier [mailto:pompermaier@okkam.it]
>> Sent: Friday, February 02, 2018 9:04 AM
>> To: user@drill.apache.org
>> Cc: James Taylor <ja...@apache.org>
>> Subject: Re: Apache Phoenix integration
>>
>> Eventually I made it to integrate Phoenix with Drill! I debugged remotely
>> the drill-embedded via Eclipse and I discovered that the problem was that
>> you need some extra jars to make it work!
>> Where can I write some documentation about debugging remotely Drill from
>> Eclipse and integration with Drill?
>>
>> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier <po...@okkam.it>
>> wrote:
>>
>>> What is the fastest way to debug the JDBC plugin from Eclipse? I don't
>>> see anything in the logs that could help...
>>> Is it possible to connect directly to the external embedded drill
>>> running on my machine if I enable jmx?
>>> it seems that the JDBC connection is established correctly but Drill
>>> throws an Exception (that is not well unwrapped by Jersey):
>>>
>>> 2018-02-02 16:54:04,520 [qtp159619134-56] INFO
>>> o.a.p.q.ConnectionQueryServicesImpl
>>> - HConnection established. Stacktrace for informational purposes:
>>> hconnection-0x1b9fe9f8
>>> java.lang.Thread.getStackTrace(Thread.java:1552)
>>> org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
>>> org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(
>>> ConnectionQueryServicesImpl.java:410)
>>> org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(
>>> ConnectionQueryServicesImpl.java:256)
>>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
>>> ConnectionQueryServicesImpl.java:2408)
>>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
>>> ConnectionQueryServicesImpl.java:2384)
>>> org.apache.phoenix.util.PhoenixContextExecutor.call(
>>> PhoenixContextExecutor.java:76)
>>> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
>>> ConnectionQueryServicesImpl.java:2384)
>>> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(
>>> PhoenixDriver.java:255)
>>> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(
>>> PhoenixEmbeddedDriver.java:150)
>>> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
>>> org.apache.commons.dbcp.DriverConnectionFactory.createConnection(
>>> DriverConnectionFactory.java:38)
>>> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(
>>> PoolableConnectionFactory.java:582)
>>> org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(
>>> BasicDataSource.java:1556)
>>> org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactor
>>> y(BasicDataSource.java:1545)
>>> org.apache.commons.dbcp.BasicDataSource.createDataSource(
>>> BasicDataSource.java:1388)
>>> org.apache.commons.dbcp.BasicDataSource.getConnection(
>>> BasicDataSource.java:1044)
>>> org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.
>>> get(JdbcUtils.java:73)
>>> org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(
>>> JdbcSchema.java:138)
>>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.<init>(
>>> JdbcStoragePlugin.java:103)
>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(
>>> NativeConstructorAccessorImpl.java:62)
>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
>>> DelegatingConstructorAccessorImpl.java:45)
>>> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>> org.apache.drill.exec.store.StoragePluginRegistryImpl.create(
>>> StoragePluginRegistryImpl.java:346)
>>> org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(
>>> StoragePluginRegistryImpl.java:239)
>>> org.apache.drill.exec.server.rest.PluginConfigWrapper.
>>> createOrUpdateInStorage(PluginConfigWrapper.java:57)
>>> org.apache.drill.exec.server.rest.StorageResources.
>>> createOrUpdatePluginJSON(StorageResources.java:162)
>>> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlugi
>>> n(
>>> StorageResources.java:177)
>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>> NativeMethodAccessorImpl.java:
>>> 62)
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>> DelegatingMethodAccessorImpl.java:43)
>>> java.lang.reflect.Method.invoke(Method.java:498)
>>> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHan
>>> dle
>>> rFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
>>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD
>>> isp
>>> atcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
>>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD
>>> isp
>>> atcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
>>> org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatche
>>> rPr ovider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherPr
>>> ovider.java:195)
>>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD
>>> isp
>>> atcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104)
>>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
>>> invoke(ResourceMethodInvoker.java:387)
>>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
>>> apply(ResourceMethodInvoker.java:331)
>>> org.glassfish.jersey.server.model.ResourceMethodInvoker.
>>> apply(ResourceMethodInvoker.java:103)
>>> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269
>>> )
>>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
>>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
>>> org.glassfish.jersey.internal.Errors.process(Errors.java:315)
>>> org.glassfish.jersey.internal.Errors.process(Errors.java:297)
>>> org.glassfish.jersey.internal.Errors.process(Errors.java:267)
>>> org.glassfish.jersey.process.internal.RequestScope.
>>> runInScope(RequestScope.java:297)
>>> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:2
>>> 52) org.glassfish.jersey.server.ApplicationHandler.handle(
>>> ApplicationHandler.java:1023)
>>> org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:37
>>> 2) org.glassfish.jersey.servlet.ServletContainer.service(
>>> ServletContainer.java:382)
>>> org.glassfish.jersey.servlet.ServletContainer.service(
>>> ServletContainer.java:345)
>>> org.glassfish.jersey.servlet.ServletContainer.service(
>>> ServletContainer.java:220)
>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)
>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:
>>> 551) org.eclipse.jetty.server.session.SessionHandler.
>>> doHandle(SessionHandler.java:219)
>>> org.eclipse.jetty.server.handler.ContextHandler.
>>> doHandle(ContextHandler.java:1111)
>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:4
>>> 78) org.eclipse.jetty.server.session.SessionHandler.
>>> doScope(SessionHandler.java:183)
>>> org.eclipse.jetty.server.handler.ContextHandler.
>>> doScope(ContextHandler.java:1045)
>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(
>>> ScopedHandler.java:141)
>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(
>>> HandlerWrapper.java:97)
>>> org.eclipse.jetty.server.Server.handle(Server.java:462)
>>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
>>> org.eclipse.jetty.server.HttpConnection.onFillable(
>>> HttpConnection.java:232)
>>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:
>>> 534) org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
>>> QueuedThreadPool.java:607)
>>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(
>>> QueuedThreadPool.java:536)
>>> java.lang.Thread.run(Thread.java:745)
>>>
>>>
>>> On Fri, Feb 2, 2018 at 10:21 AM, Flavio Pompermaier
>>> <po...@okkam.it>
>>> wrote:
>>>
>>>> Hi Saurabh,
>>>> I'd like to use Drill because in my use case I have multiple sources
>>>> (HDFS, HBase, Phoenix, Oracle, Mysql) and I'd like to allow the user
>>>> to see the data.
>>>> Without writing a plugin for each storage system I'd like to leverage
>>>> Apache Drill as a broker towards all of them.
>>>>
>>>> Right now I've tried to puth the phoenix-core.jar into the
>>>> jar/3rdparty folder but when I try to create the Phoenix storage I
>>>> get the following
>>>> error:
>>>> Please retry: error (unable to create/ update storage)
>>>>
>>>> I'm trying to debug but I don't see anything wrong..
>>>>
>>>>
>>>> On Fri, Feb 2, 2018 at 1:42 AM, Saurabh Mahapatra <
>>>> saurabhmahapatra94@gmail.com> wrote:
>>>>
>>>>> Hi Flavio,
>>>>>
>>>>> I was curious what your use case is. Phoenix provides SQL access
>>>>> with secondary indexes on HBase. Why would you want to use Drill?
>>>>>
>>>>> Thanks,
>>>>> Saurabh
>>>>>
>>>>>> On Thu, Feb 1, 2018 at 3:08 PM, Kunal Khatua <kk...@mapr.com> wrote:
>>>>>>
>>>>>> The JDBC storage plugin allows Drill to leverage any SQL system
>>>>>> that
>>>>> has
>>>>>> JDBC drivers, so it should work.
>>>>>>
>>>>>> That said, the JDBC storage plugin is a community developed
>>>>>> storage plugin, so it might not be fully tested.
>>>>>>
>>>>>> If you are looking to simply have the Phoenix JDBC driver bundled
>>>>>> into
>>>>> the
>>>>>> Drill 3rd party directory, I (think) the reason it is not bundled
>>>>>> is because we're not actively tracking compatibility of different
>>>>>> JDBC
>>>>> drivers
>>>>>> with Drill. There could also be license issues, which is why we
>>>>>> can't bundle drivers for platforms like Oracle, etc.
>>>>>>
>>>>>> ~ Kunal
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Flavio Pompermaier [mailto:pompermaier@okkam.it]
>>>>>> Sent: Thursday, February 01, 2018 9:39 AM
>>>>>> To: user@drill.apache.org
>>>>>> Subject: Apache Phoenix integration
>>>>>>
>>>>>> Hi all,
>>>>>> looking at https://urldefense.proofpoint.
>> com/v2/url?u=https-3A__drill.
>>>>>> apache.org_docs_apache-2Ddrill-2Dcontribution-2Dideas_&d=DwIBaQ&c=
>>>>>> cskdkSMqhcnjZxdQVpwTXg&r=-cT6otg6lpT_XkmYy7yg3A&m=daBPP_
>>>>>> PaenGs721v8erkPwwO543kYzW-gKUv-c1jVWM&s=KMq_T0hogIOIv_
>>>>>> ttrCLCxBeEt-AUKKxpcQt3PpeNSbs&e=
>>>>>> it seems that Apache Phoenix is not integrated with Drill right
>>>>>> now. In theory the integration should be quite straightforward
>>>>>> using the JDBC Phoenix connector, isn't it?
>>>>>> Right now there are just a few problem of class loading when
>>>>>> putting
>>>>> the
>>>>>> phoenix jar into jars/3rdparty but it shouldn't be too hard to
>>>>>> shade conflicting classes away.
>>>>>>
>>>>>> Has anyone attempted to connect to to Apache before?
>>>>>>
>>>>>> Best,
>>>>>> Flavio
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Flavio Pompermaier
>>>> Development Department
>>>>
>>>> OKKAM S.r.l.
>>>> Tel. +(39) 0461 041809 <+39%200461%20041809>
>>>>
>>>
>>>
>>>
>>
>>
>> --
>> Flavio Pompermaier
>> Development Department
>>
>> OKKAM S.r.l.
>> Tel. +(39) 0461 041809
>>