You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Peder Jakobsen | gmail <pj...@gmail.com> on 2015/12/31 18:32:53 UTC
MS Windows: Hadoop binaries required to run drill?
Hi, does hadoop have to be installed in order to run apache drill on
windows?
So far, I'm 3 hours into "Run Apache Drill in 10 Minutes", and still no
luck. Once I figure it out, I will update the documentation for windows if
you like.
Thanks,
Peder J.
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Kristine Hahn <kh...@maprtech.com>.
oops, sorry, I now see you prefaced the FROM with cp. Forget that.
Kristine Hahn
Sr. Technical Writer
415-497-8107 @krishahn skype:krishahn
On Thu, Dec 31, 2015 at 2:52 PM, Kristine Hahn <kh...@maprtech.com> wrote:
> Please try USE cp; before the select statement. Drill uses the dfs storage
> plugin, not the cp storage plugin, by default.
>
> Kristine Hahn
> Sr. Technical Writer
> 415-497-8107 @krishahn skype:krishahn
>
>
> On Thu, Dec 31, 2015 at 2:50 PM, Peder Jakobsen | gmail <
> pjakobsen@gmail.com> wrote:
>
>> I spoke too soon, perhaps (but it must be a simple I/O issue on startup,
>> no?)
>>
>>
>> C:\devel\apache-drill-1.4.0\bin>sqlline.bat -u "jdbc:drill:zk=local"
>> DRILL_ARGS - " -u jdbc:drill:zk=local"
>> HADOOP_HOME not detected...
>> HBASE_HOME not detected...
>> Calculating Drill classpath...
>> Dec 31, 2015 5:39:53 PM org.glassfish.jersey.server.ApplicationHandler
>> initialize
>> INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29
>> 01:25:26...
>> apache drill 1.4.0
>> "just drill it"
>> 0: jdbc:drill:zk=local> SELECT * FROM cp.`employee.json` LIMIT 3;
>> Dec 31, 2015 5:46:55 PM
>> org.apache.calcite.sql.validate.SqlValidatorException <init>
>> SEVERE: org.apache.calcite.sql.validate.SqlValidatorException: Table
>> 'cp.employee.json' not found
>> Dec 31, 2015 5:46:55 PM org.apache.calcite.runtime.CalciteException <init>
>> SEVERE: org.apache.calcite.runtime.CalciteContextException: From line 1,
>> column 15 to line 1, column 16: Table 'cp.employee.json' not found
>> Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 16:
>> Table
>> 'cp.employee.json' not found
>>
>>
>> [Error Id: 9d6af232-fc40-40ec-8a2a-8d082923b776 on Funky:31010]
>> (state=,code=0)
>>
>> On Thu, Dec 31, 2015 at 5:34 PM, Peder Jakobsen | gmail <
>> pjakobsen@gmail.com
>> > wrote:
>>
>> > OUCH that hurts. Sometimes you get lucky when you jump in a haystack
>> and
>> > get pricked by the missing needle right away.
>> >
>> > So I copied hive.sys.drill from my tmp folder in linux to windows, and
>> > wow, now it seems to work. Would you allow me the honour to fix this
>> bug
>> > and commit the changes even though I have not programmed in Java since
>> 2003
>> > ? ;) I can't imagine that I/O in java has changed all that much.
>> >
>> > Peder :)
>> >
>> >
>> >
>> > On Thu, Dec 31, 2015 at 5:30 PM, Peder Jakobsen | gmail <
>> > pjakobsen@gmail.com> wrote:
>> >
>> >> No. I've tried this many times.
>> >>
>> >> So at startup, it's supposed to create these files which it needs?
>> >>
>> >> It seems to do so in my linux install. I will try to copy those files
>> >> over and see if I get another error. Needle in a haystack debugging.
>> :)
>> >>
>> >> P.
>> >>
>> >> On Thu, Dec 31, 2015 at 5:27 PM, Nathan Griffith <ngriffith@dremio.com
>> >
>> >> wrote:
>> >>
>> >>> Ah! Okay. I remember it making temporary stuff *somewhere* within
>> >>> windows.
>> >>>
>> >>> If I recall correctly, a bad set of temporary files once gave me the
>> >>> exact same issue, which was fixed by deleting them. But apparently
>> >>> this is isnt helping with your case?
>> >>>
>> >>> On Thu, Dec 31, 2015 at 2:22 PM, Peder Jakobsen | gmail
>> >>> <pj...@gmail.com> wrote:
>> >>> > OK, at startup, Drill creates an empty file called hive.sys.drill
>> >>> that's
>> >>> > located in C:\tmp\drill\sys.storage_plugins
>> >>> >
>> >>> > Perhaps it's not surprising that we get "Unable to deserialize
>> >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill" (state=,code=0)"
>> >>> > considering that this file appears to be empty.
>> >>> >
>> >>> > On Linux, lots of stuff is included in this drill path: profiles,
>> >>> > sys.options & sys.storage_plugins
>> >>> >
>> >>> > Hope this helps
>> >>> >
>> >>> > P.
>> >>> >
>> >>> > On Thu, Dec 31, 2015 at 5:13 PM, Peder Jakobsen | gmail <
>> >>> pjakobsen@gmail.com
>> >>> >> wrote:
>> >>> >
>> >>> >> I deleted everything in C:\Windows\Temp. Note, when I start drill
>> >>> >> again, nothing gets created in there. I will now try to search
>> the
>> >>> >> machine for any files that may have been created at the time of
>> >>> startup.
>> >>> >> Standy by..
>> >>> >>
>> >>> >> On Thu, Dec 31, 2015 at 5:10 PM, Nathan Griffith <
>> >>> ngriffith@dremio.com>
>> >>> >> wrote:
>> >>> >>
>> >>> >>> Hey Peder,
>> >>> >>>
>> >>> >>> What's in your C:\Windows\Temp directory? Is there something that
>> >>> >>> looks like it's from Drill? If there is, try deleting it and
>> running
>> >>> >>> again.
>> >>> >>>
>> >>> >>> --Nathan
>> >>> >>>
>> >>> >>> On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
>> >>> >>> <pj...@gmail.com> wrote:
>> >>> >>> > Hi Tomer and Jacques,
>> >>> >>> >
>> >>> >>> > I tried those instructions to the letter. I still get the same
>> >>> java
>> >>> >>> > Runtime Exception "Uable to deserialize
>> >>> >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill"
>> >>> >>> >
>> >>> >>> > Let me know if you need the full stack trace, although this
>> should
>> >>> be
>> >>> >>> easy
>> >>> >>> > to replicate. I simply installed a clean version of Windows 7
>> on
>> >>> >>> > VirtualBox, and followed all instructions from Drill
>> documentation
>> >>> and
>> >>> >>> > Tomers suggestions. I can't imagine why it would work on any
>> >>> other
>> >>> >>> > windows OS unless some other tweak is done somewhere.
>> >>> >>> >
>> >>> >>> > P.
>> >>> >>> >
>> >>> >>> >
>> >>> >>> > On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <
>> >>> jacques@dremio.com>
>> >>> >>> wrote:
>> >>> >>> >
>> >>> >>> >> Just to clarify, Drill doesn't require Hadoop to be installed.
>> >>> Drill
>> >>> >>> does
>> >>> >>> >> include some Hadoop code since it uses some Hadoop
>> functionality
>> >>> under
>> >>> >>> the
>> >>> >>> >> hood as a library (even in non-hadoop installs). Drill should
>> be
>> >>> >>> including
>> >>> >>> >> the required windows executables/dlls in the distribution
>> folder
>> >>> (for
>> >>> >>> using
>> >>> >>> >> hadoop as a library) and this should be transparent to you. Let
>> >>> us know
>> >>> >>> >> what happens when you follow the link that Tomer suggested.
>> >>> >>> >>
>> >>> >>> >> --
>> >>> >>> >> Jacques Nadeau
>> >>> >>> >> CTO and Co-Founder, Dremio
>> >>> >>> >>
>> >>> >>> >> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <
>> >>> tshiran@dremio.com>
>> >>> >>> wrote:
>> >>> >>> >>
>> >>> >>> >> > No. Drill does not depend on Hadoop
>> >>> >>> >> >
>> >>> >>> >> > Can you try these instructions and see if they work:
>> >>> >>> >> >
>> >>> >>> >> >
>> >>> >>>
>> >>>
>> http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
>> >>> >>> >> >
>> >>> >>> >> >
>> >>> >>> >> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
>> >>> >>> >> > pjakobsen@gmail.com> wrote:
>> >>> >>> >> > >
>> >>> >>> >> > > Hi, does hadoop have to be installed in order to run apache
>> >>> drill
>> >>> >>> on
>> >>> >>> >> > > windows?
>> >>> >>> >> > >
>> >>> >>> >> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes",
>> and
>> >>> >>> still no
>> >>> >>> >> > > luck. Once I figure it out, I will update the
>> documentation
>> >>> for
>> >>> >>> >> windows
>> >>> >>> >> > if
>> >>> >>> >> > > you like.
>> >>> >>> >> > >
>> >>> >>> >> > > Thanks,
>> >>> >>> >> > >
>> >>> >>> >> > > Peder J.
>> >>> >>> >> >
>> >>> >>> >>
>> >>> >>>
>> >>> >>
>> >>> >>
>> >>>
>> >>
>> >>
>> >
>>
>
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Kristine Hahn <kh...@maprtech.com>.
Please try USE cp; before the select statement. Drill uses the dfs storage
plugin, not the cp storage plugin, by default.
Kristine Hahn
Sr. Technical Writer
415-497-8107 @krishahn skype:krishahn
On Thu, Dec 31, 2015 at 2:50 PM, Peder Jakobsen | gmail <pjakobsen@gmail.com
> wrote:
> I spoke too soon, perhaps (but it must be a simple I/O issue on startup,
> no?)
>
>
> C:\devel\apache-drill-1.4.0\bin>sqlline.bat -u "jdbc:drill:zk=local"
> DRILL_ARGS - " -u jdbc:drill:zk=local"
> HADOOP_HOME not detected...
> HBASE_HOME not detected...
> Calculating Drill classpath...
> Dec 31, 2015 5:39:53 PM org.glassfish.jersey.server.ApplicationHandler
> initialize
> INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29
> 01:25:26...
> apache drill 1.4.0
> "just drill it"
> 0: jdbc:drill:zk=local> SELECT * FROM cp.`employee.json` LIMIT 3;
> Dec 31, 2015 5:46:55 PM
> org.apache.calcite.sql.validate.SqlValidatorException <init>
> SEVERE: org.apache.calcite.sql.validate.SqlValidatorException: Table
> 'cp.employee.json' not found
> Dec 31, 2015 5:46:55 PM org.apache.calcite.runtime.CalciteException <init>
> SEVERE: org.apache.calcite.runtime.CalciteContextException: From line 1,
> column 15 to line 1, column 16: Table 'cp.employee.json' not found
> Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 16: Table
> 'cp.employee.json' not found
>
>
> [Error Id: 9d6af232-fc40-40ec-8a2a-8d082923b776 on Funky:31010]
> (state=,code=0)
>
> On Thu, Dec 31, 2015 at 5:34 PM, Peder Jakobsen | gmail <
> pjakobsen@gmail.com
> > wrote:
>
> > OUCH that hurts. Sometimes you get lucky when you jump in a haystack and
> > get pricked by the missing needle right away.
> >
> > So I copied hive.sys.drill from my tmp folder in linux to windows, and
> > wow, now it seems to work. Would you allow me the honour to fix this
> bug
> > and commit the changes even though I have not programmed in Java since
> 2003
> > ? ;) I can't imagine that I/O in java has changed all that much.
> >
> > Peder :)
> >
> >
> >
> > On Thu, Dec 31, 2015 at 5:30 PM, Peder Jakobsen | gmail <
> > pjakobsen@gmail.com> wrote:
> >
> >> No. I've tried this many times.
> >>
> >> So at startup, it's supposed to create these files which it needs?
> >>
> >> It seems to do so in my linux install. I will try to copy those files
> >> over and see if I get another error. Needle in a haystack debugging.
> :)
> >>
> >> P.
> >>
> >> On Thu, Dec 31, 2015 at 5:27 PM, Nathan Griffith <ng...@dremio.com>
> >> wrote:
> >>
> >>> Ah! Okay. I remember it making temporary stuff *somewhere* within
> >>> windows.
> >>>
> >>> If I recall correctly, a bad set of temporary files once gave me the
> >>> exact same issue, which was fixed by deleting them. But apparently
> >>> this is isnt helping with your case?
> >>>
> >>> On Thu, Dec 31, 2015 at 2:22 PM, Peder Jakobsen | gmail
> >>> <pj...@gmail.com> wrote:
> >>> > OK, at startup, Drill creates an empty file called hive.sys.drill
> >>> that's
> >>> > located in C:\tmp\drill\sys.storage_plugins
> >>> >
> >>> > Perhaps it's not surprising that we get "Unable to deserialize
> >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill" (state=,code=0)"
> >>> > considering that this file appears to be empty.
> >>> >
> >>> > On Linux, lots of stuff is included in this drill path: profiles,
> >>> > sys.options & sys.storage_plugins
> >>> >
> >>> > Hope this helps
> >>> >
> >>> > P.
> >>> >
> >>> > On Thu, Dec 31, 2015 at 5:13 PM, Peder Jakobsen | gmail <
> >>> pjakobsen@gmail.com
> >>> >> wrote:
> >>> >
> >>> >> I deleted everything in C:\Windows\Temp. Note, when I start drill
> >>> >> again, nothing gets created in there. I will now try to search the
> >>> >> machine for any files that may have been created at the time of
> >>> startup.
> >>> >> Standy by..
> >>> >>
> >>> >> On Thu, Dec 31, 2015 at 5:10 PM, Nathan Griffith <
> >>> ngriffith@dremio.com>
> >>> >> wrote:
> >>> >>
> >>> >>> Hey Peder,
> >>> >>>
> >>> >>> What's in your C:\Windows\Temp directory? Is there something that
> >>> >>> looks like it's from Drill? If there is, try deleting it and
> running
> >>> >>> again.
> >>> >>>
> >>> >>> --Nathan
> >>> >>>
> >>> >>> On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
> >>> >>> <pj...@gmail.com> wrote:
> >>> >>> > Hi Tomer and Jacques,
> >>> >>> >
> >>> >>> > I tried those instructions to the letter. I still get the same
> >>> java
> >>> >>> > Runtime Exception "Uable to deserialize
> >>> >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill"
> >>> >>> >
> >>> >>> > Let me know if you need the full stack trace, although this
> should
> >>> be
> >>> >>> easy
> >>> >>> > to replicate. I simply installed a clean version of Windows 7 on
> >>> >>> > VirtualBox, and followed all instructions from Drill
> documentation
> >>> and
> >>> >>> > Tomers suggestions. I can't imagine why it would work on any
> >>> other
> >>> >>> > windows OS unless some other tweak is done somewhere.
> >>> >>> >
> >>> >>> > P.
> >>> >>> >
> >>> >>> >
> >>> >>> > On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <
> >>> jacques@dremio.com>
> >>> >>> wrote:
> >>> >>> >
> >>> >>> >> Just to clarify, Drill doesn't require Hadoop to be installed.
> >>> Drill
> >>> >>> does
> >>> >>> >> include some Hadoop code since it uses some Hadoop functionality
> >>> under
> >>> >>> the
> >>> >>> >> hood as a library (even in non-hadoop installs). Drill should be
> >>> >>> including
> >>> >>> >> the required windows executables/dlls in the distribution folder
> >>> (for
> >>> >>> using
> >>> >>> >> hadoop as a library) and this should be transparent to you. Let
> >>> us know
> >>> >>> >> what happens when you follow the link that Tomer suggested.
> >>> >>> >>
> >>> >>> >> --
> >>> >>> >> Jacques Nadeau
> >>> >>> >> CTO and Co-Founder, Dremio
> >>> >>> >>
> >>> >>> >> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <
> >>> tshiran@dremio.com>
> >>> >>> wrote:
> >>> >>> >>
> >>> >>> >> > No. Drill does not depend on Hadoop
> >>> >>> >> >
> >>> >>> >> > Can you try these instructions and see if they work:
> >>> >>> >> >
> >>> >>> >> >
> >>> >>>
> >>>
> http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
> >>> >>> >> >
> >>> >>> >> >
> >>> >>> >> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
> >>> >>> >> > pjakobsen@gmail.com> wrote:
> >>> >>> >> > >
> >>> >>> >> > > Hi, does hadoop have to be installed in order to run apache
> >>> drill
> >>> >>> on
> >>> >>> >> > > windows?
> >>> >>> >> > >
> >>> >>> >> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes",
> and
> >>> >>> still no
> >>> >>> >> > > luck. Once I figure it out, I will update the documentation
> >>> for
> >>> >>> >> windows
> >>> >>> >> > if
> >>> >>> >> > > you like.
> >>> >>> >> > >
> >>> >>> >> > > Thanks,
> >>> >>> >> > >
> >>> >>> >> > > Peder J.
> >>> >>> >> >
> >>> >>> >>
> >>> >>>
> >>> >>
> >>> >>
> >>>
> >>
> >>
> >
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Peder Jakobsen | gmail <pj...@gmail.com>.
Hi Jacques, Jason, etc.
Problems on windows all resolves. Drill runs like a dream. Gotcha was due
to documentation that implies Drill can run on 32bit Windows, which is not
possible.
Suggest compiling winutils.exe as 32bit, although this a completely
uneducated stab at what seems to a win win. If not, I will update
documentation to reflect this reality, and issue a pull request.
I have never used Jira, but I will try to make a ticket there first.
Thank you again for all your help. It's encouraging to see such a strong
community around this fantastic tool.
Peder :)
On Thu, Dec 31, 2015 at 6:16 PM, Jacques Nadeau <ja...@dremio.com> wrote:
> When Drill first starts up (or encounters an empty folder for embedded
> data), it will automatically create a number of new storage plugins. This
> includes cp and dfs, as well as disabled ones for hive, hbase, etc. It
> seems like your Drill is failing to do this. Since you only copied one
> plugin into your tmp directory (the hive plugin), you will not be able to
> query from the cp plugin. You would need to copy all the default plugins to
> get this working. This potentially provides a workaround but it doesn't
> indicate why Drill is failing to initialize these settings.
>
> Can you possibly try this under the Administrator account if one exists?
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Thu, Dec 31, 2015 at 2:50 PM, Peder Jakobsen | gmail <
> pjakobsen@gmail.com
> > wrote:
>
> > I spoke too soon, perhaps (but it must be a simple I/O issue on startup,
> > no?)
> >
> >
> > C:\devel\apache-drill-1.4.0\bin>sqlline.bat -u "jdbc:drill:zk=local"
> > DRILL_ARGS - " -u jdbc:drill:zk=local"
> > HADOOP_HOME not detected...
> > HBASE_HOME not detected...
> > Calculating Drill classpath...
> > Dec 31, 2015 5:39:53 PM org.glassfish.jersey.server.ApplicationHandler
> > initialize
> > INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29
> > 01:25:26...
> > apache drill 1.4.0
> > "just drill it"
> > 0: jdbc:drill:zk=local> SELECT * FROM cp.`employee.json` LIMIT 3;
> > Dec 31, 2015 5:46:55 PM
> > org.apache.calcite.sql.validate.SqlValidatorException <init>
> > SEVERE: org.apache.calcite.sql.validate.SqlValidatorException: Table
> > 'cp.employee.json' not found
> > Dec 31, 2015 5:46:55 PM org.apache.calcite.runtime.CalciteException
> <init>
> > SEVERE: org.apache.calcite.runtime.CalciteContextException: From line 1,
> > column 15 to line 1, column 16: Table 'cp.employee.json' not found
> > Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 16:
> Table
> > 'cp.employee.json' not found
> >
> >
> > [Error Id: 9d6af232-fc40-40ec-8a2a-8d082923b776 on Funky:31010]
> > (state=,code=0)
> >
> > On Thu, Dec 31, 2015 at 5:34 PM, Peder Jakobsen | gmail <
> > pjakobsen@gmail.com
> > > wrote:
> >
> > > OUCH that hurts. Sometimes you get lucky when you jump in a haystack
> and
> > > get pricked by the missing needle right away.
> > >
> > > So I copied hive.sys.drill from my tmp folder in linux to windows, and
> > > wow, now it seems to work. Would you allow me the honour to fix this
> > bug
> > > and commit the changes even though I have not programmed in Java since
> > 2003
> > > ? ;) I can't imagine that I/O in java has changed all that much.
> > >
> > > Peder :)
> > >
> > >
> > >
> > > On Thu, Dec 31, 2015 at 5:30 PM, Peder Jakobsen | gmail <
> > > pjakobsen@gmail.com> wrote:
> > >
> > >> No. I've tried this many times.
> > >>
> > >> So at startup, it's supposed to create these files which it needs?
> > >>
> > >> It seems to do so in my linux install. I will try to copy those
> files
> > >> over and see if I get another error. Needle in a haystack debugging.
> > :)
> > >>
> > >> P.
> > >>
> > >> On Thu, Dec 31, 2015 at 5:27 PM, Nathan Griffith <
> ngriffith@dremio.com>
> > >> wrote:
> > >>
> > >>> Ah! Okay. I remember it making temporary stuff *somewhere* within
> > >>> windows.
> > >>>
> > >>> If I recall correctly, a bad set of temporary files once gave me the
> > >>> exact same issue, which was fixed by deleting them. But apparently
> > >>> this is isnt helping with your case?
> > >>>
> > >>> On Thu, Dec 31, 2015 at 2:22 PM, Peder Jakobsen | gmail
> > >>> <pj...@gmail.com> wrote:
> > >>> > OK, at startup, Drill creates an empty file called hive.sys.drill
> > >>> that's
> > >>> > located in C:\tmp\drill\sys.storage_plugins
> > >>> >
> > >>> > Perhaps it's not surprising that we get "Unable to deserialize
> > >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill" (state=,code=0)"
> > >>> > considering that this file appears to be empty.
> > >>> >
> > >>> > On Linux, lots of stuff is included in this drill path: profiles,
> > >>> > sys.options & sys.storage_plugins
> > >>> >
> > >>> > Hope this helps
> > >>> >
> > >>> > P.
> > >>> >
> > >>> > On Thu, Dec 31, 2015 at 5:13 PM, Peder Jakobsen | gmail <
> > >>> pjakobsen@gmail.com
> > >>> >> wrote:
> > >>> >
> > >>> >> I deleted everything in C:\Windows\Temp. Note, when I start
> drill
> > >>> >> again, nothing gets created in there. I will now try to search
> the
> > >>> >> machine for any files that may have been created at the time of
> > >>> startup.
> > >>> >> Standy by..
> > >>> >>
> > >>> >> On Thu, Dec 31, 2015 at 5:10 PM, Nathan Griffith <
> > >>> ngriffith@dremio.com>
> > >>> >> wrote:
> > >>> >>
> > >>> >>> Hey Peder,
> > >>> >>>
> > >>> >>> What's in your C:\Windows\Temp directory? Is there something that
> > >>> >>> looks like it's from Drill? If there is, try deleting it and
> > running
> > >>> >>> again.
> > >>> >>>
> > >>> >>> --Nathan
> > >>> >>>
> > >>> >>> On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
> > >>> >>> <pj...@gmail.com> wrote:
> > >>> >>> > Hi Tomer and Jacques,
> > >>> >>> >
> > >>> >>> > I tried those instructions to the letter. I still get the
> same
> > >>> java
> > >>> >>> > Runtime Exception "Uable to deserialize
> > >>> >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill"
> > >>> >>> >
> > >>> >>> > Let me know if you need the full stack trace, although this
> > should
> > >>> be
> > >>> >>> easy
> > >>> >>> > to replicate. I simply installed a clean version of Windows 7
> on
> > >>> >>> > VirtualBox, and followed all instructions from Drill
> > documentation
> > >>> and
> > >>> >>> > Tomers suggestions. I can't imagine why it would work on any
> > >>> other
> > >>> >>> > windows OS unless some other tweak is done somewhere.
> > >>> >>> >
> > >>> >>> > P.
> > >>> >>> >
> > >>> >>> >
> > >>> >>> > On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <
> > >>> jacques@dremio.com>
> > >>> >>> wrote:
> > >>> >>> >
> > >>> >>> >> Just to clarify, Drill doesn't require Hadoop to be installed.
> > >>> Drill
> > >>> >>> does
> > >>> >>> >> include some Hadoop code since it uses some Hadoop
> functionality
> > >>> under
> > >>> >>> the
> > >>> >>> >> hood as a library (even in non-hadoop installs). Drill should
> be
> > >>> >>> including
> > >>> >>> >> the required windows executables/dlls in the distribution
> folder
> > >>> (for
> > >>> >>> using
> > >>> >>> >> hadoop as a library) and this should be transparent to you.
> Let
> > >>> us know
> > >>> >>> >> what happens when you follow the link that Tomer suggested.
> > >>> >>> >>
> > >>> >>> >> --
> > >>> >>> >> Jacques Nadeau
> > >>> >>> >> CTO and Co-Founder, Dremio
> > >>> >>> >>
> > >>> >>> >> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <
> > >>> tshiran@dremio.com>
> > >>> >>> wrote:
> > >>> >>> >>
> > >>> >>> >> > No. Drill does not depend on Hadoop
> > >>> >>> >> >
> > >>> >>> >> > Can you try these instructions and see if they work:
> > >>> >>> >> >
> > >>> >>> >> >
> > >>> >>>
> > >>>
> > http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
> > >>> >>> >> >
> > >>> >>> >> >
> > >>> >>> >> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
> > >>> >>> >> > pjakobsen@gmail.com> wrote:
> > >>> >>> >> > >
> > >>> >>> >> > > Hi, does hadoop have to be installed in order to run
> apache
> > >>> drill
> > >>> >>> on
> > >>> >>> >> > > windows?
> > >>> >>> >> > >
> > >>> >>> >> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes",
> > and
> > >>> >>> still no
> > >>> >>> >> > > luck. Once I figure it out, I will update the
> documentation
> > >>> for
> > >>> >>> >> windows
> > >>> >>> >> > if
> > >>> >>> >> > > you like.
> > >>> >>> >> > >
> > >>> >>> >> > > Thanks,
> > >>> >>> >> > >
> > >>> >>> >> > > Peder J.
> > >>> >>> >> >
> > >>> >>> >>
> > >>> >>>
> > >>> >>
> > >>> >>
> > >>>
> > >>
> > >>
> > >
> >
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Jacques Nadeau <ja...@dremio.com>.
Indeed. Happy new year
On Dec 31, 2015 4:07 PM, "Peder Jakobsen | gmail" <pj...@gmail.com>
wrote:
> Hi Jacques, I will try it tomorrow. It's time have some Veuve Clicquot.
> :)
>
> Thank you for all your help today. Happy New Year to everyone.
>
> Peder J.
>
> On Thu, Dec 31, 2015 at 6:16 PM, Jacques Nadeau <ja...@dremio.com>
> wrote:
>
> > When Drill first starts up (or encounters an empty folder for embedded
> > data), it will automatically create a number of new storage plugins. This
> > includes cp and dfs, as well as disabled ones for hive, hbase, etc. It
> > seems like your Drill is failing to do this. Since you only copied one
> > plugin into your tmp directory (the hive plugin), you will not be able to
> > query from the cp plugin. You would need to copy all the default plugins
> to
> > get this working. This potentially provides a workaround but it doesn't
> > indicate why Drill is failing to initialize these settings.
> >
> > Can you possibly try this under the Administrator account if one exists?
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Thu, Dec 31, 2015 at 2:50 PM, Peder Jakobsen | gmail <
> > pjakobsen@gmail.com
> > > wrote:
> >
> > > I spoke too soon, perhaps (but it must be a simple I/O issue on
> startup,
> > > no?)
> > >
> > >
> > > C:\devel\apache-drill-1.4.0\bin>sqlline.bat -u "jdbc:drill:zk=local"
> > > DRILL_ARGS - " -u jdbc:drill:zk=local"
> > > HADOOP_HOME not detected...
> > > HBASE_HOME not detected...
> > > Calculating Drill classpath...
> > > Dec 31, 2015 5:39:53 PM org.glassfish.jersey.server.ApplicationHandler
> > > initialize
> > > INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29
> > > 01:25:26...
> > > apache drill 1.4.0
> > > "just drill it"
> > > 0: jdbc:drill:zk=local> SELECT * FROM cp.`employee.json` LIMIT 3;
> > > Dec 31, 2015 5:46:55 PM
> > > org.apache.calcite.sql.validate.SqlValidatorException <init>
> > > SEVERE: org.apache.calcite.sql.validate.SqlValidatorException: Table
> > > 'cp.employee.json' not found
> > > Dec 31, 2015 5:46:55 PM org.apache.calcite.runtime.CalciteException
> > <init>
> > > SEVERE: org.apache.calcite.runtime.CalciteContextException: From line
> 1,
> > > column 15 to line 1, column 16: Table 'cp.employee.json' not found
> > > Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 16:
> > Table
> > > 'cp.employee.json' not found
> > >
> > >
> > > [Error Id: 9d6af232-fc40-40ec-8a2a-8d082923b776 on Funky:31010]
> > > (state=,code=0)
> > >
> > > On Thu, Dec 31, 2015 at 5:34 PM, Peder Jakobsen | gmail <
> > > pjakobsen@gmail.com
> > > > wrote:
> > >
> > > > OUCH that hurts. Sometimes you get lucky when you jump in a haystack
> > and
> > > > get pricked by the missing needle right away.
> > > >
> > > > So I copied hive.sys.drill from my tmp folder in linux to windows,
> and
> > > > wow, now it seems to work. Would you allow me the honour to fix
> this
> > > bug
> > > > and commit the changes even though I have not programmed in Java
> since
> > > 2003
> > > > ? ;) I can't imagine that I/O in java has changed all that much.
> > > >
> > > > Peder :)
> > > >
> > > >
> > > >
> > > > On Thu, Dec 31, 2015 at 5:30 PM, Peder Jakobsen | gmail <
> > > > pjakobsen@gmail.com> wrote:
> > > >
> > > >> No. I've tried this many times.
> > > >>
> > > >> So at startup, it's supposed to create these files which it needs?
> > > >>
> > > >> It seems to do so in my linux install. I will try to copy those
> > files
> > > >> over and see if I get another error. Needle in a haystack
> debugging.
> > > :)
> > > >>
> > > >> P.
> > > >>
> > > >> On Thu, Dec 31, 2015 at 5:27 PM, Nathan Griffith <
> > ngriffith@dremio.com>
> > > >> wrote:
> > > >>
> > > >>> Ah! Okay. I remember it making temporary stuff *somewhere* within
> > > >>> windows.
> > > >>>
> > > >>> If I recall correctly, a bad set of temporary files once gave me
> the
> > > >>> exact same issue, which was fixed by deleting them. But apparently
> > > >>> this is isnt helping with your case?
> > > >>>
> > > >>> On Thu, Dec 31, 2015 at 2:22 PM, Peder Jakobsen | gmail
> > > >>> <pj...@gmail.com> wrote:
> > > >>> > OK, at startup, Drill creates an empty file called
> hive.sys.drill
> > > >>> that's
> > > >>> > located in C:\tmp\drill\sys.storage_plugins
> > > >>> >
> > > >>> > Perhaps it's not surprising that we get "Unable to deserialize
> > > >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill" (state=,code=0)"
> > > >>> > considering that this file appears to be empty.
> > > >>> >
> > > >>> > On Linux, lots of stuff is included in this drill path:
> profiles,
> > > >>> > sys.options & sys.storage_plugins
> > > >>> >
> > > >>> > Hope this helps
> > > >>> >
> > > >>> > P.
> > > >>> >
> > > >>> > On Thu, Dec 31, 2015 at 5:13 PM, Peder Jakobsen | gmail <
> > > >>> pjakobsen@gmail.com
> > > >>> >> wrote:
> > > >>> >
> > > >>> >> I deleted everything in C:\Windows\Temp. Note, when I start
> > drill
> > > >>> >> again, nothing gets created in there. I will now try to search
> > the
> > > >>> >> machine for any files that may have been created at the time of
> > > >>> startup.
> > > >>> >> Standy by..
> > > >>> >>
> > > >>> >> On Thu, Dec 31, 2015 at 5:10 PM, Nathan Griffith <
> > > >>> ngriffith@dremio.com>
> > > >>> >> wrote:
> > > >>> >>
> > > >>> >>> Hey Peder,
> > > >>> >>>
> > > >>> >>> What's in your C:\Windows\Temp directory? Is there something
> that
> > > >>> >>> looks like it's from Drill? If there is, try deleting it and
> > > running
> > > >>> >>> again.
> > > >>> >>>
> > > >>> >>> --Nathan
> > > >>> >>>
> > > >>> >>> On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
> > > >>> >>> <pj...@gmail.com> wrote:
> > > >>> >>> > Hi Tomer and Jacques,
> > > >>> >>> >
> > > >>> >>> > I tried those instructions to the letter. I still get the
> > same
> > > >>> java
> > > >>> >>> > Runtime Exception "Uable to deserialize
> > > >>> >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill"
> > > >>> >>> >
> > > >>> >>> > Let me know if you need the full stack trace, although this
> > > should
> > > >>> be
> > > >>> >>> easy
> > > >>> >>> > to replicate. I simply installed a clean version of Windows
> 7
> > on
> > > >>> >>> > VirtualBox, and followed all instructions from Drill
> > > documentation
> > > >>> and
> > > >>> >>> > Tomers suggestions. I can't imagine why it would work on
> any
> > > >>> other
> > > >>> >>> > windows OS unless some other tweak is done somewhere.
> > > >>> >>> >
> > > >>> >>> > P.
> > > >>> >>> >
> > > >>> >>> >
> > > >>> >>> > On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <
> > > >>> jacques@dremio.com>
> > > >>> >>> wrote:
> > > >>> >>> >
> > > >>> >>> >> Just to clarify, Drill doesn't require Hadoop to be
> installed.
> > > >>> Drill
> > > >>> >>> does
> > > >>> >>> >> include some Hadoop code since it uses some Hadoop
> > functionality
> > > >>> under
> > > >>> >>> the
> > > >>> >>> >> hood as a library (even in non-hadoop installs). Drill
> should
> > be
> > > >>> >>> including
> > > >>> >>> >> the required windows executables/dlls in the distribution
> > folder
> > > >>> (for
> > > >>> >>> using
> > > >>> >>> >> hadoop as a library) and this should be transparent to you.
> > Let
> > > >>> us know
> > > >>> >>> >> what happens when you follow the link that Tomer suggested.
> > > >>> >>> >>
> > > >>> >>> >> --
> > > >>> >>> >> Jacques Nadeau
> > > >>> >>> >> CTO and Co-Founder, Dremio
> > > >>> >>> >>
> > > >>> >>> >> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <
> > > >>> tshiran@dremio.com>
> > > >>> >>> wrote:
> > > >>> >>> >>
> > > >>> >>> >> > No. Drill does not depend on Hadoop
> > > >>> >>> >> >
> > > >>> >>> >> > Can you try these instructions and see if they work:
> > > >>> >>> >> >
> > > >>> >>> >> >
> > > >>> >>>
> > > >>>
> > >
> http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
> > > >>> >>> >> >
> > > >>> >>> >> >
> > > >>> >>> >> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
> > > >>> >>> >> > pjakobsen@gmail.com> wrote:
> > > >>> >>> >> > >
> > > >>> >>> >> > > Hi, does hadoop have to be installed in order to run
> > apache
> > > >>> drill
> > > >>> >>> on
> > > >>> >>> >> > > windows?
> > > >>> >>> >> > >
> > > >>> >>> >> > > So far, I'm 3 hours into "Run Apache Drill in 10
> Minutes",
> > > and
> > > >>> >>> still no
> > > >>> >>> >> > > luck. Once I figure it out, I will update the
> > documentation
> > > >>> for
> > > >>> >>> >> windows
> > > >>> >>> >> > if
> > > >>> >>> >> > > you like.
> > > >>> >>> >> > >
> > > >>> >>> >> > > Thanks,
> > > >>> >>> >> > >
> > > >>> >>> >> > > Peder J.
> > > >>> >>> >> >
> > > >>> >>> >>
> > > >>> >>>
> > > >>> >>
> > > >>> >>
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> >
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Peder Jakobsen | gmail <pj...@gmail.com>.
Hi Jacques, I will try it tomorrow. It's time have some Veuve Clicquot. :)
Thank you for all your help today. Happy New Year to everyone.
Peder J.
On Thu, Dec 31, 2015 at 6:16 PM, Jacques Nadeau <ja...@dremio.com> wrote:
> When Drill first starts up (or encounters an empty folder for embedded
> data), it will automatically create a number of new storage plugins. This
> includes cp and dfs, as well as disabled ones for hive, hbase, etc. It
> seems like your Drill is failing to do this. Since you only copied one
> plugin into your tmp directory (the hive plugin), you will not be able to
> query from the cp plugin. You would need to copy all the default plugins to
> get this working. This potentially provides a workaround but it doesn't
> indicate why Drill is failing to initialize these settings.
>
> Can you possibly try this under the Administrator account if one exists?
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Thu, Dec 31, 2015 at 2:50 PM, Peder Jakobsen | gmail <
> pjakobsen@gmail.com
> > wrote:
>
> > I spoke too soon, perhaps (but it must be a simple I/O issue on startup,
> > no?)
> >
> >
> > C:\devel\apache-drill-1.4.0\bin>sqlline.bat -u "jdbc:drill:zk=local"
> > DRILL_ARGS - " -u jdbc:drill:zk=local"
> > HADOOP_HOME not detected...
> > HBASE_HOME not detected...
> > Calculating Drill classpath...
> > Dec 31, 2015 5:39:53 PM org.glassfish.jersey.server.ApplicationHandler
> > initialize
> > INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29
> > 01:25:26...
> > apache drill 1.4.0
> > "just drill it"
> > 0: jdbc:drill:zk=local> SELECT * FROM cp.`employee.json` LIMIT 3;
> > Dec 31, 2015 5:46:55 PM
> > org.apache.calcite.sql.validate.SqlValidatorException <init>
> > SEVERE: org.apache.calcite.sql.validate.SqlValidatorException: Table
> > 'cp.employee.json' not found
> > Dec 31, 2015 5:46:55 PM org.apache.calcite.runtime.CalciteException
> <init>
> > SEVERE: org.apache.calcite.runtime.CalciteContextException: From line 1,
> > column 15 to line 1, column 16: Table 'cp.employee.json' not found
> > Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 16:
> Table
> > 'cp.employee.json' not found
> >
> >
> > [Error Id: 9d6af232-fc40-40ec-8a2a-8d082923b776 on Funky:31010]
> > (state=,code=0)
> >
> > On Thu, Dec 31, 2015 at 5:34 PM, Peder Jakobsen | gmail <
> > pjakobsen@gmail.com
> > > wrote:
> >
> > > OUCH that hurts. Sometimes you get lucky when you jump in a haystack
> and
> > > get pricked by the missing needle right away.
> > >
> > > So I copied hive.sys.drill from my tmp folder in linux to windows, and
> > > wow, now it seems to work. Would you allow me the honour to fix this
> > bug
> > > and commit the changes even though I have not programmed in Java since
> > 2003
> > > ? ;) I can't imagine that I/O in java has changed all that much.
> > >
> > > Peder :)
> > >
> > >
> > >
> > > On Thu, Dec 31, 2015 at 5:30 PM, Peder Jakobsen | gmail <
> > > pjakobsen@gmail.com> wrote:
> > >
> > >> No. I've tried this many times.
> > >>
> > >> So at startup, it's supposed to create these files which it needs?
> > >>
> > >> It seems to do so in my linux install. I will try to copy those
> files
> > >> over and see if I get another error. Needle in a haystack debugging.
> > :)
> > >>
> > >> P.
> > >>
> > >> On Thu, Dec 31, 2015 at 5:27 PM, Nathan Griffith <
> ngriffith@dremio.com>
> > >> wrote:
> > >>
> > >>> Ah! Okay. I remember it making temporary stuff *somewhere* within
> > >>> windows.
> > >>>
> > >>> If I recall correctly, a bad set of temporary files once gave me the
> > >>> exact same issue, which was fixed by deleting them. But apparently
> > >>> this is isnt helping with your case?
> > >>>
> > >>> On Thu, Dec 31, 2015 at 2:22 PM, Peder Jakobsen | gmail
> > >>> <pj...@gmail.com> wrote:
> > >>> > OK, at startup, Drill creates an empty file called hive.sys.drill
> > >>> that's
> > >>> > located in C:\tmp\drill\sys.storage_plugins
> > >>> >
> > >>> > Perhaps it's not surprising that we get "Unable to deserialize
> > >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill" (state=,code=0)"
> > >>> > considering that this file appears to be empty.
> > >>> >
> > >>> > On Linux, lots of stuff is included in this drill path: profiles,
> > >>> > sys.options & sys.storage_plugins
> > >>> >
> > >>> > Hope this helps
> > >>> >
> > >>> > P.
> > >>> >
> > >>> > On Thu, Dec 31, 2015 at 5:13 PM, Peder Jakobsen | gmail <
> > >>> pjakobsen@gmail.com
> > >>> >> wrote:
> > >>> >
> > >>> >> I deleted everything in C:\Windows\Temp. Note, when I start
> drill
> > >>> >> again, nothing gets created in there. I will now try to search
> the
> > >>> >> machine for any files that may have been created at the time of
> > >>> startup.
> > >>> >> Standy by..
> > >>> >>
> > >>> >> On Thu, Dec 31, 2015 at 5:10 PM, Nathan Griffith <
> > >>> ngriffith@dremio.com>
> > >>> >> wrote:
> > >>> >>
> > >>> >>> Hey Peder,
> > >>> >>>
> > >>> >>> What's in your C:\Windows\Temp directory? Is there something that
> > >>> >>> looks like it's from Drill? If there is, try deleting it and
> > running
> > >>> >>> again.
> > >>> >>>
> > >>> >>> --Nathan
> > >>> >>>
> > >>> >>> On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
> > >>> >>> <pj...@gmail.com> wrote:
> > >>> >>> > Hi Tomer and Jacques,
> > >>> >>> >
> > >>> >>> > I tried those instructions to the letter. I still get the
> same
> > >>> java
> > >>> >>> > Runtime Exception "Uable to deserialize
> > >>> >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill"
> > >>> >>> >
> > >>> >>> > Let me know if you need the full stack trace, although this
> > should
> > >>> be
> > >>> >>> easy
> > >>> >>> > to replicate. I simply installed a clean version of Windows 7
> on
> > >>> >>> > VirtualBox, and followed all instructions from Drill
> > documentation
> > >>> and
> > >>> >>> > Tomers suggestions. I can't imagine why it would work on any
> > >>> other
> > >>> >>> > windows OS unless some other tweak is done somewhere.
> > >>> >>> >
> > >>> >>> > P.
> > >>> >>> >
> > >>> >>> >
> > >>> >>> > On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <
> > >>> jacques@dremio.com>
> > >>> >>> wrote:
> > >>> >>> >
> > >>> >>> >> Just to clarify, Drill doesn't require Hadoop to be installed.
> > >>> Drill
> > >>> >>> does
> > >>> >>> >> include some Hadoop code since it uses some Hadoop
> functionality
> > >>> under
> > >>> >>> the
> > >>> >>> >> hood as a library (even in non-hadoop installs). Drill should
> be
> > >>> >>> including
> > >>> >>> >> the required windows executables/dlls in the distribution
> folder
> > >>> (for
> > >>> >>> using
> > >>> >>> >> hadoop as a library) and this should be transparent to you.
> Let
> > >>> us know
> > >>> >>> >> what happens when you follow the link that Tomer suggested.
> > >>> >>> >>
> > >>> >>> >> --
> > >>> >>> >> Jacques Nadeau
> > >>> >>> >> CTO and Co-Founder, Dremio
> > >>> >>> >>
> > >>> >>> >> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <
> > >>> tshiran@dremio.com>
> > >>> >>> wrote:
> > >>> >>> >>
> > >>> >>> >> > No. Drill does not depend on Hadoop
> > >>> >>> >> >
> > >>> >>> >> > Can you try these instructions and see if they work:
> > >>> >>> >> >
> > >>> >>> >> >
> > >>> >>>
> > >>>
> > http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
> > >>> >>> >> >
> > >>> >>> >> >
> > >>> >>> >> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
> > >>> >>> >> > pjakobsen@gmail.com> wrote:
> > >>> >>> >> > >
> > >>> >>> >> > > Hi, does hadoop have to be installed in order to run
> apache
> > >>> drill
> > >>> >>> on
> > >>> >>> >> > > windows?
> > >>> >>> >> > >
> > >>> >>> >> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes",
> > and
> > >>> >>> still no
> > >>> >>> >> > > luck. Once I figure it out, I will update the
> documentation
> > >>> for
> > >>> >>> >> windows
> > >>> >>> >> > if
> > >>> >>> >> > > you like.
> > >>> >>> >> > >
> > >>> >>> >> > > Thanks,
> > >>> >>> >> > >
> > >>> >>> >> > > Peder J.
> > >>> >>> >> >
> > >>> >>> >>
> > >>> >>>
> > >>> >>
> > >>> >>
> > >>>
> > >>
> > >>
> > >
> >
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Peder Jakobsen | gmail <pj...@gmail.com>.
Hi Jacques, yes when I copy all these files over manually from my Linux
machine, everything works as expected on Windows 7 32bit. ODBC drivers and
Drill explorer also work fine.
So what do you think is causing some of these files not to be written on
startup? I have permission on all folders.
For I/O operations in Drill at startup, ideally they'd be performed in a
try catch block of some sort so that the program can exit gracefully with
the appropriate feedback to the user, no?
Let me know if I can be of further help in tracking down this problem.
Peder J.
On Thu, Dec 31, 2015 at 6:16 PM, Jacques Nadeau <ja...@dremio.com> wrote:
> When Drill first starts up (or encounters an empty folder for embedded
> data), it will automatically create a number of new storage plugins. This
> includes cp and dfs, as well as disabled ones for hive, hbase, etc. It
> seems like your Drill is failing to do this. Since you only copied one
> plugin into your tmp directory (the hive plugin), you will not be able to
> query from the cp plugin. You would need to copy all the default plugins to
> get this working. This potentially provides a workaround but it doesn't
> indicate why Drill is failing to initialize these settings.
>
> Can you possibly try this under the Administrator account if one exists?
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Thu, Dec 31, 2015 at 2:50 PM, Peder Jakobsen | gmail <
> pjakobsen@gmail.com
> > wrote:
>
> > I spoke too soon, perhaps (but it must be a simple I/O issue on startup,
> > no?)
> >
> >
> > C:\devel\apache-drill-1.4.0\bin>sqlline.bat -u "jdbc:drill:zk=local"
> > DRILL_ARGS - " -u jdbc:drill:zk=local"
> > HADOOP_HOME not detected...
> > HBASE_HOME not detected...
> > Calculating Drill classpath...
> > Dec 31, 2015 5:39:53 PM org.glassfish.jersey.server.ApplicationHandler
> > initialize
> > INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29
> > 01:25:26...
> > apache drill 1.4.0
> > "just drill it"
> > 0: jdbc:drill:zk=local> SELECT * FROM cp.`employee.json` LIMIT 3;
> > Dec 31, 2015 5:46:55 PM
> > org.apache.calcite.sql.validate.SqlValidatorException <init>
> > SEVERE: org.apache.calcite.sql.validate.SqlValidatorException: Table
> > 'cp.employee.json' not found
> > Dec 31, 2015 5:46:55 PM org.apache.calcite.runtime.CalciteException
> <init>
> > SEVERE: org.apache.calcite.runtime.CalciteContextException: From line 1,
> > column 15 to line 1, column 16: Table 'cp.employee.json' not found
> > Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 16:
> Table
> > 'cp.employee.json' not found
> >
> >
> > [Error Id: 9d6af232-fc40-40ec-8a2a-8d082923b776 on Funky:31010]
> > (state=,code=0)
> >
> > On Thu, Dec 31, 2015 at 5:34 PM, Peder Jakobsen | gmail <
> > pjakobsen@gmail.com
> > > wrote:
> >
> > > OUCH that hurts. Sometimes you get lucky when you jump in a haystack
> and
> > > get pricked by the missing needle right away.
> > >
> > > So I copied hive.sys.drill from my tmp folder in linux to windows, and
> > > wow, now it seems to work. Would you allow me the honour to fix this
> > bug
> > > and commit the changes even though I have not programmed in Java since
> > 2003
> > > ? ;) I can't imagine that I/O in java has changed all that much.
> > >
> > > Peder :)
> > >
> > >
> > >
> > > On Thu, Dec 31, 2015 at 5:30 PM, Peder Jakobsen | gmail <
> > > pjakobsen@gmail.com> wrote:
> > >
> > >> No. I've tried this many times.
> > >>
> > >> So at startup, it's supposed to create these files which it needs?
> > >>
> > >> It seems to do so in my linux install. I will try to copy those
> files
> > >> over and see if I get another error. Needle in a haystack debugging.
> > :)
> > >>
> > >> P.
> > >>
> > >> On Thu, Dec 31, 2015 at 5:27 PM, Nathan Griffith <
> ngriffith@dremio.com>
> > >> wrote:
> > >>
> > >>> Ah! Okay. I remember it making temporary stuff *somewhere* within
> > >>> windows.
> > >>>
> > >>> If I recall correctly, a bad set of temporary files once gave me the
> > >>> exact same issue, which was fixed by deleting them. But apparently
> > >>> this is isnt helping with your case?
> > >>>
> > >>> On Thu, Dec 31, 2015 at 2:22 PM, Peder Jakobsen | gmail
> > >>> <pj...@gmail.com> wrote:
> > >>> > OK, at startup, Drill creates an empty file called hive.sys.drill
> > >>> that's
> > >>> > located in C:\tmp\drill\sys.storage_plugins
> > >>> >
> > >>> > Perhaps it's not surprising that we get "Unable to deserialize
> > >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill" (state=,code=0)"
> > >>> > considering that this file appears to be empty.
> > >>> >
> > >>> > On Linux, lots of stuff is included in this drill path: profiles,
> > >>> > sys.options & sys.storage_plugins
> > >>> >
> > >>> > Hope this helps
> > >>> >
> > >>> > P.
> > >>> >
> > >>> > On Thu, Dec 31, 2015 at 5:13 PM, Peder Jakobsen | gmail <
> > >>> pjakobsen@gmail.com
> > >>> >> wrote:
> > >>> >
> > >>> >> I deleted everything in C:\Windows\Temp. Note, when I start
> drill
> > >>> >> again, nothing gets created in there. I will now try to search
> the
> > >>> >> machine for any files that may have been created at the time of
> > >>> startup.
> > >>> >> Standy by..
> > >>> >>
> > >>> >> On Thu, Dec 31, 2015 at 5:10 PM, Nathan Griffith <
> > >>> ngriffith@dremio.com>
> > >>> >> wrote:
> > >>> >>
> > >>> >>> Hey Peder,
> > >>> >>>
> > >>> >>> What's in your C:\Windows\Temp directory? Is there something that
> > >>> >>> looks like it's from Drill? If there is, try deleting it and
> > running
> > >>> >>> again.
> > >>> >>>
> > >>> >>> --Nathan
> > >>> >>>
> > >>> >>> On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
> > >>> >>> <pj...@gmail.com> wrote:
> > >>> >>> > Hi Tomer and Jacques,
> > >>> >>> >
> > >>> >>> > I tried those instructions to the letter. I still get the
> same
> > >>> java
> > >>> >>> > Runtime Exception "Uable to deserialize
> > >>> >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill"
> > >>> >>> >
> > >>> >>> > Let me know if you need the full stack trace, although this
> > should
> > >>> be
> > >>> >>> easy
> > >>> >>> > to replicate. I simply installed a clean version of Windows 7
> on
> > >>> >>> > VirtualBox, and followed all instructions from Drill
> > documentation
> > >>> and
> > >>> >>> > Tomers suggestions. I can't imagine why it would work on any
> > >>> other
> > >>> >>> > windows OS unless some other tweak is done somewhere.
> > >>> >>> >
> > >>> >>> > P.
> > >>> >>> >
> > >>> >>> >
> > >>> >>> > On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <
> > >>> jacques@dremio.com>
> > >>> >>> wrote:
> > >>> >>> >
> > >>> >>> >> Just to clarify, Drill doesn't require Hadoop to be installed.
> > >>> Drill
> > >>> >>> does
> > >>> >>> >> include some Hadoop code since it uses some Hadoop
> functionality
> > >>> under
> > >>> >>> the
> > >>> >>> >> hood as a library (even in non-hadoop installs). Drill should
> be
> > >>> >>> including
> > >>> >>> >> the required windows executables/dlls in the distribution
> folder
> > >>> (for
> > >>> >>> using
> > >>> >>> >> hadoop as a library) and this should be transparent to you.
> Let
> > >>> us know
> > >>> >>> >> what happens when you follow the link that Tomer suggested.
> > >>> >>> >>
> > >>> >>> >> --
> > >>> >>> >> Jacques Nadeau
> > >>> >>> >> CTO and Co-Founder, Dremio
> > >>> >>> >>
> > >>> >>> >> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <
> > >>> tshiran@dremio.com>
> > >>> >>> wrote:
> > >>> >>> >>
> > >>> >>> >> > No. Drill does not depend on Hadoop
> > >>> >>> >> >
> > >>> >>> >> > Can you try these instructions and see if they work:
> > >>> >>> >> >
> > >>> >>> >> >
> > >>> >>>
> > >>>
> > http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
> > >>> >>> >> >
> > >>> >>> >> >
> > >>> >>> >> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
> > >>> >>> >> > pjakobsen@gmail.com> wrote:
> > >>> >>> >> > >
> > >>> >>> >> > > Hi, does hadoop have to be installed in order to run
> apache
> > >>> drill
> > >>> >>> on
> > >>> >>> >> > > windows?
> > >>> >>> >> > >
> > >>> >>> >> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes",
> > and
> > >>> >>> still no
> > >>> >>> >> > > luck. Once I figure it out, I will update the
> documentation
> > >>> for
> > >>> >>> >> windows
> > >>> >>> >> > if
> > >>> >>> >> > > you like.
> > >>> >>> >> > >
> > >>> >>> >> > > Thanks,
> > >>> >>> >> > >
> > >>> >>> >> > > Peder J.
> > >>> >>> >> >
> > >>> >>> >>
> > >>> >>>
> > >>> >>
> > >>> >>
> > >>>
> > >>
> > >>
> > >
> >
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Jacques Nadeau <ja...@dremio.com>.
When Drill first starts up (or encounters an empty folder for embedded
data), it will automatically create a number of new storage plugins. This
includes cp and dfs, as well as disabled ones for hive, hbase, etc. It
seems like your Drill is failing to do this. Since you only copied one
plugin into your tmp directory (the hive plugin), you will not be able to
query from the cp plugin. You would need to copy all the default plugins to
get this working. This potentially provides a workaround but it doesn't
indicate why Drill is failing to initialize these settings.
Can you possibly try this under the Administrator account if one exists?
--
Jacques Nadeau
CTO and Co-Founder, Dremio
On Thu, Dec 31, 2015 at 2:50 PM, Peder Jakobsen | gmail <pjakobsen@gmail.com
> wrote:
> I spoke too soon, perhaps (but it must be a simple I/O issue on startup,
> no?)
>
>
> C:\devel\apache-drill-1.4.0\bin>sqlline.bat -u "jdbc:drill:zk=local"
> DRILL_ARGS - " -u jdbc:drill:zk=local"
> HADOOP_HOME not detected...
> HBASE_HOME not detected...
> Calculating Drill classpath...
> Dec 31, 2015 5:39:53 PM org.glassfish.jersey.server.ApplicationHandler
> initialize
> INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29
> 01:25:26...
> apache drill 1.4.0
> "just drill it"
> 0: jdbc:drill:zk=local> SELECT * FROM cp.`employee.json` LIMIT 3;
> Dec 31, 2015 5:46:55 PM
> org.apache.calcite.sql.validate.SqlValidatorException <init>
> SEVERE: org.apache.calcite.sql.validate.SqlValidatorException: Table
> 'cp.employee.json' not found
> Dec 31, 2015 5:46:55 PM org.apache.calcite.runtime.CalciteException <init>
> SEVERE: org.apache.calcite.runtime.CalciteContextException: From line 1,
> column 15 to line 1, column 16: Table 'cp.employee.json' not found
> Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 16: Table
> 'cp.employee.json' not found
>
>
> [Error Id: 9d6af232-fc40-40ec-8a2a-8d082923b776 on Funky:31010]
> (state=,code=0)
>
> On Thu, Dec 31, 2015 at 5:34 PM, Peder Jakobsen | gmail <
> pjakobsen@gmail.com
> > wrote:
>
> > OUCH that hurts. Sometimes you get lucky when you jump in a haystack and
> > get pricked by the missing needle right away.
> >
> > So I copied hive.sys.drill from my tmp folder in linux to windows, and
> > wow, now it seems to work. Would you allow me the honour to fix this
> bug
> > and commit the changes even though I have not programmed in Java since
> 2003
> > ? ;) I can't imagine that I/O in java has changed all that much.
> >
> > Peder :)
> >
> >
> >
> > On Thu, Dec 31, 2015 at 5:30 PM, Peder Jakobsen | gmail <
> > pjakobsen@gmail.com> wrote:
> >
> >> No. I've tried this many times.
> >>
> >> So at startup, it's supposed to create these files which it needs?
> >>
> >> It seems to do so in my linux install. I will try to copy those files
> >> over and see if I get another error. Needle in a haystack debugging.
> :)
> >>
> >> P.
> >>
> >> On Thu, Dec 31, 2015 at 5:27 PM, Nathan Griffith <ng...@dremio.com>
> >> wrote:
> >>
> >>> Ah! Okay. I remember it making temporary stuff *somewhere* within
> >>> windows.
> >>>
> >>> If I recall correctly, a bad set of temporary files once gave me the
> >>> exact same issue, which was fixed by deleting them. But apparently
> >>> this is isnt helping with your case?
> >>>
> >>> On Thu, Dec 31, 2015 at 2:22 PM, Peder Jakobsen | gmail
> >>> <pj...@gmail.com> wrote:
> >>> > OK, at startup, Drill creates an empty file called hive.sys.drill
> >>> that's
> >>> > located in C:\tmp\drill\sys.storage_plugins
> >>> >
> >>> > Perhaps it's not surprising that we get "Unable to deserialize
> >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill" (state=,code=0)"
> >>> > considering that this file appears to be empty.
> >>> >
> >>> > On Linux, lots of stuff is included in this drill path: profiles,
> >>> > sys.options & sys.storage_plugins
> >>> >
> >>> > Hope this helps
> >>> >
> >>> > P.
> >>> >
> >>> > On Thu, Dec 31, 2015 at 5:13 PM, Peder Jakobsen | gmail <
> >>> pjakobsen@gmail.com
> >>> >> wrote:
> >>> >
> >>> >> I deleted everything in C:\Windows\Temp. Note, when I start drill
> >>> >> again, nothing gets created in there. I will now try to search the
> >>> >> machine for any files that may have been created at the time of
> >>> startup.
> >>> >> Standy by..
> >>> >>
> >>> >> On Thu, Dec 31, 2015 at 5:10 PM, Nathan Griffith <
> >>> ngriffith@dremio.com>
> >>> >> wrote:
> >>> >>
> >>> >>> Hey Peder,
> >>> >>>
> >>> >>> What's in your C:\Windows\Temp directory? Is there something that
> >>> >>> looks like it's from Drill? If there is, try deleting it and
> running
> >>> >>> again.
> >>> >>>
> >>> >>> --Nathan
> >>> >>>
> >>> >>> On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
> >>> >>> <pj...@gmail.com> wrote:
> >>> >>> > Hi Tomer and Jacques,
> >>> >>> >
> >>> >>> > I tried those instructions to the letter. I still get the same
> >>> java
> >>> >>> > Runtime Exception "Uable to deserialize
> >>> >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill"
> >>> >>> >
> >>> >>> > Let me know if you need the full stack trace, although this
> should
> >>> be
> >>> >>> easy
> >>> >>> > to replicate. I simply installed a clean version of Windows 7 on
> >>> >>> > VirtualBox, and followed all instructions from Drill
> documentation
> >>> and
> >>> >>> > Tomers suggestions. I can't imagine why it would work on any
> >>> other
> >>> >>> > windows OS unless some other tweak is done somewhere.
> >>> >>> >
> >>> >>> > P.
> >>> >>> >
> >>> >>> >
> >>> >>> > On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <
> >>> jacques@dremio.com>
> >>> >>> wrote:
> >>> >>> >
> >>> >>> >> Just to clarify, Drill doesn't require Hadoop to be installed.
> >>> Drill
> >>> >>> does
> >>> >>> >> include some Hadoop code since it uses some Hadoop functionality
> >>> under
> >>> >>> the
> >>> >>> >> hood as a library (even in non-hadoop installs). Drill should be
> >>> >>> including
> >>> >>> >> the required windows executables/dlls in the distribution folder
> >>> (for
> >>> >>> using
> >>> >>> >> hadoop as a library) and this should be transparent to you. Let
> >>> us know
> >>> >>> >> what happens when you follow the link that Tomer suggested.
> >>> >>> >>
> >>> >>> >> --
> >>> >>> >> Jacques Nadeau
> >>> >>> >> CTO and Co-Founder, Dremio
> >>> >>> >>
> >>> >>> >> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <
> >>> tshiran@dremio.com>
> >>> >>> wrote:
> >>> >>> >>
> >>> >>> >> > No. Drill does not depend on Hadoop
> >>> >>> >> >
> >>> >>> >> > Can you try these instructions and see if they work:
> >>> >>> >> >
> >>> >>> >> >
> >>> >>>
> >>>
> http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
> >>> >>> >> >
> >>> >>> >> >
> >>> >>> >> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
> >>> >>> >> > pjakobsen@gmail.com> wrote:
> >>> >>> >> > >
> >>> >>> >> > > Hi, does hadoop have to be installed in order to run apache
> >>> drill
> >>> >>> on
> >>> >>> >> > > windows?
> >>> >>> >> > >
> >>> >>> >> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes",
> and
> >>> >>> still no
> >>> >>> >> > > luck. Once I figure it out, I will update the documentation
> >>> for
> >>> >>> >> windows
> >>> >>> >> > if
> >>> >>> >> > > you like.
> >>> >>> >> > >
> >>> >>> >> > > Thanks,
> >>> >>> >> > >
> >>> >>> >> > > Peder J.
> >>> >>> >> >
> >>> >>> >>
> >>> >>>
> >>> >>
> >>> >>
> >>>
> >>
> >>
> >
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Peder Jakobsen | gmail <pj...@gmail.com>.
I spoke too soon, perhaps (but it must be a simple I/O issue on startup,
no?)
C:\devel\apache-drill-1.4.0\bin>sqlline.bat -u "jdbc:drill:zk=local"
DRILL_ARGS - " -u jdbc:drill:zk=local"
HADOOP_HOME not detected...
HBASE_HOME not detected...
Calculating Drill classpath...
Dec 31, 2015 5:39:53 PM org.glassfish.jersey.server.ApplicationHandler
initialize
INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29
01:25:26...
apache drill 1.4.0
"just drill it"
0: jdbc:drill:zk=local> SELECT * FROM cp.`employee.json` LIMIT 3;
Dec 31, 2015 5:46:55 PM
org.apache.calcite.sql.validate.SqlValidatorException <init>
SEVERE: org.apache.calcite.sql.validate.SqlValidatorException: Table
'cp.employee.json' not found
Dec 31, 2015 5:46:55 PM org.apache.calcite.runtime.CalciteException <init>
SEVERE: org.apache.calcite.runtime.CalciteContextException: From line 1,
column 15 to line 1, column 16: Table 'cp.employee.json' not found
Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 16: Table
'cp.employee.json' not found
[Error Id: 9d6af232-fc40-40ec-8a2a-8d082923b776 on Funky:31010]
(state=,code=0)
On Thu, Dec 31, 2015 at 5:34 PM, Peder Jakobsen | gmail <pjakobsen@gmail.com
> wrote:
> OUCH that hurts. Sometimes you get lucky when you jump in a haystack and
> get pricked by the missing needle right away.
>
> So I copied hive.sys.drill from my tmp folder in linux to windows, and
> wow, now it seems to work. Would you allow me the honour to fix this bug
> and commit the changes even though I have not programmed in Java since 2003
> ? ;) I can't imagine that I/O in java has changed all that much.
>
> Peder :)
>
>
>
> On Thu, Dec 31, 2015 at 5:30 PM, Peder Jakobsen | gmail <
> pjakobsen@gmail.com> wrote:
>
>> No. I've tried this many times.
>>
>> So at startup, it's supposed to create these files which it needs?
>>
>> It seems to do so in my linux install. I will try to copy those files
>> over and see if I get another error. Needle in a haystack debugging. :)
>>
>> P.
>>
>> On Thu, Dec 31, 2015 at 5:27 PM, Nathan Griffith <ng...@dremio.com>
>> wrote:
>>
>>> Ah! Okay. I remember it making temporary stuff *somewhere* within
>>> windows.
>>>
>>> If I recall correctly, a bad set of temporary files once gave me the
>>> exact same issue, which was fixed by deleting them. But apparently
>>> this is isnt helping with your case?
>>>
>>> On Thu, Dec 31, 2015 at 2:22 PM, Peder Jakobsen | gmail
>>> <pj...@gmail.com> wrote:
>>> > OK, at startup, Drill creates an empty file called hive.sys.drill
>>> that's
>>> > located in C:\tmp\drill\sys.storage_plugins
>>> >
>>> > Perhaps it's not surprising that we get "Unable to deserialize
>>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill" (state=,code=0)"
>>> > considering that this file appears to be empty.
>>> >
>>> > On Linux, lots of stuff is included in this drill path: profiles,
>>> > sys.options & sys.storage_plugins
>>> >
>>> > Hope this helps
>>> >
>>> > P.
>>> >
>>> > On Thu, Dec 31, 2015 at 5:13 PM, Peder Jakobsen | gmail <
>>> pjakobsen@gmail.com
>>> >> wrote:
>>> >
>>> >> I deleted everything in C:\Windows\Temp. Note, when I start drill
>>> >> again, nothing gets created in there. I will now try to search the
>>> >> machine for any files that may have been created at the time of
>>> startup.
>>> >> Standy by..
>>> >>
>>> >> On Thu, Dec 31, 2015 at 5:10 PM, Nathan Griffith <
>>> ngriffith@dremio.com>
>>> >> wrote:
>>> >>
>>> >>> Hey Peder,
>>> >>>
>>> >>> What's in your C:\Windows\Temp directory? Is there something that
>>> >>> looks like it's from Drill? If there is, try deleting it and running
>>> >>> again.
>>> >>>
>>> >>> --Nathan
>>> >>>
>>> >>> On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
>>> >>> <pj...@gmail.com> wrote:
>>> >>> > Hi Tomer and Jacques,
>>> >>> >
>>> >>> > I tried those instructions to the letter. I still get the same
>>> java
>>> >>> > Runtime Exception "Uable to deserialize
>>> >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill"
>>> >>> >
>>> >>> > Let me know if you need the full stack trace, although this should
>>> be
>>> >>> easy
>>> >>> > to replicate. I simply installed a clean version of Windows 7 on
>>> >>> > VirtualBox, and followed all instructions from Drill documentation
>>> and
>>> >>> > Tomers suggestions. I can't imagine why it would work on any
>>> other
>>> >>> > windows OS unless some other tweak is done somewhere.
>>> >>> >
>>> >>> > P.
>>> >>> >
>>> >>> >
>>> >>> > On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <
>>> jacques@dremio.com>
>>> >>> wrote:
>>> >>> >
>>> >>> >> Just to clarify, Drill doesn't require Hadoop to be installed.
>>> Drill
>>> >>> does
>>> >>> >> include some Hadoop code since it uses some Hadoop functionality
>>> under
>>> >>> the
>>> >>> >> hood as a library (even in non-hadoop installs). Drill should be
>>> >>> including
>>> >>> >> the required windows executables/dlls in the distribution folder
>>> (for
>>> >>> using
>>> >>> >> hadoop as a library) and this should be transparent to you. Let
>>> us know
>>> >>> >> what happens when you follow the link that Tomer suggested.
>>> >>> >>
>>> >>> >> --
>>> >>> >> Jacques Nadeau
>>> >>> >> CTO and Co-Founder, Dremio
>>> >>> >>
>>> >>> >> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <
>>> tshiran@dremio.com>
>>> >>> wrote:
>>> >>> >>
>>> >>> >> > No. Drill does not depend on Hadoop
>>> >>> >> >
>>> >>> >> > Can you try these instructions and see if they work:
>>> >>> >> >
>>> >>> >> >
>>> >>>
>>> http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
>>> >>> >> >
>>> >>> >> >
>>> >>> >> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
>>> >>> >> > pjakobsen@gmail.com> wrote:
>>> >>> >> > >
>>> >>> >> > > Hi, does hadoop have to be installed in order to run apache
>>> drill
>>> >>> on
>>> >>> >> > > windows?
>>> >>> >> > >
>>> >>> >> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes", and
>>> >>> still no
>>> >>> >> > > luck. Once I figure it out, I will update the documentation
>>> for
>>> >>> >> windows
>>> >>> >> > if
>>> >>> >> > > you like.
>>> >>> >> > >
>>> >>> >> > > Thanks,
>>> >>> >> > >
>>> >>> >> > > Peder J.
>>> >>> >> >
>>> >>> >>
>>> >>>
>>> >>
>>> >>
>>>
>>
>>
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Jason Altekruse <al...@gmail.com>.
Thank you for your extra effort working through this! We are always welcome
to new contributors, feel free to open a JIRA and a pull request with your
fixes.
Welcome to the Drill community!
- Jason
On Thu, Dec 31, 2015 at 4:34 PM, Peder Jakobsen | gmail <pjakobsen@gmail.com
> wrote:
> OUCH that hurts. Sometimes you get lucky when you jump in a haystack and
> get pricked by the missing needle right away.
>
> So I copied hive.sys.drill from my tmp folder in linux to windows, and wow,
> now it seems to work. Would you allow me the honour to fix this bug and
> commit the changes even though I have not programmed in Java since 2003 ?
> ;) I can't imagine that I/O in java has changed all that much.
>
> Peder :)
>
>
>
> On Thu, Dec 31, 2015 at 5:30 PM, Peder Jakobsen | gmail <
> pjakobsen@gmail.com
> > wrote:
>
> > No. I've tried this many times.
> >
> > So at startup, it's supposed to create these files which it needs?
> >
> > It seems to do so in my linux install. I will try to copy those files
> > over and see if I get another error. Needle in a haystack debugging. :)
> >
> > P.
> >
> > On Thu, Dec 31, 2015 at 5:27 PM, Nathan Griffith <ng...@dremio.com>
> > wrote:
> >
> >> Ah! Okay. I remember it making temporary stuff *somewhere* within
> windows.
> >>
> >> If I recall correctly, a bad set of temporary files once gave me the
> >> exact same issue, which was fixed by deleting them. But apparently
> >> this is isnt helping with your case?
> >>
> >> On Thu, Dec 31, 2015 at 2:22 PM, Peder Jakobsen | gmail
> >> <pj...@gmail.com> wrote:
> >> > OK, at startup, Drill creates an empty file called hive.sys.drill
> >> that's
> >> > located in C:\tmp\drill\sys.storage_plugins
> >> >
> >> > Perhaps it's not surprising that we get "Unable to deserialize
> >> > "/tmp/drill/sys.storage_plugins/hive.sys.drill" (state=,code=0)"
> >> > considering that this file appears to be empty.
> >> >
> >> > On Linux, lots of stuff is included in this drill path: profiles,
> >> > sys.options & sys.storage_plugins
> >> >
> >> > Hope this helps
> >> >
> >> > P.
> >> >
> >> > On Thu, Dec 31, 2015 at 5:13 PM, Peder Jakobsen | gmail <
> >> pjakobsen@gmail.com
> >> >> wrote:
> >> >
> >> >> I deleted everything in C:\Windows\Temp. Note, when I start drill
> >> >> again, nothing gets created in there. I will now try to search the
> >> >> machine for any files that may have been created at the time of
> >> startup.
> >> >> Standy by..
> >> >>
> >> >> On Thu, Dec 31, 2015 at 5:10 PM, Nathan Griffith <
> ngriffith@dremio.com
> >> >
> >> >> wrote:
> >> >>
> >> >>> Hey Peder,
> >> >>>
> >> >>> What's in your C:\Windows\Temp directory? Is there something that
> >> >>> looks like it's from Drill? If there is, try deleting it and running
> >> >>> again.
> >> >>>
> >> >>> --Nathan
> >> >>>
> >> >>> On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
> >> >>> <pj...@gmail.com> wrote:
> >> >>> > Hi Tomer and Jacques,
> >> >>> >
> >> >>> > I tried those instructions to the letter. I still get the same
> >> java
> >> >>> > Runtime Exception "Uable to deserialize
> >> >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill"
> >> >>> >
> >> >>> > Let me know if you need the full stack trace, although this should
> >> be
> >> >>> easy
> >> >>> > to replicate. I simply installed a clean version of Windows 7 on
> >> >>> > VirtualBox, and followed all instructions from Drill documentation
> >> and
> >> >>> > Tomers suggestions. I can't imagine why it would work on any
> other
> >> >>> > windows OS unless some other tweak is done somewhere.
> >> >>> >
> >> >>> > P.
> >> >>> >
> >> >>> >
> >> >>> > On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <
> jacques@dremio.com
> >> >
> >> >>> wrote:
> >> >>> >
> >> >>> >> Just to clarify, Drill doesn't require Hadoop to be installed.
> >> Drill
> >> >>> does
> >> >>> >> include some Hadoop code since it uses some Hadoop functionality
> >> under
> >> >>> the
> >> >>> >> hood as a library (even in non-hadoop installs). Drill should be
> >> >>> including
> >> >>> >> the required windows executables/dlls in the distribution folder
> >> (for
> >> >>> using
> >> >>> >> hadoop as a library) and this should be transparent to you. Let
> us
> >> know
> >> >>> >> what happens when you follow the link that Tomer suggested.
> >> >>> >>
> >> >>> >> --
> >> >>> >> Jacques Nadeau
> >> >>> >> CTO and Co-Founder, Dremio
> >> >>> >>
> >> >>> >> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <
> tshiran@dremio.com
> >> >
> >> >>> wrote:
> >> >>> >>
> >> >>> >> > No. Drill does not depend on Hadoop
> >> >>> >> >
> >> >>> >> > Can you try these instructions and see if they work:
> >> >>> >> >
> >> >>> >> >
> >> >>>
> >>
> http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
> >> >>> >> > pjakobsen@gmail.com> wrote:
> >> >>> >> > >
> >> >>> >> > > Hi, does hadoop have to be installed in order to run apache
> >> drill
> >> >>> on
> >> >>> >> > > windows?
> >> >>> >> > >
> >> >>> >> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes",
> and
> >> >>> still no
> >> >>> >> > > luck. Once I figure it out, I will update the documentation
> >> for
> >> >>> >> windows
> >> >>> >> > if
> >> >>> >> > > you like.
> >> >>> >> > >
> >> >>> >> > > Thanks,
> >> >>> >> > >
> >> >>> >> > > Peder J.
> >> >>> >> >
> >> >>> >>
> >> >>>
> >> >>
> >> >>
> >>
> >
> >
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Peder Jakobsen | gmail <pj...@gmail.com>.
OUCH that hurts. Sometimes you get lucky when you jump in a haystack and
get pricked by the missing needle right away.
So I copied hive.sys.drill from my tmp folder in linux to windows, and wow,
now it seems to work. Would you allow me the honour to fix this bug and
commit the changes even though I have not programmed in Java since 2003 ?
;) I can't imagine that I/O in java has changed all that much.
Peder :)
On Thu, Dec 31, 2015 at 5:30 PM, Peder Jakobsen | gmail <pjakobsen@gmail.com
> wrote:
> No. I've tried this many times.
>
> So at startup, it's supposed to create these files which it needs?
>
> It seems to do so in my linux install. I will try to copy those files
> over and see if I get another error. Needle in a haystack debugging. :)
>
> P.
>
> On Thu, Dec 31, 2015 at 5:27 PM, Nathan Griffith <ng...@dremio.com>
> wrote:
>
>> Ah! Okay. I remember it making temporary stuff *somewhere* within windows.
>>
>> If I recall correctly, a bad set of temporary files once gave me the
>> exact same issue, which was fixed by deleting them. But apparently
>> this is isnt helping with your case?
>>
>> On Thu, Dec 31, 2015 at 2:22 PM, Peder Jakobsen | gmail
>> <pj...@gmail.com> wrote:
>> > OK, at startup, Drill creates an empty file called hive.sys.drill
>> that's
>> > located in C:\tmp\drill\sys.storage_plugins
>> >
>> > Perhaps it's not surprising that we get "Unable to deserialize
>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill" (state=,code=0)"
>> > considering that this file appears to be empty.
>> >
>> > On Linux, lots of stuff is included in this drill path: profiles,
>> > sys.options & sys.storage_plugins
>> >
>> > Hope this helps
>> >
>> > P.
>> >
>> > On Thu, Dec 31, 2015 at 5:13 PM, Peder Jakobsen | gmail <
>> pjakobsen@gmail.com
>> >> wrote:
>> >
>> >> I deleted everything in C:\Windows\Temp. Note, when I start drill
>> >> again, nothing gets created in there. I will now try to search the
>> >> machine for any files that may have been created at the time of
>> startup.
>> >> Standy by..
>> >>
>> >> On Thu, Dec 31, 2015 at 5:10 PM, Nathan Griffith <ngriffith@dremio.com
>> >
>> >> wrote:
>> >>
>> >>> Hey Peder,
>> >>>
>> >>> What's in your C:\Windows\Temp directory? Is there something that
>> >>> looks like it's from Drill? If there is, try deleting it and running
>> >>> again.
>> >>>
>> >>> --Nathan
>> >>>
>> >>> On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
>> >>> <pj...@gmail.com> wrote:
>> >>> > Hi Tomer and Jacques,
>> >>> >
>> >>> > I tried those instructions to the letter. I still get the same
>> java
>> >>> > Runtime Exception "Uable to deserialize
>> >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill"
>> >>> >
>> >>> > Let me know if you need the full stack trace, although this should
>> be
>> >>> easy
>> >>> > to replicate. I simply installed a clean version of Windows 7 on
>> >>> > VirtualBox, and followed all instructions from Drill documentation
>> and
>> >>> > Tomers suggestions. I can't imagine why it would work on any other
>> >>> > windows OS unless some other tweak is done somewhere.
>> >>> >
>> >>> > P.
>> >>> >
>> >>> >
>> >>> > On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <jacques@dremio.com
>> >
>> >>> wrote:
>> >>> >
>> >>> >> Just to clarify, Drill doesn't require Hadoop to be installed.
>> Drill
>> >>> does
>> >>> >> include some Hadoop code since it uses some Hadoop functionality
>> under
>> >>> the
>> >>> >> hood as a library (even in non-hadoop installs). Drill should be
>> >>> including
>> >>> >> the required windows executables/dlls in the distribution folder
>> (for
>> >>> using
>> >>> >> hadoop as a library) and this should be transparent to you. Let us
>> know
>> >>> >> what happens when you follow the link that Tomer suggested.
>> >>> >>
>> >>> >> --
>> >>> >> Jacques Nadeau
>> >>> >> CTO and Co-Founder, Dremio
>> >>> >>
>> >>> >> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <tshiran@dremio.com
>> >
>> >>> wrote:
>> >>> >>
>> >>> >> > No. Drill does not depend on Hadoop
>> >>> >> >
>> >>> >> > Can you try these instructions and see if they work:
>> >>> >> >
>> >>> >> >
>> >>>
>> http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
>> >>> >> >
>> >>> >> >
>> >>> >> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
>> >>> >> > pjakobsen@gmail.com> wrote:
>> >>> >> > >
>> >>> >> > > Hi, does hadoop have to be installed in order to run apache
>> drill
>> >>> on
>> >>> >> > > windows?
>> >>> >> > >
>> >>> >> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes", and
>> >>> still no
>> >>> >> > > luck. Once I figure it out, I will update the documentation
>> for
>> >>> >> windows
>> >>> >> > if
>> >>> >> > > you like.
>> >>> >> > >
>> >>> >> > > Thanks,
>> >>> >> > >
>> >>> >> > > Peder J.
>> >>> >> >
>> >>> >>
>> >>>
>> >>
>> >>
>>
>
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Peder Jakobsen | gmail <pj...@gmail.com>.
No. I've tried this many times.
So at startup, it's supposed to create these files which it needs?
It seems to do so in my linux install. I will try to copy those files
over and see if I get another error. Needle in a haystack debugging. :)
P.
On Thu, Dec 31, 2015 at 5:27 PM, Nathan Griffith <ng...@dremio.com>
wrote:
> Ah! Okay. I remember it making temporary stuff *somewhere* within windows.
>
> If I recall correctly, a bad set of temporary files once gave me the
> exact same issue, which was fixed by deleting them. But apparently
> this is isnt helping with your case?
>
> On Thu, Dec 31, 2015 at 2:22 PM, Peder Jakobsen | gmail
> <pj...@gmail.com> wrote:
> > OK, at startup, Drill creates an empty file called hive.sys.drill that's
> > located in C:\tmp\drill\sys.storage_plugins
> >
> > Perhaps it's not surprising that we get "Unable to deserialize
> > "/tmp/drill/sys.storage_plugins/hive.sys.drill" (state=,code=0)"
> > considering that this file appears to be empty.
> >
> > On Linux, lots of stuff is included in this drill path: profiles,
> > sys.options & sys.storage_plugins
> >
> > Hope this helps
> >
> > P.
> >
> > On Thu, Dec 31, 2015 at 5:13 PM, Peder Jakobsen | gmail <
> pjakobsen@gmail.com
> >> wrote:
> >
> >> I deleted everything in C:\Windows\Temp. Note, when I start drill
> >> again, nothing gets created in there. I will now try to search the
> >> machine for any files that may have been created at the time of startup.
> >> Standy by..
> >>
> >> On Thu, Dec 31, 2015 at 5:10 PM, Nathan Griffith <ng...@dremio.com>
> >> wrote:
> >>
> >>> Hey Peder,
> >>>
> >>> What's in your C:\Windows\Temp directory? Is there something that
> >>> looks like it's from Drill? If there is, try deleting it and running
> >>> again.
> >>>
> >>> --Nathan
> >>>
> >>> On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
> >>> <pj...@gmail.com> wrote:
> >>> > Hi Tomer and Jacques,
> >>> >
> >>> > I tried those instructions to the letter. I still get the same java
> >>> > Runtime Exception "Uable to deserialize
> >>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill"
> >>> >
> >>> > Let me know if you need the full stack trace, although this should be
> >>> easy
> >>> > to replicate. I simply installed a clean version of Windows 7 on
> >>> > VirtualBox, and followed all instructions from Drill documentation
> and
> >>> > Tomers suggestions. I can't imagine why it would work on any other
> >>> > windows OS unless some other tweak is done somewhere.
> >>> >
> >>> > P.
> >>> >
> >>> >
> >>> > On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <ja...@dremio.com>
> >>> wrote:
> >>> >
> >>> >> Just to clarify, Drill doesn't require Hadoop to be installed. Drill
> >>> does
> >>> >> include some Hadoop code since it uses some Hadoop functionality
> under
> >>> the
> >>> >> hood as a library (even in non-hadoop installs). Drill should be
> >>> including
> >>> >> the required windows executables/dlls in the distribution folder
> (for
> >>> using
> >>> >> hadoop as a library) and this should be transparent to you. Let us
> know
> >>> >> what happens when you follow the link that Tomer suggested.
> >>> >>
> >>> >> --
> >>> >> Jacques Nadeau
> >>> >> CTO and Co-Founder, Dremio
> >>> >>
> >>> >> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <ts...@dremio.com>
> >>> wrote:
> >>> >>
> >>> >> > No. Drill does not depend on Hadoop
> >>> >> >
> >>> >> > Can you try these instructions and see if they work:
> >>> >> >
> >>> >> >
> >>>
> http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
> >>> >> >
> >>> >> >
> >>> >> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
> >>> >> > pjakobsen@gmail.com> wrote:
> >>> >> > >
> >>> >> > > Hi, does hadoop have to be installed in order to run apache
> drill
> >>> on
> >>> >> > > windows?
> >>> >> > >
> >>> >> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes", and
> >>> still no
> >>> >> > > luck. Once I figure it out, I will update the documentation for
> >>> >> windows
> >>> >> > if
> >>> >> > > you like.
> >>> >> > >
> >>> >> > > Thanks,
> >>> >> > >
> >>> >> > > Peder J.
> >>> >> >
> >>> >>
> >>>
> >>
> >>
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Nathan Griffith <ng...@dremio.com>.
Ah! Okay. I remember it making temporary stuff *somewhere* within windows.
If I recall correctly, a bad set of temporary files once gave me the
exact same issue, which was fixed by deleting them. But apparently
this is isnt helping with your case?
On Thu, Dec 31, 2015 at 2:22 PM, Peder Jakobsen | gmail
<pj...@gmail.com> wrote:
> OK, at startup, Drill creates an empty file called hive.sys.drill that's
> located in C:\tmp\drill\sys.storage_plugins
>
> Perhaps it's not surprising that we get "Unable to deserialize
> "/tmp/drill/sys.storage_plugins/hive.sys.drill" (state=,code=0)"
> considering that this file appears to be empty.
>
> On Linux, lots of stuff is included in this drill path: profiles,
> sys.options & sys.storage_plugins
>
> Hope this helps
>
> P.
>
> On Thu, Dec 31, 2015 at 5:13 PM, Peder Jakobsen | gmail <pjakobsen@gmail.com
>> wrote:
>
>> I deleted everything in C:\Windows\Temp. Note, when I start drill
>> again, nothing gets created in there. I will now try to search the
>> machine for any files that may have been created at the time of startup.
>> Standy by..
>>
>> On Thu, Dec 31, 2015 at 5:10 PM, Nathan Griffith <ng...@dremio.com>
>> wrote:
>>
>>> Hey Peder,
>>>
>>> What's in your C:\Windows\Temp directory? Is there something that
>>> looks like it's from Drill? If there is, try deleting it and running
>>> again.
>>>
>>> --Nathan
>>>
>>> On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
>>> <pj...@gmail.com> wrote:
>>> > Hi Tomer and Jacques,
>>> >
>>> > I tried those instructions to the letter. I still get the same java
>>> > Runtime Exception "Uable to deserialize
>>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill"
>>> >
>>> > Let me know if you need the full stack trace, although this should be
>>> easy
>>> > to replicate. I simply installed a clean version of Windows 7 on
>>> > VirtualBox, and followed all instructions from Drill documentation and
>>> > Tomers suggestions. I can't imagine why it would work on any other
>>> > windows OS unless some other tweak is done somewhere.
>>> >
>>> > P.
>>> >
>>> >
>>> > On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <ja...@dremio.com>
>>> wrote:
>>> >
>>> >> Just to clarify, Drill doesn't require Hadoop to be installed. Drill
>>> does
>>> >> include some Hadoop code since it uses some Hadoop functionality under
>>> the
>>> >> hood as a library (even in non-hadoop installs). Drill should be
>>> including
>>> >> the required windows executables/dlls in the distribution folder (for
>>> using
>>> >> hadoop as a library) and this should be transparent to you. Let us know
>>> >> what happens when you follow the link that Tomer suggested.
>>> >>
>>> >> --
>>> >> Jacques Nadeau
>>> >> CTO and Co-Founder, Dremio
>>> >>
>>> >> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <ts...@dremio.com>
>>> wrote:
>>> >>
>>> >> > No. Drill does not depend on Hadoop
>>> >> >
>>> >> > Can you try these instructions and see if they work:
>>> >> >
>>> >> >
>>> http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
>>> >> >
>>> >> >
>>> >> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
>>> >> > pjakobsen@gmail.com> wrote:
>>> >> > >
>>> >> > > Hi, does hadoop have to be installed in order to run apache drill
>>> on
>>> >> > > windows?
>>> >> > >
>>> >> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes", and
>>> still no
>>> >> > > luck. Once I figure it out, I will update the documentation for
>>> >> windows
>>> >> > if
>>> >> > > you like.
>>> >> > >
>>> >> > > Thanks,
>>> >> > >
>>> >> > > Peder J.
>>> >> >
>>> >>
>>>
>>
>>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Peder Jakobsen | gmail <pj...@gmail.com>.
OK, at startup, Drill creates an empty file called hive.sys.drill that's
located in C:\tmp\drill\sys.storage_plugins
Perhaps it's not surprising that we get "Unable to deserialize
"/tmp/drill/sys.storage_plugins/hive.sys.drill" (state=,code=0)"
considering that this file appears to be empty.
On Linux, lots of stuff is included in this drill path: profiles,
sys.options & sys.storage_plugins
Hope this helps
P.
On Thu, Dec 31, 2015 at 5:13 PM, Peder Jakobsen | gmail <pjakobsen@gmail.com
> wrote:
> I deleted everything in C:\Windows\Temp. Note, when I start drill
> again, nothing gets created in there. I will now try to search the
> machine for any files that may have been created at the time of startup.
> Standy by..
>
> On Thu, Dec 31, 2015 at 5:10 PM, Nathan Griffith <ng...@dremio.com>
> wrote:
>
>> Hey Peder,
>>
>> What's in your C:\Windows\Temp directory? Is there something that
>> looks like it's from Drill? If there is, try deleting it and running
>> again.
>>
>> --Nathan
>>
>> On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
>> <pj...@gmail.com> wrote:
>> > Hi Tomer and Jacques,
>> >
>> > I tried those instructions to the letter. I still get the same java
>> > Runtime Exception "Uable to deserialize
>> > "/tmp/drill/sys.storage_plugins/hive.sys.drill"
>> >
>> > Let me know if you need the full stack trace, although this should be
>> easy
>> > to replicate. I simply installed a clean version of Windows 7 on
>> > VirtualBox, and followed all instructions from Drill documentation and
>> > Tomers suggestions. I can't imagine why it would work on any other
>> > windows OS unless some other tweak is done somewhere.
>> >
>> > P.
>> >
>> >
>> > On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <ja...@dremio.com>
>> wrote:
>> >
>> >> Just to clarify, Drill doesn't require Hadoop to be installed. Drill
>> does
>> >> include some Hadoop code since it uses some Hadoop functionality under
>> the
>> >> hood as a library (even in non-hadoop installs). Drill should be
>> including
>> >> the required windows executables/dlls in the distribution folder (for
>> using
>> >> hadoop as a library) and this should be transparent to you. Let us know
>> >> what happens when you follow the link that Tomer suggested.
>> >>
>> >> --
>> >> Jacques Nadeau
>> >> CTO and Co-Founder, Dremio
>> >>
>> >> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <ts...@dremio.com>
>> wrote:
>> >>
>> >> > No. Drill does not depend on Hadoop
>> >> >
>> >> > Can you try these instructions and see if they work:
>> >> >
>> >> >
>> http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
>> >> >
>> >> >
>> >> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
>> >> > pjakobsen@gmail.com> wrote:
>> >> > >
>> >> > > Hi, does hadoop have to be installed in order to run apache drill
>> on
>> >> > > windows?
>> >> > >
>> >> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes", and
>> still no
>> >> > > luck. Once I figure it out, I will update the documentation for
>> >> windows
>> >> > if
>> >> > > you like.
>> >> > >
>> >> > > Thanks,
>> >> > >
>> >> > > Peder J.
>> >> >
>> >>
>>
>
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Peder Jakobsen | gmail <pj...@gmail.com>.
I deleted everything in C:\Windows\Temp. Note, when I start drill again,
nothing gets created in there. I will now try to search the machine for
any files that may have been created at the time of startup. Standy by..
On Thu, Dec 31, 2015 at 5:10 PM, Nathan Griffith <ng...@dremio.com>
wrote:
> Hey Peder,
>
> What's in your C:\Windows\Temp directory? Is there something that
> looks like it's from Drill? If there is, try deleting it and running
> again.
>
> --Nathan
>
> On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
> <pj...@gmail.com> wrote:
> > Hi Tomer and Jacques,
> >
> > I tried those instructions to the letter. I still get the same java
> > Runtime Exception "Uable to deserialize
> > "/tmp/drill/sys.storage_plugins/hive.sys.drill"
> >
> > Let me know if you need the full stack trace, although this should be
> easy
> > to replicate. I simply installed a clean version of Windows 7 on
> > VirtualBox, and followed all instructions from Drill documentation and
> > Tomers suggestions. I can't imagine why it would work on any other
> > windows OS unless some other tweak is done somewhere.
> >
> > P.
> >
> >
> > On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <ja...@dremio.com>
> wrote:
> >
> >> Just to clarify, Drill doesn't require Hadoop to be installed. Drill
> does
> >> include some Hadoop code since it uses some Hadoop functionality under
> the
> >> hood as a library (even in non-hadoop installs). Drill should be
> including
> >> the required windows executables/dlls in the distribution folder (for
> using
> >> hadoop as a library) and this should be transparent to you. Let us know
> >> what happens when you follow the link that Tomer suggested.
> >>
> >> --
> >> Jacques Nadeau
> >> CTO and Co-Founder, Dremio
> >>
> >> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <ts...@dremio.com>
> wrote:
> >>
> >> > No. Drill does not depend on Hadoop
> >> >
> >> > Can you try these instructions and see if they work:
> >> >
> >> >
> http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
> >> >
> >> >
> >> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
> >> > pjakobsen@gmail.com> wrote:
> >> > >
> >> > > Hi, does hadoop have to be installed in order to run apache drill on
> >> > > windows?
> >> > >
> >> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes", and
> still no
> >> > > luck. Once I figure it out, I will update the documentation for
> >> windows
> >> > if
> >> > > you like.
> >> > >
> >> > > Thanks,
> >> > >
> >> > > Peder J.
> >> >
> >>
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Nathan Griffith <ng...@dremio.com>.
Hey Peder,
What's in your C:\Windows\Temp directory? Is there something that
looks like it's from Drill? If there is, try deleting it and running
again.
--Nathan
On Thu, Dec 31, 2015 at 1:53 PM, Peder Jakobsen | gmail
<pj...@gmail.com> wrote:
> Hi Tomer and Jacques,
>
> I tried those instructions to the letter. I still get the same java
> Runtime Exception "Uable to deserialize
> "/tmp/drill/sys.storage_plugins/hive.sys.drill"
>
> Let me know if you need the full stack trace, although this should be easy
> to replicate. I simply installed a clean version of Windows 7 on
> VirtualBox, and followed all instructions from Drill documentation and
> Tomers suggestions. I can't imagine why it would work on any other
> windows OS unless some other tweak is done somewhere.
>
> P.
>
>
> On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <ja...@dremio.com> wrote:
>
>> Just to clarify, Drill doesn't require Hadoop to be installed. Drill does
>> include some Hadoop code since it uses some Hadoop functionality under the
>> hood as a library (even in non-hadoop installs). Drill should be including
>> the required windows executables/dlls in the distribution folder (for using
>> hadoop as a library) and this should be transparent to you. Let us know
>> what happens when you follow the link that Tomer suggested.
>>
>> --
>> Jacques Nadeau
>> CTO and Co-Founder, Dremio
>>
>> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <ts...@dremio.com> wrote:
>>
>> > No. Drill does not depend on Hadoop
>> >
>> > Can you try these instructions and see if they work:
>> >
>> > http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
>> >
>> >
>> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
>> > pjakobsen@gmail.com> wrote:
>> > >
>> > > Hi, does hadoop have to be installed in order to run apache drill on
>> > > windows?
>> > >
>> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes", and still no
>> > > luck. Once I figure it out, I will update the documentation for
>> windows
>> > if
>> > > you like.
>> > >
>> > > Thanks,
>> > >
>> > > Peder J.
>> >
>>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Peder Jakobsen | gmail <pj...@gmail.com>.
Hi Tomer and Jacques,
I tried those instructions to the letter. I still get the same java
Runtime Exception "Uable to deserialize
"/tmp/drill/sys.storage_plugins/hive.sys.drill"
Let me know if you need the full stack trace, although this should be easy
to replicate. I simply installed a clean version of Windows 7 on
VirtualBox, and followed all instructions from Drill documentation and
Tomers suggestions. I can't imagine why it would work on any other
windows OS unless some other tweak is done somewhere.
P.
On Thu, Dec 31, 2015 at 1:22 PM, Jacques Nadeau <ja...@dremio.com> wrote:
> Just to clarify, Drill doesn't require Hadoop to be installed. Drill does
> include some Hadoop code since it uses some Hadoop functionality under the
> hood as a library (even in non-hadoop installs). Drill should be including
> the required windows executables/dlls in the distribution folder (for using
> hadoop as a library) and this should be transparent to you. Let us know
> what happens when you follow the link that Tomer suggested.
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <ts...@dremio.com> wrote:
>
> > No. Drill does not depend on Hadoop
> >
> > Can you try these instructions and see if they work:
> >
> > http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
> >
> >
> > > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
> > pjakobsen@gmail.com> wrote:
> > >
> > > Hi, does hadoop have to be installed in order to run apache drill on
> > > windows?
> > >
> > > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes", and still no
> > > luck. Once I figure it out, I will update the documentation for
> windows
> > if
> > > you like.
> > >
> > > Thanks,
> > >
> > > Peder J.
> >
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Jacques Nadeau <ja...@dremio.com>.
Just to clarify, Drill doesn't require Hadoop to be installed. Drill does
include some Hadoop code since it uses some Hadoop functionality under the
hood as a library (even in non-hadoop installs). Drill should be including
the required windows executables/dlls in the distribution folder (for using
hadoop as a library) and this should be transparent to you. Let us know
what happens when you follow the link that Tomer suggested.
--
Jacques Nadeau
CTO and Co-Founder, Dremio
On Thu, Dec 31, 2015 at 10:14 AM, Tomer Shiran <ts...@dremio.com> wrote:
> No. Drill does not depend on Hadoop
>
> Can you try these instructions and see if they work:
>
> http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
>
>
> > On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <
> pjakobsen@gmail.com> wrote:
> >
> > Hi, does hadoop have to be installed in order to run apache drill on
> > windows?
> >
> > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes", and still no
> > luck. Once I figure it out, I will update the documentation for windows
> if
> > you like.
> >
> > Thanks,
> >
> > Peder J.
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Tomer Shiran <ts...@dremio.com>.
No. Drill does not depend on Hadoop
Can you try these instructions and see if they work:
http://www.dremio.com/blog/installing-apache-drill-on-microsoft-windows/
> On Dec 31, 2015, at 11:32 AM, Peder Jakobsen | gmail <pj...@gmail.com> wrote:
>
> Hi, does hadoop have to be installed in order to run apache drill on
> windows?
>
> So far, I'm 3 hours into "Run Apache Drill in 10 Minutes", and still no
> luck. Once I figure it out, I will update the documentation for windows if
> you like.
>
> Thanks,
>
> Peder J.
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Peder Jakobsen | gmail <pj...@gmail.com>.
Hi Ted.
I'm running Windows 7, 32 bit version, and Java SDK 1.7.0
I installed Apache drill according to instructions here:
https://drill.apache.org/docs/installing-drill-on-windows/
I adjusted there instructions a bit according to Tomer's suggestions.
I start drill according to:
https://drill.apache.org/docs/starting-drill-on-windows/
Here's the command and stack trace:
C:\devel\apache-drill-1.4.0\bin>sqlline.bat -u "jdbc:drill:zk=local"
DRILL_ARGS - " -u jdbc:drill:zk=local"
HADOOP_HOME not detected...
HBASE_HOME not detected...
Calculating Drill classpath...
Error: Failure in starting embedded Drillbit: java.lang.RuntimeException:
Unable to deserialize "/tmp/drill/sys.storage_plugins/hive.sys.drill"
(state=,code=0)
java.sql.SQLException: Failure in starting embedded Drillbit:
java.lang.RuntimeException: Unable to deserialize
"/tmp/drill/sys.storage_plugins/hive.sys.drill"
at
org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:127)
at
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:64)
at
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
at
net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126)
at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
at
sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
at sqlline.Commands.connect(Commands.java:1083)
at sqlline.Commands.connect(Commands.java:1015)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
at sqlline.SqlLine.dispatch(SqlLine.java:742)
at sqlline.SqlLine.initArgs(SqlLine.java:528)
at sqlline.SqlLine.begin(SqlLine.java:596)
at sqlline.SqlLine.start(SqlLine.java:375)
at sqlline.SqlLine.main(SqlLine.java:268)
Caused by: java.lang.RuntimeException: Unable to deserialize
"/tmp/drill/sys.storage_plugins/hive.sys.drill"
at
org.apache.drill.exec.store.sys.local.FilePStore.get(FilePStore.java:140)
at
org.apache.drill.exec.store.sys.local.FilePStore$Iter$DeferredEntry.getValue(FilePStore.java:219)
at
org.apache.drill.exec.store.StoragePluginRegistry.createPlugins(StoragePluginRegistry.java:197)
at
org.apache.drill.exec.store.StoragePluginRegistry.init(StoragePluginRegistry.java:161)
at org.apache.drill.exec.server.Drillbit.run(Drillbit.java:213)
at
org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:118)
... 18 more
Caused by: com.fasterxml.jackson.databind.JsonMappingException: No content
to map due to end-of-input
at [Source: [B@6d9696; line: 1, column: 1]
at
com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
at
com.fasterxml.jackson.databind.ObjectReader._initForReading(ObjectReader.java:1351)
at
com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:1252)
at
com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:912)
at
org.apache.drill.exec.store.sys.serialize.JacksonSerializer.deserialize(JacksonSerializer.java:44)
at
org.apache.drill.exec.store.sys.local.FilePStore.get(FilePStore.java:138)
... 23 more
apache drill 1.4.0
"the only truly happy people are children, the creative minority and drill
users"
0: jdbc:drill:zk=local>
*SAME ERROR OCCURS ON SHUTDOWN*
0: jdbc:drill:zk=local> !quit
Error: Failure in starting embedded Drillbit: java.lang.RuntimeException:
Unable to deserialize "/tmp/drill/sys.storage_plugins/hive.sys.drill"
(state=,code=0)
Error: Failure in starting embedded Drillbit: java.lang.RuntimeException:
Unable to deserialize "/tmp/drill/sys.storage_plugins/hive.sys.drill"
(state=,code=0)
On Thu, Dec 31, 2015 at 4:33 PM, Ted Dunning <te...@gmail.com> wrote:
> Can you say a bit more about your platform and environment?
>
> What about the errors you are seeing?
>
>
>
> On Thu, Dec 31, 2015 at 9:32 AM, Peder Jakobsen | gmail <
> pjakobsen@gmail.com
> > wrote:
>
> > Hi, does hadoop have to be installed in order to run apache drill on
> > windows?
> >
> > So far, I'm 3 hours into "Run Apache Drill in 10 Minutes", and still no
> > luck. Once I figure it out, I will update the documentation for windows
> if
> > you like.
> >
> > Thanks,
> >
> > Peder J.
> >
>
Re: MS Windows: Hadoop binaries required to run drill?
Posted by Ted Dunning <te...@gmail.com>.
Can you say a bit more about your platform and environment?
What about the errors you are seeing?
On Thu, Dec 31, 2015 at 9:32 AM, Peder Jakobsen | gmail <pjakobsen@gmail.com
> wrote:
> Hi, does hadoop have to be installed in order to run apache drill on
> windows?
>
> So far, I'm 3 hours into "Run Apache Drill in 10 Minutes", and still no
> luck. Once I figure it out, I will update the documentation for windows if
> you like.
>
> Thanks,
>
> Peder J.
>