You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@impala.apache.org by Skye Wanderman-Milne <sk...@cloudera.com> on 2016/04/05 00:11:13 UTC

Re: maven error while compiling the front end code

Ah didn't mean to take this off the dev list. Also add our new
dev@impala.incubator list.

Can you check what dependency versions you're pulling in from maven? My
guess is that you're somehow building with non-CDH5.5.2 frontend
dependencies.

On Mon, Apr 4, 2016 at 3:04 PM, Pradeep Nayak <pr...@gmail.com> wrote:

> Umm, I am trying to test this on a dev cluster which has cloudera parcels
> already installed. I just re-placed impala-frontend-0.1-SNAPSHOT.jar with
> my version. Do I need to change anything on the dev cluster ?
>
> On Mon, Apr 4, 2016 at 2:43 PM Skye Wanderman-Milne <sk...@cloudera.com>
> wrote:
>
>> Try: source $IMPALA_HOME/bin/set-classpath.sh
>>
>> If you recently switched branches, you may want to close your current
>> terminal, open a new one, and re-source $IMPALA_HOME/bin/impala-config.sh
>> and set-classpath.sh.
>>
>> Then restart your cluster (start-impala-cluster.py)
>>
>> On Mon, Apr 4, 2016 at 2:39 PM, Pradeep Nayak <pr...@gmail.com>
>> wrote:
>>
>>> Thanks Skye.
>>>
>>> I am in my impala shell. I get this following error. I see this is
>>> already compiled as part of impala-frontend. This happens only when I try
>>> to do a "select * from <table_name>" I can do "show tables; create table
>>> <table_name>" etc. Any pointers ?
>>>
>>> ERROR: AnalysisException: java.lang.NoClassDefFoundError: Could not
>>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>
>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
>>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>
>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>> com.cloudera.impala.util.HdfsCachingUtil
>>>
>>> CAUSED BY: TableLoadingException: java.lang.NoClassDefFoundError: Could
>>> not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>
>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
>>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>
>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>> com.cloudera.impala.util.HdfsCachingUtil
>>>
>>> I used this tree:
>>> https://github.com/cloudera/Impala/tree/cdh5.5.2-release
>>>
>>> On Mon, Apr 4, 2016 at 12:01 PM Skye Wanderman-Milne <sk...@cloudera.com>
>>> wrote:
>>>
>>>> I think it should be
>>>> https://github.com/cloudera/Impala/tree/cdh5-2.3.0_5.5.2
>>>>
>>>> On Mon, Apr 4, 2016 at 12:00 PM, Pradeep Nayak <pr...@gmail.com>
>>>> wrote:
>>>>
>>>>> Also to what release this parcel  *5.5.2-1.cdh5.5.2.p0.4 *map to on
>>>>> github ? I am unable to find a tag/branch for this parcel type.
>>>>>
>>>>>
>>>>> On Mon, Apr 4, 2016 at 11:58 AM Skye Wanderman-Milne <
>>>>> skye@cloudera.com> wrote:
>>>>>
>>>>>> Ah yes, both 1 and 2 are correct. Glad you figured it out!
>>>>>>
>>>>>> On Mon, Apr 4, 2016 at 11:56 AM, Pradeep Nayak <pradeep1288@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> I just figured.
>>>>>>> 1. For the first time you need to use the ./buildall.sh and for
>>>>>>> subsequent fe compilation I can just use maven
>>>>>>> 2. It doesn't build on Macbook
>>>>>>> 3. I set up a Ubuntu vm and got this compiled.
>>>>>>>
>>>>>>> Pradeep
>>>>>>>
>>>>>>> On Mon, Apr 4, 2016 at 11:07 AM Skye Wanderman-Milne <
>>>>>>> skye@cloudera.com> wrote:
>>>>>>>
>>>>>>>> Hey Pradeep,
>>>>>>>>
>>>>>>>> You should try running the $IMPALA_HOME/buildall.sh script in order
>>>>>>>> to build Impala. It has a lot of necessary setup to build Impala. Here are
>>>>>>>> the commands from buildall.sh that should provide the
>>>>>>>> impala-data-source-api jar for building the FE:
>>>>>>>>
>>>>>>>> # build the external data source API
>>>>>>>> pushd ${IMPALA_HOME}/ext-data-source
>>>>>>>> ${IMPALA_HOME}/bin/mvn-quiet.sh install -DskipTests
>>>>>>>> popd
>>>>>>>>
>>>>>>>> See
>>>>>>>> https://github.com/cloudera/Impala/blob/cdh5-trunk/buildall.sh#L283
>>>>>>>>
>>>>>>>> Hope that helps!
>>>>>>>> Skye
>>>>>>>>
>>>>>>>> On Sat, Apr 2, 2016 at 12:55 PM, Pradeep Nayak <
>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I just cloned the Impala source from github, and I am trying to
>>>>>>>>> compile the front end per Impala wiki. I get the error below. I am using
>>>>>>>>> maven 3.
>>>>>>>>>
>>>>>>>>> [ERROR] Failed to execute goal on project impala-frontend: Could
>>>>>>>>> not resolve dependencies for project
>>>>>>>>> com.cloudera.impala:impala-frontend:jar:0.1-SNAPSHOT: Failure to find
>>>>>>>>> com.cloudera.impala:impala-data-source-api:jar:1.0-SNAPSHOT in
>>>>>>>>> https://repository.cloudera.com/content/groups/cdh-releases-rcs
>>>>>>>>> was cached in the local repository, resolution will not be reattempted
>>>>>>>>> until the update interval of cdh.rcs.releases.repo has elapsed or updates
>>>>>>>>> are forced -> [Help 1
>>>>>>>>>
>>>>>>>>> This is the command I used:   mvn clean package
>>>>>>>>> dependency:copy-dependencies -DskipTests=true
>>>>>>>>>
>>>>>>>>> Any idea getting around this problem ?
>>>>>>>>>
>>>>>>>>> Cheers!
>>>>>>>>> Pradeep
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>>> Groups "Impala Dev" group.
>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>> send an email to impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "Impala Dev" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to impala-dev+unsubscribe@cloudera.org.
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>

Re: maven error while compiling the front end code

Posted by Pradeep Nayak <pr...@gmail.com>.
Skye - That's fine, I understand. :-)

However I feel this could be a potential bug. Here is how I can sort of
constantly reproduce.

1. Change the fs.defaultFS to some other value.
2. Propagate stale files and restart HDFS and its' dependent services.
3. After restart goto Impala shell, try to do a select query.
4. You will see the Noclassdef found error.
5. Restart the impala services again. (Just the daemon will do)
6. Now when you will use the impala shell you will no longer see the error.

This is when you use cloudera manager to do the restarts, make config
changes etc.

Pradeep

On Thu, Apr 7, 2016 at 7:07 PM Skye Wanderman-Milne <sk...@cloudera.com>
wrote:

> Thanks for sharing Pradeep. Unfortunately I don't have time to investigate
> this further right now, but I think this info will be useful for others
> running a similar setup as you.
>
> On Tue, Apr 5, 2016 at 1:45 PM, Pradeep Nayak <pr...@gmail.com>
> wrote:
>
>> This is what I saw in IMPALAD.info:
>>
>>
>>
>>
>>
>> I0405 16:44:39.136433 10963 impala-beeswax-server.cc:172] query():
>> query=describe tab1
>> I0405 16:44:39.136504 10963 impala-beeswax-server.cc:515] query: Query {
>>   01: query (string) = "describe tab1",
>>   03: configuration (list) = list<string>[0] {
>>   },
>>   04: hadoop_user (string) = "root",
>> }
>> I0405 16:44:39.136579 10963 impala-beeswax-server.cc:537]
>> TClientRequest.queryOptions: TQueryOptions {
>>   01: abort_on_error (bool) = false,
>>   02: max_errors (i32) = 0,
>>   03: disable_codegen (bool) = false,
>>   04: batch_size (i32) = 0,
>>   05: num_nodes (i32) = 0,
>>   06: max_scan_range_length (i64) = 0,
>>   07: num_scanner_threads (i32) = 0,
>>   08: max_io_buffers (i32) = 0,
>>   09: allow_unsupported_formats (bool) = false,
>>   10: default_order_by_limit (i64) = -1,
>>   11: debug_action (string) = "",
>>   12: mem_limit (i64) = 0,
>>   13: abort_on_default_limit_exceeded (bool) = false,
>>   15: hbase_caching (i32) = 0,
>>   16: hbase_cache_blocks (bool) = false,
>>   17: parquet_file_size (i64) = 0,
>>   18: explain_level (i32) = 1,
>>   19: sync_ddl (bool) = false,
>>   23: disable_cached_reads (bool) = false,
>>   24: disable_outermost_topn (bool) = false,
>>   25: rm_initial_mem (i64) = 0,
>>   26: query_timeout_s (i32) = 0,
>>   28: appx_count_distinct (bool) = false,
>>   29: disable_unsafe_spills (bool) = false,
>>   31: exec_single_node_rows_threshold (i32) = 100,
>> }
>> I0405 16:44:39.139252 10963 Frontend.java:818] analyze query describe tab1
>> E0405 16:44:39.139780 10963 Analyzer.java:2210] Failed to load metadata
>> for table: tab1
>> java.lang.NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>> I0405 16:44:39.140349 10963 jni-util.cc:177]
>> com.cloudera.impala.common.AnalysisException:
>> java.lang.NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>> at
>> com.cloudera.impala.analysis.DescribeStmt.analyze(DescribeStmt.java:117)
>> at
>> com.cloudera.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:342)
>> at
>> com.cloudera.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:317)
>> at com.cloudera.impala.service.Frontend.analyzeStmt(Frontend.java:827)
>> at
>> com.cloudera.impala.service.Frontend.createExecRequest(Frontend.java:856)
>> at
>> com.cloudera.impala.service.JniFrontend.createExecRequest(JniFrontend.java:147)
>> Caused by: com.cloudera.impala.catalog.TableLoadingException:
>> java.lang.NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>> at
>> com.cloudera.impala.catalog.IncompleteTable.loadFromThrift(IncompleteTable.java:111)
>> at com.cloudera.impala.catalog.Table.fromThrift(Table.java:240)
>> at
>> com.cloudera.impala.catalog.ImpaladCatalog.addTable(ImpaladCatalog.java:357)
>> at
>> com.cloudera.impala.catalog.ImpaladCatalog.addCatalogObject(ImpaladCatalog.java:246)
>> at
>> com.cloudera.impala.catalog.ImpaladCatalog.updateCatalog(ImpaladCatalog.java:132)
>> at
>> com.cloudera.impala.service.Frontend.updateCatalogCache(Frontend.java:223)
>> at
>> com.cloudera.impala.service.JniFrontend.updateCatalogCache(JniFrontend.java:164)
>> at ========.<Remote stack trace on catalogd>:
>> com.cloudera.impala.catalog.TableLoadingException:
>> java.lang.NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>> at
>> com.cloudera.impala.catalog.TableLoadingMgr$LoadRequest.get(TableLoadingMgr.java:71)
>> at
>> com.cloudera.impala.catalog.CatalogServiceCatalog.getOrLoadTable(CatalogServiceCatalog.java:499)
>> at com.cloudera.imp
>>
>>
>>
>> ala.catalog.TableLoadingMgr.loadNextTable(TableLoadingMgr.java:287)
>> at
>> com.cloudera.impala.catalog.TableLoadingMgr.access$600(TableLoadingMgr.java:47)
>> at
>> com.cloudera.impala.catalog.TableLoadingMgr$3.run(TableLoadingMgr.java:260)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.util.concurrent.ExecutionException:
>> java.lang.NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
>> at
>> com.cloudera.impala.catalog.TableLoadingMgr$LoadRequest.get(TableLoadingMgr.java:69)
>> ... 7 more
>> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>> at
>> com.cloudera.impala.catalog.HdfsTable.loadPartitions(HdfsTable.java:657)
>> at com.cloudera.impala.catalog.HdfsTable.load(HdfsTable.java:1151)
>> at com.cloudera.impala.catalog.TableLoader.load(TableLoader.java:84)
>> at
>> com.cloudera.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:232)
>> at
>> com.cloudera.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:229)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> ... 3 more
>> ()
>> I0405 16:44:39.151973 10963 status.cc:112] AnalysisException:
>> java.lang.NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: TableLoadingException: java.lang.NoClassDefFoundError: Could
>> not initialize class com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>>     @           0x7993f9  (unknown)
>>     @           0xa325bb  (unknown)
>>     @           0x96d110  (unknown)
>>     @           0x98599a  (unknown)
>>     @           0x990c9f  (unknown)
>>     @           0x9c103b  (unknown)
>>     @           0xacdce6  (unknown)
>>     @           0xac2a5b  (unknown)
>>     @           0x991b01  (unknown)
>>     @          0x15553ea  (unknown)
>>     @          0x15425ef  (unknown)
>>     @          0x1544ff6  (unknown)
>>     @           0x921ba9  (unknown)
>>     @           0x922692  (unknown)
>>     @           0xa629d5  (unknown)
>>     @           0xa63884  (unknown)
>>     @           0xc6f133  (unknown)
>>     @     0x7fb9dad04df5  start_thread
>>     @     0x7fb9d9bbf1ad  __clone
>> I0405 16:44:39.152009 10963 impala-server.cc:862] UnregisterQuery():
>> query_id=164d3f861927d713:b845784fc383df9b
>> I0405 16:44:39.152016 10963 impala-server.cc:943] Cancel():
>> query_id=164d3f861927d713:b845784fc383df9b
>>
>> On Tue, Apr 5, 2016 at 1:36 PM Pradeep Nayak <pr...@gmail.com>
>> wrote:
>>
>>> As for logs,
>>>
>>> I just see this :
>>>
>>> E0405 15:19:47.907562 10957 Analyzer.java:2210] Failed to load metadata
>>> for table: tab1
>>> java.lang.NoClassDefFoundError: Could not initialize class
>>> com.cloudera.impala.util.HdfsCachingUtil
>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
>>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>> com.cloudera.impala.util.HdfsCachingUtil
>>> E0405 15:20:52.074620 10957 Analyzer.java:2210] Failed to load metadata
>>> for table: tab1
>>> java.lang.NoClassDefFoundError: Could not initialize class
>>> com.cloudera.impala.util.HdfsCachingUtil
>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
>>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>> com.cloudera.impala.util.HdfsCachingUtil
>>>
>>> tab1 is an external table.
>>>
>>> On Tue, Apr 5, 2016 at 1:24 PM Pradeep Nayak <pr...@gmail.com>
>>> wrote:
>>>
>>>> Here is the strange part:
>>>>
>>>> Where we make changes to core-site.xml, re-deploy the stale client
>>>> configs from the cloudera manager, and restart the affected services we get
>>>> into the NoClassDefound state. In order to get past it we needed to restart
>>>> impala dameons, catalog server etc again from the cloudera manager. Then
>>>>  it begins to work.
>>>>
>>>> I can reproduce this but very inconsistently though. Is there any logs
>>>> you want me to  be looking at ?
>>>>
>>>>
>>>> On Tue, Apr 5, 2016 at 10:26 AM Skye Wanderman-Milne <sk...@cloudera.com>
>>>> wrote:
>>>>
>>>>> In case others run into similar issues, can you share how you resolved
>>>>> the NoClassDefFoundErrors?
>>>>>
>>>>> On Mon, Apr 4, 2016 at 10:21 PM, Henry Robinson <he...@cloudera.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On 4 April 2016 at 17:55, Pradeep Nayak <pr...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Also just wanted to understand one more thing.
>>>>>>> http://www.cloudera.com/documentation/enterprise/5-5-x/topics/impala_s3.html It
>>>>>>> mentions that impala needs to have a default file system which is hdfs, but
>>>>>>> not just s3a, or any other filesystem which extends Hadoop FileSystem class
>>>>>>> ? Is there plans to support non hdfs filesystem as default ?
>>>>>>>
>>>>>>
>>>>>> Yes there are - see IMPALA-1850 (code review
>>>>>> http://gerrit.cloudera.org:8080/#/c/1121/).
>>>>>>
>>>>>> Henry
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Meanwhile I got around the problem earlier. I can query tables, do a
>>>>>>> count(*), etc.
>>>>>>>
>>>>>>> However join is causing a problem...
>>>>>>>
>>>>>>> *select * from tab1 join tab2;*
>>>>>>>
>>>>>>> *Query: select * from tab1 join tab2*
>>>>>>>
>>>>>>> *ERROR:*
>>>>>>>
>>>>>>> *Failed to connect to FS*
>>>>>>>
>>>>>>> *Error(255): Unknown error 255*
>>>>>>>
>>>>>>>
>>>>>>> Pradeep
>>>>>>>
>>>>>>> On Mon, Apr 4, 2016 at 3:31 PM Pradeep Nayak <pr...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> That's what been puzzling me as well :(. Let me get back to you
>>>>>>>> with that information.
>>>>>>>>
>>>>>>>> Pradeep
>>>>>>>>
>>>>>>>> On Mon, Apr 4, 2016 at 3:28 PM Skye Wanderman-Milne <
>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>
>>>>>>>>> Actually, I didn't read your exception carefully and didn't notice
>>>>>>>>> that it can't find an Impala class, so it's probably not a dependency issue.
>>>>>>>>>
>>>>>>>>> Can you restart your cluster, rerun a select * query, and then
>>>>>>>>> search your logs for that "NoClassDefFoundError"? That will tell us which
>>>>>>>>> process the error is coming from. That class is included in the FE jar, so
>>>>>>>>> I'm not sure why it wouldn't be able to find it.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Apr 4, 2016 at 3:14 PM, Pradeep Nayak <
>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Sure Skye. I didn't notice the to: list as well until you pointed
>>>>>>>>>> out. I really appreciate your help here.
>>>>>>>>>>
>>>>>>>>>> I haven't modified the pom.xml for the front-end here:
>>>>>>>>>> https://github.com/cloudera/Impala/blob/cdh5.5.2-release/fe/pom.xml
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I am just building with this at the moment. mvn clean package
>>>>>>>>>> -DSkipTests = true from the fe folder.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 4, 2016 at 3:11 PM Skye Wanderman-Milne <
>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Ah didn't mean to take this off the dev list. Also add our new
>>>>>>>>>>> dev@impala.incubator list.
>>>>>>>>>>>
>>>>>>>>>>> Can you check what dependency versions you're pulling in from
>>>>>>>>>>> maven? My guess is that you're somehow building with non-CDH5.5.2 frontend
>>>>>>>>>>> dependencies.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 4, 2016 at 3:04 PM, Pradeep Nayak <
>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Umm, I am trying to test this on a dev cluster which has
>>>>>>>>>>>> cloudera parcels already installed. I just re-placed
>>>>>>>>>>>> impala-frontend-0.1-SNAPSHOT.jar with my version. Do I need to change
>>>>>>>>>>>> anything on the dev cluster ?
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Apr 4, 2016 at 2:43 PM Skye Wanderman-Milne <
>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Try: source $IMPALA_HOME/bin/set-classpath.sh
>>>>>>>>>>>>>
>>>>>>>>>>>>> If you recently switched branches, you may want to close your
>>>>>>>>>>>>> current terminal, open a new one, and re-source
>>>>>>>>>>>>> $IMPALA_HOME/bin/impala-config.sh and set-classpath.sh.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Then restart your cluster (start-impala-cluster.py)
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 2:39 PM, Pradeep Nayak <
>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks Skye.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I am in my impala shell. I get this following error. I see
>>>>>>>>>>>>>> this is already compiled as part of impala-frontend. This happens only when
>>>>>>>>>>>>>> I try to do a "select * from <table_name>" I can do "show tables; create
>>>>>>>>>>>>>> table <table_name>" etc. Any pointers ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ERROR: AnalysisException: java.lang.NoClassDefFoundError:
>>>>>>>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> CAUSED BY: ExecutionException:
>>>>>>>>>>>>>> java.lang.NoClassDefFoundError: Could not initialize class
>>>>>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> CAUSED BY: TableLoadingException:
>>>>>>>>>>>>>> java.lang.NoClassDefFoundError: Could not initialize class
>>>>>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> CAUSED BY: ExecutionException:
>>>>>>>>>>>>>> java.lang.NoClassDefFoundError: Could not initialize class
>>>>>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I used this tree:
>>>>>>>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5.5.2-release
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 12:01 PM Skye Wanderman-Milne <
>>>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think it should be
>>>>>>>>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5-2.3.0_5.5.2
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 12:00 PM, Pradeep Nayak <
>>>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Also to what release this parcel  *5.5.2-1.cdh5.5.2.p0.4 *map
>>>>>>>>>>>>>>>> to on github ? I am unable to find a tag/branch for this parcel type.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:58 AM Skye Wanderman-Milne <
>>>>>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ah yes, both 1 and 2 are correct. Glad you figured it out!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:56 AM, Pradeep Nayak <
>>>>>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I just figured.
>>>>>>>>>>>>>>>>>> 1. For the first time you need to use the ./buildall.sh
>>>>>>>>>>>>>>>>>> and for subsequent fe compilation I can just use maven
>>>>>>>>>>>>>>>>>> 2. It doesn't build on Macbook
>>>>>>>>>>>>>>>>>> 3. I set up a Ubuntu vm and got this compiled.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:07 AM Skye Wanderman-Milne <
>>>>>>>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hey Pradeep,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> You should try running the $IMPALA_HOME/buildall.sh
>>>>>>>>>>>>>>>>>>> script in order to build Impala. It has a lot of necessary setup to build
>>>>>>>>>>>>>>>>>>> Impala. Here are the commands from buildall.sh that should provide the
>>>>>>>>>>>>>>>>>>> impala-data-source-api jar for building the FE:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> # build the external data source API
>>>>>>>>>>>>>>>>>>> pushd ${IMPALA_HOME}/ext-data-source
>>>>>>>>>>>>>>>>>>> ${IMPALA_HOME}/bin/mvn-quiet.sh install -DskipTests
>>>>>>>>>>>>>>>>>>> popd
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> See
>>>>>>>>>>>>>>>>>>> https://github.com/cloudera/Impala/blob/cdh5-trunk/buildall.sh#L283
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hope that helps!
>>>>>>>>>>>>>>>>>>> Skye
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Sat, Apr 2, 2016 at 12:55 PM, Pradeep Nayak <
>>>>>>>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I just cloned the Impala source from github, and I am
>>>>>>>>>>>>>>>>>>>> trying to compile the front end per Impala wiki. I get the error below. I
>>>>>>>>>>>>>>>>>>>> am using maven 3.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> [ERROR] Failed to execute goal on project
>>>>>>>>>>>>>>>>>>>> impala-frontend: Could not resolve dependencies for project
>>>>>>>>>>>>>>>>>>>> com.cloudera.impala:impala-frontend:jar:0.1-SNAPSHOT: Failure to find
>>>>>>>>>>>>>>>>>>>> com.cloudera.impala:impala-data-source-api:jar:1.0-SNAPSHOT in
>>>>>>>>>>>>>>>>>>>> https://repository.cloudera.com/content/groups/cdh-releases-rcs
>>>>>>>>>>>>>>>>>>>> was cached in the local repository, resolution will not be reattempted
>>>>>>>>>>>>>>>>>>>> until the update interval of cdh.rcs.releases.repo has elapsed or updates
>>>>>>>>>>>>>>>>>>>> are forced -> [Help 1
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> This is the command I used:   mvn clean package
>>>>>>>>>>>>>>>>>>>> dependency:copy-dependencies -DskipTests=true
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Any idea getting around this problem ?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> You received this message because you are subscribed to
>>>>>>>>>>>>>>>>>>>> the Google Groups "Impala Dev" group.
>>>>>>>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving
>>>>>>>>>>>>>>>>>>>> emails from it, send an email to
>>>>>>>>>>>>>>>>>>>> impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> You received this message because you are subscribed to
>>>>>>>>>>>>>>>>>> the Google Groups "Impala Dev" group.
>>>>>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>>>>>>>>>>>>>>>>> from it, send an email to
>>>>>>>>>>>>>>>>>> impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "Impala Dev" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to impala-dev+unsubscribe@cloudera.org.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Henry Robinson
>>>>>> Software Engineer
>>>>>> Cloudera
>>>>>> 415-994-6679
>>>>>>
>>>>>
>>>>> --
>> You received this message because you are subscribed to the Google Groups
>> "Impala Dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to impala-dev+unsubscribe@cloudera.org.
>>
>
>

Re: maven error while compiling the front end code

Posted by Skye Wanderman-Milne <sk...@cloudera.com>.
Thanks for sharing Pradeep. Unfortunately I don't have time to investigate
this further right now, but I think this info will be useful for others
running a similar setup as you.

On Tue, Apr 5, 2016 at 1:45 PM, Pradeep Nayak <pr...@gmail.com> wrote:

> This is what I saw in IMPALAD.info:
>
>
>
>
>
> I0405 16:44:39.136433 10963 impala-beeswax-server.cc:172] query():
> query=describe tab1
> I0405 16:44:39.136504 10963 impala-beeswax-server.cc:515] query: Query {
>   01: query (string) = "describe tab1",
>   03: configuration (list) = list<string>[0] {
>   },
>   04: hadoop_user (string) = "root",
> }
> I0405 16:44:39.136579 10963 impala-beeswax-server.cc:537]
> TClientRequest.queryOptions: TQueryOptions {
>   01: abort_on_error (bool) = false,
>   02: max_errors (i32) = 0,
>   03: disable_codegen (bool) = false,
>   04: batch_size (i32) = 0,
>   05: num_nodes (i32) = 0,
>   06: max_scan_range_length (i64) = 0,
>   07: num_scanner_threads (i32) = 0,
>   08: max_io_buffers (i32) = 0,
>   09: allow_unsupported_formats (bool) = false,
>   10: default_order_by_limit (i64) = -1,
>   11: debug_action (string) = "",
>   12: mem_limit (i64) = 0,
>   13: abort_on_default_limit_exceeded (bool) = false,
>   15: hbase_caching (i32) = 0,
>   16: hbase_cache_blocks (bool) = false,
>   17: parquet_file_size (i64) = 0,
>   18: explain_level (i32) = 1,
>   19: sync_ddl (bool) = false,
>   23: disable_cached_reads (bool) = false,
>   24: disable_outermost_topn (bool) = false,
>   25: rm_initial_mem (i64) = 0,
>   26: query_timeout_s (i32) = 0,
>   28: appx_count_distinct (bool) = false,
>   29: disable_unsafe_spills (bool) = false,
>   31: exec_single_node_rows_threshold (i32) = 100,
> }
> I0405 16:44:39.139252 10963 Frontend.java:818] analyze query describe tab1
> E0405 16:44:39.139780 10963 Analyzer.java:2210] Failed to load metadata
> for table: tab1
> java.lang.NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
> initialize class com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
> I0405 16:44:39.140349 10963 jni-util.cc:177]
> com.cloudera.impala.common.AnalysisException:
> java.lang.NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
> initialize class com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
> at com.cloudera.impala.analysis.DescribeStmt.analyze(DescribeStmt.java:117)
> at
> com.cloudera.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:342)
> at
> com.cloudera.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:317)
> at com.cloudera.impala.service.Frontend.analyzeStmt(Frontend.java:827)
> at
> com.cloudera.impala.service.Frontend.createExecRequest(Frontend.java:856)
> at
> com.cloudera.impala.service.JniFrontend.createExecRequest(JniFrontend.java:147)
> Caused by: com.cloudera.impala.catalog.TableLoadingException:
> java.lang.NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
> initialize class com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
> at
> com.cloudera.impala.catalog.IncompleteTable.loadFromThrift(IncompleteTable.java:111)
> at com.cloudera.impala.catalog.Table.fromThrift(Table.java:240)
> at
> com.cloudera.impala.catalog.ImpaladCatalog.addTable(ImpaladCatalog.java:357)
> at
> com.cloudera.impala.catalog.ImpaladCatalog.addCatalogObject(ImpaladCatalog.java:246)
> at
> com.cloudera.impala.catalog.ImpaladCatalog.updateCatalog(ImpaladCatalog.java:132)
> at
> com.cloudera.impala.service.Frontend.updateCatalogCache(Frontend.java:223)
> at
> com.cloudera.impala.service.JniFrontend.updateCatalogCache(JniFrontend.java:164)
> at ========.<Remote stack trace on catalogd>:
> com.cloudera.impala.catalog.TableLoadingException:
> java.lang.NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
> at
> com.cloudera.impala.catalog.TableLoadingMgr$LoadRequest.get(TableLoadingMgr.java:71)
> at
> com.cloudera.impala.catalog.CatalogServiceCatalog.getOrLoadTable(CatalogServiceCatalog.java:499)
> at com.cloudera.imp
>
>
>
> ala.catalog.TableLoadingMgr.loadNextTable(TableLoadingMgr.java:287)
> at
> com.cloudera.impala.catalog.TableLoadingMgr.access$600(TableLoadingMgr.java:47)
> at
> com.cloudera.impala.catalog.TableLoadingMgr$3.run(TableLoadingMgr.java:260)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> com.cloudera.impala.catalog.TableLoadingMgr$LoadRequest.get(TableLoadingMgr.java:69)
> ... 7 more
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
> at com.cloudera.impala.catalog.HdfsTable.loadPartitions(HdfsTable.java:657)
> at com.cloudera.impala.catalog.HdfsTable.load(HdfsTable.java:1151)
> at com.cloudera.impala.catalog.TableLoader.load(TableLoader.java:84)
> at
> com.cloudera.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:232)
> at
> com.cloudera.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:229)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> ()
> I0405 16:44:39.151973 10963 status.cc:112] AnalysisException:
> java.lang.NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
> initialize class com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: TableLoadingException: java.lang.NoClassDefFoundError: Could
> not initialize class com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
> initialize class com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
>     @           0x7993f9  (unknown)
>     @           0xa325bb  (unknown)
>     @           0x96d110  (unknown)
>     @           0x98599a  (unknown)
>     @           0x990c9f  (unknown)
>     @           0x9c103b  (unknown)
>     @           0xacdce6  (unknown)
>     @           0xac2a5b  (unknown)
>     @           0x991b01  (unknown)
>     @          0x15553ea  (unknown)
>     @          0x15425ef  (unknown)
>     @          0x1544ff6  (unknown)
>     @           0x921ba9  (unknown)
>     @           0x922692  (unknown)
>     @           0xa629d5  (unknown)
>     @           0xa63884  (unknown)
>     @           0xc6f133  (unknown)
>     @     0x7fb9dad04df5  start_thread
>     @     0x7fb9d9bbf1ad  __clone
> I0405 16:44:39.152009 10963 impala-server.cc:862] UnregisterQuery():
> query_id=164d3f861927d713:b845784fc383df9b
> I0405 16:44:39.152016 10963 impala-server.cc:943] Cancel():
> query_id=164d3f861927d713:b845784fc383df9b
>
> On Tue, Apr 5, 2016 at 1:36 PM Pradeep Nayak <pr...@gmail.com>
> wrote:
>
>> As for logs,
>>
>> I just see this :
>>
>> E0405 15:19:47.907562 10957 Analyzer.java:2210] Failed to load metadata
>> for table: tab1
>> java.lang.NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>> E0405 15:20:52.074620 10957 Analyzer.java:2210] Failed to load metadata
>> for table: tab1
>> java.lang.NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>> com.cloudera.impala.util.HdfsCachingUtil
>>
>> tab1 is an external table.
>>
>> On Tue, Apr 5, 2016 at 1:24 PM Pradeep Nayak <pr...@gmail.com>
>> wrote:
>>
>>> Here is the strange part:
>>>
>>> Where we make changes to core-site.xml, re-deploy the stale client
>>> configs from the cloudera manager, and restart the affected services we get
>>> into the NoClassDefound state. In order to get past it we needed to restart
>>> impala dameons, catalog server etc again from the cloudera manager. Then
>>>  it begins to work.
>>>
>>> I can reproduce this but very inconsistently though. Is there any logs
>>> you want me to  be looking at ?
>>>
>>>
>>> On Tue, Apr 5, 2016 at 10:26 AM Skye Wanderman-Milne <sk...@cloudera.com>
>>> wrote:
>>>
>>>> In case others run into similar issues, can you share how you resolved
>>>> the NoClassDefFoundErrors?
>>>>
>>>> On Mon, Apr 4, 2016 at 10:21 PM, Henry Robinson <he...@cloudera.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On 4 April 2016 at 17:55, Pradeep Nayak <pr...@gmail.com> wrote:
>>>>>
>>>>>> Also just wanted to understand one more thing.
>>>>>> http://www.cloudera.com/documentation/enterprise/5-5-x/topics/impala_s3.html It
>>>>>> mentions that impala needs to have a default file system which is hdfs, but
>>>>>> not just s3a, or any other filesystem which extends Hadoop FileSystem class
>>>>>> ? Is there plans to support non hdfs filesystem as default ?
>>>>>>
>>>>>
>>>>> Yes there are - see IMPALA-1850 (code review
>>>>> http://gerrit.cloudera.org:8080/#/c/1121/).
>>>>>
>>>>> Henry
>>>>>
>>>>>
>>>>>>
>>>>>> Meanwhile I got around the problem earlier. I can query tables, do a
>>>>>> count(*), etc.
>>>>>>
>>>>>> However join is causing a problem...
>>>>>>
>>>>>> *select * from tab1 join tab2;*
>>>>>>
>>>>>> *Query: select * from tab1 join tab2*
>>>>>>
>>>>>> *ERROR:*
>>>>>>
>>>>>> *Failed to connect to FS*
>>>>>>
>>>>>> *Error(255): Unknown error 255*
>>>>>>
>>>>>>
>>>>>> Pradeep
>>>>>>
>>>>>> On Mon, Apr 4, 2016 at 3:31 PM Pradeep Nayak <pr...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> That's what been puzzling me as well :(. Let me get back to you with
>>>>>>> that information.
>>>>>>>
>>>>>>> Pradeep
>>>>>>>
>>>>>>> On Mon, Apr 4, 2016 at 3:28 PM Skye Wanderman-Milne <
>>>>>>> skye@cloudera.com> wrote:
>>>>>>>
>>>>>>>> Actually, I didn't read your exception carefully and didn't notice
>>>>>>>> that it can't find an Impala class, so it's probably not a dependency issue.
>>>>>>>>
>>>>>>>> Can you restart your cluster, rerun a select * query, and then
>>>>>>>> search your logs for that "NoClassDefFoundError"? That will tell us which
>>>>>>>> process the error is coming from. That class is included in the FE jar, so
>>>>>>>> I'm not sure why it wouldn't be able to find it.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Apr 4, 2016 at 3:14 PM, Pradeep Nayak <
>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Sure Skye. I didn't notice the to: list as well until you pointed
>>>>>>>>> out. I really appreciate your help here.
>>>>>>>>>
>>>>>>>>> I haven't modified the pom.xml for the front-end here:
>>>>>>>>> https://github.com/cloudera/Impala/blob/cdh5.5.2-release/fe/pom.xml
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I am just building with this at the moment. mvn clean package
>>>>>>>>> -DSkipTests = true from the fe folder.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Apr 4, 2016 at 3:11 PM Skye Wanderman-Milne <
>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>
>>>>>>>>>> Ah didn't mean to take this off the dev list. Also add our new
>>>>>>>>>> dev@impala.incubator list.
>>>>>>>>>>
>>>>>>>>>> Can you check what dependency versions you're pulling in from
>>>>>>>>>> maven? My guess is that you're somehow building with non-CDH5.5.2 frontend
>>>>>>>>>> dependencies.
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 4, 2016 at 3:04 PM, Pradeep Nayak <
>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Umm, I am trying to test this on a dev cluster which has
>>>>>>>>>>> cloudera parcels already installed. I just re-placed
>>>>>>>>>>> impala-frontend-0.1-SNAPSHOT.jar with my version. Do I need to change
>>>>>>>>>>> anything on the dev cluster ?
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 4, 2016 at 2:43 PM Skye Wanderman-Milne <
>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Try: source $IMPALA_HOME/bin/set-classpath.sh
>>>>>>>>>>>>
>>>>>>>>>>>> If you recently switched branches, you may want to close your
>>>>>>>>>>>> current terminal, open a new one, and re-source
>>>>>>>>>>>> $IMPALA_HOME/bin/impala-config.sh and set-classpath.sh.
>>>>>>>>>>>>
>>>>>>>>>>>> Then restart your cluster (start-impala-cluster.py)
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Apr 4, 2016 at 2:39 PM, Pradeep Nayak <
>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks Skye.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am in my impala shell. I get this following error. I see
>>>>>>>>>>>>> this is already compiled as part of impala-frontend. This happens only when
>>>>>>>>>>>>> I try to do a "select * from <table_name>" I can do "show tables; create
>>>>>>>>>>>>> table <table_name>" etc. Any pointers ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> ERROR: AnalysisException: java.lang.NoClassDefFoundError:
>>>>>>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>>
>>>>>>>>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError:
>>>>>>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>>
>>>>>>>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>>
>>>>>>>>>>>>> CAUSED BY: TableLoadingException:
>>>>>>>>>>>>> java.lang.NoClassDefFoundError: Could not initialize class
>>>>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>>
>>>>>>>>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError:
>>>>>>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>>
>>>>>>>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>>
>>>>>>>>>>>>> I used this tree:
>>>>>>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5.5.2-release
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 12:01 PM Skye Wanderman-Milne <
>>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think it should be
>>>>>>>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5-2.3.0_5.5.2
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 12:00 PM, Pradeep Nayak <
>>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also to what release this parcel  *5.5.2-1.cdh5.5.2.p0.4 *map
>>>>>>>>>>>>>>> to on github ? I am unable to find a tag/branch for this parcel type.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:58 AM Skye Wanderman-Milne <
>>>>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ah yes, both 1 and 2 are correct. Glad you figured it out!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:56 AM, Pradeep Nayak <
>>>>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I just figured.
>>>>>>>>>>>>>>>>> 1. For the first time you need to use the ./buildall.sh
>>>>>>>>>>>>>>>>> and for subsequent fe compilation I can just use maven
>>>>>>>>>>>>>>>>> 2. It doesn't build on Macbook
>>>>>>>>>>>>>>>>> 3. I set up a Ubuntu vm and got this compiled.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:07 AM Skye Wanderman-Milne <
>>>>>>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hey Pradeep,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> You should try running the $IMPALA_HOME/buildall.sh
>>>>>>>>>>>>>>>>>> script in order to build Impala. It has a lot of necessary setup to build
>>>>>>>>>>>>>>>>>> Impala. Here are the commands from buildall.sh that should provide the
>>>>>>>>>>>>>>>>>> impala-data-source-api jar for building the FE:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> # build the external data source API
>>>>>>>>>>>>>>>>>> pushd ${IMPALA_HOME}/ext-data-source
>>>>>>>>>>>>>>>>>> ${IMPALA_HOME}/bin/mvn-quiet.sh install -DskipTests
>>>>>>>>>>>>>>>>>> popd
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> See
>>>>>>>>>>>>>>>>>> https://github.com/cloudera/Impala/blob/cdh5-trunk/buildall.sh#L283
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hope that helps!
>>>>>>>>>>>>>>>>>> Skye
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Sat, Apr 2, 2016 at 12:55 PM, Pradeep Nayak <
>>>>>>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I just cloned the Impala source from github, and I am
>>>>>>>>>>>>>>>>>>> trying to compile the front end per Impala wiki. I get the error below. I
>>>>>>>>>>>>>>>>>>> am using maven 3.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [ERROR] Failed to execute goal on project
>>>>>>>>>>>>>>>>>>> impala-frontend: Could not resolve dependencies for project
>>>>>>>>>>>>>>>>>>> com.cloudera.impala:impala-frontend:jar:0.1-SNAPSHOT: Failure to find
>>>>>>>>>>>>>>>>>>> com.cloudera.impala:impala-data-source-api:jar:1.0-SNAPSHOT in
>>>>>>>>>>>>>>>>>>> https://repository.cloudera.com/content/groups/cdh-releases-rcs
>>>>>>>>>>>>>>>>>>> was cached in the local repository, resolution will not be reattempted
>>>>>>>>>>>>>>>>>>> until the update interval of cdh.rcs.releases.repo has elapsed or updates
>>>>>>>>>>>>>>>>>>> are forced -> [Help 1
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This is the command I used:   mvn clean package
>>>>>>>>>>>>>>>>>>> dependency:copy-dependencies -DskipTests=true
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Any idea getting around this problem ?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> You received this message because you are subscribed to
>>>>>>>>>>>>>>>>>>> the Google Groups "Impala Dev" group.
>>>>>>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>>>>>>>>>>>>>>>>>> from it, send an email to
>>>>>>>>>>>>>>>>>>> impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> You received this message because you are subscribed to
>>>>>>>>>>>>>>>>> the Google Groups "Impala Dev" group.
>>>>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>>>>>>>>>>>>>>>> from it, send an email to
>>>>>>>>>>>>>>>>> impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Impala Dev" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to impala-dev+unsubscribe@cloudera.org.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Henry Robinson
>>>>> Software Engineer
>>>>> Cloudera
>>>>> 415-994-6679
>>>>>
>>>>
>>>> --
> You received this message because you are subscribed to the Google Groups
> "Impala Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to impala-dev+unsubscribe@cloudera.org.
>

Re: maven error while compiling the front end code

Posted by Pradeep Nayak <pr...@gmail.com>.
This is what I saw in IMPALAD.info:





I0405 16:44:39.136433 10963 impala-beeswax-server.cc:172] query():
query=describe tab1
I0405 16:44:39.136504 10963 impala-beeswax-server.cc:515] query: Query {
  01: query (string) = "describe tab1",
  03: configuration (list) = list<string>[0] {
  },
  04: hadoop_user (string) = "root",
}
I0405 16:44:39.136579 10963 impala-beeswax-server.cc:537]
TClientRequest.queryOptions: TQueryOptions {
  01: abort_on_error (bool) = false,
  02: max_errors (i32) = 0,
  03: disable_codegen (bool) = false,
  04: batch_size (i32) = 0,
  05: num_nodes (i32) = 0,
  06: max_scan_range_length (i64) = 0,
  07: num_scanner_threads (i32) = 0,
  08: max_io_buffers (i32) = 0,
  09: allow_unsupported_formats (bool) = false,
  10: default_order_by_limit (i64) = -1,
  11: debug_action (string) = "",
  12: mem_limit (i64) = 0,
  13: abort_on_default_limit_exceeded (bool) = false,
  15: hbase_caching (i32) = 0,
  16: hbase_cache_blocks (bool) = false,
  17: parquet_file_size (i64) = 0,
  18: explain_level (i32) = 1,
  19: sync_ddl (bool) = false,
  23: disable_cached_reads (bool) = false,
  24: disable_outermost_topn (bool) = false,
  25: rm_initial_mem (i64) = 0,
  26: query_timeout_s (i32) = 0,
  28: appx_count_distinct (bool) = false,
  29: disable_unsafe_spills (bool) = false,
  31: exec_single_node_rows_threshold (i32) = 100,
}
I0405 16:44:39.139252 10963 Frontend.java:818] analyze query describe tab1
E0405 16:44:39.139780 10963 Analyzer.java:2210] Failed to load metadata for
table: tab1
java.lang.NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
initialize class com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
I0405 16:44:39.140349 10963 jni-util.cc:177]
com.cloudera.impala.common.AnalysisException:
java.lang.NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
initialize class com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
at com.cloudera.impala.analysis.DescribeStmt.analyze(DescribeStmt.java:117)
at
com.cloudera.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:342)
at
com.cloudera.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:317)
at com.cloudera.impala.service.Frontend.analyzeStmt(Frontend.java:827)
at com.cloudera.impala.service.Frontend.createExecRequest(Frontend.java:856)
at
com.cloudera.impala.service.JniFrontend.createExecRequest(JniFrontend.java:147)
Caused by: com.cloudera.impala.catalog.TableLoadingException:
java.lang.NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
initialize class com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
at
com.cloudera.impala.catalog.IncompleteTable.loadFromThrift(IncompleteTable.java:111)
at com.cloudera.impala.catalog.Table.fromThrift(Table.java:240)
at
com.cloudera.impala.catalog.ImpaladCatalog.addTable(ImpaladCatalog.java:357)
at
com.cloudera.impala.catalog.ImpaladCatalog.addCatalogObject(ImpaladCatalog.java:246)
at
com.cloudera.impala.catalog.ImpaladCatalog.updateCatalog(ImpaladCatalog.java:132)
at
com.cloudera.impala.service.Frontend.updateCatalogCache(Frontend.java:223)
at
com.cloudera.impala.service.JniFrontend.updateCatalogCache(JniFrontend.java:164)
at ========.<Remote stack trace on catalogd>:
com.cloudera.impala.catalog.TableLoadingException:
java.lang.NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
at
com.cloudera.impala.catalog.TableLoadingMgr$LoadRequest.get(TableLoadingMgr.java:71)
at
com.cloudera.impala.catalog.CatalogServiceCatalog.getOrLoadTable(CatalogServiceCatalog.java:499)
at com.cloudera.imp



ala.catalog.TableLoadingMgr.loadNextTable(TableLoadingMgr.java:287)
at
com.cloudera.impala.catalog.TableLoadingMgr.access$600(TableLoadingMgr.java:47)
at
com.cloudera.impala.catalog.TableLoadingMgr$3.run(TableLoadingMgr.java:260)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException:
java.lang.NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
com.cloudera.impala.catalog.TableLoadingMgr$LoadRequest.get(TableLoadingMgr.java:69)
... 7 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
at com.cloudera.impala.catalog.HdfsTable.loadPartitions(HdfsTable.java:657)
at com.cloudera.impala.catalog.HdfsTable.load(HdfsTable.java:1151)
at com.cloudera.impala.catalog.TableLoader.load(TableLoader.java:84)
at
com.cloudera.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:232)
at
com.cloudera.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:229)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
... 3 more
()
I0405 16:44:39.151973 10963 status.cc:112] AnalysisException:
java.lang.NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
initialize class com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: TableLoadingException: java.lang.NoClassDefFoundError: Could not
initialize class com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
initialize class com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
    @           0x7993f9  (unknown)
    @           0xa325bb  (unknown)
    @           0x96d110  (unknown)
    @           0x98599a  (unknown)
    @           0x990c9f  (unknown)
    @           0x9c103b  (unknown)
    @           0xacdce6  (unknown)
    @           0xac2a5b  (unknown)
    @           0x991b01  (unknown)
    @          0x15553ea  (unknown)
    @          0x15425ef  (unknown)
    @          0x1544ff6  (unknown)
    @           0x921ba9  (unknown)
    @           0x922692  (unknown)
    @           0xa629d5  (unknown)
    @           0xa63884  (unknown)
    @           0xc6f133  (unknown)
    @     0x7fb9dad04df5  start_thread
    @     0x7fb9d9bbf1ad  __clone
I0405 16:44:39.152009 10963 impala-server.cc:862] UnregisterQuery():
query_id=164d3f861927d713:b845784fc383df9b
I0405 16:44:39.152016 10963 impala-server.cc:943] Cancel():
query_id=164d3f861927d713:b845784fc383df9b

On Tue, Apr 5, 2016 at 1:36 PM Pradeep Nayak <pr...@gmail.com> wrote:

> As for logs,
>
> I just see this :
>
> E0405 15:19:47.907562 10957 Analyzer.java:2210] Failed to load metadata
> for table: tab1
> java.lang.NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
> initialize class com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
> E0405 15:20:52.074620 10957 Analyzer.java:2210] Failed to load metadata
> for table: tab1
> java.lang.NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
> initialize class com.cloudera.impala.util.HdfsCachingUtil
> CAUSED BY: NoClassDefFoundError: Could not initialize class
> com.cloudera.impala.util.HdfsCachingUtil
>
> tab1 is an external table.
>
> On Tue, Apr 5, 2016 at 1:24 PM Pradeep Nayak <pr...@gmail.com>
> wrote:
>
>> Here is the strange part:
>>
>> Where we make changes to core-site.xml, re-deploy the stale client
>> configs from the cloudera manager, and restart the affected services we get
>> into the NoClassDefound state. In order to get past it we needed to restart
>> impala dameons, catalog server etc again from the cloudera manager. Then
>>  it begins to work.
>>
>> I can reproduce this but very inconsistently though. Is there any logs
>> you want me to  be looking at ?
>>
>>
>> On Tue, Apr 5, 2016 at 10:26 AM Skye Wanderman-Milne <sk...@cloudera.com>
>> wrote:
>>
>>> In case others run into similar issues, can you share how you resolved
>>> the NoClassDefFoundErrors?
>>>
>>> On Mon, Apr 4, 2016 at 10:21 PM, Henry Robinson <he...@cloudera.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On 4 April 2016 at 17:55, Pradeep Nayak <pr...@gmail.com> wrote:
>>>>
>>>>> Also just wanted to understand one more thing.
>>>>> http://www.cloudera.com/documentation/enterprise/5-5-x/topics/impala_s3.html It
>>>>> mentions that impala needs to have a default file system which is hdfs, but
>>>>> not just s3a, or any other filesystem which extends Hadoop FileSystem class
>>>>> ? Is there plans to support non hdfs filesystem as default ?
>>>>>
>>>>
>>>> Yes there are - see IMPALA-1850 (code review
>>>> http://gerrit.cloudera.org:8080/#/c/1121/).
>>>>
>>>> Henry
>>>>
>>>>
>>>>>
>>>>> Meanwhile I got around the problem earlier. I can query tables, do a
>>>>> count(*), etc.
>>>>>
>>>>> However join is causing a problem...
>>>>>
>>>>> *select * from tab1 join tab2;*
>>>>>
>>>>> *Query: select * from tab1 join tab2*
>>>>>
>>>>> *ERROR:*
>>>>>
>>>>> *Failed to connect to FS*
>>>>>
>>>>> *Error(255): Unknown error 255*
>>>>>
>>>>>
>>>>> Pradeep
>>>>>
>>>>> On Mon, Apr 4, 2016 at 3:31 PM Pradeep Nayak <pr...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> That's what been puzzling me as well :(. Let me get back to you with
>>>>>> that information.
>>>>>>
>>>>>> Pradeep
>>>>>>
>>>>>> On Mon, Apr 4, 2016 at 3:28 PM Skye Wanderman-Milne <
>>>>>> skye@cloudera.com> wrote:
>>>>>>
>>>>>>> Actually, I didn't read your exception carefully and didn't notice
>>>>>>> that it can't find an Impala class, so it's probably not a dependency issue.
>>>>>>>
>>>>>>> Can you restart your cluster, rerun a select * query, and then
>>>>>>> search your logs for that "NoClassDefFoundError"? That will tell us which
>>>>>>> process the error is coming from. That class is included in the FE jar, so
>>>>>>> I'm not sure why it wouldn't be able to find it.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Apr 4, 2016 at 3:14 PM, Pradeep Nayak <pradeep1288@gmail.com
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Sure Skye. I didn't notice the to: list as well until you pointed
>>>>>>>> out. I really appreciate your help here.
>>>>>>>>
>>>>>>>> I haven't modified the pom.xml for the front-end here:
>>>>>>>> https://github.com/cloudera/Impala/blob/cdh5.5.2-release/fe/pom.xml
>>>>>>>>
>>>>>>>>
>>>>>>>> I am just building with this at the moment. mvn clean package
>>>>>>>> -DSkipTests = true from the fe folder.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Apr 4, 2016 at 3:11 PM Skye Wanderman-Milne <
>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>
>>>>>>>>> Ah didn't mean to take this off the dev list. Also add our new
>>>>>>>>> dev@impala.incubator list.
>>>>>>>>>
>>>>>>>>> Can you check what dependency versions you're pulling in from
>>>>>>>>> maven? My guess is that you're somehow building with non-CDH5.5.2 frontend
>>>>>>>>> dependencies.
>>>>>>>>>
>>>>>>>>> On Mon, Apr 4, 2016 at 3:04 PM, Pradeep Nayak <
>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Umm, I am trying to test this on a dev cluster which has cloudera
>>>>>>>>>> parcels already installed. I just re-placed
>>>>>>>>>> impala-frontend-0.1-SNAPSHOT.jar with my version. Do I need to change
>>>>>>>>>> anything on the dev cluster ?
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 4, 2016 at 2:43 PM Skye Wanderman-Milne <
>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Try: source $IMPALA_HOME/bin/set-classpath.sh
>>>>>>>>>>>
>>>>>>>>>>> If you recently switched branches, you may want to close your
>>>>>>>>>>> current terminal, open a new one, and re-source
>>>>>>>>>>> $IMPALA_HOME/bin/impala-config.sh and set-classpath.sh.
>>>>>>>>>>>
>>>>>>>>>>> Then restart your cluster (start-impala-cluster.py)
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 4, 2016 at 2:39 PM, Pradeep Nayak <
>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thanks Skye.
>>>>>>>>>>>>
>>>>>>>>>>>> I am in my impala shell. I get this following error. I see this
>>>>>>>>>>>> is already compiled as part of impala-frontend. This happens only when I
>>>>>>>>>>>> try to do a "select * from <table_name>" I can do "show tables; create
>>>>>>>>>>>> table <table_name>" etc. Any pointers ?
>>>>>>>>>>>>
>>>>>>>>>>>> ERROR: AnalysisException: java.lang.NoClassDefFoundError: Could
>>>>>>>>>>>> not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>
>>>>>>>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError:
>>>>>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>
>>>>>>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>
>>>>>>>>>>>> CAUSED BY: TableLoadingException:
>>>>>>>>>>>> java.lang.NoClassDefFoundError: Could not initialize class
>>>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>
>>>>>>>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError:
>>>>>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>
>>>>>>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>>
>>>>>>>>>>>> I used this tree:
>>>>>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5.5.2-release
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Apr 4, 2016 at 12:01 PM Skye Wanderman-Milne <
>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I think it should be
>>>>>>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5-2.3.0_5.5.2
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 12:00 PM, Pradeep Nayak <
>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Also to what release this parcel  *5.5.2-1.cdh5.5.2.p0.4 *map
>>>>>>>>>>>>>> to on github ? I am unable to find a tag/branch for this parcel type.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:58 AM Skye Wanderman-Milne <
>>>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ah yes, both 1 and 2 are correct. Glad you figured it out!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:56 AM, Pradeep Nayak <
>>>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I just figured.
>>>>>>>>>>>>>>>> 1. For the first time you need to use the ./buildall.sh and
>>>>>>>>>>>>>>>> for subsequent fe compilation I can just use maven
>>>>>>>>>>>>>>>> 2. It doesn't build on Macbook
>>>>>>>>>>>>>>>> 3. I set up a Ubuntu vm and got this compiled.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:07 AM Skye Wanderman-Milne <
>>>>>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hey Pradeep,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> You should try running the $IMPALA_HOME/buildall.sh script
>>>>>>>>>>>>>>>>> in order to build Impala. It has a lot of necessary setup to build Impala.
>>>>>>>>>>>>>>>>> Here are the commands from buildall.sh that should provide the
>>>>>>>>>>>>>>>>> impala-data-source-api jar for building the FE:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> # build the external data source API
>>>>>>>>>>>>>>>>> pushd ${IMPALA_HOME}/ext-data-source
>>>>>>>>>>>>>>>>> ${IMPALA_HOME}/bin/mvn-quiet.sh install -DskipTests
>>>>>>>>>>>>>>>>> popd
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> See
>>>>>>>>>>>>>>>>> https://github.com/cloudera/Impala/blob/cdh5-trunk/buildall.sh#L283
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hope that helps!
>>>>>>>>>>>>>>>>> Skye
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Sat, Apr 2, 2016 at 12:55 PM, Pradeep Nayak <
>>>>>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I just cloned the Impala source from github, and I am
>>>>>>>>>>>>>>>>>> trying to compile the front end per Impala wiki. I get the error below. I
>>>>>>>>>>>>>>>>>> am using maven 3.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> [ERROR] Failed to execute goal on project
>>>>>>>>>>>>>>>>>> impala-frontend: Could not resolve dependencies for project
>>>>>>>>>>>>>>>>>> com.cloudera.impala:impala-frontend:jar:0.1-SNAPSHOT: Failure to find
>>>>>>>>>>>>>>>>>> com.cloudera.impala:impala-data-source-api:jar:1.0-SNAPSHOT in
>>>>>>>>>>>>>>>>>> https://repository.cloudera.com/content/groups/cdh-releases-rcs
>>>>>>>>>>>>>>>>>> was cached in the local repository, resolution will not be reattempted
>>>>>>>>>>>>>>>>>> until the update interval of cdh.rcs.releases.repo has elapsed or updates
>>>>>>>>>>>>>>>>>> are forced -> [Help 1
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> This is the command I used:   mvn clean package
>>>>>>>>>>>>>>>>>> dependency:copy-dependencies -DskipTests=true
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Any idea getting around this problem ?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> You received this message because you are subscribed to
>>>>>>>>>>>>>>>>>> the Google Groups "Impala Dev" group.
>>>>>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>>>>>>>>>>>>>>>>> from it, send an email to
>>>>>>>>>>>>>>>>>> impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>>>>>> Google Groups "Impala Dev" group.
>>>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>>>>>>>>>>>>>>> from it, send an email to
>>>>>>>>>>>>>>>> impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Impala Dev" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to impala-dev+unsubscribe@cloudera.org.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Henry Robinson
>>>> Software Engineer
>>>> Cloudera
>>>> 415-994-6679
>>>>
>>>
>>>

Re: maven error while compiling the front end code

Posted by Pradeep Nayak <pr...@gmail.com>.
As for logs,

I just see this :

E0405 15:19:47.907562 10957 Analyzer.java:2210] Failed to load metadata for
table: tab1
java.lang.NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
initialize class com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
E0405 15:20:52.074620 10957 Analyzer.java:2210] Failed to load metadata for
table: tab1
java.lang.NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could not
initialize class com.cloudera.impala.util.HdfsCachingUtil
CAUSED BY: NoClassDefFoundError: Could not initialize class
com.cloudera.impala.util.HdfsCachingUtil

tab1 is an external table.

On Tue, Apr 5, 2016 at 1:24 PM Pradeep Nayak <pr...@gmail.com> wrote:

> Here is the strange part:
>
> Where we make changes to core-site.xml, re-deploy the stale client configs
> from the cloudera manager, and restart the affected services we get into
> the NoClassDefound state. In order to get past it we needed to restart
> impala dameons, catalog server etc again from the cloudera manager. Then
>  it begins to work.
>
> I can reproduce this but very inconsistently though. Is there any logs you
> want me to  be looking at ?
>
>
> On Tue, Apr 5, 2016 at 10:26 AM Skye Wanderman-Milne <sk...@cloudera.com>
> wrote:
>
>> In case others run into similar issues, can you share how you resolved
>> the NoClassDefFoundErrors?
>>
>> On Mon, Apr 4, 2016 at 10:21 PM, Henry Robinson <he...@cloudera.com>
>> wrote:
>>
>>>
>>>
>>> On 4 April 2016 at 17:55, Pradeep Nayak <pr...@gmail.com> wrote:
>>>
>>>> Also just wanted to understand one more thing.
>>>> http://www.cloudera.com/documentation/enterprise/5-5-x/topics/impala_s3.html It
>>>> mentions that impala needs to have a default file system which is hdfs, but
>>>> not just s3a, or any other filesystem which extends Hadoop FileSystem class
>>>> ? Is there plans to support non hdfs filesystem as default ?
>>>>
>>>
>>> Yes there are - see IMPALA-1850 (code review
>>> http://gerrit.cloudera.org:8080/#/c/1121/).
>>>
>>> Henry
>>>
>>>
>>>>
>>>> Meanwhile I got around the problem earlier. I can query tables, do a
>>>> count(*), etc.
>>>>
>>>> However join is causing a problem...
>>>>
>>>> *select * from tab1 join tab2;*
>>>>
>>>> *Query: select * from tab1 join tab2*
>>>>
>>>> *ERROR:*
>>>>
>>>> *Failed to connect to FS*
>>>>
>>>> *Error(255): Unknown error 255*
>>>>
>>>>
>>>> Pradeep
>>>>
>>>> On Mon, Apr 4, 2016 at 3:31 PM Pradeep Nayak <pr...@gmail.com>
>>>> wrote:
>>>>
>>>>> That's what been puzzling me as well :(. Let me get back to you with
>>>>> that information.
>>>>>
>>>>> Pradeep
>>>>>
>>>>> On Mon, Apr 4, 2016 at 3:28 PM Skye Wanderman-Milne <sk...@cloudera.com>
>>>>> wrote:
>>>>>
>>>>>> Actually, I didn't read your exception carefully and didn't notice
>>>>>> that it can't find an Impala class, so it's probably not a dependency issue.
>>>>>>
>>>>>> Can you restart your cluster, rerun a select * query, and then search
>>>>>> your logs for that "NoClassDefFoundError"? That will tell us which process
>>>>>> the error is coming from. That class is included in the FE jar, so I'm not
>>>>>> sure why it wouldn't be able to find it.
>>>>>>
>>>>>>
>>>>>> On Mon, Apr 4, 2016 at 3:14 PM, Pradeep Nayak <pr...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Sure Skye. I didn't notice the to: list as well until you pointed
>>>>>>> out. I really appreciate your help here.
>>>>>>>
>>>>>>> I haven't modified the pom.xml for the front-end here:
>>>>>>> https://github.com/cloudera/Impala/blob/cdh5.5.2-release/fe/pom.xml
>>>>>>>
>>>>>>> I am just building with this at the moment. mvn clean package
>>>>>>> -DSkipTests = true from the fe folder.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Apr 4, 2016 at 3:11 PM Skye Wanderman-Milne <
>>>>>>> skye@cloudera.com> wrote:
>>>>>>>
>>>>>>>> Ah didn't mean to take this off the dev list. Also add our new
>>>>>>>> dev@impala.incubator list.
>>>>>>>>
>>>>>>>> Can you check what dependency versions you're pulling in from
>>>>>>>> maven? My guess is that you're somehow building with non-CDH5.5.2 frontend
>>>>>>>> dependencies.
>>>>>>>>
>>>>>>>> On Mon, Apr 4, 2016 at 3:04 PM, Pradeep Nayak <
>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Umm, I am trying to test this on a dev cluster which has cloudera
>>>>>>>>> parcels already installed. I just re-placed
>>>>>>>>> impala-frontend-0.1-SNAPSHOT.jar with my version. Do I need to change
>>>>>>>>> anything on the dev cluster ?
>>>>>>>>>
>>>>>>>>> On Mon, Apr 4, 2016 at 2:43 PM Skye Wanderman-Milne <
>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>
>>>>>>>>>> Try: source $IMPALA_HOME/bin/set-classpath.sh
>>>>>>>>>>
>>>>>>>>>> If you recently switched branches, you may want to close your
>>>>>>>>>> current terminal, open a new one, and re-source
>>>>>>>>>> $IMPALA_HOME/bin/impala-config.sh and set-classpath.sh.
>>>>>>>>>>
>>>>>>>>>> Then restart your cluster (start-impala-cluster.py)
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 4, 2016 at 2:39 PM, Pradeep Nayak <
>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks Skye.
>>>>>>>>>>>
>>>>>>>>>>> I am in my impala shell. I get this following error. I see this
>>>>>>>>>>> is already compiled as part of impala-frontend. This happens only when I
>>>>>>>>>>> try to do a "select * from <table_name>" I can do "show tables; create
>>>>>>>>>>> table <table_name>" etc. Any pointers ?
>>>>>>>>>>>
>>>>>>>>>>> ERROR: AnalysisException: java.lang.NoClassDefFoundError: Could
>>>>>>>>>>> not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>
>>>>>>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError:
>>>>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>
>>>>>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>
>>>>>>>>>>> CAUSED BY: TableLoadingException:
>>>>>>>>>>> java.lang.NoClassDefFoundError: Could not initialize class
>>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>
>>>>>>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError:
>>>>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>
>>>>>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>>
>>>>>>>>>>> I used this tree:
>>>>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5.5.2-release
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 4, 2016 at 12:01 PM Skye Wanderman-Milne <
>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I think it should be
>>>>>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5-2.3.0_5.5.2
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Apr 4, 2016 at 12:00 PM, Pradeep Nayak <
>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Also to what release this parcel  *5.5.2-1.cdh5.5.2.p0.4 *map
>>>>>>>>>>>>> to on github ? I am unable to find a tag/branch for this parcel type.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:58 AM Skye Wanderman-Milne <
>>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ah yes, both 1 and 2 are correct. Glad you figured it out!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:56 AM, Pradeep Nayak <
>>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I just figured.
>>>>>>>>>>>>>>> 1. For the first time you need to use the ./buildall.sh and
>>>>>>>>>>>>>>> for subsequent fe compilation I can just use maven
>>>>>>>>>>>>>>> 2. It doesn't build on Macbook
>>>>>>>>>>>>>>> 3. I set up a Ubuntu vm and got this compiled.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:07 AM Skye Wanderman-Milne <
>>>>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hey Pradeep,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You should try running the $IMPALA_HOME/buildall.sh script
>>>>>>>>>>>>>>>> in order to build Impala. It has a lot of necessary setup to build Impala.
>>>>>>>>>>>>>>>> Here are the commands from buildall.sh that should provide the
>>>>>>>>>>>>>>>> impala-data-source-api jar for building the FE:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> # build the external data source API
>>>>>>>>>>>>>>>> pushd ${IMPALA_HOME}/ext-data-source
>>>>>>>>>>>>>>>> ${IMPALA_HOME}/bin/mvn-quiet.sh install -DskipTests
>>>>>>>>>>>>>>>> popd
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> See
>>>>>>>>>>>>>>>> https://github.com/cloudera/Impala/blob/cdh5-trunk/buildall.sh#L283
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hope that helps!
>>>>>>>>>>>>>>>> Skye
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Sat, Apr 2, 2016 at 12:55 PM, Pradeep Nayak <
>>>>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I just cloned the Impala source from github, and I am
>>>>>>>>>>>>>>>>> trying to compile the front end per Impala wiki. I get the error below. I
>>>>>>>>>>>>>>>>> am using maven 3.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> [ERROR] Failed to execute goal on project impala-frontend:
>>>>>>>>>>>>>>>>> Could not resolve dependencies for project
>>>>>>>>>>>>>>>>> com.cloudera.impala:impala-frontend:jar:0.1-SNAPSHOT: Failure to find
>>>>>>>>>>>>>>>>> com.cloudera.impala:impala-data-source-api:jar:1.0-SNAPSHOT in
>>>>>>>>>>>>>>>>> https://repository.cloudera.com/content/groups/cdh-releases-rcs
>>>>>>>>>>>>>>>>> was cached in the local repository, resolution will not be reattempted
>>>>>>>>>>>>>>>>> until the update interval of cdh.rcs.releases.repo has elapsed or updates
>>>>>>>>>>>>>>>>> are forced -> [Help 1
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This is the command I used:   mvn clean package
>>>>>>>>>>>>>>>>> dependency:copy-dependencies -DskipTests=true
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Any idea getting around this problem ?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> You received this message because you are subscribed to
>>>>>>>>>>>>>>>>> the Google Groups "Impala Dev" group.
>>>>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>>>>>>>>>>>>>>>> from it, send an email to
>>>>>>>>>>>>>>>>> impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>>>>> Google Groups "Impala Dev" group.
>>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>>>>>>>>>>>>>> from it, send an email to
>>>>>>>>>>>>>>> impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Impala Dev" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to impala-dev+unsubscribe@cloudera.org.
>>>>
>>>
>>>
>>>
>>> --
>>> Henry Robinson
>>> Software Engineer
>>> Cloudera
>>> 415-994-6679
>>>
>>
>>

Re: maven error while compiling the front end code

Posted by Pradeep Nayak <pr...@gmail.com>.
Here is the strange part:

Where we make changes to core-site.xml, re-deploy the stale client configs
from the cloudera manager, and restart the affected services we get into
the NoClassDefound state. In order to get past it we needed to restart
impala dameons, catalog server etc again from the cloudera manager. Then
 it begins to work.

I can reproduce this but very inconsistently though. Is there any logs you
want me to  be looking at ?


On Tue, Apr 5, 2016 at 10:26 AM Skye Wanderman-Milne <sk...@cloudera.com>
wrote:

> In case others run into similar issues, can you share how you resolved the
> NoClassDefFoundErrors?
>
> On Mon, Apr 4, 2016 at 10:21 PM, Henry Robinson <he...@cloudera.com>
> wrote:
>
>>
>>
>> On 4 April 2016 at 17:55, Pradeep Nayak <pr...@gmail.com> wrote:
>>
>>> Also just wanted to understand one more thing.
>>> http://www.cloudera.com/documentation/enterprise/5-5-x/topics/impala_s3.html It
>>> mentions that impala needs to have a default file system which is hdfs, but
>>> not just s3a, or any other filesystem which extends Hadoop FileSystem class
>>> ? Is there plans to support non hdfs filesystem as default ?
>>>
>>
>> Yes there are - see IMPALA-1850 (code review
>> http://gerrit.cloudera.org:8080/#/c/1121/).
>>
>> Henry
>>
>>
>>>
>>> Meanwhile I got around the problem earlier. I can query tables, do a
>>> count(*), etc.
>>>
>>> However join is causing a problem...
>>>
>>> *select * from tab1 join tab2;*
>>>
>>> *Query: select * from tab1 join tab2*
>>>
>>> *ERROR:*
>>>
>>> *Failed to connect to FS*
>>>
>>> *Error(255): Unknown error 255*
>>>
>>>
>>> Pradeep
>>>
>>> On Mon, Apr 4, 2016 at 3:31 PM Pradeep Nayak <pr...@gmail.com>
>>> wrote:
>>>
>>>> That's what been puzzling me as well :(. Let me get back to you with
>>>> that information.
>>>>
>>>> Pradeep
>>>>
>>>> On Mon, Apr 4, 2016 at 3:28 PM Skye Wanderman-Milne <sk...@cloudera.com>
>>>> wrote:
>>>>
>>>>> Actually, I didn't read your exception carefully and didn't notice
>>>>> that it can't find an Impala class, so it's probably not a dependency issue.
>>>>>
>>>>> Can you restart your cluster, rerun a select * query, and then search
>>>>> your logs for that "NoClassDefFoundError"? That will tell us which process
>>>>> the error is coming from. That class is included in the FE jar, so I'm not
>>>>> sure why it wouldn't be able to find it.
>>>>>
>>>>>
>>>>> On Mon, Apr 4, 2016 at 3:14 PM, Pradeep Nayak <pr...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Sure Skye. I didn't notice the to: list as well until you pointed
>>>>>> out. I really appreciate your help here.
>>>>>>
>>>>>> I haven't modified the pom.xml for the front-end here:
>>>>>> https://github.com/cloudera/Impala/blob/cdh5.5.2-release/fe/pom.xml
>>>>>>
>>>>>> I am just building with this at the moment. mvn clean package
>>>>>> -DSkipTests = true from the fe folder.
>>>>>>
>>>>>>
>>>>>> On Mon, Apr 4, 2016 at 3:11 PM Skye Wanderman-Milne <
>>>>>> skye@cloudera.com> wrote:
>>>>>>
>>>>>>> Ah didn't mean to take this off the dev list. Also add our new
>>>>>>> dev@impala.incubator list.
>>>>>>>
>>>>>>> Can you check what dependency versions you're pulling in from maven?
>>>>>>> My guess is that you're somehow building with non-CDH5.5.2 frontend
>>>>>>> dependencies.
>>>>>>>
>>>>>>> On Mon, Apr 4, 2016 at 3:04 PM, Pradeep Nayak <pradeep1288@gmail.com
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Umm, I am trying to test this on a dev cluster which has cloudera
>>>>>>>> parcels already installed. I just re-placed
>>>>>>>> impala-frontend-0.1-SNAPSHOT.jar with my version. Do I need to change
>>>>>>>> anything on the dev cluster ?
>>>>>>>>
>>>>>>>> On Mon, Apr 4, 2016 at 2:43 PM Skye Wanderman-Milne <
>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>
>>>>>>>>> Try: source $IMPALA_HOME/bin/set-classpath.sh
>>>>>>>>>
>>>>>>>>> If you recently switched branches, you may want to close your
>>>>>>>>> current terminal, open a new one, and re-source
>>>>>>>>> $IMPALA_HOME/bin/impala-config.sh and set-classpath.sh.
>>>>>>>>>
>>>>>>>>> Then restart your cluster (start-impala-cluster.py)
>>>>>>>>>
>>>>>>>>> On Mon, Apr 4, 2016 at 2:39 PM, Pradeep Nayak <
>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks Skye.
>>>>>>>>>>
>>>>>>>>>> I am in my impala shell. I get this following error. I see this
>>>>>>>>>> is already compiled as part of impala-frontend. This happens only when I
>>>>>>>>>> try to do a "select * from <table_name>" I can do "show tables; create
>>>>>>>>>> table <table_name>" etc. Any pointers ?
>>>>>>>>>>
>>>>>>>>>> ERROR: AnalysisException: java.lang.NoClassDefFoundError: Could
>>>>>>>>>> not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>
>>>>>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError:
>>>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>
>>>>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>
>>>>>>>>>> CAUSED BY: TableLoadingException: java.lang.NoClassDefFoundError:
>>>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>
>>>>>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError:
>>>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>
>>>>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>>
>>>>>>>>>> I used this tree:
>>>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5.5.2-release
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 4, 2016 at 12:01 PM Skye Wanderman-Milne <
>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I think it should be
>>>>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5-2.3.0_5.5.2
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 4, 2016 at 12:00 PM, Pradeep Nayak <
>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Also to what release this parcel  *5.5.2-1.cdh5.5.2.p0.4 *map
>>>>>>>>>>>> to on github ? I am unable to find a tag/branch for this parcel type.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:58 AM Skye Wanderman-Milne <
>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Ah yes, both 1 and 2 are correct. Glad you figured it out!
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:56 AM, Pradeep Nayak <
>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I just figured.
>>>>>>>>>>>>>> 1. For the first time you need to use the ./buildall.sh and
>>>>>>>>>>>>>> for subsequent fe compilation I can just use maven
>>>>>>>>>>>>>> 2. It doesn't build on Macbook
>>>>>>>>>>>>>> 3. I set up a Ubuntu vm and got this compiled.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:07 AM Skye Wanderman-Milne <
>>>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hey Pradeep,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You should try running the $IMPALA_HOME/buildall.sh script
>>>>>>>>>>>>>>> in order to build Impala. It has a lot of necessary setup to build Impala.
>>>>>>>>>>>>>>> Here are the commands from buildall.sh that should provide the
>>>>>>>>>>>>>>> impala-data-source-api jar for building the FE:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> # build the external data source API
>>>>>>>>>>>>>>> pushd ${IMPALA_HOME}/ext-data-source
>>>>>>>>>>>>>>> ${IMPALA_HOME}/bin/mvn-quiet.sh install -DskipTests
>>>>>>>>>>>>>>> popd
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> See
>>>>>>>>>>>>>>> https://github.com/cloudera/Impala/blob/cdh5-trunk/buildall.sh#L283
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hope that helps!
>>>>>>>>>>>>>>> Skye
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sat, Apr 2, 2016 at 12:55 PM, Pradeep Nayak <
>>>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I just cloned the Impala source from github, and I am
>>>>>>>>>>>>>>>> trying to compile the front end per Impala wiki. I get the error below. I
>>>>>>>>>>>>>>>> am using maven 3.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [ERROR] Failed to execute goal on project impala-frontend:
>>>>>>>>>>>>>>>> Could not resolve dependencies for project
>>>>>>>>>>>>>>>> com.cloudera.impala:impala-frontend:jar:0.1-SNAPSHOT: Failure to find
>>>>>>>>>>>>>>>> com.cloudera.impala:impala-data-source-api:jar:1.0-SNAPSHOT in
>>>>>>>>>>>>>>>> https://repository.cloudera.com/content/groups/cdh-releases-rcs
>>>>>>>>>>>>>>>> was cached in the local repository, resolution will not be reattempted
>>>>>>>>>>>>>>>> until the update interval of cdh.rcs.releases.repo has elapsed or updates
>>>>>>>>>>>>>>>> are forced -> [Help 1
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This is the command I used:   mvn clean package
>>>>>>>>>>>>>>>> dependency:copy-dependencies -DskipTests=true
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Any idea getting around this problem ?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>>>>>> Google Groups "Impala Dev" group.
>>>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>>>>>>>>>>>>>>> from it, send an email to
>>>>>>>>>>>>>>>> impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>>>> Google Groups "Impala Dev" group.
>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>>>>> it, send an email to impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Impala Dev" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to impala-dev+unsubscribe@cloudera.org.
>>>
>>
>>
>>
>> --
>> Henry Robinson
>> Software Engineer
>> Cloudera
>> 415-994-6679
>>
>
>

Re: maven error while compiling the front end code

Posted by Skye Wanderman-Milne <sk...@cloudera.com>.
In case others run into similar issues, can you share how you resolved the
NoClassDefFoundErrors?

On Mon, Apr 4, 2016 at 10:21 PM, Henry Robinson <he...@cloudera.com> wrote:

>
>
> On 4 April 2016 at 17:55, Pradeep Nayak <pr...@gmail.com> wrote:
>
>> Also just wanted to understand one more thing.
>> http://www.cloudera.com/documentation/enterprise/5-5-x/topics/impala_s3.html It
>> mentions that impala needs to have a default file system which is hdfs, but
>> not just s3a, or any other filesystem which extends Hadoop FileSystem class
>> ? Is there plans to support non hdfs filesystem as default ?
>>
>
> Yes there are - see IMPALA-1850 (code review
> http://gerrit.cloudera.org:8080/#/c/1121/).
>
> Henry
>
>
>>
>> Meanwhile I got around the problem earlier. I can query tables, do a
>> count(*), etc.
>>
>> However join is causing a problem...
>>
>> *select * from tab1 join tab2;*
>>
>> *Query: select * from tab1 join tab2*
>>
>> *ERROR:*
>>
>> *Failed to connect to FS*
>>
>> *Error(255): Unknown error 255*
>>
>>
>> Pradeep
>>
>> On Mon, Apr 4, 2016 at 3:31 PM Pradeep Nayak <pr...@gmail.com>
>> wrote:
>>
>>> That's what been puzzling me as well :(. Let me get back to you with
>>> that information.
>>>
>>> Pradeep
>>>
>>> On Mon, Apr 4, 2016 at 3:28 PM Skye Wanderman-Milne <sk...@cloudera.com>
>>> wrote:
>>>
>>>> Actually, I didn't read your exception carefully and didn't notice that
>>>> it can't find an Impala class, so it's probably not a dependency issue.
>>>>
>>>> Can you restart your cluster, rerun a select * query, and then search
>>>> your logs for that "NoClassDefFoundError"? That will tell us which process
>>>> the error is coming from. That class is included in the FE jar, so I'm not
>>>> sure why it wouldn't be able to find it.
>>>>
>>>>
>>>> On Mon, Apr 4, 2016 at 3:14 PM, Pradeep Nayak <pr...@gmail.com>
>>>> wrote:
>>>>
>>>>> Sure Skye. I didn't notice the to: list as well until you pointed out.
>>>>> I really appreciate your help here.
>>>>>
>>>>> I haven't modified the pom.xml for the front-end here:
>>>>> https://github.com/cloudera/Impala/blob/cdh5.5.2-release/fe/pom.xml
>>>>>
>>>>> I am just building with this at the moment. mvn clean package
>>>>> -DSkipTests = true from the fe folder.
>>>>>
>>>>>
>>>>> On Mon, Apr 4, 2016 at 3:11 PM Skye Wanderman-Milne <sk...@cloudera.com>
>>>>> wrote:
>>>>>
>>>>>> Ah didn't mean to take this off the dev list. Also add our new
>>>>>> dev@impala.incubator list.
>>>>>>
>>>>>> Can you check what dependency versions you're pulling in from maven?
>>>>>> My guess is that you're somehow building with non-CDH5.5.2 frontend
>>>>>> dependencies.
>>>>>>
>>>>>> On Mon, Apr 4, 2016 at 3:04 PM, Pradeep Nayak <pr...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Umm, I am trying to test this on a dev cluster which has cloudera
>>>>>>> parcels already installed. I just re-placed
>>>>>>> impala-frontend-0.1-SNAPSHOT.jar with my version. Do I need to change
>>>>>>> anything on the dev cluster ?
>>>>>>>
>>>>>>> On Mon, Apr 4, 2016 at 2:43 PM Skye Wanderman-Milne <
>>>>>>> skye@cloudera.com> wrote:
>>>>>>>
>>>>>>>> Try: source $IMPALA_HOME/bin/set-classpath.sh
>>>>>>>>
>>>>>>>> If you recently switched branches, you may want to close your
>>>>>>>> current terminal, open a new one, and re-source
>>>>>>>> $IMPALA_HOME/bin/impala-config.sh and set-classpath.sh.
>>>>>>>>
>>>>>>>> Then restart your cluster (start-impala-cluster.py)
>>>>>>>>
>>>>>>>> On Mon, Apr 4, 2016 at 2:39 PM, Pradeep Nayak <
>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Thanks Skye.
>>>>>>>>>
>>>>>>>>> I am in my impala shell. I get this following error. I see this is
>>>>>>>>> already compiled as part of impala-frontend. This happens only when I try
>>>>>>>>> to do a "select * from <table_name>" I can do "show tables; create table
>>>>>>>>> <table_name>" etc. Any pointers ?
>>>>>>>>>
>>>>>>>>> ERROR: AnalysisException: java.lang.NoClassDefFoundError: Could
>>>>>>>>> not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>
>>>>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError:
>>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>
>>>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>
>>>>>>>>> CAUSED BY: TableLoadingException: java.lang.NoClassDefFoundError:
>>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>
>>>>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError:
>>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>
>>>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>>
>>>>>>>>> I used this tree:
>>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5.5.2-release
>>>>>>>>>
>>>>>>>>> On Mon, Apr 4, 2016 at 12:01 PM Skye Wanderman-Milne <
>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>
>>>>>>>>>> I think it should be
>>>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5-2.3.0_5.5.2
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 4, 2016 at 12:00 PM, Pradeep Nayak <
>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Also to what release this parcel  *5.5.2-1.cdh5.5.2.p0.4 *map
>>>>>>>>>>> to on github ? I am unable to find a tag/branch for this parcel type.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:58 AM Skye Wanderman-Milne <
>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Ah yes, both 1 and 2 are correct. Glad you figured it out!
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:56 AM, Pradeep Nayak <
>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I just figured.
>>>>>>>>>>>>> 1. For the first time you need to use the ./buildall.sh and
>>>>>>>>>>>>> for subsequent fe compilation I can just use maven
>>>>>>>>>>>>> 2. It doesn't build on Macbook
>>>>>>>>>>>>> 3. I set up a Ubuntu vm and got this compiled.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:07 AM Skye Wanderman-Milne <
>>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hey Pradeep,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You should try running the $IMPALA_HOME/buildall.sh script in
>>>>>>>>>>>>>> order to build Impala. It has a lot of necessary setup to build Impala.
>>>>>>>>>>>>>> Here are the commands from buildall.sh that should provide the
>>>>>>>>>>>>>> impala-data-source-api jar for building the FE:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> # build the external data source API
>>>>>>>>>>>>>> pushd ${IMPALA_HOME}/ext-data-source
>>>>>>>>>>>>>> ${IMPALA_HOME}/bin/mvn-quiet.sh install -DskipTests
>>>>>>>>>>>>>> popd
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> See
>>>>>>>>>>>>>> https://github.com/cloudera/Impala/blob/cdh5-trunk/buildall.sh#L283
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hope that helps!
>>>>>>>>>>>>>> Skye
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sat, Apr 2, 2016 at 12:55 PM, Pradeep Nayak <
>>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I just cloned the Impala source from github, and I am trying
>>>>>>>>>>>>>>> to compile the front end per Impala wiki. I get the error below. I am using
>>>>>>>>>>>>>>> maven 3.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [ERROR] Failed to execute goal on project impala-frontend:
>>>>>>>>>>>>>>> Could not resolve dependencies for project
>>>>>>>>>>>>>>> com.cloudera.impala:impala-frontend:jar:0.1-SNAPSHOT: Failure to find
>>>>>>>>>>>>>>> com.cloudera.impala:impala-data-source-api:jar:1.0-SNAPSHOT in
>>>>>>>>>>>>>>> https://repository.cloudera.com/content/groups/cdh-releases-rcs
>>>>>>>>>>>>>>> was cached in the local repository, resolution will not be reattempted
>>>>>>>>>>>>>>> until the update interval of cdh.rcs.releases.repo has elapsed or updates
>>>>>>>>>>>>>>> are forced -> [Help 1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This is the command I used:   mvn clean package
>>>>>>>>>>>>>>> dependency:copy-dependencies -DskipTests=true
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Any idea getting around this problem ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>>>>> Google Groups "Impala Dev" group.
>>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>>>>>>>>>>>>>> from it, send an email to
>>>>>>>>>>>>>>> impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>>> Google Groups "Impala Dev" group.
>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>>>> it, send an email to impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>> --
>> You received this message because you are subscribed to the Google Groups
>> "Impala Dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to impala-dev+unsubscribe@cloudera.org.
>>
>
>
>
> --
> Henry Robinson
> Software Engineer
> Cloudera
> 415-994-6679
>

Re: maven error while compiling the front end code

Posted by Henry Robinson <he...@cloudera.com>.
On 4 April 2016 at 17:55, Pradeep Nayak <pr...@gmail.com> wrote:

> Also just wanted to understand one more thing.
> http://www.cloudera.com/documentation/enterprise/5-5-x/topics/impala_s3.html It
> mentions that impala needs to have a default file system which is hdfs, but
> not just s3a, or any other filesystem which extends Hadoop FileSystem class
> ? Is there plans to support non hdfs filesystem as default ?
>

Yes there are - see IMPALA-1850 (code review
http://gerrit.cloudera.org:8080/#/c/1121/).

Henry


>
> Meanwhile I got around the problem earlier. I can query tables, do a
> count(*), etc.
>
> However join is causing a problem...
>
> *select * from tab1 join tab2;*
>
> *Query: select * from tab1 join tab2*
>
> *ERROR:*
>
> *Failed to connect to FS*
>
> *Error(255): Unknown error 255*
>
>
> Pradeep
>
> On Mon, Apr 4, 2016 at 3:31 PM Pradeep Nayak <pr...@gmail.com>
> wrote:
>
>> That's what been puzzling me as well :(. Let me get back to you with that
>> information.
>>
>> Pradeep
>>
>> On Mon, Apr 4, 2016 at 3:28 PM Skye Wanderman-Milne <sk...@cloudera.com>
>> wrote:
>>
>>> Actually, I didn't read your exception carefully and didn't notice that
>>> it can't find an Impala class, so it's probably not a dependency issue.
>>>
>>> Can you restart your cluster, rerun a select * query, and then search
>>> your logs for that "NoClassDefFoundError"? That will tell us which process
>>> the error is coming from. That class is included in the FE jar, so I'm not
>>> sure why it wouldn't be able to find it.
>>>
>>>
>>> On Mon, Apr 4, 2016 at 3:14 PM, Pradeep Nayak <pr...@gmail.com>
>>> wrote:
>>>
>>>> Sure Skye. I didn't notice the to: list as well until you pointed out.
>>>> I really appreciate your help here.
>>>>
>>>> I haven't modified the pom.xml for the front-end here:
>>>> https://github.com/cloudera/Impala/blob/cdh5.5.2-release/fe/pom.xml
>>>>
>>>> I am just building with this at the moment. mvn clean package
>>>> -DSkipTests = true from the fe folder.
>>>>
>>>>
>>>> On Mon, Apr 4, 2016 at 3:11 PM Skye Wanderman-Milne <sk...@cloudera.com>
>>>> wrote:
>>>>
>>>>> Ah didn't mean to take this off the dev list. Also add our new
>>>>> dev@impala.incubator list.
>>>>>
>>>>> Can you check what dependency versions you're pulling in from maven?
>>>>> My guess is that you're somehow building with non-CDH5.5.2 frontend
>>>>> dependencies.
>>>>>
>>>>> On Mon, Apr 4, 2016 at 3:04 PM, Pradeep Nayak <pr...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Umm, I am trying to test this on a dev cluster which has cloudera
>>>>>> parcels already installed. I just re-placed
>>>>>> impala-frontend-0.1-SNAPSHOT.jar with my version. Do I need to change
>>>>>> anything on the dev cluster ?
>>>>>>
>>>>>> On Mon, Apr 4, 2016 at 2:43 PM Skye Wanderman-Milne <
>>>>>> skye@cloudera.com> wrote:
>>>>>>
>>>>>>> Try: source $IMPALA_HOME/bin/set-classpath.sh
>>>>>>>
>>>>>>> If you recently switched branches, you may want to close your
>>>>>>> current terminal, open a new one, and re-source
>>>>>>> $IMPALA_HOME/bin/impala-config.sh and set-classpath.sh.
>>>>>>>
>>>>>>> Then restart your cluster (start-impala-cluster.py)
>>>>>>>
>>>>>>> On Mon, Apr 4, 2016 at 2:39 PM, Pradeep Nayak <pradeep1288@gmail.com
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Thanks Skye.
>>>>>>>>
>>>>>>>> I am in my impala shell. I get this following error. I see this is
>>>>>>>> already compiled as part of impala-frontend. This happens only when I try
>>>>>>>> to do a "select * from <table_name>" I can do "show tables; create table
>>>>>>>> <table_name>" etc. Any pointers ?
>>>>>>>>
>>>>>>>> ERROR: AnalysisException: java.lang.NoClassDefFoundError: Could not
>>>>>>>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>
>>>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError:
>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>
>>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>
>>>>>>>> CAUSED BY: TableLoadingException: java.lang.NoClassDefFoundError:
>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>
>>>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError:
>>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>
>>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>>
>>>>>>>> I used this tree:
>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5.5.2-release
>>>>>>>>
>>>>>>>> On Mon, Apr 4, 2016 at 12:01 PM Skye Wanderman-Milne <
>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>
>>>>>>>>> I think it should be
>>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5-2.3.0_5.5.2
>>>>>>>>>
>>>>>>>>> On Mon, Apr 4, 2016 at 12:00 PM, Pradeep Nayak <
>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Also to what release this parcel  *5.5.2-1.cdh5.5.2.p0.4 *map to
>>>>>>>>>> on github ? I am unable to find a tag/branch for this parcel type.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 4, 2016 at 11:58 AM Skye Wanderman-Milne <
>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Ah yes, both 1 and 2 are correct. Glad you figured it out!
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:56 AM, Pradeep Nayak <
>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I just figured.
>>>>>>>>>>>> 1. For the first time you need to use the ./buildall.sh and for
>>>>>>>>>>>> subsequent fe compilation I can just use maven
>>>>>>>>>>>> 2. It doesn't build on Macbook
>>>>>>>>>>>> 3. I set up a Ubuntu vm and got this compiled.
>>>>>>>>>>>>
>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:07 AM Skye Wanderman-Milne <
>>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hey Pradeep,
>>>>>>>>>>>>>
>>>>>>>>>>>>> You should try running the $IMPALA_HOME/buildall.sh script in
>>>>>>>>>>>>> order to build Impala. It has a lot of necessary setup to build Impala.
>>>>>>>>>>>>> Here are the commands from buildall.sh that should provide the
>>>>>>>>>>>>> impala-data-source-api jar for building the FE:
>>>>>>>>>>>>>
>>>>>>>>>>>>> # build the external data source API
>>>>>>>>>>>>> pushd ${IMPALA_HOME}/ext-data-source
>>>>>>>>>>>>> ${IMPALA_HOME}/bin/mvn-quiet.sh install -DskipTests
>>>>>>>>>>>>> popd
>>>>>>>>>>>>>
>>>>>>>>>>>>> See
>>>>>>>>>>>>> https://github.com/cloudera/Impala/blob/cdh5-trunk/buildall.sh#L283
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hope that helps!
>>>>>>>>>>>>> Skye
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sat, Apr 2, 2016 at 12:55 PM, Pradeep Nayak <
>>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I just cloned the Impala source from github, and I am trying
>>>>>>>>>>>>>> to compile the front end per Impala wiki. I get the error below. I am using
>>>>>>>>>>>>>> maven 3.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [ERROR] Failed to execute goal on project impala-frontend:
>>>>>>>>>>>>>> Could not resolve dependencies for project
>>>>>>>>>>>>>> com.cloudera.impala:impala-frontend:jar:0.1-SNAPSHOT: Failure to find
>>>>>>>>>>>>>> com.cloudera.impala:impala-data-source-api:jar:1.0-SNAPSHOT in
>>>>>>>>>>>>>> https://repository.cloudera.com/content/groups/cdh-releases-rcs
>>>>>>>>>>>>>> was cached in the local repository, resolution will not be reattempted
>>>>>>>>>>>>>> until the update interval of cdh.rcs.releases.repo has elapsed or updates
>>>>>>>>>>>>>> are forced -> [Help 1
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is the command I used:   mvn clean package
>>>>>>>>>>>>>> dependency:copy-dependencies -DskipTests=true
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Any idea getting around this problem ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>>>> Google Groups "Impala Dev" group.
>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>>>>> it, send an email to impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>> Google Groups "Impala Dev" group.
>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>>> it, send an email to impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>> --
> You received this message because you are subscribed to the Google Groups
> "Impala Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to impala-dev+unsubscribe@cloudera.org.
>



-- 
Henry Robinson
Software Engineer
Cloudera
415-994-6679

Re: maven error while compiling the front end code

Posted by Pradeep Nayak <pr...@gmail.com>.
Also just wanted to understand one more thing.
http://www.cloudera.com/documentation/enterprise/5-5-x/topics/impala_s3.html It
mentions that impala needs to have a default file system which is hdfs, but
not just s3a, or any other filesystem which extends Hadoop FileSystem class
? Is there plans to support non hdfs filesystem as default ?

Meanwhile I got around the problem earlier. I can query tables, do a
count(*), etc.

However join is causing a problem...

*select * from tab1 join tab2;*

*Query: select * from tab1 join tab2*

*ERROR:*

*Failed to connect to FS*

*Error(255): Unknown error 255*


Pradeep

On Mon, Apr 4, 2016 at 3:31 PM Pradeep Nayak <pr...@gmail.com> wrote:

> That's what been puzzling me as well :(. Let me get back to you with that
> information.
>
> Pradeep
>
> On Mon, Apr 4, 2016 at 3:28 PM Skye Wanderman-Milne <sk...@cloudera.com>
> wrote:
>
>> Actually, I didn't read your exception carefully and didn't notice that
>> it can't find an Impala class, so it's probably not a dependency issue.
>>
>> Can you restart your cluster, rerun a select * query, and then search
>> your logs for that "NoClassDefFoundError"? That will tell us which process
>> the error is coming from. That class is included in the FE jar, so I'm not
>> sure why it wouldn't be able to find it.
>>
>>
>> On Mon, Apr 4, 2016 at 3:14 PM, Pradeep Nayak <pr...@gmail.com>
>> wrote:
>>
>>> Sure Skye. I didn't notice the to: list as well until you pointed out. I
>>> really appreciate your help here.
>>>
>>> I haven't modified the pom.xml for the front-end here:
>>> https://github.com/cloudera/Impala/blob/cdh5.5.2-release/fe/pom.xml
>>>
>>> I am just building with this at the moment. mvn clean package
>>> -DSkipTests = true from the fe folder.
>>>
>>>
>>> On Mon, Apr 4, 2016 at 3:11 PM Skye Wanderman-Milne <sk...@cloudera.com>
>>> wrote:
>>>
>>>> Ah didn't mean to take this off the dev list. Also add our new
>>>> dev@impala.incubator list.
>>>>
>>>> Can you check what dependency versions you're pulling in from maven? My
>>>> guess is that you're somehow building with non-CDH5.5.2 frontend
>>>> dependencies.
>>>>
>>>> On Mon, Apr 4, 2016 at 3:04 PM, Pradeep Nayak <pr...@gmail.com>
>>>> wrote:
>>>>
>>>>> Umm, I am trying to test this on a dev cluster which has cloudera
>>>>> parcels already installed. I just re-placed
>>>>> impala-frontend-0.1-SNAPSHOT.jar with my version. Do I need to change
>>>>> anything on the dev cluster ?
>>>>>
>>>>> On Mon, Apr 4, 2016 at 2:43 PM Skye Wanderman-Milne <sk...@cloudera.com>
>>>>> wrote:
>>>>>
>>>>>> Try: source $IMPALA_HOME/bin/set-classpath.sh
>>>>>>
>>>>>> If you recently switched branches, you may want to close your current
>>>>>> terminal, open a new one, and re-source $IMPALA_HOME/bin/impala-config.sh
>>>>>> and set-classpath.sh.
>>>>>>
>>>>>> Then restart your cluster (start-impala-cluster.py)
>>>>>>
>>>>>> On Mon, Apr 4, 2016 at 2:39 PM, Pradeep Nayak <pr...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks Skye.
>>>>>>>
>>>>>>> I am in my impala shell. I get this following error. I see this is
>>>>>>> already compiled as part of impala-frontend. This happens only when I try
>>>>>>> to do a "select * from <table_name>" I can do "show tables; create table
>>>>>>> <table_name>" etc. Any pointers ?
>>>>>>>
>>>>>>> ERROR: AnalysisException: java.lang.NoClassDefFoundError: Could not
>>>>>>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>
>>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could
>>>>>>> not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>
>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>
>>>>>>> CAUSED BY: TableLoadingException: java.lang.NoClassDefFoundError:
>>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>
>>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could
>>>>>>> not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>
>>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>>
>>>>>>> I used this tree:
>>>>>>> https://github.com/cloudera/Impala/tree/cdh5.5.2-release
>>>>>>>
>>>>>>> On Mon, Apr 4, 2016 at 12:01 PM Skye Wanderman-Milne <
>>>>>>> skye@cloudera.com> wrote:
>>>>>>>
>>>>>>>> I think it should be
>>>>>>>> https://github.com/cloudera/Impala/tree/cdh5-2.3.0_5.5.2
>>>>>>>>
>>>>>>>> On Mon, Apr 4, 2016 at 12:00 PM, Pradeep Nayak <
>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Also to what release this parcel  *5.5.2-1.cdh5.5.2.p0.4 *map to
>>>>>>>>> on github ? I am unable to find a tag/branch for this parcel type.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Apr 4, 2016 at 11:58 AM Skye Wanderman-Milne <
>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>
>>>>>>>>>> Ah yes, both 1 and 2 are correct. Glad you figured it out!
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 4, 2016 at 11:56 AM, Pradeep Nayak <
>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I just figured.
>>>>>>>>>>> 1. For the first time you need to use the ./buildall.sh and for
>>>>>>>>>>> subsequent fe compilation I can just use maven
>>>>>>>>>>> 2. It doesn't build on Macbook
>>>>>>>>>>> 3. I set up a Ubuntu vm and got this compiled.
>>>>>>>>>>>
>>>>>>>>>>> Pradeep
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 4, 2016 at 11:07 AM Skye Wanderman-Milne <
>>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hey Pradeep,
>>>>>>>>>>>>
>>>>>>>>>>>> You should try running the $IMPALA_HOME/buildall.sh script in
>>>>>>>>>>>> order to build Impala. It has a lot of necessary setup to build Impala.
>>>>>>>>>>>> Here are the commands from buildall.sh that should provide the
>>>>>>>>>>>> impala-data-source-api jar for building the FE:
>>>>>>>>>>>>
>>>>>>>>>>>> # build the external data source API
>>>>>>>>>>>> pushd ${IMPALA_HOME}/ext-data-source
>>>>>>>>>>>> ${IMPALA_HOME}/bin/mvn-quiet.sh install -DskipTests
>>>>>>>>>>>> popd
>>>>>>>>>>>>
>>>>>>>>>>>> See
>>>>>>>>>>>> https://github.com/cloudera/Impala/blob/cdh5-trunk/buildall.sh#L283
>>>>>>>>>>>>
>>>>>>>>>>>> Hope that helps!
>>>>>>>>>>>> Skye
>>>>>>>>>>>>
>>>>>>>>>>>> On Sat, Apr 2, 2016 at 12:55 PM, Pradeep Nayak <
>>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I just cloned the Impala source from github, and I am trying
>>>>>>>>>>>>> to compile the front end per Impala wiki. I get the error below. I am using
>>>>>>>>>>>>> maven 3.
>>>>>>>>>>>>>
>>>>>>>>>>>>> [ERROR] Failed to execute goal on project impala-frontend:
>>>>>>>>>>>>> Could not resolve dependencies for project
>>>>>>>>>>>>> com.cloudera.impala:impala-frontend:jar:0.1-SNAPSHOT: Failure to find
>>>>>>>>>>>>> com.cloudera.impala:impala-data-source-api:jar:1.0-SNAPSHOT in
>>>>>>>>>>>>> https://repository.cloudera.com/content/groups/cdh-releases-rcs
>>>>>>>>>>>>> was cached in the local repository, resolution will not be reattempted
>>>>>>>>>>>>> until the update interval of cdh.rcs.releases.repo has elapsed or updates
>>>>>>>>>>>>> are forced -> [Help 1
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is the command I used:   mvn clean package
>>>>>>>>>>>>> dependency:copy-dependencies -DskipTests=true
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any idea getting around this problem ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers!
>>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>>> Google Groups "Impala Dev" group.
>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>>>> it, send an email to impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>> Google Groups "Impala Dev" group.
>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>> it, send an email to impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>

Re: maven error while compiling the front end code

Posted by Pradeep Nayak <pr...@gmail.com>.
That's what been puzzling me as well :(. Let me get back to you with that
information.

Pradeep

On Mon, Apr 4, 2016 at 3:28 PM Skye Wanderman-Milne <sk...@cloudera.com>
wrote:

> Actually, I didn't read your exception carefully and didn't notice that it
> can't find an Impala class, so it's probably not a dependency issue.
>
> Can you restart your cluster, rerun a select * query, and then search your
> logs for that "NoClassDefFoundError"? That will tell us which process the
> error is coming from. That class is included in the FE jar, so I'm not sure
> why it wouldn't be able to find it.
>
>
> On Mon, Apr 4, 2016 at 3:14 PM, Pradeep Nayak <pr...@gmail.com>
> wrote:
>
>> Sure Skye. I didn't notice the to: list as well until you pointed out. I
>> really appreciate your help here.
>>
>> I haven't modified the pom.xml for the front-end here:
>> https://github.com/cloudera/Impala/blob/cdh5.5.2-release/fe/pom.xml
>>
>> I am just building with this at the moment. mvn clean package -DSkipTests
>> = true from the fe folder.
>>
>>
>> On Mon, Apr 4, 2016 at 3:11 PM Skye Wanderman-Milne <sk...@cloudera.com>
>> wrote:
>>
>>> Ah didn't mean to take this off the dev list. Also add our new
>>> dev@impala.incubator list.
>>>
>>> Can you check what dependency versions you're pulling in from maven? My
>>> guess is that you're somehow building with non-CDH5.5.2 frontend
>>> dependencies.
>>>
>>> On Mon, Apr 4, 2016 at 3:04 PM, Pradeep Nayak <pr...@gmail.com>
>>> wrote:
>>>
>>>> Umm, I am trying to test this on a dev cluster which has cloudera
>>>> parcels already installed. I just re-placed
>>>> impala-frontend-0.1-SNAPSHOT.jar with my version. Do I need to change
>>>> anything on the dev cluster ?
>>>>
>>>> On Mon, Apr 4, 2016 at 2:43 PM Skye Wanderman-Milne <sk...@cloudera.com>
>>>> wrote:
>>>>
>>>>> Try: source $IMPALA_HOME/bin/set-classpath.sh
>>>>>
>>>>> If you recently switched branches, you may want to close your current
>>>>> terminal, open a new one, and re-source $IMPALA_HOME/bin/impala-config.sh
>>>>> and set-classpath.sh.
>>>>>
>>>>> Then restart your cluster (start-impala-cluster.py)
>>>>>
>>>>> On Mon, Apr 4, 2016 at 2:39 PM, Pradeep Nayak <pr...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Thanks Skye.
>>>>>>
>>>>>> I am in my impala shell. I get this following error. I see this is
>>>>>> already compiled as part of impala-frontend. This happens only when I try
>>>>>> to do a "select * from <table_name>" I can do "show tables; create table
>>>>>> <table_name>" etc. Any pointers ?
>>>>>>
>>>>>> ERROR: AnalysisException: java.lang.NoClassDefFoundError: Could not
>>>>>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>
>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could
>>>>>> not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>
>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>
>>>>>> CAUSED BY: TableLoadingException: java.lang.NoClassDefFoundError:
>>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>
>>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could
>>>>>> not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>>
>>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>>
>>>>>> I used this tree:
>>>>>> https://github.com/cloudera/Impala/tree/cdh5.5.2-release
>>>>>>
>>>>>> On Mon, Apr 4, 2016 at 12:01 PM Skye Wanderman-Milne <
>>>>>> skye@cloudera.com> wrote:
>>>>>>
>>>>>>> I think it should be
>>>>>>> https://github.com/cloudera/Impala/tree/cdh5-2.3.0_5.5.2
>>>>>>>
>>>>>>> On Mon, Apr 4, 2016 at 12:00 PM, Pradeep Nayak <
>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>
>>>>>>>> Also to what release this parcel  *5.5.2-1.cdh5.5.2.p0.4 *map to
>>>>>>>> on github ? I am unable to find a tag/branch for this parcel type.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Apr 4, 2016 at 11:58 AM Skye Wanderman-Milne <
>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>
>>>>>>>>> Ah yes, both 1 and 2 are correct. Glad you figured it out!
>>>>>>>>>
>>>>>>>>> On Mon, Apr 4, 2016 at 11:56 AM, Pradeep Nayak <
>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I just figured.
>>>>>>>>>> 1. For the first time you need to use the ./buildall.sh and for
>>>>>>>>>> subsequent fe compilation I can just use maven
>>>>>>>>>> 2. It doesn't build on Macbook
>>>>>>>>>> 3. I set up a Ubuntu vm and got this compiled.
>>>>>>>>>>
>>>>>>>>>> Pradeep
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 4, 2016 at 11:07 AM Skye Wanderman-Milne <
>>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hey Pradeep,
>>>>>>>>>>>
>>>>>>>>>>> You should try running the $IMPALA_HOME/buildall.sh script in
>>>>>>>>>>> order to build Impala. It has a lot of necessary setup to build Impala.
>>>>>>>>>>> Here are the commands from buildall.sh that should provide the
>>>>>>>>>>> impala-data-source-api jar for building the FE:
>>>>>>>>>>>
>>>>>>>>>>> # build the external data source API
>>>>>>>>>>> pushd ${IMPALA_HOME}/ext-data-source
>>>>>>>>>>> ${IMPALA_HOME}/bin/mvn-quiet.sh install -DskipTests
>>>>>>>>>>> popd
>>>>>>>>>>>
>>>>>>>>>>> See
>>>>>>>>>>> https://github.com/cloudera/Impala/blob/cdh5-trunk/buildall.sh#L283
>>>>>>>>>>>
>>>>>>>>>>> Hope that helps!
>>>>>>>>>>> Skye
>>>>>>>>>>>
>>>>>>>>>>> On Sat, Apr 2, 2016 at 12:55 PM, Pradeep Nayak <
>>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I just cloned the Impala source from github, and I am trying to
>>>>>>>>>>>> compile the front end per Impala wiki. I get the error below. I am using
>>>>>>>>>>>> maven 3.
>>>>>>>>>>>>
>>>>>>>>>>>> [ERROR] Failed to execute goal on project impala-frontend:
>>>>>>>>>>>> Could not resolve dependencies for project
>>>>>>>>>>>> com.cloudera.impala:impala-frontend:jar:0.1-SNAPSHOT: Failure to find
>>>>>>>>>>>> com.cloudera.impala:impala-data-source-api:jar:1.0-SNAPSHOT in
>>>>>>>>>>>> https://repository.cloudera.com/content/groups/cdh-releases-rcs
>>>>>>>>>>>> was cached in the local repository, resolution will not be reattempted
>>>>>>>>>>>> until the update interval of cdh.rcs.releases.repo has elapsed or updates
>>>>>>>>>>>> are forced -> [Help 1
>>>>>>>>>>>>
>>>>>>>>>>>> This is the command I used:   mvn clean package
>>>>>>>>>>>> dependency:copy-dependencies -DskipTests=true
>>>>>>>>>>>>
>>>>>>>>>>>> Any idea getting around this problem ?
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers!
>>>>>>>>>>>> Pradeep
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>> Google Groups "Impala Dev" group.
>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>>> it, send an email to impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>> Google Groups "Impala Dev" group.
>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>>> send an email to impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>
>

Re: maven error while compiling the front end code

Posted by Skye Wanderman-Milne <sk...@cloudera.com>.
Actually, I didn't read your exception carefully and didn't notice that it
can't find an Impala class, so it's probably not a dependency issue.

Can you restart your cluster, rerun a select * query, and then search your
logs for that "NoClassDefFoundError"? That will tell us which process the
error is coming from. That class is included in the FE jar, so I'm not sure
why it wouldn't be able to find it.

On Mon, Apr 4, 2016 at 3:14 PM, Pradeep Nayak <pr...@gmail.com> wrote:

> Sure Skye. I didn't notice the to: list as well until you pointed out. I
> really appreciate your help here.
>
> I haven't modified the pom.xml for the front-end here:
> https://github.com/cloudera/Impala/blob/cdh5.5.2-release/fe/pom.xml
>
> I am just building with this at the moment. mvn clean package -DSkipTests
> = true from the fe folder.
>
>
> On Mon, Apr 4, 2016 at 3:11 PM Skye Wanderman-Milne <sk...@cloudera.com>
> wrote:
>
>> Ah didn't mean to take this off the dev list. Also add our new
>> dev@impala.incubator list.
>>
>> Can you check what dependency versions you're pulling in from maven? My
>> guess is that you're somehow building with non-CDH5.5.2 frontend
>> dependencies.
>>
>> On Mon, Apr 4, 2016 at 3:04 PM, Pradeep Nayak <pr...@gmail.com>
>> wrote:
>>
>>> Umm, I am trying to test this on a dev cluster which has cloudera
>>> parcels already installed. I just re-placed
>>> impala-frontend-0.1-SNAPSHOT.jar with my version. Do I need to change
>>> anything on the dev cluster ?
>>>
>>> On Mon, Apr 4, 2016 at 2:43 PM Skye Wanderman-Milne <sk...@cloudera.com>
>>> wrote:
>>>
>>>> Try: source $IMPALA_HOME/bin/set-classpath.sh
>>>>
>>>> If you recently switched branches, you may want to close your current
>>>> terminal, open a new one, and re-source $IMPALA_HOME/bin/impala-config.sh
>>>> and set-classpath.sh.
>>>>
>>>> Then restart your cluster (start-impala-cluster.py)
>>>>
>>>> On Mon, Apr 4, 2016 at 2:39 PM, Pradeep Nayak <pr...@gmail.com>
>>>> wrote:
>>>>
>>>>> Thanks Skye.
>>>>>
>>>>> I am in my impala shell. I get this following error. I see this is
>>>>> already compiled as part of impala-frontend. This happens only when I try
>>>>> to do a "select * from <table_name>" I can do "show tables; create table
>>>>> <table_name>" etc. Any pointers ?
>>>>>
>>>>> ERROR: AnalysisException: java.lang.NoClassDefFoundError: Could not
>>>>> initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>
>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could
>>>>> not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>
>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>
>>>>> CAUSED BY: TableLoadingException: java.lang.NoClassDefFoundError:
>>>>> Could not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>
>>>>> CAUSED BY: ExecutionException: java.lang.NoClassDefFoundError: Could
>>>>> not initialize class com.cloudera.impala.util.HdfsCachingUtil
>>>>>
>>>>> CAUSED BY: NoClassDefFoundError: Could not initialize class
>>>>> com.cloudera.impala.util.HdfsCachingUtil
>>>>>
>>>>> I used this tree:
>>>>> https://github.com/cloudera/Impala/tree/cdh5.5.2-release
>>>>>
>>>>> On Mon, Apr 4, 2016 at 12:01 PM Skye Wanderman-Milne <
>>>>> skye@cloudera.com> wrote:
>>>>>
>>>>>> I think it should be
>>>>>> https://github.com/cloudera/Impala/tree/cdh5-2.3.0_5.5.2
>>>>>>
>>>>>> On Mon, Apr 4, 2016 at 12:00 PM, Pradeep Nayak <pradeep1288@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Also to what release this parcel  *5.5.2-1.cdh5.5.2.p0.4 *map to on
>>>>>>> github ? I am unable to find a tag/branch for this parcel type.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Apr 4, 2016 at 11:58 AM Skye Wanderman-Milne <
>>>>>>> skye@cloudera.com> wrote:
>>>>>>>
>>>>>>>> Ah yes, both 1 and 2 are correct. Glad you figured it out!
>>>>>>>>
>>>>>>>> On Mon, Apr 4, 2016 at 11:56 AM, Pradeep Nayak <
>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I just figured.
>>>>>>>>> 1. For the first time you need to use the ./buildall.sh and for
>>>>>>>>> subsequent fe compilation I can just use maven
>>>>>>>>> 2. It doesn't build on Macbook
>>>>>>>>> 3. I set up a Ubuntu vm and got this compiled.
>>>>>>>>>
>>>>>>>>> Pradeep
>>>>>>>>>
>>>>>>>>> On Mon, Apr 4, 2016 at 11:07 AM Skye Wanderman-Milne <
>>>>>>>>> skye@cloudera.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hey Pradeep,
>>>>>>>>>>
>>>>>>>>>> You should try running the $IMPALA_HOME/buildall.sh script in
>>>>>>>>>> order to build Impala. It has a lot of necessary setup to build Impala.
>>>>>>>>>> Here are the commands from buildall.sh that should provide the
>>>>>>>>>> impala-data-source-api jar for building the FE:
>>>>>>>>>>
>>>>>>>>>> # build the external data source API
>>>>>>>>>> pushd ${IMPALA_HOME}/ext-data-source
>>>>>>>>>> ${IMPALA_HOME}/bin/mvn-quiet.sh install -DskipTests
>>>>>>>>>> popd
>>>>>>>>>>
>>>>>>>>>> See
>>>>>>>>>> https://github.com/cloudera/Impala/blob/cdh5-trunk/buildall.sh#L283
>>>>>>>>>>
>>>>>>>>>> Hope that helps!
>>>>>>>>>> Skye
>>>>>>>>>>
>>>>>>>>>> On Sat, Apr 2, 2016 at 12:55 PM, Pradeep Nayak <
>>>>>>>>>> pradeep1288@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I just cloned the Impala source from github, and I am trying to
>>>>>>>>>>> compile the front end per Impala wiki. I get the error below. I am using
>>>>>>>>>>> maven 3.
>>>>>>>>>>>
>>>>>>>>>>> [ERROR] Failed to execute goal on project impala-frontend: Could
>>>>>>>>>>> not resolve dependencies for project
>>>>>>>>>>> com.cloudera.impala:impala-frontend:jar:0.1-SNAPSHOT: Failure to find
>>>>>>>>>>> com.cloudera.impala:impala-data-source-api:jar:1.0-SNAPSHOT in
>>>>>>>>>>> https://repository.cloudera.com/content/groups/cdh-releases-rcs
>>>>>>>>>>> was cached in the local repository, resolution will not be reattempted
>>>>>>>>>>> until the update interval of cdh.rcs.releases.repo has elapsed or updates
>>>>>>>>>>> are forced -> [Help 1
>>>>>>>>>>>
>>>>>>>>>>> This is the command I used:   mvn clean package
>>>>>>>>>>> dependency:copy-dependencies -DskipTests=true
>>>>>>>>>>>
>>>>>>>>>>> Any idea getting around this problem ?
>>>>>>>>>>>
>>>>>>>>>>> Cheers!
>>>>>>>>>>> Pradeep
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>> Google Groups "Impala Dev" group.
>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>> it, send an email to impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>>> Groups "Impala Dev" group.
>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>> send an email to impala-dev+unsubscribe@cloudera.org.
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>