You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2018/06/08 02:32:00 UTC
[jira] [Resolved] (SPARK-24482) Not able to run simple SOLR
application
[ https://issues.apache.org/jira/browse/SPARK-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-24482.
----------------------------------
Resolution: Invalid
It sounds like more specific to the thirdpart library. Let's leave this resolved until it looks clear to an issue within Spark.
> Not able to run simple SOLR application
> ---------------------------------------
>
> Key: SPARK-24482
> URL: https://issues.apache.org/jira/browse/SPARK-24482
> Project: Spark
> Issue Type: Brainstorming
> Components: Spark Submit, YARN
> Affects Versions: 1.6.0
> Reporter: Pravin Dahare
> Priority: Major
>
> I am not able to run solr applciation may be due to "org.apache.httpcomponents" jars.
> I am using "org.apache.httpcomponents" 4.3.1 jars and my cluster has old jars:4.2.5.
> Event though I am excluding 4.2.5 jars , issue still exists. Adding exception at bottom. Any help is appreciated.
> =================================================
> spark2-submit \
> --master yarn \
> --deploy-mode client \
> --class com.citi.solrsearch.SolrSearch \
> --conf spark.executor.extraJavaOptions=-Djava.security.auth.login.config=/home/icttdnee/Sprinter/solrCitiscreening/jaas-client.conf \
> --conf spark.jars.excludes=/opt/cloudera/parcels/CDH-5.9.1-1.cdh5.9.1.p2964.3211/jars/httpclient-4.2.5.jar,/opt/cloudera/parcels/CDH-5.9.1-1.cdh5.9.1.p2964.3211/jars/httpcore-4.2.5.jar \
> --conf spark.driver.userClassPathFirst=false \
> --conf spark.executor.userClassPathFirst=false \
> --conf spark.driver.extraClassPath=/home/icttdnee/citiscreen_solr/httpclient-4.3.1.jar,/home/icttdnee/citiscreen_solr/httpcore-4.3.1.jar,/home/icttdnee/citiscreen_solr/httpclient-4.3.1.jar,/home/icttdnee/citiscreen_solr/httpcore-4.3.1.jar \
> --jars /home/icttdnee/citiscreen_solr/spark-solr-2.1.0.jar,/home/pd94468/scala-logging_2.11-3.4.0.jar,/home/icttdnee/citiscreen_solr/spark-core_2.11-1.6.0.jar,/home/icttdnee/citiscreen_solr/httpclient-4.3.1.jar,/home/icttdnee/citiscreen_solr/httpcore-4.3.1.jar \
> /home/icttdnee/citiscreen_solr/spark-solr-1.0-SNAPSHOT-jar-with-dependencies.jar \
> https://bdgtr012x07h2.nam.nsroot.net:8985/solr/icttdnee_ttsd_collection \
> &q=alert_id:[16000%20TO%20*]
> =================================================
> Maven Snippet:
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-core_2.10</artifactId>
> <version>2.0.2</version>
> </dependency>
> <!-- https://mvnrepository.com/artifact/com.lucidworks.spark/spark-solr -->
> <dependency>
> <groupId>com.lucidworks.spark</groupId>
> <artifactId>spark-solr</artifactId>
> <version>3.5.0</version>
> </dependency>
> <!-- https://mvnrepository.com/artifact/org.apache.solr/solr-core -->
> <dependency>
> <groupId>org.apache.solr</groupId>
> <artifactId>solr-core</artifactId>
> <version>4.10.3</version>
> </dependency>
> <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->
> <dependency>
> <groupId>org.apache.httpcomponents</groupId>
> <artifactId>httpclient</artifactId>
> <version>4.3.1</version>
> </dependency>
> <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpcore -->
> <dependency>
> <groupId>org.apache.httpcomponents</groupId>
> <artifactId>httpcore</artifactId>
> <version>4.3.1</version>
> </dependency>
> ==============================================================
>
>
> 18/06/07 05:08:00 INFO spark.SparkContext: Registered listener com.pepperdata.spark.metrics.PepperdataSparkListener
> Exception in thread "main" com.google.common.util.concurrent.ExecutionError: java.lang.NoSuchFieldError: INSTANCE
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2232)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969)
> at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829)
> at com.lucidworks.spark.util.SolrSupport$.getCachedCloudClient(SolrSupport.scala:245)
> at com.lucidworks.spark.util.SolrQuerySupport$.getUniqueKey(SolrQuerySupport.scala:91)
> at com.lucidworks.spark.rdd.SolrRDD.<init>(SolrRDD.scala:40)
> at com.lucidworks.spark.rdd.SelectSolrRDD.<init>(SelectSolrRDD.scala:30)
> at com.lucidworks.spark.rdd.SelectSolrRDD$.apply(SelectSolrRDD.scala:168)
> at com.lucidworks.spark.rdd.SolrJavaRDD.get(SolrJavaRDD.java:48)
> at com.citi.solrsearch.SolrSearch.main(SolrSearch.java:48)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
> at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> Caused by: java.lang.NoSuchFieldError: INSTANCE
> at org.apache.http.impl.conn.HttpClientConnectionOperator.<init>(HttpClientConnectionOperator.java:75)
> at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:149)
> at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:136)
> at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:112)
> at org.apache.solr.client.solrj.impl.HttpClientUtil.createPoolingConnectionManager(HttpClientUtil.java:228)
> at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:223)
> at com.lucidworks.spark.util.SolrSupport$.getSolrCloudClient(SolrSupport.scala:218)
> at com.lucidworks.spark.util.SolrSupport$.getNewSolrCloudClient(SolrSupport.scala:241)
> at com.lucidworks.spark.util.CacheCloudSolrClient$$anon$1.load(SolrSupport.scala:39)
> at com.lucidworks.spark.util.CacheCloudSolrClient$$anon$1.load(SolrSupport.scala:37)
> at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
> at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
> at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org