Error making REST call from streaming app

["Afshartous, Nick" <na...@turbine.com>]

Hi,


We got the following exception trying to initiate a REST call from the Spark app.


This is running Spark 1.5.2 in AWS / Yarn.  Its only happened one time during the course of a streaming app

that has been running for months.


Just curious if anyone could shed some more light on root cause.


Thanks,

--

    Nick


> User class threw exception: org.apache.spark.SparkException: Job aborted due to stage failure: Task 323 in stage 15154.0 failed 4 times, most recent failure: Lost task 323.3 in stage 15154.0 (TID 2010826, ip-10-247-128-182.ec2.internal): com.sun.jersey.spi.service.ServiceConfigurationError: com.sun.jersey.spi.inject.InjectableProvider: : java.io.FileNotFoundException: /mnt/yarn/usercache/hadoop/appcache/application_1452625196513_0026/container_1452625196513_0026_02_000003/__app__.jar (No such file or directory)
> at com.sun.jersey.spi.service.ServiceFinder.fail(ServiceFinder.java:610)
> at com.sun.jersey.spi.service.ServiceFinder.parse(ServiceFinder.java:682)
> at com.sun.jersey.spi.service.ServiceFinder.access$500(ServiceFinder.java:159)
> at com.sun.jersey.spi.service.ServiceFinder$AbstractLazyIterator.hasNext(ServiceFinder.java:739)
> at com.sun.jersey.spi.service.ServiceFinder.toClassArray(ServiceFinder.java:595)
> at com.sun.jersey.core.spi.component.ProviderServices.getServiceClasses(ProviderServices.java:318)
> at com.sun.jersey.core.spi.component.ProviderServices.getProviderAndServiceClasses(ProviderServices.java:297)
> at com.sun.jersey.core.spi.component.ProviderServices.getProvidersAndServices(ProviderServices.java:204)
> at com.sun.jersey.core.spi.factory.InjectableProviderFactory.configure(InjectableProviderFactory.java:106)
> at com.sun.jersey.api.client.Client.init(Client.java:263)
> at com.sun.jersey.api.client.Client.access$000(Client.java:118)
> at com.sun.jersey.api.client.Client$1.f(Client.java:191)
> at com.sun.jersey.api.client.Client$1.f(Client.java:187)
> at com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:193)
> at com.sun.jersey.api.client.Client.<init>(Client.java:187)
> at com.sun.jersey.api.client.Client.<init>(Client.java:159)
> at com.sun.jersey.api.client.Client.create(Client.java:669)
> at com.wb.analytics.schemaservice.fingerprint.FingerPrintRestClient.getSchema(FingerPrintRestClient.java:48)
> at com.wb.analytics.schemaservice.fingerprint.FingerPrintService.getSchemaFromService(FingerPrintService.java:80)