You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by jay vyas <ja...@gmail.com> on 2015/01/29 00:12:01 UTC
spark akka fork : is the source anywhere?
Hi spark. Where is akka coming from in spark ?
I see the distribution referenced is a spark artifact... but not in the
apache namespace.
<akka.group>org.spark-project.akka</akka.group>
<akka.version>2.3.4-spark</akka.version>
Clearly this is a deliberate thought out change (See SPARK-1812), but its
not clear where 2.3.4 spark is coming from and who is maintaining its
release?
--
jay vyas
PS
I've had some conversations with will benton as well about this, and its
clear that some modifications to akka are needed, or else a protobug error
occurs, which amount to serialization incompatibilities, hence if one wants
to build spark from sources, the patched akka is required (or else, manual
patching needs to be done)...
15/01/28 22:58:10 ERROR ActorSystemImpl: Uncaught fatal error from thread
[sparkWorker-akka.remote.default-remote-dispatcher-6] shutting down
ActorSystem [sparkWorker] java.lang.VerifyError: class
akka.remote.WireFormats$AkkaControlMessage overrides final method
getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
Re: spark akka fork : is the source anywhere?
Posted by Patrick Wendell <pw...@gmail.com>.
It's maintained here:
https://github.com/pwendell/akka/tree/2.2.3-shaded-proto
Over time, this is something that would be great to get rid of, per rxin
On Wed, Jan 28, 2015 at 3:33 PM, Reynold Xin <rx...@databricks.com> wrote:
> Hopefully problems like this will go away entirely in the next couple of
> releases. https://issues.apache.org/jira/browse/SPARK-5293
>
>
>
> On Wed, Jan 28, 2015 at 3:12 PM, jay vyas <ja...@gmail.com>
> wrote:
>
>> Hi spark. Where is akka coming from in spark ?
>>
>> I see the distribution referenced is a spark artifact... but not in the
>> apache namespace.
>>
>> <akka.group>org.spark-project.akka</akka.group>
>> <akka.version>2.3.4-spark</akka.version>
>>
>> Clearly this is a deliberate thought out change (See SPARK-1812), but its
>> not clear where 2.3.4 spark is coming from and who is maintaining its
>> release?
>>
>> --
>> jay vyas
>>
>> PS
>>
>> I've had some conversations with will benton as well about this, and its
>> clear that some modifications to akka are needed, or else a protobug error
>> occurs, which amount to serialization incompatibilities, hence if one wants
>> to build spark from sources, the patched akka is required (or else, manual
>> patching needs to be done)...
>>
>> 15/01/28 22:58:10 ERROR ActorSystemImpl: Uncaught fatal error from thread
>> [sparkWorker-akka.remote.default-remote-dispatcher-6] shutting down
>> ActorSystem [sparkWorker] java.lang.VerifyError: class
>> akka.remote.WireFormats$AkkaControlMessage overrides final method
>> getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: spark akka fork : is the source anywhere?
Posted by Reynold Xin <rx...@databricks.com>.
Hopefully problems like this will go away entirely in the next couple of
releases. https://issues.apache.org/jira/browse/SPARK-5293
On Wed, Jan 28, 2015 at 3:12 PM, jay vyas <ja...@gmail.com>
wrote:
> Hi spark. Where is akka coming from in spark ?
>
> I see the distribution referenced is a spark artifact... but not in the
> apache namespace.
>
> <akka.group>org.spark-project.akka</akka.group>
> <akka.version>2.3.4-spark</akka.version>
>
> Clearly this is a deliberate thought out change (See SPARK-1812), but its
> not clear where 2.3.4 spark is coming from and who is maintaining its
> release?
>
> --
> jay vyas
>
> PS
>
> I've had some conversations with will benton as well about this, and its
> clear that some modifications to akka are needed, or else a protobug error
> occurs, which amount to serialization incompatibilities, hence if one wants
> to build spark from sources, the patched akka is required (or else, manual
> patching needs to be done)...
>
> 15/01/28 22:58:10 ERROR ActorSystemImpl: Uncaught fatal error from thread
> [sparkWorker-akka.remote.default-remote-dispatcher-6] shutting down
> ActorSystem [sparkWorker] java.lang.VerifyError: class
> akka.remote.WireFormats$AkkaControlMessage overrides final method
> getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
>