You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Joe Xie <zh...@fb.com> on 2011/08/22 23:19:57 UTC
About Running Spark on a cluster with mesos Re: WELCOME to
mesos-dev@incubator.apache.org
Hi there:
I has build and installed mesos (pre-protobuf branch for spark) and spark,
and deployed them to a cluster. However, I have following error when I run
spark task on the mesos URL of my master, the spark task runs fine in
local mode. Any suggestion or help? Thanks a lot!
Joe
My os: CentOS 5.2
Python: 2.6
Gcc: 4.2.1
Swig:1.3.40
./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor on port
50501
11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered actor on
port 50501
11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0 with cache
11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0 with 1
partitions
11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current cache
locations
11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0
11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final stage: List()
11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents: List()
11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0, which has
no missing parents
11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1 tasks
Exception in thread "Spark scheduler" java.lang.UnsatisfiedLinkError:
org.apache.mesos.MesosSchedulerDriver.initialize()V
at org.apache.mesos.MesosSchedulerDriver.initialize(Native Method)
at
org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:42)
at
org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:46)
at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
Sent to wrong address previously
>
>
>On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
><me...@incubator.apache.org> wrote:
>
>>Hi! This is the ezmlm program. I'm managing the
>>mesos-dev@incubator.apache.org mailing list.
>>
>>I'm working for my owner, who can be reached
>>at mesos-dev-owner@incubator.apache.org.
>>
>>Acknowledgment: I have added the address
>>
>> zhiyong@fb.com
>>
>>to the mesos-dev mailing list.
>>
>>Welcome to mesos-dev@incubator.apache.org!
>>
>>Please save this message so that you know the address you are
>>subscribed under, in case you later want to unsubscribe or change your
>>subscription address.
>>
>>
>>--- Administrative commands for the mesos-dev list ---
>>
>>I can handle administrative requests automatically. Please
>>do not send them to the list address! Instead, send
>>your message to the correct command address:
>>
>>To subscribe to the list, send a message to:
>> <me...@incubator.apache.org>
>>
>>To remove your address from the list, send a message to:
>> <me...@incubator.apache.org>
>>
>>Send mail to the following for info and FAQ for this list:
>> <me...@incubator.apache.org>
>> <me...@incubator.apache.org>
>>
>>Similar addresses exist for the digest list:
>> <me...@incubator.apache.org>
>> <me...@incubator.apache.org>
>>
>>To get messages 123 through 145 (a maximum of 100 per request), mail:
>> <me...@incubator.apache.org>
>>
>>To get an index with subject and author for messages 123-456 , mail:
>> <me...@incubator.apache.org>
>>
>>They are always returned as sets of 100, max 2000 per request,
>>so you'll actually get 100-499.
>>
>>To receive all messages with the same subject as message 12345,
>>send a short message to:
>> <me...@incubator.apache.org>
>>
>>The messages should contain one line or word of text to avoid being
>>treated as sp@m, but I will ignore their content.
>>Only the ADDRESS you send to is important.
>>
>>You can start a subscription for an alternate address,
>>for example "john@host.domain", just add a hyphen and your
>>address (with '=' instead of '@') after the command word:
>><me...@incubator.apache.org>
>>
>>To stop subscription for this address, mail:
>><me...@incubator.apache.org>
>>
>>In both cases, I'll send a confirmation message to that address. When
>>you receive it, simply reply to it to complete your subscription.
>>
>>If despite following these instructions, you do not get the
>>desired results, please contact my owner at
>>mesos-dev-owner@incubator.apache.org. Please be patient, my owner is a
>>lot slower than I am ;-)
>>
>>--- Enclosed is a copy of the request I received.
>>
>>Return-Path: <zh...@fb.com>
>>Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07 -0000
>>Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:07
>>+0000
>>X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>> tests=SPF_PASS
>>X-Spam-Check-By: apache.org
>>Received-SPF: pass (nike.apache.org: domain of zhiyong@fb.com designates
>>67.231.145.42 as permitted sender)
>>Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>(67.231.145.42)
>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:00
>>+0000
>>Received: from pps.filterd (m0004348 [127.0.0.1])
>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id p7LNTI6g015638
>> for
>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.ap
>>a
>>che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>[66.220.144.38])
>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>> for
>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.ap
>>a
>>che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>Received: from SC-MBX01-5.TheFacebook.com ([fe80::f05e:c7f1:c7dd:e640])
>>by
>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id 14.01.0289.001;
>>Sun,
>> 21 Aug 2011 16:29:14 -0700
>>From: Joe Xie <zh...@fb.com>
>>To:
>>"mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.ap
>>a
>>che.org"
>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.a
>>p
>>ache.org>
>>Subject: Re: confirm subscribe to mesos-dev@incubator.apache.org
>>Thread-Topic: confirm subscribe to mesos-dev@incubator.apache.org
>>Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>Date: Sun, 21 Aug 2011 23:29:13 +0000
>>Message-ID: <CA...@fb.com>
>>In-Reply-To: <13...@incubator.apache.org>
>>Accept-Language: en-US
>>Content-Language: en-US
>>X-MS-Has-Attach:
>>X-MS-TNEF-Correlator:
>>x-originating-ip: [192.168.18.252]
>>Content-Type: text/plain; charset="us-ascii"
>>Content-ID: <B4...@fb.com>
>>Content-Transfer-Encoding: quoted-printable
>>MIME-Version: 1.0
>>X-Virus-Checked: Checked by ClamAV on apache.org
>>
>>Thanks!
>>Joe
>>On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>><me...@incubator.apache.org> wrote:
>>
>>>Hi! This is the ezmlm program. I'm managing the
>>>mesos-dev@incubator.apache.org mailing list.
>>>
>>>I'm working for my owner, who can be reached
>>>at mesos-dev-owner@incubator.apache.org.
>>>
>>>To confirm that you would like
>>>
>>> zhiyong@fb.com
>>>
>>>added to the mesos-dev mailing list, please send
>>>a short reply to this address:
>>>
>>> =20
>>>mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@incubator.
>>>a
>>>p=
>>ac
>>>he.org
>>>
>>>Usually, this happens when you just hit the "reply" button.
>>>If this does not work, simply copy the address and paste it into
>>>the "To:" field of a new message.
>>>
>>>or click here:
>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@in
>>>c
>>>u=
>>ba
>>>tor.apache.org
>>>
>>>This confirmation serves two purposes. First, it verifies that I am able
>>>to get mail through to you. Second, it protects you in case someone
>>>forges a subscription request in your name.
>>>
>>>Please note that ALL Apache dev- and user- mailing lists are publicly
>>>archived. Do familiarize yourself with Apache's public archive policy
>>>at
>>>
>>> http://www.apache.org/foundation/public-archives.html
>>>
>>>prior to subscribing and posting messages to
>>>mesos-dev@incubator.apache.org.
>>>If you're not sure whether or not the policy applies to this mailing
>>>list,
>>>assume it does unless the list name contains the word "private" in it.
>>>
>>>Some mail programs are broken and cannot handle long addresses. If you
>>>cannot reply to this request, instead send a message to
>>><me...@incubator.apache.org> and put the
>>>entire address listed above into the "Subject:" line.
>>>
>>>
>>>--- Administrative commands for the mesos-dev list ---
>>>
>>>I can handle administrative requests automatically. Please
>>>do not send them to the list address! Instead, send
>>>your message to the correct command address:
>>>
>>>To subscribe to the list, send a message to:
>>> <me...@incubator.apache.org>
>>>
>>>To remove your address from the list, send a message to:
>>> <me...@incubator.apache.org>
>>>
>>>Send mail to the following for info and FAQ for this list:
>>> <me...@incubator.apache.org>
>>> <me...@incubator.apache.org>
>>>
>>>Similar addresses exist for the digest list:
>>> <me...@incubator.apache.org>
>>> <me...@incubator.apache.org>
>>>
>>>To get messages 123 through 145 (a maximum of 100 per request), mail:
>>> <me...@incubator.apache.org>
>>>
>>>To get an index with subject and author for messages 123-456 , mail:
>>> <me...@incubator.apache.org>
>>>
>>>They are always returned as sets of 100, max 2000 per request,
>>>so you'll actually get 100-499.
>>>
>>>To receive all messages with the same subject as message 12345,
>>>send a short message to:
>>> <me...@incubator.apache.org>
>>>
>>>The messages should contain one line or word of text to avoid being
>>>treated as sp@m, but I will ignore their content.
>>>Only the ADDRESS you send to is important.
>>>
>>>You can start a subscription for an alternate address,
>>>for example "john@host.domain", just add a hyphen and your
>>>address (with '=3D' instead of '@') after the command word:
>>><me...@incubator.apache.org>
>>>
>>>To stop subscription for this address, mail:
>>><me...@incubator.apache.org>
>>>
>>>In both cases, I'll send a confirmation message to that address. When
>>>you receive it, simply reply to it to complete your subscription.
>>>
>>>If despite following these instructions, you do not get the
>>>desired results, please contact my owner at
>>>mesos-dev-owner@incubator.apache.org. Please be patient, my owner is a
>>>lot slower than I am ;-)
>>>
>>>--- Enclosed is a copy of the request I received.
>>>
>>>Return-Path: <zh...@fb.com>
>>>Received: (qmail 19986 invoked by uid 99); 21 Aug 2011 22:43:26 -0000
>>>Received: from athena.apache.org (HELO athena.apache.org)
>>>(140.211.11.136)
>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:26
>>>+0000
>>>X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>X-Spam-Check-By: apache.org
>>>Received-SPF: pass (athena.apache.org: domain of zhiyong@fb.com
>>>designates 67.231.145.42 as permitted sender)
>>>Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>(67.231.145.42)
>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:19
>>>+0000
>>>Received: from pps.filterd (m0004347 [127.0.0.1])
>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id p7LMcbSq017336
>>> for <me...@incubator.apache.org>; Sun, 21 Aug 2011
>>>15:42:58 -0700
>>>Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>[66.220.144.38])
>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128 verify=3DNOT)
>>> for <me...@incubator.apache.org>; Sun, 21 Aug 2011
>>>15:42:58 -0700
>>>Received: from SC-MBX01-5.TheFacebook.com ([fe80::f05e:c7f1:c7dd:e640])
>>>by
>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>14.01.0289.001;
>>>Sun,
>>> 21 Aug 2011 15:42:53 -0700
>>>From: Joe Xie <zh...@fb.com>
>>>To: "mesos-dev-subscribe@incubator.apache.org"
>>> <me...@incubator.apache.org>
>>>Subject: can i join this
>>>Thread-Topic: can i join this
>>>Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>Message-ID: <CA...@fb.com>
>>>Accept-Language: en-US
>>>Content-Language: en-US
>>>X-MS-Has-Attach:=20
>>>X-MS-TNEF-Correlator:
>>>x-originating-ip: [192.168.18.252]
>>>Content-Type: multipart/alternative;
>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>MIME-Version: 1.0
>>>
>>>--_000_CA76D7F74256zhiyongfbcom_
>>>Content-Type: text/plain; charset=3D"us-ascii"
>>>Content-Transfer-Encoding: quoted-printable
>>>
>>>Best,
>>>Joe
>>>
>>>--_000_CA76D7F74256zhiyongfbcom_
>>>Content-Type: text/html; charset=3D"us-ascii"
>>>Content-ID: <F6...@fb.com>
>>>Content-Transfer-Encoding: quoted-printable
>>>
>>><html>
>>><head>
>>><meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>charset=3D3Dus-ascii"=3D
>>>>
>>></head>
>>><body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode: space;
>>>-webkit-lin=3D
>>>e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px;
>>>font-fami=3D
>>>ly: Calibri, sans-serif; ">
>>><div>Best,</div>
>>><div>Joe</div>
>>></body>
>>></html>
>>>
>>>--_000_CA76D7F74256zhiyongfbcom_--
>>
>
Re: About Running Spark on a cluster with mesos Re: WELCOME to mesos-dev@incubator.apache.org
Posted by Matei Zaharia <ma...@eecs.berkeley.edu>.
Great. Yeah, .bashrc doesn't get read in processes that were launched through SSH.
Matei
On Aug 22, 2011, at 4:42 PM, Joe Xie wrote:
> Thank you Matei! It's working now. I checked the stderr, it seems I have
> to specify the SCALA_HOME even if it's in .bashrc.
> Best,
> Joe
>
> On 8/22/11 4:13 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>
>> This probably means that your task crashed. Take a look at the logs Mesos
>> collects for your task, which will be in
>> MESOS_HOME/work/slave-id/framework-id/attempt-id/stdout and stderr.
>>
>> Sometimes this happens because spark-env.sh isn't configured on all the
>> nodes to contain SCALA_HOME, JAVA_HOME and MESOS_HOME. It can also happen
>> because your code is not available on all the machines and you get a
>> ClassNotFoundError. You can either copy the code to the same place
>> everywhere and put that in the SPARK_CLASSPATH variable in spark-env.sh,
>> or package your code into a JAR and use the "jars" parameter to
>> SparkContext to pass it (in which case Spark will copy it to all the
>> nodes).
>>
>> Matei
>>
>> On Aug 22, 2011, at 4:04 PM, Joe Xie wrote:
>>
>>> Btw, I checkout my webui for master, it indicates "No offers are active"
>>> under "Resource Offers" while I have two machines as slave, so there
>>> might
>>> be problem with my configuration.
>>> Best,
>>> JOE
>>>
>>> On 8/22/11 3:59 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>
>>>> Thank you Matei,
>>>>
>>>> it's running for me with Spark 0.3 + pre-protobuf, and I'm testing with
>>>> provided example "run spark.examples.SparkPi
>>>> mesos://1@10.30.95.184:5050
>>>> 3". However, it seems I keep losing TID after starting the task, and
>>>> then
>>>> job aborted with 4 tries. It might be the problem on my side too, did
>>>> this
>>>> happen before? I could go for trying masters for spark + mesos.
>>>> 11/08/22 15:48:27 INFO spark.SimpleJob: Lost TID 13 (task 0:2)
>>>> 11/08/22 15:48:27 ERROR spark.SimpleJob: Task 0:2 failed more than 4
>>>> times; aborting job
>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from TID
>>>> 12
>>>> because its job is gone
>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from TID
>>>> 14
>>>> because its job is gone
>>>>
>>>> Thanks!
>>>> Joe
>>>>
>>>>
>>>>
>>>> On 8/22/11 3:38 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>
>>>>> Thank you Matei! I can go for Spark 0.3 + pre-protobuf. I understand
>>>>> there
>>>>> might be lots of new updates on the master branch. Is there any
>>>>> significant difference in current master with Spark 0.3 +
>>>>> pre-protobuf? I
>>>>> could also try to keep a separate branch with stable masters if that's
>>>>> the
>>>>> case.
>>>>> Best,
>>>>> Joe
>>>>> On 8/22/11 3:21 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>>>
>>>>>> If you want the most stable versions right now, I recommend Spark 0.3
>>>>>> (download it from the website or the 0.3-scala-2.9 tag in Git) and
>>>>>> pre-protobuf Mesos. However, the current master Spark + master Mesos
>>>>>> should also work OK; it's just that further commits may break things
>>>>>> temporarily.
>>>>>>
>>>>>> Matei
>>>>>>
>>>>>> On Aug 22, 2011, at 3:17 PM, Joe Xie wrote:
>>>>>>
>>>>>>> Hi Matei,
>>>>>>> Thanks for your feedback. I bet I'm using the master brach of Spark
>>>>>>> and
>>>>>>> pre-protobuf version of Mesos. Do you mean I should go for Spark
>>>>>>> (master)
>>>>>>> + protobuf (master). Let's me know the most stable combination of
>>>>>>> braches
>>>>>>> from Spark and pre-protobuf, I can try them out.
>>>>>>>
>>>>>>> Best,
>>>>>>> Joe
>>>>>>>
>>>>>>>
>>>>>>> On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>>>>>
>>>>>>>> Are you using the master branch of Spark, or one of the releases
>>>>>>>> (e.g.
>>>>>>>> 0.3)? The releases work with the pre-protobuf version of Mesos but
>>>>>>>> the
>>>>>>>> master branch of Spark actually requires the master branch of Mesos
>>>>>>>> now.
>>>>>>>> That could cause the kind of error you're seeing.
>>>>>>>>
>>>>>>>> Matei
>>>>>>>>
>>>>>>>> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>>>>>>>>
>>>>>>>>> Sorry this should have been sent to user mail list.
>>>>>>>>>
>>>>>>>>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi there:
>>>>>>>>>>
>>>>>>>>>> I has build and installed mesos (pre-protobuf branch for spark)
>>>>>>>>>> and
>>>>>>>>>> spark,
>>>>>>>>>> and deployed them to a cluster. However, I have following error
>>>>>>>>>> when
>>>>>>>>>> I
>>>>>>>>>> run
>>>>>>>>>> spark task on the mesos URL of my master, the spark task runs
>>>>>>>>>> fine
>>>>>>>>>> in
>>>>>>>>>> local mode. Any suggestion or help? Thanks a lot!
>>>>>>>>>> Joe
>>>>>>>>>>
>>>>>>>>>> My os: CentOS 5.2
>>>>>>>>>> Python: 2.6
>>>>>>>>>> Gcc: 4.2.1
>>>>>>>>>> Swig:1.3.40
>>>>>>>>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor
>>>>>>>>>> on
>>>>>>>>>> port
>>>>>>>>>> 50501
>>>>>>>>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered
>>>>>>>>>> actor
>>>>>>>>>> on
>>>>>>>>>> port 50501
>>>>>>>>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0
>>>>>>>>>> with
>>>>>>>>>> cache
>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0
>>>>>>>>>> with
>>>>>>>>>> 1
>>>>>>>>>> partitions
>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current
>>>>>>>>>> cache
>>>>>>>>>> locations
>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0
>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final
>>>>>>>>>> stage:
>>>>>>>>>> List()
>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents:
>>>>>>>>>> List()
>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0,
>>>>>>>>>> which
>>>>>>>>>> has
>>>>>>>>>> no missing parents
>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1
>>>>>>>>>> tasks
>>>>>>>>>> Exception in thread "Spark scheduler"
>>>>>>>>>> java.lang.UnsatisfiedLinkError:
>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>>>>>>>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native
>>>>>>>>>> Method)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.
>>>>>>>>>> ja
>>>>>>>>>> v
>>>>>>>>>> a
>>>>>>>>>> :4
>>>>>>>>>> 2)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.
>>>>>>>>>> ja
>>>>>>>>>> v
>>>>>>>>>> a
>>>>>>>>>> :4
>>>>>>>>>> 6)
>>>>>>>>>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>>>>>>>>
>>>>>>>>>> Sent to wrong address previously
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>
>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>
>>>>>>>>>>>> Acknowledgment: I have added the address
>>>>>>>>>>>>
>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>
>>>>>>>>>>>> to the mesos-dev mailing list.
>>>>>>>>>>>>
>>>>>>>>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>>>>>>>>
>>>>>>>>>>>> Please save this message so that you know the address you are
>>>>>>>>>>>> subscribed under, in case you later want to unsubscribe or
>>>>>>>>>>>> change
>>>>>>>>>>>> your
>>>>>>>>>>>> subscription address.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>
>>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>
>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per request),
>>>>>>>>>>>> mail:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> To get an index with subject and author for messages 123-456 ,
>>>>>>>>>>>> mail:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>
>>>>>>>>>>>> To receive all messages with the same subject as message 12345,
>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>>>>> being
>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>
>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>>> address (with '=' instead of '@') after the command word:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>> address.
>>>>>>>>>>>> When
>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>> subscription.
>>>>>>>>>>>>
>>>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my
>>>>>>>>>>>> owner
>>>>>>>>>>>> is
>>>>>>>>>>>> a
>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>
>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>
>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07
>>>>>>>>>>>> -0000
>>>>>>>>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>>>>>>>> (192.87.106.230)
>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>> 23:30:07
>>>>>>>>>>>> +0000
>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>>>>>>>>> tests=SPF_PASS
>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>> Received-SPF: pass (nike.apache.org: domain of zhiyong@fb.com
>>>>>>>>>>>> designates
>>>>>>>>>>>> 67.231.145.42 as permitted sender)
>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>> mx0a-00082601.pphosted.com)
>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>> 23:30:00
>>>>>>>>>>>> +0000
>>>>>>>>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>>>>>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>> p7LNTI6g015638
>>>>>>>>>>>> for
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@inc
>>>>>>>>>>>> ub
>>>>>>>>>>>> a
>>>>>>>>>>>> t
>>>>>>>>>>>> or
>>>>>>>>>>>> .a
>>>>>>>>>>>> p
>>>>>>>>>>>> a
>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>>>>>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>>>>>>>>>>>> for
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@inc
>>>>>>>>>>>> ub
>>>>>>>>>>>> a
>>>>>>>>>>>> t
>>>>>>>>>>>> or
>>>>>>>>>>>> .a
>>>>>>>>>>>> p
>>>>>>>>>>>> a
>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>> by
>>>>>>>>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>> Sun,
>>>>>>>>>>>> 21 Aug 2011 16:29:14 -0700
>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>> To:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> "mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@inc
>>>>>>>>>>>> ub
>>>>>>>>>>>> a
>>>>>>>>>>>> t
>>>>>>>>>>>> or
>>>>>>>>>>>> .a
>>>>>>>>>>>> p
>>>>>>>>>>>> a
>>>>>>>>>>>> che.org"
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@in
>>>>>>>>>>>> cu
>>>>>>>>>>>> b
>>>>>>>>>>>> a
>>>>>>>>>>>> to
>>>>>>>>>>>> r.
>>>>>>>>>>>> a
>>>>>>>>>>>> p
>>>>>>>>>>>> ache.org>
>>>>>>>>>>>> Subject: Re: confirm subscribe to
>>>>>>>>>>>> mesos-dev@incubator.apache.org
>>>>>>>>>>>> Thread-Topic: confirm subscribe to
>>>>>>>>>>>> mesos-dev@incubator.apache.org
>>>>>>>>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>>>>>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>> In-Reply-To: <13...@incubator.apache.org>
>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>> X-MS-Has-Attach:
>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>>>>>>>> Content-ID: <B4...@fb.com>
>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>> Joe
>>>>>>>>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>>
>>>>>>>>>>>>> To confirm that you would like
>>>>>>>>>>>>>
>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>
>>>>>>>>>>>>> added to the mesos-dev mailing list, please send
>>>>>>>>>>>>> a short reply to this address:
>>>>>>>>>>>>>
>>>>>>>>>>>>> =20
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@i
>>>>>>>>>>>>> nc
>>>>>>>>>>>>> u
>>>>>>>>>>>>> b
>>>>>>>>>>>>> at
>>>>>>>>>>>>> or
>>>>>>>>>>>>> .
>>>>>>>>>>>>> a
>>>>>>>>>>>>> p=
>>>>>>>>>>>> ac
>>>>>>>>>>>>> he.org
>>>>>>>>>>>>>
>>>>>>>>>>>>> Usually, this happens when you just hit the "reply" button.
>>>>>>>>>>>>> If this does not work, simply copy the address and paste it
>>>>>>>>>>>>> into
>>>>>>>>>>>>> the "To:" field of a new message.
>>>>>>>>>>>>>
>>>>>>>>>>>>> or click here:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3D
>>>>>>>>>>>>> fb
>>>>>>>>>>>>> .
>>>>>>>>>>>>> c
>>>>>>>>>>>>> om
>>>>>>>>>>>>> @i
>>>>>>>>>>>>> n
>>>>>>>>>>>>> c
>>>>>>>>>>>>> u=
>>>>>>>>>>>> ba
>>>>>>>>>>>>> tor.apache.org
>>>>>>>>>>>>>
>>>>>>>>>>>>> This confirmation serves two purposes. First, it verifies
>>>>>>>>>>>>> that I
>>>>>>>>>>>>> am
>>>>>>>>>>>>> able
>>>>>>>>>>>>> to get mail through to you. Second, it protects you in case
>>>>>>>>>>>>> someone
>>>>>>>>>>>>> forges a subscription request in your name.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please note that ALL Apache dev- and user- mailing lists are
>>>>>>>>>>>>> publicly
>>>>>>>>>>>>> archived. Do familiarize yourself with Apache's public
>>>>>>>>>>>>> archive
>>>>>>>>>>>>> policy
>>>>>>>>>>>>> at
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>>>>>>>>
>>>>>>>>>>>>> prior to subscribing and posting messages to
>>>>>>>>>>>>> mesos-dev@incubator.apache.org.
>>>>>>>>>>>>> If you're not sure whether or not the policy applies to this
>>>>>>>>>>>>> mailing
>>>>>>>>>>>>> list,
>>>>>>>>>>>>> assume it does unless the list name contains the word
>>>>>>>>>>>>> "private"
>>>>>>>>>>>>> in
>>>>>>>>>>>>> it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Some mail programs are broken and cannot handle long
>>>>>>>>>>>>> addresses.
>>>>>>>>>>>>> If
>>>>>>>>>>>>> you
>>>>>>>>>>>>> cannot reply to this request, instead send a message to
>>>>>>>>>>>>> <me...@incubator.apache.org> and put the
>>>>>>>>>>>>> entire address listed above into the "Subject:" line.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>>
>>>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>>
>>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>> request),
>>>>>>>>>>>>> mail:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> To get an index with subject and author for messages 123-456 ,
>>>>>>>>>>>>> mail:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>>
>>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>> 12345,
>>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>>>>>> being
>>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>>
>>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>>>> address (with '=3D' instead of '@') after the command word:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>>
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>> address.
>>>>>>>>>>>>> When
>>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>> subscription.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my
>>>>>>>>>>>>> owner
>>>>>>>>>>>>> is a
>>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>>
>>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>> -0000
>>>>>>>>>>>>> Received: from athena.apache.org (HELO athena.apache.org)
>>>>>>>>>>>>> (140.211.11.136)
>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>> +0000
>>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>>>>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>>> Received-SPF: pass (athena.apache.org: domain of
>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>> mx0a-00082601.pphosted.com)
>>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>> 22:43:19
>>>>>>>>>>>>> +0000
>>>>>>>>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>>>>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>>> p7LMcbSq017336
>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug
>>>>>>>>>>>>> 2011
>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>>>>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128
>>>>>>>>>>>>> verify=3DNOT)
>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug
>>>>>>>>>>>>> 2011
>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>>> by
>>>>>>>>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>>> Sun,
>>>>>>>>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>> Subject: can i join this
>>>>>>>>>>>>> Thread-Topic: can i join this
>>>>>>>>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>>>>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>>> X-MS-Has-Attach:=20
>>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>>> Content-Type: multipart/alternative;
>>>>>>>>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>>
>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>
>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>>>>>>>>> Content-ID: <F6...@fb.com>
>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>
>>>>>>>>>>>>> <html>
>>>>>>>>>>>>> <head>
>>>>>>>>>>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>>>>>>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>>>>>>>>
>>>>>>>>>>>>> </head>
>>>>>>>>>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode:
>>>>>>>>>>>>> space;
>>>>>>>>>>>>> -webkit-lin=3D
>>>>>>>>>>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size:
>>>>>>>>>>>>> 14px;
>>>>>>>>>>>>> font-fami=3D
>>>>>>>>>>>>> ly: Calibri, sans-serif; ">
>>>>>>>>>>>>> <div>Best,</div>
>>>>>>>>>>>>> <div>Joe</div>
>>>>>>>>>>>>> </body>
>>>>>>>>>>>>> </html>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: About spark-shell in mesos cluster mode Re: WELCOME to mesos-dev@incubator.apache.org
Posted by Matei Zaharia <ma...@eecs.berkeley.edu>.
Oh OK, got it. If you put the data in the Hadoop filesystem, you can also use a hdfs:// URL with textFile and then all the nodes will be able to access it.
Matei
On Aug 24, 2011, at 9:20 AM, Joe Xie wrote:
> Thanks! I thought all the logs are all at master and did not find there,
> find them on slave and fix the problem now. My problem is that the data
> was not available to slave machines when I use sc.textFile(data) in
> spark-shell cluster mode.
> Best,
> Joe
>
> On 8/24/11 8:27 AM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>
>> Again, you probably need to look at the stderr and stdout logs on the
>> slaves. "task lost" just means that the task crashed or could not be
>> launched.
>>
>> Matei
>>
>> On Aug 24, 2011, at 7:53 AM, Joe Xie wrote:
>>
>>> Hi Matei,
>>>
>>> when I start with "MASTER=1@mymesoshost:5050 ./spark-shell" it seems I
>>> keep losing TID after starting the task, and failed after 4 tries. I'm
>>> trying to looking into code now, but let me know if have any suggestion
>>> too.
>>>
>>> 11/08/24 07:52:34 ERROR spark.SimpleJob: Task 0:6 failed more than 4
>>> times; aborting job
>>> 11/08/24 07:52:34 INFO spark.MesosScheduler: Ignoring update from TID 41
>>> because its job is gone
>>>
>>>
>>> Thanks again,
>>> Joe
>>>
>>> On 8/23/11 6:52 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>
>>>> Thanks! I was use another email.
>>>>
>>>> From: Mail Delivery Subsystem <ma...@googlemail.com>
>>>> Date: Tue, Aug 23, 2011 at 3:29 PM
>>>> Subject: Delivery Status Notification (Failure)
>>>> To: zyxienju@gmail.com
>>>> Delivery to the following recipient failed permanently:
>>>> spark-users@eecs.berkeley.edu
>>>> Technical details of permanent failure:
>>>> Google tried to deliver your message, but it was rejected by the
>>>> recipient
>>>> domain. We recommend contacting the other email provider for further
>>>> information about the cause of this error. The error that the other
>>>> server
>>>> returned was: 550 550 unknown user (state 14).
>>>>
>>>>
>>>>
>>>> On 8/23/11 4:50 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>>
>>>>> Weird, did you subscribe to the spark-users mailing list? I don't
>>>>> think
>>>>> you can post to it without doing that. Go to
>>>>> https://lists.eecs.berkeley.edu/sympa/subscribe/spark-users.
>>>>>
>>>>> Anyway, to run the shell on a cluster, you need to set the MASTER
>>>>> environment variable to the Mesos master URL you want to use (similar
>>>>> to
>>>>> the one you pass to the standalone programs). So for example, type
>>>>>
>>>>> MASTER=1@localhost:5050 ./spark-shell
>>>>>
>>>>> Matei
>>>>>
>>>>>
>>>>> On Aug 23, 2011, at 4:39 PM, Joe Xie wrote:
>>>>>
>>>>>> Hi there,
>>>>>>
>>>>>> may I ask how can I run spark-shell in cluster mode? The web wiki say
>>>>>> change the configuration of master, but I did not find detailed info
>>>>>> yet.
>>>>>> It seems my email failed at spark-user maillist.
>>>>>> Thanks!
>>>>>> Joe
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 8/22/11 4:42 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>
>>>>>>> Thank you Matei! It's working now. I checked the stderr, it seems I
>>>>>>> have
>>>>>>> to specify the SCALA_HOME even if it's in .bashrc.
>>>>>>> Best,
>>>>>>> Joe
>>>>>>>
>>>>>>> On 8/22/11 4:13 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>>>>>
>>>>>>>> This probably means that your task crashed. Take a look at the logs
>>>>>>>> Mesos
>>>>>>>> collects for your task, which will be in
>>>>>>>> MESOS_HOME/work/slave-id/framework-id/attempt-id/stdout and stderr.
>>>>>>>>
>>>>>>>> Sometimes this happens because spark-env.sh isn't configured on all
>>>>>>>> the
>>>>>>>> nodes to contain SCALA_HOME, JAVA_HOME and MESOS_HOME. It can also
>>>>>>>> happen
>>>>>>>> because your code is not available on all the machines and you get
>>>>>>>> a
>>>>>>>> ClassNotFoundError. You can either copy the code to the same place
>>>>>>>> everywhere and put that in the SPARK_CLASSPATH variable in
>>>>>>>> spark-env.sh,
>>>>>>>> or package your code into a JAR and use the "jars" parameter to
>>>>>>>> SparkContext to pass it (in which case Spark will copy it to all
>>>>>>>> the
>>>>>>>> nodes).
>>>>>>>>
>>>>>>>> Matei
>>>>>>>>
>>>>>>>> On Aug 22, 2011, at 4:04 PM, Joe Xie wrote:
>>>>>>>>
>>>>>>>>> Btw, I checkout my webui for master, it indicates "No offers are
>>>>>>>>> active"
>>>>>>>>> under "Resource Offers" while I have two machines as slave, so
>>>>>>>>> there
>>>>>>>>> might
>>>>>>>>> be problem with my configuration.
>>>>>>>>> Best,
>>>>>>>>> JOE
>>>>>>>>>
>>>>>>>>> On 8/22/11 3:59 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>>>
>>>>>>>>>> Thank you Matei,
>>>>>>>>>>
>>>>>>>>>> it's running for me with Spark 0.3 + pre-protobuf, and I'm
>>>>>>>>>> testing
>>>>>>>>>> with
>>>>>>>>>> provided example "run spark.examples.SparkPi
>>>>>>>>>> mesos://1@10.30.95.184:5050
>>>>>>>>>> 3". However, it seems I keep losing TID after starting the task,
>>>>>>>>>> and
>>>>>>>>>> then
>>>>>>>>>> job aborted with 4 tries. It might be the problem on my side too,
>>>>>>>>>> did
>>>>>>>>>> this
>>>>>>>>>> happen before? I could go for trying masters for spark + mesos.
>>>>>>>>>> 11/08/22 15:48:27 INFO spark.SimpleJob: Lost TID 13 (task 0:2)
>>>>>>>>>> 11/08/22 15:48:27 ERROR spark.SimpleJob: Task 0:2 failed more
>>>>>>>>>> than
>>>>>>>>>> 4
>>>>>>>>>> times; aborting job
>>>>>>>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from
>>>>>>>>>> TID
>>>>>>>>>> 12
>>>>>>>>>> because its job is gone
>>>>>>>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from
>>>>>>>>>> TID
>>>>>>>>>> 14
>>>>>>>>>> because its job is gone
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>> Joe
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 8/22/11 3:38 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thank you Matei! I can go for Spark 0.3 + pre-protobuf. I
>>>>>>>>>>> understand
>>>>>>>>>>> there
>>>>>>>>>>> might be lots of new updates on the master branch. Is there any
>>>>>>>>>>> significant difference in current master with Spark 0.3 +
>>>>>>>>>>> pre-protobuf? I
>>>>>>>>>>> could also try to keep a separate branch with stable masters if
>>>>>>>>>>> that's
>>>>>>>>>>> the
>>>>>>>>>>> case.
>>>>>>>>>>> Best,
>>>>>>>>>>> Joe
>>>>>>>>>>> On 8/22/11 3:21 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> If you want the most stable versions right now, I recommend
>>>>>>>>>>>> Spark
>>>>>>>>>>>> 0.3
>>>>>>>>>>>> (download it from the website or the 0.3-scala-2.9 tag in Git)
>>>>>>>>>>>> and
>>>>>>>>>>>> pre-protobuf Mesos. However, the current master Spark + master
>>>>>>>>>>>> Mesos
>>>>>>>>>>>> should also work OK; it's just that further commits may break
>>>>>>>>>>>> things
>>>>>>>>>>>> temporarily.
>>>>>>>>>>>>
>>>>>>>>>>>> Matei
>>>>>>>>>>>>
>>>>>>>>>>>> On Aug 22, 2011, at 3:17 PM, Joe Xie wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Matei,
>>>>>>>>>>>>> Thanks for your feedback. I bet I'm using the master brach of
>>>>>>>>>>>>> Spark
>>>>>>>>>>>>> and
>>>>>>>>>>>>> pre-protobuf version of Mesos. Do you mean I should go for
>>>>>>>>>>>>> Spark
>>>>>>>>>>>>> (master)
>>>>>>>>>>>>> + protobuf (master). Let's me know the most stable
>>>>>>>>>>>>> combination
>>>>>>>>>>>>> of
>>>>>>>>>>>>> braches
>>>>>>>>>>>>> from Spark and pre-protobuf, I can try them out.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Are you using the master branch of Spark, or one of the
>>>>>>>>>>>>>> releases
>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>> 0.3)? The releases work with the pre-protobuf version of
>>>>>>>>>>>>>> Mesos
>>>>>>>>>>>>>> but
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> master branch of Spark actually requires the master branch of
>>>>>>>>>>>>>> Mesos
>>>>>>>>>>>>>> now.
>>>>>>>>>>>>>> That could cause the kind of error you're seeing.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Matei
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Sorry this should have been sent to user mail list.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi there:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I has build and installed mesos (pre-protobuf branch for
>>>>>>>>>>>>>>>> spark)
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> spark,
>>>>>>>>>>>>>>>> and deployed them to a cluster. However, I have following
>>>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>> run
>>>>>>>>>>>>>>>> spark task on the mesos URL of my master, the spark task
>>>>>>>>>>>>>>>> runs
>>>>>>>>>>>>>>>> fine
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> local mode. Any suggestion or help? Thanks a lot!
>>>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> My os: CentOS 5.2
>>>>>>>>>>>>>>>> Python: 2.6
>>>>>>>>>>>>>>>> Gcc: 4.2.1
>>>>>>>>>>>>>>>> Swig:1.3.40
>>>>>>>>>>>>>>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered
>>>>>>>>>>>>>>>> actor
>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>> port
>>>>>>>>>>>>>>>> 50501
>>>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor:
>>>>>>>>>>>>>>>> Registered
>>>>>>>>>>>>>>>> actor
>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>> port 50501
>>>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD
>>>>>>>>>>>>>>>> ID
>>>>>>>>>>>>>>>> 0
>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>> cache
>>>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering
>>>>>>>>>>>>>>>> RDD
>>>>>>>>>>>>>>>> 0
>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>> 1
>>>>>>>>>>>>>>>> partitions
>>>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for
>>>>>>>>>>>>>>>> current
>>>>>>>>>>>>>>>> cache
>>>>>>>>>>>>>>>> locations
>>>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage:
>>>>>>>>>>>>>>>> Stage
>>>>>>>>>>>>>>>> 0
>>>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of
>>>>>>>>>>>>>>>> final
>>>>>>>>>>>>>>>> stage:
>>>>>>>>>>>>>>>> List()
>>>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing
>>>>>>>>>>>>>>>> parents:
>>>>>>>>>>>>>>>> List()
>>>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting
>>>>>>>>>>>>>>>> Stage
>>>>>>>>>>>>>>>> 0,
>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>> no missing parents
>>>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job
>>>>>>>>>>>>>>>> with 1
>>>>>>>>>>>>>>>> tasks
>>>>>>>>>>>>>>>> Exception in thread "Spark scheduler"
>>>>>>>>>>>>>>>> java.lang.UnsatisfiedLinkError:
>>>>>>>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>>>>>>>>>>>>>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native
>>>>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerD
>>>>>>>>>>>>>>>> ri
>>>>>>>>>>>>>>>> v
>>>>>>>>>>>>>>>> er
>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>> ja
>>>>>>>>>>>>>>>> v
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> :4
>>>>>>>>>>>>>>>> 2)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerD
>>>>>>>>>>>>>>>> ri
>>>>>>>>>>>>>>>> v
>>>>>>>>>>>>>>>> er
>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>> ja
>>>>>>>>>>>>>>>> v
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> :4
>>>>>>>>>>>>>>>> 6)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Sent to wrong address previously
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Acknowledgment: I have added the address
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> to the mesos-dev mailing list.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Please save this message so that you know the address you
>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>> subscribed under, in case you later want to unsubscribe
>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>> subscription address.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I can handle administrative requests automatically.
>>>>>>>>>>>>>>>>>> Please
>>>>>>>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Send mail to the following for info and FAQ for this
>>>>>>>>>>>>>>>>>> list:
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>>>>>>> request),
>>>>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To get an index with subject and author for messages
>>>>>>>>>>>>>>>>>> 123-456 ,
>>>>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per
>>>>>>>>>>>>>>>>>> request,
>>>>>>>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>>>>>>> 12345,
>>>>>>>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The messages should contain one line or word of text to
>>>>>>>>>>>>>>>>>> avoid
>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and
>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>> address (with '=' instead of '@') after the command word:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> <mesos-dev-subscribe-john=host.domain@incubator.apache.org
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> <mesos-dev-unsubscribe-john=host.domain@incubator.apache.o
>>>>>>>>>>>>>>>>>> rg
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>>>>>>> address.
>>>>>>>>>>>>>>>>>> When
>>>>>>>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>>>>>>> subscription.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> If despite following these instructions, you do not get
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient,
>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>> owner
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>>>>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>>>>>>> 23:30:07
>>>>>>>>>>>>>>>>>> -0000
>>>>>>>>>>>>>>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>>>>>>>>>>>>>> (192.87.106.230)
>>>>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>>>>>> 23:30:07
>>>>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>>>>>>>>>>>>>>> tests=SPF_PASS
>>>>>>>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>>>>>>>> Received-SPF: pass (nike.apache.org: domain of
>>>>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>>>> designates
>>>>>>>>>>>>>>>>>> 67.231.145.42 as permitted sender)
>>>>>>>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>>>>>>> mx0a-00082601.pphosted.com)
>>>>>>>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>>>>>> 23:30:00
>>>>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>>>>>>>>>>>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>>>>>>>> p7LNTI6g015638
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.c
>>>>>>>>>>>>>>>>>> om
>>>>>>>>>>>>>>>>>> @
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> t
>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>>>>>>>> Received: from mail.thefacebook.com
>>>>>>>>>>>>>>>>>> (corpout1.snc1.tfbnw.net
>>>>>>>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>>>>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>>>>>>>>>>>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128
>>>>>>>>>>>>>>>>>> verify=NOT)
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.c
>>>>>>>>>>>>>>>>>> om
>>>>>>>>>>>>>>>>>> @
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> t
>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>>>>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>>>>>>>> Sun,
>>>>>>>>>>>>>>>>>> 21 Aug 2011 16:29:14 -0700
>>>>>>>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>>>>>>>> To:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> "mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.c
>>>>>>>>>>>>>>>>>> om
>>>>>>>>>>>>>>>>>> @
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> t
>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> che.org"
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.
>>>>>>>>>>>>>>>>>> co
>>>>>>>>>>>>>>>>>> m
>>>>>>>>>>>>>>>>>> @i
>>>>>>>>>>>>>>>>>> n
>>>>>>>>>>>>>>>>>> cu
>>>>>>>>>>>>>>>>>> b
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> r.
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>>>> ache.org>
>>>>>>>>>>>>>>>>>> Subject: Re: confirm subscribe to
>>>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org
>>>>>>>>>>>>>>>>>> Thread-Topic: confirm subscribe to
>>>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org
>>>>>>>>>>>>>>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>>>>>>>>>>>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>>>>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>>>>>>>> In-Reply-To:
>>>>>>>>>>>>>>>>>> <13...@incubator.apache.org>
>>>>>>>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>>>>>>>> X-MS-Has-Attach:
>>>>>>>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>>>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>>>>>>>>>>>>>> Content-ID: <B4...@fb.com>
>>>>>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> To confirm that you would like
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> added to the mesos-dev mailing list, please send
>>>>>>>>>>>>>>>>>>> a short reply to this address:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> =20
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb
>>>>>>>>>>>>>>>>>>> .c
>>>>>>>>>>>>>>>>>>> o
>>>>>>>>>>>>>>>>>>> m@
>>>>>>>>>>>>>>>>>>> i
>>>>>>>>>>>>>>>>>>> nc
>>>>>>>>>>>>>>>>>>> u
>>>>>>>>>>>>>>>>>>> b
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> p=
>>>>>>>>>>>>>>>>>> ac
>>>>>>>>>>>>>>>>>>> he.org
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Usually, this happens when you just hit the "reply"
>>>>>>>>>>>>>>>>>>> button.
>>>>>>>>>>>>>>>>>>> If this does not work, simply copy the address and paste
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>> the "To:" field of a new message.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> or click here:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiy
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> g
>>>>>>>>>>>>>>>>>>> =3
>>>>>>>>>>>>>>>>>>> D
>>>>>>>>>>>>>>>>>>> fb
>>>>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>>>>> om
>>>>>>>>>>>>>>>>>>> @i
>>>>>>>>>>>>>>>>>>> n
>>>>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>>>>> u=
>>>>>>>>>>>>>>>>>> ba
>>>>>>>>>>>>>>>>>>> tor.apache.org
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This confirmation serves two purposes. First, it
>>>>>>>>>>>>>>>>>>> verifies
>>>>>>>>>>>>>>>>>>> that I
>>>>>>>>>>>>>>>>>>> am
>>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>>> to get mail through to you. Second, it protects you in
>>>>>>>>>>>>>>>>>>> case
>>>>>>>>>>>>>>>>>>> someone
>>>>>>>>>>>>>>>>>>> forges a subscription request in your name.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Please note that ALL Apache dev- and user- mailing lists
>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>> publicly
>>>>>>>>>>>>>>>>>>> archived. Do familiarize yourself with Apache's public
>>>>>>>>>>>>>>>>>>> archive
>>>>>>>>>>>>>>>>>>> policy
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> prior to subscribing and posting messages to
>>>>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org.
>>>>>>>>>>>>>>>>>>> If you're not sure whether or not the policy applies to
>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>> mailing
>>>>>>>>>>>>>>>>>>> list,
>>>>>>>>>>>>>>>>>>> assume it does unless the list name contains the word
>>>>>>>>>>>>>>>>>>> "private"
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> it.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Some mail programs are broken and cannot handle long
>>>>>>>>>>>>>>>>>>> addresses.
>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>> cannot reply to this request, instead send a message to
>>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org> and put the
>>>>>>>>>>>>>>>>>>> entire address listed above into the "Subject:" line.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I can handle administrative requests automatically.
>>>>>>>>>>>>>>>>>>> Please
>>>>>>>>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Send mail to the following for info and FAQ for this
>>>>>>>>>>>>>>>>>>> list:
>>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>>>>>>>> request),
>>>>>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> To get an index with subject and author for messages
>>>>>>>>>>>>>>>>>>> 123-456
>>>>>>>>>>>>>>>>>>> ,
>>>>>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per
>>>>>>>>>>>>>>>>>>> request,
>>>>>>>>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>>>>>>>> 12345,
>>>>>>>>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The messages should contain one line or word of text to
>>>>>>>>>>>>>>>>>>> avoid
>>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and
>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>> address (with '=3D' instead of '@') after the command
>>>>>>>>>>>>>>>>>>> word:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> <mesos-dev-subscribe-john=3Dhost.domain@incubator.apache.
>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>> g
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> <mesos-dev-unsubscribe-john=3Dhost.domain@incubator.apach
>>>>>>>>>>>>>>>>>>> e.
>>>>>>>>>>>>>>>>>>> o
>>>>>>>>>>>>>>>>>>> rg
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>>>>>>>> address.
>>>>>>>>>>>>>>>>>>> When
>>>>>>>>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>>>>>>>> subscription.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If despite following these instructions, you do not get
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient,
>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>> owner
>>>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>>>>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>>>>>>>> -0000
>>>>>>>>>>>>>>>>>>> Received: from athena.apache.org (HELO
>>>>>>>>>>>>>>>>>>> athena.apache.org)
>>>>>>>>>>>>>>>>>>> (140.211.11.136)
>>>>>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug
>>>>>>>>>>>>>>>>>>> 2011
>>>>>>>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>>>>>>>>>>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>>>>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>>>>>>>>> Received-SPF: pass (athena.apache.org: domain of
>>>>>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>>>>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>>>>>>>> mx0a-00082601.pphosted.com)
>>>>>>>>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug
>>>>>>>>>>>>>>>>>>> 2011
>>>>>>>>>>>>>>>>>>> 22:43:19
>>>>>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>>>>>>>>>>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>>>>>>>>> p7LMcbSq017336
>>>>>>>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21
>>>>>>>>>>>>>>>>>>> Aug
>>>>>>>>>>>>>>>>>>> 2011
>>>>>>>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>>>>>>>> Received: from mail.thefacebook.com
>>>>>>>>>>>>>>>>>>> (corpout1.snc1.tfbnw.net
>>>>>>>>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>>>>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>>>>>>>>>>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128
>>>>>>>>>>>>>>>>>>> verify=3DNOT)
>>>>>>>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21
>>>>>>>>>>>>>>>>>>> Aug
>>>>>>>>>>>>>>>>>>> 2011
>>>>>>>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>>>>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>>>>>>>>> Sun,
>>>>>>>>>>>>>>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>>>>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>>>>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>> Subject: can i join this
>>>>>>>>>>>>>>>>>>> Thread-Topic: can i join this
>>>>>>>>>>>>>>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>>>>>>>>>>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>>>>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>>>>>>>>> X-MS-Has-Attach:=20
>>>>>>>>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>>>>>>>>> Content-Type: multipart/alternative;
>>>>>>>>>>>>>>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>>>>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>>>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>>>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>>>>>>>>>>>>>>> Content-ID: <F6...@fb.com>
>>>>>>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> <html>
>>>>>>>>>>>>>>>>>>> <head>
>>>>>>>>>>>>>>>>>>> <meta http-equiv=3D3D"Content-Type"
>>>>>>>>>>>>>>>>>>> content=3D3D"text/html;
>>>>>>>>>>>>>>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> </head>
>>>>>>>>>>>>>>>>>>> <body style=3D3D"word-wrap: break-word;
>>>>>>>>>>>>>>>>>>> -webkit-nbsp-mode:
>>>>>>>>>>>>>>>>>>> space;
>>>>>>>>>>>>>>>>>>> -webkit-lin=3D
>>>>>>>>>>>>>>>>>>> e-break: after-white-space; color: rgb(0, 0, 0);
>>>>>>>>>>>>>>>>>>> font-size:
>>>>>>>>>>>>>>>>>>> 14px;
>>>>>>>>>>>>>>>>>>> font-fami=3D
>>>>>>>>>>>>>>>>>>> ly: Calibri, sans-serif; ">
>>>>>>>>>>>>>>>>>>> <div>Best,</div>
>>>>>>>>>>>>>>>>>>> <div>Joe</div>
>>>>>>>>>>>>>>>>>>> </body>
>>>>>>>>>>>>>>>>>>> </html>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: About spark-shell in mesos cluster mode Re: WELCOME to
mesos-dev@incubator.apache.org
Posted by Joe Xie <zh...@fb.com>.
Thanks! I thought all the logs are all at master and did not find there,
find them on slave and fix the problem now. My problem is that the data
was not available to slave machines when I use sc.textFile(data) in
spark-shell cluster mode.
Best,
Joe
On 8/24/11 8:27 AM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>Again, you probably need to look at the stderr and stdout logs on the
>slaves. "task lost" just means that the task crashed or could not be
>launched.
>
>Matei
>
>On Aug 24, 2011, at 7:53 AM, Joe Xie wrote:
>
>> Hi Matei,
>>
>> when I start with "MASTER=1@mymesoshost:5050 ./spark-shell" it seems I
>> keep losing TID after starting the task, and failed after 4 tries. I'm
>> trying to looking into code now, but let me know if have any suggestion
>> too.
>>
>> 11/08/24 07:52:34 ERROR spark.SimpleJob: Task 0:6 failed more than 4
>> times; aborting job
>> 11/08/24 07:52:34 INFO spark.MesosScheduler: Ignoring update from TID 41
>> because its job is gone
>>
>>
>> Thanks again,
>> Joe
>>
>> On 8/23/11 6:52 PM, "Joe Xie" <zh...@fb.com> wrote:
>>
>>> Thanks! I was use another email.
>>>
>>> From: Mail Delivery Subsystem <ma...@googlemail.com>
>>> Date: Tue, Aug 23, 2011 at 3:29 PM
>>> Subject: Delivery Status Notification (Failure)
>>> To: zyxienju@gmail.com
>>> Delivery to the following recipient failed permanently:
>>> spark-users@eecs.berkeley.edu
>>> Technical details of permanent failure:
>>> Google tried to deliver your message, but it was rejected by the
>>>recipient
>>> domain. We recommend contacting the other email provider for further
>>> information about the cause of this error. The error that the other
>>>server
>>> returned was: 550 550 unknown user (state 14).
>>>
>>>
>>>
>>> On 8/23/11 4:50 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>
>>>> Weird, did you subscribe to the spark-users mailing list? I don't
>>>>think
>>>> you can post to it without doing that. Go to
>>>> https://lists.eecs.berkeley.edu/sympa/subscribe/spark-users.
>>>>
>>>> Anyway, to run the shell on a cluster, you need to set the MASTER
>>>> environment variable to the Mesos master URL you want to use (similar
>>>>to
>>>> the one you pass to the standalone programs). So for example, type
>>>>
>>>> MASTER=1@localhost:5050 ./spark-shell
>>>>
>>>> Matei
>>>>
>>>>
>>>> On Aug 23, 2011, at 4:39 PM, Joe Xie wrote:
>>>>
>>>>> Hi there,
>>>>>
>>>>> may I ask how can I run spark-shell in cluster mode? The web wiki say
>>>>> change the configuration of master, but I did not find detailed info
>>>>> yet.
>>>>> It seems my email failed at spark-user maillist.
>>>>> Thanks!
>>>>> Joe
>>>>>
>>>>>
>>>>>
>>>>> On 8/22/11 4:42 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>
>>>>>> Thank you Matei! It's working now. I checked the stderr, it seems I
>>>>>> have
>>>>>> to specify the SCALA_HOME even if it's in .bashrc.
>>>>>> Best,
>>>>>> Joe
>>>>>>
>>>>>> On 8/22/11 4:13 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>>>>
>>>>>>> This probably means that your task crashed. Take a look at the logs
>>>>>>> Mesos
>>>>>>> collects for your task, which will be in
>>>>>>> MESOS_HOME/work/slave-id/framework-id/attempt-id/stdout and stderr.
>>>>>>>
>>>>>>> Sometimes this happens because spark-env.sh isn't configured on all
>>>>>>> the
>>>>>>> nodes to contain SCALA_HOME, JAVA_HOME and MESOS_HOME. It can also
>>>>>>> happen
>>>>>>> because your code is not available on all the machines and you get
>>>>>>>a
>>>>>>> ClassNotFoundError. You can either copy the code to the same place
>>>>>>> everywhere and put that in the SPARK_CLASSPATH variable in
>>>>>>> spark-env.sh,
>>>>>>> or package your code into a JAR and use the "jars" parameter to
>>>>>>> SparkContext to pass it (in which case Spark will copy it to all
>>>>>>>the
>>>>>>> nodes).
>>>>>>>
>>>>>>> Matei
>>>>>>>
>>>>>>> On Aug 22, 2011, at 4:04 PM, Joe Xie wrote:
>>>>>>>
>>>>>>>> Btw, I checkout my webui for master, it indicates "No offers are
>>>>>>>> active"
>>>>>>>> under "Resource Offers" while I have two machines as slave, so
>>>>>>>>there
>>>>>>>> might
>>>>>>>> be problem with my configuration.
>>>>>>>> Best,
>>>>>>>> JOE
>>>>>>>>
>>>>>>>> On 8/22/11 3:59 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>>
>>>>>>>>> Thank you Matei,
>>>>>>>>>
>>>>>>>>> it's running for me with Spark 0.3 + pre-protobuf, and I'm
>>>>>>>>>testing
>>>>>>>>> with
>>>>>>>>> provided example "run spark.examples.SparkPi
>>>>>>>>> mesos://1@10.30.95.184:5050
>>>>>>>>> 3". However, it seems I keep losing TID after starting the task,
>>>>>>>>> and
>>>>>>>>> then
>>>>>>>>> job aborted with 4 tries. It might be the problem on my side too,
>>>>>>>>> did
>>>>>>>>> this
>>>>>>>>> happen before? I could go for trying masters for spark + mesos.
>>>>>>>>> 11/08/22 15:48:27 INFO spark.SimpleJob: Lost TID 13 (task 0:2)
>>>>>>>>> 11/08/22 15:48:27 ERROR spark.SimpleJob: Task 0:2 failed more
>>>>>>>>>than
>>>>>>>>> 4
>>>>>>>>> times; aborting job
>>>>>>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from
>>>>>>>>> TID
>>>>>>>>> 12
>>>>>>>>> because its job is gone
>>>>>>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from
>>>>>>>>> TID
>>>>>>>>> 14
>>>>>>>>> because its job is gone
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>> Joe
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 8/22/11 3:38 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>>>
>>>>>>>>>> Thank you Matei! I can go for Spark 0.3 + pre-protobuf. I
>>>>>>>>>> understand
>>>>>>>>>> there
>>>>>>>>>> might be lots of new updates on the master branch. Is there any
>>>>>>>>>> significant difference in current master with Spark 0.3 +
>>>>>>>>>> pre-protobuf? I
>>>>>>>>>> could also try to keep a separate branch with stable masters if
>>>>>>>>>> that's
>>>>>>>>>> the
>>>>>>>>>> case.
>>>>>>>>>> Best,
>>>>>>>>>> Joe
>>>>>>>>>> On 8/22/11 3:21 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> If you want the most stable versions right now, I recommend
>>>>>>>>>>>Spark
>>>>>>>>>>> 0.3
>>>>>>>>>>> (download it from the website or the 0.3-scala-2.9 tag in Git)
>>>>>>>>>>> and
>>>>>>>>>>> pre-protobuf Mesos. However, the current master Spark + master
>>>>>>>>>>> Mesos
>>>>>>>>>>> should also work OK; it's just that further commits may break
>>>>>>>>>>> things
>>>>>>>>>>> temporarily.
>>>>>>>>>>>
>>>>>>>>>>> Matei
>>>>>>>>>>>
>>>>>>>>>>> On Aug 22, 2011, at 3:17 PM, Joe Xie wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Matei,
>>>>>>>>>>>> Thanks for your feedback. I bet I'm using the master brach of
>>>>>>>>>>>> Spark
>>>>>>>>>>>> and
>>>>>>>>>>>> pre-protobuf version of Mesos. Do you mean I should go for
>>>>>>>>>>>>Spark
>>>>>>>>>>>> (master)
>>>>>>>>>>>> + protobuf (master). Let's me know the most stable
>>>>>>>>>>>>combination
>>>>>>>>>>>> of
>>>>>>>>>>>> braches
>>>>>>>>>>>> from Spark and pre-protobuf, I can try them out.
>>>>>>>>>>>>
>>>>>>>>>>>> Best,
>>>>>>>>>>>> Joe
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Are you using the master branch of Spark, or one of the
>>>>>>>>>>>>> releases
>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>> 0.3)? The releases work with the pre-protobuf version of
>>>>>>>>>>>>>Mesos
>>>>>>>>>>>>> but
>>>>>>>>>>>>> the
>>>>>>>>>>>>> master branch of Spark actually requires the master branch of
>>>>>>>>>>>>> Mesos
>>>>>>>>>>>>> now.
>>>>>>>>>>>>> That could cause the kind of error you're seeing.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Matei
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sorry this should have been sent to user mail list.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi there:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I has build and installed mesos (pre-protobuf branch for
>>>>>>>>>>>>>>> spark)
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> spark,
>>>>>>>>>>>>>>> and deployed them to a cluster. However, I have following
>>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>> run
>>>>>>>>>>>>>>> spark task on the mesos URL of my master, the spark task
>>>>>>>>>>>>>>>runs
>>>>>>>>>>>>>>> fine
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> local mode. Any suggestion or help? Thanks a lot!
>>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> My os: CentOS 5.2
>>>>>>>>>>>>>>> Python: 2.6
>>>>>>>>>>>>>>> Gcc: 4.2.1
>>>>>>>>>>>>>>> Swig:1.3.40
>>>>>>>>>>>>>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered
>>>>>>>>>>>>>>> actor
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> port
>>>>>>>>>>>>>>> 50501
>>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor:
>>>>>>>>>>>>>>> Registered
>>>>>>>>>>>>>>> actor
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> port 50501
>>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD
>>>>>>>>>>>>>>>ID
>>>>>>>>>>>>>>> 0
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> cache
>>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering
>>>>>>>>>>>>>>> RDD
>>>>>>>>>>>>>>> 0
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> 1
>>>>>>>>>>>>>>> partitions
>>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for
>>>>>>>>>>>>>>> current
>>>>>>>>>>>>>>> cache
>>>>>>>>>>>>>>> locations
>>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage:
>>>>>>>>>>>>>>> Stage
>>>>>>>>>>>>>>> 0
>>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of
>>>>>>>>>>>>>>>final
>>>>>>>>>>>>>>> stage:
>>>>>>>>>>>>>>> List()
>>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing
>>>>>>>>>>>>>>>parents:
>>>>>>>>>>>>>>> List()
>>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting
>>>>>>>>>>>>>>>Stage
>>>>>>>>>>>>>>> 0,
>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>> no missing parents
>>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job
>>>>>>>>>>>>>>>with 1
>>>>>>>>>>>>>>> tasks
>>>>>>>>>>>>>>> Exception in thread "Spark scheduler"
>>>>>>>>>>>>>>> java.lang.UnsatisfiedLinkError:
>>>>>>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>>>>>>>>>>>>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native
>>>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerD
>>>>>>>>>>>>>>>ri
>>>>>>>>>>>>>>> v
>>>>>>>>>>>>>>> er
>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>> ja
>>>>>>>>>>>>>>> v
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> :4
>>>>>>>>>>>>>>> 2)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerD
>>>>>>>>>>>>>>>ri
>>>>>>>>>>>>>>> v
>>>>>>>>>>>>>>> er
>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>> ja
>>>>>>>>>>>>>>> v
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> :4
>>>>>>>>>>>>>>> 6)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Sent to wrong address previously
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Acknowledgment: I have added the address
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> to the mesos-dev mailing list.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Please save this message so that you know the address you
>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>> subscribed under, in case you later want to unsubscribe
>>>>>>>>>>>>>>>>>or
>>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>> subscription address.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I can handle administrative requests automatically.
>>>>>>>>>>>>>>>>>Please
>>>>>>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Send mail to the following for info and FAQ for this
>>>>>>>>>>>>>>>>>list:
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>>>>>> request),
>>>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To get an index with subject and author for messages
>>>>>>>>>>>>>>>>> 123-456 ,
>>>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per
>>>>>>>>>>>>>>>>> request,
>>>>>>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>>>>>> 12345,
>>>>>>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The messages should contain one line or word of text to
>>>>>>>>>>>>>>>>> avoid
>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and
>>>>>>>>>>>>>>>>>your
>>>>>>>>>>>>>>>>> address (with '=' instead of '@') after the command word:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>><mesos-dev-subscribe-john=host.domain@incubator.apache.org
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>><mesos-dev-unsubscribe-john=host.domain@incubator.apache.o
>>>>>>>>>>>>>>>>>rg
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>>>>>> address.
>>>>>>>>>>>>>>>>> When
>>>>>>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>>>>>> subscription.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If despite following these instructions, you do not get
>>>>>>>>>>>>>>>>>the
>>>>>>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient,
>>>>>>>>>>>>>>>>>my
>>>>>>>>>>>>>>>>> owner
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>>>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>>>>>> 23:30:07
>>>>>>>>>>>>>>>>> -0000
>>>>>>>>>>>>>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>>>>>>>>>>>>> (192.87.106.230)
>>>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>>>>> 23:30:07
>>>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>>>>>>>>>>>>>> tests=SPF_PASS
>>>>>>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>>>>>>> Received-SPF: pass (nike.apache.org: domain of
>>>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>>> designates
>>>>>>>>>>>>>>>>> 67.231.145.42 as permitted sender)
>>>>>>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>>>>>> mx0a-00082601.pphosted.com)
>>>>>>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>>>>> 23:30:00
>>>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>>>>>>>>>>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>>>>>>> p7LNTI6g015638
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.c
>>>>>>>>>>>>>>>>>om
>>>>>>>>>>>>>>>>> @
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> t
>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>>>>>>> Received: from mail.thefacebook.com
>>>>>>>>>>>>>>>>> (corpout1.snc1.tfbnw.net
>>>>>>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>>>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>>>>>>>>>>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128
>>>>>>>>>>>>>>>>> verify=NOT)
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.c
>>>>>>>>>>>>>>>>>om
>>>>>>>>>>>>>>>>> @
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> t
>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>>>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>>>>>>> Sun,
>>>>>>>>>>>>>>>>> 21 Aug 2011 16:29:14 -0700
>>>>>>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>>>>>>> To:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>"mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.c
>>>>>>>>>>>>>>>>>om
>>>>>>>>>>>>>>>>> @
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> t
>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> che.org"
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.
>>>>>>>>>>>>>>>>>co
>>>>>>>>>>>>>>>>> m
>>>>>>>>>>>>>>>>> @i
>>>>>>>>>>>>>>>>> n
>>>>>>>>>>>>>>>>> cu
>>>>>>>>>>>>>>>>> b
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> r.
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>>> ache.org>
>>>>>>>>>>>>>>>>> Subject: Re: confirm subscribe to
>>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org
>>>>>>>>>>>>>>>>> Thread-Topic: confirm subscribe to
>>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org
>>>>>>>>>>>>>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>>>>>>>>>>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>>>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>>>>>>> In-Reply-To:
>>>>>>>>>>>>>>>>><13...@incubator.apache.org>
>>>>>>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>>>>>>> X-MS-Has-Attach:
>>>>>>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>>>>>>>>>>>>> Content-ID: <B4...@fb.com>
>>>>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To confirm that you would like
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> added to the mesos-dev mailing list, please send
>>>>>>>>>>>>>>>>>> a short reply to this address:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> =20
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb
>>>>>>>>>>>>>>>>>>.c
>>>>>>>>>>>>>>>>>> o
>>>>>>>>>>>>>>>>>> m@
>>>>>>>>>>>>>>>>>> i
>>>>>>>>>>>>>>>>>> nc
>>>>>>>>>>>>>>>>>> u
>>>>>>>>>>>>>>>>>> b
>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> p=
>>>>>>>>>>>>>>>>> ac
>>>>>>>>>>>>>>>>>> he.org
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Usually, this happens when you just hit the "reply"
>>>>>>>>>>>>>>>>>> button.
>>>>>>>>>>>>>>>>>> If this does not work, simply copy the address and paste
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>> the "To:" field of a new message.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> or click here:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiy
>>>>>>>>>>>>>>>>>>on
>>>>>>>>>>>>>>>>>> g
>>>>>>>>>>>>>>>>>> =3
>>>>>>>>>>>>>>>>>> D
>>>>>>>>>>>>>>>>>> fb
>>>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>>>> om
>>>>>>>>>>>>>>>>>> @i
>>>>>>>>>>>>>>>>>> n
>>>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>>>> u=
>>>>>>>>>>>>>>>>> ba
>>>>>>>>>>>>>>>>>> tor.apache.org
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> This confirmation serves two purposes. First, it
>>>>>>>>>>>>>>>>>>verifies
>>>>>>>>>>>>>>>>>> that I
>>>>>>>>>>>>>>>>>> am
>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>> to get mail through to you. Second, it protects you in
>>>>>>>>>>>>>>>>>> case
>>>>>>>>>>>>>>>>>> someone
>>>>>>>>>>>>>>>>>> forges a subscription request in your name.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Please note that ALL Apache dev- and user- mailing lists
>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>> publicly
>>>>>>>>>>>>>>>>>> archived. Do familiarize yourself with Apache's public
>>>>>>>>>>>>>>>>>> archive
>>>>>>>>>>>>>>>>>> policy
>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> prior to subscribing and posting messages to
>>>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org.
>>>>>>>>>>>>>>>>>> If you're not sure whether or not the policy applies to
>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>> mailing
>>>>>>>>>>>>>>>>>> list,
>>>>>>>>>>>>>>>>>> assume it does unless the list name contains the word
>>>>>>>>>>>>>>>>>> "private"
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> it.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Some mail programs are broken and cannot handle long
>>>>>>>>>>>>>>>>>> addresses.
>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>> cannot reply to this request, instead send a message to
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org> and put the
>>>>>>>>>>>>>>>>>> entire address listed above into the "Subject:" line.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I can handle administrative requests automatically.
>>>>>>>>>>>>>>>>>>Please
>>>>>>>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Send mail to the following for info and FAQ for this
>>>>>>>>>>>>>>>>>>list:
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>>>>>>> request),
>>>>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To get an index with subject and author for messages
>>>>>>>>>>>>>>>>>> 123-456
>>>>>>>>>>>>>>>>>> ,
>>>>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per
>>>>>>>>>>>>>>>>>> request,
>>>>>>>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>>>>>>> 12345,
>>>>>>>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The messages should contain one line or word of text to
>>>>>>>>>>>>>>>>>> avoid
>>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and
>>>>>>>>>>>>>>>>>>your
>>>>>>>>>>>>>>>>>> address (with '=3D' instead of '@') after the command
>>>>>>>>>>>>>>>>>> word:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>><mesos-dev-subscribe-john=3Dhost.domain@incubator.apache.
>>>>>>>>>>>>>>>>>>or
>>>>>>>>>>>>>>>>>> g
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>><mesos-dev-unsubscribe-john=3Dhost.domain@incubator.apach
>>>>>>>>>>>>>>>>>>e.
>>>>>>>>>>>>>>>>>> o
>>>>>>>>>>>>>>>>>> rg
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>>>>>>> address.
>>>>>>>>>>>>>>>>>> When
>>>>>>>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>>>>>>> subscription.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> If despite following these instructions, you do not get
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient,
>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>> owner
>>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>>>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>>>>>>> -0000
>>>>>>>>>>>>>>>>>> Received: from athena.apache.org (HELO
>>>>>>>>>>>>>>>>>>athena.apache.org)
>>>>>>>>>>>>>>>>>> (140.211.11.136)
>>>>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug
>>>>>>>>>>>>>>>>>>2011
>>>>>>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>>>>>>>>>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>>>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>>>>>>>> Received-SPF: pass (athena.apache.org: domain of
>>>>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>>>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>>>>>>> mx0a-00082601.pphosted.com)
>>>>>>>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug
>>>>>>>>>>>>>>>>>>2011
>>>>>>>>>>>>>>>>>> 22:43:19
>>>>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>>>>>>>>>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>>>>>>>> p7LMcbSq017336
>>>>>>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21
>>>>>>>>>>>>>>>>>> Aug
>>>>>>>>>>>>>>>>>> 2011
>>>>>>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>>>>>>> Received: from mail.thefacebook.com
>>>>>>>>>>>>>>>>>> (corpout1.snc1.tfbnw.net
>>>>>>>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>>>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>>>>>>>>>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128
>>>>>>>>>>>>>>>>>> verify=3DNOT)
>>>>>>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21
>>>>>>>>>>>>>>>>>> Aug
>>>>>>>>>>>>>>>>>> 2011
>>>>>>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>>>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>>>>>>>> Sun,
>>>>>>>>>>>>>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>>>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>>>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>> Subject: can i join this
>>>>>>>>>>>>>>>>>> Thread-Topic: can i join this
>>>>>>>>>>>>>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>>>>>>>>>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>>>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>>>>>>>> X-MS-Has-Attach:=20
>>>>>>>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>>>>>>>> Content-Type: multipart/alternative;
>>>>>>>>>>>>>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>>>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>>>>>>>>>>>>>> Content-ID: <F6...@fb.com>
>>>>>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> <html>
>>>>>>>>>>>>>>>>>> <head>
>>>>>>>>>>>>>>>>>> <meta http-equiv=3D3D"Content-Type"
>>>>>>>>>>>>>>>>>> content=3D3D"text/html;
>>>>>>>>>>>>>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> </head>
>>>>>>>>>>>>>>>>>> <body style=3D3D"word-wrap: break-word;
>>>>>>>>>>>>>>>>>>-webkit-nbsp-mode:
>>>>>>>>>>>>>>>>>> space;
>>>>>>>>>>>>>>>>>> -webkit-lin=3D
>>>>>>>>>>>>>>>>>> e-break: after-white-space; color: rgb(0, 0, 0);
>>>>>>>>>>>>>>>>>> font-size:
>>>>>>>>>>>>>>>>>> 14px;
>>>>>>>>>>>>>>>>>> font-fami=3D
>>>>>>>>>>>>>>>>>> ly: Calibri, sans-serif; ">
>>>>>>>>>>>>>>>>>> <div>Best,</div>
>>>>>>>>>>>>>>>>>> <div>Joe</div>
>>>>>>>>>>>>>>>>>> </body>
>>>>>>>>>>>>>>>>>> </html>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: About spark-shell in mesos cluster mode Re: WELCOME to mesos-dev@incubator.apache.org
Posted by Matei Zaharia <ma...@eecs.berkeley.edu>.
Again, you probably need to look at the stderr and stdout logs on the slaves. "task lost" just means that the task crashed or could not be launched.
Matei
On Aug 24, 2011, at 7:53 AM, Joe Xie wrote:
> Hi Matei,
>
> when I start with "MASTER=1@mymesoshost:5050 ./spark-shell" it seems I
> keep losing TID after starting the task, and failed after 4 tries. I'm
> trying to looking into code now, but let me know if have any suggestion
> too.
>
> 11/08/24 07:52:34 ERROR spark.SimpleJob: Task 0:6 failed more than 4
> times; aborting job
> 11/08/24 07:52:34 INFO spark.MesosScheduler: Ignoring update from TID 41
> because its job is gone
>
>
> Thanks again,
> Joe
>
> On 8/23/11 6:52 PM, "Joe Xie" <zh...@fb.com> wrote:
>
>> Thanks! I was use another email.
>>
>> From: Mail Delivery Subsystem <ma...@googlemail.com>
>> Date: Tue, Aug 23, 2011 at 3:29 PM
>> Subject: Delivery Status Notification (Failure)
>> To: zyxienju@gmail.com
>> Delivery to the following recipient failed permanently:
>> spark-users@eecs.berkeley.edu
>> Technical details of permanent failure:
>> Google tried to deliver your message, but it was rejected by the recipient
>> domain. We recommend contacting the other email provider for further
>> information about the cause of this error. The error that the other server
>> returned was: 550 550 unknown user (state 14).
>>
>>
>>
>> On 8/23/11 4:50 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>
>>> Weird, did you subscribe to the spark-users mailing list? I don't think
>>> you can post to it without doing that. Go to
>>> https://lists.eecs.berkeley.edu/sympa/subscribe/spark-users.
>>>
>>> Anyway, to run the shell on a cluster, you need to set the MASTER
>>> environment variable to the Mesos master URL you want to use (similar to
>>> the one you pass to the standalone programs). So for example, type
>>>
>>> MASTER=1@localhost:5050 ./spark-shell
>>>
>>> Matei
>>>
>>>
>>> On Aug 23, 2011, at 4:39 PM, Joe Xie wrote:
>>>
>>>> Hi there,
>>>>
>>>> may I ask how can I run spark-shell in cluster mode? The web wiki say
>>>> change the configuration of master, but I did not find detailed info
>>>> yet.
>>>> It seems my email failed at spark-user maillist.
>>>> Thanks!
>>>> Joe
>>>>
>>>>
>>>>
>>>> On 8/22/11 4:42 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>
>>>>> Thank you Matei! It's working now. I checked the stderr, it seems I
>>>>> have
>>>>> to specify the SCALA_HOME even if it's in .bashrc.
>>>>> Best,
>>>>> Joe
>>>>>
>>>>> On 8/22/11 4:13 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>>>
>>>>>> This probably means that your task crashed. Take a look at the logs
>>>>>> Mesos
>>>>>> collects for your task, which will be in
>>>>>> MESOS_HOME/work/slave-id/framework-id/attempt-id/stdout and stderr.
>>>>>>
>>>>>> Sometimes this happens because spark-env.sh isn't configured on all
>>>>>> the
>>>>>> nodes to contain SCALA_HOME, JAVA_HOME and MESOS_HOME. It can also
>>>>>> happen
>>>>>> because your code is not available on all the machines and you get a
>>>>>> ClassNotFoundError. You can either copy the code to the same place
>>>>>> everywhere and put that in the SPARK_CLASSPATH variable in
>>>>>> spark-env.sh,
>>>>>> or package your code into a JAR and use the "jars" parameter to
>>>>>> SparkContext to pass it (in which case Spark will copy it to all the
>>>>>> nodes).
>>>>>>
>>>>>> Matei
>>>>>>
>>>>>> On Aug 22, 2011, at 4:04 PM, Joe Xie wrote:
>>>>>>
>>>>>>> Btw, I checkout my webui for master, it indicates "No offers are
>>>>>>> active"
>>>>>>> under "Resource Offers" while I have two machines as slave, so there
>>>>>>> might
>>>>>>> be problem with my configuration.
>>>>>>> Best,
>>>>>>> JOE
>>>>>>>
>>>>>>> On 8/22/11 3:59 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>
>>>>>>>> Thank you Matei,
>>>>>>>>
>>>>>>>> it's running for me with Spark 0.3 + pre-protobuf, and I'm testing
>>>>>>>> with
>>>>>>>> provided example "run spark.examples.SparkPi
>>>>>>>> mesos://1@10.30.95.184:5050
>>>>>>>> 3". However, it seems I keep losing TID after starting the task,
>>>>>>>> and
>>>>>>>> then
>>>>>>>> job aborted with 4 tries. It might be the problem on my side too,
>>>>>>>> did
>>>>>>>> this
>>>>>>>> happen before? I could go for trying masters for spark + mesos.
>>>>>>>> 11/08/22 15:48:27 INFO spark.SimpleJob: Lost TID 13 (task 0:2)
>>>>>>>> 11/08/22 15:48:27 ERROR spark.SimpleJob: Task 0:2 failed more than
>>>>>>>> 4
>>>>>>>> times; aborting job
>>>>>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from
>>>>>>>> TID
>>>>>>>> 12
>>>>>>>> because its job is gone
>>>>>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from
>>>>>>>> TID
>>>>>>>> 14
>>>>>>>> because its job is gone
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>> Joe
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 8/22/11 3:38 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>>
>>>>>>>>> Thank you Matei! I can go for Spark 0.3 + pre-protobuf. I
>>>>>>>>> understand
>>>>>>>>> there
>>>>>>>>> might be lots of new updates on the master branch. Is there any
>>>>>>>>> significant difference in current master with Spark 0.3 +
>>>>>>>>> pre-protobuf? I
>>>>>>>>> could also try to keep a separate branch with stable masters if
>>>>>>>>> that's
>>>>>>>>> the
>>>>>>>>> case.
>>>>>>>>> Best,
>>>>>>>>> Joe
>>>>>>>>> On 8/22/11 3:21 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> If you want the most stable versions right now, I recommend Spark
>>>>>>>>>> 0.3
>>>>>>>>>> (download it from the website or the 0.3-scala-2.9 tag in Git)
>>>>>>>>>> and
>>>>>>>>>> pre-protobuf Mesos. However, the current master Spark + master
>>>>>>>>>> Mesos
>>>>>>>>>> should also work OK; it's just that further commits may break
>>>>>>>>>> things
>>>>>>>>>> temporarily.
>>>>>>>>>>
>>>>>>>>>> Matei
>>>>>>>>>>
>>>>>>>>>> On Aug 22, 2011, at 3:17 PM, Joe Xie wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Matei,
>>>>>>>>>>> Thanks for your feedback. I bet I'm using the master brach of
>>>>>>>>>>> Spark
>>>>>>>>>>> and
>>>>>>>>>>> pre-protobuf version of Mesos. Do you mean I should go for Spark
>>>>>>>>>>> (master)
>>>>>>>>>>> + protobuf (master). Let's me know the most stable combination
>>>>>>>>>>> of
>>>>>>>>>>> braches
>>>>>>>>>>> from Spark and pre-protobuf, I can try them out.
>>>>>>>>>>>
>>>>>>>>>>> Best,
>>>>>>>>>>> Joe
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Are you using the master branch of Spark, or one of the
>>>>>>>>>>>> releases
>>>>>>>>>>>> (e.g.
>>>>>>>>>>>> 0.3)? The releases work with the pre-protobuf version of Mesos
>>>>>>>>>>>> but
>>>>>>>>>>>> the
>>>>>>>>>>>> master branch of Spark actually requires the master branch of
>>>>>>>>>>>> Mesos
>>>>>>>>>>>> now.
>>>>>>>>>>>> That could cause the kind of error you're seeing.
>>>>>>>>>>>>
>>>>>>>>>>>> Matei
>>>>>>>>>>>>
>>>>>>>>>>>> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Sorry this should have been sent to user mail list.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi there:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I has build and installed mesos (pre-protobuf branch for
>>>>>>>>>>>>>> spark)
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>> spark,
>>>>>>>>>>>>>> and deployed them to a cluster. However, I have following
>>>>>>>>>>>>>> error
>>>>>>>>>>>>>> when
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>> run
>>>>>>>>>>>>>> spark task on the mesos URL of my master, the spark task runs
>>>>>>>>>>>>>> fine
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>> local mode. Any suggestion or help? Thanks a lot!
>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> My os: CentOS 5.2
>>>>>>>>>>>>>> Python: 2.6
>>>>>>>>>>>>>> Gcc: 4.2.1
>>>>>>>>>>>>>> Swig:1.3.40
>>>>>>>>>>>>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered
>>>>>>>>>>>>>> actor
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>> port
>>>>>>>>>>>>>> 50501
>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor:
>>>>>>>>>>>>>> Registered
>>>>>>>>>>>>>> actor
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>> port 50501
>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID
>>>>>>>>>>>>>> 0
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>> cache
>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering
>>>>>>>>>>>>>> RDD
>>>>>>>>>>>>>> 0
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>> 1
>>>>>>>>>>>>>> partitions
>>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for
>>>>>>>>>>>>>> current
>>>>>>>>>>>>>> cache
>>>>>>>>>>>>>> locations
>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage:
>>>>>>>>>>>>>> Stage
>>>>>>>>>>>>>> 0
>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final
>>>>>>>>>>>>>> stage:
>>>>>>>>>>>>>> List()
>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents:
>>>>>>>>>>>>>> List()
>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage
>>>>>>>>>>>>>> 0,
>>>>>>>>>>>>>> which
>>>>>>>>>>>>>> has
>>>>>>>>>>>>>> no missing parents
>>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1
>>>>>>>>>>>>>> tasks
>>>>>>>>>>>>>> Exception in thread "Spark scheduler"
>>>>>>>>>>>>>> java.lang.UnsatisfiedLinkError:
>>>>>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>>>>>>>>>>>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native
>>>>>>>>>>>>>> Method)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDri
>>>>>>>>>>>>>> v
>>>>>>>>>>>>>> er
>>>>>>>>>>>>>> .
>>>>>>>>>>>>>> ja
>>>>>>>>>>>>>> v
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> :4
>>>>>>>>>>>>>> 2)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDri
>>>>>>>>>>>>>> v
>>>>>>>>>>>>>> er
>>>>>>>>>>>>>> .
>>>>>>>>>>>>>> ja
>>>>>>>>>>>>>> v
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> :4
>>>>>>>>>>>>>> 6)
>>>>>>>>>>>>>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sent to wrong address previously
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Acknowledgment: I have added the address
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> to the mesos-dev mailing list.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please save this message so that you know the address you
>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>> subscribed under, in case you later want to unsubscribe or
>>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>> subscription address.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>>>>> request),
>>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To get an index with subject and author for messages
>>>>>>>>>>>>>>>> 123-456 ,
>>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per
>>>>>>>>>>>>>>>> request,
>>>>>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>>>>> 12345,
>>>>>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The messages should contain one line or word of text to
>>>>>>>>>>>>>>>> avoid
>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>>>>>>> address (with '=' instead of '@') after the command word:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <mesos-dev-unsubscribe-john=host.domain@incubator.apache.org
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>>>>> address.
>>>>>>>>>>>>>>>> When
>>>>>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>>>>> subscription.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my
>>>>>>>>>>>>>>>> owner
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>>>>> 23:30:07
>>>>>>>>>>>>>>>> -0000
>>>>>>>>>>>>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>>>>>>>>>>>> (192.87.106.230)
>>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>>>> 23:30:07
>>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>>>>>>>>>>>>> tests=SPF_PASS
>>>>>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>>>>>> Received-SPF: pass (nike.apache.org: domain of
>>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>> designates
>>>>>>>>>>>>>>>> 67.231.145.42 as permitted sender)
>>>>>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>>>>> mx0a-00082601.pphosted.com)
>>>>>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>>>> 23:30:00
>>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>>>>>>>>>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>>>>>> p7LNTI6g015638
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com
>>>>>>>>>>>>>>>> @
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> t
>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>>>>>> Received: from mail.thefacebook.com
>>>>>>>>>>>>>>>> (corpout1.snc1.tfbnw.net
>>>>>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>>>>>>>>>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128
>>>>>>>>>>>>>>>> verify=NOT)
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com
>>>>>>>>>>>>>>>> @
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> t
>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>>>>>> Sun,
>>>>>>>>>>>>>>>> 21 Aug 2011 16:29:14 -0700
>>>>>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>>>>>> To:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> "mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com
>>>>>>>>>>>>>>>> @
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> t
>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> che.org"
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.co
>>>>>>>>>>>>>>>> m
>>>>>>>>>>>>>>>> @i
>>>>>>>>>>>>>>>> n
>>>>>>>>>>>>>>>> cu
>>>>>>>>>>>>>>>> b
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> r.
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>> ache.org>
>>>>>>>>>>>>>>>> Subject: Re: confirm subscribe to
>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org
>>>>>>>>>>>>>>>> Thread-Topic: confirm subscribe to
>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org
>>>>>>>>>>>>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>>>>>>>>>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>>>>>> In-Reply-To: <13...@incubator.apache.org>
>>>>>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>>>>>> X-MS-Has-Attach:
>>>>>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>>>>>>>>>>>> Content-ID: <B4...@fb.com>
>>>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To confirm that you would like
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> added to the mesos-dev mailing list, please send
>>>>>>>>>>>>>>>>> a short reply to this address:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> =20
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.c
>>>>>>>>>>>>>>>>> o
>>>>>>>>>>>>>>>>> m@
>>>>>>>>>>>>>>>>> i
>>>>>>>>>>>>>>>>> nc
>>>>>>>>>>>>>>>>> u
>>>>>>>>>>>>>>>>> b
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> p=
>>>>>>>>>>>>>>>> ac
>>>>>>>>>>>>>>>>> he.org
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Usually, this happens when you just hit the "reply"
>>>>>>>>>>>>>>>>> button.
>>>>>>>>>>>>>>>>> If this does not work, simply copy the address and paste
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>> the "To:" field of a new message.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> or click here:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyon
>>>>>>>>>>>>>>>>> g
>>>>>>>>>>>>>>>>> =3
>>>>>>>>>>>>>>>>> D
>>>>>>>>>>>>>>>>> fb
>>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>>> om
>>>>>>>>>>>>>>>>> @i
>>>>>>>>>>>>>>>>> n
>>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>>> u=
>>>>>>>>>>>>>>>> ba
>>>>>>>>>>>>>>>>> tor.apache.org
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This confirmation serves two purposes. First, it verifies
>>>>>>>>>>>>>>>>> that I
>>>>>>>>>>>>>>>>> am
>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>> to get mail through to you. Second, it protects you in
>>>>>>>>>>>>>>>>> case
>>>>>>>>>>>>>>>>> someone
>>>>>>>>>>>>>>>>> forges a subscription request in your name.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Please note that ALL Apache dev- and user- mailing lists
>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>> publicly
>>>>>>>>>>>>>>>>> archived. Do familiarize yourself with Apache's public
>>>>>>>>>>>>>>>>> archive
>>>>>>>>>>>>>>>>> policy
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> prior to subscribing and posting messages to
>>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org.
>>>>>>>>>>>>>>>>> If you're not sure whether or not the policy applies to
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> mailing
>>>>>>>>>>>>>>>>> list,
>>>>>>>>>>>>>>>>> assume it does unless the list name contains the word
>>>>>>>>>>>>>>>>> "private"
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> it.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Some mail programs are broken and cannot handle long
>>>>>>>>>>>>>>>>> addresses.
>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>> cannot reply to this request, instead send a message to
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org> and put the
>>>>>>>>>>>>>>>>> entire address listed above into the "Subject:" line.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>>>>>> request),
>>>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To get an index with subject and author for messages
>>>>>>>>>>>>>>>>> 123-456
>>>>>>>>>>>>>>>>> ,
>>>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per
>>>>>>>>>>>>>>>>> request,
>>>>>>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>>>>>> 12345,
>>>>>>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The messages should contain one line or word of text to
>>>>>>>>>>>>>>>>> avoid
>>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>>>>>>>> address (with '=3D' instead of '@') after the command
>>>>>>>>>>>>>>>>> word:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> <mesos-dev-subscribe-john=3Dhost.domain@incubator.apache.or
>>>>>>>>>>>>>>>>> g
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> <mesos-dev-unsubscribe-john=3Dhost.domain@incubator.apache.
>>>>>>>>>>>>>>>>> o
>>>>>>>>>>>>>>>>> rg
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>>>>>> address.
>>>>>>>>>>>>>>>>> When
>>>>>>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>>>>>> subscription.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If despite following these instructions, you do not get
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient,
>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>> owner
>>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>>>>>> -0000
>>>>>>>>>>>>>>>>> Received: from athena.apache.org (HELO athena.apache.org)
>>>>>>>>>>>>>>>>> (140.211.11.136)
>>>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>>>>>>>>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>>>>>>> Received-SPF: pass (athena.apache.org: domain of
>>>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>>>>>> mx0a-00082601.pphosted.com)
>>>>>>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>>>>> 22:43:19
>>>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>>>>>>>>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>>>>>>> p7LMcbSq017336
>>>>>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21
>>>>>>>>>>>>>>>>> Aug
>>>>>>>>>>>>>>>>> 2011
>>>>>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>>>>>> Received: from mail.thefacebook.com
>>>>>>>>>>>>>>>>> (corpout1.snc1.tfbnw.net
>>>>>>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>>>>>>>>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128
>>>>>>>>>>>>>>>>> verify=3DNOT)
>>>>>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21
>>>>>>>>>>>>>>>>> Aug
>>>>>>>>>>>>>>>>> 2011
>>>>>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>>>>>>> Sun,
>>>>>>>>>>>>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>> Subject: can i join this
>>>>>>>>>>>>>>>>> Thread-Topic: can i join this
>>>>>>>>>>>>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>>>>>>>>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>>>>>>> X-MS-Has-Attach:=20
>>>>>>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>>>>>>> Content-Type: multipart/alternative;
>>>>>>>>>>>>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>>>>>>>>>>>>> Content-ID: <F6...@fb.com>
>>>>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> <html>
>>>>>>>>>>>>>>>>> <head>
>>>>>>>>>>>>>>>>> <meta http-equiv=3D3D"Content-Type"
>>>>>>>>>>>>>>>>> content=3D3D"text/html;
>>>>>>>>>>>>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> </head>
>>>>>>>>>>>>>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode:
>>>>>>>>>>>>>>>>> space;
>>>>>>>>>>>>>>>>> -webkit-lin=3D
>>>>>>>>>>>>>>>>> e-break: after-white-space; color: rgb(0, 0, 0);
>>>>>>>>>>>>>>>>> font-size:
>>>>>>>>>>>>>>>>> 14px;
>>>>>>>>>>>>>>>>> font-fami=3D
>>>>>>>>>>>>>>>>> ly: Calibri, sans-serif; ">
>>>>>>>>>>>>>>>>> <div>Best,</div>
>>>>>>>>>>>>>>>>> <div>Joe</div>
>>>>>>>>>>>>>>>>> </body>
>>>>>>>>>>>>>>>>> </html>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: About spark-shell in mesos cluster mode Re: WELCOME to
mesos-dev@incubator.apache.org
Posted by Joe Xie <zh...@fb.com>.
Hi Matei,
when I start with "MASTER=1@mymesoshost:5050 ./spark-shell" it seems I
keep losing TID after starting the task, and failed after 4 tries. I'm
trying to looking into code now, but let me know if have any suggestion
too.
11/08/24 07:52:34 ERROR spark.SimpleJob: Task 0:6 failed more than 4
times; aborting job
11/08/24 07:52:34 INFO spark.MesosScheduler: Ignoring update from TID 41
because its job is gone
Thanks again,
Joe
On 8/23/11 6:52 PM, "Joe Xie" <zh...@fb.com> wrote:
>Thanks! I was use another email.
>
>From: Mail Delivery Subsystem <ma...@googlemail.com>
>Date: Tue, Aug 23, 2011 at 3:29 PM
>Subject: Delivery Status Notification (Failure)
>To: zyxienju@gmail.com
>Delivery to the following recipient failed permanently:
>spark-users@eecs.berkeley.edu
>Technical details of permanent failure:
>Google tried to deliver your message, but it was rejected by the recipient
>domain. We recommend contacting the other email provider for further
>information about the cause of this error. The error that the other server
>returned was: 550 550 unknown user (state 14).
>
>
>
>On 8/23/11 4:50 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>
>>Weird, did you subscribe to the spark-users mailing list? I don't think
>>you can post to it without doing that. Go to
>>https://lists.eecs.berkeley.edu/sympa/subscribe/spark-users.
>>
>>Anyway, to run the shell on a cluster, you need to set the MASTER
>>environment variable to the Mesos master URL you want to use (similar to
>>the one you pass to the standalone programs). So for example, type
>>
>>MASTER=1@localhost:5050 ./spark-shell
>>
>>Matei
>>
>>
>>On Aug 23, 2011, at 4:39 PM, Joe Xie wrote:
>>
>>> Hi there,
>>>
>>> may I ask how can I run spark-shell in cluster mode? The web wiki say
>>> change the configuration of master, but I did not find detailed info
>>>yet.
>>> It seems my email failed at spark-user maillist.
>>> Thanks!
>>> Joe
>>>
>>>
>>>
>>> On 8/22/11 4:42 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>
>>>> Thank you Matei! It's working now. I checked the stderr, it seems I
>>>>have
>>>> to specify the SCALA_HOME even if it's in .bashrc.
>>>> Best,
>>>> Joe
>>>>
>>>> On 8/22/11 4:13 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>>
>>>>> This probably means that your task crashed. Take a look at the logs
>>>>>Mesos
>>>>> collects for your task, which will be in
>>>>> MESOS_HOME/work/slave-id/framework-id/attempt-id/stdout and stderr.
>>>>>
>>>>> Sometimes this happens because spark-env.sh isn't configured on all
>>>>>the
>>>>> nodes to contain SCALA_HOME, JAVA_HOME and MESOS_HOME. It can also
>>>>>happen
>>>>> because your code is not available on all the machines and you get a
>>>>> ClassNotFoundError. You can either copy the code to the same place
>>>>> everywhere and put that in the SPARK_CLASSPATH variable in
>>>>>spark-env.sh,
>>>>> or package your code into a JAR and use the "jars" parameter to
>>>>> SparkContext to pass it (in which case Spark will copy it to all the
>>>>> nodes).
>>>>>
>>>>> Matei
>>>>>
>>>>> On Aug 22, 2011, at 4:04 PM, Joe Xie wrote:
>>>>>
>>>>>> Btw, I checkout my webui for master, it indicates "No offers are
>>>>>> active"
>>>>>> under "Resource Offers" while I have two machines as slave, so there
>>>>>> might
>>>>>> be problem with my configuration.
>>>>>> Best,
>>>>>> JOE
>>>>>>
>>>>>> On 8/22/11 3:59 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>
>>>>>>> Thank you Matei,
>>>>>>>
>>>>>>> it's running for me with Spark 0.3 + pre-protobuf, and I'm testing
>>>>>>> with
>>>>>>> provided example "run spark.examples.SparkPi
>>>>>>> mesos://1@10.30.95.184:5050
>>>>>>> 3". However, it seems I keep losing TID after starting the task,
>>>>>>>and
>>>>>>> then
>>>>>>> job aborted with 4 tries. It might be the problem on my side too,
>>>>>>>did
>>>>>>> this
>>>>>>> happen before? I could go for trying masters for spark + mesos.
>>>>>>> 11/08/22 15:48:27 INFO spark.SimpleJob: Lost TID 13 (task 0:2)
>>>>>>> 11/08/22 15:48:27 ERROR spark.SimpleJob: Task 0:2 failed more than
>>>>>>>4
>>>>>>> times; aborting job
>>>>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from
>>>>>>>TID
>>>>>>> 12
>>>>>>> because its job is gone
>>>>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from
>>>>>>>TID
>>>>>>> 14
>>>>>>> because its job is gone
>>>>>>>
>>>>>>> Thanks!
>>>>>>> Joe
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 8/22/11 3:38 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>
>>>>>>>> Thank you Matei! I can go for Spark 0.3 + pre-protobuf. I
>>>>>>>>understand
>>>>>>>> there
>>>>>>>> might be lots of new updates on the master branch. Is there any
>>>>>>>> significant difference in current master with Spark 0.3 +
>>>>>>>> pre-protobuf? I
>>>>>>>> could also try to keep a separate branch with stable masters if
>>>>>>>> that's
>>>>>>>> the
>>>>>>>> case.
>>>>>>>> Best,
>>>>>>>> Joe
>>>>>>>> On 8/22/11 3:21 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu>
>>>>>>>>wrote:
>>>>>>>>
>>>>>>>>> If you want the most stable versions right now, I recommend Spark
>>>>>>>>> 0.3
>>>>>>>>> (download it from the website or the 0.3-scala-2.9 tag in Git)
>>>>>>>>>and
>>>>>>>>> pre-protobuf Mesos. However, the current master Spark + master
>>>>>>>>>Mesos
>>>>>>>>> should also work OK; it's just that further commits may break
>>>>>>>>>things
>>>>>>>>> temporarily.
>>>>>>>>>
>>>>>>>>> Matei
>>>>>>>>>
>>>>>>>>> On Aug 22, 2011, at 3:17 PM, Joe Xie wrote:
>>>>>>>>>
>>>>>>>>>> Hi Matei,
>>>>>>>>>> Thanks for your feedback. I bet I'm using the master brach of
>>>>>>>>>>Spark
>>>>>>>>>> and
>>>>>>>>>> pre-protobuf version of Mesos. Do you mean I should go for Spark
>>>>>>>>>> (master)
>>>>>>>>>> + protobuf (master). Let's me know the most stable combination
>>>>>>>>>>of
>>>>>>>>>> braches
>>>>>>>>>> from Spark and pre-protobuf, I can try them out.
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Joe
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Are you using the master branch of Spark, or one of the
>>>>>>>>>>>releases
>>>>>>>>>>> (e.g.
>>>>>>>>>>> 0.3)? The releases work with the pre-protobuf version of Mesos
>>>>>>>>>>>but
>>>>>>>>>>> the
>>>>>>>>>>> master branch of Spark actually requires the master branch of
>>>>>>>>>>> Mesos
>>>>>>>>>>> now.
>>>>>>>>>>> That could cause the kind of error you're seeing.
>>>>>>>>>>>
>>>>>>>>>>> Matei
>>>>>>>>>>>
>>>>>>>>>>> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Sorry this should have been sent to user mail list.
>>>>>>>>>>>>
>>>>>>>>>>>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi there:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I has build and installed mesos (pre-protobuf branch for
>>>>>>>>>>>>>spark)
>>>>>>>>>>>>> and
>>>>>>>>>>>>> spark,
>>>>>>>>>>>>> and deployed them to a cluster. However, I have following
>>>>>>>>>>>>>error
>>>>>>>>>>>>> when
>>>>>>>>>>>>> I
>>>>>>>>>>>>> run
>>>>>>>>>>>>> spark task on the mesos URL of my master, the spark task runs
>>>>>>>>>>>>> fine
>>>>>>>>>>>>> in
>>>>>>>>>>>>> local mode. Any suggestion or help? Thanks a lot!
>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>
>>>>>>>>>>>>> My os: CentOS 5.2
>>>>>>>>>>>>> Python: 2.6
>>>>>>>>>>>>> Gcc: 4.2.1
>>>>>>>>>>>>> Swig:1.3.40
>>>>>>>>>>>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered
>>>>>>>>>>>>>actor
>>>>>>>>>>>>> on
>>>>>>>>>>>>> port
>>>>>>>>>>>>> 50501
>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor:
>>>>>>>>>>>>>Registered
>>>>>>>>>>>>> actor
>>>>>>>>>>>>> on
>>>>>>>>>>>>> port 50501
>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID
>>>>>>>>>>>>>0
>>>>>>>>>>>>> with
>>>>>>>>>>>>> cache
>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering
>>>>>>>>>>>>>RDD
>>>>>>>>>>>>> 0
>>>>>>>>>>>>> with
>>>>>>>>>>>>> 1
>>>>>>>>>>>>> partitions
>>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for
>>>>>>>>>>>>> current
>>>>>>>>>>>>> cache
>>>>>>>>>>>>> locations
>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage:
>>>>>>>>>>>>>Stage
>>>>>>>>>>>>> 0
>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final
>>>>>>>>>>>>> stage:
>>>>>>>>>>>>> List()
>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents:
>>>>>>>>>>>>> List()
>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage
>>>>>>>>>>>>>0,
>>>>>>>>>>>>> which
>>>>>>>>>>>>> has
>>>>>>>>>>>>> no missing parents
>>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1
>>>>>>>>>>>>> tasks
>>>>>>>>>>>>> Exception in thread "Spark scheduler"
>>>>>>>>>>>>> java.lang.UnsatisfiedLinkError:
>>>>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>>>>>>>>>>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native
>>>>>>>>>>>>> Method)
>>>>>>>>>>>>> at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDri
>>>>>>>>>>>>>v
>>>>>>>>>>>>>er
>>>>>>>>>>>>> .
>>>>>>>>>>>>> ja
>>>>>>>>>>>>> v
>>>>>>>>>>>>> a
>>>>>>>>>>>>> :4
>>>>>>>>>>>>> 2)
>>>>>>>>>>>>> at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDri
>>>>>>>>>>>>>v
>>>>>>>>>>>>>er
>>>>>>>>>>>>> .
>>>>>>>>>>>>> ja
>>>>>>>>>>>>> v
>>>>>>>>>>>>> a
>>>>>>>>>>>>> :4
>>>>>>>>>>>>> 6)
>>>>>>>>>>>>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sent to wrong address previously
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Acknowledgment: I have added the address
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> to the mesos-dev mailing list.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please save this message so that you know the address you
>>>>>>>>>>>>>>>are
>>>>>>>>>>>>>>> subscribed under, in case you later want to unsubscribe or
>>>>>>>>>>>>>>> change
>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>> subscription address.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>>>> request),
>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To get an index with subject and author for messages
>>>>>>>>>>>>>>>123-456 ,
>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per
>>>>>>>>>>>>>>>request,
>>>>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>>>> 12345,
>>>>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The messages should contain one line or word of text to
>>>>>>>>>>>>>>>avoid
>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>>>>>> address (with '=' instead of '@') after the command word:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>><mesos-dev-unsubscribe-john=host.domain@incubator.apache.org
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>>>> address.
>>>>>>>>>>>>>>> When
>>>>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>>>> subscription.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my
>>>>>>>>>>>>>>> owner
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>>>> 23:30:07
>>>>>>>>>>>>>>> -0000
>>>>>>>>>>>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>>>>>>>>>>> (192.87.106.230)
>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>>> 23:30:07
>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>>>>>>>>>>>> tests=SPF_PASS
>>>>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>>>>> Received-SPF: pass (nike.apache.org: domain of
>>>>>>>>>>>>>>>zhiyong@fb.com
>>>>>>>>>>>>>>> designates
>>>>>>>>>>>>>>> 67.231.145.42 as permitted sender)
>>>>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>>>> mx0a-00082601.pphosted.com)
>>>>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>>> 23:30:00
>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>>>>>>>>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>>>>> p7LNTI6g015638
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com
>>>>>>>>>>>>>>>@
>>>>>>>>>>>>>>>in
>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> t
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>>>>> Received: from mail.thefacebook.com
>>>>>>>>>>>>>>>(corpout1.snc1.tfbnw.net
>>>>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>>>>>>>>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128
>>>>>>>>>>>>>>>verify=NOT)
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com
>>>>>>>>>>>>>>>@
>>>>>>>>>>>>>>>in
>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> t
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>>>>> Sun,
>>>>>>>>>>>>>>> 21 Aug 2011 16:29:14 -0700
>>>>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>>>>> To:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>"mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com
>>>>>>>>>>>>>>>@
>>>>>>>>>>>>>>>in
>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> t
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> che.org"
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.co
>>>>>>>>>>>>>>>m
>>>>>>>>>>>>>>>@i
>>>>>>>>>>>>>>> n
>>>>>>>>>>>>>>> cu
>>>>>>>>>>>>>>> b
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> r.
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>> ache.org>
>>>>>>>>>>>>>>> Subject: Re: confirm subscribe to
>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org
>>>>>>>>>>>>>>> Thread-Topic: confirm subscribe to
>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org
>>>>>>>>>>>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>>>>>>>>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>>>>> In-Reply-To: <13...@incubator.apache.org>
>>>>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>>>>> X-MS-Has-Attach:
>>>>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>>>>>>>>>>> Content-ID: <B4...@fb.com>
>>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To confirm that you would like
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> added to the mesos-dev mailing list, please send
>>>>>>>>>>>>>>>> a short reply to this address:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> =20
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.c
>>>>>>>>>>>>>>>>o
>>>>>>>>>>>>>>>>m@
>>>>>>>>>>>>>>>> i
>>>>>>>>>>>>>>>> nc
>>>>>>>>>>>>>>>> u
>>>>>>>>>>>>>>>> b
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> p=
>>>>>>>>>>>>>>> ac
>>>>>>>>>>>>>>>> he.org
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Usually, this happens when you just hit the "reply"
>>>>>>>>>>>>>>>>button.
>>>>>>>>>>>>>>>> If this does not work, simply copy the address and paste
>>>>>>>>>>>>>>>>it
>>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>> the "To:" field of a new message.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> or click here:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyon
>>>>>>>>>>>>>>>>g
>>>>>>>>>>>>>>>>=3
>>>>>>>>>>>>>>>> D
>>>>>>>>>>>>>>>> fb
>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>> om
>>>>>>>>>>>>>>>> @i
>>>>>>>>>>>>>>>> n
>>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>>> u=
>>>>>>>>>>>>>>> ba
>>>>>>>>>>>>>>>> tor.apache.org
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This confirmation serves two purposes. First, it verifies
>>>>>>>>>>>>>>>> that I
>>>>>>>>>>>>>>>> am
>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>> to get mail through to you. Second, it protects you in
>>>>>>>>>>>>>>>>case
>>>>>>>>>>>>>>>> someone
>>>>>>>>>>>>>>>> forges a subscription request in your name.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please note that ALL Apache dev- and user- mailing lists
>>>>>>>>>>>>>>>>are
>>>>>>>>>>>>>>>> publicly
>>>>>>>>>>>>>>>> archived. Do familiarize yourself with Apache's public
>>>>>>>>>>>>>>>> archive
>>>>>>>>>>>>>>>> policy
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> prior to subscribing and posting messages to
>>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org.
>>>>>>>>>>>>>>>> If you're not sure whether or not the policy applies to
>>>>>>>>>>>>>>>>this
>>>>>>>>>>>>>>>> mailing
>>>>>>>>>>>>>>>> list,
>>>>>>>>>>>>>>>> assume it does unless the list name contains the word
>>>>>>>>>>>>>>>> "private"
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> it.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Some mail programs are broken and cannot handle long
>>>>>>>>>>>>>>>> addresses.
>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>> cannot reply to this request, instead send a message to
>>>>>>>>>>>>>>>> <me...@incubator.apache.org> and put the
>>>>>>>>>>>>>>>> entire address listed above into the "Subject:" line.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>>>>> request),
>>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To get an index with subject and author for messages
>>>>>>>>>>>>>>>>123-456
>>>>>>>>>>>>>>>> ,
>>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per
>>>>>>>>>>>>>>>> request,
>>>>>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>>>>> 12345,
>>>>>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The messages should contain one line or word of text to
>>>>>>>>>>>>>>>>avoid
>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>>>>>>> address (with '=3D' instead of '@') after the command
>>>>>>>>>>>>>>>>word:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>><mesos-dev-subscribe-john=3Dhost.domain@incubator.apache.or
>>>>>>>>>>>>>>>>g
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>><mesos-dev-unsubscribe-john=3Dhost.domain@incubator.apache.
>>>>>>>>>>>>>>>>o
>>>>>>>>>>>>>>>>rg
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>>>>> address.
>>>>>>>>>>>>>>>> When
>>>>>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>>>>> subscription.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If despite following these instructions, you do not get
>>>>>>>>>>>>>>>>the
>>>>>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient,
>>>>>>>>>>>>>>>>my
>>>>>>>>>>>>>>>> owner
>>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>>>>> -0000
>>>>>>>>>>>>>>>> Received: from athena.apache.org (HELO athena.apache.org)
>>>>>>>>>>>>>>>> (140.211.11.136)
>>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>>>>>>>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>>>>>> Received-SPF: pass (athena.apache.org: domain of
>>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>>>>> mx0a-00082601.pphosted.com)
>>>>>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>>>> 22:43:19
>>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>>>>>>>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>>>>>> p7LMcbSq017336
>>>>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21
>>>>>>>>>>>>>>>>Aug
>>>>>>>>>>>>>>>> 2011
>>>>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>>>>> Received: from mail.thefacebook.com
>>>>>>>>>>>>>>>>(corpout1.snc1.tfbnw.net
>>>>>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>>>>>>>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128
>>>>>>>>>>>>>>>> verify=3DNOT)
>>>>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21
>>>>>>>>>>>>>>>>Aug
>>>>>>>>>>>>>>>> 2011
>>>>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>>>>>> Sun,
>>>>>>>>>>>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>> Subject: can i join this
>>>>>>>>>>>>>>>> Thread-Topic: can i join this
>>>>>>>>>>>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>>>>>>>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>>>>>> X-MS-Has-Attach:=20
>>>>>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>>>>>> Content-Type: multipart/alternative;
>>>>>>>>>>>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>>>>>>>>>>>> Content-ID: <F6...@fb.com>
>>>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> <html>
>>>>>>>>>>>>>>>> <head>
>>>>>>>>>>>>>>>> <meta http-equiv=3D3D"Content-Type"
>>>>>>>>>>>>>>>>content=3D3D"text/html;
>>>>>>>>>>>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> </head>
>>>>>>>>>>>>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode:
>>>>>>>>>>>>>>>> space;
>>>>>>>>>>>>>>>> -webkit-lin=3D
>>>>>>>>>>>>>>>> e-break: after-white-space; color: rgb(0, 0, 0);
>>>>>>>>>>>>>>>>font-size:
>>>>>>>>>>>>>>>> 14px;
>>>>>>>>>>>>>>>> font-fami=3D
>>>>>>>>>>>>>>>> ly: Calibri, sans-serif; ">
>>>>>>>>>>>>>>>> <div>Best,</div>
>>>>>>>>>>>>>>>> <div>Joe</div>
>>>>>>>>>>>>>>>> </body>
>>>>>>>>>>>>>>>> </html>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: About spark-shell in mesos cluster mode Re: WELCOME to
mesos-dev@incubator.apache.org
Posted by Joe Xie <zh...@fb.com>.
Thanks! I was use another email.
From: Mail Delivery Subsystem <ma...@googlemail.com>
Date: Tue, Aug 23, 2011 at 3:29 PM
Subject: Delivery Status Notification (Failure)
To: zyxienju@gmail.com
Delivery to the following recipient failed permanently:
spark-users@eecs.berkeley.edu
Technical details of permanent failure:
Google tried to deliver your message, but it was rejected by the recipient
domain. We recommend contacting the other email provider for further
information about the cause of this error. The error that the other server
returned was: 550 550 unknown user (state 14).
On 8/23/11 4:50 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>Weird, did you subscribe to the spark-users mailing list? I don't think
>you can post to it without doing that. Go to
>https://lists.eecs.berkeley.edu/sympa/subscribe/spark-users.
>
>Anyway, to run the shell on a cluster, you need to set the MASTER
>environment variable to the Mesos master URL you want to use (similar to
>the one you pass to the standalone programs). So for example, type
>
>MASTER=1@localhost:5050 ./spark-shell
>
>Matei
>
>
>On Aug 23, 2011, at 4:39 PM, Joe Xie wrote:
>
>> Hi there,
>>
>> may I ask how can I run spark-shell in cluster mode? The web wiki say
>> change the configuration of master, but I did not find detailed info
>>yet.
>> It seems my email failed at spark-user maillist.
>> Thanks!
>> Joe
>>
>>
>>
>> On 8/22/11 4:42 PM, "Joe Xie" <zh...@fb.com> wrote:
>>
>>> Thank you Matei! It's working now. I checked the stderr, it seems I
>>>have
>>> to specify the SCALA_HOME even if it's in .bashrc.
>>> Best,
>>> Joe
>>>
>>> On 8/22/11 4:13 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>
>>>> This probably means that your task crashed. Take a look at the logs
>>>>Mesos
>>>> collects for your task, which will be in
>>>> MESOS_HOME/work/slave-id/framework-id/attempt-id/stdout and stderr.
>>>>
>>>> Sometimes this happens because spark-env.sh isn't configured on all
>>>>the
>>>> nodes to contain SCALA_HOME, JAVA_HOME and MESOS_HOME. It can also
>>>>happen
>>>> because your code is not available on all the machines and you get a
>>>> ClassNotFoundError. You can either copy the code to the same place
>>>> everywhere and put that in the SPARK_CLASSPATH variable in
>>>>spark-env.sh,
>>>> or package your code into a JAR and use the "jars" parameter to
>>>> SparkContext to pass it (in which case Spark will copy it to all the
>>>> nodes).
>>>>
>>>> Matei
>>>>
>>>> On Aug 22, 2011, at 4:04 PM, Joe Xie wrote:
>>>>
>>>>> Btw, I checkout my webui for master, it indicates "No offers are
>>>>> active"
>>>>> under "Resource Offers" while I have two machines as slave, so there
>>>>> might
>>>>> be problem with my configuration.
>>>>> Best,
>>>>> JOE
>>>>>
>>>>> On 8/22/11 3:59 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>
>>>>>> Thank you Matei,
>>>>>>
>>>>>> it's running for me with Spark 0.3 + pre-protobuf, and I'm testing
>>>>>> with
>>>>>> provided example "run spark.examples.SparkPi
>>>>>> mesos://1@10.30.95.184:5050
>>>>>> 3". However, it seems I keep losing TID after starting the task, and
>>>>>> then
>>>>>> job aborted with 4 tries. It might be the problem on my side too,
>>>>>>did
>>>>>> this
>>>>>> happen before? I could go for trying masters for spark + mesos.
>>>>>> 11/08/22 15:48:27 INFO spark.SimpleJob: Lost TID 13 (task 0:2)
>>>>>> 11/08/22 15:48:27 ERROR spark.SimpleJob: Task 0:2 failed more than 4
>>>>>> times; aborting job
>>>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from
>>>>>>TID
>>>>>> 12
>>>>>> because its job is gone
>>>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from
>>>>>>TID
>>>>>> 14
>>>>>> because its job is gone
>>>>>>
>>>>>> Thanks!
>>>>>> Joe
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 8/22/11 3:38 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>
>>>>>>> Thank you Matei! I can go for Spark 0.3 + pre-protobuf. I
>>>>>>>understand
>>>>>>> there
>>>>>>> might be lots of new updates on the master branch. Is there any
>>>>>>> significant difference in current master with Spark 0.3 +
>>>>>>> pre-protobuf? I
>>>>>>> could also try to keep a separate branch with stable masters if
>>>>>>> that's
>>>>>>> the
>>>>>>> case.
>>>>>>> Best,
>>>>>>> Joe
>>>>>>> On 8/22/11 3:21 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu>
>>>>>>>wrote:
>>>>>>>
>>>>>>>> If you want the most stable versions right now, I recommend Spark
>>>>>>>> 0.3
>>>>>>>> (download it from the website or the 0.3-scala-2.9 tag in Git) and
>>>>>>>> pre-protobuf Mesos. However, the current master Spark + master
>>>>>>>>Mesos
>>>>>>>> should also work OK; it's just that further commits may break
>>>>>>>>things
>>>>>>>> temporarily.
>>>>>>>>
>>>>>>>> Matei
>>>>>>>>
>>>>>>>> On Aug 22, 2011, at 3:17 PM, Joe Xie wrote:
>>>>>>>>
>>>>>>>>> Hi Matei,
>>>>>>>>> Thanks for your feedback. I bet I'm using the master brach of
>>>>>>>>>Spark
>>>>>>>>> and
>>>>>>>>> pre-protobuf version of Mesos. Do you mean I should go for Spark
>>>>>>>>> (master)
>>>>>>>>> + protobuf (master). Let's me know the most stable combination
>>>>>>>>>of
>>>>>>>>> braches
>>>>>>>>> from Spark and pre-protobuf, I can try them out.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Joe
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Are you using the master branch of Spark, or one of the releases
>>>>>>>>>> (e.g.
>>>>>>>>>> 0.3)? The releases work with the pre-protobuf version of Mesos
>>>>>>>>>>but
>>>>>>>>>> the
>>>>>>>>>> master branch of Spark actually requires the master branch of
>>>>>>>>>> Mesos
>>>>>>>>>> now.
>>>>>>>>>> That could cause the kind of error you're seeing.
>>>>>>>>>>
>>>>>>>>>> Matei
>>>>>>>>>>
>>>>>>>>>> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>>>>>>>>>>
>>>>>>>>>>> Sorry this should have been sent to user mail list.
>>>>>>>>>>>
>>>>>>>>>>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi there:
>>>>>>>>>>>>
>>>>>>>>>>>> I has build and installed mesos (pre-protobuf branch for
>>>>>>>>>>>>spark)
>>>>>>>>>>>> and
>>>>>>>>>>>> spark,
>>>>>>>>>>>> and deployed them to a cluster. However, I have following
>>>>>>>>>>>>error
>>>>>>>>>>>> when
>>>>>>>>>>>> I
>>>>>>>>>>>> run
>>>>>>>>>>>> spark task on the mesos URL of my master, the spark task runs
>>>>>>>>>>>> fine
>>>>>>>>>>>> in
>>>>>>>>>>>> local mode. Any suggestion or help? Thanks a lot!
>>>>>>>>>>>> Joe
>>>>>>>>>>>>
>>>>>>>>>>>> My os: CentOS 5.2
>>>>>>>>>>>> Python: 2.6
>>>>>>>>>>>> Gcc: 4.2.1
>>>>>>>>>>>> Swig:1.3.40
>>>>>>>>>>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered
>>>>>>>>>>>>actor
>>>>>>>>>>>> on
>>>>>>>>>>>> port
>>>>>>>>>>>> 50501
>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered
>>>>>>>>>>>> actor
>>>>>>>>>>>> on
>>>>>>>>>>>> port 50501
>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID
>>>>>>>>>>>>0
>>>>>>>>>>>> with
>>>>>>>>>>>> cache
>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering
>>>>>>>>>>>>RDD
>>>>>>>>>>>> 0
>>>>>>>>>>>> with
>>>>>>>>>>>> 1
>>>>>>>>>>>> partitions
>>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for
>>>>>>>>>>>> current
>>>>>>>>>>>> cache
>>>>>>>>>>>> locations
>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage:
>>>>>>>>>>>>Stage
>>>>>>>>>>>> 0
>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final
>>>>>>>>>>>> stage:
>>>>>>>>>>>> List()
>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents:
>>>>>>>>>>>> List()
>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage
>>>>>>>>>>>>0,
>>>>>>>>>>>> which
>>>>>>>>>>>> has
>>>>>>>>>>>> no missing parents
>>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1
>>>>>>>>>>>> tasks
>>>>>>>>>>>> Exception in thread "Spark scheduler"
>>>>>>>>>>>> java.lang.UnsatisfiedLinkError:
>>>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>>>>>>>>>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native
>>>>>>>>>>>> Method)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriv
>>>>>>>>>>>>er
>>>>>>>>>>>> .
>>>>>>>>>>>> ja
>>>>>>>>>>>> v
>>>>>>>>>>>> a
>>>>>>>>>>>> :4
>>>>>>>>>>>> 2)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriv
>>>>>>>>>>>>er
>>>>>>>>>>>> .
>>>>>>>>>>>> ja
>>>>>>>>>>>> v
>>>>>>>>>>>> a
>>>>>>>>>>>> :4
>>>>>>>>>>>> 6)
>>>>>>>>>>>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>>>>>>>>>>
>>>>>>>>>>>> Sent to wrong address previously
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Acknowledgment: I have added the address
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> to the mesos-dev mailing list.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please save this message so that you know the address you
>>>>>>>>>>>>>>are
>>>>>>>>>>>>>> subscribed under, in case you later want to unsubscribe or
>>>>>>>>>>>>>> change
>>>>>>>>>>>>>> your
>>>>>>>>>>>>>> subscription address.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>>> request),
>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To get an index with subject and author for messages
>>>>>>>>>>>>>>123-456 ,
>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per
>>>>>>>>>>>>>>request,
>>>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>>> 12345,
>>>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The messages should contain one line or word of text to
>>>>>>>>>>>>>>avoid
>>>>>>>>>>>>>> being
>>>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>>>>> address (with '=' instead of '@') after the command word:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>><me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>>> address.
>>>>>>>>>>>>>> When
>>>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>>> subscription.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my
>>>>>>>>>>>>>> owner
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>>> 23:30:07
>>>>>>>>>>>>>> -0000
>>>>>>>>>>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>>>>>>>>>> (192.87.106.230)
>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>> 23:30:07
>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>>>>>>>>>>> tests=SPF_PASS
>>>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>>>> Received-SPF: pass (nike.apache.org: domain of
>>>>>>>>>>>>>>zhiyong@fb.com
>>>>>>>>>>>>>> designates
>>>>>>>>>>>>>> 67.231.145.42 as permitted sender)
>>>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>>> mx0a-00082601.pphosted.com)
>>>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>> 23:30:00
>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>>>>>>>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>>>> p7LNTI6g015638
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@
>>>>>>>>>>>>>>in
>>>>>>>>>>>>>> c
>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> t
>>>>>>>>>>>>>> or
>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>> p
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>>>>>>>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@
>>>>>>>>>>>>>>in
>>>>>>>>>>>>>> c
>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> t
>>>>>>>>>>>>>> or
>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>> p
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>>>> by
>>>>>>>>>>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>>>> Sun,
>>>>>>>>>>>>>> 21 Aug 2011 16:29:14 -0700
>>>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>>>> To:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>"mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@
>>>>>>>>>>>>>>in
>>>>>>>>>>>>>> c
>>>>>>>>>>>>>> ub
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> t
>>>>>>>>>>>>>> or
>>>>>>>>>>>>>> .a
>>>>>>>>>>>>>> p
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> che.org"
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com
>>>>>>>>>>>>>>@i
>>>>>>>>>>>>>> n
>>>>>>>>>>>>>> cu
>>>>>>>>>>>>>> b
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> r.
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> p
>>>>>>>>>>>>>> ache.org>
>>>>>>>>>>>>>> Subject: Re: confirm subscribe to
>>>>>>>>>>>>>> mesos-dev@incubator.apache.org
>>>>>>>>>>>>>> Thread-Topic: confirm subscribe to
>>>>>>>>>>>>>> mesos-dev@incubator.apache.org
>>>>>>>>>>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>>>>>>>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>>>> In-Reply-To: <13...@incubator.apache.org>
>>>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>>>> X-MS-Has-Attach:
>>>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>>>>>>>>>> Content-ID: <B4...@fb.com>
>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To confirm that you would like
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> added to the mesos-dev mailing list, please send
>>>>>>>>>>>>>>> a short reply to this address:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> =20
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.co
>>>>>>>>>>>>>>>m@
>>>>>>>>>>>>>>> i
>>>>>>>>>>>>>>> nc
>>>>>>>>>>>>>>> u
>>>>>>>>>>>>>>> b
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> p=
>>>>>>>>>>>>>> ac
>>>>>>>>>>>>>>> he.org
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Usually, this happens when you just hit the "reply" button.
>>>>>>>>>>>>>>> If this does not work, simply copy the address and paste it
>>>>>>>>>>>>>>> into
>>>>>>>>>>>>>>> the "To:" field of a new message.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> or click here:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong
>>>>>>>>>>>>>>>=3
>>>>>>>>>>>>>>> D
>>>>>>>>>>>>>>> fb
>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>> om
>>>>>>>>>>>>>>> @i
>>>>>>>>>>>>>>> n
>>>>>>>>>>>>>>> c
>>>>>>>>>>>>>>> u=
>>>>>>>>>>>>>> ba
>>>>>>>>>>>>>>> tor.apache.org
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This confirmation serves two purposes. First, it verifies
>>>>>>>>>>>>>>> that I
>>>>>>>>>>>>>>> am
>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>> to get mail through to you. Second, it protects you in case
>>>>>>>>>>>>>>> someone
>>>>>>>>>>>>>>> forges a subscription request in your name.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please note that ALL Apache dev- and user- mailing lists
>>>>>>>>>>>>>>>are
>>>>>>>>>>>>>>> publicly
>>>>>>>>>>>>>>> archived. Do familiarize yourself with Apache's public
>>>>>>>>>>>>>>> archive
>>>>>>>>>>>>>>> policy
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> prior to subscribing and posting messages to
>>>>>>>>>>>>>>> mesos-dev@incubator.apache.org.
>>>>>>>>>>>>>>> If you're not sure whether or not the policy applies to
>>>>>>>>>>>>>>>this
>>>>>>>>>>>>>>> mailing
>>>>>>>>>>>>>>> list,
>>>>>>>>>>>>>>> assume it does unless the list name contains the word
>>>>>>>>>>>>>>> "private"
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Some mail programs are broken and cannot handle long
>>>>>>>>>>>>>>> addresses.
>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>> cannot reply to this request, instead send a message to
>>>>>>>>>>>>>>> <me...@incubator.apache.org> and put the
>>>>>>>>>>>>>>> entire address listed above into the "Subject:" line.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>>>> request),
>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To get an index with subject and author for messages
>>>>>>>>>>>>>>>123-456
>>>>>>>>>>>>>>> ,
>>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per
>>>>>>>>>>>>>>> request,
>>>>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>>>> 12345,
>>>>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The messages should contain one line or word of text to
>>>>>>>>>>>>>>>avoid
>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>>>>>> address (with '=3D' instead of '@') after the command word:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>><mesos-dev-subscribe-john=3Dhost.domain@incubator.apache.org
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>><mesos-dev-unsubscribe-john=3Dhost.domain@incubator.apache.o
>>>>>>>>>>>>>>>rg
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>>>> address.
>>>>>>>>>>>>>>> When
>>>>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>>>> subscription.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my
>>>>>>>>>>>>>>> owner
>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>>>> -0000
>>>>>>>>>>>>>>> Received: from athena.apache.org (HELO athena.apache.org)
>>>>>>>>>>>>>>> (140.211.11.136)
>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>>>>>>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>>>>> Received-SPF: pass (athena.apache.org: domain of
>>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>>>> mx0a-00082601.pphosted.com)
>>>>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>>> 22:43:19
>>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>>>>>>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>>>>> p7LMcbSq017336
>>>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21
>>>>>>>>>>>>>>>Aug
>>>>>>>>>>>>>>> 2011
>>>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>>>> Received: from mail.thefacebook.com
>>>>>>>>>>>>>>>(corpout1.snc1.tfbnw.net
>>>>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>>>>>>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128
>>>>>>>>>>>>>>> verify=3DNOT)
>>>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21
>>>>>>>>>>>>>>>Aug
>>>>>>>>>>>>>>> 2011
>>>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>>>>> Sun,
>>>>>>>>>>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>> Subject: can i join this
>>>>>>>>>>>>>>> Thread-Topic: can i join this
>>>>>>>>>>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>>>>>>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>>>>> X-MS-Has-Attach:=20
>>>>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>>>>> Content-Type: multipart/alternative;
>>>>>>>>>>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>>>>>>>>>>> Content-ID: <F6...@fb.com>
>>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> <html>
>>>>>>>>>>>>>>> <head>
>>>>>>>>>>>>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>>>>>>>>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> </head>
>>>>>>>>>>>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode:
>>>>>>>>>>>>>>> space;
>>>>>>>>>>>>>>> -webkit-lin=3D
>>>>>>>>>>>>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size:
>>>>>>>>>>>>>>> 14px;
>>>>>>>>>>>>>>> font-fami=3D
>>>>>>>>>>>>>>> ly: Calibri, sans-serif; ">
>>>>>>>>>>>>>>> <div>Best,</div>
>>>>>>>>>>>>>>> <div>Joe</div>
>>>>>>>>>>>>>>> </body>
>>>>>>>>>>>>>>> </html>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: About spark-shell in mesos cluster mode Re: WELCOME to mesos-dev@incubator.apache.org
Posted by Matei Zaharia <ma...@eecs.berkeley.edu>.
Weird, did you subscribe to the spark-users mailing list? I don't think you can post to it without doing that. Go to https://lists.eecs.berkeley.edu/sympa/subscribe/spark-users.
Anyway, to run the shell on a cluster, you need to set the MASTER environment variable to the Mesos master URL you want to use (similar to the one you pass to the standalone programs). So for example, type
MASTER=1@localhost:5050 ./spark-shell
Matei
On Aug 23, 2011, at 4:39 PM, Joe Xie wrote:
> Hi there,
>
> may I ask how can I run spark-shell in cluster mode? The web wiki say
> change the configuration of master, but I did not find detailed info yet.
> It seems my email failed at spark-user maillist.
> Thanks!
> Joe
>
>
>
> On 8/22/11 4:42 PM, "Joe Xie" <zh...@fb.com> wrote:
>
>> Thank you Matei! It's working now. I checked the stderr, it seems I have
>> to specify the SCALA_HOME even if it's in .bashrc.
>> Best,
>> Joe
>>
>> On 8/22/11 4:13 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>
>>> This probably means that your task crashed. Take a look at the logs Mesos
>>> collects for your task, which will be in
>>> MESOS_HOME/work/slave-id/framework-id/attempt-id/stdout and stderr.
>>>
>>> Sometimes this happens because spark-env.sh isn't configured on all the
>>> nodes to contain SCALA_HOME, JAVA_HOME and MESOS_HOME. It can also happen
>>> because your code is not available on all the machines and you get a
>>> ClassNotFoundError. You can either copy the code to the same place
>>> everywhere and put that in the SPARK_CLASSPATH variable in spark-env.sh,
>>> or package your code into a JAR and use the "jars" parameter to
>>> SparkContext to pass it (in which case Spark will copy it to all the
>>> nodes).
>>>
>>> Matei
>>>
>>> On Aug 22, 2011, at 4:04 PM, Joe Xie wrote:
>>>
>>>> Btw, I checkout my webui for master, it indicates "No offers are
>>>> active"
>>>> under "Resource Offers" while I have two machines as slave, so there
>>>> might
>>>> be problem with my configuration.
>>>> Best,
>>>> JOE
>>>>
>>>> On 8/22/11 3:59 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>
>>>>> Thank you Matei,
>>>>>
>>>>> it's running for me with Spark 0.3 + pre-protobuf, and I'm testing
>>>>> with
>>>>> provided example "run spark.examples.SparkPi
>>>>> mesos://1@10.30.95.184:5050
>>>>> 3". However, it seems I keep losing TID after starting the task, and
>>>>> then
>>>>> job aborted with 4 tries. It might be the problem on my side too, did
>>>>> this
>>>>> happen before? I could go for trying masters for spark + mesos.
>>>>> 11/08/22 15:48:27 INFO spark.SimpleJob: Lost TID 13 (task 0:2)
>>>>> 11/08/22 15:48:27 ERROR spark.SimpleJob: Task 0:2 failed more than 4
>>>>> times; aborting job
>>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from TID
>>>>> 12
>>>>> because its job is gone
>>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from TID
>>>>> 14
>>>>> because its job is gone
>>>>>
>>>>> Thanks!
>>>>> Joe
>>>>>
>>>>>
>>>>>
>>>>> On 8/22/11 3:38 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>
>>>>>> Thank you Matei! I can go for Spark 0.3 + pre-protobuf. I understand
>>>>>> there
>>>>>> might be lots of new updates on the master branch. Is there any
>>>>>> significant difference in current master with Spark 0.3 +
>>>>>> pre-protobuf? I
>>>>>> could also try to keep a separate branch with stable masters if
>>>>>> that's
>>>>>> the
>>>>>> case.
>>>>>> Best,
>>>>>> Joe
>>>>>> On 8/22/11 3:21 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>>>>
>>>>>>> If you want the most stable versions right now, I recommend Spark
>>>>>>> 0.3
>>>>>>> (download it from the website or the 0.3-scala-2.9 tag in Git) and
>>>>>>> pre-protobuf Mesos. However, the current master Spark + master Mesos
>>>>>>> should also work OK; it's just that further commits may break things
>>>>>>> temporarily.
>>>>>>>
>>>>>>> Matei
>>>>>>>
>>>>>>> On Aug 22, 2011, at 3:17 PM, Joe Xie wrote:
>>>>>>>
>>>>>>>> Hi Matei,
>>>>>>>> Thanks for your feedback. I bet I'm using the master brach of Spark
>>>>>>>> and
>>>>>>>> pre-protobuf version of Mesos. Do you mean I should go for Spark
>>>>>>>> (master)
>>>>>>>> + protobuf (master). Let's me know the most stable combination of
>>>>>>>> braches
>>>>>>>> from Spark and pre-protobuf, I can try them out.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Joe
>>>>>>>>
>>>>>>>>
>>>>>>>> On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Are you using the master branch of Spark, or one of the releases
>>>>>>>>> (e.g.
>>>>>>>>> 0.3)? The releases work with the pre-protobuf version of Mesos but
>>>>>>>>> the
>>>>>>>>> master branch of Spark actually requires the master branch of
>>>>>>>>> Mesos
>>>>>>>>> now.
>>>>>>>>> That could cause the kind of error you're seeing.
>>>>>>>>>
>>>>>>>>> Matei
>>>>>>>>>
>>>>>>>>> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>>>>>>>>>
>>>>>>>>>> Sorry this should have been sent to user mail list.
>>>>>>>>>>
>>>>>>>>>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi there:
>>>>>>>>>>>
>>>>>>>>>>> I has build and installed mesos (pre-protobuf branch for spark)
>>>>>>>>>>> and
>>>>>>>>>>> spark,
>>>>>>>>>>> and deployed them to a cluster. However, I have following error
>>>>>>>>>>> when
>>>>>>>>>>> I
>>>>>>>>>>> run
>>>>>>>>>>> spark task on the mesos URL of my master, the spark task runs
>>>>>>>>>>> fine
>>>>>>>>>>> in
>>>>>>>>>>> local mode. Any suggestion or help? Thanks a lot!
>>>>>>>>>>> Joe
>>>>>>>>>>>
>>>>>>>>>>> My os: CentOS 5.2
>>>>>>>>>>> Python: 2.6
>>>>>>>>>>> Gcc: 4.2.1
>>>>>>>>>>> Swig:1.3.40
>>>>>>>>>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor
>>>>>>>>>>> on
>>>>>>>>>>> port
>>>>>>>>>>> 50501
>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered
>>>>>>>>>>> actor
>>>>>>>>>>> on
>>>>>>>>>>> port 50501
>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0
>>>>>>>>>>> with
>>>>>>>>>>> cache
>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD
>>>>>>>>>>> 0
>>>>>>>>>>> with
>>>>>>>>>>> 1
>>>>>>>>>>> partitions
>>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for
>>>>>>>>>>> current
>>>>>>>>>>> cache
>>>>>>>>>>> locations
>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage
>>>>>>>>>>> 0
>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final
>>>>>>>>>>> stage:
>>>>>>>>>>> List()
>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents:
>>>>>>>>>>> List()
>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0,
>>>>>>>>>>> which
>>>>>>>>>>> has
>>>>>>>>>>> no missing parents
>>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1
>>>>>>>>>>> tasks
>>>>>>>>>>> Exception in thread "Spark scheduler"
>>>>>>>>>>> java.lang.UnsatisfiedLinkError:
>>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>>>>>>>>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native
>>>>>>>>>>> Method)
>>>>>>>>>>> at
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver
>>>>>>>>>>> .
>>>>>>>>>>> ja
>>>>>>>>>>> v
>>>>>>>>>>> a
>>>>>>>>>>> :4
>>>>>>>>>>> 2)
>>>>>>>>>>> at
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver
>>>>>>>>>>> .
>>>>>>>>>>> ja
>>>>>>>>>>> v
>>>>>>>>>>> a
>>>>>>>>>>> :4
>>>>>>>>>>> 6)
>>>>>>>>>>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>>>>>>>>>
>>>>>>>>>>> Sent to wrong address previously
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Acknowledgment: I have added the address
>>>>>>>>>>>>>
>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>
>>>>>>>>>>>>> to the mesos-dev mailing list.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please save this message so that you know the address you are
>>>>>>>>>>>>> subscribed under, in case you later want to unsubscribe or
>>>>>>>>>>>>> change
>>>>>>>>>>>>> your
>>>>>>>>>>>>> subscription address.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>>
>>>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>>
>>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>> request),
>>>>>>>>>>>>> mail:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> To get an index with subject and author for messages 123-456 ,
>>>>>>>>>>>>> mail:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>>
>>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>> 12345,
>>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>>>>>> being
>>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>>
>>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>>>> address (with '=' instead of '@') after the command word:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>> address.
>>>>>>>>>>>>> When
>>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>> subscription.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my
>>>>>>>>>>>>> owner
>>>>>>>>>>>>> is
>>>>>>>>>>>>> a
>>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>>
>>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>> 23:30:07
>>>>>>>>>>>>> -0000
>>>>>>>>>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>>>>>>>>> (192.87.106.230)
>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>> 23:30:07
>>>>>>>>>>>>> +0000
>>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>>>>>>>>>> tests=SPF_PASS
>>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>>> Received-SPF: pass (nike.apache.org: domain of zhiyong@fb.com
>>>>>>>>>>>>> designates
>>>>>>>>>>>>> 67.231.145.42 as permitted sender)
>>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>> mx0a-00082601.pphosted.com)
>>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>> 23:30:00
>>>>>>>>>>>>> +0000
>>>>>>>>>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>>>>>>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>>> p7LNTI6g015638
>>>>>>>>>>>>> for
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@in
>>>>>>>>>>>>> c
>>>>>>>>>>>>> ub
>>>>>>>>>>>>> a
>>>>>>>>>>>>> t
>>>>>>>>>>>>> or
>>>>>>>>>>>>> .a
>>>>>>>>>>>>> p
>>>>>>>>>>>>> a
>>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>>>>>>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>>>>>>>>>>>>> for
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@in
>>>>>>>>>>>>> c
>>>>>>>>>>>>> ub
>>>>>>>>>>>>> a
>>>>>>>>>>>>> t
>>>>>>>>>>>>> or
>>>>>>>>>>>>> .a
>>>>>>>>>>>>> p
>>>>>>>>>>>>> a
>>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>>> by
>>>>>>>>>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>>> Sun,
>>>>>>>>>>>>> 21 Aug 2011 16:29:14 -0700
>>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>>> To:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> "mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@in
>>>>>>>>>>>>> c
>>>>>>>>>>>>> ub
>>>>>>>>>>>>> a
>>>>>>>>>>>>> t
>>>>>>>>>>>>> or
>>>>>>>>>>>>> .a
>>>>>>>>>>>>> p
>>>>>>>>>>>>> a
>>>>>>>>>>>>> che.org"
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@i
>>>>>>>>>>>>> n
>>>>>>>>>>>>> cu
>>>>>>>>>>>>> b
>>>>>>>>>>>>> a
>>>>>>>>>>>>> to
>>>>>>>>>>>>> r.
>>>>>>>>>>>>> a
>>>>>>>>>>>>> p
>>>>>>>>>>>>> ache.org>
>>>>>>>>>>>>> Subject: Re: confirm subscribe to
>>>>>>>>>>>>> mesos-dev@incubator.apache.org
>>>>>>>>>>>>> Thread-Topic: confirm subscribe to
>>>>>>>>>>>>> mesos-dev@incubator.apache.org
>>>>>>>>>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>>>>>>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>>> In-Reply-To: <13...@incubator.apache.org>
>>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>>> X-MS-Has-Attach:
>>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>>>>>>>>> Content-ID: <B4...@fb.com>
>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>> Joe
>>>>>>>>>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To confirm that you would like
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> added to the mesos-dev mailing list, please send
>>>>>>>>>>>>>> a short reply to this address:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> =20
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@
>>>>>>>>>>>>>> i
>>>>>>>>>>>>>> nc
>>>>>>>>>>>>>> u
>>>>>>>>>>>>>> b
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> or
>>>>>>>>>>>>>> .
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> p=
>>>>>>>>>>>>> ac
>>>>>>>>>>>>>> he.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Usually, this happens when you just hit the "reply" button.
>>>>>>>>>>>>>> If this does not work, simply copy the address and paste it
>>>>>>>>>>>>>> into
>>>>>>>>>>>>>> the "To:" field of a new message.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> or click here:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3
>>>>>>>>>>>>>> D
>>>>>>>>>>>>>> fb
>>>>>>>>>>>>>> .
>>>>>>>>>>>>>> c
>>>>>>>>>>>>>> om
>>>>>>>>>>>>>> @i
>>>>>>>>>>>>>> n
>>>>>>>>>>>>>> c
>>>>>>>>>>>>>> u=
>>>>>>>>>>>>> ba
>>>>>>>>>>>>>> tor.apache.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This confirmation serves two purposes. First, it verifies
>>>>>>>>>>>>>> that I
>>>>>>>>>>>>>> am
>>>>>>>>>>>>>> able
>>>>>>>>>>>>>> to get mail through to you. Second, it protects you in case
>>>>>>>>>>>>>> someone
>>>>>>>>>>>>>> forges a subscription request in your name.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please note that ALL Apache dev- and user- mailing lists are
>>>>>>>>>>>>>> publicly
>>>>>>>>>>>>>> archived. Do familiarize yourself with Apache's public
>>>>>>>>>>>>>> archive
>>>>>>>>>>>>>> policy
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> prior to subscribing and posting messages to
>>>>>>>>>>>>>> mesos-dev@incubator.apache.org.
>>>>>>>>>>>>>> If you're not sure whether or not the policy applies to this
>>>>>>>>>>>>>> mailing
>>>>>>>>>>>>>> list,
>>>>>>>>>>>>>> assume it does unless the list name contains the word
>>>>>>>>>>>>>> "private"
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>> it.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Some mail programs are broken and cannot handle long
>>>>>>>>>>>>>> addresses.
>>>>>>>>>>>>>> If
>>>>>>>>>>>>>> you
>>>>>>>>>>>>>> cannot reply to this request, instead send a message to
>>>>>>>>>>>>>> <me...@incubator.apache.org> and put the
>>>>>>>>>>>>>> entire address listed above into the "Subject:" line.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>>> request),
>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To get an index with subject and author for messages 123-456
>>>>>>>>>>>>>> ,
>>>>>>>>>>>>>> mail:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per
>>>>>>>>>>>>>> request,
>>>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>>> 12345,
>>>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>>>>>>> being
>>>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>>>>> address (with '=3D' instead of '@') after the command word:
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <mesos-dev-unsubscribe-john=3Dhost.domain@incubator.apache.org
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>>> address.
>>>>>>>>>>>>>> When
>>>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>>> subscription.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my
>>>>>>>>>>>>>> owner
>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>>> -0000
>>>>>>>>>>>>>> Received: from athena.apache.org (HELO athena.apache.org)
>>>>>>>>>>>>>> (140.211.11.136)
>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>>>>>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>>>> Received-SPF: pass (athena.apache.org: domain of
>>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>>> mx0a-00082601.pphosted.com)
>>>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>>> 22:43:19
>>>>>>>>>>>>>> +0000
>>>>>>>>>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>>>>>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>>>> p7LMcbSq017336
>>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug
>>>>>>>>>>>>>> 2011
>>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>>>>>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128
>>>>>>>>>>>>>> verify=3DNOT)
>>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug
>>>>>>>>>>>>>> 2011
>>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>>>> by
>>>>>>>>>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>>>> Sun,
>>>>>>>>>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>> Subject: can i join this
>>>>>>>>>>>>>> Thread-Topic: can i join this
>>>>>>>>>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>>>>>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>>>> X-MS-Has-Attach:=20
>>>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>>>> Content-Type: multipart/alternative;
>>>>>>>>>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>>>>>>>>>> Content-ID: <F6...@fb.com>
>>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <html>
>>>>>>>>>>>>>> <head>
>>>>>>>>>>>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>>>>>>>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> </head>
>>>>>>>>>>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode:
>>>>>>>>>>>>>> space;
>>>>>>>>>>>>>> -webkit-lin=3D
>>>>>>>>>>>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size:
>>>>>>>>>>>>>> 14px;
>>>>>>>>>>>>>> font-fami=3D
>>>>>>>>>>>>>> ly: Calibri, sans-serif; ">
>>>>>>>>>>>>>> <div>Best,</div>
>>>>>>>>>>>>>> <div>Joe</div>
>>>>>>>>>>>>>> </body>
>>>>>>>>>>>>>> </html>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
About spark-shell in mesos cluster mode Re: WELCOME to
mesos-dev@incubator.apache.org
Posted by Joe Xie <zh...@fb.com>.
Hi there,
may I ask how can I run spark-shell in cluster mode? The web wiki say
change the configuration of master, but I did not find detailed info yet.
It seems my email failed at spark-user maillist.
Thanks!
Joe
On 8/22/11 4:42 PM, "Joe Xie" <zh...@fb.com> wrote:
>Thank you Matei! It's working now. I checked the stderr, it seems I have
>to specify the SCALA_HOME even if it's in .bashrc.
>Best,
>Joe
>
>On 8/22/11 4:13 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>
>>This probably means that your task crashed. Take a look at the logs Mesos
>>collects for your task, which will be in
>>MESOS_HOME/work/slave-id/framework-id/attempt-id/stdout and stderr.
>>
>>Sometimes this happens because spark-env.sh isn't configured on all the
>>nodes to contain SCALA_HOME, JAVA_HOME and MESOS_HOME. It can also happen
>>because your code is not available on all the machines and you get a
>>ClassNotFoundError. You can either copy the code to the same place
>>everywhere and put that in the SPARK_CLASSPATH variable in spark-env.sh,
>>or package your code into a JAR and use the "jars" parameter to
>>SparkContext to pass it (in which case Spark will copy it to all the
>>nodes).
>>
>>Matei
>>
>>On Aug 22, 2011, at 4:04 PM, Joe Xie wrote:
>>
>>> Btw, I checkout my webui for master, it indicates "No offers are
>>>active"
>>> under "Resource Offers" while I have two machines as slave, so there
>>>might
>>> be problem with my configuration.
>>> Best,
>>> JOE
>>>
>>> On 8/22/11 3:59 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>
>>>> Thank you Matei,
>>>>
>>>> it's running for me with Spark 0.3 + pre-protobuf, and I'm testing
>>>>with
>>>> provided example "run spark.examples.SparkPi
>>>>mesos://1@10.30.95.184:5050
>>>> 3". However, it seems I keep losing TID after starting the task, and
>>>>then
>>>> job aborted with 4 tries. It might be the problem on my side too, did
>>>>this
>>>> happen before? I could go for trying masters for spark + mesos.
>>>> 11/08/22 15:48:27 INFO spark.SimpleJob: Lost TID 13 (task 0:2)
>>>> 11/08/22 15:48:27 ERROR spark.SimpleJob: Task 0:2 failed more than 4
>>>> times; aborting job
>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from TID
>>>>12
>>>> because its job is gone
>>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from TID
>>>>14
>>>> because its job is gone
>>>>
>>>> Thanks!
>>>> Joe
>>>>
>>>>
>>>>
>>>> On 8/22/11 3:38 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>
>>>>> Thank you Matei! I can go for Spark 0.3 + pre-protobuf. I understand
>>>>> there
>>>>> might be lots of new updates on the master branch. Is there any
>>>>> significant difference in current master with Spark 0.3 +
>>>>>pre-protobuf? I
>>>>> could also try to keep a separate branch with stable masters if
>>>>>that's
>>>>> the
>>>>> case.
>>>>> Best,
>>>>> Joe
>>>>> On 8/22/11 3:21 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>>>
>>>>>> If you want the most stable versions right now, I recommend Spark
>>>>>>0.3
>>>>>> (download it from the website or the 0.3-scala-2.9 tag in Git) and
>>>>>> pre-protobuf Mesos. However, the current master Spark + master Mesos
>>>>>> should also work OK; it's just that further commits may break things
>>>>>> temporarily.
>>>>>>
>>>>>> Matei
>>>>>>
>>>>>> On Aug 22, 2011, at 3:17 PM, Joe Xie wrote:
>>>>>>
>>>>>>> Hi Matei,
>>>>>>> Thanks for your feedback. I bet I'm using the master brach of Spark
>>>>>>> and
>>>>>>> pre-protobuf version of Mesos. Do you mean I should go for Spark
>>>>>>> (master)
>>>>>>> + protobuf (master). Let's me know the most stable combination of
>>>>>>> braches
>>>>>>> from Spark and pre-protobuf, I can try them out.
>>>>>>>
>>>>>>> Best,
>>>>>>> Joe
>>>>>>>
>>>>>>>
>>>>>>> On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu>
>>>>>>>wrote:
>>>>>>>
>>>>>>>> Are you using the master branch of Spark, or one of the releases
>>>>>>>> (e.g.
>>>>>>>> 0.3)? The releases work with the pre-protobuf version of Mesos but
>>>>>>>> the
>>>>>>>> master branch of Spark actually requires the master branch of
>>>>>>>>Mesos
>>>>>>>> now.
>>>>>>>> That could cause the kind of error you're seeing.
>>>>>>>>
>>>>>>>> Matei
>>>>>>>>
>>>>>>>> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>>>>>>>>
>>>>>>>>> Sorry this should have been sent to user mail list.
>>>>>>>>>
>>>>>>>>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi there:
>>>>>>>>>>
>>>>>>>>>> I has build and installed mesos (pre-protobuf branch for spark)
>>>>>>>>>>and
>>>>>>>>>> spark,
>>>>>>>>>> and deployed them to a cluster. However, I have following error
>>>>>>>>>> when
>>>>>>>>>> I
>>>>>>>>>> run
>>>>>>>>>> spark task on the mesos URL of my master, the spark task runs
>>>>>>>>>>fine
>>>>>>>>>> in
>>>>>>>>>> local mode. Any suggestion or help? Thanks a lot!
>>>>>>>>>> Joe
>>>>>>>>>>
>>>>>>>>>> My os: CentOS 5.2
>>>>>>>>>> Python: 2.6
>>>>>>>>>> Gcc: 4.2.1
>>>>>>>>>> Swig:1.3.40
>>>>>>>>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor
>>>>>>>>>>on
>>>>>>>>>> port
>>>>>>>>>> 50501
>>>>>>>>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered
>>>>>>>>>> actor
>>>>>>>>>> on
>>>>>>>>>> port 50501
>>>>>>>>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0
>>>>>>>>>> with
>>>>>>>>>> cache
>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD
>>>>>>>>>>0
>>>>>>>>>> with
>>>>>>>>>> 1
>>>>>>>>>> partitions
>>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for
>>>>>>>>>>current
>>>>>>>>>> cache
>>>>>>>>>> locations
>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage
>>>>>>>>>>0
>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final
>>>>>>>>>> stage:
>>>>>>>>>> List()
>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents:
>>>>>>>>>> List()
>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0,
>>>>>>>>>> which
>>>>>>>>>> has
>>>>>>>>>> no missing parents
>>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1
>>>>>>>>>>tasks
>>>>>>>>>> Exception in thread "Spark scheduler"
>>>>>>>>>> java.lang.UnsatisfiedLinkError:
>>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>>>>>>>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native
>>>>>>>>>>Method)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver
>>>>>>>>>>.
>>>>>>>>>>ja
>>>>>>>>>> v
>>>>>>>>>> a
>>>>>>>>>> :4
>>>>>>>>>> 2)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver
>>>>>>>>>>.
>>>>>>>>>>ja
>>>>>>>>>> v
>>>>>>>>>> a
>>>>>>>>>> :4
>>>>>>>>>> 6)
>>>>>>>>>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>>>>>>>>
>>>>>>>>>> Sent to wrong address previously
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>
>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>
>>>>>>>>>>>> Acknowledgment: I have added the address
>>>>>>>>>>>>
>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>
>>>>>>>>>>>> to the mesos-dev mailing list.
>>>>>>>>>>>>
>>>>>>>>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>>>>>>>>
>>>>>>>>>>>> Please save this message so that you know the address you are
>>>>>>>>>>>> subscribed under, in case you later want to unsubscribe or
>>>>>>>>>>>>change
>>>>>>>>>>>> your
>>>>>>>>>>>> subscription address.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>
>>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>
>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>request),
>>>>>>>>>>>> mail:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> To get an index with subject and author for messages 123-456 ,
>>>>>>>>>>>> mail:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>
>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>12345,
>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>>>>> being
>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>
>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>>> address (with '=' instead of '@') after the command word:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>address.
>>>>>>>>>>>> When
>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>subscription.
>>>>>>>>>>>>
>>>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my
>>>>>>>>>>>>owner
>>>>>>>>>>>> is
>>>>>>>>>>>> a
>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>
>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>
>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>23:30:07
>>>>>>>>>>>> -0000
>>>>>>>>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>>>>>>>> (192.87.106.230)
>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>> 23:30:07
>>>>>>>>>>>> +0000
>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>>>>>>>>> tests=SPF_PASS
>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>> Received-SPF: pass (nike.apache.org: domain of zhiyong@fb.com
>>>>>>>>>>>> designates
>>>>>>>>>>>> 67.231.145.42 as permitted sender)
>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>mx0a-00082601.pphosted.com)
>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>> 23:30:00
>>>>>>>>>>>> +0000
>>>>>>>>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>>>>>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>> p7LNTI6g015638
>>>>>>>>>>>> for
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@in
>>>>>>>>>>>>c
>>>>>>>>>>>>ub
>>>>>>>>>>>> a
>>>>>>>>>>>> t
>>>>>>>>>>>> or
>>>>>>>>>>>> .a
>>>>>>>>>>>> p
>>>>>>>>>>>> a
>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>>>>>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>>>>>>>>>>>> for
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@in
>>>>>>>>>>>>c
>>>>>>>>>>>>ub
>>>>>>>>>>>> a
>>>>>>>>>>>> t
>>>>>>>>>>>> or
>>>>>>>>>>>> .a
>>>>>>>>>>>> p
>>>>>>>>>>>> a
>>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>> by
>>>>>>>>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>> Sun,
>>>>>>>>>>>> 21 Aug 2011 16:29:14 -0700
>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>> To:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>"mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@in
>>>>>>>>>>>>c
>>>>>>>>>>>>ub
>>>>>>>>>>>> a
>>>>>>>>>>>> t
>>>>>>>>>>>> or
>>>>>>>>>>>> .a
>>>>>>>>>>>> p
>>>>>>>>>>>> a
>>>>>>>>>>>> che.org"
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@i
>>>>>>>>>>>>n
>>>>>>>>>>>>cu
>>>>>>>>>>>> b
>>>>>>>>>>>> a
>>>>>>>>>>>> to
>>>>>>>>>>>> r.
>>>>>>>>>>>> a
>>>>>>>>>>>> p
>>>>>>>>>>>> ache.org>
>>>>>>>>>>>> Subject: Re: confirm subscribe to
>>>>>>>>>>>>mesos-dev@incubator.apache.org
>>>>>>>>>>>> Thread-Topic: confirm subscribe to
>>>>>>>>>>>>mesos-dev@incubator.apache.org
>>>>>>>>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>>>>>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>> In-Reply-To: <13...@incubator.apache.org>
>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>> X-MS-Has-Attach:
>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>>>>>>>> Content-ID: <B4...@fb.com>
>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>> Joe
>>>>>>>>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>>
>>>>>>>>>>>>> To confirm that you would like
>>>>>>>>>>>>>
>>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>>
>>>>>>>>>>>>> added to the mesos-dev mailing list, please send
>>>>>>>>>>>>> a short reply to this address:
>>>>>>>>>>>>>
>>>>>>>>>>>>> =20
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@
>>>>>>>>>>>>>i
>>>>>>>>>>>>>nc
>>>>>>>>>>>>> u
>>>>>>>>>>>>> b
>>>>>>>>>>>>> at
>>>>>>>>>>>>> or
>>>>>>>>>>>>> .
>>>>>>>>>>>>> a
>>>>>>>>>>>>> p=
>>>>>>>>>>>> ac
>>>>>>>>>>>>> he.org
>>>>>>>>>>>>>
>>>>>>>>>>>>> Usually, this happens when you just hit the "reply" button.
>>>>>>>>>>>>> If this does not work, simply copy the address and paste it
>>>>>>>>>>>>>into
>>>>>>>>>>>>> the "To:" field of a new message.
>>>>>>>>>>>>>
>>>>>>>>>>>>> or click here:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3
>>>>>>>>>>>>>D
>>>>>>>>>>>>>fb
>>>>>>>>>>>>> .
>>>>>>>>>>>>> c
>>>>>>>>>>>>> om
>>>>>>>>>>>>> @i
>>>>>>>>>>>>> n
>>>>>>>>>>>>> c
>>>>>>>>>>>>> u=
>>>>>>>>>>>> ba
>>>>>>>>>>>>> tor.apache.org
>>>>>>>>>>>>>
>>>>>>>>>>>>> This confirmation serves two purposes. First, it verifies
>>>>>>>>>>>>>that I
>>>>>>>>>>>>> am
>>>>>>>>>>>>> able
>>>>>>>>>>>>> to get mail through to you. Second, it protects you in case
>>>>>>>>>>>>> someone
>>>>>>>>>>>>> forges a subscription request in your name.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please note that ALL Apache dev- and user- mailing lists are
>>>>>>>>>>>>> publicly
>>>>>>>>>>>>> archived. Do familiarize yourself with Apache's public
>>>>>>>>>>>>>archive
>>>>>>>>>>>>> policy
>>>>>>>>>>>>> at
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>>>>>>>>
>>>>>>>>>>>>> prior to subscribing and posting messages to
>>>>>>>>>>>>> mesos-dev@incubator.apache.org.
>>>>>>>>>>>>> If you're not sure whether or not the policy applies to this
>>>>>>>>>>>>> mailing
>>>>>>>>>>>>> list,
>>>>>>>>>>>>> assume it does unless the list name contains the word
>>>>>>>>>>>>>"private"
>>>>>>>>>>>>> in
>>>>>>>>>>>>> it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Some mail programs are broken and cannot handle long
>>>>>>>>>>>>>addresses.
>>>>>>>>>>>>> If
>>>>>>>>>>>>> you
>>>>>>>>>>>>> cannot reply to this request, instead send a message to
>>>>>>>>>>>>> <me...@incubator.apache.org> and put the
>>>>>>>>>>>>> entire address listed above into the "Subject:" line.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>>
>>>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>>
>>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>>request),
>>>>>>>>>>>>> mail:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> To get an index with subject and author for messages 123-456
>>>>>>>>>>>>>,
>>>>>>>>>>>>> mail:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per
>>>>>>>>>>>>>request,
>>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>>
>>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>>12345,
>>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>>>>>> being
>>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>>
>>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>>>> address (with '=3D' instead of '@') after the command word:
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>>
>>>>>>>>>>>>><mesos-dev-unsubscribe-john=3Dhost.domain@incubator.apache.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>>address.
>>>>>>>>>>>>> When
>>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>>> subscription.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my
>>>>>>>>>>>>> owner
>>>>>>>>>>>>> is a
>>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>>
>>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>>22:43:26
>>>>>>>>>>>>> -0000
>>>>>>>>>>>>> Received: from athena.apache.org (HELO athena.apache.org)
>>>>>>>>>>>>> (140.211.11.136)
>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>>> +0000
>>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>>>>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>>> Received-SPF: pass (athena.apache.org: domain of
>>>>>>>>>>>>>zhiyong@fb.com
>>>>>>>>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>>mx0a-00082601.pphosted.com)
>>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>>> 22:43:19
>>>>>>>>>>>>> +0000
>>>>>>>>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>>>>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>>> p7LMcbSq017336
>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug
>>>>>>>>>>>>> 2011
>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>>>>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128
>>>>>>>>>>>>> verify=3DNOT)
>>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug
>>>>>>>>>>>>> 2011
>>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>>> by
>>>>>>>>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>>> Sun,
>>>>>>>>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>> Subject: can i join this
>>>>>>>>>>>>> Thread-Topic: can i join this
>>>>>>>>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>>>>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>>> X-MS-Has-Attach:=20
>>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>>> Content-Type: multipart/alternative;
>>>>>>>>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>>
>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>
>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>>>>>>>>> Content-ID: <F6...@fb.com>
>>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>>
>>>>>>>>>>>>> <html>
>>>>>>>>>>>>> <head>
>>>>>>>>>>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>>>>>>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>>>>>>>>
>>>>>>>>>>>>> </head>
>>>>>>>>>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode:
>>>>>>>>>>>>> space;
>>>>>>>>>>>>> -webkit-lin=3D
>>>>>>>>>>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size:
>>>>>>>>>>>>> 14px;
>>>>>>>>>>>>> font-fami=3D
>>>>>>>>>>>>> ly: Calibri, sans-serif; ">
>>>>>>>>>>>>> <div>Best,</div>
>>>>>>>>>>>>> <div>Joe</div>
>>>>>>>>>>>>> </body>
>>>>>>>>>>>>> </html>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: About Running Spark on a cluster with mesos Re: WELCOME to
mesos-dev@incubator.apache.org
Posted by Joe Xie <zh...@fb.com>.
Thank you Matei! It's working now. I checked the stderr, it seems I have
to specify the SCALA_HOME even if it's in .bashrc.
Best,
Joe
On 8/22/11 4:13 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>This probably means that your task crashed. Take a look at the logs Mesos
>collects for your task, which will be in
>MESOS_HOME/work/slave-id/framework-id/attempt-id/stdout and stderr.
>
>Sometimes this happens because spark-env.sh isn't configured on all the
>nodes to contain SCALA_HOME, JAVA_HOME and MESOS_HOME. It can also happen
>because your code is not available on all the machines and you get a
>ClassNotFoundError. You can either copy the code to the same place
>everywhere and put that in the SPARK_CLASSPATH variable in spark-env.sh,
>or package your code into a JAR and use the "jars" parameter to
>SparkContext to pass it (in which case Spark will copy it to all the
>nodes).
>
>Matei
>
>On Aug 22, 2011, at 4:04 PM, Joe Xie wrote:
>
>> Btw, I checkout my webui for master, it indicates "No offers are active"
>> under "Resource Offers" while I have two machines as slave, so there
>>might
>> be problem with my configuration.
>> Best,
>> JOE
>>
>> On 8/22/11 3:59 PM, "Joe Xie" <zh...@fb.com> wrote:
>>
>>> Thank you Matei,
>>>
>>> it's running for me with Spark 0.3 + pre-protobuf, and I'm testing with
>>> provided example "run spark.examples.SparkPi
>>>mesos://1@10.30.95.184:5050
>>> 3". However, it seems I keep losing TID after starting the task, and
>>>then
>>> job aborted with 4 tries. It might be the problem on my side too, did
>>>this
>>> happen before? I could go for trying masters for spark + mesos.
>>> 11/08/22 15:48:27 INFO spark.SimpleJob: Lost TID 13 (task 0:2)
>>> 11/08/22 15:48:27 ERROR spark.SimpleJob: Task 0:2 failed more than 4
>>> times; aborting job
>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from TID
>>>12
>>> because its job is gone
>>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from TID
>>>14
>>> because its job is gone
>>>
>>> Thanks!
>>> Joe
>>>
>>>
>>>
>>> On 8/22/11 3:38 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>
>>>> Thank you Matei! I can go for Spark 0.3 + pre-protobuf. I understand
>>>> there
>>>> might be lots of new updates on the master branch. Is there any
>>>> significant difference in current master with Spark 0.3 +
>>>>pre-protobuf? I
>>>> could also try to keep a separate branch with stable masters if that's
>>>> the
>>>> case.
>>>> Best,
>>>> Joe
>>>> On 8/22/11 3:21 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>>
>>>>> If you want the most stable versions right now, I recommend Spark 0.3
>>>>> (download it from the website or the 0.3-scala-2.9 tag in Git) and
>>>>> pre-protobuf Mesos. However, the current master Spark + master Mesos
>>>>> should also work OK; it's just that further commits may break things
>>>>> temporarily.
>>>>>
>>>>> Matei
>>>>>
>>>>> On Aug 22, 2011, at 3:17 PM, Joe Xie wrote:
>>>>>
>>>>>> Hi Matei,
>>>>>> Thanks for your feedback. I bet I'm using the master brach of Spark
>>>>>> and
>>>>>> pre-protobuf version of Mesos. Do you mean I should go for Spark
>>>>>> (master)
>>>>>> + protobuf (master). Let's me know the most stable combination of
>>>>>> braches
>>>>>> from Spark and pre-protobuf, I can try them out.
>>>>>>
>>>>>> Best,
>>>>>> Joe
>>>>>>
>>>>>>
>>>>>> On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>>>>
>>>>>>> Are you using the master branch of Spark, or one of the releases
>>>>>>> (e.g.
>>>>>>> 0.3)? The releases work with the pre-protobuf version of Mesos but
>>>>>>> the
>>>>>>> master branch of Spark actually requires the master branch of Mesos
>>>>>>> now.
>>>>>>> That could cause the kind of error you're seeing.
>>>>>>>
>>>>>>> Matei
>>>>>>>
>>>>>>> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>>>>>>>
>>>>>>>> Sorry this should have been sent to user mail list.
>>>>>>>>
>>>>>>>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>>
>>>>>>>>> Hi there:
>>>>>>>>>
>>>>>>>>> I has build and installed mesos (pre-protobuf branch for spark)
>>>>>>>>>and
>>>>>>>>> spark,
>>>>>>>>> and deployed them to a cluster. However, I have following error
>>>>>>>>> when
>>>>>>>>> I
>>>>>>>>> run
>>>>>>>>> spark task on the mesos URL of my master, the spark task runs
>>>>>>>>>fine
>>>>>>>>> in
>>>>>>>>> local mode. Any suggestion or help? Thanks a lot!
>>>>>>>>> Joe
>>>>>>>>>
>>>>>>>>> My os: CentOS 5.2
>>>>>>>>> Python: 2.6
>>>>>>>>> Gcc: 4.2.1
>>>>>>>>> Swig:1.3.40
>>>>>>>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor
>>>>>>>>>on
>>>>>>>>> port
>>>>>>>>> 50501
>>>>>>>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered
>>>>>>>>> actor
>>>>>>>>> on
>>>>>>>>> port 50501
>>>>>>>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0
>>>>>>>>> with
>>>>>>>>> cache
>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0
>>>>>>>>> with
>>>>>>>>> 1
>>>>>>>>> partitions
>>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current
>>>>>>>>> cache
>>>>>>>>> locations
>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0
>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final
>>>>>>>>> stage:
>>>>>>>>> List()
>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents:
>>>>>>>>> List()
>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0,
>>>>>>>>> which
>>>>>>>>> has
>>>>>>>>> no missing parents
>>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1
>>>>>>>>>tasks
>>>>>>>>> Exception in thread "Spark scheduler"
>>>>>>>>> java.lang.UnsatisfiedLinkError:
>>>>>>>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>>>>>>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native
>>>>>>>>>Method)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.
>>>>>>>>>ja
>>>>>>>>> v
>>>>>>>>> a
>>>>>>>>> :4
>>>>>>>>> 2)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.
>>>>>>>>>ja
>>>>>>>>> v
>>>>>>>>> a
>>>>>>>>> :4
>>>>>>>>> 6)
>>>>>>>>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>>>>>>>
>>>>>>>>> Sent to wrong address previously
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>
>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>
>>>>>>>>>>> Acknowledgment: I have added the address
>>>>>>>>>>>
>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>
>>>>>>>>>>> to the mesos-dev mailing list.
>>>>>>>>>>>
>>>>>>>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>>>>>>>
>>>>>>>>>>> Please save this message so that you know the address you are
>>>>>>>>>>> subscribed under, in case you later want to unsubscribe or
>>>>>>>>>>>change
>>>>>>>>>>> your
>>>>>>>>>>> subscription address.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>
>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>
>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per request),
>>>>>>>>>>> mail:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> To get an index with subject and author for messages 123-456 ,
>>>>>>>>>>> mail:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>
>>>>>>>>>>> To receive all messages with the same subject as message 12345,
>>>>>>>>>>> send a short message to:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>>>> being
>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>
>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>> address (with '=' instead of '@') after the command word:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>address.
>>>>>>>>>>> When
>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>subscription.
>>>>>>>>>>>
>>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my
>>>>>>>>>>>owner
>>>>>>>>>>> is
>>>>>>>>>>> a
>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>
>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>
>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07
>>>>>>>>>>> -0000
>>>>>>>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>>>>>>> (192.87.106.230)
>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>> 23:30:07
>>>>>>>>>>> +0000
>>>>>>>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>>>>>>>> tests=SPF_PASS
>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>> Received-SPF: pass (nike.apache.org: domain of zhiyong@fb.com
>>>>>>>>>>> designates
>>>>>>>>>>> 67.231.145.42 as permitted sender)
>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>mx0a-00082601.pphosted.com)
>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>> 23:30:00
>>>>>>>>>>> +0000
>>>>>>>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>>>>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>> p7LNTI6g015638
>>>>>>>>>>> for
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@inc
>>>>>>>>>>>ub
>>>>>>>>>>> a
>>>>>>>>>>> t
>>>>>>>>>>> or
>>>>>>>>>>> .a
>>>>>>>>>>> p
>>>>>>>>>>> a
>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>>>>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>>>>>>>>>>> for
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@inc
>>>>>>>>>>>ub
>>>>>>>>>>> a
>>>>>>>>>>> t
>>>>>>>>>>> or
>>>>>>>>>>> .a
>>>>>>>>>>> p
>>>>>>>>>>> a
>>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>> by
>>>>>>>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>> Sun,
>>>>>>>>>>> 21 Aug 2011 16:29:14 -0700
>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>> To:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>"mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@inc
>>>>>>>>>>>ub
>>>>>>>>>>> a
>>>>>>>>>>> t
>>>>>>>>>>> or
>>>>>>>>>>> .a
>>>>>>>>>>> p
>>>>>>>>>>> a
>>>>>>>>>>> che.org"
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@in
>>>>>>>>>>>cu
>>>>>>>>>>> b
>>>>>>>>>>> a
>>>>>>>>>>> to
>>>>>>>>>>> r.
>>>>>>>>>>> a
>>>>>>>>>>> p
>>>>>>>>>>> ache.org>
>>>>>>>>>>> Subject: Re: confirm subscribe to
>>>>>>>>>>>mesos-dev@incubator.apache.org
>>>>>>>>>>> Thread-Topic: confirm subscribe to
>>>>>>>>>>>mesos-dev@incubator.apache.org
>>>>>>>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>>>>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>> In-Reply-To: <13...@incubator.apache.org>
>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>> X-MS-Has-Attach:
>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>>>>>>> Content-ID: <B4...@fb.com>
>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>>>>>>>
>>>>>>>>>>> Thanks!
>>>>>>>>>>> Joe
>>>>>>>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>>
>>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>>
>>>>>>>>>>>> To confirm that you would like
>>>>>>>>>>>>
>>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>>
>>>>>>>>>>>> added to the mesos-dev mailing list, please send
>>>>>>>>>>>> a short reply to this address:
>>>>>>>>>>>>
>>>>>>>>>>>> =20
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@i
>>>>>>>>>>>>nc
>>>>>>>>>>>> u
>>>>>>>>>>>> b
>>>>>>>>>>>> at
>>>>>>>>>>>> or
>>>>>>>>>>>> .
>>>>>>>>>>>> a
>>>>>>>>>>>> p=
>>>>>>>>>>> ac
>>>>>>>>>>>> he.org
>>>>>>>>>>>>
>>>>>>>>>>>> Usually, this happens when you just hit the "reply" button.
>>>>>>>>>>>> If this does not work, simply copy the address and paste it
>>>>>>>>>>>>into
>>>>>>>>>>>> the "To:" field of a new message.
>>>>>>>>>>>>
>>>>>>>>>>>> or click here:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3D
>>>>>>>>>>>>fb
>>>>>>>>>>>> .
>>>>>>>>>>>> c
>>>>>>>>>>>> om
>>>>>>>>>>>> @i
>>>>>>>>>>>> n
>>>>>>>>>>>> c
>>>>>>>>>>>> u=
>>>>>>>>>>> ba
>>>>>>>>>>>> tor.apache.org
>>>>>>>>>>>>
>>>>>>>>>>>> This confirmation serves two purposes. First, it verifies
>>>>>>>>>>>>that I
>>>>>>>>>>>> am
>>>>>>>>>>>> able
>>>>>>>>>>>> to get mail through to you. Second, it protects you in case
>>>>>>>>>>>> someone
>>>>>>>>>>>> forges a subscription request in your name.
>>>>>>>>>>>>
>>>>>>>>>>>> Please note that ALL Apache dev- and user- mailing lists are
>>>>>>>>>>>> publicly
>>>>>>>>>>>> archived. Do familiarize yourself with Apache's public
>>>>>>>>>>>>archive
>>>>>>>>>>>> policy
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>>>>>>>
>>>>>>>>>>>> prior to subscribing and posting messages to
>>>>>>>>>>>> mesos-dev@incubator.apache.org.
>>>>>>>>>>>> If you're not sure whether or not the policy applies to this
>>>>>>>>>>>> mailing
>>>>>>>>>>>> list,
>>>>>>>>>>>> assume it does unless the list name contains the word
>>>>>>>>>>>>"private"
>>>>>>>>>>>> in
>>>>>>>>>>>> it.
>>>>>>>>>>>>
>>>>>>>>>>>> Some mail programs are broken and cannot handle long
>>>>>>>>>>>>addresses.
>>>>>>>>>>>> If
>>>>>>>>>>>> you
>>>>>>>>>>>> cannot reply to this request, instead send a message to
>>>>>>>>>>>> <me...@incubator.apache.org> and put the
>>>>>>>>>>>> entire address listed above into the "Subject:" line.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>>
>>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>>
>>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per
>>>>>>>>>>>>request),
>>>>>>>>>>>> mail:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> To get an index with subject and author for messages 123-456 ,
>>>>>>>>>>>> mail:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>>
>>>>>>>>>>>> To receive all messages with the same subject as message
>>>>>>>>>>>>12345,
>>>>>>>>>>>> send a short message to:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>>>>> being
>>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>>
>>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>>> address (with '=3D' instead of '@') after the command word:
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>>>
>>>>>>>>>>>><me...@incubator.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>> In both cases, I'll send a confirmation message to that
>>>>>>>>>>>>address.
>>>>>>>>>>>> When
>>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>>> subscription.
>>>>>>>>>>>>
>>>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my
>>>>>>>>>>>> owner
>>>>>>>>>>>> is a
>>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>>
>>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>>
>>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011
>>>>>>>>>>>>22:43:26
>>>>>>>>>>>> -0000
>>>>>>>>>>>> Received: from athena.apache.org (HELO athena.apache.org)
>>>>>>>>>>>> (140.211.11.136)
>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>> 22:43:26
>>>>>>>>>>>> +0000
>>>>>>>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>>>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>>> Received-SPF: pass (athena.apache.org: domain of
>>>>>>>>>>>>zhiyong@fb.com
>>>>>>>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>>>>>>>> Received: from [67.231.145.42] (HELO
>>>>>>>>>>>>mx0a-00082601.pphosted.com)
>>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>>> 22:43:19
>>>>>>>>>>>> +0000
>>>>>>>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>>>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>>> p7LMcbSq017336
>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug
>>>>>>>>>>>> 2011
>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>>>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128
>>>>>>>>>>>> verify=3DNOT)
>>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug
>>>>>>>>>>>> 2011
>>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>>> by
>>>>>>>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>>> Sun,
>>>>>>>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>> Subject: can i join this
>>>>>>>>>>>> Thread-Topic: can i join this
>>>>>>>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>>>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>>> X-MS-Has-Attach:=20
>>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>>> Content-Type: multipart/alternative;
>>>>>>>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>>
>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>
>>>>>>>>>>>> Best,
>>>>>>>>>>>> Joe
>>>>>>>>>>>>
>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>>>>>>>> Content-ID: <F6...@fb.com>
>>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>>
>>>>>>>>>>>> <html>
>>>>>>>>>>>> <head>
>>>>>>>>>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>>>>>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>>>>>>>
>>>>>>>>>>>> </head>
>>>>>>>>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode:
>>>>>>>>>>>> space;
>>>>>>>>>>>> -webkit-lin=3D
>>>>>>>>>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size:
>>>>>>>>>>>> 14px;
>>>>>>>>>>>> font-fami=3D
>>>>>>>>>>>> ly: Calibri, sans-serif; ">
>>>>>>>>>>>> <div>Best,</div>
>>>>>>>>>>>> <div>Joe</div>
>>>>>>>>>>>> </body>
>>>>>>>>>>>> </html>
>>>>>>>>>>>>
>>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: About Running Spark on a cluster with mesos Re: WELCOME to mesos-dev@incubator.apache.org
Posted by Matei Zaharia <ma...@eecs.berkeley.edu>.
This probably means that your task crashed. Take a look at the logs Mesos collects for your task, which will be in MESOS_HOME/work/slave-id/framework-id/attempt-id/stdout and stderr.
Sometimes this happens because spark-env.sh isn't configured on all the nodes to contain SCALA_HOME, JAVA_HOME and MESOS_HOME. It can also happen because your code is not available on all the machines and you get a ClassNotFoundError. You can either copy the code to the same place everywhere and put that in the SPARK_CLASSPATH variable in spark-env.sh, or package your code into a JAR and use the "jars" parameter to SparkContext to pass it (in which case Spark will copy it to all the nodes).
Matei
On Aug 22, 2011, at 4:04 PM, Joe Xie wrote:
> Btw, I checkout my webui for master, it indicates "No offers are active"
> under "Resource Offers" while I have two machines as slave, so there might
> be problem with my configuration.
> Best,
> JOE
>
> On 8/22/11 3:59 PM, "Joe Xie" <zh...@fb.com> wrote:
>
>> Thank you Matei,
>>
>> it's running for me with Spark 0.3 + pre-protobuf, and I'm testing with
>> provided example "run spark.examples.SparkPi mesos://1@10.30.95.184:5050
>> 3". However, it seems I keep losing TID after starting the task, and then
>> job aborted with 4 tries. It might be the problem on my side too, did this
>> happen before? I could go for trying masters for spark + mesos.
>> 11/08/22 15:48:27 INFO spark.SimpleJob: Lost TID 13 (task 0:2)
>> 11/08/22 15:48:27 ERROR spark.SimpleJob: Task 0:2 failed more than 4
>> times; aborting job
>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from TID 12
>> because its job is gone
>> 11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from TID 14
>> because its job is gone
>>
>> Thanks!
>> Joe
>>
>>
>>
>> On 8/22/11 3:38 PM, "Joe Xie" <zh...@fb.com> wrote:
>>
>>> Thank you Matei! I can go for Spark 0.3 + pre-protobuf. I understand
>>> there
>>> might be lots of new updates on the master branch. Is there any
>>> significant difference in current master with Spark 0.3 + pre-protobuf? I
>>> could also try to keep a separate branch with stable masters if that's
>>> the
>>> case.
>>> Best,
>>> Joe
>>> On 8/22/11 3:21 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>
>>>> If you want the most stable versions right now, I recommend Spark 0.3
>>>> (download it from the website or the 0.3-scala-2.9 tag in Git) and
>>>> pre-protobuf Mesos. However, the current master Spark + master Mesos
>>>> should also work OK; it's just that further commits may break things
>>>> temporarily.
>>>>
>>>> Matei
>>>>
>>>> On Aug 22, 2011, at 3:17 PM, Joe Xie wrote:
>>>>
>>>>> Hi Matei,
>>>>> Thanks for your feedback. I bet I'm using the master brach of Spark
>>>>> and
>>>>> pre-protobuf version of Mesos. Do you mean I should go for Spark
>>>>> (master)
>>>>> + protobuf (master). Let's me know the most stable combination of
>>>>> braches
>>>>> from Spark and pre-protobuf, I can try them out.
>>>>>
>>>>> Best,
>>>>> Joe
>>>>>
>>>>>
>>>>> On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>>>
>>>>>> Are you using the master branch of Spark, or one of the releases
>>>>>> (e.g.
>>>>>> 0.3)? The releases work with the pre-protobuf version of Mesos but
>>>>>> the
>>>>>> master branch of Spark actually requires the master branch of Mesos
>>>>>> now.
>>>>>> That could cause the kind of error you're seeing.
>>>>>>
>>>>>> Matei
>>>>>>
>>>>>> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>>>>>>
>>>>>>> Sorry this should have been sent to user mail list.
>>>>>>>
>>>>>>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>>
>>>>>>>> Hi there:
>>>>>>>>
>>>>>>>> I has build and installed mesos (pre-protobuf branch for spark) and
>>>>>>>> spark,
>>>>>>>> and deployed them to a cluster. However, I have following error
>>>>>>>> when
>>>>>>>> I
>>>>>>>> run
>>>>>>>> spark task on the mesos URL of my master, the spark task runs fine
>>>>>>>> in
>>>>>>>> local mode. Any suggestion or help? Thanks a lot!
>>>>>>>> Joe
>>>>>>>>
>>>>>>>> My os: CentOS 5.2
>>>>>>>> Python: 2.6
>>>>>>>> Gcc: 4.2.1
>>>>>>>> Swig:1.3.40
>>>>>>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor on
>>>>>>>> port
>>>>>>>> 50501
>>>>>>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered
>>>>>>>> actor
>>>>>>>> on
>>>>>>>> port 50501
>>>>>>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0
>>>>>>>> with
>>>>>>>> cache
>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0
>>>>>>>> with
>>>>>>>> 1
>>>>>>>> partitions
>>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current
>>>>>>>> cache
>>>>>>>> locations
>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0
>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final
>>>>>>>> stage:
>>>>>>>> List()
>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents:
>>>>>>>> List()
>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0,
>>>>>>>> which
>>>>>>>> has
>>>>>>>> no missing parents
>>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1 tasks
>>>>>>>> Exception in thread "Spark scheduler"
>>>>>>>> java.lang.UnsatisfiedLinkError:
>>>>>>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>>>>>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native Method)
>>>>>>>> at
>>>>>>>>
>>>>>>>>
>>>>>>>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.ja
>>>>>>>> v
>>>>>>>> a
>>>>>>>> :4
>>>>>>>> 2)
>>>>>>>> at
>>>>>>>>
>>>>>>>>
>>>>>>>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.ja
>>>>>>>> v
>>>>>>>> a
>>>>>>>> :4
>>>>>>>> 6)
>>>>>>>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>>>>>>
>>>>>>>> Sent to wrong address previously
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>
>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>
>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>
>>>>>>>>>> Acknowledgment: I have added the address
>>>>>>>>>>
>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>
>>>>>>>>>> to the mesos-dev mailing list.
>>>>>>>>>>
>>>>>>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>>>>>>
>>>>>>>>>> Please save this message so that you know the address you are
>>>>>>>>>> subscribed under, in case you later want to unsubscribe or change
>>>>>>>>>> your
>>>>>>>>>> subscription address.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>
>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>
>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per request),
>>>>>>>>>> mail:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> To get an index with subject and author for messages 123-456 ,
>>>>>>>>>> mail:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>
>>>>>>>>>> To receive all messages with the same subject as message 12345,
>>>>>>>>>> send a short message to:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>>> being
>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>
>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>> address (with '=' instead of '@') after the command word:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> In both cases, I'll send a confirmation message to that address.
>>>>>>>>>> When
>>>>>>>>>> you receive it, simply reply to it to complete your subscription.
>>>>>>>>>>
>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my owner
>>>>>>>>>> is
>>>>>>>>>> a
>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>
>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>
>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07
>>>>>>>>>> -0000
>>>>>>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>>>>>> (192.87.106.230)
>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>> 23:30:07
>>>>>>>>>> +0000
>>>>>>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>>>>>>> tests=SPF_PASS
>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>> Received-SPF: pass (nike.apache.org: domain of zhiyong@fb.com
>>>>>>>>>> designates
>>>>>>>>>> 67.231.145.42 as permitted sender)
>>>>>>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>>>>>>> (67.231.145.42)
>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>> 23:30:00
>>>>>>>>>> +0000
>>>>>>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>>>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>> p7LNTI6g015638
>>>>>>>>>> for
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incub
>>>>>>>>>> a
>>>>>>>>>> t
>>>>>>>>>> or
>>>>>>>>>> .a
>>>>>>>>>> p
>>>>>>>>>> a
>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>>>> [66.220.144.38])
>>>>>>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>>>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>>>>>>>>>> for
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incub
>>>>>>>>>> a
>>>>>>>>>> t
>>>>>>>>>> or
>>>>>>>>>> .a
>>>>>>>>>> p
>>>>>>>>>> a
>>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>> by
>>>>>>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>> Sun,
>>>>>>>>>> 21 Aug 2011 16:29:14 -0700
>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>> To:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> "mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incub
>>>>>>>>>> a
>>>>>>>>>> t
>>>>>>>>>> or
>>>>>>>>>> .a
>>>>>>>>>> p
>>>>>>>>>> a
>>>>>>>>>> che.org"
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incu
>>>>>>>>>> b
>>>>>>>>>> a
>>>>>>>>>> to
>>>>>>>>>> r.
>>>>>>>>>> a
>>>>>>>>>> p
>>>>>>>>>> ache.org>
>>>>>>>>>> Subject: Re: confirm subscribe to mesos-dev@incubator.apache.org
>>>>>>>>>> Thread-Topic: confirm subscribe to mesos-dev@incubator.apache.org
>>>>>>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>>>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>> In-Reply-To: <13...@incubator.apache.org>
>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>> Content-Language: en-US
>>>>>>>>>> X-MS-Has-Attach:
>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>>>>>> Content-ID: <B4...@fb.com>
>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>> Joe
>>>>>>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>>
>>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>>
>>>>>>>>>>> To confirm that you would like
>>>>>>>>>>>
>>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>>
>>>>>>>>>>> added to the mesos-dev mailing list, please send
>>>>>>>>>>> a short reply to this address:
>>>>>>>>>>>
>>>>>>>>>>> =20
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@inc
>>>>>>>>>>> u
>>>>>>>>>>> b
>>>>>>>>>>> at
>>>>>>>>>>> or
>>>>>>>>>>> .
>>>>>>>>>>> a
>>>>>>>>>>> p=
>>>>>>>>>> ac
>>>>>>>>>>> he.org
>>>>>>>>>>>
>>>>>>>>>>> Usually, this happens when you just hit the "reply" button.
>>>>>>>>>>> If this does not work, simply copy the address and paste it into
>>>>>>>>>>> the "To:" field of a new message.
>>>>>>>>>>>
>>>>>>>>>>> or click here:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb
>>>>>>>>>>> .
>>>>>>>>>>> c
>>>>>>>>>>> om
>>>>>>>>>>> @i
>>>>>>>>>>> n
>>>>>>>>>>> c
>>>>>>>>>>> u=
>>>>>>>>>> ba
>>>>>>>>>>> tor.apache.org
>>>>>>>>>>>
>>>>>>>>>>> This confirmation serves two purposes. First, it verifies that I
>>>>>>>>>>> am
>>>>>>>>>>> able
>>>>>>>>>>> to get mail through to you. Second, it protects you in case
>>>>>>>>>>> someone
>>>>>>>>>>> forges a subscription request in your name.
>>>>>>>>>>>
>>>>>>>>>>> Please note that ALL Apache dev- and user- mailing lists are
>>>>>>>>>>> publicly
>>>>>>>>>>> archived. Do familiarize yourself with Apache's public archive
>>>>>>>>>>> policy
>>>>>>>>>>> at
>>>>>>>>>>>
>>>>>>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>>>>>>
>>>>>>>>>>> prior to subscribing and posting messages to
>>>>>>>>>>> mesos-dev@incubator.apache.org.
>>>>>>>>>>> If you're not sure whether or not the policy applies to this
>>>>>>>>>>> mailing
>>>>>>>>>>> list,
>>>>>>>>>>> assume it does unless the list name contains the word "private"
>>>>>>>>>>> in
>>>>>>>>>>> it.
>>>>>>>>>>>
>>>>>>>>>>> Some mail programs are broken and cannot handle long addresses.
>>>>>>>>>>> If
>>>>>>>>>>> you
>>>>>>>>>>> cannot reply to this request, instead send a message to
>>>>>>>>>>> <me...@incubator.apache.org> and put the
>>>>>>>>>>> entire address listed above into the "Subject:" line.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>>
>>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>>
>>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per request),
>>>>>>>>>>> mail:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> To get an index with subject and author for messages 123-456 ,
>>>>>>>>>>> mail:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>>
>>>>>>>>>>> To receive all messages with the same subject as message 12345,
>>>>>>>>>>> send a short message to:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>>>> being
>>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>>
>>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>>> address (with '=3D' instead of '@') after the command word:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>>
>>>>>>>>>>> In both cases, I'll send a confirmation message to that address.
>>>>>>>>>>> When
>>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>> subscription.
>>>>>>>>>>>
>>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my
>>>>>>>>>>> owner
>>>>>>>>>>> is a
>>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>>
>>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>>
>>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011 22:43:26
>>>>>>>>>>> -0000
>>>>>>>>>>> Received: from athena.apache.org (HELO athena.apache.org)
>>>>>>>>>>> (140.211.11.136)
>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>> 22:43:26
>>>>>>>>>>> +0000
>>>>>>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>>> Received-SPF: pass (athena.apache.org: domain of zhiyong@fb.com
>>>>>>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>>>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>>>>>>>> (67.231.145.42)
>>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>> 22:43:19
>>>>>>>>>>> +0000
>>>>>>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>> p7LMcbSq017336
>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug
>>>>>>>>>>> 2011
>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>>>>> [66.220.144.38])
>>>>>>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128
>>>>>>>>>>> verify=3DNOT)
>>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug
>>>>>>>>>>> 2011
>>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>>> by
>>>>>>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>>> Sun,
>>>>>>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>> Subject: can i join this
>>>>>>>>>>> Thread-Topic: can i join this
>>>>>>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>>> Content-Language: en-US
>>>>>>>>>>> X-MS-Has-Attach:=20
>>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>>> Content-Type: multipart/alternative;
>>>>>>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>>
>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>
>>>>>>>>>>> Best,
>>>>>>>>>>> Joe
>>>>>>>>>>>
>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>>>>>>> Content-ID: <F6...@fb.com>
>>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>>
>>>>>>>>>>> <html>
>>>>>>>>>>> <head>
>>>>>>>>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>>>>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>>>>>>
>>>>>>>>>>> </head>
>>>>>>>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode:
>>>>>>>>>>> space;
>>>>>>>>>>> -webkit-lin=3D
>>>>>>>>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size:
>>>>>>>>>>> 14px;
>>>>>>>>>>> font-fami=3D
>>>>>>>>>>> ly: Calibri, sans-serif; ">
>>>>>>>>>>> <div>Best,</div>
>>>>>>>>>>> <div>Joe</div>
>>>>>>>>>>> </body>
>>>>>>>>>>> </html>
>>>>>>>>>>>
>>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: About Running Spark on a cluster with mesos Re: WELCOME to
mesos-dev@incubator.apache.org
Posted by Joe Xie <zh...@fb.com>.
Btw, I checkout my webui for master, it indicates "No offers are active"
under "Resource Offers" while I have two machines as slave, so there might
be problem with my configuration.
Best,
JOE
On 8/22/11 3:59 PM, "Joe Xie" <zh...@fb.com> wrote:
>Thank you Matei,
>
>it's running for me with Spark 0.3 + pre-protobuf, and I'm testing with
>provided example "run spark.examples.SparkPi mesos://1@10.30.95.184:5050
>3". However, it seems I keep losing TID after starting the task, and then
>job aborted with 4 tries. It might be the problem on my side too, did this
>happen before? I could go for trying masters for spark + mesos.
>11/08/22 15:48:27 INFO spark.SimpleJob: Lost TID 13 (task 0:2)
>11/08/22 15:48:27 ERROR spark.SimpleJob: Task 0:2 failed more than 4
>times; aborting job
>11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from TID 12
>because its job is gone
>11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from TID 14
>because its job is gone
>
>Thanks!
>Joe
>
>
>
>On 8/22/11 3:38 PM, "Joe Xie" <zh...@fb.com> wrote:
>
>>Thank you Matei! I can go for Spark 0.3 + pre-protobuf. I understand
>>there
>>might be lots of new updates on the master branch. Is there any
>>significant difference in current master with Spark 0.3 + pre-protobuf? I
>>could also try to keep a separate branch with stable masters if that's
>>the
>>case.
>>Best,
>>Joe
>>On 8/22/11 3:21 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>
>>>If you want the most stable versions right now, I recommend Spark 0.3
>>>(download it from the website or the 0.3-scala-2.9 tag in Git) and
>>>pre-protobuf Mesos. However, the current master Spark + master Mesos
>>>should also work OK; it's just that further commits may break things
>>>temporarily.
>>>
>>>Matei
>>>
>>>On Aug 22, 2011, at 3:17 PM, Joe Xie wrote:
>>>
>>>> Hi Matei,
>>>> Thanks for your feedback. I bet I'm using the master brach of Spark
>>>>and
>>>> pre-protobuf version of Mesos. Do you mean I should go for Spark
>>>>(master)
>>>> + protobuf (master). Let's me know the most stable combination of
>>>>braches
>>>> from Spark and pre-protobuf, I can try them out.
>>>>
>>>> Best,
>>>> Joe
>>>>
>>>>
>>>> On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>>
>>>>> Are you using the master branch of Spark, or one of the releases
>>>>>(e.g.
>>>>> 0.3)? The releases work with the pre-protobuf version of Mesos but
>>>>>the
>>>>> master branch of Spark actually requires the master branch of Mesos
>>>>>now.
>>>>> That could cause the kind of error you're seeing.
>>>>>
>>>>> Matei
>>>>>
>>>>> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>>>>>
>>>>>> Sorry this should have been sent to user mail list.
>>>>>>
>>>>>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>>
>>>>>>> Hi there:
>>>>>>>
>>>>>>> I has build and installed mesos (pre-protobuf branch for spark) and
>>>>>>> spark,
>>>>>>> and deployed them to a cluster. However, I have following error
>>>>>>>when
>>>>>>>I
>>>>>>> run
>>>>>>> spark task on the mesos URL of my master, the spark task runs fine
>>>>>>>in
>>>>>>> local mode. Any suggestion or help? Thanks a lot!
>>>>>>> Joe
>>>>>>>
>>>>>>> My os: CentOS 5.2
>>>>>>> Python: 2.6
>>>>>>> Gcc: 4.2.1
>>>>>>> Swig:1.3.40
>>>>>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor on
>>>>>>> port
>>>>>>> 50501
>>>>>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered
>>>>>>>actor
>>>>>>>on
>>>>>>> port 50501
>>>>>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0
>>>>>>>with
>>>>>>> cache
>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0
>>>>>>>with
>>>>>>> 1
>>>>>>> partitions
>>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current
>>>>>>>cache
>>>>>>> locations
>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0
>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final
>>>>>>>stage:
>>>>>>> List()
>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents:
>>>>>>>List()
>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0,
>>>>>>>which
>>>>>>> has
>>>>>>> no missing parents
>>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1 tasks
>>>>>>> Exception in thread "Spark scheduler"
>>>>>>>java.lang.UnsatisfiedLinkError:
>>>>>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>>>>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native Method)
>>>>>>> at
>>>>>>>
>>>>>>>
>>>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.ja
>>>>>>>v
>>>>>>>a
>>>>>>>:4
>>>>>>> 2)
>>>>>>> at
>>>>>>>
>>>>>>>
>>>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.ja
>>>>>>>v
>>>>>>>a
>>>>>>>:4
>>>>>>> 6)
>>>>>>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>>>>>
>>>>>>> Sent to wrong address previously
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>
>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>
>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>
>>>>>>>>> Acknowledgment: I have added the address
>>>>>>>>>
>>>>>>>>> zhiyong@fb.com
>>>>>>>>>
>>>>>>>>> to the mesos-dev mailing list.
>>>>>>>>>
>>>>>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>>>>>
>>>>>>>>> Please save this message so that you know the address you are
>>>>>>>>> subscribed under, in case you later want to unsubscribe or change
>>>>>>>>> your
>>>>>>>>> subscription address.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>
>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>> your message to the correct command address:
>>>>>>>>>
>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per request),
>>>>>>>>>mail:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> To get an index with subject and author for messages 123-456 ,
>>>>>>>>>mail:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>
>>>>>>>>> To receive all messages with the same subject as message 12345,
>>>>>>>>> send a short message to:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>>being
>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>
>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>> address (with '=' instead of '@') after the command word:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> In both cases, I'll send a confirmation message to that address.
>>>>>>>>>When
>>>>>>>>> you receive it, simply reply to it to complete your subscription.
>>>>>>>>>
>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>> desired results, please contact my owner at
>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my owner
>>>>>>>>>is
>>>>>>>>> a
>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>
>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>
>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07
>>>>>>>>>-0000
>>>>>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>>>>> (192.87.106.230)
>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>23:30:07
>>>>>>>>> +0000
>>>>>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>>>>>> tests=SPF_PASS
>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>> Received-SPF: pass (nike.apache.org: domain of zhiyong@fb.com
>>>>>>>>> designates
>>>>>>>>> 67.231.145.42 as permitted sender)
>>>>>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>>>>>> (67.231.145.42)
>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>23:30:00
>>>>>>>>> +0000
>>>>>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>p7LNTI6g015638
>>>>>>>>> for
>>>>>>>>>
>>>>>>>>>
>>>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incub
>>>>>>>>>a
>>>>>>>>>t
>>>>>>>>>or
>>>>>>>>> .a
>>>>>>>>> p
>>>>>>>>> a
>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>>> [66.220.144.38])
>>>>>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>>>>>>>>> for
>>>>>>>>>
>>>>>>>>>
>>>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incub
>>>>>>>>>a
>>>>>>>>>t
>>>>>>>>>or
>>>>>>>>> .a
>>>>>>>>> p
>>>>>>>>> a
>>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>> by
>>>>>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>>>>>> 14.01.0289.001;
>>>>>>>>> Sun,
>>>>>>>>> 21 Aug 2011 16:29:14 -0700
>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>> To:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>"mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incub
>>>>>>>>>a
>>>>>>>>>t
>>>>>>>>>or
>>>>>>>>> .a
>>>>>>>>> p
>>>>>>>>> a
>>>>>>>>> che.org"
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incu
>>>>>>>>>b
>>>>>>>>>a
>>>>>>>>>to
>>>>>>>>> r.
>>>>>>>>> a
>>>>>>>>> p
>>>>>>>>> ache.org>
>>>>>>>>> Subject: Re: confirm subscribe to mesos-dev@incubator.apache.org
>>>>>>>>> Thread-Topic: confirm subscribe to mesos-dev@incubator.apache.org
>>>>>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>> In-Reply-To: <13...@incubator.apache.org>
>>>>>>>>> Accept-Language: en-US
>>>>>>>>> Content-Language: en-US
>>>>>>>>> X-MS-Has-Attach:
>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>>>>> Content-ID: <B4...@fb.com>
>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>> MIME-Version: 1.0
>>>>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>> Joe
>>>>>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>>
>>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>>
>>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>>
>>>>>>>>>> To confirm that you would like
>>>>>>>>>>
>>>>>>>>>> zhiyong@fb.com
>>>>>>>>>>
>>>>>>>>>> added to the mesos-dev mailing list, please send
>>>>>>>>>> a short reply to this address:
>>>>>>>>>>
>>>>>>>>>> =20
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@inc
>>>>>>>>>>u
>>>>>>>>>>b
>>>>>>>>>>at
>>>>>>>>>> or
>>>>>>>>>> .
>>>>>>>>>> a
>>>>>>>>>> p=
>>>>>>>>> ac
>>>>>>>>>> he.org
>>>>>>>>>>
>>>>>>>>>> Usually, this happens when you just hit the "reply" button.
>>>>>>>>>> If this does not work, simply copy the address and paste it into
>>>>>>>>>> the "To:" field of a new message.
>>>>>>>>>>
>>>>>>>>>> or click here:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb
>>>>>>>>>>.
>>>>>>>>>>c
>>>>>>>>>>om
>>>>>>>>>> @i
>>>>>>>>>> n
>>>>>>>>>> c
>>>>>>>>>> u=
>>>>>>>>> ba
>>>>>>>>>> tor.apache.org
>>>>>>>>>>
>>>>>>>>>> This confirmation serves two purposes. First, it verifies that I
>>>>>>>>>>am
>>>>>>>>>> able
>>>>>>>>>> to get mail through to you. Second, it protects you in case
>>>>>>>>>>someone
>>>>>>>>>> forges a subscription request in your name.
>>>>>>>>>>
>>>>>>>>>> Please note that ALL Apache dev- and user- mailing lists are
>>>>>>>>>> publicly
>>>>>>>>>> archived. Do familiarize yourself with Apache's public archive
>>>>>>>>>> policy
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>>>>>
>>>>>>>>>> prior to subscribing and posting messages to
>>>>>>>>>> mesos-dev@incubator.apache.org.
>>>>>>>>>> If you're not sure whether or not the policy applies to this
>>>>>>>>>>mailing
>>>>>>>>>> list,
>>>>>>>>>> assume it does unless the list name contains the word "private"
>>>>>>>>>>in
>>>>>>>>>> it.
>>>>>>>>>>
>>>>>>>>>> Some mail programs are broken and cannot handle long addresses.
>>>>>>>>>>If
>>>>>>>>>> you
>>>>>>>>>> cannot reply to this request, instead send a message to
>>>>>>>>>> <me...@incubator.apache.org> and put the
>>>>>>>>>> entire address listed above into the "Subject:" line.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>>
>>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>>> your message to the correct command address:
>>>>>>>>>>
>>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per request),
>>>>>>>>>> mail:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> To get an index with subject and author for messages 123-456 ,
>>>>>>>>>>mail:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>>
>>>>>>>>>> To receive all messages with the same subject as message 12345,
>>>>>>>>>> send a short message to:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>>>being
>>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>>
>>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>>> address (with '=3D' instead of '@') after the command word:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>>
>>>>>>>>>> In both cases, I'll send a confirmation message to that address.
>>>>>>>>>> When
>>>>>>>>>> you receive it, simply reply to it to complete your
>>>>>>>>>>subscription.
>>>>>>>>>>
>>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>>> desired results, please contact my owner at
>>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my
>>>>>>>>>>owner
>>>>>>>>>> is a
>>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>>
>>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>>
>>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011 22:43:26
>>>>>>>>>> -0000
>>>>>>>>>> Received: from athena.apache.org (HELO athena.apache.org)
>>>>>>>>>> (140.211.11.136)
>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>22:43:26
>>>>>>>>>> +0000
>>>>>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>>> Received-SPF: pass (athena.apache.org: domain of zhiyong@fb.com
>>>>>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>>>>>>> (67.231.145.42)
>>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>>22:43:19
>>>>>>>>>> +0000
>>>>>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>>p7LMcbSq017336
>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug
>>>>>>>>>>2011
>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>>>> [66.220.144.38])
>>>>>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128
>>>>>>>>>>verify=3DNOT)
>>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug
>>>>>>>>>>2011
>>>>>>>>>> 15:42:58 -0700
>>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>>> by
>>>>>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>>>>>> 14.01.0289.001;
>>>>>>>>>> Sun,
>>>>>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>> Subject: can i join this
>>>>>>>>>> Thread-Topic: can i join this
>>>>>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>>> Accept-Language: en-US
>>>>>>>>>> Content-Language: en-US
>>>>>>>>>> X-MS-Has-Attach:=20
>>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>>> Content-Type: multipart/alternative;
>>>>>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>>
>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Joe
>>>>>>>>>>
>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>>>>>> Content-ID: <F6...@fb.com>
>>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>>
>>>>>>>>>> <html>
>>>>>>>>>> <head>
>>>>>>>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>>>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>>>>>
>>>>>>>>>> </head>
>>>>>>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode:
>>>>>>>>>>space;
>>>>>>>>>> -webkit-lin=3D
>>>>>>>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size:
>>>>>>>>>>14px;
>>>>>>>>>> font-fami=3D
>>>>>>>>>> ly: Calibri, sans-serif; ">
>>>>>>>>>> <div>Best,</div>
>>>>>>>>>> <div>Joe</div>
>>>>>>>>>> </body>
>>>>>>>>>> </html>
>>>>>>>>>>
>>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: About Running Spark on a cluster with mesos Re: WELCOME to
mesos-dev@incubator.apache.org
Posted by Joe Xie <zh...@fb.com>.
Thank you Matei,
it's running for me with Spark 0.3 + pre-protobuf, and I'm testing with
provided example "run spark.examples.SparkPi mesos://1@10.30.95.184:5050
3". However, it seems I keep losing TID after starting the task, and then
job aborted with 4 tries. It might be the problem on my side too, did this
happen before? I could go for trying masters for spark + mesos.
11/08/22 15:48:27 INFO spark.SimpleJob: Lost TID 13 (task 0:2)
11/08/22 15:48:27 ERROR spark.SimpleJob: Task 0:2 failed more than 4
times; aborting job
11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from TID 12
because its job is gone
11/08/22 15:48:28 INFO spark.MesosScheduler: Ignoring update from TID 14
because its job is gone
Thanks!
Joe
On 8/22/11 3:38 PM, "Joe Xie" <zh...@fb.com> wrote:
>Thank you Matei! I can go for Spark 0.3 + pre-protobuf. I understand there
>might be lots of new updates on the master branch. Is there any
>significant difference in current master with Spark 0.3 + pre-protobuf? I
>could also try to keep a separate branch with stable masters if that's the
>case.
>Best,
>Joe
>On 8/22/11 3:21 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>
>>If you want the most stable versions right now, I recommend Spark 0.3
>>(download it from the website or the 0.3-scala-2.9 tag in Git) and
>>pre-protobuf Mesos. However, the current master Spark + master Mesos
>>should also work OK; it's just that further commits may break things
>>temporarily.
>>
>>Matei
>>
>>On Aug 22, 2011, at 3:17 PM, Joe Xie wrote:
>>
>>> Hi Matei,
>>> Thanks for your feedback. I bet I'm using the master brach of Spark and
>>> pre-protobuf version of Mesos. Do you mean I should go for Spark
>>>(master)
>>> + protobuf (master). Let's me know the most stable combination of
>>>braches
>>> from Spark and pre-protobuf, I can try them out.
>>>
>>> Best,
>>> Joe
>>>
>>>
>>> On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>>
>>>> Are you using the master branch of Spark, or one of the releases (e.g.
>>>> 0.3)? The releases work with the pre-protobuf version of Mesos but the
>>>> master branch of Spark actually requires the master branch of Mesos
>>>>now.
>>>> That could cause the kind of error you're seeing.
>>>>
>>>> Matei
>>>>
>>>> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>>>>
>>>>> Sorry this should have been sent to user mail list.
>>>>>
>>>>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>>
>>>>>> Hi there:
>>>>>>
>>>>>> I has build and installed mesos (pre-protobuf branch for spark) and
>>>>>> spark,
>>>>>> and deployed them to a cluster. However, I have following error when
>>>>>>I
>>>>>> run
>>>>>> spark task on the mesos URL of my master, the spark task runs fine
>>>>>>in
>>>>>> local mode. Any suggestion or help? Thanks a lot!
>>>>>> Joe
>>>>>>
>>>>>> My os: CentOS 5.2
>>>>>> Python: 2.6
>>>>>> Gcc: 4.2.1
>>>>>> Swig:1.3.40
>>>>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor on
>>>>>> port
>>>>>> 50501
>>>>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered actor
>>>>>>on
>>>>>> port 50501
>>>>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>>>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0 with
>>>>>> cache
>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0
>>>>>>with
>>>>>> 1
>>>>>> partitions
>>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current
>>>>>>cache
>>>>>> locations
>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0
>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final stage:
>>>>>> List()
>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents: List()
>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0,
>>>>>>which
>>>>>> has
>>>>>> no missing parents
>>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1 tasks
>>>>>> Exception in thread "Spark scheduler"
>>>>>>java.lang.UnsatisfiedLinkError:
>>>>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>>>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native Method)
>>>>>> at
>>>>>>
>>>>>>
>>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.jav
>>>>>>a
>>>>>>:4
>>>>>> 2)
>>>>>> at
>>>>>>
>>>>>>
>>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.jav
>>>>>>a
>>>>>>:4
>>>>>> 6)
>>>>>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>>>>
>>>>>> Sent to wrong address previously
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>
>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>
>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>
>>>>>>>> Acknowledgment: I have added the address
>>>>>>>>
>>>>>>>> zhiyong@fb.com
>>>>>>>>
>>>>>>>> to the mesos-dev mailing list.
>>>>>>>>
>>>>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>>>>
>>>>>>>> Please save this message so that you know the address you are
>>>>>>>> subscribed under, in case you later want to unsubscribe or change
>>>>>>>> your
>>>>>>>> subscription address.
>>>>>>>>
>>>>>>>>
>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>
>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>> your message to the correct command address:
>>>>>>>>
>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> To get messages 123 through 145 (a maximum of 100 per request),
>>>>>>>>mail:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> To get an index with subject and author for messages 123-456 ,
>>>>>>>>mail:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>>> so you'll actually get 100-499.
>>>>>>>>
>>>>>>>> To receive all messages with the same subject as message 12345,
>>>>>>>> send a short message to:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>being
>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>
>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>> address (with '=' instead of '@') after the command word:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> To stop subscription for this address, mail:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> In both cases, I'll send a confirmation message to that address.
>>>>>>>>When
>>>>>>>> you receive it, simply reply to it to complete your subscription.
>>>>>>>>
>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>> desired results, please contact my owner at
>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my owner
>>>>>>>>is
>>>>>>>> a
>>>>>>>> lot slower than I am ;-)
>>>>>>>>
>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>
>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07
>>>>>>>>-0000
>>>>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>>>> (192.87.106.230)
>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>23:30:07
>>>>>>>> +0000
>>>>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>>>>> tests=SPF_PASS
>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>> Received-SPF: pass (nike.apache.org: domain of zhiyong@fb.com
>>>>>>>> designates
>>>>>>>> 67.231.145.42 as permitted sender)
>>>>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>>>>> (67.231.145.42)
>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>23:30:00
>>>>>>>> +0000
>>>>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id p7LNTI6g015638
>>>>>>>> for
>>>>>>>>
>>>>>>>>
>>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incuba
>>>>>>>>t
>>>>>>>>or
>>>>>>>> .a
>>>>>>>> p
>>>>>>>> a
>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>> [66.220.144.38])
>>>>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>>>>>>>> for
>>>>>>>>
>>>>>>>>
>>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incuba
>>>>>>>>t
>>>>>>>>or
>>>>>>>> .a
>>>>>>>> p
>>>>>>>> a
>>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>> by
>>>>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>>>>> 14.01.0289.001;
>>>>>>>> Sun,
>>>>>>>> 21 Aug 2011 16:29:14 -0700
>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>> To:
>>>>>>>>
>>>>>>>>
>>>>>>>>"mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incuba
>>>>>>>>t
>>>>>>>>or
>>>>>>>> .a
>>>>>>>> p
>>>>>>>> a
>>>>>>>> che.org"
>>>>>>>>
>>>>>>>>
>>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incub
>>>>>>>>a
>>>>>>>>to
>>>>>>>> r.
>>>>>>>> a
>>>>>>>> p
>>>>>>>> ache.org>
>>>>>>>> Subject: Re: confirm subscribe to mesos-dev@incubator.apache.org
>>>>>>>> Thread-Topic: confirm subscribe to mesos-dev@incubator.apache.org
>>>>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>> In-Reply-To: <13...@incubator.apache.org>
>>>>>>>> Accept-Language: en-US
>>>>>>>> Content-Language: en-US
>>>>>>>> X-MS-Has-Attach:
>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>>>> Content-ID: <B4...@fb.com>
>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>> MIME-Version: 1.0
>>>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>> Joe
>>>>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>>
>>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>>
>>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>>
>>>>>>>>> To confirm that you would like
>>>>>>>>>
>>>>>>>>> zhiyong@fb.com
>>>>>>>>>
>>>>>>>>> added to the mesos-dev mailing list, please send
>>>>>>>>> a short reply to this address:
>>>>>>>>>
>>>>>>>>> =20
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@incu
>>>>>>>>>b
>>>>>>>>>at
>>>>>>>>> or
>>>>>>>>> .
>>>>>>>>> a
>>>>>>>>> p=
>>>>>>>> ac
>>>>>>>>> he.org
>>>>>>>>>
>>>>>>>>> Usually, this happens when you just hit the "reply" button.
>>>>>>>>> If this does not work, simply copy the address and paste it into
>>>>>>>>> the "To:" field of a new message.
>>>>>>>>>
>>>>>>>>> or click here:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.
>>>>>>>>>c
>>>>>>>>>om
>>>>>>>>> @i
>>>>>>>>> n
>>>>>>>>> c
>>>>>>>>> u=
>>>>>>>> ba
>>>>>>>>> tor.apache.org
>>>>>>>>>
>>>>>>>>> This confirmation serves two purposes. First, it verifies that I
>>>>>>>>>am
>>>>>>>>> able
>>>>>>>>> to get mail through to you. Second, it protects you in case
>>>>>>>>>someone
>>>>>>>>> forges a subscription request in your name.
>>>>>>>>>
>>>>>>>>> Please note that ALL Apache dev- and user- mailing lists are
>>>>>>>>> publicly
>>>>>>>>> archived. Do familiarize yourself with Apache's public archive
>>>>>>>>> policy
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>>>>
>>>>>>>>> prior to subscribing and posting messages to
>>>>>>>>> mesos-dev@incubator.apache.org.
>>>>>>>>> If you're not sure whether or not the policy applies to this
>>>>>>>>>mailing
>>>>>>>>> list,
>>>>>>>>> assume it does unless the list name contains the word "private"
>>>>>>>>>in
>>>>>>>>> it.
>>>>>>>>>
>>>>>>>>> Some mail programs are broken and cannot handle long addresses.
>>>>>>>>>If
>>>>>>>>> you
>>>>>>>>> cannot reply to this request, instead send a message to
>>>>>>>>> <me...@incubator.apache.org> and put the
>>>>>>>>> entire address listed above into the "Subject:" line.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>>
>>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>>> your message to the correct command address:
>>>>>>>>>
>>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> To get messages 123 through 145 (a maximum of 100 per request),
>>>>>>>>> mail:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> To get an index with subject and author for messages 123-456 ,
>>>>>>>>>mail:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>>>> so you'll actually get 100-499.
>>>>>>>>>
>>>>>>>>> To receive all messages with the same subject as message 12345,
>>>>>>>>> send a short message to:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>>being
>>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>>
>>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>>> address (with '=3D' instead of '@') after the command word:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> To stop subscription for this address, mail:
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>>
>>>>>>>>> In both cases, I'll send a confirmation message to that address.
>>>>>>>>> When
>>>>>>>>> you receive it, simply reply to it to complete your subscription.
>>>>>>>>>
>>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>>> desired results, please contact my owner at
>>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my owner
>>>>>>>>> is a
>>>>>>>>> lot slower than I am ;-)
>>>>>>>>>
>>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>>
>>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011 22:43:26
>>>>>>>>> -0000
>>>>>>>>> Received: from athena.apache.org (HELO athena.apache.org)
>>>>>>>>> (140.211.11.136)
>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>22:43:26
>>>>>>>>> +0000
>>>>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>>> Received-SPF: pass (athena.apache.org: domain of zhiyong@fb.com
>>>>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>>>>>> (67.231.145.42)
>>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>>22:43:19
>>>>>>>>> +0000
>>>>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id
>>>>>>>>>p7LMcbSq017336
>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug 2011
>>>>>>>>> 15:42:58 -0700
>>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>>> [66.220.144.38])
>>>>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128
>>>>>>>>>verify=3DNOT)
>>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug 2011
>>>>>>>>> 15:42:58 -0700
>>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>>> by
>>>>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>>>>> 14.01.0289.001;
>>>>>>>>> Sun,
>>>>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>> Subject: can i join this
>>>>>>>>> Thread-Topic: can i join this
>>>>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>>> Accept-Language: en-US
>>>>>>>>> Content-Language: en-US
>>>>>>>>> X-MS-Has-Attach:=20
>>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>>> Content-Type: multipart/alternative;
>>>>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>>>>> MIME-Version: 1.0
>>>>>>>>>
>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Joe
>>>>>>>>>
>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>>>>> Content-ID: <F6...@fb.com>
>>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>>
>>>>>>>>> <html>
>>>>>>>>> <head>
>>>>>>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>>>>
>>>>>>>>> </head>
>>>>>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode: space;
>>>>>>>>> -webkit-lin=3D
>>>>>>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px;
>>>>>>>>> font-fami=3D
>>>>>>>>> ly: Calibri, sans-serif; ">
>>>>>>>>> <div>Best,</div>
>>>>>>>>> <div>Joe</div>
>>>>>>>>> </body>
>>>>>>>>> </html>
>>>>>>>>>
>>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: About Running Spark on a cluster with mesos Re: WELCOME to
mesos-dev@incubator.apache.org
Posted by Joe Xie <zh...@fb.com>.
Thank you Matei! I can go for Spark 0.3 + pre-protobuf. I understand there
might be lots of new updates on the master branch. Is there any
significant difference in current master with Spark 0.3 + pre-protobuf? I
could also try to keep a separate branch with stable masters if that's the
case.
Best,
Joe
On 8/22/11 3:21 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>If you want the most stable versions right now, I recommend Spark 0.3
>(download it from the website or the 0.3-scala-2.9 tag in Git) and
>pre-protobuf Mesos. However, the current master Spark + master Mesos
>should also work OK; it's just that further commits may break things
>temporarily.
>
>Matei
>
>On Aug 22, 2011, at 3:17 PM, Joe Xie wrote:
>
>> Hi Matei,
>> Thanks for your feedback. I bet I'm using the master brach of Spark and
>> pre-protobuf version of Mesos. Do you mean I should go for Spark
>>(master)
>> + protobuf (master). Let's me know the most stable combination of
>>braches
>> from Spark and pre-protobuf, I can try them out.
>>
>> Best,
>> Joe
>>
>>
>> On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>>
>>> Are you using the master branch of Spark, or one of the releases (e.g.
>>> 0.3)? The releases work with the pre-protobuf version of Mesos but the
>>> master branch of Spark actually requires the master branch of Mesos
>>>now.
>>> That could cause the kind of error you're seeing.
>>>
>>> Matei
>>>
>>> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>>>
>>>> Sorry this should have been sent to user mail list.
>>>>
>>>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>>
>>>>> Hi there:
>>>>>
>>>>> I has build and installed mesos (pre-protobuf branch for spark) and
>>>>> spark,
>>>>> and deployed them to a cluster. However, I have following error when
>>>>>I
>>>>> run
>>>>> spark task on the mesos URL of my master, the spark task runs fine in
>>>>> local mode. Any suggestion or help? Thanks a lot!
>>>>> Joe
>>>>>
>>>>> My os: CentOS 5.2
>>>>> Python: 2.6
>>>>> Gcc: 4.2.1
>>>>> Swig:1.3.40
>>>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor on
>>>>> port
>>>>> 50501
>>>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered actor
>>>>>on
>>>>> port 50501
>>>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0 with
>>>>> cache
>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0
>>>>>with
>>>>> 1
>>>>> partitions
>>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current
>>>>>cache
>>>>> locations
>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0
>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final stage:
>>>>> List()
>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents: List()
>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0,
>>>>>which
>>>>> has
>>>>> no missing parents
>>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1 tasks
>>>>> Exception in thread "Spark scheduler" java.lang.UnsatisfiedLinkError:
>>>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native Method)
>>>>> at
>>>>>
>>>>>
>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java
>>>>>:4
>>>>> 2)
>>>>> at
>>>>>
>>>>>
>>>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java
>>>>>:4
>>>>> 6)
>>>>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>>>
>>>>> Sent to wrong address previously
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>
>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>
>>>>>>> I'm working for my owner, who can be reached
>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>
>>>>>>> Acknowledgment: I have added the address
>>>>>>>
>>>>>>> zhiyong@fb.com
>>>>>>>
>>>>>>> to the mesos-dev mailing list.
>>>>>>>
>>>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>>>
>>>>>>> Please save this message so that you know the address you are
>>>>>>> subscribed under, in case you later want to unsubscribe or change
>>>>>>> your
>>>>>>> subscription address.
>>>>>>>
>>>>>>>
>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>
>>>>>>> I can handle administrative requests automatically. Please
>>>>>>> do not send them to the list address! Instead, send
>>>>>>> your message to the correct command address:
>>>>>>>
>>>>>>> To subscribe to the list, send a message to:
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> To remove your address from the list, send a message to:
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>> <me...@incubator.apache.org>
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> Similar addresses exist for the digest list:
>>>>>>> <me...@incubator.apache.org>
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> To get messages 123 through 145 (a maximum of 100 per request),
>>>>>>>mail:
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> To get an index with subject and author for messages 123-456 ,
>>>>>>>mail:
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>> so you'll actually get 100-499.
>>>>>>>
>>>>>>> To receive all messages with the same subject as message 12345,
>>>>>>> send a short message to:
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> The messages should contain one line or word of text to avoid being
>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>
>>>>>>> You can start a subscription for an alternate address,
>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>> address (with '=' instead of '@') after the command word:
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> To stop subscription for this address, mail:
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> In both cases, I'll send a confirmation message to that address.
>>>>>>>When
>>>>>>> you receive it, simply reply to it to complete your subscription.
>>>>>>>
>>>>>>> If despite following these instructions, you do not get the
>>>>>>> desired results, please contact my owner at
>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my owner
>>>>>>>is
>>>>>>> a
>>>>>>> lot slower than I am ;-)
>>>>>>>
>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>
>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07
>>>>>>>-0000
>>>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>>> (192.87.106.230)
>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:07
>>>>>>> +0000
>>>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>>>> tests=SPF_PASS
>>>>>>> X-Spam-Check-By: apache.org
>>>>>>> Received-SPF: pass (nike.apache.org: domain of zhiyong@fb.com
>>>>>>> designates
>>>>>>> 67.231.145.42 as permitted sender)
>>>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>>>> (67.231.145.42)
>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:00
>>>>>>> +0000
>>>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id p7LNTI6g015638
>>>>>>> for
>>>>>>>
>>>>>>>
>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubat
>>>>>>>or
>>>>>>> .a
>>>>>>> p
>>>>>>> a
>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>> [66.220.144.38])
>>>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>>>>>>> for
>>>>>>>
>>>>>>>
>>>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubat
>>>>>>>or
>>>>>>> .a
>>>>>>> p
>>>>>>> a
>>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>> by
>>>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>>>> 14.01.0289.001;
>>>>>>> Sun,
>>>>>>> 21 Aug 2011 16:29:14 -0700
>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>> To:
>>>>>>>
>>>>>>>
>>>>>>>"mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubat
>>>>>>>or
>>>>>>> .a
>>>>>>> p
>>>>>>> a
>>>>>>> che.org"
>>>>>>>
>>>>>>>
>>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incuba
>>>>>>>to
>>>>>>> r.
>>>>>>> a
>>>>>>> p
>>>>>>> ache.org>
>>>>>>> Subject: Re: confirm subscribe to mesos-dev@incubator.apache.org
>>>>>>> Thread-Topic: confirm subscribe to mesos-dev@incubator.apache.org
>>>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>> In-Reply-To: <13...@incubator.apache.org>
>>>>>>> Accept-Language: en-US
>>>>>>> Content-Language: en-US
>>>>>>> X-MS-Has-Attach:
>>>>>>> X-MS-TNEF-Correlator:
>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>>> Content-ID: <B4...@fb.com>
>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>> MIME-Version: 1.0
>>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>>>
>>>>>>> Thanks!
>>>>>>> Joe
>>>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>>
>>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>>
>>>>>>>> I'm working for my owner, who can be reached
>>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>>
>>>>>>>> To confirm that you would like
>>>>>>>>
>>>>>>>> zhiyong@fb.com
>>>>>>>>
>>>>>>>> added to the mesos-dev mailing list, please send
>>>>>>>> a short reply to this address:
>>>>>>>>
>>>>>>>> =20
>>>>>>>>
>>>>>>>>
>>>>>>>>mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@incub
>>>>>>>>at
>>>>>>>> or
>>>>>>>> .
>>>>>>>> a
>>>>>>>> p=
>>>>>>> ac
>>>>>>>> he.org
>>>>>>>>
>>>>>>>> Usually, this happens when you just hit the "reply" button.
>>>>>>>> If this does not work, simply copy the address and paste it into
>>>>>>>> the "To:" field of a new message.
>>>>>>>>
>>>>>>>> or click here:
>>>>>>>>
>>>>>>>>
>>>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.c
>>>>>>>>om
>>>>>>>> @i
>>>>>>>> n
>>>>>>>> c
>>>>>>>> u=
>>>>>>> ba
>>>>>>>> tor.apache.org
>>>>>>>>
>>>>>>>> This confirmation serves two purposes. First, it verifies that I
>>>>>>>>am
>>>>>>>> able
>>>>>>>> to get mail through to you. Second, it protects you in case
>>>>>>>>someone
>>>>>>>> forges a subscription request in your name.
>>>>>>>>
>>>>>>>> Please note that ALL Apache dev- and user- mailing lists are
>>>>>>>> publicly
>>>>>>>> archived. Do familiarize yourself with Apache's public archive
>>>>>>>> policy
>>>>>>>> at
>>>>>>>>
>>>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>>>
>>>>>>>> prior to subscribing and posting messages to
>>>>>>>> mesos-dev@incubator.apache.org.
>>>>>>>> If you're not sure whether or not the policy applies to this
>>>>>>>>mailing
>>>>>>>> list,
>>>>>>>> assume it does unless the list name contains the word "private" in
>>>>>>>> it.
>>>>>>>>
>>>>>>>> Some mail programs are broken and cannot handle long addresses. If
>>>>>>>> you
>>>>>>>> cannot reply to this request, instead send a message to
>>>>>>>> <me...@incubator.apache.org> and put the
>>>>>>>> entire address listed above into the "Subject:" line.
>>>>>>>>
>>>>>>>>
>>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>>
>>>>>>>> I can handle administrative requests automatically. Please
>>>>>>>> do not send them to the list address! Instead, send
>>>>>>>> your message to the correct command address:
>>>>>>>>
>>>>>>>> To subscribe to the list, send a message to:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> To remove your address from the list, send a message to:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> Similar addresses exist for the digest list:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> To get messages 123 through 145 (a maximum of 100 per request),
>>>>>>>> mail:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> To get an index with subject and author for messages 123-456 ,
>>>>>>>>mail:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>>> so you'll actually get 100-499.
>>>>>>>>
>>>>>>>> To receive all messages with the same subject as message 12345,
>>>>>>>> send a short message to:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> The messages should contain one line or word of text to avoid
>>>>>>>>being
>>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>>
>>>>>>>> You can start a subscription for an alternate address,
>>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>>> address (with '=3D' instead of '@') after the command word:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> To stop subscription for this address, mail:
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>>
>>>>>>>> In both cases, I'll send a confirmation message to that address.
>>>>>>>> When
>>>>>>>> you receive it, simply reply to it to complete your subscription.
>>>>>>>>
>>>>>>>> If despite following these instructions, you do not get the
>>>>>>>> desired results, please contact my owner at
>>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my owner
>>>>>>>> is a
>>>>>>>> lot slower than I am ;-)
>>>>>>>>
>>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>>
>>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011 22:43:26
>>>>>>>> -0000
>>>>>>>> Received: from athena.apache.org (HELO athena.apache.org)
>>>>>>>> (140.211.11.136)
>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>22:43:26
>>>>>>>> +0000
>>>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>>>> X-Spam-Check-By: apache.org
>>>>>>>> Received-SPF: pass (athena.apache.org: domain of zhiyong@fb.com
>>>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>>>>> (67.231.145.42)
>>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011
>>>>>>>>22:43:19
>>>>>>>> +0000
>>>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id p7LMcbSq017336
>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug 2011
>>>>>>>> 15:42:58 -0700
>>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>>> [66.220.144.38])
>>>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128
>>>>>>>>verify=3DNOT)
>>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug 2011
>>>>>>>> 15:42:58 -0700
>>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>>> by
>>>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>>>> 14.01.0289.001;
>>>>>>>> Sun,
>>>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>>>> <me...@incubator.apache.org>
>>>>>>>> Subject: can i join this
>>>>>>>> Thread-Topic: can i join this
>>>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>>> Accept-Language: en-US
>>>>>>>> Content-Language: en-US
>>>>>>>> X-MS-Has-Attach:=20
>>>>>>>> X-MS-TNEF-Correlator:
>>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>>> Content-Type: multipart/alternative;
>>>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>>>> MIME-Version: 1.0
>>>>>>>>
>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Joe
>>>>>>>>
>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>>>> Content-ID: <F6...@fb.com>
>>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>>
>>>>>>>> <html>
>>>>>>>> <head>
>>>>>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>>>
>>>>>>>> </head>
>>>>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode: space;
>>>>>>>> -webkit-lin=3D
>>>>>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px;
>>>>>>>> font-fami=3D
>>>>>>>> ly: Calibri, sans-serif; ">
>>>>>>>> <div>Best,</div>
>>>>>>>> <div>Joe</div>
>>>>>>>> </body>
>>>>>>>> </html>
>>>>>>>>
>>>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: About Running Spark on a cluster with mesos Re: WELCOME to mesos-dev@incubator.apache.org
Posted by Matei Zaharia <ma...@eecs.berkeley.edu>.
If you want the most stable versions right now, I recommend Spark 0.3 (download it from the website or the 0.3-scala-2.9 tag in Git) and pre-protobuf Mesos. However, the current master Spark + master Mesos should also work OK; it's just that further commits may break things temporarily.
Matei
On Aug 22, 2011, at 3:17 PM, Joe Xie wrote:
> Hi Matei,
> Thanks for your feedback. I bet I'm using the master brach of Spark and
> pre-protobuf version of Mesos. Do you mean I should go for Spark (master)
> + protobuf (master). Let's me know the most stable combination of braches
> from Spark and pre-protobuf, I can try them out.
>
> Best,
> Joe
>
>
> On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>
>> Are you using the master branch of Spark, or one of the releases (e.g.
>> 0.3)? The releases work with the pre-protobuf version of Mesos but the
>> master branch of Spark actually requires the master branch of Mesos now.
>> That could cause the kind of error you're seeing.
>>
>> Matei
>>
>> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>>
>>> Sorry this should have been sent to user mail list.
>>>
>>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>>
>>>> Hi there:
>>>>
>>>> I has build and installed mesos (pre-protobuf branch for spark) and
>>>> spark,
>>>> and deployed them to a cluster. However, I have following error when I
>>>> run
>>>> spark task on the mesos URL of my master, the spark task runs fine in
>>>> local mode. Any suggestion or help? Thanks a lot!
>>>> Joe
>>>>
>>>> My os: CentOS 5.2
>>>> Python: 2.6
>>>> Gcc: 4.2.1
>>>> Swig:1.3.40
>>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor on
>>>> port
>>>> 50501
>>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered actor on
>>>> port 50501
>>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0 with
>>>> cache
>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0 with
>>>> 1
>>>> partitions
>>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current cache
>>>> locations
>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0
>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final stage:
>>>> List()
>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents: List()
>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0, which
>>>> has
>>>> no missing parents
>>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1 tasks
>>>> Exception in thread "Spark scheduler" java.lang.UnsatisfiedLinkError:
>>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native Method)
>>>> at
>>>>
>>>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:4
>>>> 2)
>>>> at
>>>>
>>>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:4
>>>> 6)
>>>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>>
>>>> Sent to wrong address previously
>>>>
>>>>
>>>>>
>>>>>
>>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>>> <me...@incubator.apache.org> wrote:
>>>>>
>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>
>>>>>> I'm working for my owner, who can be reached
>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>
>>>>>> Acknowledgment: I have added the address
>>>>>>
>>>>>> zhiyong@fb.com
>>>>>>
>>>>>> to the mesos-dev mailing list.
>>>>>>
>>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>>
>>>>>> Please save this message so that you know the address you are
>>>>>> subscribed under, in case you later want to unsubscribe or change
>>>>>> your
>>>>>> subscription address.
>>>>>>
>>>>>>
>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>
>>>>>> I can handle administrative requests automatically. Please
>>>>>> do not send them to the list address! Instead, send
>>>>>> your message to the correct command address:
>>>>>>
>>>>>> To subscribe to the list, send a message to:
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> To remove your address from the list, send a message to:
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>> <me...@incubator.apache.org>
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> Similar addresses exist for the digest list:
>>>>>> <me...@incubator.apache.org>
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> To get messages 123 through 145 (a maximum of 100 per request), mail:
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> To get an index with subject and author for messages 123-456 , mail:
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>> so you'll actually get 100-499.
>>>>>>
>>>>>> To receive all messages with the same subject as message 12345,
>>>>>> send a short message to:
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> The messages should contain one line or word of text to avoid being
>>>>>> treated as sp@m, but I will ignore their content.
>>>>>> Only the ADDRESS you send to is important.
>>>>>>
>>>>>> You can start a subscription for an alternate address,
>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>> address (with '=' instead of '@') after the command word:
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> To stop subscription for this address, mail:
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> In both cases, I'll send a confirmation message to that address. When
>>>>>> you receive it, simply reply to it to complete your subscription.
>>>>>>
>>>>>> If despite following these instructions, you do not get the
>>>>>> desired results, please contact my owner at
>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my owner is
>>>>>> a
>>>>>> lot slower than I am ;-)
>>>>>>
>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>
>>>>>> Return-Path: <zh...@fb.com>
>>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07 -0000
>>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>> (192.87.106.230)
>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:07
>>>>>> +0000
>>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>>> tests=SPF_PASS
>>>>>> X-Spam-Check-By: apache.org
>>>>>> Received-SPF: pass (nike.apache.org: domain of zhiyong@fb.com
>>>>>> designates
>>>>>> 67.231.145.42 as permitted sender)
>>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>>> (67.231.145.42)
>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:00
>>>>>> +0000
>>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id p7LNTI6g015638
>>>>>> for
>>>>>>
>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator
>>>>>> .a
>>>>>> p
>>>>>> a
>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>> [66.220.144.38])
>>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>>>>>> for
>>>>>>
>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator
>>>>>> .a
>>>>>> p
>>>>>> a
>>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>> by
>>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>>> 14.01.0289.001;
>>>>>> Sun,
>>>>>> 21 Aug 2011 16:29:14 -0700
>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>> To:
>>>>>>
>>>>>> "mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator
>>>>>> .a
>>>>>> p
>>>>>> a
>>>>>> che.org"
>>>>>>
>>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubato
>>>>>> r.
>>>>>> a
>>>>>> p
>>>>>> ache.org>
>>>>>> Subject: Re: confirm subscribe to mesos-dev@incubator.apache.org
>>>>>> Thread-Topic: confirm subscribe to mesos-dev@incubator.apache.org
>>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>>> Message-ID: <CA...@fb.com>
>>>>>> In-Reply-To: <13...@incubator.apache.org>
>>>>>> Accept-Language: en-US
>>>>>> Content-Language: en-US
>>>>>> X-MS-Has-Attach:
>>>>>> X-MS-TNEF-Correlator:
>>>>>> x-originating-ip: [192.168.18.252]
>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>> Content-ID: <B4...@fb.com>
>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>> MIME-Version: 1.0
>>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>>
>>>>>> Thanks!
>>>>>> Joe
>>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>>> <me...@incubator.apache.org> wrote:
>>>>>>
>>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>>
>>>>>>> I'm working for my owner, who can be reached
>>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>>
>>>>>>> To confirm that you would like
>>>>>>>
>>>>>>> zhiyong@fb.com
>>>>>>>
>>>>>>> added to the mesos-dev mailing list, please send
>>>>>>> a short reply to this address:
>>>>>>>
>>>>>>> =20
>>>>>>>
>>>>>>> mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@incubat
>>>>>>> or
>>>>>>> .
>>>>>>> a
>>>>>>> p=
>>>>>> ac
>>>>>>> he.org
>>>>>>>
>>>>>>> Usually, this happens when you just hit the "reply" button.
>>>>>>> If this does not work, simply copy the address and paste it into
>>>>>>> the "To:" field of a new message.
>>>>>>>
>>>>>>> or click here:
>>>>>>>
>>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com
>>>>>>> @i
>>>>>>> n
>>>>>>> c
>>>>>>> u=
>>>>>> ba
>>>>>>> tor.apache.org
>>>>>>>
>>>>>>> This confirmation serves two purposes. First, it verifies that I am
>>>>>>> able
>>>>>>> to get mail through to you. Second, it protects you in case someone
>>>>>>> forges a subscription request in your name.
>>>>>>>
>>>>>>> Please note that ALL Apache dev- and user- mailing lists are
>>>>>>> publicly
>>>>>>> archived. Do familiarize yourself with Apache's public archive
>>>>>>> policy
>>>>>>> at
>>>>>>>
>>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>>
>>>>>>> prior to subscribing and posting messages to
>>>>>>> mesos-dev@incubator.apache.org.
>>>>>>> If you're not sure whether or not the policy applies to this mailing
>>>>>>> list,
>>>>>>> assume it does unless the list name contains the word "private" in
>>>>>>> it.
>>>>>>>
>>>>>>> Some mail programs are broken and cannot handle long addresses. If
>>>>>>> you
>>>>>>> cannot reply to this request, instead send a message to
>>>>>>> <me...@incubator.apache.org> and put the
>>>>>>> entire address listed above into the "Subject:" line.
>>>>>>>
>>>>>>>
>>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>>
>>>>>>> I can handle administrative requests automatically. Please
>>>>>>> do not send them to the list address! Instead, send
>>>>>>> your message to the correct command address:
>>>>>>>
>>>>>>> To subscribe to the list, send a message to:
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> To remove your address from the list, send a message to:
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>>> <me...@incubator.apache.org>
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> Similar addresses exist for the digest list:
>>>>>>> <me...@incubator.apache.org>
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> To get messages 123 through 145 (a maximum of 100 per request),
>>>>>>> mail:
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> To get an index with subject and author for messages 123-456 , mail:
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>>> so you'll actually get 100-499.
>>>>>>>
>>>>>>> To receive all messages with the same subject as message 12345,
>>>>>>> send a short message to:
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> The messages should contain one line or word of text to avoid being
>>>>>>> treated as sp@m, but I will ignore their content.
>>>>>>> Only the ADDRESS you send to is important.
>>>>>>>
>>>>>>> You can start a subscription for an alternate address,
>>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>>> address (with '=3D' instead of '@') after the command word:
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> To stop subscription for this address, mail:
>>>>>>> <me...@incubator.apache.org>
>>>>>>>
>>>>>>> In both cases, I'll send a confirmation message to that address.
>>>>>>> When
>>>>>>> you receive it, simply reply to it to complete your subscription.
>>>>>>>
>>>>>>> If despite following these instructions, you do not get the
>>>>>>> desired results, please contact my owner at
>>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my owner
>>>>>>> is a
>>>>>>> lot slower than I am ;-)
>>>>>>>
>>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>>
>>>>>>> Return-Path: <zh...@fb.com>
>>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011 22:43:26
>>>>>>> -0000
>>>>>>> Received: from athena.apache.org (HELO athena.apache.org)
>>>>>>> (140.211.11.136)
>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:26
>>>>>>> +0000
>>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>>> X-Spam-Check-By: apache.org
>>>>>>> Received-SPF: pass (athena.apache.org: domain of zhiyong@fb.com
>>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>>>> (67.231.145.42)
>>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:19
>>>>>>> +0000
>>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id p7LMcbSq017336
>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug 2011
>>>>>>> 15:42:58 -0700
>>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>>> [66.220.144.38])
>>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128 verify=3DNOT)
>>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug 2011
>>>>>>> 15:42:58 -0700
>>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>> ([fe80::f05e:c7f1:c7dd:e640])
>>>>>>> by
>>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>>> 14.01.0289.001;
>>>>>>> Sun,
>>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>>> <me...@incubator.apache.org>
>>>>>>> Subject: can i join this
>>>>>>> Thread-Topic: can i join this
>>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>>> Message-ID: <CA...@fb.com>
>>>>>>> Accept-Language: en-US
>>>>>>> Content-Language: en-US
>>>>>>> X-MS-Has-Attach:=20
>>>>>>> X-MS-TNEF-Correlator:
>>>>>>> x-originating-ip: [192.168.18.252]
>>>>>>> Content-Type: multipart/alternative;
>>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>>> MIME-Version: 1.0
>>>>>>>
>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>
>>>>>>> Best,
>>>>>>> Joe
>>>>>>>
>>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>>> Content-ID: <F6...@fb.com>
>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>>
>>>>>>> <html>
>>>>>>> <head>
>>>>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>>
>>>>>>> </head>
>>>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode: space;
>>>>>>> -webkit-lin=3D
>>>>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px;
>>>>>>> font-fami=3D
>>>>>>> ly: Calibri, sans-serif; ">
>>>>>>> <div>Best,</div>
>>>>>>> <div>Joe</div>
>>>>>>> </body>
>>>>>>> </html>
>>>>>>>
>>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: About Running Spark on a cluster with mesos Re: WELCOME to
mesos-dev@incubator.apache.org
Posted by Joe Xie <zh...@fb.com>.
Hi Matei,
Thanks for your feedback. I bet I'm using the master brach of Spark and
pre-protobuf version of Mesos. Do you mean I should go for Spark (master)
+ protobuf (master). Let's me know the most stable combination of braches
from Spark and pre-protobuf, I can try them out.
Best,
Joe
On 8/22/11 3:09 PM, "Matei Zaharia" <ma...@eecs.berkeley.edu> wrote:
>Are you using the master branch of Spark, or one of the releases (e.g.
>0.3)? The releases work with the pre-protobuf version of Mesos but the
>master branch of Spark actually requires the master branch of Mesos now.
>That could cause the kind of error you're seeing.
>
>Matei
>
>On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
>
>> Sorry this should have been sent to user mail list.
>>
>> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>>
>>> Hi there:
>>>
>>> I has build and installed mesos (pre-protobuf branch for spark) and
>>>spark,
>>> and deployed them to a cluster. However, I have following error when I
>>>run
>>> spark task on the mesos URL of my master, the spark task runs fine in
>>> local mode. Any suggestion or help? Thanks a lot!
>>> Joe
>>>
>>> My os: CentOS 5.2
>>> Python: 2.6
>>> Gcc: 4.2.1
>>> Swig:1.3.40
>>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor on
>>>port
>>> 50501
>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered actor on
>>> port 50501
>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0 with
>>>cache
>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0 with
>>>1
>>> partitions
>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current cache
>>> locations
>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0
>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final stage:
>>> List()
>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents: List()
>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0, which
>>>has
>>> no missing parents
>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1 tasks
>>> Exception in thread "Spark scheduler" java.lang.UnsatisfiedLinkError:
>>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native Method)
>>> at
>>>
>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:4
>>>2)
>>> at
>>>
>>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:4
>>>6)
>>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>>
>>> Sent to wrong address previously
>>>
>>>
>>>>
>>>>
>>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>>> <me...@incubator.apache.org> wrote:
>>>>
>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>
>>>>> I'm working for my owner, who can be reached
>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>
>>>>> Acknowledgment: I have added the address
>>>>>
>>>>> zhiyong@fb.com
>>>>>
>>>>> to the mesos-dev mailing list.
>>>>>
>>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>>
>>>>> Please save this message so that you know the address you are
>>>>> subscribed under, in case you later want to unsubscribe or change
>>>>>your
>>>>> subscription address.
>>>>>
>>>>>
>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>
>>>>> I can handle administrative requests automatically. Please
>>>>> do not send them to the list address! Instead, send
>>>>> your message to the correct command address:
>>>>>
>>>>> To subscribe to the list, send a message to:
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> To remove your address from the list, send a message to:
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> Send mail to the following for info and FAQ for this list:
>>>>> <me...@incubator.apache.org>
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> Similar addresses exist for the digest list:
>>>>> <me...@incubator.apache.org>
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> To get messages 123 through 145 (a maximum of 100 per request), mail:
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> To get an index with subject and author for messages 123-456 , mail:
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>> so you'll actually get 100-499.
>>>>>
>>>>> To receive all messages with the same subject as message 12345,
>>>>> send a short message to:
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> The messages should contain one line or word of text to avoid being
>>>>> treated as sp@m, but I will ignore their content.
>>>>> Only the ADDRESS you send to is important.
>>>>>
>>>>> You can start a subscription for an alternate address,
>>>>> for example "john@host.domain", just add a hyphen and your
>>>>> address (with '=' instead of '@') after the command word:
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> To stop subscription for this address, mail:
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> In both cases, I'll send a confirmation message to that address. When
>>>>> you receive it, simply reply to it to complete your subscription.
>>>>>
>>>>> If despite following these instructions, you do not get the
>>>>> desired results, please contact my owner at
>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my owner is
>>>>>a
>>>>> lot slower than I am ;-)
>>>>>
>>>>> --- Enclosed is a copy of the request I received.
>>>>>
>>>>> Return-Path: <zh...@fb.com>
>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07 -0000
>>>>> Received: from nike.apache.org (HELO nike.apache.org)
>>>>>(192.87.106.230)
>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:07
>>>>> +0000
>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>>> tests=SPF_PASS
>>>>> X-Spam-Check-By: apache.org
>>>>> Received-SPF: pass (nike.apache.org: domain of zhiyong@fb.com
>>>>>designates
>>>>> 67.231.145.42 as permitted sender)
>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>> (67.231.145.42)
>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:00
>>>>> +0000
>>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id p7LNTI6g015638
>>>>> for
>>>>>
>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator
>>>>>.a
>>>>> p
>>>>> a
>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>> [66.220.144.38])
>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>>>>> for
>>>>>
>>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator
>>>>>.a
>>>>> p
>>>>> a
>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>([fe80::f05e:c7f1:c7dd:e640])
>>>>> by
>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>>> 14.01.0289.001;
>>>>> Sun,
>>>>> 21 Aug 2011 16:29:14 -0700
>>>>> From: Joe Xie <zh...@fb.com>
>>>>> To:
>>>>>
>>>>>"mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator
>>>>>.a
>>>>> p
>>>>> a
>>>>> che.org"
>>>>>
>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubato
>>>>>r.
>>>>> a
>>>>> p
>>>>> ache.org>
>>>>> Subject: Re: confirm subscribe to mesos-dev@incubator.apache.org
>>>>> Thread-Topic: confirm subscribe to mesos-dev@incubator.apache.org
>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>>> Message-ID: <CA...@fb.com>
>>>>> In-Reply-To: <13...@incubator.apache.org>
>>>>> Accept-Language: en-US
>>>>> Content-Language: en-US
>>>>> X-MS-Has-Attach:
>>>>> X-MS-TNEF-Correlator:
>>>>> x-originating-ip: [192.168.18.252]
>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>> Content-ID: <B4...@fb.com>
>>>>> Content-Transfer-Encoding: quoted-printable
>>>>> MIME-Version: 1.0
>>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>>
>>>>> Thanks!
>>>>> Joe
>>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>>> <me...@incubator.apache.org> wrote:
>>>>>
>>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>>
>>>>>> I'm working for my owner, who can be reached
>>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>>
>>>>>> To confirm that you would like
>>>>>>
>>>>>> zhiyong@fb.com
>>>>>>
>>>>>> added to the mesos-dev mailing list, please send
>>>>>> a short reply to this address:
>>>>>>
>>>>>> =20
>>>>>>
>>>>>>mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@incubat
>>>>>>or
>>>>>> .
>>>>>> a
>>>>>> p=
>>>>> ac
>>>>>> he.org
>>>>>>
>>>>>> Usually, this happens when you just hit the "reply" button.
>>>>>> If this does not work, simply copy the address and paste it into
>>>>>> the "To:" field of a new message.
>>>>>>
>>>>>> or click here:
>>>>>>
>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com
>>>>>>@i
>>>>>> n
>>>>>> c
>>>>>> u=
>>>>> ba
>>>>>> tor.apache.org
>>>>>>
>>>>>> This confirmation serves two purposes. First, it verifies that I am
>>>>>> able
>>>>>> to get mail through to you. Second, it protects you in case someone
>>>>>> forges a subscription request in your name.
>>>>>>
>>>>>> Please note that ALL Apache dev- and user- mailing lists are
>>>>>>publicly
>>>>>> archived. Do familiarize yourself with Apache's public archive
>>>>>>policy
>>>>>> at
>>>>>>
>>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>>
>>>>>> prior to subscribing and posting messages to
>>>>>> mesos-dev@incubator.apache.org.
>>>>>> If you're not sure whether or not the policy applies to this mailing
>>>>>> list,
>>>>>> assume it does unless the list name contains the word "private" in
>>>>>>it.
>>>>>>
>>>>>> Some mail programs are broken and cannot handle long addresses. If
>>>>>>you
>>>>>> cannot reply to this request, instead send a message to
>>>>>> <me...@incubator.apache.org> and put the
>>>>>> entire address listed above into the "Subject:" line.
>>>>>>
>>>>>>
>>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>>
>>>>>> I can handle administrative requests automatically. Please
>>>>>> do not send them to the list address! Instead, send
>>>>>> your message to the correct command address:
>>>>>>
>>>>>> To subscribe to the list, send a message to:
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> To remove your address from the list, send a message to:
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> Send mail to the following for info and FAQ for this list:
>>>>>> <me...@incubator.apache.org>
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> Similar addresses exist for the digest list:
>>>>>> <me...@incubator.apache.org>
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> To get messages 123 through 145 (a maximum of 100 per request),
>>>>>>mail:
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> To get an index with subject and author for messages 123-456 , mail:
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>>> so you'll actually get 100-499.
>>>>>>
>>>>>> To receive all messages with the same subject as message 12345,
>>>>>> send a short message to:
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> The messages should contain one line or word of text to avoid being
>>>>>> treated as sp@m, but I will ignore their content.
>>>>>> Only the ADDRESS you send to is important.
>>>>>>
>>>>>> You can start a subscription for an alternate address,
>>>>>> for example "john@host.domain", just add a hyphen and your
>>>>>> address (with '=3D' instead of '@') after the command word:
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> To stop subscription for this address, mail:
>>>>>> <me...@incubator.apache.org>
>>>>>>
>>>>>> In both cases, I'll send a confirmation message to that address.
>>>>>>When
>>>>>> you receive it, simply reply to it to complete your subscription.
>>>>>>
>>>>>> If despite following these instructions, you do not get the
>>>>>> desired results, please contact my owner at
>>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my owner
>>>>>>is a
>>>>>> lot slower than I am ;-)
>>>>>>
>>>>>> --- Enclosed is a copy of the request I received.
>>>>>>
>>>>>> Return-Path: <zh...@fb.com>
>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011 22:43:26
>>>>>>-0000
>>>>>> Received: from athena.apache.org (HELO athena.apache.org)
>>>>>> (140.211.11.136)
>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:26
>>>>>> +0000
>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>>> X-Spam-Check-By: apache.org
>>>>>> Received-SPF: pass (athena.apache.org: domain of zhiyong@fb.com
>>>>>> designates 67.231.145.42 as permitted sender)
>>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>>> (67.231.145.42)
>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:19
>>>>>> +0000
>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id p7LMcbSq017336
>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug 2011
>>>>>> 15:42:58 -0700
>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>>> [66.220.144.38])
>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128 verify=3DNOT)
>>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug 2011
>>>>>> 15:42:58 -0700
>>>>>> Received: from SC-MBX01-5.TheFacebook.com
>>>>>>([fe80::f05e:c7f1:c7dd:e640])
>>>>>> by
>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>>> 14.01.0289.001;
>>>>>> Sun,
>>>>>> 21 Aug 2011 15:42:53 -0700
>>>>>> From: Joe Xie <zh...@fb.com>
>>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>>> <me...@incubator.apache.org>
>>>>>> Subject: can i join this
>>>>>> Thread-Topic: can i join this
>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>>> Message-ID: <CA...@fb.com>
>>>>>> Accept-Language: en-US
>>>>>> Content-Language: en-US
>>>>>> X-MS-Has-Attach:=20
>>>>>> X-MS-TNEF-Correlator:
>>>>>> x-originating-ip: [192.168.18.252]
>>>>>> Content-Type: multipart/alternative;
>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>>> MIME-Version: 1.0
>>>>>>
>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>
>>>>>> Best,
>>>>>> Joe
>>>>>>
>>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>>> Content-ID: <F6...@fb.com>
>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>
>>>>>> <html>
>>>>>> <head>
>>>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>>>> charset=3D3Dus-ascii"=3D
>>>>>>>
>>>>>> </head>
>>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode: space;
>>>>>> -webkit-lin=3D
>>>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px;
>>>>>> font-fami=3D
>>>>>> ly: Calibri, sans-serif; ">
>>>>>> <div>Best,</div>
>>>>>> <div>Joe</div>
>>>>>> </body>
>>>>>> </html>
>>>>>>
>>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>>
>>>>
>>>
>>
>
Re: About Running Spark on a cluster with mesos Re: WELCOME to mesos-dev@incubator.apache.org
Posted by Matei Zaharia <ma...@eecs.berkeley.edu>.
Are you using the master branch of Spark, or one of the releases (e.g. 0.3)? The releases work with the pre-protobuf version of Mesos but the master branch of Spark actually requires the master branch of Mesos now. That could cause the kind of error you're seeing.
Matei
On Aug 22, 2011, at 3:06 PM, Joe Xie wrote:
> Sorry this should have been sent to user mail list.
>
> On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>
>> Hi there:
>>
>> I has build and installed mesos (pre-protobuf branch for spark) and spark,
>> and deployed them to a cluster. However, I have following error when I run
>> spark task on the mesos URL of my master, the spark task runs fine in
>> local mode. Any suggestion or help? Thanks a lot!
>> Joe
>>
>> My os: CentOS 5.2
>> Python: 2.6
>> Gcc: 4.2.1
>> Swig:1.3.40
>> ./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor on port
>> 50501
>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered actor on
>> port 50501
>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0 with cache
>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0 with 1
>> partitions
>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current cache
>> locations
>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0
>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final stage:
>> List()
>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents: List()
>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0, which has
>> no missing parents
>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1 tasks
>> Exception in thread "Spark scheduler" java.lang.UnsatisfiedLinkError:
>> org.apache.mesos.MesosSchedulerDriver.initialize()V
>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native Method)
>> at
>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:42)
>> at
>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:46)
>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>>
>> Sent to wrong address previously
>>
>>
>>>
>>>
>>> On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>>> <me...@incubator.apache.org> wrote:
>>>
>>>> Hi! This is the ezmlm program. I'm managing the
>>>> mesos-dev@incubator.apache.org mailing list.
>>>>
>>>> I'm working for my owner, who can be reached
>>>> at mesos-dev-owner@incubator.apache.org.
>>>>
>>>> Acknowledgment: I have added the address
>>>>
>>>> zhiyong@fb.com
>>>>
>>>> to the mesos-dev mailing list.
>>>>
>>>> Welcome to mesos-dev@incubator.apache.org!
>>>>
>>>> Please save this message so that you know the address you are
>>>> subscribed under, in case you later want to unsubscribe or change your
>>>> subscription address.
>>>>
>>>>
>>>> --- Administrative commands for the mesos-dev list ---
>>>>
>>>> I can handle administrative requests automatically. Please
>>>> do not send them to the list address! Instead, send
>>>> your message to the correct command address:
>>>>
>>>> To subscribe to the list, send a message to:
>>>> <me...@incubator.apache.org>
>>>>
>>>> To remove your address from the list, send a message to:
>>>> <me...@incubator.apache.org>
>>>>
>>>> Send mail to the following for info and FAQ for this list:
>>>> <me...@incubator.apache.org>
>>>> <me...@incubator.apache.org>
>>>>
>>>> Similar addresses exist for the digest list:
>>>> <me...@incubator.apache.org>
>>>> <me...@incubator.apache.org>
>>>>
>>>> To get messages 123 through 145 (a maximum of 100 per request), mail:
>>>> <me...@incubator.apache.org>
>>>>
>>>> To get an index with subject and author for messages 123-456 , mail:
>>>> <me...@incubator.apache.org>
>>>>
>>>> They are always returned as sets of 100, max 2000 per request,
>>>> so you'll actually get 100-499.
>>>>
>>>> To receive all messages with the same subject as message 12345,
>>>> send a short message to:
>>>> <me...@incubator.apache.org>
>>>>
>>>> The messages should contain one line or word of text to avoid being
>>>> treated as sp@m, but I will ignore their content.
>>>> Only the ADDRESS you send to is important.
>>>>
>>>> You can start a subscription for an alternate address,
>>>> for example "john@host.domain", just add a hyphen and your
>>>> address (with '=' instead of '@') after the command word:
>>>> <me...@incubator.apache.org>
>>>>
>>>> To stop subscription for this address, mail:
>>>> <me...@incubator.apache.org>
>>>>
>>>> In both cases, I'll send a confirmation message to that address. When
>>>> you receive it, simply reply to it to complete your subscription.
>>>>
>>>> If despite following these instructions, you do not get the
>>>> desired results, please contact my owner at
>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my owner is a
>>>> lot slower than I am ;-)
>>>>
>>>> --- Enclosed is a copy of the request I received.
>>>>
>>>> Return-Path: <zh...@fb.com>
>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07 -0000
>>>> Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:07
>>>> +0000
>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>> tests=SPF_PASS
>>>> X-Spam-Check-By: apache.org
>>>> Received-SPF: pass (nike.apache.org: domain of zhiyong@fb.com designates
>>>> 67.231.145.42 as permitted sender)
>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>> (67.231.145.42)
>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:00
>>>> +0000
>>>> Received: from pps.filterd (m0004348 [127.0.0.1])
>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id p7LNTI6g015638
>>>> for
>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.a
>>>> p
>>>> a
>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>> [66.220.144.38])
>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>>>> for
>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.a
>>>> p
>>>> a
>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>> Received: from SC-MBX01-5.TheFacebook.com ([fe80::f05e:c7f1:c7dd:e640])
>>>> by
>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>> 14.01.0289.001;
>>>> Sun,
>>>> 21 Aug 2011 16:29:14 -0700
>>>> From: Joe Xie <zh...@fb.com>
>>>> To:
>>>> "mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.a
>>>> p
>>>> a
>>>> che.org"
>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.
>>>> a
>>>> p
>>>> ache.org>
>>>> Subject: Re: confirm subscribe to mesos-dev@incubator.apache.org
>>>> Thread-Topic: confirm subscribe to mesos-dev@incubator.apache.org
>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>> Message-ID: <CA...@fb.com>
>>>> In-Reply-To: <13...@incubator.apache.org>
>>>> Accept-Language: en-US
>>>> Content-Language: en-US
>>>> X-MS-Has-Attach:
>>>> X-MS-TNEF-Correlator:
>>>> x-originating-ip: [192.168.18.252]
>>>> Content-Type: text/plain; charset="us-ascii"
>>>> Content-ID: <B4...@fb.com>
>>>> Content-Transfer-Encoding: quoted-printable
>>>> MIME-Version: 1.0
>>>> X-Virus-Checked: Checked by ClamAV on apache.org
>>>>
>>>> Thanks!
>>>> Joe
>>>> On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>>> <me...@incubator.apache.org> wrote:
>>>>
>>>>> Hi! This is the ezmlm program. I'm managing the
>>>>> mesos-dev@incubator.apache.org mailing list.
>>>>>
>>>>> I'm working for my owner, who can be reached
>>>>> at mesos-dev-owner@incubator.apache.org.
>>>>>
>>>>> To confirm that you would like
>>>>>
>>>>> zhiyong@fb.com
>>>>>
>>>>> added to the mesos-dev mailing list, please send
>>>>> a short reply to this address:
>>>>>
>>>>> =20
>>>>> mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@incubator
>>>>> .
>>>>> a
>>>>> p=
>>>> ac
>>>>> he.org
>>>>>
>>>>> Usually, this happens when you just hit the "reply" button.
>>>>> If this does not work, simply copy the address and paste it into
>>>>> the "To:" field of a new message.
>>>>>
>>>>> or click here:
>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@i
>>>>> n
>>>>> c
>>>>> u=
>>>> ba
>>>>> tor.apache.org
>>>>>
>>>>> This confirmation serves two purposes. First, it verifies that I am
>>>>> able
>>>>> to get mail through to you. Second, it protects you in case someone
>>>>> forges a subscription request in your name.
>>>>>
>>>>> Please note that ALL Apache dev- and user- mailing lists are publicly
>>>>> archived. Do familiarize yourself with Apache's public archive policy
>>>>> at
>>>>>
>>>>> http://www.apache.org/foundation/public-archives.html
>>>>>
>>>>> prior to subscribing and posting messages to
>>>>> mesos-dev@incubator.apache.org.
>>>>> If you're not sure whether or not the policy applies to this mailing
>>>>> list,
>>>>> assume it does unless the list name contains the word "private" in it.
>>>>>
>>>>> Some mail programs are broken and cannot handle long addresses. If you
>>>>> cannot reply to this request, instead send a message to
>>>>> <me...@incubator.apache.org> and put the
>>>>> entire address listed above into the "Subject:" line.
>>>>>
>>>>>
>>>>> --- Administrative commands for the mesos-dev list ---
>>>>>
>>>>> I can handle administrative requests automatically. Please
>>>>> do not send them to the list address! Instead, send
>>>>> your message to the correct command address:
>>>>>
>>>>> To subscribe to the list, send a message to:
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> To remove your address from the list, send a message to:
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> Send mail to the following for info and FAQ for this list:
>>>>> <me...@incubator.apache.org>
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> Similar addresses exist for the digest list:
>>>>> <me...@incubator.apache.org>
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> To get messages 123 through 145 (a maximum of 100 per request), mail:
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> To get an index with subject and author for messages 123-456 , mail:
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> They are always returned as sets of 100, max 2000 per request,
>>>>> so you'll actually get 100-499.
>>>>>
>>>>> To receive all messages with the same subject as message 12345,
>>>>> send a short message to:
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> The messages should contain one line or word of text to avoid being
>>>>> treated as sp@m, but I will ignore their content.
>>>>> Only the ADDRESS you send to is important.
>>>>>
>>>>> You can start a subscription for an alternate address,
>>>>> for example "john@host.domain", just add a hyphen and your
>>>>> address (with '=3D' instead of '@') after the command word:
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> To stop subscription for this address, mail:
>>>>> <me...@incubator.apache.org>
>>>>>
>>>>> In both cases, I'll send a confirmation message to that address. When
>>>>> you receive it, simply reply to it to complete your subscription.
>>>>>
>>>>> If despite following these instructions, you do not get the
>>>>> desired results, please contact my owner at
>>>>> mesos-dev-owner@incubator.apache.org. Please be patient, my owner is a
>>>>> lot slower than I am ;-)
>>>>>
>>>>> --- Enclosed is a copy of the request I received.
>>>>>
>>>>> Return-Path: <zh...@fb.com>
>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011 22:43:26 -0000
>>>>> Received: from athena.apache.org (HELO athena.apache.org)
>>>>> (140.211.11.136)
>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:26
>>>>> +0000
>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>> X-Spam-Check-By: apache.org
>>>>> Received-SPF: pass (athena.apache.org: domain of zhiyong@fb.com
>>>>> designates 67.231.145.42 as permitted sender)
>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>> (67.231.145.42)
>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:19
>>>>> +0000
>>>>> Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id p7LMcbSq017336
>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug 2011
>>>>> 15:42:58 -0700
>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>> [66.220.144.38])
>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128 verify=3DNOT)
>>>>> for <me...@incubator.apache.org>; Sun, 21 Aug 2011
>>>>> 15:42:58 -0700
>>>>> Received: from SC-MBX01-5.TheFacebook.com ([fe80::f05e:c7f1:c7dd:e640])
>>>>> by
>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>> 14.01.0289.001;
>>>>> Sun,
>>>>> 21 Aug 2011 15:42:53 -0700
>>>>> From: Joe Xie <zh...@fb.com>
>>>>> To: "mesos-dev-subscribe@incubator.apache.org"
>>>>> <me...@incubator.apache.org>
>>>>> Subject: can i join this
>>>>> Thread-Topic: can i join this
>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>> Message-ID: <CA...@fb.com>
>>>>> Accept-Language: en-US
>>>>> Content-Language: en-US
>>>>> X-MS-Has-Attach:=20
>>>>> X-MS-TNEF-Correlator:
>>>>> x-originating-ip: [192.168.18.252]
>>>>> Content-Type: multipart/alternative;
>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>> MIME-Version: 1.0
>>>>>
>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>> Content-Type: text/plain; charset=3D"us-ascii"
>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>
>>>>> Best,
>>>>> Joe
>>>>>
>>>>> --_000_CA76D7F74256zhiyongfbcom_
>>>>> Content-Type: text/html; charset=3D"us-ascii"
>>>>> Content-ID: <F6...@fb.com>
>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>
>>>>> <html>
>>>>> <head>
>>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>>> charset=3D3Dus-ascii"=3D
>>>>>>
>>>>> </head>
>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode: space;
>>>>> -webkit-lin=3D
>>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px;
>>>>> font-fami=3D
>>>>> ly: Calibri, sans-serif; ">
>>>>> <div>Best,</div>
>>>>> <div>Joe</div>
>>>>> </body>
>>>>> </html>
>>>>>
>>>>> --_000_CA76D7F74256zhiyongfbcom_--
>>>>
>>>
>>
>
Re: About Running Spark on a cluster with mesos Re: WELCOME to
mesos-dev@incubator.apache.org
Posted by Joe Xie <zh...@fb.com>.
Sorry this should have been sent to user mail list.
On 8/22/11 2:19 PM, "Joe Xie" <zh...@fb.com> wrote:
>Hi there:
>
>I has build and installed mesos (pre-protobuf branch for spark) and spark,
>and deployed them to a cluster. However, I have following error when I run
>spark task on the mesos URL of my master, the spark task runs fine in
>local mode. Any suggestion or help? Thanks a lot!
>Joe
>
>My os: CentOS 5.2
>Python: 2.6
>Gcc: 4.2.1
>Swig:1.3.40
>./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050 1
>11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor on port
>50501
>11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered actor on
>port 50501
>11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0 with cache
>11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0 with 1
>partitions
>11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current cache
>locations
>11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0
>11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final stage:
>List()
>11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents: List()
>11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0, which has
>no missing parents
>11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1 tasks
>Exception in thread "Spark scheduler" java.lang.UnsatisfiedLinkError:
>org.apache.mesos.MesosSchedulerDriver.initialize()V
> at org.apache.mesos.MesosSchedulerDriver.initialize(Native Method)
> at
>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:42)
> at
>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:46)
> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>
>Sent to wrong address previously
>
>
>>
>>
>>On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>><me...@incubator.apache.org> wrote:
>>
>>>Hi! This is the ezmlm program. I'm managing the
>>>mesos-dev@incubator.apache.org mailing list.
>>>
>>>I'm working for my owner, who can be reached
>>>at mesos-dev-owner@incubator.apache.org.
>>>
>>>Acknowledgment: I have added the address
>>>
>>> zhiyong@fb.com
>>>
>>>to the mesos-dev mailing list.
>>>
>>>Welcome to mesos-dev@incubator.apache.org!
>>>
>>>Please save this message so that you know the address you are
>>>subscribed under, in case you later want to unsubscribe or change your
>>>subscription address.
>>>
>>>
>>>--- Administrative commands for the mesos-dev list ---
>>>
>>>I can handle administrative requests automatically. Please
>>>do not send them to the list address! Instead, send
>>>your message to the correct command address:
>>>
>>>To subscribe to the list, send a message to:
>>> <me...@incubator.apache.org>
>>>
>>>To remove your address from the list, send a message to:
>>> <me...@incubator.apache.org>
>>>
>>>Send mail to the following for info and FAQ for this list:
>>> <me...@incubator.apache.org>
>>> <me...@incubator.apache.org>
>>>
>>>Similar addresses exist for the digest list:
>>> <me...@incubator.apache.org>
>>> <me...@incubator.apache.org>
>>>
>>>To get messages 123 through 145 (a maximum of 100 per request), mail:
>>> <me...@incubator.apache.org>
>>>
>>>To get an index with subject and author for messages 123-456 , mail:
>>> <me...@incubator.apache.org>
>>>
>>>They are always returned as sets of 100, max 2000 per request,
>>>so you'll actually get 100-499.
>>>
>>>To receive all messages with the same subject as message 12345,
>>>send a short message to:
>>> <me...@incubator.apache.org>
>>>
>>>The messages should contain one line or word of text to avoid being
>>>treated as sp@m, but I will ignore their content.
>>>Only the ADDRESS you send to is important.
>>>
>>>You can start a subscription for an alternate address,
>>>for example "john@host.domain", just add a hyphen and your
>>>address (with '=' instead of '@') after the command word:
>>><me...@incubator.apache.org>
>>>
>>>To stop subscription for this address, mail:
>>><me...@incubator.apache.org>
>>>
>>>In both cases, I'll send a confirmation message to that address. When
>>>you receive it, simply reply to it to complete your subscription.
>>>
>>>If despite following these instructions, you do not get the
>>>desired results, please contact my owner at
>>>mesos-dev-owner@incubator.apache.org. Please be patient, my owner is a
>>>lot slower than I am ;-)
>>>
>>>--- Enclosed is a copy of the request I received.
>>>
>>>Return-Path: <zh...@fb.com>
>>>Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07 -0000
>>>Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:07
>>>+0000
>>>X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>> tests=SPF_PASS
>>>X-Spam-Check-By: apache.org
>>>Received-SPF: pass (nike.apache.org: domain of zhiyong@fb.com designates
>>>67.231.145.42 as permitted sender)
>>>Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>(67.231.145.42)
>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:00
>>>+0000
>>>Received: from pps.filterd (m0004348 [127.0.0.1])
>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id p7LNTI6g015638
>>> for
>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.a
>>>p
>>>a
>>>che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>[66.220.144.38])
>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>>> for
>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.a
>>>p
>>>a
>>>che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>Received: from SC-MBX01-5.TheFacebook.com ([fe80::f05e:c7f1:c7dd:e640])
>>>by
>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>14.01.0289.001;
>>>Sun,
>>> 21 Aug 2011 16:29:14 -0700
>>>From: Joe Xie <zh...@fb.com>
>>>To:
>>>"mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.a
>>>p
>>>a
>>>che.org"
>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.
>>>a
>>>p
>>>ache.org>
>>>Subject: Re: confirm subscribe to mesos-dev@incubator.apache.org
>>>Thread-Topic: confirm subscribe to mesos-dev@incubator.apache.org
>>>Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>Message-ID: <CA...@fb.com>
>>>In-Reply-To: <13...@incubator.apache.org>
>>>Accept-Language: en-US
>>>Content-Language: en-US
>>>X-MS-Has-Attach:
>>>X-MS-TNEF-Correlator:
>>>x-originating-ip: [192.168.18.252]
>>>Content-Type: text/plain; charset="us-ascii"
>>>Content-ID: <B4...@fb.com>
>>>Content-Transfer-Encoding: quoted-printable
>>>MIME-Version: 1.0
>>>X-Virus-Checked: Checked by ClamAV on apache.org
>>>
>>>Thanks!
>>>Joe
>>>On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>><me...@incubator.apache.org> wrote:
>>>
>>>>Hi! This is the ezmlm program. I'm managing the
>>>>mesos-dev@incubator.apache.org mailing list.
>>>>
>>>>I'm working for my owner, who can be reached
>>>>at mesos-dev-owner@incubator.apache.org.
>>>>
>>>>To confirm that you would like
>>>>
>>>> zhiyong@fb.com
>>>>
>>>>added to the mesos-dev mailing list, please send
>>>>a short reply to this address:
>>>>
>>>> =20
>>>>mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@incubator
>>>>.
>>>>a
>>>>p=
>>>ac
>>>>he.org
>>>>
>>>>Usually, this happens when you just hit the "reply" button.
>>>>If this does not work, simply copy the address and paste it into
>>>>the "To:" field of a new message.
>>>>
>>>>or click here:
>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@i
>>>>n
>>>>c
>>>>u=
>>>ba
>>>>tor.apache.org
>>>>
>>>>This confirmation serves two purposes. First, it verifies that I am
>>>>able
>>>>to get mail through to you. Second, it protects you in case someone
>>>>forges a subscription request in your name.
>>>>
>>>>Please note that ALL Apache dev- and user- mailing lists are publicly
>>>>archived. Do familiarize yourself with Apache's public archive policy
>>>>at
>>>>
>>>> http://www.apache.org/foundation/public-archives.html
>>>>
>>>>prior to subscribing and posting messages to
>>>>mesos-dev@incubator.apache.org.
>>>>If you're not sure whether or not the policy applies to this mailing
>>>>list,
>>>>assume it does unless the list name contains the word "private" in it.
>>>>
>>>>Some mail programs are broken and cannot handle long addresses. If you
>>>>cannot reply to this request, instead send a message to
>>>><me...@incubator.apache.org> and put the
>>>>entire address listed above into the "Subject:" line.
>>>>
>>>>
>>>>--- Administrative commands for the mesos-dev list ---
>>>>
>>>>I can handle administrative requests automatically. Please
>>>>do not send them to the list address! Instead, send
>>>>your message to the correct command address:
>>>>
>>>>To subscribe to the list, send a message to:
>>>> <me...@incubator.apache.org>
>>>>
>>>>To remove your address from the list, send a message to:
>>>> <me...@incubator.apache.org>
>>>>
>>>>Send mail to the following for info and FAQ for this list:
>>>> <me...@incubator.apache.org>
>>>> <me...@incubator.apache.org>
>>>>
>>>>Similar addresses exist for the digest list:
>>>> <me...@incubator.apache.org>
>>>> <me...@incubator.apache.org>
>>>>
>>>>To get messages 123 through 145 (a maximum of 100 per request), mail:
>>>> <me...@incubator.apache.org>
>>>>
>>>>To get an index with subject and author for messages 123-456 , mail:
>>>> <me...@incubator.apache.org>
>>>>
>>>>They are always returned as sets of 100, max 2000 per request,
>>>>so you'll actually get 100-499.
>>>>
>>>>To receive all messages with the same subject as message 12345,
>>>>send a short message to:
>>>> <me...@incubator.apache.org>
>>>>
>>>>The messages should contain one line or word of text to avoid being
>>>>treated as sp@m, but I will ignore their content.
>>>>Only the ADDRESS you send to is important.
>>>>
>>>>You can start a subscription for an alternate address,
>>>>for example "john@host.domain", just add a hyphen and your
>>>>address (with '=3D' instead of '@') after the command word:
>>>><me...@incubator.apache.org>
>>>>
>>>>To stop subscription for this address, mail:
>>>><me...@incubator.apache.org>
>>>>
>>>>In both cases, I'll send a confirmation message to that address. When
>>>>you receive it, simply reply to it to complete your subscription.
>>>>
>>>>If despite following these instructions, you do not get the
>>>>desired results, please contact my owner at
>>>>mesos-dev-owner@incubator.apache.org. Please be patient, my owner is a
>>>>lot slower than I am ;-)
>>>>
>>>>--- Enclosed is a copy of the request I received.
>>>>
>>>>Return-Path: <zh...@fb.com>
>>>>Received: (qmail 19986 invoked by uid 99); 21 Aug 2011 22:43:26 -0000
>>>>Received: from athena.apache.org (HELO athena.apache.org)
>>>>(140.211.11.136)
>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:26
>>>>+0000
>>>>X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>X-Spam-Check-By: apache.org
>>>>Received-SPF: pass (athena.apache.org: domain of zhiyong@fb.com
>>>>designates 67.231.145.42 as permitted sender)
>>>>Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>(67.231.145.42)
>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:19
>>>>+0000
>>>>Received: from pps.filterd (m0004347 [127.0.0.1])
>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id p7LMcbSq017336
>>>> for <me...@incubator.apache.org>; Sun, 21 Aug 2011
>>>>15:42:58 -0700
>>>>Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>[66.220.144.38])
>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128 verify=3DNOT)
>>>> for <me...@incubator.apache.org>; Sun, 21 Aug 2011
>>>>15:42:58 -0700
>>>>Received: from SC-MBX01-5.TheFacebook.com ([fe80::f05e:c7f1:c7dd:e640])
>>>>by
>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>14.01.0289.001;
>>>>Sun,
>>>> 21 Aug 2011 15:42:53 -0700
>>>>From: Joe Xie <zh...@fb.com>
>>>>To: "mesos-dev-subscribe@incubator.apache.org"
>>>> <me...@incubator.apache.org>
>>>>Subject: can i join this
>>>>Thread-Topic: can i join this
>>>>Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>Message-ID: <CA...@fb.com>
>>>>Accept-Language: en-US
>>>>Content-Language: en-US
>>>>X-MS-Has-Attach:=20
>>>>X-MS-TNEF-Correlator:
>>>>x-originating-ip: [192.168.18.252]
>>>>Content-Type: multipart/alternative;
>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>MIME-Version: 1.0
>>>>
>>>>--_000_CA76D7F74256zhiyongfbcom_
>>>>Content-Type: text/plain; charset=3D"us-ascii"
>>>>Content-Transfer-Encoding: quoted-printable
>>>>
>>>>Best,
>>>>Joe
>>>>
>>>>--_000_CA76D7F74256zhiyongfbcom_
>>>>Content-Type: text/html; charset=3D"us-ascii"
>>>>Content-ID: <F6...@fb.com>
>>>>Content-Transfer-Encoding: quoted-printable
>>>>
>>>><html>
>>>><head>
>>>><meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>>charset=3D3Dus-ascii"=3D
>>>>>
>>>></head>
>>>><body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode: space;
>>>>-webkit-lin=3D
>>>>e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px;
>>>>font-fami=3D
>>>>ly: Calibri, sans-serif; ">
>>>><div>Best,</div>
>>>><div>Joe</div>
>>>></body>
>>>></html>
>>>>
>>>>--_000_CA76D7F74256zhiyongfbcom_--
>>>
>>
>