You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Ranga <sr...@gmail.com> on 2015/03/17 23:45:05 UTC
StorageLevel: OFF_HEAP
Hi
I am trying to use the OFF_HEAP storage level in my Spark (1.2.1) cluster.
The Tachyon (0.6.0-SNAPSHOT) nodes seem to be up and running. However, when
I try to persist the RDD, I get the following error:
ERROR [2015-03-16 22:22:54,017] ({Executor task launch worker-0}
TachyonFS.java[connect]:364) - Invalid method name:
'getUserUnderfsTempFolder'
ERROR [2015-03-16 22:22:54,050] ({Executor task launch worker-0}
TachyonFS.java[getFileId]:1020) - Invalid method name: 'user_getFileId'
Is this because of a version mis-match?
On a different note, I was wondering if Tachyon has been used in a
production environment by anybody in this group?
Appreciate your help with this.
- Ranga
Re: StorageLevel: OFF_HEAP
Posted by Ranga <sr...@gmail.com>.
Thanks Ted. Will do.
On Wed, Mar 18, 2015 at 2:27 PM, Ted Yu <yu...@gmail.com> wrote:
> Ranga:
> Please apply the patch from:
> https://github.com/apache/spark/pull/4867
>
> And rebuild Spark - the build would use Tachyon-0.6.1
>
> Cheers
>
> On Wed, Mar 18, 2015 at 2:23 PM, Ranga <sr...@gmail.com> wrote:
>
>> Hi Haoyuan
>>
>> No. I assumed that Spark-1.3.0 was already built with Tachyon-0.6.0. If
>> not, I can rebuild and try. Could you let me know how to rebuild with 0.6.0?
>> Thanks for your help.
>>
>>
>> - Ranga
>>
>> On Wed, Mar 18, 2015 at 12:59 PM, Haoyuan Li <ha...@gmail.com>
>> wrote:
>>
>>> Did you recompile it with Tachyon 0.6.0?
>>>
>>> Also, Tachyon 0.6.1 has been released this morning:
>>> http://tachyon-project.org/ ; https://github.com/amplab/tachyon/releases
>>>
>>> Best regards,
>>>
>>> Haoyuan
>>>
>>> On Wed, Mar 18, 2015 at 11:48 AM, Ranga <sr...@gmail.com> wrote:
>>>
>>>> I just tested with Spark-1.3.0 + Tachyon-0.6.0 and still see the same
>>>> issue. Here are the logs:
>>>> 15/03/18 11:44:11 ERROR : Invalid method name: 'getDataFolder'
>>>> 15/03/18 11:44:11 ERROR : Invalid method name: 'user_getFileId'
>>>> 15/03/18 11:44:11 ERROR storage.TachyonBlockManager: Failed 10 attempts
>>>> to create tachyon dir in
>>>> /tmp_spark_tachyon/spark-e3538a20-5e42-48a4-ad67-4b97aded90e4/<driver>
>>>>
>>>> Thanks for any other pointers.
>>>>
>>>>
>>>> - Ranga
>>>>
>>>>
>>>>
>>>> On Wed, Mar 18, 2015 at 9:53 AM, Ranga <sr...@gmail.com> wrote:
>>>>
>>>>> Thanks for the information. Will rebuild with 0.6.0 till the patch is
>>>>> merged.
>>>>>
>>>>> On Tue, Mar 17, 2015 at 7:24 PM, Ted Yu <yu...@gmail.com> wrote:
>>>>>
>>>>>> Ranga:
>>>>>> Take a look at https://github.com/apache/spark/pull/4867
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>> On Tue, Mar 17, 2015 at 6:08 PM, fightfate@163.com <fightfate@163.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hi， Ranga
>>>>>>>
>>>>>>> That's true. Typically a version mis-match issue. Note that spark
>>>>>>> 1.2.1 has tachyon built in with version 0.5.0 , I think you may need to
>>>>>>> rebuild spark
>>>>>>> with your current tachyon release.
>>>>>>> We had used tachyon for several of our spark projects in a
>>>>>>> production environment.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Sun.
>>>>>>>
>>>>>>> ------------------------------
>>>>>>> fightfate@163.com
>>>>>>>
>>>>>>>
>>>>>>> *From:* Ranga <sr...@gmail.com>
>>>>>>> *Date:* 2015-03-18 06:45
>>>>>>> *To:* user@spark.apache.org
>>>>>>> *Subject:* StorageLevel: OFF_HEAP
>>>>>>> Hi
>>>>>>>
>>>>>>> I am trying to use the OFF_HEAP storage level in my Spark (1.2.1)
>>>>>>> cluster. The Tachyon (0.6.0-SNAPSHOT) nodes seem to be up and running.
>>>>>>> However, when I try to persist the RDD, I get the following error:
>>>>>>>
>>>>>>> ERROR [2015-03-16 22:22:54,017] ({Executor task launch worker-0}
>>>>>>> TachyonFS.java[connect]:364) - Invalid method name:
>>>>>>> 'getUserUnderfsTempFolder'
>>>>>>> ERROR [2015-03-16 22:22:54,050] ({Executor task launch worker-0}
>>>>>>> TachyonFS.java[getFileId]:1020) - Invalid method name: 'user_getFileId'
>>>>>>>
>>>>>>> Is this because of a version mis-match?
>>>>>>>
>>>>>>> On a different note, I was wondering if Tachyon has been used in a
>>>>>>> production environment by anybody in this group?
>>>>>>>
>>>>>>> Appreciate your help with this.
>>>>>>>
>>>>>>>
>>>>>>> - Ranga
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Haoyuan Li
>>> AMPLab, EECS, UC Berkeley
>>> http://www.cs.berkeley.edu/~haoyuan/
>>>
>>
>>
>
Re: StorageLevel: OFF_HEAP
Posted by Ted Yu <yu...@gmail.com>.
Ranga:
Please apply the patch from:
https://github.com/apache/spark/pull/4867
And rebuild Spark - the build would use Tachyon-0.6.1
Cheers
On Wed, Mar 18, 2015 at 2:23 PM, Ranga <sr...@gmail.com> wrote:
> Hi Haoyuan
>
> No. I assumed that Spark-1.3.0 was already built with Tachyon-0.6.0. If
> not, I can rebuild and try. Could you let me know how to rebuild with 0.6.0?
> Thanks for your help.
>
>
> - Ranga
>
> On Wed, Mar 18, 2015 at 12:59 PM, Haoyuan Li <ha...@gmail.com> wrote:
>
>> Did you recompile it with Tachyon 0.6.0?
>>
>> Also, Tachyon 0.6.1 has been released this morning:
>> http://tachyon-project.org/ ; https://github.com/amplab/tachyon/releases
>>
>> Best regards,
>>
>> Haoyuan
>>
>> On Wed, Mar 18, 2015 at 11:48 AM, Ranga <sr...@gmail.com> wrote:
>>
>>> I just tested with Spark-1.3.0 + Tachyon-0.6.0 and still see the same
>>> issue. Here are the logs:
>>> 15/03/18 11:44:11 ERROR : Invalid method name: 'getDataFolder'
>>> 15/03/18 11:44:11 ERROR : Invalid method name: 'user_getFileId'
>>> 15/03/18 11:44:11 ERROR storage.TachyonBlockManager: Failed 10 attempts
>>> to create tachyon dir in
>>> /tmp_spark_tachyon/spark-e3538a20-5e42-48a4-ad67-4b97aded90e4/<driver>
>>>
>>> Thanks for any other pointers.
>>>
>>>
>>> - Ranga
>>>
>>>
>>>
>>> On Wed, Mar 18, 2015 at 9:53 AM, Ranga <sr...@gmail.com> wrote:
>>>
>>>> Thanks for the information. Will rebuild with 0.6.0 till the patch is
>>>> merged.
>>>>
>>>> On Tue, Mar 17, 2015 at 7:24 PM, Ted Yu <yu...@gmail.com> wrote:
>>>>
>>>>> Ranga:
>>>>> Take a look at https://github.com/apache/spark/pull/4867
>>>>>
>>>>> Cheers
>>>>>
>>>>> On Tue, Mar 17, 2015 at 6:08 PM, fightfate@163.com <fi...@163.com>
>>>>> wrote:
>>>>>
>>>>>> Hi， Ranga
>>>>>>
>>>>>> That's true. Typically a version mis-match issue. Note that spark
>>>>>> 1.2.1 has tachyon built in with version 0.5.0 , I think you may need to
>>>>>> rebuild spark
>>>>>> with your current tachyon release.
>>>>>> We had used tachyon for several of our spark projects in a production
>>>>>> environment.
>>>>>>
>>>>>> Thanks,
>>>>>> Sun.
>>>>>>
>>>>>> ------------------------------
>>>>>> fightfate@163.com
>>>>>>
>>>>>>
>>>>>> *From:* Ranga <sr...@gmail.com>
>>>>>> *Date:* 2015-03-18 06:45
>>>>>> *To:* user@spark.apache.org
>>>>>> *Subject:* StorageLevel: OFF_HEAP
>>>>>> Hi
>>>>>>
>>>>>> I am trying to use the OFF_HEAP storage level in my Spark (1.2.1)
>>>>>> cluster. The Tachyon (0.6.0-SNAPSHOT) nodes seem to be up and running.
>>>>>> However, when I try to persist the RDD, I get the following error:
>>>>>>
>>>>>> ERROR [2015-03-16 22:22:54,017] ({Executor task launch worker-0}
>>>>>> TachyonFS.java[connect]:364) - Invalid method name:
>>>>>> 'getUserUnderfsTempFolder'
>>>>>> ERROR [2015-03-16 22:22:54,050] ({Executor task launch worker-0}
>>>>>> TachyonFS.java[getFileId]:1020) - Invalid method name: 'user_getFileId'
>>>>>>
>>>>>> Is this because of a version mis-match?
>>>>>>
>>>>>> On a different note, I was wondering if Tachyon has been used in a
>>>>>> production environment by anybody in this group?
>>>>>>
>>>>>> Appreciate your help with this.
>>>>>>
>>>>>>
>>>>>> - Ranga
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> Haoyuan Li
>> AMPLab, EECS, UC Berkeley
>> http://www.cs.berkeley.edu/~haoyuan/
>>
>
>
Re: StorageLevel: OFF_HEAP
Posted by Ranga <sr...@gmail.com>.
Hi Haoyuan
No. I assumed that Spark-1.3.0 was already built with Tachyon-0.6.0. If
not, I can rebuild and try. Could you let me know how to rebuild with 0.6.0?
Thanks for your help.
- Ranga
On Wed, Mar 18, 2015 at 12:59 PM, Haoyuan Li <ha...@gmail.com> wrote:
> Did you recompile it with Tachyon 0.6.0?
>
> Also, Tachyon 0.6.1 has been released this morning:
> http://tachyon-project.org/ ; https://github.com/amplab/tachyon/releases
>
> Best regards,
>
> Haoyuan
>
> On Wed, Mar 18, 2015 at 11:48 AM, Ranga <sr...@gmail.com> wrote:
>
>> I just tested with Spark-1.3.0 + Tachyon-0.6.0 and still see the same
>> issue. Here are the logs:
>> 15/03/18 11:44:11 ERROR : Invalid method name: 'getDataFolder'
>> 15/03/18 11:44:11 ERROR : Invalid method name: 'user_getFileId'
>> 15/03/18 11:44:11 ERROR storage.TachyonBlockManager: Failed 10 attempts
>> to create tachyon dir in
>> /tmp_spark_tachyon/spark-e3538a20-5e42-48a4-ad67-4b97aded90e4/<driver>
>>
>> Thanks for any other pointers.
>>
>>
>> - Ranga
>>
>>
>>
>> On Wed, Mar 18, 2015 at 9:53 AM, Ranga <sr...@gmail.com> wrote:
>>
>>> Thanks for the information. Will rebuild with 0.6.0 till the patch is
>>> merged.
>>>
>>> On Tue, Mar 17, 2015 at 7:24 PM, Ted Yu <yu...@gmail.com> wrote:
>>>
>>>> Ranga:
>>>> Take a look at https://github.com/apache/spark/pull/4867
>>>>
>>>> Cheers
>>>>
>>>> On Tue, Mar 17, 2015 at 6:08 PM, fightfate@163.com <fi...@163.com>
>>>> wrote:
>>>>
>>>>> Hi， Ranga
>>>>>
>>>>> That's true. Typically a version mis-match issue. Note that spark
>>>>> 1.2.1 has tachyon built in with version 0.5.0 , I think you may need to
>>>>> rebuild spark
>>>>> with your current tachyon release.
>>>>> We had used tachyon for several of our spark projects in a production
>>>>> environment.
>>>>>
>>>>> Thanks,
>>>>> Sun.
>>>>>
>>>>> ------------------------------
>>>>> fightfate@163.com
>>>>>
>>>>>
>>>>> *From:* Ranga <sr...@gmail.com>
>>>>> *Date:* 2015-03-18 06:45
>>>>> *To:* user@spark.apache.org
>>>>> *Subject:* StorageLevel: OFF_HEAP
>>>>> Hi
>>>>>
>>>>> I am trying to use the OFF_HEAP storage level in my Spark (1.2.1)
>>>>> cluster. The Tachyon (0.6.0-SNAPSHOT) nodes seem to be up and running.
>>>>> However, when I try to persist the RDD, I get the following error:
>>>>>
>>>>> ERROR [2015-03-16 22:22:54,017] ({Executor task launch worker-0}
>>>>> TachyonFS.java[connect]:364) - Invalid method name:
>>>>> 'getUserUnderfsTempFolder'
>>>>> ERROR [2015-03-16 22:22:54,050] ({Executor task launch worker-0}
>>>>> TachyonFS.java[getFileId]:1020) - Invalid method name: 'user_getFileId'
>>>>>
>>>>> Is this because of a version mis-match?
>>>>>
>>>>> On a different note, I was wondering if Tachyon has been used in a
>>>>> production environment by anybody in this group?
>>>>>
>>>>> Appreciate your help with this.
>>>>>
>>>>>
>>>>> - Ranga
>>>>>
>>>>>
>>>>
>>>
>>
>
>
> --
> Haoyuan Li
> AMPLab, EECS, UC Berkeley
> http://www.cs.berkeley.edu/~haoyuan/
>
Re: StorageLevel: OFF_HEAP
Posted by Haoyuan Li <ha...@gmail.com>.
Did you recompile it with Tachyon 0.6.0?
Also, Tachyon 0.6.1 has been released this morning:
http://tachyon-project.org/ ; https://github.com/amplab/tachyon/releases
Best regards,
Haoyuan
On Wed, Mar 18, 2015 at 11:48 AM, Ranga <sr...@gmail.com> wrote:
> I just tested with Spark-1.3.0 + Tachyon-0.6.0 and still see the same
> issue. Here are the logs:
> 15/03/18 11:44:11 ERROR : Invalid method name: 'getDataFolder'
> 15/03/18 11:44:11 ERROR : Invalid method name: 'user_getFileId'
> 15/03/18 11:44:11 ERROR storage.TachyonBlockManager: Failed 10 attempts to
> create tachyon dir in
> /tmp_spark_tachyon/spark-e3538a20-5e42-48a4-ad67-4b97aded90e4/<driver>
>
> Thanks for any other pointers.
>
>
> - Ranga
>
>
>
> On Wed, Mar 18, 2015 at 9:53 AM, Ranga <sr...@gmail.com> wrote:
>
>> Thanks for the information. Will rebuild with 0.6.0 till the patch is
>> merged.
>>
>> On Tue, Mar 17, 2015 at 7:24 PM, Ted Yu <yu...@gmail.com> wrote:
>>
>>> Ranga:
>>> Take a look at https://github.com/apache/spark/pull/4867
>>>
>>> Cheers
>>>
>>> On Tue, Mar 17, 2015 at 6:08 PM, fightfate@163.com <fi...@163.com>
>>> wrote:
>>>
>>>> Hi， Ranga
>>>>
>>>> That's true. Typically a version mis-match issue. Note that spark 1.2.1
>>>> has tachyon built in with version 0.5.0 , I think you may need to rebuild
>>>> spark
>>>> with your current tachyon release.
>>>> We had used tachyon for several of our spark projects in a production
>>>> environment.
>>>>
>>>> Thanks,
>>>> Sun.
>>>>
>>>> ------------------------------
>>>> fightfate@163.com
>>>>
>>>>
>>>> *From:* Ranga <sr...@gmail.com>
>>>> *Date:* 2015-03-18 06:45
>>>> *To:* user@spark.apache.org
>>>> *Subject:* StorageLevel: OFF_HEAP
>>>> Hi
>>>>
>>>> I am trying to use the OFF_HEAP storage level in my Spark (1.2.1)
>>>> cluster. The Tachyon (0.6.0-SNAPSHOT) nodes seem to be up and running.
>>>> However, when I try to persist the RDD, I get the following error:
>>>>
>>>> ERROR [2015-03-16 22:22:54,017] ({Executor task launch worker-0}
>>>> TachyonFS.java[connect]:364) - Invalid method name:
>>>> 'getUserUnderfsTempFolder'
>>>> ERROR [2015-03-16 22:22:54,050] ({Executor task launch worker-0}
>>>> TachyonFS.java[getFileId]:1020) - Invalid method name: 'user_getFileId'
>>>>
>>>> Is this because of a version mis-match?
>>>>
>>>> On a different note, I was wondering if Tachyon has been used in a
>>>> production environment by anybody in this group?
>>>>
>>>> Appreciate your help with this.
>>>>
>>>>
>>>> - Ranga
>>>>
>>>>
>>>
>>
>
--
Haoyuan Li
AMPLab, EECS, UC Berkeley
http://www.cs.berkeley.edu/~haoyuan/
Re: StorageLevel: OFF_HEAP
Posted by Ranga <sr...@gmail.com>.
I just tested with Spark-1.3.0 + Tachyon-0.6.0 and still see the same
issue. Here are the logs:
15/03/18 11:44:11 ERROR : Invalid method name: 'getDataFolder'
15/03/18 11:44:11 ERROR : Invalid method name: 'user_getFileId'
15/03/18 11:44:11 ERROR storage.TachyonBlockManager: Failed 10 attempts to
create tachyon dir in
/tmp_spark_tachyon/spark-e3538a20-5e42-48a4-ad67-4b97aded90e4/<driver>
Thanks for any other pointers.
- Ranga
On Wed, Mar 18, 2015 at 9:53 AM, Ranga <sr...@gmail.com> wrote:
> Thanks for the information. Will rebuild with 0.6.0 till the patch is
> merged.
>
> On Tue, Mar 17, 2015 at 7:24 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> Ranga:
>> Take a look at https://github.com/apache/spark/pull/4867
>>
>> Cheers
>>
>> On Tue, Mar 17, 2015 at 6:08 PM, fightfate@163.com <fi...@163.com>
>> wrote:
>>
>>> Hi， Ranga
>>>
>>> That's true. Typically a version mis-match issue. Note that spark 1.2.1
>>> has tachyon built in with version 0.5.0 , I think you may need to rebuild
>>> spark
>>> with your current tachyon release.
>>> We had used tachyon for several of our spark projects in a production
>>> environment.
>>>
>>> Thanks,
>>> Sun.
>>>
>>> ------------------------------
>>> fightfate@163.com
>>>
>>>
>>> *From:* Ranga <sr...@gmail.com>
>>> *Date:* 2015-03-18 06:45
>>> *To:* user@spark.apache.org
>>> *Subject:* StorageLevel: OFF_HEAP
>>> Hi
>>>
>>> I am trying to use the OFF_HEAP storage level in my Spark (1.2.1)
>>> cluster. The Tachyon (0.6.0-SNAPSHOT) nodes seem to be up and running.
>>> However, when I try to persist the RDD, I get the following error:
>>>
>>> ERROR [2015-03-16 22:22:54,017] ({Executor task launch worker-0}
>>> TachyonFS.java[connect]:364) - Invalid method name:
>>> 'getUserUnderfsTempFolder'
>>> ERROR [2015-03-16 22:22:54,050] ({Executor task launch worker-0}
>>> TachyonFS.java[getFileId]:1020) - Invalid method name: 'user_getFileId'
>>>
>>> Is this because of a version mis-match?
>>>
>>> On a different note, I was wondering if Tachyon has been used in a
>>> production environment by anybody in this group?
>>>
>>> Appreciate your help with this.
>>>
>>>
>>> - Ranga
>>>
>>>
>>
>
Re: StorageLevel: OFF_HEAP
Posted by Ranga <sr...@gmail.com>.
Thanks for the information. Will rebuild with 0.6.0 till the patch is
merged.
On Tue, Mar 17, 2015 at 7:24 PM, Ted Yu <yu...@gmail.com> wrote:
> Ranga:
> Take a look at https://github.com/apache/spark/pull/4867
>
> Cheers
>
> On Tue, Mar 17, 2015 at 6:08 PM, fightfate@163.com <fi...@163.com>
> wrote:
>
>> Hi， Ranga
>>
>> That's true. Typically a version mis-match issue. Note that spark 1.2.1
>> has tachyon built in with version 0.5.0 , I think you may need to rebuild
>> spark
>> with your current tachyon release.
>> We had used tachyon for several of our spark projects in a production
>> environment.
>>
>> Thanks,
>> Sun.
>>
>> ------------------------------
>> fightfate@163.com
>>
>>
>> *From:* Ranga <sr...@gmail.com>
>> *Date:* 2015-03-18 06:45
>> *To:* user@spark.apache.org
>> *Subject:* StorageLevel: OFF_HEAP
>> Hi
>>
>> I am trying to use the OFF_HEAP storage level in my Spark (1.2.1)
>> cluster. The Tachyon (0.6.0-SNAPSHOT) nodes seem to be up and running.
>> However, when I try to persist the RDD, I get the following error:
>>
>> ERROR [2015-03-16 22:22:54,017] ({Executor task launch worker-0}
>> TachyonFS.java[connect]:364) - Invalid method name:
>> 'getUserUnderfsTempFolder'
>> ERROR [2015-03-16 22:22:54,050] ({Executor task launch worker-0}
>> TachyonFS.java[getFileId]:1020) - Invalid method name: 'user_getFileId'
>>
>> Is this because of a version mis-match?
>>
>> On a different note, I was wondering if Tachyon has been used in a
>> production environment by anybody in this group?
>>
>> Appreciate your help with this.
>>
>>
>> - Ranga
>>
>>
>
Re: StorageLevel: OFF_HEAP
Posted by Ted Yu <yu...@gmail.com>.
Ranga:
Take a look at https://github.com/apache/spark/pull/4867
Cheers
On Tue, Mar 17, 2015 at 6:08 PM, fightfate@163.com <fi...@163.com>
wrote:
> Hi， Ranga
>
> That's true. Typically a version mis-match issue. Note that spark 1.2.1
> has tachyon built in with version 0.5.0 , I think you may need to rebuild
> spark
> with your current tachyon release.
> We had used tachyon for several of our spark projects in a production
> environment.
>
> Thanks,
> Sun.
>
> ------------------------------
> fightfate@163.com
>
>
> *From:* Ranga <sr...@gmail.com>
> *Date:* 2015-03-18 06:45
> *To:* user@spark.apache.org
> *Subject:* StorageLevel: OFF_HEAP
> Hi
>
> I am trying to use the OFF_HEAP storage level in my Spark (1.2.1) cluster.
> The Tachyon (0.6.0-SNAPSHOT) nodes seem to be up and running. However, when
> I try to persist the RDD, I get the following error:
>
> ERROR [2015-03-16 22:22:54,017] ({Executor task launch worker-0}
> TachyonFS.java[connect]:364) - Invalid method name:
> 'getUserUnderfsTempFolder'
> ERROR [2015-03-16 22:22:54,050] ({Executor task launch worker-0}
> TachyonFS.java[getFileId]:1020) - Invalid method name: 'user_getFileId'
>
> Is this because of a version mis-match?
>
> On a different note, I was wondering if Tachyon has been used in a
> production environment by anybody in this group?
>
> Appreciate your help with this.
>
>
> - Ranga
>
>
Re: StorageLevel: OFF_HEAP
Posted by "fightfate@163.com" <fi...@163.com>.
Hi， Ranga
That's true. Typically a version mis-match issue. Note that spark 1.2.1 has tachyon built in with version 0.5.0 , I think you may need to rebuild spark
with your current tachyon release.
We had used tachyon for several of our spark projects in a production environment.
Thanks,
Sun.
fightfate@163.com
From: Ranga
Date: 2015-03-18 06:45
To: user@spark.apache.org
Subject: StorageLevel: OFF_HEAP
Hi
I am trying to use the OFF_HEAP storage level in my Spark (1.2.1) cluster. The Tachyon (0.6.0-SNAPSHOT) nodes seem to be up and running. However, when I try to persist the RDD, I get the following error:
ERROR [2015-03-16 22:22:54,017] ({Executor task launch worker-0} TachyonFS.java[connect]:364) - Invalid method name: 'getUserUnderfsTempFolder'
ERROR [2015-03-16 22:22:54,050] ({Executor task launch worker-0} TachyonFS.java[getFileId]:1020) - Invalid method name: 'user_getFileId'
Is this because of a version mis-match?
On a different note, I was wondering if Tachyon has been used in a production environment by anybody in this group?
Appreciate your help with this.
- Ranga