You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by ja...@accenture.com on 2017/05/09 06:49:51 UTC
RDD.cacheDataSet() not working intermittently
Hi,
I have a scenario in which I am caching my RDDs for future use. But I observed that when I use my RDD, complete DAG is re-executed and RDD gets created again.
How can I avoid this scenario and make sure that RDD.cacheDataSet() caches RDD every time.
Regards,
Jasbir Singh
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com
RDD.cacheDataSet() not working intermittently
Posted by ja...@accenture.com.
Hi,
I have a scenario in which I am caching my RDDs for future use. But I observed that when I use my RDD, complete DAG is re-executed and RDD gets created again.
How can I avoid this scenario and make sure that RDD.cacheDataSet() caches RDD every time.
Regards,
Jasbir Singh
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com
RDD.cacheDataSet() not working intermittently
Posted by ja...@accenture.com.
Hi,
I have a scenario in which I am caching my RDDs for future use. But I observed that when I use my RDD, complete DAG is re-executed and RDD gets created again.
How can I avoid this scenario and make sure that RDD.cacheDataSet() caches RDD every time.
Regards,
Jasbir Singh
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com