You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Xuelin Cao <xu...@gmail.com> on 2015/01/13 06:50:57 UTC
How to create an empty RDD with a given type?
Hi,
I'd like to create a transform function, that convert RDD[String] to
RDD[Int]
Occasionally, the input RDD could be an empty RDD. I just want to
directly create an empty RDD[Int] if the input RDD is empty. And, I don't
want to return None as the result.
Is there an easy way to do that?
Re: How to create an empty RDD with a given type?
Posted by Sean Owen <so...@cloudera.com>.
PS you will probably have to cast to get the type RDD[Int]:
https://issues.apache.org/jira/browse/SPARK-2331
On Tue, Jan 13, 2015 at 5:50 AM, Xuelin Cao <xu...@gmail.com> wrote:
>
>
> Hi,
>
> I'd like to create a transform function, that convert RDD[String] to
> RDD[Int]
>
> Occasionally, the input RDD could be an empty RDD. I just want to
> directly create an empty RDD[Int] if the input RDD is empty. And, I don't
> want to return None as the result.
>
> Is there an easy way to do that?
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: How to create an empty RDD with a given type?
Posted by Xuelin Cao <xu...@gmail.com>.
Got it, thanks!
On Tue, Jan 13, 2015 at 2:00 PM, Justin Yip <yi...@gmail.com> wrote:
> Xuelin,
>
> There is a function called emtpyRDD under spark context
> <http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.SparkContext> which
> serves this purpose.
>
> Justin
>
> On Mon, Jan 12, 2015 at 9:50 PM, Xuelin Cao <xu...@gmail.com>
> wrote:
>
>>
>>
>> Hi,
>>
>> I'd like to create a transform function, that convert RDD[String] to
>> RDD[Int]
>>
>> Occasionally, the input RDD could be an empty RDD. I just want to
>> directly create an empty RDD[Int] if the input RDD is empty. And, I don't
>> want to return None as the result.
>>
>> Is there an easy way to do that?
>>
>>
>>
>
Re: How to create an empty RDD with a given type?
Posted by Justin Yip <yi...@gmail.com>.
Xuelin,
There is a function called emtpyRDD under spark context
<http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.SparkContext>
which
serves this purpose.
Justin
On Mon, Jan 12, 2015 at 9:50 PM, Xuelin Cao <xu...@gmail.com> wrote:
>
>
> Hi,
>
> I'd like to create a transform function, that convert RDD[String] to
> RDD[Int]
>
> Occasionally, the input RDD could be an empty RDD. I just want to
> directly create an empty RDD[Int] if the input RDD is empty. And, I don't
> want to return None as the result.
>
> Is there an easy way to do that?
>
>
>