You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Deep Pradhan <pr...@gmail.com> on 2014/09/24 17:21:03 UTC
RDD of Iterable[String]
Can we iterate over RDD of Iterable[String]? How do we do that?
Because the entire Iterable[String] seems to be a single element in the RDD.
Thank You
Re: RDD of Iterable[String]
Posted by nir <ni...@gmail.com>.
Have you found answer to this? I am also looking for exact same solution.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/RDD-of-Iterable-String-tp15016p23329.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: RDD of Iterable[String]
Posted by Liquan Pei <li...@gmail.com>.
Hi Deep,
I believe that you are referring to the map for Iterable[String]
suppose you have
iter:Iterable[String]
you can do
newIter = iter.map(item => Item + "a" )
which will create an new Iterable[String] with each element appending an
"a" to all string in iter.
Does this answer your question?
Liquan
On Thu, Sep 25, 2014 at 12:43 AM, Deep Pradhan <pr...@gmail.com>
wrote:
> what should come in the map??
> Thanks Liquan for answering me...
> I really need some help..I am stuck in some thing.
>
>
> On Thu, Sep 25, 2014 at 12:57 PM, Deep Pradhan <pr...@gmail.com>
> wrote:
>
>> what should come in the map??
>>
>> On Wed, Sep 24, 2014 at 10:52 PM, Liquan Pei <li...@gmail.com> wrote:
>>
>>> Hi Deep,
>>>
>>> The Iterable trait in scala has methods like map and reduce that you can
>>> use to iterate elements of Iterable[String]. You can also create an
>>> Iterator from the Iterable. For example, suppose you have
>>>
>>> val rdd: RDD[Iterable[String]]
>>>
>>> you can do
>>>
>>> rdd.map { x => //x has type Iterable[String]
>>> x.map(...) // Process elements in iterable[String]
>>> val iter:Iterator[String] = x.iterator
>>> while(iter.hasNext) {
>>> iter.next()
>>> }
>>> }
>>>
>>> Hope this helps!
>>>
>>> Liquan
>>>
>>> On Wed, Sep 24, 2014 at 8:21 AM, Deep Pradhan <pradhandeep1991@gmail.com
>>> > wrote:
>>>
>>>> Can we iterate over RDD of Iterable[String]? How do we do that?
>>>> Because the entire Iterable[String] seems to be a single element in the
>>>> RDD.
>>>>
>>>> Thank You
>>>>
>>>
>>>
>>>
>>> --
>>> Liquan Pei
>>> Department of Physics
>>> University of Massachusetts Amherst
>>>
>>
>>
>
--
Liquan Pei
Department of Physics
University of Massachusetts Amherst
Re: RDD of Iterable[String]
Posted by Liquan Pei <li...@gmail.com>.
Hi Deep,
The Iterable trait in scala has methods like map and reduce that you can
use to iterate elements of Iterable[String]. You can also create an
Iterator from the Iterable. For example, suppose you have
val rdd: RDD[Iterable[String]]
you can do
rdd.map { x => //x has type Iterable[String]
x.map(...) // Process elements in iterable[String]
val iter:Iterator[String] = x.iterator
while(iter.hasNext) {
iter.next()
}
}
Hope this helps!
Liquan
On Wed, Sep 24, 2014 at 8:21 AM, Deep Pradhan <pr...@gmail.com>
wrote:
> Can we iterate over RDD of Iterable[String]? How do we do that?
> Because the entire Iterable[String] seems to be a single element in the
> RDD.
>
> Thank You
>
--
Liquan Pei
Department of Physics
University of Massachusetts Amherst