You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by 萝卜丝炒饭 <14...@qq.com> on 2017/06/15 08:11:27 UTC
the dependence length of RDD, can its size be greater than 1 pleaae?
Hi all,
The RDD code keeps a member as below:
dependencies_ : seq[Dependency[_]]
It is a seq, that means it can keep more than one dependency.
I have an issue about this.
Is it possible that its size is greater than one please?
If yes, how to produce it please? Would you like show me some code please?
thanks
Robin Shao
Re: the dependence length of RDD, can its size be greater than 1 pleaae?
Posted by 萝卜丝炒饭 <14...@qq.com>.
Hi Owen,
More issues about this topic.
Is two the up limit od dependency please?
In the code, firstParent RDD is used to get partitions.Why is firstparent RDD used please?
If first parent RDD has 5 partitions and sevond has 6, is it reasonable to use first please?
thanks
---Original---
From: "Sean Owen"<so...@cloudera.com>
Date: 2017/6/15 16:13:11
To: "user"<us...@qq.com>;
Subject: Re: the dependence length of RDD, can its size be greater than 1 pleaae?
Yes. Imagine an RDD that results from a union of other RDDs.
On Thu, Jun 15, 2017, 09:11 萝卜丝炒饭 <14...@qq.com> wrote:
Hi all,
The RDD code keeps a member as below:
dependencies_ : seq[Dependency[_]]
It is a seq, that means it can keep more than one dependency.
I have an issue about this.
Is it possible that its size is greater than one please?
If yes, how to produce it please? Would you like show me some code please?
thanks
Robin Shao
Re: the dependence length of RDD, can its size be greater than 1 pleaae?
Posted by 萝卜丝炒饭 <14...@qq.com>.
Hi Owen,
More issues about this topic.
Is two the up limit od dependency please?
In the code, firstParent RDD is used to get partitions.Why is firstparent RDD used please?
If first parent RDD has 5 partitions and sevond has 6, is it reasonable to use first please?
thanks
---Original---
From: "Sean Owen"<so...@cloudera.com>
Date: 2017/6/15 16:13:11
To: "user"<us...@qq.com>;
Subject: Re: the dependence length of RDD, can its size be greater than 1 pleaae?
Yes. Imagine an RDD that results from a union of other RDDs.
On Thu, Jun 15, 2017, 09:11 萝卜丝炒饭 <14...@qq.com> wrote:
Hi all,
The RDD code keeps a member as below:
dependencies_ : seq[Dependency[_]]
It is a seq, that means it can keep more than one dependency.
I have an issue about this.
Is it possible that its size is greater than one please?
If yes, how to produce it please? Would you like show me some code please?
thanks
Robin Shao
Re: the dependence length of RDD, can its size be greater than 1 pleaae?
Posted by Sean Owen <so...@cloudera.com>.
Yes. Imagine an RDD that results from a union of other RDDs.
On Thu, Jun 15, 2017, 09:11 萝卜丝炒饭 <14...@qq.com> wrote:
> Hi all,
>
> The RDD code keeps a member as below:
> dependencies_ : seq[Dependency[_]]
>
> It is a seq, that means it can keep more than one dependency.
>
> I have an issue about this.
> Is it possible that its size is greater than one please?
> If yes, how to produce it please? Would you like show me some code please?
>
> thanks
> Robin Shao
>
Re: the dependence length of RDD, can its size be greater than 1 pleaae?
Posted by Reynold Xin <rx...@databricks.com>.
A join?
On Thu, Jun 15, 2017 at 1:11 AM 萝卜丝炒饭 <14...@qq.com> wrote:
> Hi all,
>
> The RDD code keeps a member as below:
> dependencies_ : seq[Dependency[_]]
>
> It is a seq, that means it can keep more than one dependency.
>
> I have an issue about this.
> Is it possible that its size is greater than one please?
> If yes, how to produce it please? Would you like show me some code please?
>
> thanks
> Robin Shao
>
Re: the dependence length of RDD, can its size be greater than 1 pleaae?
Posted by Sean Owen <so...@cloudera.com>.
Yes. Imagine an RDD that results from a union of other RDDs.
On Thu, Jun 15, 2017, 09:11 萝卜丝炒饭 <14...@qq.com> wrote:
> Hi all,
>
> The RDD code keeps a member as below:
> dependencies_ : seq[Dependency[_]]
>
> It is a seq, that means it can keep more than one dependency.
>
> I have an issue about this.
> Is it possible that its size is greater than one please?
> If yes, how to produce it please? Would you like show me some code please?
>
> thanks
> Robin Shao
>
Re: the dependence length of RDD, can its size be greater than 1 pleaae?
Posted by Reynold Xin <rx...@databricks.com>.
A join?
On Thu, Jun 15, 2017 at 1:11 AM 萝卜丝炒饭 <14...@qq.com> wrote:
> Hi all,
>
> The RDD code keeps a member as below:
> dependencies_ : seq[Dependency[_]]
>
> It is a seq, that means it can keep more than one dependency.
>
> I have an issue about this.
> Is it possible that its size is greater than one please?
> If yes, how to produce it please? Would you like show me some code please?
>
> thanks
> Robin Shao
>