is the temporary output's sequence id stable in when pipeline runs every time

[陈竞 <cj...@gmail.com>]

i found out that crunch will give any temporary output an sequence id,which
is generated when construct the data graph. my problem is that: is the
temporary output's sequence id stable in when pipeline runs every time?

Re: is the temporary output's sequence id stable in when pipeline runs every time

[陈竞 <cj...@gmail.com>]

i saw the source code:

public class Graph implements Iterable<Vertex> {

    private final Map<PCollectionImpl, Vertex> vertices;
    private final Map<Pair<Vertex, Vertex>, Edge> edges;
    private final Map<Vertex, List<Vertex>> dependencies;

PCollectionImpl use the default hashcode(), and the vertices is
materialized by HashSet, so the iterator order of Graph maybe vary
every time,

which will make temporay table's id varies every time.


2016-09-23 13:59 GMT+08:00 Josh Wills <jo...@gmail.com>:

> I don't think we guarantee stability, no, though we do our best to support
> it in most cases.
>
> On Thu, Sep 22, 2016 at 10:24 PM 陈竞 <cj...@gmail.com> wrote:
>
>> i found out that crunch will give any temporary output an sequence
>> id,which is generated when construct the data graph. my problem is that: is
>> the temporary output's sequence id stable in when pipeline runs every time?
>>
>


-- 
陈竞，中科院计算技术研究所，高性能计算机中心
Jing Chen HPCC.ICT.AC China

Re: is the temporary output's sequence id stable in when pipeline runs every time

[Josh Wills <jo...@gmail.com>]

I don't think we guarantee stability, no, though we do our best to support
it in most cases.
On Thu, Sep 22, 2016 at 10:24 PM 陈竞 <cj...@gmail.com> wrote:

> i found out that crunch will give any temporary output an sequence
> id,which is generated when construct the data graph. my problem is that: is
> the temporary output's sequence id stable in when pipeline runs every time?
>