Role History Question

[Rui Zhang <rz...@vertica.com>]

Hi, everyone,

Does Role History always ensure the nodes will restart on the same 
machines? If it is, do I have a way to know what these machines are or I 
have to use other tools such as ZooKeeper to do this?

This is important for my development, because Vertica stores catalog and 
data on those machines if the role history does not ensure this, we will 
lose data when restarting.

Thanks,

-- 
Rui Zhang
Software engineer Intern
Vertica, an HP Company
rzhang@vertica.com

Re: Role History Question

[Steve Loughran <st...@hortonworks.com>]

On 30 July 2014 17:34, Rui Zhang <rz...@vertica.com> wrote:

> Hi, everyone,
>
> Does Role History always ensure the nodes will restart on the same
> machines? If it is, do I have a way to know what these machines are or I
> have to use other tools such as ZooKeeper to do this?
>
>
it's best effort ... we record what is happening and when trying to add new
nodes we try to reposition nodes on those machines if

1. there's no instance of that role already on the machine
2. the machine hasn't been unreliable when launching nodes

Logic for "unreliable" is a bit minimal right now; we have a startup window
and if the launch fails in that the node is placed at the end of the queue
of nodes.

The requests are made to YARN as "best-effort placement" and not "mandatory
placement" -YARN may make other decisions. Mandatory placement is tricky as
implies no assignment until the target host is live and with space. The
flag can't be mixed with "best-effort placement" in containers with the
same priority.



> This is important for my development, because Vertica stores catalog and
> data on those machines if the role history does not ensure this, we will
> lose data when restarting.
>

We've been thinking about how to do some snapshotting of local data to
HDFS; other apps need it too. Would this work? The problem here is that we
can't guarantee clean container shutdown, hence container snapshot at
shutdown



>
> Thanks,
>
> --
> Rui Zhang
> Software engineer Intern
> Vertica, an HP Company
> rzhang@vertica.com
>
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.