Why TableNamespaceManager#upsert abort master if failing to update zookeeper

[Stephen Jiang <sy...@gmail.com>]

I am working on moving namespace DDL to Proc-V2 and come to see this code.
Looks like the logic exists since St.Ack implemented namespace in
HBASE-8408.

I just wonder whether the failure of zookeeper update is serve enough to
shut down master (in my new code, I don't abort - I want to make sure that
the abort is not needed - note the upsert would fail if zookeep update
fails and rollback in my new code - so we will not have orphaned ns table
row)

  private void upsert(Table table, NamespaceDescriptor ns) throws
IOException {

    validateTableAndRegionCount(ns);

    Put p = new Put(Bytes.toBytes(ns.getName()));

    p.addImmutable(HTableDescriptor.NAMESPACE_FAMILY_INFO_BYTES,

        HTableDescriptor.NAMESPACE_COL_DESC_BYTES,

        ProtobufUtil.toProtoNamespaceDescriptor(ns).toByteArray());

    table.put(p);

    try {

      zkNamespaceManager.update(ns);

    } catch(IOException ex) {

      String msg = "Failed to update namespace information in ZK. Aborting."
;

      LOG.fatal(msg, ex);

      masterServices.abort(msg, ex);

    }

  }

Re: Why TableNamespaceManager#upsert abort master if failing to update zookeeper

[Matteo Bertozzi <th...@gmail.com>]

I think the master is aborting because the listeners on that zknode will
have the wrong information.
so aborting was probably a shortcut to say, something went wrong we don't
know what to do. hope that on restart and maybe via hbck the state will be
fixed.

with proc-v2 you don't need to abort, but you need to rollback or at least
keep retrying until that znode is updated before declaring that that
operation was completed successfully.


Matteo


On Fri, Jul 17, 2015 at 4:24 PM, Stephen Jiang <sy...@gmail.com>
wrote:

> I am working on moving namespace DDL to Proc-V2 and come to see this code.
> Looks like the logic exists since St.Ack implemented namespace in
> HBASE-8408.
>
> I just wonder whether the failure of zookeeper update is serve enough to
> shut down master (in my new code, I don't abort - I want to make sure that
> the abort is not needed - note the upsert would fail if zookeep update
> fails and rollback in my new code - so we will not have orphaned ns table
> row)
>
>   private void upsert(Table table, NamespaceDescriptor ns) throws
> IOException {
>
>     validateTableAndRegionCount(ns);
>
>     Put p = new Put(Bytes.toBytes(ns.getName()));
>
>     p.addImmutable(HTableDescriptor.NAMESPACE_FAMILY_INFO_BYTES,
>
>         HTableDescriptor.NAMESPACE_COL_DESC_BYTES,
>
>         ProtobufUtil.toProtoNamespaceDescriptor(ns).toByteArray());
>
>     table.put(p);
>
>     try {
>
>       zkNamespaceManager.update(ns);
>
>     } catch(IOException ex) {
>
>       String msg = "Failed to update namespace information in ZK.
> Aborting."
> ;
>
>       LOG.fatal(msg, ex);
>
>       masterServices.abort(msg, ex);
>
>     }
>
>   }
>