You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by "Vikas M.B." <vi...@gmail.com> on 2014/08/14 23:41:07 UTC

How to write metadata only if store job is successful

Hi,

     One of the use case in my custom storage UDF  is to write certain
metadata to HDFS location only if a store job has been successful.

So, I tried to do this in the cleanupOnSuccess() method overridden from
StoreFunc class.

In particular, i am setting certain properties in UDFContext’s
UDFProperties object in setStoreLocation method.

In the  cleanupOnSuccess()  method, when I retrieve the same UDFProperties
object, it is empty. I am retrieving the UDFProperties object from
UDFContext based on the UDF class name in both cases.


So, my question is whether the same UDFProperties object would exist until
the invocation of cleanupOnSuccess() method.
If not, is there any better way to achieve the use case of writing metadata
to HDFS only if a store job has been successful?

Thanks,
Vikas