get schema in StorageFunc

[Stan Rosenberg <sr...@proclivitysystems.com>]

Hi All,

I'd like to get the schema of a relation that is used in conjunction
with my custom StorageFunc.  I found 'checkSchema' to be useful for
this case, however, it seems to work only in local mode.  When run in
distributed mode, 'checkSchema' is not invoked in mappers.
Is there some other means to retrieve the schema?

Thanks in advance,

stan

Re: get schema in StorageFunc

[Dmitriy Ryaboy <dv...@gmail.com>]

Stan, you have to serialize the schema into the job conf using the
signature passed to you in setUDFContextSignature.

your code will look something like this:

@Override

public void checkSchema(ResourceSchema s) throws IOException {

    this.schema = s;

    getUDFProperties().setProperty(contextSignature + "_schema"
, ObjectSerializer.serialize(schema));

}
@Override

public void setUDFContextSignature(String signature) {

    this.contextSignature = signature;

}

@Override

public void setStoreLocation(String location, Job job) throws IOException {

    String serializedSchema = getUDFProperties().getProperty(
contextSignature + "_schema");

    if (serializedSchema!= null) {

        this.schema = (ResourceSchema)
ObjectSerializer.deserialize(serializedSchema);

    }

}




On Mon, Nov 7, 2011 at 7:01 PM, Stan Rosenberg <
srosenberg@proclivitysystems.com> wrote:

> Hi All,
>
> I'd like to get the schema of a relation that is used in conjunction
> with my custom StorageFunc.  I found 'checkSchema' to be useful for
> this case, however, it seems to work only in local mode.  When run in
> distributed mode, 'checkSchema' is not invoked in mappers.
> Is there some other means to retrieve the schema?
>
> Thanks in advance,
>
> stan
>