You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Chris Riccomini <cr...@linkedin.com> on 2009/05/26 18:43:25 UTC

StoreFunc with schema

Hi All,

I¹m trying to write a StoreFunc to store data using Voldemort Serialization
(http://project-voldemort.com/). This serialization involves storing a json
map (describing the schema) in the metadata section of the HDFS file.

Unfortunately, the StoreFunc interface just gives me:

bindTo(OutputStream os)

I have no way (that I can tell) to get the HDFS file path that I need. I
really don¹t want to pass the path as a constructor argument if I can avoid
it.

As best as I can tell, PigRecordWriter is taking filename, and creating an
output stream for me. I¹d rather have the file path (like LoadFunc).

Any thoughts/suggestions?

Thanks in advance!
Chris

Re: StoreFunc with schema

Posted by Alan Gates <ga...@yahoo-inc.com>.
There is work being done to change this, see https://issues.apache.org/jira/browse/PIG-652

Alan.

On May 26, 2009, at 9:59 AM, Chris Riccomini wrote:

> Hi All,
>
> I’m trying to write a StoreFunc to store data using Voldemort  
> Serialization
> (http://project-voldemort.com/). This serialization involves storing  
> a json
> map (describing the schema) in the metadata section of the HDFS file.
>
> Unfortunately, the StoreFunc interface just gives me:
>
> bindTo(OutputStream os)
>
> I have no way (that I can tell) to get the HDFS file path that I  
> need. I
> really don’t want to pass the path as a constructor argument if I  
> can avoid
> it.
>
> As best as I can tell, PigRecordWriter is taking filename, and  
> creating an
> output stream for me. I’d rather have the file path (like LoadFunc).
>
> Any thoughts/suggestions?
>
> Thanks in advance!
> Chris

StoreFunc with schema

Posted by Chris Riccomini <cr...@linkedin.com>.
Hi All,

I¹m trying to write a StoreFunc to store data using Voldemort Serialization
(http://project-voldemort.com/). This serialization involves storing a json
map (describing the schema) in the metadata section of the HDFS file.

Unfortunately, the StoreFunc interface just gives me:

bindTo(OutputStream os)

I have no way (that I can tell) to get the HDFS file path that I need. I
really don¹t want to pass the path as a constructor argument if I can avoid
it.

As best as I can tell, PigRecordWriter is taking filename, and creating an
output stream for me. I¹d rather have the file path (like LoadFunc).

Any thoughts/suggestions?

Thanks in advance!
Chris