You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pig.apache.org by Apache Wiki <wi...@apache.org> on 2010/11/24 03:51:16 UTC

[Pig Wiki] Update of "PigStorageWithInputPath" by daijy

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.

The "PigStorageWithInputPath" page has been changed by daijy.
http://wiki.apache.org/pig/PigStorageWithInputPath

--------------------------------------------------

New page:
= Use Case =
When you want to load a globbing, but want to know which file you are working on during the loading. Eg, 

{{{
A = load '*.txt' using PigStorageWithInputPath();
}}}

You want to know which exact file you are loading in your getNext().

= Solution =
Here is a code snip to get the file name:

{{{
public class PigStorageWithInputPath extends PigStorage {
    Path path = null;

    @Override
    public void prepareToRead(RecordReader reader, PigSplit split) {
        super.prepareToRead(reader, split);
        path = ((FileSplit)split.getWrappedSplit()).getPath();
    }

    @Override
    public Tuple getNext() throws IOException {
        Tuple myTuple = super.getNext();
        if (myTuple != null)
            myTuple.append(path.toString());
        return myTuple;
    }
}

}}}