You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pig.apache.org by Apache Wiki <wi...@apache.org> on 2010/11/24 03:51:16 UTC
[Pig Wiki] Update of "PigStorageWithInputPath" by daijy
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.
The "PigStorageWithInputPath" page has been changed by daijy.
http://wiki.apache.org/pig/PigStorageWithInputPath
--------------------------------------------------
New page:
= Use Case =
When you want to load a globbing, but want to know which file you are working on during the loading. Eg,
{{{
A = load '*.txt' using PigStorageWithInputPath();
}}}
You want to know which exact file you are loading in your getNext().
= Solution =
Here is a code snip to get the file name:
{{{
public class PigStorageWithInputPath extends PigStorage {
Path path = null;
@Override
public void prepareToRead(RecordReader reader, PigSplit split) {
super.prepareToRead(reader, split);
path = ((FileSplit)split.getWrappedSplit()).getPath();
}
@Override
public Tuple getNext() throws IOException {
Tuple myTuple = super.getNext();
if (myTuple != null)
myTuple.append(path.toString());
return myTuple;
}
}
}}}