You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Ido Hadanny (Created) (JIRA)" <ji...@apache.org> on 2012/03/01 18:44:00 UTC
[jira] [Created] (PIG-2567) CSVLoader does not interpret empty
fields to NULLs
CSVLoader does not interpret empty fields to NULLs
--------------------------------------------------
Key: PIG-2567
URL: https://issues.apache.org/jira/browse/PIG-2567
Project: Pig
Issue Type: Bug
Components: piggybank
Affects Versions: 0.8.1
Reporter: Ido Hadanny
Priority: Minor
when specifying an empty field like so "a,,b"
CSVLoader turns them into empty strings.
while PigStorage converts them to nulls.
PigStorage:
private void readField(byte[] buf, int start, int end) {
if (start == end) {
// NULL value
mProtoTuple.add(null);
} else {
mProtoTuple.add(new DataByteArray(buf, start, end));
}
}
CSVLoader:
private void readField(ByteBuffer buf, int fieldID) {
158
if (mRequiredColumns==null || (mRequiredColumns.length>fieldID && mRequiredColumns[fieldID])) {
159
byte[] bytes = new byte[buf.position()];
160
buf.rewind();
161
buf.get(bytes, 0, bytes.length);
162
mProtoTuple.add(new DataByteArray(bytes));
163
}
164
buf.clear();
165
}
166
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2567) CSVLoader does not interpret empty
fields to NULLs
Posted by "Anuroopa George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478959#comment-13478959 ]
Anuroopa George commented on PIG-2567:
--------------------------------------
Can you please post the script which you used as well as the output your are getting?
> CSVLoader does not interpret empty fields to NULLs
> --------------------------------------------------
>
> Key: PIG-2567
> URL: https://issues.apache.org/jira/browse/PIG-2567
> Project: Pig
> Issue Type: Bug
> Components: piggybank
> Affects Versions: 0.8.1
> Reporter: Ido Hadanny
> Priority: Minor
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> when specifying an empty field like so "a,,b"
> CSVLoader turns them into empty strings.
> while PigStorage converts them to nulls.
> PigStorage:
> private void readField(byte[] buf, int start, int end) {
> if (start == end) {
> // NULL value
> mProtoTuple.add(null);
> } else {
> mProtoTuple.add(new DataByteArray(buf, start, end));
> }
> }
> CSVLoader:
> private void readField(ByteBuffer buf, int fieldID) {
> 158
> if (mRequiredColumns==null || (mRequiredColumns.length>fieldID && mRequiredColumns[fieldID])) {
> 159
> byte[] bytes = new byte[buf.position()];
> 160
> buf.rewind();
> 161
> buf.get(bytes, 0, bytes.length);
> 162
> mProtoTuple.add(new DataByteArray(bytes));
> 163
> }
> 164
> buf.clear();
> 165
> }
> 166
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira