Pig filter against flatten column

[Badrinarayanan S <ba...@fifthgentech.com>]

Hi, 

 

I am trying to run a filter against a column which is the result of a
flatten operation. But the filter clause throws an exception as
org.apache.pig.data.DataByteArray cannot be cast to java.lang.String.  The
exception is against the line doing the matches filter. If I change matches
to eq, I am not getting the exception and I don't get any result though I
have ColumnName having 'Page'.

 

Suspect the datatype of the ColumnName (which is result of flatten) of the
relation VisitPages is still bytearray. I have tried casting it to chararray
still same exception. However if I describe of VisitDetails it shows as
chararray. Any suggestions?

 

Below for the pig script:

 

Visits = LOAD 'cassandra://test/Visits' USING CassandraStorage() as (Id,
DetailsBag:bag{DetailsTuple:tuple(VisitTimestamp:chararray,
VisitDetails:bag{VisitColumns:tuple(ColumnName:chararray,
ColumnValue:chararray)})});

VisitsFlattened = FOREACH Visits GENERATE Id,
FLATTEN(DetailsBag.VisitDetails);

VisitDetailsFlattened = FOREACH VisitsFlattened GENERATE Id,
FLATTEN(VisitDetails);

VisitDetails = FOREACH VisitDetailsFlattened GENERATE (chararray)ColumnName,
(chararray)ColumnValue, (chararray)Id;

DESCRIBE VisitDetails;

VisitPages = FILTER VisitDetails BY (ColumnName MATCHES 'Page');

dump VisitPages;

 

......

 

 

Thanks,

badri



Disclaimer: This message (including any attachments) is being sent from Fifth Generation Technologies India (P) Ltd. (5G) and may contain information that is proprietary, confidential and privileged. If you are not the intended recipient, please inform the sender immediately by reply e-mail and delete this message and attachments from your system, without retaining a copy. Any unauthorized use or dissemination of this message in whole or in part is strictly prohibited. 5G shall  not be liable for the improper or incomplete transmission of the information contained in this  communication nor for any delay in its receipt or damage to your system. 5G does not guarantee that the integrity of this communication has been maintained nor that this communication is free of viruses, interceptions or interference.