You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Michael Moores <mm...@real.com> on 2010/10/15 00:49:27 UTC
ColumnFamilyRecordReader issue
My Hadoop TaskTracker is using the Cassandra CplumnFamilyInputFormat, and appears to be finding records (the data is serialized below in the log output),
but the cassandra Column class is throwing a validation exception indicating "Required field 'clock' was not present!".
My Cassandra cluster version is 0.7-beta2 and i also loaded the same JAR in my hadoop configuration.
So does this point to my client side (hadoop task) possibly using the wrong cassandra version?
MapAttempt TASK_TYPE="MAP" TASKID="task_201010140951_0020_m_000006" TASK_ATTEMPT_ID="attempt_201010140951_0020_m_000006_3" START_TIME="1287095149722" TRACKER_NAME="tracker_kv-app06\.dev\.real\.com:localhost\.localdomain/127\.0\.0\.1:54957" HTTP_PORT="50060" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201010140951_0020_m_000006" TASK_ATTEMPT_ID="attempt_201010140951_0020_m_000006_3" TASK_STATUS="FAILED" FINISH_TIME="1287095154592" HOSTNAME="kv-app06\.dev\.real\.com" ERROR="java\.lang\.RuntimeException: org\.apache\.thrift\.protocol\.TProtocolException: Required field 'clock' was not present! Struct: Column(name:69 64, value:6D 65 73 73 61 67 65 2D 70 72 65 66 65 72 65 6E 63 65 73, clock:null)
at org\.apache\.cassandra\.hadoop\.ColumnFamilyRecordReader$RowIterator\.maybeInit(ColumnFamilyRecordReader\.java:266)
at org\.apache\.cassandra\.hadoop\.ColumnFamilyRecordReader$RowIterator\.computeNext(ColumnFamilyRecordReader\.java:281)
at org\.apache\.cassandra\.hadoop\.ColumnFamilyRecordReader$RowIterator\.computeNext(ColumnFamilyRecordReader\.java:178)
at com\.google\.common\.collect\.AbstractIterator\.tryToComputeNext(AbstractIterator\.java:136)
at com\.google\.common\.collect\.AbstractIterator\.hasNext(AbstractIterator\.java:131)
at org\.apache\.cassandra\.hadoop\.ColumnFamilyRecordReader\.nextKeyValue(ColumnFamilyRecordReader\.java:137)
at org\.apache\.hadoop\.mapred\.MapTask$NewTrackingRecordReader\.nextKeyValue(MapTask\.java:423)
at org\.apache\.hadoop\.mapreduce\.MapContext\.nextKeyValue(MapContext\.java:67)
at org\.apache\.hadoop\.mapreduce\.Mapper\.run(Mapper\.java:143)
at org\.apache\.hadoop\.mapred\.MapTask\.runNewMapper(MapTask\.java:621)
at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:305)
at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
Caused by: org\.apache\.thrift\.protocol\.TProtocolException: Required field 'clock' was not present! Struct: Column(name:69 64, value:6D 65 73 73 61 67 65 2D 70 72 65 66 65 72 65 6E 63 65 73, clock:null)
at org\.apache\.cassandra\.thrift\.Column\.validate(Column\.java:615)
at org\.apache\.cassandra\.thrift\.Column\.read(Column\.java:527)
at org\.apache\.cassandra\.thrift\.ColumnOrSuperColumn\.read(ColumnOrSuperColumn\.java:351)
at org\.apache\.cassandra\.thrift\.KeySlice\.read(KeySlice\.java:391)
at org\.apache\.cassandra\.thrift\.Cassandra$get_range_slices_result\.read(Cassandra\.java:11859)
at org\.apache\.cassandra\.thrift\.Cassandra$Client\.recv_get_range_slices(Cassandra\.java:743)
at org\.apache\.cassandra\.thrift\.Cassandra$Client\.get_range_slices(Cassandra\.java:715)
at org\.apache\.cassandra\.hadoop\.ColumnFamilyRecordReader$RowIterator\.maybeInit(ColumnFamilyRecordReader\.java:244)
Re: ColumnFamilyRecordReader issue
Posted by Aaron Morton <aa...@thelastpickle.com>.
Clock was present in beta 1 and then removed. The Beta2 thrift client does not have this check in it.
Double check your install and make sure it's all beta 2.
Aaron
On 15 Oct, 2010,at 11:49 AM, Michael Moores <mm...@real.com> wrote:
My Hadoop TaskTracker is using the Cassandra CplumnFamilyInputFormat, and appears to be finding records (the data is serialized below in the log output),
but the cassandra Column class is throwing a validation exception indicating "Required field 'clock' was not present!".
My Cassandra cluster version is 0.7-beta2 and i also loaded the same JAR in my hadoop configuration.
So does this point to my client side (hadoop task) possibly using the wrong cassandra version?
MapAttempt TASK_TYPE="MAP" TASKID="task_201010140951_0020_m_000006" TASK_ATTEMPT_ID="attempt_201010140951_0020_m_000006_3" START_TIME="1287095149722" TRACKER_NAME="tracker_kv-app06\.dev\.real\.com:localhost\.localdomain/127\.0\.0\.1:54957" HTTP_PORT="50060" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201010140951_0020_m_000006" TASK_ATTEMPT_ID="attempt_201010140951_0020_m_000006_3" TASK_STATUS="FAILED" FINISH_TIME="1287095154592" HOSTNAME="kv-app06\.dev\.real\.com" ERROR="java\.lang\.RuntimeException: org\.apache\.thrift\.protocol\.TProtocolException: Required field 'clock' was not present! Struct: Column(name:69 64, value:6D 65 73 73 61 67 65 2D 70 72 65 66 65 72 65 6E 63 65 73, clock:null)
at org\.apache\.cassandra\.hadoop\.ColumnFamilyRecordReader$RowIterator\.maybeInit(ColumnFamilyRecordReader\.java:266)
at org\.apache\.cassandra\.hadoop\.ColumnFamilyRecordReader$RowIterator\.computeNext(ColumnFamilyRecordReader\.java:281)
at org\.apache\.cassandra\.hadoop\.ColumnFamilyRecordReader$RowIterator\.computeNext(ColumnFamilyRecordReader\.java:178)
at com\.google\.common\.collect\.AbstractIterator\.tryToComputeNext(AbstractIterator\.java:136)
at com\.google\.common\.collect\.AbstractIterator\hasNext(AbstractIterator\.java:131)
at org\.apache\.cassandra\.hadoop\.ColumnFamilyRecordReader\.nextKeyValue(ColumnFamilyRecordReader\.java:137)
at org\.apache\.hadoop\.mapred\.MapTask$NewTrackingRecordReader\.nextKeyValue(MapTask\.java:423)
at org\.apache\.hadoop\.mapreduce\.MapContext\nextKeyValue(MapContext\.java:67)
at org\.apache\.hadoop\.mapreduce\.Mapper\.run(Mapper\.java:143)
at org\.apache\.hadoop\.mapred\.MapTask\.runNewMapper(MapTask\.java:621)
at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:305)
at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
Caused by: org\.apache\.thrift\.protocol\.TProtocolException: Required field 'clock' was not present! Struct: Column(name:69 64, value:6D 65 73 73 61 67 65 2D 70 72 65 66 65 72 65 6E 63 65 73, clock:null)
at org\.apache\.cassandra\.thrift\.Column\.validate(Column\.java:615)
at org\.apache\.cassandra\.thrift\.Column\.read(Column\.java:527)
at org\.apache\.cassandra\.thrift\.ColumnOrSuperColumn\.read(ColumnOrSuperColumn\.java:351)
at org\.apache\.cassandra\.thrift\.KeySlice\.read(KeySlice\.java:391)
at org\.apache\.cassandra\.thrift\.Cassandra$get_range_slices_result\.read(Cassandra\.java:11859)
at org\.apache\.cassandra\.thrift\.Cassandra$Client\.recv_get_range_slices(Cassandra\.java:743)
at org\.apache\.cassandra\.thrift\.Cassandra$Client\.get_range_slices(Cassandra\.java:715)
at org\.apache\.cassandra\.hadoop\.ColumnFamilyRecordReader$RowIterator\.maybeInit(ColumnFamilyRecordReader\.java:244)