You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Du Lam <de...@gmail.com> on 2014/10/03 11:34:35 UTC
avro mapreduce output
Hi,
In mapreduce with reduce output format of <NullWritable,
AvroValue<myavroclass>>
Is it possible for me to get the reduce output without the string
"key:none" and "value" , i.e. only the content of myavroclass itself??
Thanks!
what i get now:
{u'value': {u'MYCOLUMN1': 'Hello', u'MYCOLUMN2': 'World'}, u'key': None}
what i actually wanted:
{u'MYCOLUMN1': 'Hello', u'MYCOLUMN2': 'World'}
RE: avro mapreduce output
Posted by java8964 <ja...@hotmail.com>.
Avro data should be a binary format in its own version. Why you got something like JSON?
What output format class you use?
Yong
Date: Fri, 3 Oct 2014 17:34:35 +0800
Subject: avro mapreduce output
From: delim123456@gmail.com
To: user@hadoop.apache.org
Hi,
In mapreduce with reduce output format of <NullWritable, AvroValue<myavroclass>>
Is it possible for me to get the reduce output without the string "key:none" and "value" , i.e. only the content of myavroclass itself??Thanks!
what i get now:{u'value': {u'MYCOLUMN1': 'Hello', u'MYCOLUMN2': 'World'}, u'key': None}what i actually wanted:{u'MYCOLUMN1': 'Hello', u'MYCOLUMN2': 'World'}
RE: avro mapreduce output
Posted by java8964 <ja...@hotmail.com>.
Avro data should be a binary format in its own version. Why you got something like JSON?
What output format class you use?
Yong
Date: Fri, 3 Oct 2014 17:34:35 +0800
Subject: avro mapreduce output
From: delim123456@gmail.com
To: user@hadoop.apache.org
Hi,
In mapreduce with reduce output format of <NullWritable, AvroValue<myavroclass>>
Is it possible for me to get the reduce output without the string "key:none" and "value" , i.e. only the content of myavroclass itself??Thanks!
what i get now:{u'value': {u'MYCOLUMN1': 'Hello', u'MYCOLUMN2': 'World'}, u'key': None}what i actually wanted:{u'MYCOLUMN1': 'Hello', u'MYCOLUMN2': 'World'}
RE: avro mapreduce output
Posted by java8964 <ja...@hotmail.com>.
Avro data should be a binary format in its own version. Why you got something like JSON?
What output format class you use?
Yong
Date: Fri, 3 Oct 2014 17:34:35 +0800
Subject: avro mapreduce output
From: delim123456@gmail.com
To: user@hadoop.apache.org
Hi,
In mapreduce with reduce output format of <NullWritable, AvroValue<myavroclass>>
Is it possible for me to get the reduce output without the string "key:none" and "value" , i.e. only the content of myavroclass itself??Thanks!
what i get now:{u'value': {u'MYCOLUMN1': 'Hello', u'MYCOLUMN2': 'World'}, u'key': None}what i actually wanted:{u'MYCOLUMN1': 'Hello', u'MYCOLUMN2': 'World'}
RE: avro mapreduce output
Posted by java8964 <ja...@hotmail.com>.
Avro data should be a binary format in its own version. Why you got something like JSON?
What output format class you use?
Yong
Date: Fri, 3 Oct 2014 17:34:35 +0800
Subject: avro mapreduce output
From: delim123456@gmail.com
To: user@hadoop.apache.org
Hi,
In mapreduce with reduce output format of <NullWritable, AvroValue<myavroclass>>
Is it possible for me to get the reduce output without the string "key:none" and "value" , i.e. only the content of myavroclass itself??Thanks!
what i get now:{u'value': {u'MYCOLUMN1': 'Hello', u'MYCOLUMN2': 'World'}, u'key': None}what i actually wanted:{u'MYCOLUMN1': 'Hello', u'MYCOLUMN2': 'World'}