You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by sauyuktha shetty <sh...@gmail.com> on 2015/11/23 08:35:01 UTC

Fwd: Doubt in DoubleWritable

Hi,

I am new to mapreduce. This is my first map reduce project and may look
naive. Please guide me regarding this issue I am facing.

I was trying to execute a MapReduce code that involved calculating average
values.

I found that Hadoop does not have a DoubleArrayWritabloe type and hence
extended ArrayWritable to create DoubleArrayWritable type. When I try to do
a get on it though it returns an object of type Writable rather than
DoubleWritable, which I am unable to typecast.

Below is the snippet of my code and error message.

ubuntu@ip-172-31-6-72:~/extended/data$ javac -classpath
"/home/ubuntu/extended/hadoop-2.4.0-64bit/hadoop-common/hadoop-dist/target/hadoop-2.4.0//etc/hadoop:/home/ubuntu/extended/hadoop-2.4.0-64bit/hadoop-common/hadoop-dist/target/hadoop-2.4.0//share/hadoop/common/lib/*:/home/ubuntu/extended/hadoop-2.4.0-64bit/hadoop-common/hadoop-dist/target/hadoop-2.4.0//share/hadoop/common/*:/home/ubuntu/extended/hadoop-2.4.0-64bit/hadoop-common/hadoop-dist/target/hadoop-2.4.0//share/hadoop/hdfs:/home/ubuntu/extended/hadoop-2.4.0-64bit/hadoop-common/hadoop-dist/target/hadoop-2.4.0//share/hadoop/hdfs/lib/*:/home/ubuntu/extended/hadoop-2.4.0-64bit/hadoop-common/hadoop-dist/target/hadoop-2.4.0//share/hadoop/hdfs/*:/home/ubuntu/extended/hadoop-2.4.0-64bit/hadoop-common/hadoop-dist/target/hadoop-2.4.0//share/hadoop/yarn/lib/*:/home/ubuntu/extended/hadoop-2.4.0-64bit/hadoop-common/hadoop-dist/target/hadoop-2.4.0//share/hadoop/yarn/*:/home/ubuntu/extended/hadoop-2.4.0-64bit/hadoop-common/hadoop-dist/target/hadoop-2.4.0//share/hadoop/mapreduce/lib/*:/home/ubuntu/extended/hadoop-2.4.0-64bit/hadoop-common/hadoop-dist/target/hadoop-2.4.0//share/hadoop/mapreduce/*:/home/ubuntu/extended/hadoop-2.4.0-64bit/hadoop-common/hadoop-dist/target/hadoop-2.4.0//contrib/capacity-scheduler/*.jar;."
MapReduce.java  NewMapper.java DoubleArrayWritable.java NewReducer.java -d
output
NewMapper.java:15: error: method does not override or implement a method
from a supertype
  @Override
  ^
NewReducer.java:34: error: incomparable types: Writable and int
      if (value[1]!=0)
                  ^
NewReducer.java:37: error: bad operand types for binary operator '+'
          sumvalueTemp = sumvalueTemp + value[0];
                                      ^
  first type:  double
  second type: Writable
NewReducer.java:39: error: incomparable types: Writable and int
      if (value[3]!=0)
                  ^
NewReducer.java:42: error: bad operand types for binary operator '+'
          sumvalueDewpoint = sumvalueDewpoint + value[2];
                                              ^
  first type:  double
  second type: Writable
NewReducer.java:44: error: incomparable types: Writable and int
      if (value[5]!=0)
                  ^
NewReducer.java:47: error: bad operand types for binary operator '+'
          sumvalueWindspeed = sumvalueWindspeed + value[4];
                                                ^
  first type:  double
  second type: Writable
7 errors

Regards,
Sauyuktha

Re: Doubt in DoubleWritable

Posted by unmesha sreeveni <un...@gmail.com>.

Please try this
for (DoubleArrayWritable avalue : values) {
Writable[] value = avalue.get();
// DoubleWritable[] value = new DoubleWritable[6];
// for(int k=0;k<6;k++){
// value[k] = DoubleWritable(wvalue[k]);
// }
//parse accordingly
if (Double.parseDouble(value[1].toString()) != 0) {
total_records_Temp = total_records_Temp + 1;
sumvalueTemp = sumvalueTemp + Double.parseDouble(value[0].toString());
}
if (Double.parseDouble(value[3].toString()) != 0) {
total_records_Dewpoint = total_records_Dewpoint + 1;
sumvalueDewpoint = sumvalueDewpoint +
Double.parseDouble(value[2].toString());
}
if (Double.parseDouble(value[5].toString()) != 0) {
total_records_Windspeed = total_records_Windspeed + 1;
sumvalueWindspeed = sumvalueWindspeed +
Double.parseDouble(value[4].toString());
}
}
Attaching the code


-- 
*Thanks & Regards *


*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/

Re: Doubt in DoubleWritable

Posted by unmesha sreeveni <un...@gmail.com>.

Please try this
for (DoubleArrayWritable avalue : values) {
Writable[] value = avalue.get();
// DoubleWritable[] value = new DoubleWritable[6];
// for(int k=0;k<6;k++){
// value[k] = DoubleWritable(wvalue[k]);
// }
//parse accordingly
if (Double.parseDouble(value[1].toString()) != 0) {
total_records_Temp = total_records_Temp + 1;
sumvalueTemp = sumvalueTemp + Double.parseDouble(value[0].toString());
}
if (Double.parseDouble(value[3].toString()) != 0) {
total_records_Dewpoint = total_records_Dewpoint + 1;
sumvalueDewpoint = sumvalueDewpoint +
Double.parseDouble(value[2].toString());
}
if (Double.parseDouble(value[5].toString()) != 0) {
total_records_Windspeed = total_records_Windspeed + 1;
sumvalueWindspeed = sumvalueWindspeed +
Double.parseDouble(value[4].toString());
}
}
Attaching the code


-- 
*Thanks & Regards *


*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/

Re: Doubt in DoubleWritable

Posted by unmesha sreeveni <un...@gmail.com>.

Please try this
for (DoubleArrayWritable avalue : values) {
Writable[] value = avalue.get();
// DoubleWritable[] value = new DoubleWritable[6];
// for(int k=0;k<6;k++){
// value[k] = DoubleWritable(wvalue[k]);
// }
//parse accordingly
if (Double.parseDouble(value[1].toString()) != 0) {
total_records_Temp = total_records_Temp + 1;
sumvalueTemp = sumvalueTemp + Double.parseDouble(value[0].toString());
}
if (Double.parseDouble(value[3].toString()) != 0) {
total_records_Dewpoint = total_records_Dewpoint + 1;
sumvalueDewpoint = sumvalueDewpoint +
Double.parseDouble(value[2].toString());
}
if (Double.parseDouble(value[5].toString()) != 0) {
total_records_Windspeed = total_records_Windspeed + 1;
sumvalueWindspeed = sumvalueWindspeed +
Double.parseDouble(value[4].toString());
}
}
Attaching the code


-- 
*Thanks & Regards *


*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/

Re: Doubt in DoubleWritable

Posted by unmesha sreeveni <un...@gmail.com>.

Please try this
for (DoubleArrayWritable avalue : values) {
Writable[] value = avalue.get();
// DoubleWritable[] value = new DoubleWritable[6];
// for(int k=0;k<6;k++){
// value[k] = DoubleWritable(wvalue[k]);
// }
//parse accordingly
if (Double.parseDouble(value[1].toString()) != 0) {
total_records_Temp = total_records_Temp + 1;
sumvalueTemp = sumvalueTemp + Double.parseDouble(value[0].toString());
}
if (Double.parseDouble(value[3].toString()) != 0) {
total_records_Dewpoint = total_records_Dewpoint + 1;
sumvalueDewpoint = sumvalueDewpoint +
Double.parseDouble(value[2].toString());
}
if (Double.parseDouble(value[5].toString()) != 0) {
total_records_Windspeed = total_records_Windspeed + 1;
sumvalueWindspeed = sumvalueWindspeed +
Double.parseDouble(value[4].toString());
}
}
Attaching the code


-- 
*Thanks & Regards *


*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/