Set comparison

[SK <sk...@gmail.com>]

Hi,

I have a Spark method that returns RDD[String], which I am converting to a
set and then comparing it to the expected output as shown in the following
code. 

1. val expected_res = Set("ID1", "ID2", "ID3")  // expected output

2. val result:RDD[String] = getData(input)  //method returns RDD[String]
3. val set_val = result.collect().toSet     // convert to set. 
4. println(set_val)                                 // prints:  Set("ID1",
"ID2", "ID3")
5. println(expected_res)                        // prints:  Set(ID1, ID2,
ID3)

// verify output
6. if( set_val == expected_res)
7.    println("true")                          // this does not get printed

The value returned by the method is almost same as expected output, but the
verification is failing. I am not sure why the expected_res in Line 5 does
not print the quotes even though Line 1 has them. Could  that be the reason
the comparison is failing? What is the right way to do the above comparison?

thanks





--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Set-comparison-tp7696.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Set comparison

[Ye Xianjin <ad...@gmail.com>]

If you want string with quotes, you have to escape it with '\'. It's exactly what you did in the modified version.

Sent from my iPhone

> On 2014年6月17日, at 5:43, SK <sk...@gmail.com> wrote:
> 
> In Line 1, I have expected_res as a set of strings with quotes. So I thought
> it would include the quotes during comparison.
> 
> Anyway I modified expected_res = Set("\"ID1\"", "\"ID2\"", "\"ID3\"") and
> that seems to work.
> 
> thanks.
> 
> 
> 
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Set-comparison-tp7696p7699.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Set comparison

[SK <sk...@gmail.com>]

In Line 1, I have expected_res as a set of strings with quotes. So I thought
it would include the quotes during comparison.

Anyway I modified expected_res = Set("\"ID1\"", "\"ID2\"", "\"ID3\"") and
that seems to work.

thanks.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Set-comparison-tp7696p7699.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Set comparison

[Sean Owen <so...@cloudera.com>]

On Mon, Jun 16, 2014 at 10:09 PM, SK <sk...@gmail.com> wrote:
>
> The value returned by the method is almost same as expected output, but the
> verification is failing. I am not sure why the expected_res in Line 5 does
> not print the quotes even though Line 1 has them. Could  that be the reason
> the comparison is failing? What is the right way to do the above
> comparison?
>
>
Most likely because the strings in set_val do actually start and end with
double-quotes? That is just what the output suggests. It has a string like
"ID1", not ID1