You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by "Kali.tummala@gmail.com" <Ka...@gmail.com> on 2015/10/12 13:28:33 UTC
Is there any better way of writing this code
Hi All,
just wonderign is there any better way of writing this below code, I am new
to spark an I feel what I wrote is pretty simple and basic and straight
forward is there any better way of writing using functional paradigm.
val QuoteRDD=quotefile.map(x => x.split("\\|")).
filter(line => line(0).contains("1017")).
map(x => ((x(5)+x(4)) , (x(5),x(4),x(1) ,
if (x(15) =="B")
(
{if (x(25) == "") x(9) else x(25)},
{if (x(37) == "") x(11) else x(37)}
)
else if (x(15) =="C" )
(
{if (x(24) == "") (x(9)) else x(24)},
{if (x(30) == "") (x(11)) else x(30)}
)
else if (x(15) =="A")
{(x(9),x(11))}
)))
Thanks
Sri
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Is-there-any-better-way-of-writing-this-code-tp25027.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Is there any better way of writing this code
Posted by Sean Owen <so...@cloudera.com>.
A few small-scale code style tips which might improve readability:
You can in some cases write something like _.split("...") instead of x
=> x.split("...")
A series of if-else conditions on x(15) can be replaced with "x(15)
match { case"B" => ... }"
.isEmpty may be more readable than == ""
With so many fields, you may consider writing a simple case class to
represent the object instead so you can write ".someField" instead of
"(11)"
On Mon, Oct 12, 2015 at 12:28 PM, Kali.tummala@gmail.com
<Ka...@gmail.com> wrote:
> Hi All,
>
> just wonderign is there any better way of writing this below code, I am new
> to spark an I feel what I wrote is pretty simple and basic and straight
> forward is there any better way of writing using functional paradigm.
>
> val QuoteRDD=quotefile.map(x => x.split("\\|")).
> filter(line => line(0).contains("1017")).
> map(x => ((x(5)+x(4)) , (x(5),x(4),x(1) ,
> if (x(15) =="B")
> (
> {if (x(25) == "") x(9) else x(25)},
> {if (x(37) == "") x(11) else x(37)}
> )
> else if (x(15) =="C" )
> (
> {if (x(24) == "") (x(9)) else x(24)},
> {if (x(30) == "") (x(11)) else x(30)}
> )
> else if (x(15) =="A")
> {(x(9),x(11))}
> )))
>
> Thanks
> Sri
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Is-there-any-better-way-of-writing-this-code-tp25027.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org