You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Amjad Syed <am...@gmail.com> on 2014/06/15 21:13:15 UTC
Hadoop with SAN
Hello,
I am new to hadoop.
We have a SAN solution which we want to utilize for hadoop.
I have been asked by management to do a feasible study if hadoop can be
used with SAN.
But so far i have read that SAN is not recommended for hadoop and local
disk are stored.
Can some one please provide me information if Hadoop can be configured with
SAN? If yes, how will it effect the performance and what would be the
downside. of using SAN.
Thanks,
Amjad
Re: Hadoop with SAN
Posted by Jay Vyas <ja...@gmail.com>.
You can either use san to back your datanodes, or implement a custom FileSystem over your san storage. Either would have different drawbacks depending on your requirements.
Re: Hadoop with SAN
Posted by Jay Vyas <ja...@gmail.com>.
You can either use san to back your datanodes, or implement a custom FileSystem over your san storage. Either would have different drawbacks depending on your requirements.
Re: Hadoop with SAN
Posted by Jay Vyas <ja...@gmail.com>.
You can either use san to back your datanodes, or implement a custom FileSystem over your san storage. Either would have different drawbacks depending on your requirements.
Re: Hadoop with SAN
Posted by Jay Vyas <ja...@gmail.com>.
You can either use san to back your datanodes, or implement a custom FileSystem over your san storage. Either would have different drawbacks depending on your requirements.
RE: Hadoop with SAN
Posted by "Natarajan, Prabakaran 1. (NSN - IN/Bangalore)" <pr...@nsn.com>.
Hi
I had a same question ☺
Yes, performance impact will be there. But we can reduce the impact. We have to create a RaidGroup with type RAID-0 and represent this RaidGroup with only one LUN (don’t share this RaidGroup to any other LUN).
This is an alternative for JBOD in SAN storage.
You read more from my post
https://www.linkedin.com/groups/SAN-as-JBOD-VNX-Hadoop-3750755.S.5883183617227644929?view=&gid=3750755&item=5883183617227644929&type=member&commentID=discussion%3A5883183617227644929%3Agroup%3A3750755&trk=hb_ntf_COMMENTED_ON_GROUP_DISCUSSION_YOU_CREATED#commentID_discussion%3A5883183617227644929%3Agroup%3A3750755
Thanks
Prabakaran.N
From: ext Amjad Syed [mailto:amjadcsu@gmail.com]
Sent: Monday, June 16, 2014 12:43 AM
To: user@hadoop.apache.org
Subject: Hadoop with SAN
Hello,
I am new to hadoop.
We have a SAN solution which we want to utilize for hadoop.
I have been asked by management to do a feasible study if hadoop can be used with SAN.
But so far i have read that SAN is not recommended for hadoop and local disk are stored.
Can some one please provide me information if Hadoop can be configured with SAN? If yes, how will it effect the performance and what would be the downside. of using SAN.
Thanks,
Amjad
RE: Hadoop with SAN
Posted by "Natarajan, Prabakaran 1. (NSN - IN/Bangalore)" <pr...@nsn.com>.
Hi
I had a same question ☺
Yes, performance impact will be there. But we can reduce the impact. We have to create a RaidGroup with type RAID-0 and represent this RaidGroup with only one LUN (don’t share this RaidGroup to any other LUN).
This is an alternative for JBOD in SAN storage.
You read more from my post
https://www.linkedin.com/groups/SAN-as-JBOD-VNX-Hadoop-3750755.S.5883183617227644929?view=&gid=3750755&item=5883183617227644929&type=member&commentID=discussion%3A5883183617227644929%3Agroup%3A3750755&trk=hb_ntf_COMMENTED_ON_GROUP_DISCUSSION_YOU_CREATED#commentID_discussion%3A5883183617227644929%3Agroup%3A3750755
Thanks
Prabakaran.N
From: ext Amjad Syed [mailto:amjadcsu@gmail.com]
Sent: Monday, June 16, 2014 12:43 AM
To: user@hadoop.apache.org
Subject: Hadoop with SAN
Hello,
I am new to hadoop.
We have a SAN solution which we want to utilize for hadoop.
I have been asked by management to do a feasible study if hadoop can be used with SAN.
But so far i have read that SAN is not recommended for hadoop and local disk are stored.
Can some one please provide me information if Hadoop can be configured with SAN? If yes, how will it effect the performance and what would be the downside. of using SAN.
Thanks,
Amjad
RE: Hadoop with SAN
Posted by "Natarajan, Prabakaran 1. (NSN - IN/Bangalore)" <pr...@nsn.com>.
Hi
I had a same question ☺
Yes, performance impact will be there. But we can reduce the impact. We have to create a RaidGroup with type RAID-0 and represent this RaidGroup with only one LUN (don’t share this RaidGroup to any other LUN).
This is an alternative for JBOD in SAN storage.
You read more from my post
https://www.linkedin.com/groups/SAN-as-JBOD-VNX-Hadoop-3750755.S.5883183617227644929?view=&gid=3750755&item=5883183617227644929&type=member&commentID=discussion%3A5883183617227644929%3Agroup%3A3750755&trk=hb_ntf_COMMENTED_ON_GROUP_DISCUSSION_YOU_CREATED#commentID_discussion%3A5883183617227644929%3Agroup%3A3750755
Thanks
Prabakaran.N
From: ext Amjad Syed [mailto:amjadcsu@gmail.com]
Sent: Monday, June 16, 2014 12:43 AM
To: user@hadoop.apache.org
Subject: Hadoop with SAN
Hello,
I am new to hadoop.
We have a SAN solution which we want to utilize for hadoop.
I have been asked by management to do a feasible study if hadoop can be used with SAN.
But so far i have read that SAN is not recommended for hadoop and local disk are stored.
Can some one please provide me information if Hadoop can be configured with SAN? If yes, how will it effect the performance and what would be the downside. of using SAN.
Thanks,
Amjad
RE: Hadoop with SAN
Posted by "Natarajan, Prabakaran 1. (NSN - IN/Bangalore)" <pr...@nsn.com>.
Hi
I had a same question ☺
Yes, performance impact will be there. But we can reduce the impact. We have to create a RaidGroup with type RAID-0 and represent this RaidGroup with only one LUN (don’t share this RaidGroup to any other LUN).
This is an alternative for JBOD in SAN storage.
You read more from my post
https://www.linkedin.com/groups/SAN-as-JBOD-VNX-Hadoop-3750755.S.5883183617227644929?view=&gid=3750755&item=5883183617227644929&type=member&commentID=discussion%3A5883183617227644929%3Agroup%3A3750755&trk=hb_ntf_COMMENTED_ON_GROUP_DISCUSSION_YOU_CREATED#commentID_discussion%3A5883183617227644929%3Agroup%3A3750755
Thanks
Prabakaran.N
From: ext Amjad Syed [mailto:amjadcsu@gmail.com]
Sent: Monday, June 16, 2014 12:43 AM
To: user@hadoop.apache.org
Subject: Hadoop with SAN
Hello,
I am new to hadoop.
We have a SAN solution which we want to utilize for hadoop.
I have been asked by management to do a feasible study if hadoop can be used with SAN.
But so far i have read that SAN is not recommended for hadoop and local disk are stored.
Can some one please provide me information if Hadoop can be configured with SAN? If yes, how will it effect the performance and what would be the downside. of using SAN.
Thanks,
Amjad