You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Demai Ni <ni...@gmail.com> on 2015/09/09 01:46:00 UTC
HDFS ShortCircuit Read on Mac?
hi, folks,
wondering anyone has setup HDFS shortcircuit Read on Mac? I installed
hadoop through homebrew on Mac. It is up and running. But I cannot
config "dfs.domain.socket.path" as instructed here:
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html
Since there is no dn_socket on the mac.
any pointers are appreciated.
Demai
Re: HDFS ShortCircuit Read on Mac?
Posted by Demai Ni <ni...@gmail.com>.
Chris, many thanks for the quick response. I will disable the shortcircuit
on my mac for now. :-) .... Demai
On Tue, Sep 8, 2015 at 4:57 PM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> Hello Demai,
>
> HDFS short-circuit read currently does not work on Mac, due to some
> platform differences in handling of domain sockets. The last time I
> checked, our Hadoop code was exceeding a maximum path length enforced on
> Mac for domain socket paths. I haven't had availability to look at this in
> a while, but the prior work is tracked in JIRA issues HDFS-3296 and
> HADOOP-11957 if you want to see the current progress.
>
> --Chris Nauroth
>
> From: Demai Ni <ni...@gmail.com>
> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Date: Tuesday, September 8, 2015 at 4:46 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: HDFS ShortCircuit Read on Mac?
>
> hi, folks,
>
> wondering anyone has setup HDFS shortcircuit Read on Mac? I installed
> hadoop through homebrew on Mac. It is up and running. But I cannot
> config "dfs.domain.socket.path" as instructed here:
>
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html
> Since there is no dn_socket on the mac.
>
> any pointers are appreciated.
>
> Demai
>
>
Re: HDFS ShortCircuit Read on Mac?
Posted by Demai Ni <ni...@gmail.com>.
Chris, many thanks for the quick response. I will disable the shortcircuit
on my mac for now. :-) .... Demai
On Tue, Sep 8, 2015 at 4:57 PM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> Hello Demai,
>
> HDFS short-circuit read currently does not work on Mac, due to some
> platform differences in handling of domain sockets. The last time I
> checked, our Hadoop code was exceeding a maximum path length enforced on
> Mac for domain socket paths. I haven't had availability to look at this in
> a while, but the prior work is tracked in JIRA issues HDFS-3296 and
> HADOOP-11957 if you want to see the current progress.
>
> --Chris Nauroth
>
> From: Demai Ni <ni...@gmail.com>
> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Date: Tuesday, September 8, 2015 at 4:46 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: HDFS ShortCircuit Read on Mac?
>
> hi, folks,
>
> wondering anyone has setup HDFS shortcircuit Read on Mac? I installed
> hadoop through homebrew on Mac. It is up and running. But I cannot
> config "dfs.domain.socket.path" as instructed here:
>
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html
> Since there is no dn_socket on the mac.
>
> any pointers are appreciated.
>
> Demai
>
>
Re: HDFS ShortCircuit Read on Mac?
Posted by Demai Ni <ni...@gmail.com>.
Chris, many thanks for the quick response. I will disable the shortcircuit
on my mac for now. :-) .... Demai
On Tue, Sep 8, 2015 at 4:57 PM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> Hello Demai,
>
> HDFS short-circuit read currently does not work on Mac, due to some
> platform differences in handling of domain sockets. The last time I
> checked, our Hadoop code was exceeding a maximum path length enforced on
> Mac for domain socket paths. I haven't had availability to look at this in
> a while, but the prior work is tracked in JIRA issues HDFS-3296 and
> HADOOP-11957 if you want to see the current progress.
>
> --Chris Nauroth
>
> From: Demai Ni <ni...@gmail.com>
> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Date: Tuesday, September 8, 2015 at 4:46 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: HDFS ShortCircuit Read on Mac?
>
> hi, folks,
>
> wondering anyone has setup HDFS shortcircuit Read on Mac? I installed
> hadoop through homebrew on Mac. It is up and running. But I cannot
> config "dfs.domain.socket.path" as instructed here:
>
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html
> Since there is no dn_socket on the mac.
>
> any pointers are appreciated.
>
> Demai
>
>
Re: HDFS ShortCircuit Read on Mac?
Posted by Demai Ni <ni...@gmail.com>.
Chris, many thanks for the quick response. I will disable the shortcircuit
on my mac for now. :-) .... Demai
On Tue, Sep 8, 2015 at 4:57 PM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> Hello Demai,
>
> HDFS short-circuit read currently does not work on Mac, due to some
> platform differences in handling of domain sockets. The last time I
> checked, our Hadoop code was exceeding a maximum path length enforced on
> Mac for domain socket paths. I haven't had availability to look at this in
> a while, but the prior work is tracked in JIRA issues HDFS-3296 and
> HADOOP-11957 if you want to see the current progress.
>
> --Chris Nauroth
>
> From: Demai Ni <ni...@gmail.com>
> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Date: Tuesday, September 8, 2015 at 4:46 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: HDFS ShortCircuit Read on Mac?
>
> hi, folks,
>
> wondering anyone has setup HDFS shortcircuit Read on Mac? I installed
> hadoop through homebrew on Mac. It is up and running. But I cannot
> config "dfs.domain.socket.path" as instructed here:
>
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html
> Since there is no dn_socket on the mac.
>
> any pointers are appreciated.
>
> Demai
>
>
Re: HDFS ShortCircuit Read on Mac?
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Demai,
HDFS short-circuit read currently does not work on Mac, due to some platform differences in handling of domain sockets. The last time I checked, our Hadoop code was exceeding a maximum path length enforced on Mac for domain socket paths. I haven't had availability to look at this in a while, but the prior work is tracked in JIRA issues HDFS-3296 and HADOOP-11957 if you want to see the current progress.
--Chris Nauroth
From: Demai Ni <ni...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Tuesday, September 8, 2015 at 4:46 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: HDFS ShortCircuit Read on Mac?
hi, folks,
wondering anyone has setup HDFS shortcircuit Read on Mac? I installed hadoop through homebrew on Mac. It is up and running. But I cannot config "dfs.domain.socket.path" as instructed here:
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html
Since there is no dn_socket on the mac.
any pointers are appreciated.
Demai
Re: HDFS ShortCircuit Read on Mac?
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Demai,
HDFS short-circuit read currently does not work on Mac, due to some platform differences in handling of domain sockets. The last time I checked, our Hadoop code was exceeding a maximum path length enforced on Mac for domain socket paths. I haven't had availability to look at this in a while, but the prior work is tracked in JIRA issues HDFS-3296 and HADOOP-11957 if you want to see the current progress.
--Chris Nauroth
From: Demai Ni <ni...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Tuesday, September 8, 2015 at 4:46 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: HDFS ShortCircuit Read on Mac?
hi, folks,
wondering anyone has setup HDFS shortcircuit Read on Mac? I installed hadoop through homebrew on Mac. It is up and running. But I cannot config "dfs.domain.socket.path" as instructed here:
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html
Since there is no dn_socket on the mac.
any pointers are appreciated.
Demai
Re: HDFS ShortCircuit Read on Mac?
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Demai,
HDFS short-circuit read currently does not work on Mac, due to some platform differences in handling of domain sockets. The last time I checked, our Hadoop code was exceeding a maximum path length enforced on Mac for domain socket paths. I haven't had availability to look at this in a while, but the prior work is tracked in JIRA issues HDFS-3296 and HADOOP-11957 if you want to see the current progress.
--Chris Nauroth
From: Demai Ni <ni...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Tuesday, September 8, 2015 at 4:46 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: HDFS ShortCircuit Read on Mac?
hi, folks,
wondering anyone has setup HDFS shortcircuit Read on Mac? I installed hadoop through homebrew on Mac. It is up and running. But I cannot config "dfs.domain.socket.path" as instructed here:
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html
Since there is no dn_socket on the mac.
any pointers are appreciated.
Demai
Re: HDFS ShortCircuit Read on Mac?
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Demai,
HDFS short-circuit read currently does not work on Mac, due to some platform differences in handling of domain sockets. The last time I checked, our Hadoop code was exceeding a maximum path length enforced on Mac for domain socket paths. I haven't had availability to look at this in a while, but the prior work is tracked in JIRA issues HDFS-3296 and HADOOP-11957 if you want to see the current progress.
--Chris Nauroth
From: Demai Ni <ni...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Tuesday, September 8, 2015 at 4:46 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: HDFS ShortCircuit Read on Mac?
hi, folks,
wondering anyone has setup HDFS shortcircuit Read on Mac? I installed hadoop through homebrew on Mac. It is up and running. But I cannot config "dfs.domain.socket.path" as instructed here:
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html
Since there is no dn_socket on the mac.
any pointers are appreciated.
Demai