You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Rob Blah <tm...@gmail.com> on 2013/09/26 16:46:00 UTC
Extending DFSInputStream class
Hi
I would like to wrap DFSInputStream by extension. However it seems that the
DFSInputStream constructor is package private. Is there anyway to achieve
my goal? Also just out of curiosity why you have made this class
inaccessible for developers, or am I missing something?
regards
tmp
Re: Extending DFSInputStream class
Posted by Rob Blah <tm...@gmail.com>.
I am throwing here this followup question.
Is it safe to extend (overwrite) DFSInputStream class, in terms of future
Yarn implementation design/changes etc. Unfortunately I have to overwrite
DFSInputStream as If I would overwrite HdfsDataInputStream (as I should) I
can't overwrite all the API i am suppose to (since in the inheritance tree
DataInputStream defines methods as final). Is my solution safe in the
future? Will it work correctly with HDFS in future? This question is
specially directed to YARN developers.
2013/9/26 Rob Blah <tm...@gmail.com>
> I have specific complex stream scheme, which I want to hide from the user
> (short answer), also some security reasons (limiting possible read buffer
> size).
>
>
> 2013/9/26 java8964 java8964 <ja...@hotmail.com>
>
>> Just curious, any reason you don't want to use the DFSDataInputStream?
>>
>> Yong
>>
>> ------------------------------
>> Date: Thu, 26 Sep 2013 16:46:00 +0200
>> Subject: Extending DFSInputStream class
>> From: tmp5330@gmail.com
>> To: user@hadoop.apache.org
>>
>>
>> Hi
>>
>> I would like to wrap DFSInputStream by extension. However it seems that
>> the DFSInputStream constructor is package private. Is there anyway to
>> achieve my goal? Also just out of curiosity why you have made this class
>> inaccessible for developers, or am I missing something?
>>
>> regards
>> tmp
>>
>
>
Re: Extending DFSInputStream class
Posted by Rob Blah <tm...@gmail.com>.
I am throwing here this followup question.
Is it safe to extend (overwrite) DFSInputStream class, in terms of future
Yarn implementation design/changes etc. Unfortunately I have to overwrite
DFSInputStream as If I would overwrite HdfsDataInputStream (as I should) I
can't overwrite all the API i am suppose to (since in the inheritance tree
DataInputStream defines methods as final). Is my solution safe in the
future? Will it work correctly with HDFS in future? This question is
specially directed to YARN developers.
2013/9/26 Rob Blah <tm...@gmail.com>
> I have specific complex stream scheme, which I want to hide from the user
> (short answer), also some security reasons (limiting possible read buffer
> size).
>
>
> 2013/9/26 java8964 java8964 <ja...@hotmail.com>
>
>> Just curious, any reason you don't want to use the DFSDataInputStream?
>>
>> Yong
>>
>> ------------------------------
>> Date: Thu, 26 Sep 2013 16:46:00 +0200
>> Subject: Extending DFSInputStream class
>> From: tmp5330@gmail.com
>> To: user@hadoop.apache.org
>>
>>
>> Hi
>>
>> I would like to wrap DFSInputStream by extension. However it seems that
>> the DFSInputStream constructor is package private. Is there anyway to
>> achieve my goal? Also just out of curiosity why you have made this class
>> inaccessible for developers, or am I missing something?
>>
>> regards
>> tmp
>>
>
>
Re: Extending DFSInputStream class
Posted by Rob Blah <tm...@gmail.com>.
I am throwing here this followup question.
Is it safe to extend (overwrite) DFSInputStream class, in terms of future
Yarn implementation design/changes etc. Unfortunately I have to overwrite
DFSInputStream as If I would overwrite HdfsDataInputStream (as I should) I
can't overwrite all the API i am suppose to (since in the inheritance tree
DataInputStream defines methods as final). Is my solution safe in the
future? Will it work correctly with HDFS in future? This question is
specially directed to YARN developers.
2013/9/26 Rob Blah <tm...@gmail.com>
> I have specific complex stream scheme, which I want to hide from the user
> (short answer), also some security reasons (limiting possible read buffer
> size).
>
>
> 2013/9/26 java8964 java8964 <ja...@hotmail.com>
>
>> Just curious, any reason you don't want to use the DFSDataInputStream?
>>
>> Yong
>>
>> ------------------------------
>> Date: Thu, 26 Sep 2013 16:46:00 +0200
>> Subject: Extending DFSInputStream class
>> From: tmp5330@gmail.com
>> To: user@hadoop.apache.org
>>
>>
>> Hi
>>
>> I would like to wrap DFSInputStream by extension. However it seems that
>> the DFSInputStream constructor is package private. Is there anyway to
>> achieve my goal? Also just out of curiosity why you have made this class
>> inaccessible for developers, or am I missing something?
>>
>> regards
>> tmp
>>
>
>
Re: Extending DFSInputStream class
Posted by Rob Blah <tm...@gmail.com>.
I am throwing here this followup question.
Is it safe to extend (overwrite) DFSInputStream class, in terms of future
Yarn implementation design/changes etc. Unfortunately I have to overwrite
DFSInputStream as If I would overwrite HdfsDataInputStream (as I should) I
can't overwrite all the API i am suppose to (since in the inheritance tree
DataInputStream defines methods as final). Is my solution safe in the
future? Will it work correctly with HDFS in future? This question is
specially directed to YARN developers.
2013/9/26 Rob Blah <tm...@gmail.com>
> I have specific complex stream scheme, which I want to hide from the user
> (short answer), also some security reasons (limiting possible read buffer
> size).
>
>
> 2013/9/26 java8964 java8964 <ja...@hotmail.com>
>
>> Just curious, any reason you don't want to use the DFSDataInputStream?
>>
>> Yong
>>
>> ------------------------------
>> Date: Thu, 26 Sep 2013 16:46:00 +0200
>> Subject: Extending DFSInputStream class
>> From: tmp5330@gmail.com
>> To: user@hadoop.apache.org
>>
>>
>> Hi
>>
>> I would like to wrap DFSInputStream by extension. However it seems that
>> the DFSInputStream constructor is package private. Is there anyway to
>> achieve my goal? Also just out of curiosity why you have made this class
>> inaccessible for developers, or am I missing something?
>>
>> regards
>> tmp
>>
>
>
Re: Extending DFSInputStream class
Posted by Rob Blah <tm...@gmail.com>.
I have specific complex stream scheme, which I want to hide from the user
(short answer), also some security reasons (limiting possible read buffer
size).
2013/9/26 java8964 java8964 <ja...@hotmail.com>
> Just curious, any reason you don't want to use the DFSDataInputStream?
>
> Yong
>
> ------------------------------
> Date: Thu, 26 Sep 2013 16:46:00 +0200
> Subject: Extending DFSInputStream class
> From: tmp5330@gmail.com
> To: user@hadoop.apache.org
>
>
> Hi
>
> I would like to wrap DFSInputStream by extension. However it seems that
> the DFSInputStream constructor is package private. Is there anyway to
> achieve my goal? Also just out of curiosity why you have made this class
> inaccessible for developers, or am I missing something?
>
> regards
> tmp
>
Re: Extending DFSInputStream class
Posted by Rob Blah <tm...@gmail.com>.
I have specific complex stream scheme, which I want to hide from the user
(short answer), also some security reasons (limiting possible read buffer
size).
2013/9/26 java8964 java8964 <ja...@hotmail.com>
> Just curious, any reason you don't want to use the DFSDataInputStream?
>
> Yong
>
> ------------------------------
> Date: Thu, 26 Sep 2013 16:46:00 +0200
> Subject: Extending DFSInputStream class
> From: tmp5330@gmail.com
> To: user@hadoop.apache.org
>
>
> Hi
>
> I would like to wrap DFSInputStream by extension. However it seems that
> the DFSInputStream constructor is package private. Is there anyway to
> achieve my goal? Also just out of curiosity why you have made this class
> inaccessible for developers, or am I missing something?
>
> regards
> tmp
>
Re: Extending DFSInputStream class
Posted by Rob Blah <tm...@gmail.com>.
I have specific complex stream scheme, which I want to hide from the user
(short answer), also some security reasons (limiting possible read buffer
size).
2013/9/26 java8964 java8964 <ja...@hotmail.com>
> Just curious, any reason you don't want to use the DFSDataInputStream?
>
> Yong
>
> ------------------------------
> Date: Thu, 26 Sep 2013 16:46:00 +0200
> Subject: Extending DFSInputStream class
> From: tmp5330@gmail.com
> To: user@hadoop.apache.org
>
>
> Hi
>
> I would like to wrap DFSInputStream by extension. However it seems that
> the DFSInputStream constructor is package private. Is there anyway to
> achieve my goal? Also just out of curiosity why you have made this class
> inaccessible for developers, or am I missing something?
>
> regards
> tmp
>
Re: Extending DFSInputStream class
Posted by Rob Blah <tm...@gmail.com>.
I have specific complex stream scheme, which I want to hide from the user
(short answer), also some security reasons (limiting possible read buffer
size).
2013/9/26 java8964 java8964 <ja...@hotmail.com>
> Just curious, any reason you don't want to use the DFSDataInputStream?
>
> Yong
>
> ------------------------------
> Date: Thu, 26 Sep 2013 16:46:00 +0200
> Subject: Extending DFSInputStream class
> From: tmp5330@gmail.com
> To: user@hadoop.apache.org
>
>
> Hi
>
> I would like to wrap DFSInputStream by extension. However it seems that
> the DFSInputStream constructor is package private. Is there anyway to
> achieve my goal? Also just out of curiosity why you have made this class
> inaccessible for developers, or am I missing something?
>
> regards
> tmp
>
RE: Extending DFSInputStream class
Posted by java8964 java8964 <ja...@hotmail.com>.
Just curious, any reason you don't want to use the DFSDataInputStream?
Yong
Date: Thu, 26 Sep 2013 16:46:00 +0200
Subject: Extending DFSInputStream class
From: tmp5330@gmail.com
To: user@hadoop.apache.org
Hi
I would like to wrap DFSInputStream by extension. However it seems that the DFSInputStream constructor is package private. Is there anyway to achieve my goal? Also just out of curiosity why you have made this class inaccessible for developers, or am I missing something?
regards
tmp
Re: Extending DFSInputStream class
Posted by Jay Vyas <ja...@gmail.com>.
The way we have gotten around this in the past is extending and then
copying the private code and creating a brand new implementation.
On Thu, Sep 26, 2013 at 10:50 AM, Jay Vyas <ja...@gmail.com> wrote:
> This is actually somewhat common in some of the hadoop core classes :
> Private constructors and inner classes. I think in the long term jiras
> should be opened for these to make them public and pluggable with public
> parameterized constructors wherever possible, so that modularizations can
> be provided.
>
>
> On Thu, Sep 26, 2013 at 10:46 AM, Rob Blah <tm...@gmail.com> wrote:
>
>> Hi
>>
>> I would like to wrap DFSInputStream by extension. However it seems that
>> the DFSInputStream constructor is package private. Is there anyway to
>> achieve my goal? Also just out of curiosity why you have made this class
>> inaccessible for developers, or am I missing something?
>>
>> regards
>> tmp
>>
>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Extending DFSInputStream class
Posted by Jay Vyas <ja...@gmail.com>.
The way we have gotten around this in the past is extending and then
copying the private code and creating a brand new implementation.
On Thu, Sep 26, 2013 at 10:50 AM, Jay Vyas <ja...@gmail.com> wrote:
> This is actually somewhat common in some of the hadoop core classes :
> Private constructors and inner classes. I think in the long term jiras
> should be opened for these to make them public and pluggable with public
> parameterized constructors wherever possible, so that modularizations can
> be provided.
>
>
> On Thu, Sep 26, 2013 at 10:46 AM, Rob Blah <tm...@gmail.com> wrote:
>
>> Hi
>>
>> I would like to wrap DFSInputStream by extension. However it seems that
>> the DFSInputStream constructor is package private. Is there anyway to
>> achieve my goal? Also just out of curiosity why you have made this class
>> inaccessible for developers, or am I missing something?
>>
>> regards
>> tmp
>>
>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Extending DFSInputStream class
Posted by Jay Vyas <ja...@gmail.com>.
The way we have gotten around this in the past is extending and then
copying the private code and creating a brand new implementation.
On Thu, Sep 26, 2013 at 10:50 AM, Jay Vyas <ja...@gmail.com> wrote:
> This is actually somewhat common in some of the hadoop core classes :
> Private constructors and inner classes. I think in the long term jiras
> should be opened for these to make them public and pluggable with public
> parameterized constructors wherever possible, so that modularizations can
> be provided.
>
>
> On Thu, Sep 26, 2013 at 10:46 AM, Rob Blah <tm...@gmail.com> wrote:
>
>> Hi
>>
>> I would like to wrap DFSInputStream by extension. However it seems that
>> the DFSInputStream constructor is package private. Is there anyway to
>> achieve my goal? Also just out of curiosity why you have made this class
>> inaccessible for developers, or am I missing something?
>>
>> regards
>> tmp
>>
>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Extending DFSInputStream class
Posted by Jay Vyas <ja...@gmail.com>.
The way we have gotten around this in the past is extending and then
copying the private code and creating a brand new implementation.
On Thu, Sep 26, 2013 at 10:50 AM, Jay Vyas <ja...@gmail.com> wrote:
> This is actually somewhat common in some of the hadoop core classes :
> Private constructors and inner classes. I think in the long term jiras
> should be opened for these to make them public and pluggable with public
> parameterized constructors wherever possible, so that modularizations can
> be provided.
>
>
> On Thu, Sep 26, 2013 at 10:46 AM, Rob Blah <tm...@gmail.com> wrote:
>
>> Hi
>>
>> I would like to wrap DFSInputStream by extension. However it seems that
>> the DFSInputStream constructor is package private. Is there anyway to
>> achieve my goal? Also just out of curiosity why you have made this class
>> inaccessible for developers, or am I missing something?
>>
>> regards
>> tmp
>>
>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Extending DFSInputStream class
Posted by Jay Vyas <ja...@gmail.com>.
This is actually somewhat common in some of the hadoop core classes :
Private constructors and inner classes. I think in the long term jiras
should be opened for these to make them public and pluggable with public
parameterized constructors wherever possible, so that modularizations can
be provided.
On Thu, Sep 26, 2013 at 10:46 AM, Rob Blah <tm...@gmail.com> wrote:
> Hi
>
> I would like to wrap DFSInputStream by extension. However it seems that
> the DFSInputStream constructor is package private. Is there anyway to
> achieve my goal? Also just out of curiosity why you have made this class
> inaccessible for developers, or am I missing something?
>
> regards
> tmp
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Extending DFSInputStream class
Posted by Jay Vyas <ja...@gmail.com>.
This is actually somewhat common in some of the hadoop core classes :
Private constructors and inner classes. I think in the long term jiras
should be opened for these to make them public and pluggable with public
parameterized constructors wherever possible, so that modularizations can
be provided.
On Thu, Sep 26, 2013 at 10:46 AM, Rob Blah <tm...@gmail.com> wrote:
> Hi
>
> I would like to wrap DFSInputStream by extension. However it seems that
> the DFSInputStream constructor is package private. Is there anyway to
> achieve my goal? Also just out of curiosity why you have made this class
> inaccessible for developers, or am I missing something?
>
> regards
> tmp
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Extending DFSInputStream class
Posted by Jay Vyas <ja...@gmail.com>.
This is actually somewhat common in some of the hadoop core classes :
Private constructors and inner classes. I think in the long term jiras
should be opened for these to make them public and pluggable with public
parameterized constructors wherever possible, so that modularizations can
be provided.
On Thu, Sep 26, 2013 at 10:46 AM, Rob Blah <tm...@gmail.com> wrote:
> Hi
>
> I would like to wrap DFSInputStream by extension. However it seems that
> the DFSInputStream constructor is package private. Is there anyway to
> achieve my goal? Also just out of curiosity why you have made this class
> inaccessible for developers, or am I missing something?
>
> regards
> tmp
>
--
Jay Vyas
http://jayunit100.blogspot.com
RE: Extending DFSInputStream class
Posted by java8964 java8964 <ja...@hotmail.com>.
Just curious, any reason you don't want to use the DFSDataInputStream?
Yong
Date: Thu, 26 Sep 2013 16:46:00 +0200
Subject: Extending DFSInputStream class
From: tmp5330@gmail.com
To: user@hadoop.apache.org
Hi
I would like to wrap DFSInputStream by extension. However it seems that the DFSInputStream constructor is package private. Is there anyway to achieve my goal? Also just out of curiosity why you have made this class inaccessible for developers, or am I missing something?
regards
tmp
RE: Extending DFSInputStream class
Posted by java8964 java8964 <ja...@hotmail.com>.
Just curious, any reason you don't want to use the DFSDataInputStream?
Yong
Date: Thu, 26 Sep 2013 16:46:00 +0200
Subject: Extending DFSInputStream class
From: tmp5330@gmail.com
To: user@hadoop.apache.org
Hi
I would like to wrap DFSInputStream by extension. However it seems that the DFSInputStream constructor is package private. Is there anyway to achieve my goal? Also just out of curiosity why you have made this class inaccessible for developers, or am I missing something?
regards
tmp
RE: Extending DFSInputStream class
Posted by java8964 java8964 <ja...@hotmail.com>.
Just curious, any reason you don't want to use the DFSDataInputStream?
Yong
Date: Thu, 26 Sep 2013 16:46:00 +0200
Subject: Extending DFSInputStream class
From: tmp5330@gmail.com
To: user@hadoop.apache.org
Hi
I would like to wrap DFSInputStream by extension. However it seems that the DFSInputStream constructor is package private. Is there anyway to achieve my goal? Also just out of curiosity why you have made this class inaccessible for developers, or am I missing something?
regards
tmp
Re: Extending DFSInputStream class
Posted by Jay Vyas <ja...@gmail.com>.
This is actually somewhat common in some of the hadoop core classes :
Private constructors and inner classes. I think in the long term jiras
should be opened for these to make them public and pluggable with public
parameterized constructors wherever possible, so that modularizations can
be provided.
On Thu, Sep 26, 2013 at 10:46 AM, Rob Blah <tm...@gmail.com> wrote:
> Hi
>
> I would like to wrap DFSInputStream by extension. However it seems that
> the DFSInputStream constructor is package private. Is there anyway to
> achieve my goal? Also just out of curiosity why you have made this class
> inaccessible for developers, or am I missing something?
>
> regards
> tmp
>
--
Jay Vyas
http://jayunit100.blogspot.com