You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Thanh Hong Dai <hd...@tma.com.vn> on 2015/08/03 05:49:35 UTC
Documentation inconsistency about append write in HDFS
In the latest version of the documentation
(http://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-hdfs/Hdfs
Design.html#Simple_Coherency_Model and also documentation for version 2.x),
it's mentioned that "A file once created, written, and closed need not be
changed. " and "There is a plan to support appending-writes to files in the
future."
However, as far as I know, HDFS has supported append write since 0.21, based
on this JIRA (https://issues.apache.org/jira/browse/HDFS-265) and the old
version of the documentation in 2012
(https://web.archive.org/web/20121221171824/http://hadoop.apache.org/docs/hd
fs/current/hdfs_design.html#Appending-Writes+and+File+Syncs
<https://web.archive.org/web/20121221171824/http:/hadoop.apache.org/docs/hdf
s/current/hdfs_design.html#Appending-Writes+and+File+Syncs> )
Various posts on the Internet also suggests that append write has been
available in HDFS, and will always be available in Hadoop version 2 branch.
Can we update the documentation to reflect the most recent change? (Or will
append write be deprecated or is it not ready for production use?)
Re: Documentation inconsistency about append write in HDFS
Posted by Arpit Agarwal <aa...@hortonworks.com>.
Hi Thanh,
Thanks for bringing it up. Append is available in 2.x releases as you pointed out and is production-ready.
Can you please file a doc bug at https://issues.apache.org/jira/browse/HADOOP?
On Aug 2, 2015, at 8:49 PM, Thanh Hong Dai <hd...@tma.com.vn>> wrote:
In the latest version of the documentation (http://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Simple_Coherency_Model and also documentation for version 2.x), it’s mentioned that “A file once created, written, and closed need not be changed. “ and “There is a plan to support appending-writes to files in the future.”
However, as far as I know, HDFS has supported append write since 0.21, based on this JIRA (https://issues.apache.org/jira/browse/HDFS-265) and the old version of the documentation in 2012 (https://web.archive.org/web/20121221171824/http://hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs<https://web.archive.org/web/20121221171824/http:/hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs>)
Various posts on the Internet also suggests that append write has been available in HDFS, and will always be available in Hadoop version 2 branch.
Can we update the documentation to reflect the most recent change? (Or will append write be deprecated or is it not ready for production use?)
RE: Documentation inconsistency about append write in HDFS
Posted by Thanh Hong Dai <hd...@tma.com.vn>.
I have posted an issue on the bug tracker: https://issues.apache.org/jira/browse/HDFS-8852
I’m not too familiar with the project, so I will leave it to more qualified people to do it.
From: Ravi Prakash [mailto:ravihoo@ymail.com]
Sent: Tuesday, 4 August, 2015 3:52 AM
To: user@hadoop.apache.org
Subject: Re: Documentation inconsistency about append write in HDFS
Thanks Thanh! Yes! Could you please post a patch?
On Sunday, August 2, 2015 8:50 PM, Thanh Hong Dai <hdthanh@tma.com.vn <ma...@tma.com.vn> > wrote:
In the latest version of the documentation (http://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Simple_Coherency_Model and also documentation for version 2.x), it’s mentioned that “A file once created, written, and closed need not be changed. “ and “There is a plan to support appending-writes to files in the future.”
However, as far as I know, HDFS has supported append write since 0.21, based on this JIRA (https://issues.apache.org/jira/browse/HDFS-265) and the old version of the documentation in 2012 (https://web.archive.org/web/20121221171824/http://hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs)
Various posts on the Internet also suggests that append write has been available in HDFS, and will always be available in Hadoop version 2 branch.
Can we update the documentation to reflect the most recent change? (Or will append write be deprecated or is it not ready for production use?)
RE: Documentation inconsistency about append write in HDFS
Posted by Thanh Hong Dai <hd...@tma.com.vn>.
I have posted an issue on the bug tracker: https://issues.apache.org/jira/browse/HDFS-8852
I’m not too familiar with the project, so I will leave it to more qualified people to do it.
From: Ravi Prakash [mailto:ravihoo@ymail.com]
Sent: Tuesday, 4 August, 2015 3:52 AM
To: user@hadoop.apache.org
Subject: Re: Documentation inconsistency about append write in HDFS
Thanks Thanh! Yes! Could you please post a patch?
On Sunday, August 2, 2015 8:50 PM, Thanh Hong Dai <hdthanh@tma.com.vn <ma...@tma.com.vn> > wrote:
In the latest version of the documentation (http://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Simple_Coherency_Model and also documentation for version 2.x), it’s mentioned that “A file once created, written, and closed need not be changed. “ and “There is a plan to support appending-writes to files in the future.”
However, as far as I know, HDFS has supported append write since 0.21, based on this JIRA (https://issues.apache.org/jira/browse/HDFS-265) and the old version of the documentation in 2012 (https://web.archive.org/web/20121221171824/http://hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs)
Various posts on the Internet also suggests that append write has been available in HDFS, and will always be available in Hadoop version 2 branch.
Can we update the documentation to reflect the most recent change? (Or will append write be deprecated or is it not ready for production use?)
RE: Documentation inconsistency about append write in HDFS
Posted by Thanh Hong Dai <hd...@tma.com.vn>.
I have posted an issue on the bug tracker: https://issues.apache.org/jira/browse/HDFS-8852
I’m not too familiar with the project, so I will leave it to more qualified people to do it.
From: Ravi Prakash [mailto:ravihoo@ymail.com]
Sent: Tuesday, 4 August, 2015 3:52 AM
To: user@hadoop.apache.org
Subject: Re: Documentation inconsistency about append write in HDFS
Thanks Thanh! Yes! Could you please post a patch?
On Sunday, August 2, 2015 8:50 PM, Thanh Hong Dai <hdthanh@tma.com.vn <ma...@tma.com.vn> > wrote:
In the latest version of the documentation (http://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Simple_Coherency_Model and also documentation for version 2.x), it’s mentioned that “A file once created, written, and closed need not be changed. “ and “There is a plan to support appending-writes to files in the future.”
However, as far as I know, HDFS has supported append write since 0.21, based on this JIRA (https://issues.apache.org/jira/browse/HDFS-265) and the old version of the documentation in 2012 (https://web.archive.org/web/20121221171824/http://hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs)
Various posts on the Internet also suggests that append write has been available in HDFS, and will always be available in Hadoop version 2 branch.
Can we update the documentation to reflect the most recent change? (Or will append write be deprecated or is it not ready for production use?)
RE: Documentation inconsistency about append write in HDFS
Posted by Thanh Hong Dai <hd...@tma.com.vn>.
I have posted an issue on the bug tracker: https://issues.apache.org/jira/browse/HDFS-8852
I’m not too familiar with the project, so I will leave it to more qualified people to do it.
From: Ravi Prakash [mailto:ravihoo@ymail.com]
Sent: Tuesday, 4 August, 2015 3:52 AM
To: user@hadoop.apache.org
Subject: Re: Documentation inconsistency about append write in HDFS
Thanks Thanh! Yes! Could you please post a patch?
On Sunday, August 2, 2015 8:50 PM, Thanh Hong Dai <hdthanh@tma.com.vn <ma...@tma.com.vn> > wrote:
In the latest version of the documentation (http://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Simple_Coherency_Model and also documentation for version 2.x), it’s mentioned that “A file once created, written, and closed need not be changed. “ and “There is a plan to support appending-writes to files in the future.”
However, as far as I know, HDFS has supported append write since 0.21, based on this JIRA (https://issues.apache.org/jira/browse/HDFS-265) and the old version of the documentation in 2012 (https://web.archive.org/web/20121221171824/http://hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs)
Various posts on the Internet also suggests that append write has been available in HDFS, and will always be available in Hadoop version 2 branch.
Can we update the documentation to reflect the most recent change? (Or will append write be deprecated or is it not ready for production use?)
Re: Documentation inconsistency about append write in HDFS
Posted by Ravi Prakash <ra...@ymail.com>.
Thanks Thanh! Yes! Could you please post a patch?
On Sunday, August 2, 2015 8:50 PM, Thanh Hong Dai <hd...@tma.com.vn> wrote:
<!--#yiv9197574444 _filtered #yiv9197574444 {font-family:"MS Mincho";panose-1:2 2 6 9 4 2 5 8 3 4;} _filtered #yiv9197574444 {font-family:"Cambria Math";panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv9197574444 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv9197574444 {panose-1:2 2 6 9 4 2 5 8 3 4;}#yiv9197574444 #yiv9197574444 p.yiv9197574444MsoNormal, #yiv9197574444 li.yiv9197574444MsoNormal, #yiv9197574444 div.yiv9197574444MsoNormal {margin:0cm;margin-bottom:.0001pt;font-size:11.0pt;font-family:"Calibri", sans-serif;}#yiv9197574444 a:link, #yiv9197574444 span.yiv9197574444MsoHyperlink {color:#0563C1;text-decoration:underline;}#yiv9197574444 a:visited, #yiv9197574444 span.yiv9197574444MsoHyperlinkFollowed {color:#954F72;text-decoration:underline;}#yiv9197574444 span.yiv9197574444EmailStyle17 {font-family:"Calibri", sans-serif;color:windowtext;}#yiv9197574444 .yiv9197574444MsoChpDefault {font-family:"Calibri", sans-serif;} _filtered #yiv9197574444 {margin:72.0pt 72.0pt 72.0pt 72.0pt;}#yiv9197574444 div.yiv9197574444WordSection1 {}-->In the latest version of the documentation (http://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Simple_Coherency_Model and also documentation for version 2.x), it’s mentioned that “A file once created, written, and closed need not be changed. “ and “There is a plan to support appending-writes to files in the future.” However, as far as I know, HDFS has supported append write since 0.21, based on this JIRA (https://issues.apache.org/jira/browse/HDFS-265) and the old version of the documentation in 2012 (https://web.archive.org/web/20121221171824/http://hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs) Various posts on the Internet also suggests that append write has been available in HDFS, and will always be available in Hadoop version 2 branch. Can we update the documentation to reflect the most recent change? (Or will append write be deprecated or is it not ready for production use?)
Re: Documentation inconsistency about append write in HDFS
Posted by Arpit Agarwal <aa...@hortonworks.com>.
Hi Thanh,
Thanks for bringing it up. Append is available in 2.x releases as you pointed out and is production-ready.
Can you please file a doc bug at https://issues.apache.org/jira/browse/HADOOP?
On Aug 2, 2015, at 8:49 PM, Thanh Hong Dai <hd...@tma.com.vn>> wrote:
In the latest version of the documentation (http://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Simple_Coherency_Model and also documentation for version 2.x), it’s mentioned that “A file once created, written, and closed need not be changed. “ and “There is a plan to support appending-writes to files in the future.”
However, as far as I know, HDFS has supported append write since 0.21, based on this JIRA (https://issues.apache.org/jira/browse/HDFS-265) and the old version of the documentation in 2012 (https://web.archive.org/web/20121221171824/http://hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs<https://web.archive.org/web/20121221171824/http:/hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs>)
Various posts on the Internet also suggests that append write has been available in HDFS, and will always be available in Hadoop version 2 branch.
Can we update the documentation to reflect the most recent change? (Or will append write be deprecated or is it not ready for production use?)
Re: Documentation inconsistency about append write in HDFS
Posted by Arpit Agarwal <aa...@hortonworks.com>.
Hi Thanh,
Thanks for bringing it up. Append is available in 2.x releases as you pointed out and is production-ready.
Can you please file a doc bug at https://issues.apache.org/jira/browse/HADOOP?
On Aug 2, 2015, at 8:49 PM, Thanh Hong Dai <hd...@tma.com.vn>> wrote:
In the latest version of the documentation (http://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Simple_Coherency_Model and also documentation for version 2.x), it’s mentioned that “A file once created, written, and closed need not be changed. “ and “There is a plan to support appending-writes to files in the future.”
However, as far as I know, HDFS has supported append write since 0.21, based on this JIRA (https://issues.apache.org/jira/browse/HDFS-265) and the old version of the documentation in 2012 (https://web.archive.org/web/20121221171824/http://hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs<https://web.archive.org/web/20121221171824/http:/hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs>)
Various posts on the Internet also suggests that append write has been available in HDFS, and will always be available in Hadoop version 2 branch.
Can we update the documentation to reflect the most recent change? (Or will append write be deprecated or is it not ready for production use?)
Re: Documentation inconsistency about append write in HDFS
Posted by Ravi Prakash <ra...@ymail.com>.
Thanks Thanh! Yes! Could you please post a patch?
On Sunday, August 2, 2015 8:50 PM, Thanh Hong Dai <hd...@tma.com.vn> wrote:
<!--#yiv9197574444 _filtered #yiv9197574444 {font-family:"MS Mincho";panose-1:2 2 6 9 4 2 5 8 3 4;} _filtered #yiv9197574444 {font-family:"Cambria Math";panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv9197574444 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv9197574444 {panose-1:2 2 6 9 4 2 5 8 3 4;}#yiv9197574444 #yiv9197574444 p.yiv9197574444MsoNormal, #yiv9197574444 li.yiv9197574444MsoNormal, #yiv9197574444 div.yiv9197574444MsoNormal {margin:0cm;margin-bottom:.0001pt;font-size:11.0pt;font-family:"Calibri", sans-serif;}#yiv9197574444 a:link, #yiv9197574444 span.yiv9197574444MsoHyperlink {color:#0563C1;text-decoration:underline;}#yiv9197574444 a:visited, #yiv9197574444 span.yiv9197574444MsoHyperlinkFollowed {color:#954F72;text-decoration:underline;}#yiv9197574444 span.yiv9197574444EmailStyle17 {font-family:"Calibri", sans-serif;color:windowtext;}#yiv9197574444 .yiv9197574444MsoChpDefault {font-family:"Calibri", sans-serif;} _filtered #yiv9197574444 {margin:72.0pt 72.0pt 72.0pt 72.0pt;}#yiv9197574444 div.yiv9197574444WordSection1 {}-->In the latest version of the documentation (http://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Simple_Coherency_Model and also documentation for version 2.x), it’s mentioned that “A file once created, written, and closed need not be changed. “ and “There is a plan to support appending-writes to files in the future.” However, as far as I know, HDFS has supported append write since 0.21, based on this JIRA (https://issues.apache.org/jira/browse/HDFS-265) and the old version of the documentation in 2012 (https://web.archive.org/web/20121221171824/http://hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs) Various posts on the Internet also suggests that append write has been available in HDFS, and will always be available in Hadoop version 2 branch. Can we update the documentation to reflect the most recent change? (Or will append write be deprecated or is it not ready for production use?)
Re: Documentation inconsistency about append write in HDFS
Posted by Arpit Agarwal <aa...@hortonworks.com>.
Hi Thanh,
Thanks for bringing it up. Append is available in 2.x releases as you pointed out and is production-ready.
Can you please file a doc bug at https://issues.apache.org/jira/browse/HADOOP?
On Aug 2, 2015, at 8:49 PM, Thanh Hong Dai <hd...@tma.com.vn>> wrote:
In the latest version of the documentation (http://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Simple_Coherency_Model and also documentation for version 2.x), it’s mentioned that “A file once created, written, and closed need not be changed. “ and “There is a plan to support appending-writes to files in the future.”
However, as far as I know, HDFS has supported append write since 0.21, based on this JIRA (https://issues.apache.org/jira/browse/HDFS-265) and the old version of the documentation in 2012 (https://web.archive.org/web/20121221171824/http://hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs<https://web.archive.org/web/20121221171824/http:/hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs>)
Various posts on the Internet also suggests that append write has been available in HDFS, and will always be available in Hadoop version 2 branch.
Can we update the documentation to reflect the most recent change? (Or will append write be deprecated or is it not ready for production use?)
Re: Documentation inconsistency about append write in HDFS
Posted by Ravi Prakash <ra...@ymail.com>.
Thanks Thanh! Yes! Could you please post a patch?
On Sunday, August 2, 2015 8:50 PM, Thanh Hong Dai <hd...@tma.com.vn> wrote:
<!--#yiv9197574444 _filtered #yiv9197574444 {font-family:"MS Mincho";panose-1:2 2 6 9 4 2 5 8 3 4;} _filtered #yiv9197574444 {font-family:"Cambria Math";panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv9197574444 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv9197574444 {panose-1:2 2 6 9 4 2 5 8 3 4;}#yiv9197574444 #yiv9197574444 p.yiv9197574444MsoNormal, #yiv9197574444 li.yiv9197574444MsoNormal, #yiv9197574444 div.yiv9197574444MsoNormal {margin:0cm;margin-bottom:.0001pt;font-size:11.0pt;font-family:"Calibri", sans-serif;}#yiv9197574444 a:link, #yiv9197574444 span.yiv9197574444MsoHyperlink {color:#0563C1;text-decoration:underline;}#yiv9197574444 a:visited, #yiv9197574444 span.yiv9197574444MsoHyperlinkFollowed {color:#954F72;text-decoration:underline;}#yiv9197574444 span.yiv9197574444EmailStyle17 {font-family:"Calibri", sans-serif;color:windowtext;}#yiv9197574444 .yiv9197574444MsoChpDefault {font-family:"Calibri", sans-serif;} _filtered #yiv9197574444 {margin:72.0pt 72.0pt 72.0pt 72.0pt;}#yiv9197574444 div.yiv9197574444WordSection1 {}-->In the latest version of the documentation (http://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Simple_Coherency_Model and also documentation for version 2.x), it’s mentioned that “A file once created, written, and closed need not be changed. “ and “There is a plan to support appending-writes to files in the future.” However, as far as I know, HDFS has supported append write since 0.21, based on this JIRA (https://issues.apache.org/jira/browse/HDFS-265) and the old version of the documentation in 2012 (https://web.archive.org/web/20121221171824/http://hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs) Various posts on the Internet also suggests that append write has been available in HDFS, and will always be available in Hadoop version 2 branch. Can we update the documentation to reflect the most recent change? (Or will append write be deprecated or is it not ready for production use?)
Re: Documentation inconsistency about append write in HDFS
Posted by Ravi Prakash <ra...@ymail.com>.
Thanks Thanh! Yes! Could you please post a patch?
On Sunday, August 2, 2015 8:50 PM, Thanh Hong Dai <hd...@tma.com.vn> wrote:
<!--#yiv9197574444 _filtered #yiv9197574444 {font-family:"MS Mincho";panose-1:2 2 6 9 4 2 5 8 3 4;} _filtered #yiv9197574444 {font-family:"Cambria Math";panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv9197574444 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv9197574444 {panose-1:2 2 6 9 4 2 5 8 3 4;}#yiv9197574444 #yiv9197574444 p.yiv9197574444MsoNormal, #yiv9197574444 li.yiv9197574444MsoNormal, #yiv9197574444 div.yiv9197574444MsoNormal {margin:0cm;margin-bottom:.0001pt;font-size:11.0pt;font-family:"Calibri", sans-serif;}#yiv9197574444 a:link, #yiv9197574444 span.yiv9197574444MsoHyperlink {color:#0563C1;text-decoration:underline;}#yiv9197574444 a:visited, #yiv9197574444 span.yiv9197574444MsoHyperlinkFollowed {color:#954F72;text-decoration:underline;}#yiv9197574444 span.yiv9197574444EmailStyle17 {font-family:"Calibri", sans-serif;color:windowtext;}#yiv9197574444 .yiv9197574444MsoChpDefault {font-family:"Calibri", sans-serif;} _filtered #yiv9197574444 {margin:72.0pt 72.0pt 72.0pt 72.0pt;}#yiv9197574444 div.yiv9197574444WordSection1 {}-->In the latest version of the documentation (http://hadoop.apache.org/docs/current2/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Simple_Coherency_Model and also documentation for version 2.x), it’s mentioned that “A file once created, written, and closed need not be changed. “ and “There is a plan to support appending-writes to files in the future.” However, as far as I know, HDFS has supported append write since 0.21, based on this JIRA (https://issues.apache.org/jira/browse/HDFS-265) and the old version of the documentation in 2012 (https://web.archive.org/web/20121221171824/http://hadoop.apache.org/docs/hdfs/current/hdfs_design.html#Appending-Writes+and+File+Syncs) Various posts on the Internet also suggests that append write has been available in HDFS, and will always be available in Hadoop version 2 branch. Can we update the documentation to reflect the most recent change? (Or will append write be deprecated or is it not ready for production use?)