You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Thomas Stidsborg Sylvest <ts...@danskecommodities.com> on 2013/08/06 15:18:38 UTC
HDFS 1.1.0 - File Append
Hi,
I am in the process of developing a framework around Hadoop that enables RabbitMQ messages to be persisted in HDFS. The messages will continuously stream into the system, as they are stock prices or weather data etc. Unfortunately it looks like I will not be able to append to a file in HDFS version 1.x.x. as per:
HADOOP-8230. Major improvement reported by eli2 and fixed by eli
Enable sync by default and disable append
Append is not supported in Hadoop 1.x. Please upgrade to 2.x if you need append. If you enabled dfs.support.append for HBase, you're OK, as durable sync (why HBase required dfs.support.append) is now enabled by default. If you really need the previous functionality, to turn on the append functionality set the flag "dfs.support.broken.append" to true.
Link: http://hadoop.apache.org/docs/r1.1.1/releasenotes.html
Could anyone please elaborate on this release note message? Why is it possible for HBase to append? Can I create a program somehow that is able to safely and robustly append to files?
I am running HortonWorks Windows distribution (that is Hadoop version 1.1.0) on a cluster of 3 machines.
Many thanks for your help in advance.
Kind Regards,
Thomas
Best regards,
Thomas Stidsborg Sylvest
Software Developer
Mobile:
+45 3141 4831
tss@danskecommodities.com <ma...@danskecommodities.com>
________________________________
[cid:image001.png@01CE92B8.39E66000]
Danske Commodities A/S
Vaerkmestergade 3
DK-8000 Aarhus C
http://www.danskecommodities.com <http://www.danskecommodities.com/>
[Visit DC on Linkedin]<http://www.linkedin.com/company/danske-commodities>
[Visit DC on facebook]<http://www.facebook.com/Danske.Commodities>
[Download my vCard]<http://emailsignature.danskecommodities.com/tss.vcf>
Danske Commodities e-mail policy <http://www.danskecommodities.com/About/Privacy>
Re: HDFS 1.1.0 - File Append
Posted by Harsh J <ha...@cloudera.com>.
HBase does not use append(…) anywhere. It only syncs(…) over its WAL stream
and although sync(…) was an API. Append on 1.x is pretty much broken for
various edge cases and therefore also marked unsupported. If you need
Flume's append usage to work reliably well, you will need to use a 2.x
based HDFS.
On Tue, Aug 6, 2013 at 6:48 PM, Thomas Stidsborg Sylvest <
tss@danskecommodities.com> wrote:
> Hi,****
>
> ** **
>
> I am in the process of developing a framework around Hadoop that enables
> RabbitMQ messages to be persisted in HDFS. The messages will continuously
> stream into the system, as they are stock prices or weather data etc.
> Unfortunately it looks like I will not be able to append to a file in HDFS
> version 1.x.x. as per: ** **
>
> ** **
>
> *HADOOP-8230. Major improvement reported by eli2 and fixed by eli *
>
> *Enable sync by default and disable append*
>
> *Append is not supported in Hadoop 1.x. Please upgrade to 2.x if you need
> append. If you enabled dfs.support.append for HBase, you're OK, as durable
> sync (why HBase required dfs.support.append) is now enabled by default. If
> you really need the previous functionality, to turn on the append
> functionality set the flag "dfs.support.broken.append" to true.*
>
> ** **
>
> Link: http://hadoop.apache.org/docs/r1.1.1/releasenotes.html****
>
> ** **
>
> Could anyone please elaborate on this release note message? Why is it
> possible for HBase to append? Can I create a program somehow that is able
> to safely and robustly append to files?****
>
> ** **
>
> I am running HortonWorks Windows distribution (that is Hadoop version
> 1.1.0) on a cluster of 3 machines.****
>
> ** **
>
> Many thanks for your help in advance.****
>
> ** **
>
> Kind Regards,****
>
> ** **
>
> Thomas****
>
> ** **
>
> Best regards, ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> Thomas Stidsborg Sylvest
> Software Developer****
>
> ****
>
> Mobile: ****
>
> +45 3141 4831 ****
>
> ****
>
> tss@danskecommodities.com ****
>
> ****
> ------------------------------
>
> ****
>
> Danske Commodities A/S
> Vaerkmestergade 3
> DK-8000 Aarhus C
>
> http://www.danskecommodities.com ****
>
> ** **
>
> [image: Visit DC on Linkedin]<http://www.linkedin.com/company/danske-commodities>
> ****
>
> ****
>
> [image: Visit DC on facebook] <http://www.facebook.com/Danske.Commodities>
> ****
>
> ****
>
> [image: Download my vCard]<http://emailsignature.danskecommodities.com/tss.vcf>
> ****
>
> Danske Commodities e-mail policy
> <http://www.danskecommodities.com/About/Privacy>****
>
--
Harsh J
Re: HDFS 1.1.0 - File Append
Posted by Harsh J <ha...@cloudera.com>.
HBase does not use append(…) anywhere. It only syncs(…) over its WAL stream
and although sync(…) was an API. Append on 1.x is pretty much broken for
various edge cases and therefore also marked unsupported. If you need
Flume's append usage to work reliably well, you will need to use a 2.x
based HDFS.
On Tue, Aug 6, 2013 at 6:48 PM, Thomas Stidsborg Sylvest <
tss@danskecommodities.com> wrote:
> Hi,****
>
> ** **
>
> I am in the process of developing a framework around Hadoop that enables
> RabbitMQ messages to be persisted in HDFS. The messages will continuously
> stream into the system, as they are stock prices or weather data etc.
> Unfortunately it looks like I will not be able to append to a file in HDFS
> version 1.x.x. as per: ** **
>
> ** **
>
> *HADOOP-8230. Major improvement reported by eli2 and fixed by eli *
>
> *Enable sync by default and disable append*
>
> *Append is not supported in Hadoop 1.x. Please upgrade to 2.x if you need
> append. If you enabled dfs.support.append for HBase, you're OK, as durable
> sync (why HBase required dfs.support.append) is now enabled by default. If
> you really need the previous functionality, to turn on the append
> functionality set the flag "dfs.support.broken.append" to true.*
>
> ** **
>
> Link: http://hadoop.apache.org/docs/r1.1.1/releasenotes.html****
>
> ** **
>
> Could anyone please elaborate on this release note message? Why is it
> possible for HBase to append? Can I create a program somehow that is able
> to safely and robustly append to files?****
>
> ** **
>
> I am running HortonWorks Windows distribution (that is Hadoop version
> 1.1.0) on a cluster of 3 machines.****
>
> ** **
>
> Many thanks for your help in advance.****
>
> ** **
>
> Kind Regards,****
>
> ** **
>
> Thomas****
>
> ** **
>
> Best regards, ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> Thomas Stidsborg Sylvest
> Software Developer****
>
> ****
>
> Mobile: ****
>
> +45 3141 4831 ****
>
> ****
>
> tss@danskecommodities.com ****
>
> ****
> ------------------------------
>
> ****
>
> Danske Commodities A/S
> Vaerkmestergade 3
> DK-8000 Aarhus C
>
> http://www.danskecommodities.com ****
>
> ** **
>
> [image: Visit DC on Linkedin]<http://www.linkedin.com/company/danske-commodities>
> ****
>
> ****
>
> [image: Visit DC on facebook] <http://www.facebook.com/Danske.Commodities>
> ****
>
> ****
>
> [image: Download my vCard]<http://emailsignature.danskecommodities.com/tss.vcf>
> ****
>
> Danske Commodities e-mail policy
> <http://www.danskecommodities.com/About/Privacy>****
>
--
Harsh J
Re: HDFS 1.1.0 - File Append
Posted by Adam Muise <am...@hortonworks.com>.
Thomas,
Try using Flume to ingest the realtime message from RabbitMQ. Flume ingests
event data and has pluggable components: source -> channel -> sink.
http://flume.apache.org
There is an HDFS sink already that allows you to land and bunch data as you
like it. It will handle all of the landing logic to create temp and
permanent files as required in HDFS. The RabbitMQ source may require some
development but it's not hard. I'll see if one of my colleagues can publish
the source code of the one we wrote.
You might want to hit the flume user group for further support.
Cheers,
On Tue, Aug 6, 2013 at 9:18 AM, Thomas Stidsborg Sylvest <
tss@danskecommodities.com> wrote:
> Hi,****
>
> ** **
>
> I am in the process of developing a framework around Hadoop that enables
> RabbitMQ messages to be persisted in HDFS. The messages will continuously
> stream into the system, as they are stock prices or weather data etc.
> Unfortunately it looks like I will not be able to append to a file in HDFS
> version 1.x.x. as per: ** **
>
> ** **
>
> *HADOOP-8230. Major improvement reported by eli2 and fixed by eli *
>
> *Enable sync by default and disable append*
>
> *Append is not supported in Hadoop 1.x. Please upgrade to 2.x if you need
> append. If you enabled dfs.support.append for HBase, you're OK, as durable
> sync (why HBase required dfs.support.append) is now enabled by default. If
> you really need the previous functionality, to turn on the append
> functionality set the flag "dfs.support.broken.append" to true.*
>
> ** **
>
> Link: http://hadoop.apache.org/docs/r1.1.1/releasenotes.html****
>
> ** **
>
> Could anyone please elaborate on this release note message? Why is it
> possible for HBase to append? Can I create a program somehow that is able
> to safely and robustly append to files?****
>
> ** **
>
> I am running HortonWorks Windows distribution (that is Hadoop version
> 1.1.0) on a cluster of 3 machines.****
>
> ** **
>
> Many thanks for your help in advance.****
>
> ** **
>
> Kind Regards,****
>
> ** **
>
> Thomas****
>
> ** **
>
> Best regards, ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> Thomas Stidsborg Sylvest
> Software Developer****
>
> ****
>
> Mobile: ****
>
> +45 3141 4831 ****
>
> ****
>
> tss@danskecommodities.com ****
>
> ****
> ------------------------------
>
> ****
>
> Danske Commodities A/S
> Vaerkmestergade 3
> DK-8000 Aarhus C
>
> http://www.danskecommodities.com ****
>
> ** **
>
> [image: Visit DC on Linkedin]<http://www.linkedin.com/company/danske-commodities>
> ****
>
> ****
>
> [image: Visit DC on facebook] <http://www.facebook.com/Danske.Commodities>
> ****
>
> ****
>
> [image: Download my vCard]<http://emailsignature.danskecommodities.com/tss.vcf>
> ****
>
> Danske Commodities e-mail policy
> <http://www.danskecommodities.com/About/Privacy>****
>
--
*
*
*
*
*Adam Muise*
Solution Engineer
*Hortonworks*
amuise@hortonworks.com
416-417-4037
Hortonworks - Develops, Distributes and Supports Enterprise Apache
Hadoop.<http://hortonworks.com/>
Hortonworks Virtual Sandbox <http://hortonworks.com/sandbox>
Hadoop: Disruptive Possibilities by Jeff
Needham<http://hortonworks.com/resources/?did=72&cat=1>
Re: HDFS 1.1.0 - File Append
Posted by Harsh J <ha...@cloudera.com>.
HBase does not use append(…) anywhere. It only syncs(…) over its WAL stream
and although sync(…) was an API. Append on 1.x is pretty much broken for
various edge cases and therefore also marked unsupported. If you need
Flume's append usage to work reliably well, you will need to use a 2.x
based HDFS.
On Tue, Aug 6, 2013 at 6:48 PM, Thomas Stidsborg Sylvest <
tss@danskecommodities.com> wrote:
> Hi,****
>
> ** **
>
> I am in the process of developing a framework around Hadoop that enables
> RabbitMQ messages to be persisted in HDFS. The messages will continuously
> stream into the system, as they are stock prices or weather data etc.
> Unfortunately it looks like I will not be able to append to a file in HDFS
> version 1.x.x. as per: ** **
>
> ** **
>
> *HADOOP-8230. Major improvement reported by eli2 and fixed by eli *
>
> *Enable sync by default and disable append*
>
> *Append is not supported in Hadoop 1.x. Please upgrade to 2.x if you need
> append. If you enabled dfs.support.append for HBase, you're OK, as durable
> sync (why HBase required dfs.support.append) is now enabled by default. If
> you really need the previous functionality, to turn on the append
> functionality set the flag "dfs.support.broken.append" to true.*
>
> ** **
>
> Link: http://hadoop.apache.org/docs/r1.1.1/releasenotes.html****
>
> ** **
>
> Could anyone please elaborate on this release note message? Why is it
> possible for HBase to append? Can I create a program somehow that is able
> to safely and robustly append to files?****
>
> ** **
>
> I am running HortonWorks Windows distribution (that is Hadoop version
> 1.1.0) on a cluster of 3 machines.****
>
> ** **
>
> Many thanks for your help in advance.****
>
> ** **
>
> Kind Regards,****
>
> ** **
>
> Thomas****
>
> ** **
>
> Best regards, ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> Thomas Stidsborg Sylvest
> Software Developer****
>
> ****
>
> Mobile: ****
>
> +45 3141 4831 ****
>
> ****
>
> tss@danskecommodities.com ****
>
> ****
> ------------------------------
>
> ****
>
> Danske Commodities A/S
> Vaerkmestergade 3
> DK-8000 Aarhus C
>
> http://www.danskecommodities.com ****
>
> ** **
>
> [image: Visit DC on Linkedin]<http://www.linkedin.com/company/danske-commodities>
> ****
>
> ****
>
> [image: Visit DC on facebook] <http://www.facebook.com/Danske.Commodities>
> ****
>
> ****
>
> [image: Download my vCard]<http://emailsignature.danskecommodities.com/tss.vcf>
> ****
>
> Danske Commodities e-mail policy
> <http://www.danskecommodities.com/About/Privacy>****
>
--
Harsh J
Re: HDFS 1.1.0 - File Append
Posted by Harsh J <ha...@cloudera.com>.
HBase does not use append(…) anywhere. It only syncs(…) over its WAL stream
and although sync(…) was an API. Append on 1.x is pretty much broken for
various edge cases and therefore also marked unsupported. If you need
Flume's append usage to work reliably well, you will need to use a 2.x
based HDFS.
On Tue, Aug 6, 2013 at 6:48 PM, Thomas Stidsborg Sylvest <
tss@danskecommodities.com> wrote:
> Hi,****
>
> ** **
>
> I am in the process of developing a framework around Hadoop that enables
> RabbitMQ messages to be persisted in HDFS. The messages will continuously
> stream into the system, as they are stock prices or weather data etc.
> Unfortunately it looks like I will not be able to append to a file in HDFS
> version 1.x.x. as per: ** **
>
> ** **
>
> *HADOOP-8230. Major improvement reported by eli2 and fixed by eli *
>
> *Enable sync by default and disable append*
>
> *Append is not supported in Hadoop 1.x. Please upgrade to 2.x if you need
> append. If you enabled dfs.support.append for HBase, you're OK, as durable
> sync (why HBase required dfs.support.append) is now enabled by default. If
> you really need the previous functionality, to turn on the append
> functionality set the flag "dfs.support.broken.append" to true.*
>
> ** **
>
> Link: http://hadoop.apache.org/docs/r1.1.1/releasenotes.html****
>
> ** **
>
> Could anyone please elaborate on this release note message? Why is it
> possible for HBase to append? Can I create a program somehow that is able
> to safely and robustly append to files?****
>
> ** **
>
> I am running HortonWorks Windows distribution (that is Hadoop version
> 1.1.0) on a cluster of 3 machines.****
>
> ** **
>
> Many thanks for your help in advance.****
>
> ** **
>
> Kind Regards,****
>
> ** **
>
> Thomas****
>
> ** **
>
> Best regards, ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> Thomas Stidsborg Sylvest
> Software Developer****
>
> ****
>
> Mobile: ****
>
> +45 3141 4831 ****
>
> ****
>
> tss@danskecommodities.com ****
>
> ****
> ------------------------------
>
> ****
>
> Danske Commodities A/S
> Vaerkmestergade 3
> DK-8000 Aarhus C
>
> http://www.danskecommodities.com ****
>
> ** **
>
> [image: Visit DC on Linkedin]<http://www.linkedin.com/company/danske-commodities>
> ****
>
> ****
>
> [image: Visit DC on facebook] <http://www.facebook.com/Danske.Commodities>
> ****
>
> ****
>
> [image: Download my vCard]<http://emailsignature.danskecommodities.com/tss.vcf>
> ****
>
> Danske Commodities e-mail policy
> <http://www.danskecommodities.com/About/Privacy>****
>
--
Harsh J
Re: HDFS 1.1.0 - File Append
Posted by Adam Muise <am...@hortonworks.com>.
Thomas,
Try using Flume to ingest the realtime message from RabbitMQ. Flume ingests
event data and has pluggable components: source -> channel -> sink.
http://flume.apache.org
There is an HDFS sink already that allows you to land and bunch data as you
like it. It will handle all of the landing logic to create temp and
permanent files as required in HDFS. The RabbitMQ source may require some
development but it's not hard. I'll see if one of my colleagues can publish
the source code of the one we wrote.
You might want to hit the flume user group for further support.
Cheers,
On Tue, Aug 6, 2013 at 9:18 AM, Thomas Stidsborg Sylvest <
tss@danskecommodities.com> wrote:
> Hi,****
>
> ** **
>
> I am in the process of developing a framework around Hadoop that enables
> RabbitMQ messages to be persisted in HDFS. The messages will continuously
> stream into the system, as they are stock prices or weather data etc.
> Unfortunately it looks like I will not be able to append to a file in HDFS
> version 1.x.x. as per: ** **
>
> ** **
>
> *HADOOP-8230. Major improvement reported by eli2 and fixed by eli *
>
> *Enable sync by default and disable append*
>
> *Append is not supported in Hadoop 1.x. Please upgrade to 2.x if you need
> append. If you enabled dfs.support.append for HBase, you're OK, as durable
> sync (why HBase required dfs.support.append) is now enabled by default. If
> you really need the previous functionality, to turn on the append
> functionality set the flag "dfs.support.broken.append" to true.*
>
> ** **
>
> Link: http://hadoop.apache.org/docs/r1.1.1/releasenotes.html****
>
> ** **
>
> Could anyone please elaborate on this release note message? Why is it
> possible for HBase to append? Can I create a program somehow that is able
> to safely and robustly append to files?****
>
> ** **
>
> I am running HortonWorks Windows distribution (that is Hadoop version
> 1.1.0) on a cluster of 3 machines.****
>
> ** **
>
> Many thanks for your help in advance.****
>
> ** **
>
> Kind Regards,****
>
> ** **
>
> Thomas****
>
> ** **
>
> Best regards, ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> Thomas Stidsborg Sylvest
> Software Developer****
>
> ****
>
> Mobile: ****
>
> +45 3141 4831 ****
>
> ****
>
> tss@danskecommodities.com ****
>
> ****
> ------------------------------
>
> ****
>
> Danske Commodities A/S
> Vaerkmestergade 3
> DK-8000 Aarhus C
>
> http://www.danskecommodities.com ****
>
> ** **
>
> [image: Visit DC on Linkedin]<http://www.linkedin.com/company/danske-commodities>
> ****
>
> ****
>
> [image: Visit DC on facebook] <http://www.facebook.com/Danske.Commodities>
> ****
>
> ****
>
> [image: Download my vCard]<http://emailsignature.danskecommodities.com/tss.vcf>
> ****
>
> Danske Commodities e-mail policy
> <http://www.danskecommodities.com/About/Privacy>****
>
--
*
*
*
*
*Adam Muise*
Solution Engineer
*Hortonworks*
amuise@hortonworks.com
416-417-4037
Hortonworks - Develops, Distributes and Supports Enterprise Apache
Hadoop.<http://hortonworks.com/>
Hortonworks Virtual Sandbox <http://hortonworks.com/sandbox>
Hadoop: Disruptive Possibilities by Jeff
Needham<http://hortonworks.com/resources/?did=72&cat=1>
Re: HDFS 1.1.0 - File Append
Posted by Adam Muise <am...@hortonworks.com>.
Thomas,
Try using Flume to ingest the realtime message from RabbitMQ. Flume ingests
event data and has pluggable components: source -> channel -> sink.
http://flume.apache.org
There is an HDFS sink already that allows you to land and bunch data as you
like it. It will handle all of the landing logic to create temp and
permanent files as required in HDFS. The RabbitMQ source may require some
development but it's not hard. I'll see if one of my colleagues can publish
the source code of the one we wrote.
You might want to hit the flume user group for further support.
Cheers,
On Tue, Aug 6, 2013 at 9:18 AM, Thomas Stidsborg Sylvest <
tss@danskecommodities.com> wrote:
> Hi,****
>
> ** **
>
> I am in the process of developing a framework around Hadoop that enables
> RabbitMQ messages to be persisted in HDFS. The messages will continuously
> stream into the system, as they are stock prices or weather data etc.
> Unfortunately it looks like I will not be able to append to a file in HDFS
> version 1.x.x. as per: ** **
>
> ** **
>
> *HADOOP-8230. Major improvement reported by eli2 and fixed by eli *
>
> *Enable sync by default and disable append*
>
> *Append is not supported in Hadoop 1.x. Please upgrade to 2.x if you need
> append. If you enabled dfs.support.append for HBase, you're OK, as durable
> sync (why HBase required dfs.support.append) is now enabled by default. If
> you really need the previous functionality, to turn on the append
> functionality set the flag "dfs.support.broken.append" to true.*
>
> ** **
>
> Link: http://hadoop.apache.org/docs/r1.1.1/releasenotes.html****
>
> ** **
>
> Could anyone please elaborate on this release note message? Why is it
> possible for HBase to append? Can I create a program somehow that is able
> to safely and robustly append to files?****
>
> ** **
>
> I am running HortonWorks Windows distribution (that is Hadoop version
> 1.1.0) on a cluster of 3 machines.****
>
> ** **
>
> Many thanks for your help in advance.****
>
> ** **
>
> Kind Regards,****
>
> ** **
>
> Thomas****
>
> ** **
>
> Best regards, ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> Thomas Stidsborg Sylvest
> Software Developer****
>
> ****
>
> Mobile: ****
>
> +45 3141 4831 ****
>
> ****
>
> tss@danskecommodities.com ****
>
> ****
> ------------------------------
>
> ****
>
> Danske Commodities A/S
> Vaerkmestergade 3
> DK-8000 Aarhus C
>
> http://www.danskecommodities.com ****
>
> ** **
>
> [image: Visit DC on Linkedin]<http://www.linkedin.com/company/danske-commodities>
> ****
>
> ****
>
> [image: Visit DC on facebook] <http://www.facebook.com/Danske.Commodities>
> ****
>
> ****
>
> [image: Download my vCard]<http://emailsignature.danskecommodities.com/tss.vcf>
> ****
>
> Danske Commodities e-mail policy
> <http://www.danskecommodities.com/About/Privacy>****
>
--
*
*
*
*
*Adam Muise*
Solution Engineer
*Hortonworks*
amuise@hortonworks.com
416-417-4037
Hortonworks - Develops, Distributes and Supports Enterprise Apache
Hadoop.<http://hortonworks.com/>
Hortonworks Virtual Sandbox <http://hortonworks.com/sandbox>
Hadoop: Disruptive Possibilities by Jeff
Needham<http://hortonworks.com/resources/?did=72&cat=1>
Re: HDFS 1.1.0 - File Append
Posted by Adam Muise <am...@hortonworks.com>.
Thomas,
Try using Flume to ingest the realtime message from RabbitMQ. Flume ingests
event data and has pluggable components: source -> channel -> sink.
http://flume.apache.org
There is an HDFS sink already that allows you to land and bunch data as you
like it. It will handle all of the landing logic to create temp and
permanent files as required in HDFS. The RabbitMQ source may require some
development but it's not hard. I'll see if one of my colleagues can publish
the source code of the one we wrote.
You might want to hit the flume user group for further support.
Cheers,
On Tue, Aug 6, 2013 at 9:18 AM, Thomas Stidsborg Sylvest <
tss@danskecommodities.com> wrote:
> Hi,****
>
> ** **
>
> I am in the process of developing a framework around Hadoop that enables
> RabbitMQ messages to be persisted in HDFS. The messages will continuously
> stream into the system, as they are stock prices or weather data etc.
> Unfortunately it looks like I will not be able to append to a file in HDFS
> version 1.x.x. as per: ** **
>
> ** **
>
> *HADOOP-8230. Major improvement reported by eli2 and fixed by eli *
>
> *Enable sync by default and disable append*
>
> *Append is not supported in Hadoop 1.x. Please upgrade to 2.x if you need
> append. If you enabled dfs.support.append for HBase, you're OK, as durable
> sync (why HBase required dfs.support.append) is now enabled by default. If
> you really need the previous functionality, to turn on the append
> functionality set the flag "dfs.support.broken.append" to true.*
>
> ** **
>
> Link: http://hadoop.apache.org/docs/r1.1.1/releasenotes.html****
>
> ** **
>
> Could anyone please elaborate on this release note message? Why is it
> possible for HBase to append? Can I create a program somehow that is able
> to safely and robustly append to files?****
>
> ** **
>
> I am running HortonWorks Windows distribution (that is Hadoop version
> 1.1.0) on a cluster of 3 machines.****
>
> ** **
>
> Many thanks for your help in advance.****
>
> ** **
>
> Kind Regards,****
>
> ** **
>
> Thomas****
>
> ** **
>
> Best regards, ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> ****
>
> Thomas Stidsborg Sylvest
> Software Developer****
>
> ****
>
> Mobile: ****
>
> +45 3141 4831 ****
>
> ****
>
> tss@danskecommodities.com ****
>
> ****
> ------------------------------
>
> ****
>
> Danske Commodities A/S
> Vaerkmestergade 3
> DK-8000 Aarhus C
>
> http://www.danskecommodities.com ****
>
> ** **
>
> [image: Visit DC on Linkedin]<http://www.linkedin.com/company/danske-commodities>
> ****
>
> ****
>
> [image: Visit DC on facebook] <http://www.facebook.com/Danske.Commodities>
> ****
>
> ****
>
> [image: Download my vCard]<http://emailsignature.danskecommodities.com/tss.vcf>
> ****
>
> Danske Commodities e-mail policy
> <http://www.danskecommodities.com/About/Privacy>****
>
--
*
*
*
*
*Adam Muise*
Solution Engineer
*Hortonworks*
amuise@hortonworks.com
416-417-4037
Hortonworks - Develops, Distributes and Supports Enterprise Apache
Hadoop.<http://hortonworks.com/>
Hortonworks Virtual Sandbox <http://hortonworks.com/sandbox>
Hadoop: Disruptive Possibilities by Jeff
Needham<http://hortonworks.com/resources/?did=72&cat=1>