You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openoffice.apache.org by Rory O'Farrell <of...@iol.ie> on 2017/07/31 16:39:05 UTC
Save process and files turned to hashes
John_Ha, a valued contributor to the OO User Forum, has asked me to post this to the developer list. It is a continuation of a previous thread with the same title at
http://www.mail-archive.com/dev@openoffice.apache.org/msg15177.html
-----------------------
John_Ha writes
I think that the hashtags are misleading.
The problem is that occasionally Writer creates a flat ASCII file which is full of NULL characters and saves the file as a .odt file.
This .odt file is NOT a zipped file and it does NOT have any of the usual (content.xml or styles.xml) files. This .odt file is just a flat ASCII file, often very large (the same size as the original document?), but completely full of NULL characters. Go to https://forum.openoffice.org/en/forum/ucp.php?i=pm&mode=view&f=0&p=11590 for an example file - crappyfile.odt is 24 kBytes of NULL characters.
Go to [Hint] How did I fix my ODT file at https://forum.openoffice.org/en/forum/viewtopic.php?f=7&t=1532. The thread has been viewed 194,820 times suggesting this is a serious problem of considerable interest to users. No other thread in the forum has been read as often.
When the user attempts to open what is effectively a flat ASCII file, Writer recognises it is an ASCII file and opens it as if it was a .txt file, and offers the filter pop-up. The NULL characters are then displayed as hashtags.
I think that the questions the developers ought to be asking include:
1 At what point in the save process is space on the disk reserved to write the .odt file?
2 Why is this space full of NULL characters - why isn't it random junk from the disk? How are the NULL characters written?
3 What happens to prevent the genuine file being written?
4 Why is the file full of NULL characters saved as a .odt file?
5 How can Writer save a file as a .odt file which is not a ZIP file? Why was the ZIP process not activated before the file was saved?
6 Note that Writer continues to write to the disk long after (as much as 30 seconds after on a slow network installation) the blue dotted bar crossing the bottom of the screen has stopped. Does this have an effect? What happens if something interrupts Writer while it is doing these silent writes?
7 There are many, many problems seen on the forum (e.g. spell check stops working) which are fixed by creating a new User Profile. As parts of the User Profile (eg registrymodifications.xcu and others) seem to get written AFTER the .odt file has been written, is a corrupted User Profile a manifestation of the same, or similar, problem?
8 Can the .odt file be written as an atomic process such that either "the file as it was when it opened for editing" is saved; or "the file as it is now" is saved. Note that the temporary file C:\Users\my_name\AppData\Local\Temp\svftc2x7.tmp\svftdera.tmp (or similar random name) is a copy of the .odt file as it was opened; and is only deleted when the file is saved. Can a check be made and this temporary file not be deleted until it is known that the proper .odt file has been successfully saved?
9 It is only GUESSING which suggests that over hasty shutting of a laptop lid could be the cause of this. I struggle to see how this could cause it because I understood that hibernation / shutting a laptop lid causes a graceful shut down, and not one where data might be lost. If this is the problem, then is the long delay after the blue bar has ceased causing the problem, and any data waiting to be written is lost? Does Writer handle the "graceful shutdown" instruction from Windows properly?
10 I also think that USB sticks are a red-herring. Later versions of Windows come with the default setting of not using cacheing (the user has to switch it on) so USB sticks can be withdrawn very soon.
In conclusion: I think it needs an analysis of what happens during a Save to understand
a) at what point is a large, flat ASCII file full of NULL characters created?
b) how can this file be saved as a .odt file when .odt files are ZIPped files?
I think that this analysis will lead to a better understanding of where the problem lies.
John_Ha
-----------------------------------
--
Rory O'Farrell <of...@iol.ie>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: Save process and files turned to hashes
Posted by Rory O'Farrell <of...@iol.ie>.
Patricia
I have forwarded your message to John.
Rory
On Mon, 31 Jul 2017 10:09:59 -0700
Patricia Shanahan <pa...@acm.org> wrote:
> This issue is my current focus, so the analysis is very valuable and
> timely. It seems likely that there is a common problem, applying to both
> documents and profiles, of AOO believing it has finished writing
> something too early.
>
> Unfortunately, I don't think we have any developers who are familiar
> with AOO's file write code. I am trying to learn my way around it, but
> that may take some time.
>
> It would be useful to have a survey of the configurations in which these
> problems happen. Are they associated with specific operating systems?
> More prevalent with network drives?
>
> Would it be useful for me to read the original thread, or is this an
> effective summary?
>
> On 7/31/2017 9:39 AM, Rory O'Farrell wrote:
> > John_Ha, a valued contributor to the OO User Forum, has asked me to post this to the developer list. It is a continuation of a previous thread with the same title at
> > http://www.mail-archive.com/dev@openoffice.apache.org/msg15177.html
> >
> > -----------------------
> > John_Ha writes
> >
> > I think that the hashtags are misleading.
> >
> > The problem is that occasionally Writer creates a flat ASCII file which is full of NULL characters and saves the file as a .odt file.
> >
> > This .odt file is NOT a zipped file and it does NOT have any of the usual (content.xml or styles.xml) files. This .odt file is just a flat ASCII file, often very large (the same size as the original document?), but completely full of NULL characters. Go to https://forum.openoffice.org/en/forum/ucp.php?i=pm&mode=view&f=0&p=11590 for an example file - crappyfile.odt is 24 kBytes of NULL characters.
> >
> > Go to [Hint] How did I fix my ODT file at https://forum.openoffice.org/en/forum/viewtopic.php?f=7&t=1532. The thread has been viewed 194,820 times suggesting this is a serious problem of considerable interest to users. No other thread in the forum has been read as often.
> >
> > When the user attempts to open what is effectively a flat ASCII file, Writer recognises it is an ASCII file and opens it as if it was a .txt file, and offers the filter pop-up. The NULL characters are then displayed as hashtags.
> >
> > I think that the questions the developers ought to be asking include:
> >
> > 1 At what point in the save process is space on the disk reserved to write the .odt file?
> >
> > 2 Why is this space full of NULL characters - why isn't it random junk from the disk? How are the NULL characters written?
> >
> > 3 What happens to prevent the genuine file being written?
> >
> > 4 Why is the file full of NULL characters saved as a .odt file?
> >
> > 5 How can Writer save a file as a .odt file which is not a ZIP file? Why was the ZIP process not activated before the file was saved?
> >
> > 6 Note that Writer continues to write to the disk long after (as much as 30 seconds after on a slow network installation) the blue dotted bar crossing the bottom of the screen has stopped. Does this have an effect? What happens if something interrupts Writer while it is doing these silent writes?
> >
> > 7 There are many, many problems seen on the forum (e.g. spell check stops working) which are fixed by creating a new User Profile. As parts of the User Profile (eg registrymodifications.xcu and others) seem to get written AFTER the .odt file has been written, is a corrupted User Profile a manifestation of the same, or similar, problem?
> >
> > 8 Can the .odt file be written as an atomic process such that either "the file as it was when it opened for editing" is saved; or "the file as it is now" is saved. Note that the temporary file C:\Users\my_name\AppData\Local\Temp\svftc2x7.tmp\svftdera.tmp (or similar random name) is a copy of the .odt file as it was opened; and is only deleted when the file is saved. Can a check be made and this temporary file not be deleted until it is known that the proper .odt file has been successfully saved?
> >
> > 9 It is only GUESSING which suggests that over hasty shutting of a laptop lid could be the cause of this. I struggle to see how this could cause it because I understood that hibernation / shutting a laptop lid causes a graceful shut down, and not one where data might be lost. If this is the problem, then is the long delay after the blue bar has ceased causing the problem, and any data waiting to be written is lost? Does Writer handle the "graceful shutdown" instruction from Windows properly?
> >
> > 10 I also think that USB sticks are a red-herring. Later versions of Windows come with the default setting of not using cacheing (the user has to switch it on) so USB sticks can be withdrawn very soon.
> >
> > In conclusion: I think it needs an analysis of what happens during a Save to understand
> >
> > a) at what point is a large, flat ASCII file full of NULL characters created?
> >
> > b) how can this file be saved as a .odt file when .odt files are ZIPped files?
> >
> > I think that this analysis will lead to a better understanding of where the problem lies.
> >
> > John_Ha
> > -----------------------------------
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>
--
Rory O'Farrell <of...@iol.ie>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: Save process and files turned to hashes
Posted by Hagar Delest <de...@gmail.com>.
Le 31/07/2017 à 19:09, Patricia Shanahan a écrit :
> It would be useful to have a survey of the configurations in which these problems happen. Are they associated with specific operating systems? More prevalent with network drives?
See: https://forum.openoffice.org/en/forum/viewtopic.php?f=6&t=17677#p81363
Mostly Windows users since it's the most common platform. But some other operating systems also.
Very few cases with network drives.
Hagar
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: Save process and files turned to hashes
Posted by Patricia Shanahan <pa...@acm.org>.
This issue is my current focus, so the analysis is very valuable and
timely. It seems likely that there is a common problem, applying to both
documents and profiles, of AOO believing it has finished writing
something too early.
Unfortunately, I don't think we have any developers who are familiar
with AOO's file write code. I am trying to learn my way around it, but
that may take some time.
It would be useful to have a survey of the configurations in which these
problems happen. Are they associated with specific operating systems?
More prevalent with network drives?
Would it be useful for me to read the original thread, or is this an
effective summary?
On 7/31/2017 9:39 AM, Rory O'Farrell wrote:
> John_Ha, a valued contributor to the OO User Forum, has asked me to post this to the developer list. It is a continuation of a previous thread with the same title at
> http://www.mail-archive.com/dev@openoffice.apache.org/msg15177.html
>
> -----------------------
> John_Ha writes
>
> I think that the hashtags are misleading.
>
> The problem is that occasionally Writer creates a flat ASCII file which is full of NULL characters and saves the file as a .odt file.
>
> This .odt file is NOT a zipped file and it does NOT have any of the usual (content.xml or styles.xml) files. This .odt file is just a flat ASCII file, often very large (the same size as the original document?), but completely full of NULL characters. Go to https://forum.openoffice.org/en/forum/ucp.php?i=pm&mode=view&f=0&p=11590 for an example file - crappyfile.odt is 24 kBytes of NULL characters.
>
> Go to [Hint] How did I fix my ODT file at https://forum.openoffice.org/en/forum/viewtopic.php?f=7&t=1532. The thread has been viewed 194,820 times suggesting this is a serious problem of considerable interest to users. No other thread in the forum has been read as often.
>
> When the user attempts to open what is effectively a flat ASCII file, Writer recognises it is an ASCII file and opens it as if it was a .txt file, and offers the filter pop-up. The NULL characters are then displayed as hashtags.
>
> I think that the questions the developers ought to be asking include:
>
> 1 At what point in the save process is space on the disk reserved to write the .odt file?
>
> 2 Why is this space full of NULL characters - why isn't it random junk from the disk? How are the NULL characters written?
>
> 3 What happens to prevent the genuine file being written?
>
> 4 Why is the file full of NULL characters saved as a .odt file?
>
> 5 How can Writer save a file as a .odt file which is not a ZIP file? Why was the ZIP process not activated before the file was saved?
>
> 6 Note that Writer continues to write to the disk long after (as much as 30 seconds after on a slow network installation) the blue dotted bar crossing the bottom of the screen has stopped. Does this have an effect? What happens if something interrupts Writer while it is doing these silent writes?
>
> 7 There are many, many problems seen on the forum (e.g. spell check stops working) which are fixed by creating a new User Profile. As parts of the User Profile (eg registrymodifications.xcu and others) seem to get written AFTER the .odt file has been written, is a corrupted User Profile a manifestation of the same, or similar, problem?
>
> 8 Can the .odt file be written as an atomic process such that either "the file as it was when it opened for editing" is saved; or "the file as it is now" is saved. Note that the temporary file C:\Users\my_name\AppData\Local\Temp\svftc2x7.tmp\svftdera.tmp (or similar random name) is a copy of the .odt file as it was opened; and is only deleted when the file is saved. Can a check be made and this temporary file not be deleted until it is known that the proper .odt file has been successfully saved?
>
> 9 It is only GUESSING which suggests that over hasty shutting of a laptop lid could be the cause of this. I struggle to see how this could cause it because I understood that hibernation / shutting a laptop lid causes a graceful shut down, and not one where data might be lost. If this is the problem, then is the long delay after the blue bar has ceased causing the problem, and any data waiting to be written is lost? Does Writer handle the "graceful shutdown" instruction from Windows properly?
>
> 10 I also think that USB sticks are a red-herring. Later versions of Windows come with the default setting of not using cacheing (the user has to switch it on) so USB sticks can be withdrawn very soon.
>
> In conclusion: I think it needs an analysis of what happens during a Save to understand
>
> a) at what point is a large, flat ASCII file full of NULL characters created?
>
> b) how can this file be saved as a .odt file when .odt files are ZIPped files?
>
> I think that this analysis will lead to a better understanding of where the problem lies.
>
> John_Ha
> -----------------------------------
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org