You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Sai Pullabhotla <sa...@jmethods.com> on 2009/05/26 12:38:29 UTC
Re: [jira] Commented: (FTPSERVER-306) some clients won't transform
NEW LINE characters to \r\n when sending in ASCII mode so after sending a
file the new lines will be gone.
>> when FtpServer receives a file in ASCII mode our current code replaces \r
for \r\n and ignores \n, but since it seems FileZilla
>> doesn't transform new lines to \r\n we will never find a \r and the new
line characters will be silently ignored.
Hmm... shouldn't it be just doing the following:
1. Replace \r\n with System.getProperty("line.separator");
2. Write everything else as is.
Sai Pullabhotla
www.jMethods.com
On Tue, May 26, 2009 at 2:34 AM, David Latorre (JIRA) <ji...@apache.org>wrote:
>
> [
> https://issues.apache.org/jira/browse/FTPSERVER-306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712889#action_12712889]
>
> David Latorre commented on FTPSERVER-306:
> -----------------------------------------
>
> Sending a file with \n separated lines in ASCII mode with FileZilla for
> windows will result in a file with no new lines:
>
> when FtpServer receives a file in ASCII mode our current code replaces \r
> for \r\n and ignores \n, but since it seems FileZilla doesn't transform new
> lines to \r\n we will never find a \r and the new line characters will be
> silently ignored.
>
> This is a bug in Filezilla (Might be because im sending "Unix" files from
> windows) but I doubt every other client is performing this transformation.
> So, when we find a \n we should check if last character was \r and otherwise
> insert the new line sequence.
>
>
>
>
>
>
> > some clients won't transform NEW LINE characters to \r\n when sending in
> ASCII mode so after sending a file the new lines will be gone.
> >
> -----------------------------------------------------------------------------------------------------------------------------------------
> >
> > Key: FTPSERVER-306
> > URL: https://issues.apache.org/jira/browse/FTPSERVER-306
> > Project: FtpServer
> > Issue Type: Improvement
> > Reporter: David Latorre
> >
>
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
Re: [jira] Commented: (FTPSERVER-306) some clients won't transform
NEW LINE characters to \r\n when sending in ASCII mode so after sending a
file the new lines will be gone.
Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Wed, May 27, 2009 at 8:43 PM, Niklas Gustavsson <ni...@protocol7.com> wrote:
> On Wed, May 27, 2009 at 3:07 PM, David Latorre <dv...@gmail.com> wrote:
>> I added a fix and some tests but ToNetASCIIOutputStream in
>> commons-util will always replace to \r\n so I'm uploading a modified
>> version. With this fix ascii mode should not eat line separators
>> anymore.
Oh, also, do we need to handle \n line endings?
/niklas
Re: [jira] Commented: (FTPSERVER-306) some clients won't transform
NEW LINE characters to \r\n when sending in ASCII mode so after sending a
file the new lines will be gone.
Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Thu, May 28, 2009 at 10:09 AM, David Latorre <dv...@gmail.com> wrote:
> Oh sorry I didn't understand you before, that's of course another
> option - we only transform the canonical line ending and leave the
> rest as is. I don't know what are the advantages / caveats of those
> options though. Your proposal seems to be more strict to the standard
> and thus, most preferrable. But in practice, this could mean that
> sending the very same file from commons-ftp will result in a
> different file than if you sent it from filezilla (I don't know if
> clients take the filezilla approach - just convert the system's line
> ending or they do as commons-ftp, convert all of them). But I guess
> this is not an issue, they're just line ending in the end ...
>
> The discussion about the correctness of Filezilla in its
> interpretation to the RFC is not conclusive either. RFC says: " The
> sender converts the data from an internal character representation to
> the standard 8-bit NVT-ASCII" , just "an internal representation",
> not "the internal represention for its system" ...
>
>
> So unless someone has a strong reason to need uniform EOLs
> corresponding to the target system, then we might assume that the
> issue with different clients resulting in different files can be
> ignored, and transform only the canonical line ending.
I don't have an real opinion on this, merely asking the question.
Maybe we should have a look at what other servers do.
> By the way, do we support REST when using ASCII mode? I would say no,
> but i'm not checking the code right now.
Looks like we do, which could lead to file corruption right?
/niklas
Re: [jira] Commented: (FTPSERVER-306) some clients won't transform
NEW LINE characters to \r\n when sending in ASCII mode so after sending a
file the new lines will be gone.
Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Thu, May 28, 2009 at 12:20 PM, Sai Pullabhotla
<sa...@jmethods.com> wrote:
> I agree with Dave Roberts - On STOR, tranform CRLFs to local EOL. On RETR,
> transform local EOL to CRLF. Absolutely no other changes.
After thinking some more about this, I agree.
/niklas
Re: [jira] Commented: (FTPSERVER-306) some clients won't transform
NEW LINE characters to \r\n when sending in ASCII mode so after sending a
file the new lines will be gone.
Posted by Sai Pullabhotla <sa...@jmethods.com>.
I agree with Dave Roberts - On STOR, tranform CRLFs to local EOL. On RETR,
transform local EOL to CRLF. Absolutely no other changes.
Sai Pullabhotla
www.jMethods.com
On Thu, May 28, 2009 at 4:50 AM, Dave Roberts <
dave.roberts@saaconsultants.com> wrote:
> David Latorre wrote:
> > So unless someone has a strong reason to need uniform EOLs
> >
> > corresponding to the target system, then we might assume that the
> >
> > issue with different clients resulting in different files can be
> >
> > ignored, and transform only the canonical line ending.
>
> My understanding of the spec has always been that in ASCII mode,
> both sides must convert between the local OS specific line endings
> and the canonical CR/LF pairing. If the file isn't in the correct
> format for the local OS, then the right results cannot be guaranteed.
>
> So I would agree that we should only transform to/from the canonical
> line ending.
>
Re: [jira] Commented: (FTPSERVER-306) some clients won't transform
NEW LINE characters to \r\n when sending in ASCII mode so after sending
a file the new lines will be gone.
Posted by Dave Roberts <da...@saaconsultants.com>.
David Latorre wrote:
> So unless someone has a strong reason to need uniform EOLs
>
> corresponding to the target system, then we might assume that the
>
> issue with different clients resulting in different files can be
>
> ignored, and transform only the canonical line ending.
My understanding of the spec has always been that in ASCII mode,
both sides must convert between the local OS specific line endings
and the canonical CR/LF pairing. If the file isn't in the correct
format for the local OS, then the right results cannot be guaranteed.
So I would agree that we should only transform to/from the canonical
line ending.
Re: [jira] Commented: (FTPSERVER-306) some clients won't transform
NEW LINE characters to \r\n when sending in ASCII mode so after sending a
file the new lines will be gone.
Posted by David Latorre <dv...@gmail.com>.
2009/5/28 Niklas Gustavsson <ni...@protocol7.com>:
> Sorry, my bad, I mixed up \r and \n. Let me rephrase:
>
> 1. If we get sent a \n only, should we print EOL or \n? Since \n is
> not the canonical line ending in the RFC, we might not want to treat
> this as a line ending that we should convert into the local line
> ending.
> 2. if we get sent a \r\n, we should clearly print EOL
> 3. If we get sent a \r only, we currently print EOL. Given the
> argument above, is this correct?
>
Oh sorry I didn't understand you before, that's of course another
option - we only transform the canonical line ending and leave the
rest as is. I don't know what are the advantages / caveats of those
options though. Your proposal seems to be more strict to the standard
and thus, most preferrable. But in practice, this could mean that
sending the very same file from commons-ftp will result in a
different file than if you sent it from filezilla (I don't know if
clients take the filezilla approach - just convert the system's line
ending or they do as commons-ftp, convert all of them). But I guess
this is not an issue, they're just line ending in the end ...
The discussion about the correctness of Filezilla in its
interpretation to the RFC is not conclusive either. RFC says: " The
sender converts the data from an internal character representation to
the standard 8-bit NVT-ASCII" , just "an internal representation",
not "the internal represention for its system" ...
So unless someone has a strong reason to need uniform EOLs
corresponding to the target system, then we might assume that the
issue with different clients resulting in different files can be
ignored, and transform only the canonical line ending.
By the way, do we support REST when using ASCII mode? I would say no,
but i'm not checking the code right now.
> /niklas
>
> On Thu, May 28, 2009 at 12:13 AM, David Latorre <dv...@gmail.com> wrote:
>> 2009/5/27 Niklas Gustavsson <ni...@protocol7.com>:
>>> On Wed, May 27, 2009 at 3:07 PM, David Latorre <dv...@gmail.com> wrote:
>>>> I added a fix and some tests but ToNetASCIIOutputStream in
>>>> commons-util will always replace to \r\n so I'm uploading a modified
>>>> version. With this fix ascii mode should not eat line separators
>>>> anymore.
>>>
>>> Thanks! Looking into the fix I'm not sure we should print EOL on \r.
>>> Since \r is not the canonical line ending in the RFC, I think we
>>> should rather print \r as it is in the stream. What do you think?
>>
>> I don't really know if \r has other users than being part of the New
>> Line sequence. I guess \r could take us back to the beginning of the
>> line ... but it doesn't make much sense in a file.
>>
>> My reasons to leave \r as the character substituted by EOL are 1)
>> Because it was previously written in that way. 2) Mac OS 9 and
>> before had \r as the new line separator, so if the line endings of a
>> Mac OS 9 text file weren't transformed to \r\n and we were ignoring
>> \r , we would swallow the new line again. On the other hand, if we
>> write the \r character, we cannot transform to Unix line endings
>> which have no \r...
>>
>> So, I'm sure there are other options and they might be more
>> performant, but this solution seems to work in several situations
>> which are usually not supported and the code is fairly easy to
>> understand ...
>>
>> Of course, for me , the best solution is to entirely remove the ASCII
>> mode from the RFC :-D
>>
>>
>>
>>
>>
>>
>>>
>>> /niklas
>>>
>>
>
Re: [jira] Commented: (FTPSERVER-306) some clients won't transform
NEW LINE characters to \r\n when sending in ASCII mode so after sending a
file the new lines will be gone.
Posted by Niklas Gustavsson <ni...@protocol7.com>.
Sorry, my bad, I mixed up \r and \n. Let me rephrase:
1. If we get sent a \n only, should we print EOL or \n? Since \n is
not the canonical line ending in the RFC, we might not want to treat
this as a line ending that we should convert into the local line
ending.
2. if we get sent a \r\n, we should clearly print EOL
3. If we get sent a \r only, we currently print EOL. Given the
argument above, is this correct?
/niklas
On Thu, May 28, 2009 at 12:13 AM, David Latorre <dv...@gmail.com> wrote:
> 2009/5/27 Niklas Gustavsson <ni...@protocol7.com>:
>> On Wed, May 27, 2009 at 3:07 PM, David Latorre <dv...@gmail.com> wrote:
>>> I added a fix and some tests but ToNetASCIIOutputStream in
>>> commons-util will always replace to \r\n so I'm uploading a modified
>>> version. With this fix ascii mode should not eat line separators
>>> anymore.
>>
>> Thanks! Looking into the fix I'm not sure we should print EOL on \r.
>> Since \r is not the canonical line ending in the RFC, I think we
>> should rather print \r as it is in the stream. What do you think?
>
> I don't really know if \r has other users than being part of the New
> Line sequence. I guess \r could take us back to the beginning of the
> line ... but it doesn't make much sense in a file.
>
> My reasons to leave \r as the character substituted by EOL are 1)
> Because it was previously written in that way. 2) Mac OS 9 and
> before had \r as the new line separator, so if the line endings of a
> Mac OS 9 text file weren't transformed to \r\n and we were ignoring
> \r , we would swallow the new line again. On the other hand, if we
> write the \r character, we cannot transform to Unix line endings
> which have no \r...
>
> So, I'm sure there are other options and they might be more
> performant, but this solution seems to work in several situations
> which are usually not supported and the code is fairly easy to
> understand ...
>
> Of course, for me , the best solution is to entirely remove the ASCII
> mode from the RFC :-D
>
>
>
>
>
>
>>
>> /niklas
>>
>
Re: [jira] Commented: (FTPSERVER-306) some clients won't transform
NEW LINE characters to \r\n when sending in ASCII mode so after sending a
file the new lines will be gone.
Posted by David Latorre <dv...@gmail.com>.
2009/5/27 Niklas Gustavsson <ni...@protocol7.com>:
> On Wed, May 27, 2009 at 3:07 PM, David Latorre <dv...@gmail.com> wrote:
>> I added a fix and some tests but ToNetASCIIOutputStream in
>> commons-util will always replace to \r\n so I'm uploading a modified
>> version. With this fix ascii mode should not eat line separators
>> anymore.
>
> Thanks! Looking into the fix I'm not sure we should print EOL on \r.
> Since \r is not the canonical line ending in the RFC, I think we
> should rather print \r as it is in the stream. What do you think?
I don't really know if \r has other users than being part of the New
Line sequence. I guess \r could take us back to the beginning of the
line ... but it doesn't make much sense in a file.
My reasons to leave \r as the character substituted by EOL are 1)
Because it was previously written in that way. 2) Mac OS 9 and
before had \r as the new line separator, so if the line endings of a
Mac OS 9 text file weren't transformed to \r\n and we were ignoring
\r , we would swallow the new line again. On the other hand, if we
write the \r character, we cannot transform to Unix line endings
which have no \r...
So, I'm sure there are other options and they might be more
performant, but this solution seems to work in several situations
which are usually not supported and the code is fairly easy to
understand ...
Of course, for me , the best solution is to entirely remove the ASCII
mode from the RFC :-D
>
> /niklas
>
Re: [jira] Commented: (FTPSERVER-306) some clients won't transform
NEW LINE characters to \r\n when sending in ASCII mode so after sending a
file the new lines will be gone.
Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Wed, May 27, 2009 at 3:07 PM, David Latorre <dv...@gmail.com> wrote:
> I added a fix and some tests but ToNetASCIIOutputStream in
> commons-util will always replace to \r\n so I'm uploading a modified
> version. With this fix ascii mode should not eat line separators
> anymore.
Thanks! Looking into the fix I'm not sure we should print EOL on \r.
Since \r is not the canonical line ending in the RFC, I think we
should rather print \r as it is in the stream. What do you think?
/niklas
Re: [jira] Commented: (FTPSERVER-306) some clients won't transform
NEW LINE characters to \r\n when sending in ASCII mode so after sending a
file the new lines will be gone.
Posted by David Latorre <dv...@gmail.com>.
I added a fix and some tests but ToNetASCIIOutputStream in
commons-util will always replace to \r\n so I'm uploading a modified
version. With this fix ascii mode should not eat line separators
anymore.
2009/5/26 Niklas Gustavsson <ni...@protocol7.com>:
> On Tue, May 26, 2009 at 12:38 PM, Sai Pullabhotla
> <sa...@jmethods.com> wrote:
>>>> when FtpServer receives a file in ASCII mode our current code replaces \r
>> for \r\n and ignores \n, but since it seems FileZilla
>>>> doesn't transform new lines to \r\n we will never find a \r and the new
>> line characters will be silently ignored.
>>
>> Hmm... shouldn't it be just doing the following:
>>
>> 1. Replace \r\n with System.getProperty("line.separator");
>> 2. Write everything else as is.
>
> Agreed. However, that's not what we currently do since if the client
> only sends \n we will strip out the line ending. This is a bug on our
> side that we should fix. You can probably agrue that FileZilla is
> doing the right thing as \r is probably not to be regarded as the
> "internal character representation" (as described in the RFC) on
> Windows.
>
> Anyways, we should add a bunch of tests for this and fix.
>
> /niklas
>
Re: [jira] Commented: (FTPSERVER-306) some clients won't transform
NEW LINE characters to \r\n when sending in ASCII mode so after sending a
file the new lines will be gone.
Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Tue, May 26, 2009 at 12:38 PM, Sai Pullabhotla
<sa...@jmethods.com> wrote:
>>> when FtpServer receives a file in ASCII mode our current code replaces \r
> for \r\n and ignores \n, but since it seems FileZilla
>>> doesn't transform new lines to \r\n we will never find a \r and the new
> line characters will be silently ignored.
>
> Hmm... shouldn't it be just doing the following:
>
> 1. Replace \r\n with System.getProperty("line.separator");
> 2. Write everything else as is.
Agreed. However, that's not what we currently do since if the client
only sends \n we will strip out the line ending. This is a bug on our
side that we should fix. You can probably agrue that FileZilla is
doing the right thing as \r is probably not to be regarded as the
"internal character representation" (as described in the RFC) on
Windows.
Anyways, we should add a bunch of tests for this and fix.
/niklas