You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ponymail.apache.org by sebb <se...@gmail.com> on 2020/08/19 10:53:30 UTC
Re: [incubator-ponymail-foal] branch master updated: add a convert_lf
option for fixing LF line endings on the fly with f=f
On Wed, 19 Aug 2020 at 09:31, <hu...@apache.org> wrote:
>
> This is an automated email from the ASF dual-hosted git repository.
>
> humbedooh pushed a commit to branch master
> in repository https://gitbox.apache.org/repos/asf/incubator-ponymail-foal.git
>
>
> The following commit(s) were added to refs/heads/master by this push:
> new d065941 add a convert_lf option for fixing LF line endings on the fly with f=f
> new 6b1fe8f Merge branch 'master' of github.com:apache/incubator-ponymail-foal
> d065941 is described below
>
> commit d065941c953bdcbdbce0e91260bc9f2d2dd41ad4
> Author: Daniel Gruno <hu...@apache.org>
> AuthorDate: Wed Aug 19 10:31:09 2020 +0200
>
> add a convert_lf option for fixing LF line endings on the fly with f=f
>
> also remove unused imports that were lingering..
> This new feature is disabled by default, as it breaks unit testing.
> Need to find a way around that.
> ---
> tools/archiver.py | 14 +++++++++-----
> 1 file changed, 9 insertions(+), 5 deletions(-)
>
> diff --git a/tools/archiver.py b/tools/archiver.py
> index 32c2597..cbe66e4 100755
> --- a/tools/archiver.py
> +++ b/tools/archiver.py
> @@ -54,8 +54,6 @@ import traceback
> import typing
> import uuid
>
> -import certifi
> -import chardet
> import formatflowed
> import netaddr
> import yaml
> @@ -193,14 +191,20 @@ class Body:
> def encode(self, charset="utf-8", errors="strict"):
> return self.string.encode(charset, errors=errors)
>
> - def unflow(self):
> + def unflow(self, convert_lf = False):
> if self.string:
> if self.flowed:
> - return formatflowed.convertToWrapped(
> - self.string.encode(self.character_set, errors="ignore"),
> + # Convert lone LF to CRLF if found
> + if convert_lf:
> + fixed_string = "\r\n".join([x.strip() for x in self.string.split("\n")])
> + else:
> + fixed_string = self.string
> + flow_fixed = formatflowed.convertToWrapped(
> + fixed_string.encode(self.character_set, errors="ignore"),
> wrap_fixed=False,
> character_set=self.character_set,
> )
I think you need to change CRLF back to LF after wrapping.
> + return flow_fixed
> return self.string
>
>
>
Re: [incubator-ponymail-foal] branch master updated: add a convert_lf
option for fixing LF line endings on the fly with f=f
Posted by sebb <se...@gmail.com>.
On Wed, 19 Aug 2020 at 11:58, Daniel Gruno <hu...@apache.org> wrote:
>
> On 19/08/2020 12.53, sebb wrote:
> > On Wed, 19 Aug 2020 at 09:31, <hu...@apache.org> wrote:
> >>
> >> This is an automated email from the ASF dual-hosted git repository.
> >>
> >> humbedooh pushed a commit to branch master
> >> in repository https://gitbox.apache.org/repos/asf/incubator-ponymail-foal.git
> >>
> >>
> >> The following commit(s) were added to refs/heads/master by this push:
> >> new d065941 add a convert_lf option for fixing LF line endings on the fly with f=f
> >> new 6b1fe8f Merge branch 'master' of github.com:apache/incubator-ponymail-foal
> >> d065941 is described below
> >>
> >> commit d065941c953bdcbdbce0e91260bc9f2d2dd41ad4
> >> Author: Daniel Gruno <hu...@apache.org>
> >> AuthorDate: Wed Aug 19 10:31:09 2020 +0200
> >>
> >> add a convert_lf option for fixing LF line endings on the fly with f=f
> >>
> >> also remove unused imports that were lingering..
> >> This new feature is disabled by default, as it breaks unit testing.
> >> Need to find a way around that.
> >> ---
> >> tools/archiver.py | 14 +++++++++-----
> >> 1 file changed, 9 insertions(+), 5 deletions(-)
> >>
> >> diff --git a/tools/archiver.py b/tools/archiver.py
> >> index 32c2597..cbe66e4 100755
> >> --- a/tools/archiver.py
> >> +++ b/tools/archiver.py
> >> @@ -54,8 +54,6 @@ import traceback
> >> import typing
> >> import uuid
> >>
> >> -import certifi
> >> -import chardet
> >> import formatflowed
> >> import netaddr
> >> import yaml
> >> @@ -193,14 +191,20 @@ class Body:
> >> def encode(self, charset="utf-8", errors="strict"):
> >> return self.string.encode(charset, errors=errors)
> >>
> >> - def unflow(self):
> >> + def unflow(self, convert_lf = False):
> >> if self.string:
> >> if self.flowed:
> >> - return formatflowed.convertToWrapped(
> >> - self.string.encode(self.character_set, errors="ignore"),
> >> + # Convert lone LF to CRLF if found
> >> + if convert_lf:
> >> + fixed_string = "\r\n".join([x.strip() for x in self.string.split("\n")])
> >> + else:
> >> + fixed_string = self.string
> >> + flow_fixed = formatflowed.convertToWrapped(
> >> + fixed_string.encode(self.character_set, errors="ignore"),
> >> wrap_fixed=False,
> >> character_set=self.character_set,
> >> )
> >
> > I think you need to change CRLF back to LF after wrapping.
>
> I was looking at that, but I did not find any compelling reason to do
> so, as this unflowed text only goes into the body of the digested
> document (for displaying in the UI).
>
> If you can think of a good reason to convert it back, I'm more than
> happy to add that into the mix. I suppose it should only convert if the
> original text had LF and nor CRLF?
Yes, good point.
If it is decided not to revert the change, the reason should be noted
in the code to help future maintainers.
> >
> >> + return flow_fixed
> >> return self.string
> >>
> >>
> >>
>
Re: [incubator-ponymail-foal] branch master updated: add a convert_lf
option for fixing LF line endings on the fly with f=f
Posted by Daniel Gruno <hu...@apache.org>.
On 19/08/2020 12.53, sebb wrote:
> On Wed, 19 Aug 2020 at 09:31, <hu...@apache.org> wrote:
>>
>> This is an automated email from the ASF dual-hosted git repository.
>>
>> humbedooh pushed a commit to branch master
>> in repository https://gitbox.apache.org/repos/asf/incubator-ponymail-foal.git
>>
>>
>> The following commit(s) were added to refs/heads/master by this push:
>> new d065941 add a convert_lf option for fixing LF line endings on the fly with f=f
>> new 6b1fe8f Merge branch 'master' of github.com:apache/incubator-ponymail-foal
>> d065941 is described below
>>
>> commit d065941c953bdcbdbce0e91260bc9f2d2dd41ad4
>> Author: Daniel Gruno <hu...@apache.org>
>> AuthorDate: Wed Aug 19 10:31:09 2020 +0200
>>
>> add a convert_lf option for fixing LF line endings on the fly with f=f
>>
>> also remove unused imports that were lingering..
>> This new feature is disabled by default, as it breaks unit testing.
>> Need to find a way around that.
>> ---
>> tools/archiver.py | 14 +++++++++-----
>> 1 file changed, 9 insertions(+), 5 deletions(-)
>>
>> diff --git a/tools/archiver.py b/tools/archiver.py
>> index 32c2597..cbe66e4 100755
>> --- a/tools/archiver.py
>> +++ b/tools/archiver.py
>> @@ -54,8 +54,6 @@ import traceback
>> import typing
>> import uuid
>>
>> -import certifi
>> -import chardet
>> import formatflowed
>> import netaddr
>> import yaml
>> @@ -193,14 +191,20 @@ class Body:
>> def encode(self, charset="utf-8", errors="strict"):
>> return self.string.encode(charset, errors=errors)
>>
>> - def unflow(self):
>> + def unflow(self, convert_lf = False):
>> if self.string:
>> if self.flowed:
>> - return formatflowed.convertToWrapped(
>> - self.string.encode(self.character_set, errors="ignore"),
>> + # Convert lone LF to CRLF if found
>> + if convert_lf:
>> + fixed_string = "\r\n".join([x.strip() for x in self.string.split("\n")])
>> + else:
>> + fixed_string = self.string
>> + flow_fixed = formatflowed.convertToWrapped(
>> + fixed_string.encode(self.character_set, errors="ignore"),
>> wrap_fixed=False,
>> character_set=self.character_set,
>> )
>
> I think you need to change CRLF back to LF after wrapping.
I was looking at that, but I did not find any compelling reason to do
so, as this unflowed text only goes into the body of the digested
document (for displaying in the UI).
If you can think of a good reason to convert it back, I'm more than
happy to add that into the mix. I suppose it should only convert if the
original text had LF and nor CRLF?
>
>> + return flow_fixed
>> return self.string
>>
>>
>>