Secretary workbench - testing message parsing

[sebb <se...@gmail.com>]

I think it would be useful for the Secretary workbench to be able to
reparse a message if necessary.

For example, there is a message (201810/6d84281261/) which has two
attachments, but one of them is not shown in the YAML summary. When I
tried parsing the message on my system, it correctly detected them
both. Either one of the attachments was dropped from the YAML file, or
it was never added in the first place. It would be useful to know
which is the case.

I think it would just be a question of re-running the deliver.rb
script with the appropriate file.

WDYT?

S.

Re: Secretary workbench - testing message parsing

[sebb <se...@gmail.com>]

On Fri, 19 Oct 2018 at 12:16, Shane Curcuru <as...@shanecurcuru.org> wrote:
>
> sebb wrote on 10/19/18 5:51 AM:
> > I think it would be useful for the Secretary workbench to be able to
> > reparse a message if necessary.
> >
> > For example, there is a message (201810/6d84281261/) which has two
> > attachments, but one of them is not shown in the YAML summary. When I
> > tried parsing the message on my system, it correctly detected them
> > both. Either one of the attachments was dropped from the YAML file, or
> > it was never added in the first place. It would be useful to know
> > which is the case.
> >
> > I think it would just be a question of re-running the deliver.rb
> > script with the appropriate file.
> >
> > WDYT?
>
> Yes, *as long as* it's bulletproof and crystal-clear to a user when this
> is happening, to prevent unintentionally either adding another record or
> overwriting an existing record that you didn't mean to.  Not sure if
> that's debug / only show what would happen / don't actually write data
> flags or what.
>
> Maybe my concern isn't important, but I want to make sure that in the
> future a new Secretary helper couldn't mistakenly use this to re-process
> all of last month's emails or something, which could make quite a mess
> if the script mistakenly re-updated a bunch of data.

I was thinking of a script to reprocess a single email by providing
its month and hash.
The user would have to know these.

Ideally the script would run the existing deliver script; however it
does not return the hash or the new YAML entry so it would not be easy
to detect what changed.

One way round this might be to replicate the code in a new script
which does not automatically perform the update unless certain
conditions are met, e.g.. no change to hash or month and no change to
the mail source copy - i.e. only update the YAML hash.
However this means keeping two sources in step, so perhaps the deliver
script could take a parameter which makes it perform the checks.> --
>
> - Shane
>   Whimsy PMC
>   The Apache Software Foundation

Re: Secretary workbench - testing message parsing

[Shane Curcuru <as...@shanecurcuru.org>]

sebb wrote on 10/19/18 5:51 AM:
> I think it would be useful for the Secretary workbench to be able to
> reparse a message if necessary.
> 
> For example, there is a message (201810/6d84281261/) which has two
> attachments, but one of them is not shown in the YAML summary. When I
> tried parsing the message on my system, it correctly detected them
> both. Either one of the attachments was dropped from the YAML file, or
> it was never added in the first place. It would be useful to know
> which is the case.
> 
> I think it would just be a question of re-running the deliver.rb
> script with the appropriate file.
> 
> WDYT?

Yes, *as long as* it's bulletproof and crystal-clear to a user when this
is happening, to prevent unintentionally either adding another record or
overwriting an existing record that you didn't mean to.  Not sure if
that's debug / only show what would happen / don't actually write data
flags or what.

Maybe my concern isn't important, but I want to make sure that in the
future a new Secretary helper couldn't mistakenly use this to re-process
all of last month's emails or something, which could make quite a mess
if the script mistakenly re-updated a bunch of data.

-- 

- Shane
  Whimsy PMC
  The Apache Software Foundation

Re: Secretary workbench - testing message parsing

[sebb <se...@gmail.com>]

On Sat, 20 Oct 2018 at 17:30, Sam Ruby <ru...@intertwingly.net> wrote:
>
> On Fri, Oct 19, 2018 at 5:51 AM sebb <se...@gmail.com> wrote:
> >
> > I think it would be useful for the Secretary workbench to be able to
> > reparse a message if necessary.
> >
> > For example, there is a message (201810/6d84281261/) which has two
> > attachments, but one of them is not shown in the YAML summary. When I
> > tried parsing the message on my system, it correctly detected them
> > both. Either one of the attachments was dropped from the YAML file, or
> > it was never added in the first place. It would be useful to know
> > which is the case.
> >
> > I think it would just be a question of re-running the deliver.rb
> > script with the appropriate file.
> >
> > WDYT?
>
> Rerunning the script does restore the entry in the YML file:
>
> ruby whimsy/www/secretary/workbench/deliver.rb <
> /srv/mail/secretary/201810.mail/6d8428126

Thanks!

However it's not there now.

Somehow it's getting dropped.

> - Sam Ruby

Re: Secretary workbench - testing message parsing

[Sam Ruby <ru...@intertwingly.net>]

On Fri, Oct 19, 2018 at 5:51 AM sebb <se...@gmail.com> wrote:
>
> I think it would be useful for the Secretary workbench to be able to
> reparse a message if necessary.
>
> For example, there is a message (201810/6d84281261/) which has two
> attachments, but one of them is not shown in the YAML summary. When I
> tried parsing the message on my system, it correctly detected them
> both. Either one of the attachments was dropped from the YAML file, or
> it was never added in the first place. It would be useful to know
> which is the case.
>
> I think it would just be a question of re-running the deliver.rb
> script with the appropriate file.
>
> WDYT?

Rerunning the script does restore the entry in the YML file:

ruby whimsy/www/secretary/workbench/deliver.rb <
/srv/mail/secretary/201810.mail/6d8428126

- Sam Ruby