You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ponymail.apache.org by hu...@apache.org on 2020/09/10 12:41:50 UTC
[incubator-ponymail-foal] branch master updated (8d5d1a4 -> 19cf33d)
This is an automated email from the ASF dual-hosted git repository.
humbedooh pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-ponymail-foal.git.
from 8d5d1a4 Lighten up the mood a bit
new 1a2c33d add a notes field. This can be a single text or an array of texts
new 19cf33d Add some notes to email docs when migrating
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
tools/mappings.yaml | 2 ++
tools/migrate.py | 13 +++++++++++++
2 files changed, 15 insertions(+)
[incubator-ponymail-foal] 01/02: add a notes field. This can be a
single text or an array of texts
Posted by hu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
humbedooh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-ponymail-foal.git
commit 1a2c33d8900ecce16e2e8459333edaaa91b4fc24
Author: Daniel Gruno <hu...@apache.org>
AuthorDate: Thu Sep 10 14:36:47 2020 +0200
add a notes field. This can be a single text or an array of texts
---
tools/mappings.yaml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tools/mappings.yaml b/tools/mappings.yaml
index 8828396..8c881b7 100644
--- a/tools/mappings.yaml
+++ b/tools/mappings.yaml
@@ -87,6 +87,8 @@ mbox:
type: text
to:
type: text
+ notes:
+ type: text
notification:
properties:
date:
[incubator-ponymail-foal] 02/02: Add some notes to email docs when
migrating
Posted by hu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
humbedooh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-ponymail-foal.git
commit 19cf33d0773eeb163bbd9d2e278a4f997e4fda83
Author: Daniel Gruno <hu...@apache.org>
AuthorDate: Thu Sep 10 14:41:21 2020 +0200
Add some notes to email docs when migrating
---
tools/migrate.py | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/tools/migrate.py b/tools/migrate.py
index a1d14b8..a6620ee 100644
--- a/tools/migrate.py
+++ b/tools/migrate.py
@@ -57,6 +57,7 @@ async def main():
# Let's get started..!
start_time = time.time()
+ now = start_time
processed = 0
count = await es.count(index=old_dbname, doc_type="mbox")
no_emails = count['count']
@@ -104,6 +105,18 @@ async def main():
source['_source']['permalinks'] = doc['_source']['permalinks']
doc['_source']['dbid'] = hashlib.sha3_256(source_text).hexdigest()
+ # Append migration details to notes field in doc
+ doc['_source']['notes'] = doc['_source'].get('notes', [])
+ # We want a list, not a single string
+ if isinstance(doc['_source']['notes'], str):
+ doc['_source']['notes'] = [doc['_source']['notes']]
+ doc['_source']['notes'].append("MIGRATE: Document migrated from Pony Mail to Pony Mail Foal at %u, "
+ "using foal migrator v/%s" % (now, MIGRATION_MAGIC_NUMBER))
+ # If we re-indexed the document, make a note of that as well.
+ if do_dkim:
+ doc['_source']['notes'].append("REINDEX: Document re-indexed with DKIM_ID at %u, "
+ "from %s to %s" % (now, dkim_id, old_id))
+
# Copy to new DB
bulk_array.append({
'index': dbname_mbox,