You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ponymail.apache.org by GitBox <gi...@apache.org> on 2021/09/18 22:40:28 UTC
[GitHub] [incubator-ponymail-foal] Humbedooh opened a new pull request #60: Add in monthly activity in the search results
Humbedooh opened a new pull request #60:
URL: https://github.com/apache/incubator-ponymail-foal/pull/60
This touches on #57 by introducing an active_months data entry in the search results.
As get_activity_span (and get_years before that) has a built-in check for private list access,
this should be safe to relay directly to the user.
Once merged in, we can address the JS part of the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ponymail.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ponymail-foal] sebbASF commented on pull request #60: Add in monthly activity in the search results
Posted by GitBox <gi...@apache.org>.
sebbASF commented on pull request #60:
URL: https://github.com/apache/incubator-ponymail-foal/pull/60#issuecomment-922501674
> As for whether to make it optional, I think this needs testing on a large system to see what sort of extra cost it incurs.
What extra cost are you talking about?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ponymail.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ponymail-foal] sebbASF removed a comment on pull request #60: Add in monthly activity in the search results
Posted by GitBox <gi...@apache.org>.
sebbASF removed a comment on pull request #60:
URL: https://github.com/apache/incubator-ponymail-foal/pull/60#issuecomment-922501674
> As for whether to make it optional, I think this needs testing on a large system to see what sort of extra cost it incurs.
What extra cost are you talking about?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ponymail.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ponymail-foal] Humbedooh commented on pull request #60: Add in monthly activity in the search results
Posted by GitBox <gi...@apache.org>.
Humbedooh commented on pull request #60:
URL: https://github.com/apache/incubator-ponymail-foal/pull/60#issuecomment-922500515
> Are there any cases where the data is not needed - e.g. on phones?
Possibly, but that is tricky to detect, and since we're only talking maybe a kilobyte of extra data, I think it's okay to leave in.
As for whether to make it optional, I think this needs testing on a large system to see what sort of extra cost it incurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ponymail.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ponymail-foal] Humbedooh commented on a change in pull request #60: Add in monthly activity in the search results
Posted by GitBox <gi...@apache.org>.
Humbedooh commented on a change in pull request #60:
URL: https://github.com/apache/incubator-ponymail-foal/pull/60#discussion_r711776068
##########
File path: server/plugins/messages.py
##########
@@ -445,26 +445,36 @@ async def get_years(session, query_defuzzed):
{"bool": {"should": [{"term": {"private": False}}, {"terms": {"list_raw": private_lists_accessible}}]}}
]
- # Get oldest and youngest doc in single scan
+ # Get oldest and youngest doc in single scan, as well as a monthly histogram
res = await session.database.search(
index=session.database.dbs.mbox,
size=0,
body={"query": {"bool": query_defuzzed},
"aggs": {
"first": {"min": {"field": "epoch"}},
"last": {"max": {"field": "epoch"}},
+ "active_months": {
+ "date_histogram": {
+ "field": "date",
+ "calendar_interval": "month",
+ "format": "yyyy-MM"
+ }
+ }
},
}
)
oldest = datetime.datetime.fromtimestamp(0)
youngest = datetime.datetime.fromtimestamp(0)
+ monthly_activity = {}
if res["aggregations"]:
aggs = res["aggregations"]
oldest = datetime.datetime.fromtimestamp(aggs["first"]["value"] or 0)
youngest = datetime.datetime.fromtimestamp(aggs["last"]["value"] or 0)
+ for bucket in aggs["active_months"].get("buckets", []):
+ monthly_activity[bucket["key_as_string"]] = bucket["doc_count"]
Review comment:
Good idea, done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ponymail.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ponymail-foal] sebbASF commented on a change in pull request #60: Add in monthly activity in the search results
Posted by GitBox <gi...@apache.org>.
sebbASF commented on a change in pull request #60:
URL: https://github.com/apache/incubator-ponymail-foal/pull/60#discussion_r711657033
##########
File path: server/plugins/messages.py
##########
@@ -445,26 +445,36 @@ async def get_years(session, query_defuzzed):
{"bool": {"should": [{"term": {"private": False}}, {"terms": {"list_raw": private_lists_accessible}}]}}
]
- # Get oldest and youngest doc in single scan
+ # Get oldest and youngest doc in single scan, as well as a monthly histogram
res = await session.database.search(
index=session.database.dbs.mbox,
size=0,
body={"query": {"bool": query_defuzzed},
"aggs": {
"first": {"min": {"field": "epoch"}},
"last": {"max": {"field": "epoch"}},
+ "active_months": {
+ "date_histogram": {
+ "field": "date",
+ "calendar_interval": "month",
+ "format": "yyyy-MM"
+ }
+ }
},
}
)
oldest = datetime.datetime.fromtimestamp(0)
youngest = datetime.datetime.fromtimestamp(0)
+ monthly_activity = {}
if res["aggregations"]:
aggs = res["aggregations"]
oldest = datetime.datetime.fromtimestamp(aggs["first"]["value"] or 0)
youngest = datetime.datetime.fromtimestamp(aggs["last"]["value"] or 0)
+ for bucket in aggs["active_months"].get("buckets", []):
+ monthly_activity[bucket["key_as_string"]] = bucket["doc_count"]
Review comment:
Maybe skip buckets with 0 count?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ponymail.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ponymail-foal] sebbASF commented on a change in pull request #60: Add in monthly activity in the search results
Posted by GitBox <gi...@apache.org>.
sebbASF commented on a change in pull request #60:
URL: https://github.com/apache/incubator-ponymail-foal/pull/60#discussion_r711657033
##########
File path: server/plugins/messages.py
##########
@@ -445,26 +445,36 @@ async def get_years(session, query_defuzzed):
{"bool": {"should": [{"term": {"private": False}}, {"terms": {"list_raw": private_lists_accessible}}]}}
]
- # Get oldest and youngest doc in single scan
+ # Get oldest and youngest doc in single scan, as well as a monthly histogram
res = await session.database.search(
index=session.database.dbs.mbox,
size=0,
body={"query": {"bool": query_defuzzed},
"aggs": {
"first": {"min": {"field": "epoch"}},
"last": {"max": {"field": "epoch"}},
+ "active_months": {
+ "date_histogram": {
+ "field": "date",
+ "calendar_interval": "month",
+ "format": "yyyy-MM"
+ }
+ }
},
}
)
oldest = datetime.datetime.fromtimestamp(0)
youngest = datetime.datetime.fromtimestamp(0)
+ monthly_activity = {}
if res["aggregations"]:
aggs = res["aggregations"]
oldest = datetime.datetime.fromtimestamp(aggs["first"]["value"] or 0)
youngest = datetime.datetime.fromtimestamp(aggs["last"]["value"] or 0)
+ for bucket in aggs["active_months"].get("buckets", []):
+ monthly_activity[bucket["key_as_string"]] = bucket["doc_count"]
Review comment:
Maybe skip buckets with 0 count?
This would mean some changes to the JS, but would reduce the response size
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ponymail.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ponymail-foal] Humbedooh commented on pull request #60: Add in monthly activity in the search results
Posted by GitBox <gi...@apache.org>.
Humbedooh commented on pull request #60:
URL: https://github.com/apache/incubator-ponymail-foal/pull/60#issuecomment-922502610
I was referring to the extra cost of doing a monthly histogram. Seems we got some wires crossed :)
For mobile, I don't know that we can reliably detect this at the moment, so I think it's best to just include in all responses. There will be cases where a calendar is hidden during a search but then later shown, if the width of the window changes for instance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ponymail.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ponymail-foal] Humbedooh edited a comment on pull request #60: Add in monthly activity in the search results
Posted by GitBox <gi...@apache.org>.
Humbedooh edited a comment on pull request #60:
URL: https://github.com/apache/incubator-ponymail-foal/pull/60#issuecomment-922500515
> Are there any cases where the data is not needed - e.g. on phones?
Possibly, but that is tricky to detect, and since we're only talking maybe a kilobyte of extra data, I think it's okay to leave in.
As for whether to make it optional, I think this needs testing on a large system to see what sort of extra cost it incurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ponymail.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ponymail-foal] sebbASF commented on pull request #60: Add in monthly activity in the search results
Posted by GitBox <gi...@apache.org>.
sebbASF commented on pull request #60:
URL: https://github.com/apache/incubator-ponymail-foal/pull/60#issuecomment-922452431
Maybe the monthly activity analysis should be optional?
This would be easy to do.
Are there any cases where the data is not needed - e.g. on phones?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ponymail.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ponymail-foal] sebbASF commented on pull request #60: Add in monthly activity in the search results
Posted by GitBox <gi...@apache.org>.
sebbASF commented on pull request #60:
URL: https://github.com/apache/incubator-ponymail-foal/pull/60#issuecomment-922502242
Note that the extra cost I was referring to was the network cost for mobile devices, not the cost of generating the data.
We should not be sending them data they do not need.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ponymail.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ponymail-foal] sebbASF merged pull request #60: Add in monthly activity in the search results
Posted by GitBox <gi...@apache.org>.
sebbASF merged pull request #60:
URL: https://github.com/apache/incubator-ponymail-foal/pull/60
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ponymail.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ponymail-foal] Humbedooh commented on pull request #60: Add in monthly activity in the search results
Posted by GitBox <gi...@apache.org>.
Humbedooh commented on pull request #60:
URL: https://github.com/apache/incubator-ponymail-foal/pull/60#issuecomment-922501293
It appears to not be too heavy on the machinery, so I'm inclined to leave it as a standard behavior, I don't think we need to make it optional. What we will later need to do is adjust the calendar click behavior, so that is shows the month clicked but within the search parameters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ponymail.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org