You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/10/25 23:40:31 UTC

[GitHub] [arrow] toddfarmer opened a new issue, #14510: MIGRATION: Evaluate user mapping

toddfarmer opened a new issue, #14510:
URL: https://github.com/apache/arrow/issues/14510

   In anticipation of a potential migration from ASF Jira to GitHub for issue tracking, an assessment is needed on whether we can and should map user accounts in migrated content. GitHub and ASF Jira use different user identifiers, which often differ (or collide with unrelated users) across systems. This has implications for:
   
   * Issue reporter
   * Issue assignee
   * Watchers
   * Comment authors
   * User mentions
   
   Issue assignee could be retained if we have a mapping table to convert Jira user accounts to corresponding GitHub user accounts for individuals. This could be "best effort", and fall back to simply recording the Jira user name and display name in a comment if no mapping is supplied.
   
   Issue reporter cannot be set using the [GitHub issue import API](https://gist.github.com/jonmagic/5282384165e0f86ef105), so I plan to record the reporter displayName and username in a prefix to the issue description (see [example here](https://github.com/toddfarmer/test_import/issues/18).
   
   Watchers cannot be set via the import API. We may be able to supply a simple script for interested users to execute that would set themselves as a watcher on GitHub issues that were migrated from Jira issues they were watching.
   
   Comment authors cannot be set via the import API. I've prefixed the issue comment with the original reporter displayname and username, and a link back to the original Jira comment.
   
   User mentions can be mapped as part of content mapping. If there is no user account mapping, this could fall back to using the ASF Jira username in a code block (e.g., `[~toddfarmer]`).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] toddfarmer commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
toddfarmer commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1297777824

   Testing adding subscriptions via mentions.
   
   Separately, I noticed that we cannot migrate who closed an issue. I don't think this matters, particularly if we can map and retain assignee information for closed issues, but it's worth calling out that GitHub issues have an explicit "closed_by" field, which will be populated by the account used to perform the import operation.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] toddfarmer commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
toddfarmer commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1301145405

   > Or does the import API even avoid notifications for those later comments for users that were @ mentioned in comments created by the import API?
   
   That does seem to be the case, yes. User mentions executed added via the import API seem to not affect the issue notification subscriptions. Subsequent comments on the issue where a user was mentioned in import content do not trigger notification; an additional comment mentioning the user is required to subscribe them to notification. This could be accomplished through a two-part process:
   
   1. Import the existing issue content via the import API
   2. Add a comment referencing users who were subscribed the the source issue (author, assignee, watchers).
   
   If we coordinate with GitHub support to suppress notifications during the import process, we may be able to manipulate the notification subscribers via the added comment during a window where no notification will be generated, avoiding excess noise to users.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] jorisvandenbossche commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1300643568

   > The import process explicitly does not generate notifications, 
   
   But that's OK for the purpose of adding "watchers", I think? As a watcher of an issue, I don't need notifications for all issues that I am watching when they are migrated, but I do want notifications when someone adds a new comment _after_ the migration. 
   
   Or does the import API even avoid notifications for those later comments for users that were @ mentioned in comments created by the import API?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] jorisvandenbossche commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1291596468

   > Issue reporter cannot be set using the [GitHub issue import API](https://gist.github.com/jonmagic/5282384165e0f86ef105), so I plan to record the reporter displayName and username in a prefix to the issue description 
   
   Python actually did that when migrating their issues to GitHub last year, but I suppose they did that with the help of GitHub employees (and not with only the public import APIs). 
   https://discuss.python.org/t/github-issues-migration-mannequin-users/14965
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] pitrou commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
pitrou commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1297718647

   Ah, I didn't know that (though it makes sense). Then perhaps a post-import comment would work indeed?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] ianmcook commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
ianmcook commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1292542849

   I updated the list to include both the `key` and `name` fields from GitHub, which sometimes differ. I'm not sure which of these fields will be most convenient to use for mapping. The `key` field might be safer to use because it never contains spaces.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] ianmcook commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
ianmcook commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1379790090

   Thanks @rok! The `transfer_arrow_subscriptions.py` script worked perfectly for me.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] pitrou commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
pitrou commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1297128721

   I would suggest to append the GH handles (if available) of the issue reporter and watchers at the end of the issue description, together with the name and JIRA profile address.
   
   This is so important so that those people get GH notifications for the issues they were subscribed to on JIRA.
   
   For example:
   ```
   In anticipation of a potential migration from ASF Jira...
   
   * Reporter: @toddfarmer ([Todd Farmer](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=toddfarmer))
   * Watchers: @pitrou ([Antoine Pitrou](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=apitrou)), @ianmcook ([Ian Cook](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=icook))
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] ianmcook commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
ianmcook commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1292432915

   >Issue assignee could be retained if we have a mapping table to convert Jira user accounts to corresponding GitHub user accounts for individuals.
   
   As noted [here on the mailing list](https://lists.apache.org/thread/94fc39yhgnmv2gvj5rhogsrftqmyztqy):
   
   To help with this, I created [a couple of small scripts](https://gist.github.com/ianmcook/0f1538ebc8268a88cd4e0a0a61445287) that extract details from resolved Arrow Jira issues and the associated merged GitHub PRs, then perform some cleaning, to generate a [list](https://docs.google.com/spreadsheets/d/1C8Pxmad93TJjIeN1lX9wkw1KORKocNSLYtUUPlxHmG0/) of matching Jira user identifiers and GitHub user identifiers.
   
   This is an incomplete list: it does not include users who have created Arrow Jira issues but never created an Arrow PR that was merged.
   
   Please do some sanity-checking on this list before using it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
rok commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1376110299

   I've also prepared a [subscription transfer script](https://github.com/rok/arrow-migration/blob/main/transfer_arrow_subscriptions.py) to copy issue subscriptions from jira to github: https://github.com/rok/arrow-migration/blob/main/transfer_arrow_subscriptions.py
   
   It uses a static github user-to-issue id map that will be created at migration time (currently it points to a test import) and github's API to subscribe to issues. I hope this will satisfy needs of contributors that currently rely on Jira notifications.
   
   The script currently subscribes the user to test import issues which wouldn't be very useful. I'll update it once migration is complete for any potential users.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] toddfarmer commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
toddfarmer commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1297713101

   The import process explicitly does not generate notifications, I believe (that's part of the advertised value of using it). If you are proposing a post-import comment be added that explicitly mentions all mapped GitHub user accounts that were assigned or watching (that we are able to map) in a normal GitHub issue update API call, I suspect it would. I'll test to validate.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] toddfarmer commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
toddfarmer commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1314338076

   It turns out that it requires an extra Jira API call for every migrated issue to collect the watcher list. I have concerns about API rate limiting here. I'll test this out with a full export to see if rate limits are hit. If they are, I may add a secondary process to sync the watchers afterwards.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] jorisvandenbossche commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1310523674

   > This could be accomplished through a two-part process:
   
   That sounds good to me


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] pitrou commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
pitrou commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1297708259

   Wouldn't my proposal above work?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
rok commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1379390833

   I've updated the user mapping so the script is now usable.
   If you were a reporter/creator/assignee/watcher of an issue [the script](https://github.com/rok/arrow-migration/blob/main/transfer_arrow_subscriptions.py) will subscribe you to the issues's copy on github.
   You need to provide your github username and a token with notifications permission.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok closed issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
rok closed issue #14510: MIGRATION: Evaluate user mapping
URL: https://github.com/apache/arrow/issues/14510


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
rok commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1374892917

   I've tried contacting github regarding mannequin users via [public discussions](https://github.com/orgs/community/discussions/42402), support.github.com and github.com/services, but they were not very responsive. ASF infra didn't have any suggestions so I propose we simply use @asfimport to import our issues (example of [lucene](https://github.com/apache/lucene/issues/2835)).
   
   * assignees - they can be set at import time and don't trigger notifications. I expect assignees are subscribed to the issue. I think we should assign at import time.
   
   * watchers - I was able to extract them from jira (see [example issue](https://github.com/datatart/import_dry_run_4/issues/18257)). However I'm not sure mass subscribing everyone to issues where they were watchers is a good idea as we'd have to do it with @ mentions which would generate a lot of spam. We could solve this by making Jira watchers GitHub assignees (up to 10 are possible), but I'd prefer the subscription script option Todd is proposing.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] toddfarmer commented on issue #14510: MIGRATION: Evaluate user mapping

Posted by GitBox <gi...@apache.org>.
toddfarmer commented on issue #14510:
URL: https://github.com/apache/arrow/issues/14510#issuecomment-1297705902

   My current belief is that we cannot effectively migrate issue watchers/subscriptions from Jira to GitHub issues, and that it may be necessary to create a simple Python script for individuals to execute post-migration that will:
   
   1. Query Jira to find existing issues they are subscribed to
   2. Obtain the migrated GitHub issue identifier from a designated field in the Jira issue
   3. Subscribe them to the migrated GitHub issue
   
   The above seems something we can reasonable automate in a simple script, along with documentation on how to set up necessary auth to successfully execute it. But it seems we can't directly import the watchers, even if we can map user identifiers. Unless I'm missing it, the only possible viable alternative that retains issue subscriptions would be to [perform an OAuth impersonation dance](https://developer.github.com/enterprise/2.3/v3/users/administration/#create-an-impersonation-oauth-token) for every needed user/subscription - and as was noted in the links @jorisvandenbossche cited above, that seems to only be available for GitHub Enterprise customers (seems unlikely that applies to us).
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org