You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@diversity.apache.org by Sage Sharp <sh...@otter.technology> on 2019/07/05 21:49:15 UTC
Engaging potential ASF Outreachy mentors
[Moving this to a separate thread, and again, please do not discuss funding
sources or board notifications on this thread.]
Joan had specific questions as to how to engage Outreachy mentors. I would
suggest sending them links to Outreachy's documentation of mentor duties
and the mentor FAQ:
https://www.outreachy.org/mentor/#mentor
https://www.outreachy.org/mentor/mentor-faq/
Outreachy mentors need to submit their project description through the
Outreachy website. I've attached the project information we ask for in a
text file. The community sign up doesn't start until August 1, so there's
no way for mentors to sign up for the next round until then. If the ASF D&I
committee wants to review mentor's project before August, they can use the
format to start gathering proposals.
More information on Joan's specific questions below:
On Thu, Jun 27, 2019 at 11:59 AM Joan Touzet <wo...@apache.org> wrote:
> On 2019-06-27 13:22, Naomi S wrote:
> > On Thu, 27 Jun 2019 at 18:52, Ross Gardler
> > <Ro...@microsoft.com.invalid> wrote:
> >
> >> I have seen NO discussion on the topic of who the mentors are and
> minimal
> >> discussion on how we evaluate proposals.
> >>
> >
> > I will note that this very thread ("[DISCUSS] Outreachy framework
> > proposal") was supposed to be the start of that conversation.
>
> And I would like to continue that topic, but until the noise abates and
> the S/N is restored here, I refuse to do so. I simply haven't the energy
> to wade through the piles of vitriol.
>
> I'll leave a few points here, and hopefully someone can pick up on them:
>
> * We need a double blind evaluation system that masks project name,
> applicant name, specific details, etc.
>
(FYI - using the term "anonymized evaluation system" is a better term
"double blind evaluation system". I've seen vision-impaired people complain
about how this term is an unpleasant one for them.)
I assume you mean an anonymized evaluation system for deciding which
Outreachy internship projects to accept? The Outreachy website asks for a
lot of project-specific details (like communication channel links,
documentation for how to make a first contribution, etc.). I don't think it
will be possible to make an informed decision about whether a project is
going to be successful in Outreachy without looking at those details. You
would need someone to do a lot of redaction on the project descriptions to
achieve anonymity.
Or are you talking about an anonymous review system to help coordinators
needing to make a decision about which intern selections? That would be
needed if there are more interns than you have funding for.
Coordinators will already have access to information about applicants and
intern selects through the Outreachy website, so implementing an anonymized
intern selection system might not be feasible for that. The information
that coordinators and mentors can see on the Outreachy website has some
details about the applicants. Each project will have a page that lists
applicants who have recorded a contribution or filled out a final
application. That page lists the applicant's name, email address, location
and timezone, pronouns, and a summary of how many days free the applicant
has during the internship. The final application lists answers to questions
like "What is your experience in other free software communities?" and
"What other experience do you have that is relevant to this project?"
Mentors have tried to implement anonymous reviews of Outreachy applicants'
contributions. I know some projects in Mozilla have turned on a GitHub
feature to hide the name of the person submitting a contribution. The
coordinator gathers final applications and recorded contributions, redacts
personal information, and shares it with the mentors. Then the mentors make
a decision on which applicant to accept, all without knowing details about
the applicant.
If the ASF wants to anonymize intern selections for review, it would need a
neutral third party to redact personal information and identifying
information from the intern's final application and the project.
> * We need a way to ensure that mentors actually are capable of
> committing the time necessary for this programme, to check in with
> the admins on regular intervals to ensure this is happening, and to
> blacklist them from future involvement if they are unable to meet
> those expectations and they do not have an acceptable excuse, since
> real money will have been wasted. (Doctor's note, death in the
> family, etc.)
>
Outreachy requires mentors to check a box during project sign up to specify
that they understand this is a 5 hour time commitment during the
internship. There's another box for acknowledging the 10 hour time
commitment during the contribution period. The Outreachy organizers don't
do anything as elaborate as what you're proposing, but there's nothing
stopping coordinators from doing so.
Outreachy already has feedback points three times during the internship (2
weeks in, 8 weeks in, and 13 weeks in). Mentors give feedback on their
interns, including their response time, whether the intern should be paid
their stipend for that term, and whether the internship should be extended.
Interns give private feedback on their mentors, including the mentor's
response time to their questions and how long it takes the mentors to give
feedback on their contributions.
The mentor and intern feedback is private, only read by Outreachy
organizers. That's because interns sometimes talk about how their personal
life has been impacting the amount of time they could put into their
internship. These are often very personal matters, e.g. "I'm going through
a divorce" or "I'm moving to a new country" or "I have this specific health
issue". That's not something we want to automatically share with
coordinators.
However, if you're concerned about mentor response times, I can let you
know if I see concerning trends in mentor response time or intern comments
on the ASF mentors.
> * We need a double-check that the projects involved agree to follow
> not just the ASF CoC, but the Outreachy CoC, and any other imposed
> requirements
> * We need a ruler by which we can measure the quality of the project
> in terms of its suitability for an intern - what will the intern
> *gain* by working on the project? Or is it just self-interest of
> the requesting PMC/org? Quantified and qualified, preferably.
>
One of the questions we ask mentors during project sign up is "What benefit
will the intern have for working on this project?" I've also talked about
other project criteria you might want to use here:
https://lists.apache.org/thread.html/f790fa5bffd7c2aac3fb495593456d55bed940815891511c6d3fd039@%3Cdev.diversity.apache.org%3E
> * We need to review the tagged areas Outreachy provides to ensure
> that the applications we vet cover a broad range of opportunities
> within our Foundation, not 100% documentation or internal-facing
> tools.
> * We should find a volunteer to reach out to our compatriots within
> Fedora and Debian to see how they coordinate and vet their
> similar opportunity programs.
>
I also would recommend talking with the Mozilla coordinators, as they have
more projects that they vet (10-15 per round). I can connect you with
Dustin (the main Mozilla coordinator). I would need a specific set of
people to connect him with, rather than dragging him to a public mailing
list thread that is at risk of meandering off topic.
Sage Sharp
Outreachy Organizer
Re: Engaging potential ASF Outreachy mentors
Posted by Matt Sicker <bo...@gmail.com>.
For the projects I’ve mentored so far, I’ve typically had a few different
people express interest in the project, but by the time they submit
applications and initial contributions, the list narrows down a bit. Unless
you somehow manage to attract numerous well qualified individuals all at
the same time and they don’t get discouraged by the competition, as a
mentor, you’ll likely know which person or people (if possible) you’d like
to accept based on your interactions with them. There’s already an
application process to even be considered as an Outreachy intern applicant
before you contribute to any project, and I believe that’s where a lot of
the more personal information about each individual is considered.
On Tue, Jul 9, 2019 at 11:25, Joan Touzet <wo...@apache.org> wrote:
> Hi Sage,
>
> Thank you so much for this reply.
>
> On 2019-07-05 17:49, Sage Sharp wrote:
> > [Moving this to a separate thread, and again, please do not discuss
> > funding sources or board notifications on this thread.]
> >
> > Joan had specific questions as to how to engage Outreachy mentors. I
> > would suggest sending them links to Outreachy's documentation of mentor
> > duties and the mentor FAQ:
> >
> > https://www.outreachy.org/mentor/#mentor
> > https://www.outreachy.org/mentor/mentor-faq/
> >
> > Outreachy mentors need to submit their project description through the
> > Outreachy website. I've attached the project information we ask for in a
> > text file. The community sign up doesn't start until August 1, so
> > there's no way for mentors to sign up for the next round until then. If
> > the ASF D&I committee wants to review mentor's project before August,
> > they can use the format to start gathering proposals.
> >
> > More information on Joan's specific questions below:
> >
> > On Thu, Jun 27, 2019 at 11:59 AM Joan Touzet <wohali@apache.org
> > <ma...@apache.org>> wrote:
> >
> > On 2019-06-27 13:22, Naomi S wrote:
> > > On Thu, 27 Jun 2019 at 18:52, Ross Gardler
> > > <Ross.Gardler@microsoft.com
> > <ma...@microsoft.com>.invalid> wrote:
> > >
> > >> I have seen NO discussion on the topic of who the mentors are
> > and minimal
> > >> discussion on how we evaluate proposals.
> > >>
> > >
> > > I will note that this very thread ("[DISCUSS] Outreachy framework
> > > proposal") was supposed to be the start of that conversation.
> >
> > And I would like to continue that topic, but until the noise abates
> and
> > the S/N is restored here, I refuse to do so. I simply haven't the
> energy
> > to wade through the piles of vitriol.
> >
> > I'll leave a few points here, and hopefully someone can pick up on
> them:
> >
> > * We need a double blind evaluation system that masks project name,
> > applicant name, specific details, etc.
> >
> >
> > (FYI - using the term "anonymized evaluation system" is a better term
> > "double blind evaluation system". I've seen vision-impaired people
> > complain about how this term is an unpleasant one for them.)
>
> Thank you so much! I should have realized this myself. I'll start using
> "double-anonymized" or "anonymized" instead.
>
> > I assume you mean an anonymized evaluation system for deciding which
> > Outreachy internship projects to accept? The Outreachy website asks for
> > a lot of project-specific details (like communication channel links,
> > documentation for how to make a first contribution, etc.). I don't think
> > it will be possible to make an informed decision about whether a project
> > is going to be successful in Outreachy without looking at those details.
> > You would need someone to do a lot of redaction on the project
> > descriptions to achieve anonymity.
>
> No, you're right, this wouldn't be easy and doesn't make a lot of sense.
>
> > Or are you talking about an anonymous review system to help coordinators
> > needing to make a decision about which intern selections? That would be
> > needed if there are more interns than you have funding for.
>
> Yes, this is what I was driving at.
>
> > Coordinators will already have access to information about applicants
> > and intern selects through the Outreachy website, so implementing an
> > anonymized intern selection system might not be feasible for that. The
> > information that coordinators and mentors can see on the Outreachy
> > website has some details about the applicants. Each project will have a
> > page that lists applicants who have recorded a contribution or filled
> > out a final application. That page lists the applicant's name, email
> > address, location and timezone, pronouns, and a summary of how many days
> > free the applicant has during the internship. The final application
> > lists answers to questions like "What is your experience in other free
> > software communities?" and "What other experience do you have that is
> > relevant to this project?"
> >
> > Mentors have tried to implement anonymous reviews of Outreachy
> > applicants' contributions. I know some projects in Mozilla have turned
> > on a GitHub feature to hide the name of the person submitting a
> > contribution. The coordinator gathers final applications and recorded
> > contributions, redacts personal information, and shares it with the
> > mentors. Then the mentors make a decision on which applicant to accept,
> > all without knowing details about the applicant.
> >
> > If the ASF wants to anonymize intern selections for review, it would
> > need a neutral third party to redact personal information and
> > identifying information from the intern's final application and the
> project.
>
> I am concerned about both conscious and non-conscious bias entering into
> the selection process, but perhaps I am worried over nothing. How
> frequently do projects receive more than one intern application?
>
> > * We need a way to ensure that mentors actually are capable of
> > committing the time necessary for this programme, to check in with
> > the admins on regular intervals to ensure this is happening, and to
> > blacklist them from future involvement if they are unable to meet
> > those expectations and they do not have an acceptable excuse, since
> > real money will have been wasted. (Doctor's note, death in the
> > family, etc.)
> >
> >
> > Outreachy requires mentors to check a box during project sign up to
> > specify that they understand this is a 5 hour time commitment during the
> > internship. There's another box for acknowledging the 10 hour time
> > commitment during the contribution period. The Outreachy organizers
> > don't do anything as elaborate as what you're proposing, but there's
> > nothing stopping coordinators from doing so.
>
> Yeah, this was really an internal point for us, and the coordinator can
> help ensure this point is made. I don't want the ASF to have a black eye
> because people signed up to mentor, then didn't follow through. Sure,
> things happen unexpectedly, but we should at least do a double-check
> that the mentorship applicant knows what they're signing up for, vs.
> thinking this is a "land grab for warm bodies" (which it really is not.)
>
> > Outreachy already has feedback points three times during the internship
> > (2 weeks in, 8 weeks in, and 13 weeks in). Mentors give feedback on
> > their interns, including their response time, whether the intern should
> > be paid their stipend for that term, and whether the internship should
> > be extended. Interns give private feedback on their mentors, including
> > the mentor's response time to their questions and how long it takes the
> > mentors to give feedback on their contributions.
> >
> > The mentor and intern feedback is private, only read by Outreachy
> > organizers. That's because interns sometimes talk about how their
> > personal life has been impacting the amount of time they could put into
> > their internship. These are often very personal matters, e.g. "I'm going
> > through a divorce" or "I'm moving to a new country" or "I have this
> > specific health issue". That's not something we want to automatically
> > share with coordinators.
> >
> > However, if you're concerned about mentor response times, I can let you
> > know if I see concerning trends in mentor response time or intern
> > comments on the ASF mentors.
>
> This would be very valuable feedback to the Coordinator. Yes, please! My
> main goal is to ensure the interns get a good experience out of this,
> and as a volunteer-run-and-led organisation, unexpected absences are
> more common than most of us would like.
>
> > * We need a double-check that the projects involved agree to follow
> > not just the ASF CoC, but the Outreachy CoC, and any other imposed
> > requirements
>
> Glad to hear this isn't an issue for you :)
>
> > * We need a ruler by which we can measure the quality of the project
> > in terms of its suitability for an intern - what will the intern
> > *gain* by working on the project? Or is it just self-interest of
> > the requesting PMC/org? Quantified and qualified, preferably.
> >
> >
> > One of the questions we ask mentors during project sign up is "What
> > benefit will the intern have for working on this project?" I've also
> > talked about other project criteria you might want to use here:
> >
> >
> https://lists.apache.org/thread.html/f790fa5bffd7c2aac3fb495593456d55bed940815891511c6d3fd039@%3Cdev.diversity.apache.org%3E
>
> This is all really good, and I hope the selected Coordinator on our side
> picks up on all of this.
>
> > * We need to review the tagged areas Outreachy provides to ensure
> > that the applications we vet cover a broad range of opportunities
> > within our Foundation, not 100% documentation or internal-facing
> > tools.
>
> This is really internal direction for our Coordinator, and I think it
> will be simple enough to handle.
>
> > * We should find a volunteer to reach out to our compatriots within
> > Fedora and Debian to see how they coordinate and vet their
> > similar opportunity programs.
> >
> >
> > I also would recommend talking with the Mozilla coordinators, as they
> > have more projects that they vet (10-15 per round). I can connect you
> > with Dustin (the main Mozilla coordinator). I would need a specific set
> > of people to connect him with, rather than dragging him to a public
> > mailing list thread that is at risk of meandering off topic.
>
> Fantastic! This, plus the other Mozilla name you mentioned on the call
> with Gris et. al., should get us very far.
>
> Thanks again for all your support!
>
> -Joan "now we're cookin'" Touzet
>
> --
Matt Sicker <bo...@gmail.com>
Re: Engaging potential ASF Outreachy mentors
Posted by Joan Touzet <wo...@apache.org>.
Hi Sage,
Thank you so much for this reply.
On 2019-07-05 17:49, Sage Sharp wrote:
> [Moving this to a separate thread, and again, please do not discuss
> funding sources or board notifications on this thread.]
>
> Joan had specific questions as to how to engage Outreachy mentors. I
> would suggest sending them links to Outreachy's documentation of mentor
> duties and the mentor FAQ:
>
> https://www.outreachy.org/mentor/#mentor
> https://www.outreachy.org/mentor/mentor-faq/
>
> Outreachy mentors need to submit their project description through the
> Outreachy website. I've attached the project information we ask for in a
> text file. The community sign up doesn't start until August 1, so
> there's no way for mentors to sign up for the next round until then. If
> the ASF D&I committee wants to review mentor's project before August,
> they can use the format to start gathering proposals.
>
> More information on Joan's specific questions below:
>
> On Thu, Jun 27, 2019 at 11:59 AM Joan Touzet <wohali@apache.org
> <ma...@apache.org>> wrote:
>
> On 2019-06-27 13:22, Naomi S wrote:
> > On Thu, 27 Jun 2019 at 18:52, Ross Gardler
> > <Ross.Gardler@microsoft.com
> <ma...@microsoft.com>.invalid> wrote:
> >
> >> I have seen NO discussion on the topic of who the mentors are
> and minimal
> >> discussion on how we evaluate proposals.
> >>
> >
> > I will note that this very thread ("[DISCUSS] Outreachy framework
> > proposal") was supposed to be the start of that conversation.
>
> And I would like to continue that topic, but until the noise abates and
> the S/N is restored here, I refuse to do so. I simply haven't the energy
> to wade through the piles of vitriol.
>
> I'll leave a few points here, and hopefully someone can pick up on them:
>
> * We need a double blind evaluation system that masks project name,
> applicant name, specific details, etc.
>
>
> (FYI - using the term "anonymized evaluation system" is a better term
> "double blind evaluation system". I've seen vision-impaired people
> complain about how this term is an unpleasant one for them.)
Thank you so much! I should have realized this myself. I'll start using
"double-anonymized" or "anonymized" instead.
> I assume you mean an anonymized evaluation system for deciding which
> Outreachy internship projects to accept? The Outreachy website asks for
> a lot of project-specific details (like communication channel links,
> documentation for how to make a first contribution, etc.). I don't think
> it will be possible to make an informed decision about whether a project
> is going to be successful in Outreachy without looking at those details.
> You would need someone to do a lot of redaction on the project
> descriptions to achieve anonymity.
No, you're right, this wouldn't be easy and doesn't make a lot of sense.
> Or are you talking about an anonymous review system to help coordinators
> needing to make a decision about which intern selections? That would be
> needed if there are more interns than you have funding for.
Yes, this is what I was driving at.
> Coordinators will already have access to information about applicants
> and intern selects through the Outreachy website, so implementing an
> anonymized intern selection system might not be feasible for that. The
> information that coordinators and mentors can see on the Outreachy
> website has some details about the applicants. Each project will have a
> page that lists applicants who have recorded a contribution or filled
> out a final application. That page lists the applicant's name, email
> address, location and timezone, pronouns, and a summary of how many days
> free the applicant has during the internship. The final application
> lists answers to questions like "What is your experience in other free
> software communities?" and "What other experience do you have that is
> relevant to this project?"
>
> Mentors have tried to implement anonymous reviews of Outreachy
> applicants' contributions. I know some projects in Mozilla have turned
> on a GitHub feature to hide the name of the person submitting a
> contribution. The coordinator gathers final applications and recorded
> contributions, redacts personal information, and shares it with the
> mentors. Then the mentors make a decision on which applicant to accept,
> all without knowing details about the applicant.
>
> If the ASF wants to anonymize intern selections for review, it would
> need a neutral third party to redact personal information and
> identifying information from the intern's final application and the project.
I am concerned about both conscious and non-conscious bias entering into
the selection process, but perhaps I am worried over nothing. How
frequently do projects receive more than one intern application?
> * We need a way to ensure that mentors actually are capable of
> committing the time necessary for this programme, to check in with
> the admins on regular intervals to ensure this is happening, and to
> blacklist them from future involvement if they are unable to meet
> those expectations and they do not have an acceptable excuse, since
> real money will have been wasted. (Doctor's note, death in the
> family, etc.)
>
>
> Outreachy requires mentors to check a box during project sign up to
> specify that they understand this is a 5 hour time commitment during the
> internship. There's another box for acknowledging the 10 hour time
> commitment during the contribution period. The Outreachy organizers
> don't do anything as elaborate as what you're proposing, but there's
> nothing stopping coordinators from doing so.
Yeah, this was really an internal point for us, and the coordinator can
help ensure this point is made. I don't want the ASF to have a black eye
because people signed up to mentor, then didn't follow through. Sure,
things happen unexpectedly, but we should at least do a double-check
that the mentorship applicant knows what they're signing up for, vs.
thinking this is a "land grab for warm bodies" (which it really is not.)
> Outreachy already has feedback points three times during the internship
> (2 weeks in, 8 weeks in, and 13 weeks in). Mentors give feedback on
> their interns, including their response time, whether the intern should
> be paid their stipend for that term, and whether the internship should
> be extended. Interns give private feedback on their mentors, including
> the mentor's response time to their questions and how long it takes the
> mentors to give feedback on their contributions.
>
> The mentor and intern feedback is private, only read by Outreachy
> organizers. That's because interns sometimes talk about how their
> personal life has been impacting the amount of time they could put into
> their internship. These are often very personal matters, e.g. "I'm going
> through a divorce" or "I'm moving to a new country" or "I have this
> specific health issue". That's not something we want to automatically
> share with coordinators.
>
> However, if you're concerned about mentor response times, I can let you
> know if I see concerning trends in mentor response time or intern
> comments on the ASF mentors.
This would be very valuable feedback to the Coordinator. Yes, please! My
main goal is to ensure the interns get a good experience out of this,
and as a volunteer-run-and-led organisation, unexpected absences are
more common than most of us would like.
> * We need a double-check that the projects involved agree to follow
> not just the ASF CoC, but the Outreachy CoC, and any other imposed
> requirements
Glad to hear this isn't an issue for you :)
> * We need a ruler by which we can measure the quality of the project
> in terms of its suitability for an intern - what will the intern
> *gain* by working on the project? Or is it just self-interest of
> the requesting PMC/org? Quantified and qualified, preferably.
>
>
> One of the questions we ask mentors during project sign up is "What
> benefit will the intern have for working on this project?" I've also
> talked about other project criteria you might want to use here:
>
> https://lists.apache.org/thread.html/f790fa5bffd7c2aac3fb495593456d55bed940815891511c6d3fd039@%3Cdev.diversity.apache.org%3E
This is all really good, and I hope the selected Coordinator on our side
picks up on all of this.
> * We need to review the tagged areas Outreachy provides to ensure
> that the applications we vet cover a broad range of opportunities
> within our Foundation, not 100% documentation or internal-facing
> tools.
This is really internal direction for our Coordinator, and I think it
will be simple enough to handle.
> * We should find a volunteer to reach out to our compatriots within
> Fedora and Debian to see how they coordinate and vet their
> similar opportunity programs.
>
>
> I also would recommend talking with the Mozilla coordinators, as they
> have more projects that they vet (10-15 per round). I can connect you
> with Dustin (the main Mozilla coordinator). I would need a specific set
> of people to connect him with, rather than dragging him to a public
> mailing list thread that is at risk of meandering off topic.
Fantastic! This, plus the other Mozilla name you mentioned on the call
with Gris et. al., should get us very far.
Thanks again for all your support!
-Joan "now we're cookin'" Touzet