[jira] [Comment Edited] (DI-32) Ask legal-discuss@ about offering incentives to survey takers

["Georg Link (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DI-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16981978#comment-16981978 ] 

Georg Link edited comment on DI-32 at 11/25/19 11:32 PM:
---------------------------------------------------------

This message is obsolete. We decided in [dialog|https://the-asf.slack.com/archives/CNN750WER/p1574702069327500] on the Slack channel to NOT offer incentives.

 

Here is why we decided against incentives:

{quote}We have agreed on the following standpoint regarding the gift card raffle.
  
 We cannot speak to the legality of entering participants for a chance of winning a gift card. First, because we are no lawyers. Second, because we could not find sufficient information online. Third, because of what Largent (2016) wrote: “Unfortunately, the various laws, regulations, and ethical guidelines that govern the conduct of human subjects research offer relatively little in the way of specific guidance about the factors and/or features that render offers of payment ethically acceptable.”
  
 Largant’s observation is concerned with any type of paying research participants. The uncertainty exists despite a general agreement in academic literature that research participants should be paid at least to cover their expenses related to the participation (e.g., transportation, opportunity cost). Our idea of entering participants into a raffle is a very specific case. It is widely practiced but has opposition as well. For example, Brown et al. (2006) published their reasoning for not allowing at their university to incentivize research participants with entries into a raffle.
  
 After reading up on the subject of paying participants, I collected four major concerns, some of which we can mitigate. This is not a complete list of concerns with paying research participants.
  
 1) *Amount* — the reward should not exceed the cost of time invested by participants. Some Institutional Review Boards (IRB) have guidelines like $20/h max compensation. If we assume that our survey only takes 15 minutes, this would mean we should pay at most $5. Let’s say we double it, and make it $10/gift card - that is a reasonable amount that I’ve seen before and it is the smallest gift card value offered by Amazon.
  
 2) *Informed Consent* — participants have to be informed about their cost and benefit of participating. We clearly give them the cost by letting them know number of questions and time the survey takes them. We can improve the clarity for the benefit. The benefit we provide is a chance to win one of several gift cards. This chance is determined by the number of participants as each participant can enter to win and thereby reduces the chances for everyone else. We can provide participants with a worst case probability, let’s say we expect no more than 10,000 participants and we have 100 gift cards of $10, then each participant has a chance of about 1% or better. Ethical concerns persist because our minds have biases that overvalue such probabilities.
  
 3) *Justice* — paying only some participants is not just, according to some ethical standpoints. I personally agree with a different view that it is just for everyone to have the same chance of winning, but I am also privileged and winning or losing does not impact my financial situation in any meaningful way.
  
 4) *Logistics* — paying in Amazon gift cards in USD does not work internationally. Last I know, Amazon is not in all markets and does not convert gift cards to local currency.
  
 We acknowledge that there are benefits to offering a raffle reward to participants. We also acknowledge that ASF is not a university conducting research. We acknowledge that not offering a financial incentive may result in fewer responses. However, not offering such incentive may also contribute to better response quality because it filters out those who take the survey only for the chance to win a gift card (and those who respond multiple times for better win chances).
  
 *In Conclusion, we recommend to NOT use a raffle for incentivizing survey participants.* 
  
 REFERENCES:

 * Brown, J. S., Schonfeld, T. L., & Gordon, B. G. (2006). “You May Have Already Won...“: An Examination of the Use of Lottery Payments in Research. IRB: Ethics & Human Research, 28(1), 12–16. Retrieved from [https://www.jstor.org/stable/30033185]
 * Largent, E. A. (2016). Recently proposed changes to legal and ethical guidelines governing human subjects research. Journal of Law and the Biosciences, 3(1), 206–216. [https://doi.org/10.1093/jlb/lsw001]{quote}


was (Author: georglink):
This message is obsolete. We decided in [dialog|https://the-asf.slack.com/archives/CNN750WER/p1574702069327500] on the Slack channel to NOT offer incentives.

 

Here is why we decided against incentives:


We have agreed on the following standpoint regarding the gift card raffle.
 
We cannot speak to the legality of entering participants for a chance of winning a gift card. First, because we are no lawyers. Second, because we could not find sufficient information online. Third, because of what Largent (2016) wrote: “Unfortunately, the various laws, regulations, and ethical guidelines that govern the conduct of human subjects research offer relatively little in the way of specific guidance about the factors and/or features that render offers of payment ethically acceptable.”
 
Largant’s observation is concerned with any type of paying research participants. The uncertainty exists despite a general agreement in academic literature that research participants should be paid at least to cover their expenses related to the participation (e.g., transportation, opportunity cost). Our idea of entering participants into a raffle is a very specific case. It is widely practiced but has opposition as well. For example, Brown et al. (2006) published their reasoning for not allowing at their university to incentivize research participants with entries into a raffle.
 
After reading up on the subject of paying participants, I collected four major concerns, some of which we can mitigate. This is not a complete list of concerns with paying research participants.
 
1) *Amount* — the reward should not exceed the cost of time invested by participants. Some Institutional Review Boards (IRB) have guidelines like $20/h max compensation. If we assume that our survey only takes 15 minutes, this would mean we should pay at most $5. Let’s say we double it, and make it $10/gift card - that is a reasonable amount that I’ve seen before and it is the smallest gift card value offered by Amazon.
 
2) *Informed Consent* — participants have to be informed about their cost and benefit of participating. We clearly give them the cost by letting them know number of questions and time the survey takes them. We can improve the clarity for the benefit. The benefit we provide is a chance to win one of several gift cards. This chance is determined by the number of participants as each participant can enter to win and thereby reduces the chances for everyone else. We can provide participants with a worst case probability, let’s say we expect no more than 10,000 participants and we have 100 gift cards of $10, then each participant has a chance of about 1% or better. Ethical concerns persist because our minds have biases that overvalue such probabilities.
 
3) *Justice* — paying only some participants is not just, according to some ethical standpoints. I personally agree with a different view that it is just for everyone to have the same chance of winning, but I am also privileged and winning or losing does not impact my financial situation in any meaningful way.
 
4) *Logistics* — paying in Amazon gift cards in USD does not work internationally. Last I know, Amazon is not in all markets and does not convert gift cards to local currency.
 
We acknowledge that there are benefits to offering a raffle reward to participants. We also acknowledge that ASF is not a university conducting research. We acknowledge that not offering a financial incentive may result in fewer responses. However, not offering such incentive may also contribute to better response quality because it filters out those who take the survey only for the chance to win a gift card (and those who respond multiple times for better win chances).
 
*In Conclusion, we recommend to NOT use a raffle for incentivizing survey participants.* 
 
REFERENCES: * Brown, J. S., Schonfeld, T. L., & Gordon, B. G. (2006). “You May Have Already Won...“: An Examination of the Use of Lottery Payments in Research. IRB: Ethics & Human Research, 28(1), 12–16. Retrieved from [https://www.jstor.org/stable/30033185]
 * Largent, E. A. (2016). Recently proposed changes to legal and ethical guidelines governing human subjects research. Journal of Law and the Biosciences, 3(1), 206–216. [https://doi.org/10.1093/jlb/lsw001]

> Ask legal-discuss@ about offering incentives to survey takers
> -------------------------------------------------------------
>
>                 Key: DI-32
>                 URL: https://issues.apache.org/jira/browse/DI-32
>             Project: Diversity and Inclusion
>          Issue Type: Task
>          Components: Demographics Survey
>            Reporter: Griselda Cuevas Zambrano
>            Assignee: Daniel Gruno
>            Priority: Major
>              Labels: research
>             Fix For: EDI Survey v2.0
>
>
> Ask the legal-discuss@ alias if it's possible to offer incentives to survey takers, and if yes, what process should we follow. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)