Authors: Victoria-Jane Welk, RaeAnn Anderson, Brittany Mancha, Zackary Anderson, Lilly Christen
Categories: Article, verbal coercion, sexual violence, cognitive interviewing, content validity
Source: Translational issues in psychological science
Doi: 10.1037/tps0000421
Authors: Victoria-Jane Welk, RaeAnn Anderson, Brittany Mancha, Zackary Anderson, Lilly Christen
Verbal coercion, a form of sexual violence, is a common problem that is related to various psychological and physical outcomes. Cognitive interviewing is an important technique used to understand how people perceive items on a questionnaire. The current study utilized cognitive interviewing and sought to examine the content validity of a potentially ambiguous item, “my partner insisted on sex”, from the Revised Conflict Tactics Scales, a measure of intimate partner violence. Participants were mostly college students at a local Midwestern University (N = 39). Participants responses were qualitatively coded, and the following themes coercion (42.68%) (physical (4.88%) and verbal (37.80%)), consent (30.49%), compliance (4.88%), hesitancy to have sex (13.41%), personal experiences (6.10%), and perpetrator’s intent (2.44%). The results show that most participants viewed the item, “partner insisted on sex” as a verbally coercive tactic; this theme was mentioned more often than non-coercive themes. Participants were also asked to rate quantitatively how consensual they considered the insistence item, and comparison items representing other tactic types. Three tactics were rated as clearly nonconsensual using a p < .05 standard, including the insistence item (M = 1.946, SD = 1.70) with a range from “0-not consensual at all” to “7-completely consensual”. This suggests evidence of consent validity since the item was perceived as representing coercive behavior.
Sexual victimization is any experience of sexual contact that occurs without freely given consent (Basile et al., 2014). Across the lifespan, in the United States, it is estimated that 43.9% of women and 23.4% of men have experienced sexual victimization (Breiding, 2015). Sexual victimization can occur by various tactics, including verbally coercive tactics. Verbal coercion can be defined as any verbal behavior used to obtain sexual activity without consent and can include a wide variety of verbal behaviors like someone begging, pleading, or making threats to coerce and therefore override, ignore, or otherwise disregard true consent, which is freely given (Livingston et al., 2004). Verbally coercive tactics are one of the most common tactics used to victimize (DeGue & DiLillo, 2005; French et al., 2015; Mellins et al., 2017); approximately one in six U.S. women have experienced verbal coercion victimization (Centers for Disease Control and Prevention). Yet, because verbally coercive tactics are so highly variable and potentially context-dependent (Anderson et al., in press) these tactics are difficult to validly and reliably operationalize in research. Thus, the goal of this study is to qualitatively examine the content validity of a potentially ambiguous verbal coercion item, “my partner insisted on sex” by utilizing cognitive interviewing and thematic analysis. By thematically coding open-ended responses, we are able to potentially differentiate verbal coercion from similar but distinct constructs like compliance, a form of sexual victimization, (consenting to sex that is not physically desired). We also integrate quantitative data to examine content validity via consentedness ratings. The overall goal of our study is to generate data that can inform the development or refinement of sexual violence measures.
Prior research suggests that verbal coercion has been measured in highly variable ways; for example, from the 15 measures listed in Supplemental Table 1, each measure includes 1 to 21 separate items measuring verbal coercion (Anderson et al., in press). Items also range widely in level of detail. Some items detail behaviors that are likely to be seen as more clearly non-consensual such as, “He threatened me” ((Conflict in Adolescent Relationships Inventory (CADRI)) whereas others are more consistent with rape myths and may not be perceived by participants as lacking in consent such as, “My partner reminded me of gifts or other benefits he gave me so that I would feel obligated to have sex with him” (Sexual Coercion in Intimate Relationships Scale (SCIRS)). These examples also contrast the degree of behavioral specificity, with the latter item much more detailed and therefore, more likely to elicit accurate responses even in participants who would not think of themselves as victimized. In this study we focused especially on verbal coercion in the form of insistence. Insistence is a subtype of verbal coercion and could take a variety of forms including repeatedly asking or begging (Kaestle, 2009; Loshek, 2015). However, the behavior of verbally insisting on something can also be a healthy, normative behavior in certain contexts. For example, insisting on one’s rights in an appropriate assertion. Thus, insistence behaviors may represent a boundary condition for verbal coercion and may be a useful model and starting point for examining how to differentiate verbal coercion from healthy, normative verbal expression of requests for sex. See the Supplemental Material for a discussion of false-positive and false-negative in sexual violence measurement.
One issue unique to measuring verbal coercion victimization is clearly differentiating verbal coercion from sexual compliance. This can be difficult because sometimes measures use the term “want” to mean consent; yet research demonstrates that when it comes to sexuality, want and consent can mean different things with “want” also indicating physical desire (Peterson & Muehlenhard, 2007). This can be especially complicated because wanting and consenting may overlap or may not. For example, a person may find Dwayne “the Rock” Johnson, a highly arousing and wanted sex partner, but not consent to a sex offer from him because they are already in a monogamous relationship. The sex would be wanted but if it still occurred just after declining the offer, it would be non-consenual^1^. The reverse may also be true that sex could be consensual, but unwanted. Sexual compliance is sex that is not wanted (e.g., desired) but is consented to without coercion, often for the purposes of relationship concerns (Darden et al., 2019; Impett & Peplau, 2002; Katz & Schneider, 2015; O’Sullivan & Allgier, 1998; Peterson & Muehlenhard, 2007). For example, a person may recognize that their partner has a higher sex drive than they do and therefore, comply to a sexual request for relationship harmony. However, research also suggests that women are more likely to comply to sex in relationship where they have less control (Conroy et al., 2015) or when there is a history of sexual victimization. This suggests compliance and victimization while distinct concepts may co-occur in an individual’s life (Katz & Tirone, 2010). Therefore, it is important to differentiate sexual compliance from verbal coercion to reduce potential false positives in measurement.
The conceptual distinction between consent and compliance illustrates how complex consent can be. Consent is a key part of how sexual victimization is measured (Cook et al., 2011) and therefore consentedness ratings of sexual victimization items are a helpful indicator of validity.
In cognitive interviewing paradigms participants are prompted with items and asked to explain their understanding of the items and their thought process as they interpret and respond to the items (Beatty & Willis, 2007). This approach is often used in questionnaire development to examine issues of potential flow, wording, and comprehension. However, it can also be fruitful for ascertaining content validity by prompting participants to explain their understanding of items (Pane Seifert et al., 2022). Thus, if the content of responses from participants is consistent with the construct as designed by researchers, there is evidence for content validity. Indeed, cognitive interviewing has become a recommended approach for examining content validity (DeVellis & Thorpe, 2021). Cognitive interviewing only requires a sample size of at least 20–30 participants because this type of design and resulting data does not rely on typical inferential statistics (Centers for Disease Control and Prevention, 2014; Perneger et al., 2015). Rather, this data is often analyzed qualitatively, including thematically.
Although this method is often reported as having taken place, the findings are rarely detailed and often only mentioned as a part of a process with no outcome data provided in the Results section. For example, in the Juvenile Victimization Questionnaire manual, it states that evaluating youth comprehension of the JVQ, “is a key reliability and validity question that has been seriously under-studied to date” and ensuring comprehension is the rationale providing for completing cognitive interviews (Hamby et al., 2004). Yet the description of the findings of the 24 cognitive interviews were described only as simplifying and shortening of items. Surely, such key reliability and validity findings deserve greater description! Yet, the lack of documenting cognitive interview findings has been a typical approach and indeed, the JVQ authors deserve credit for actually documenting their process more than most. Documenting these types of findings in more detail is important for theory building and exploring mechanistic processes that could inform violence measure development. Theory and mechanisms underscoring violence measurement have been underdeveloped; given the many examples in the literature of participant reporting discrepancies whether they have been victimized or not, clearly better measurement research and theory is needed (Anderson et al., 2018; Cascardi & Muzycyn, 2016; DiLillo et al., 2006). Findings from cognitive interviewing could help elucidate these processes (Hester et al., 2023). For example, Evans et al., (2016) found that when participants were uncertain about their interpretation of intimate partner violence items on the Composite Abuse Scale items, this led to participants’ underreporting their experiences.
The goal of the current study was to use cognitive interviewing to examine the content validity of a potentially ambiguous verbal coercion item, “my partner insisted on sex.” We chose to focus on this item as potentially exemplary of the challenge of differentiating sexual victimization from sexual compliance to explore the boundary conditions of verbal coercion. Further, this item is taken from one of the most frequently cited violence measures, the Revised Conflict Tactics Scales (CTS2: Straus et al., 1996), an often-used intimate partner violence measure which is also recommended as a measure of sexual violence (Bagwell-Gray et al., 2015). Given that approximately 40% of perpetrators are intimate partners (Basile et al., 2022), and that intimate partner perpetrators are more likely to use verbal coercion (Wegner et al., 2014), we wanted to use an item easily applicable across many relationship and sexual contexts. Yet, to our knowledge, the content validity of this measure has not been examined. By prompting participants to open-endedly explain their interpretation of this item and then thematically coding these responses, we can examine whether participants interpret this item as verbally coercive, demonstrating content validity, or interpret it in another way (i.e. a non-coercive theme is identified) suggesting lack of content validity. Thus, both the nature of the items identified, and the frequency of the themes are interpreted as indicators of content validity. In addition to using cognitive interviewing to elicit responses, we collected quantitative ratings of the perceived consentedness of this item and other items that varied in the tactic type (e.g., verbal, alcohol, et cetera) to compare and contrast perceptions of consent within and between participants to further examine content validity. Content validity would be demonstrated by expected variation in consentedness ratings by tactic type (verbal < alcohol-related) and by consentedness ratings that are rated in the non-consensual range.
There were 39 young adult participants (23 women, 15 men) who’s ages ranged from 18–34 years-old (M = 22.42, SD = 4.44); with most participants being college students (79.5%). Participants were mostly White (87.2%), some Hispanic or Latina/o/x (7.7%), Black/African American (7.7%), Native American (7.7%), and Asian or Pacific Islander (5.1%). Participants were mostly heterosexual (76.9%), some bisexual (12.8%), gay or lesbian (2.6%), and queer (5.1%).
Participants were recruited for cognitive interviewing, which lasted about 20 to 60 minutes. The cognitive interviewing protocol can be found https://osf.io/chpge/. During the cognitive interview, participants were asked questions pertaining to sexual violence questionnaires. Cognitive interviewing was used to establish the content validity by seeing if participants generate themes related verbal coercion, compliance, or insistence to the item, “my partner insisted on sex”. Interviewers used both reactive and proactive probes. Reactive probes were used to clarify participants examples and responses while proactive probes were used to ask participants specific questions. See the Supplemental Material for a description of recruitment for the study.
The interviewers, principal investigator, and authors all completed a reflexive/positionality statement based off Jacobson and Mustafa’s (2019) article. There were nine interviewers, all of whom were young women with various racial identities (White, Indigenous), socioeconomic status, sexual orientation (bisexual, heterosexual), and United States region that they were raised in (Mountain West, MidWest). Each interviewer conducted one to ten interviews. The coders each completed a reflexive/positionality statement as well. They reflected on their identities and background before completing the coding, as suggested by Hill and colleagues (1997). The research assistants were raised in various regions in the United States (MidWest, Northeast), and have varying racial (White, Native American), year in education (undergraduate, graduate), and sexual (bisexual, heterosexual, and questioning).
Participants responses from the cognitive interviewing were analyzed with inductive coding, specifically using a consensual qualitative research approach (Hill et al., 2005). A qualitative coding training was led by a postdoctoral researcher, who has years of experience with qualitative analysis and has published multiple articles that contain qualitative research. The training was on Zoom and was recorded for future reference.
Coding was conducted by three different research assistants; these coders did not conduct cognitive interviews and were unaware of the study goals. Coding consists of identifying key ideas or themes in the data. Thus, the coding process results in textual pieces of data called codes or in this paper, themes. Thematic analysis was chosen to examine if sexual violence questionnaires capture what they intended to capture. By utilizing thematic analysis, researchers can examine if participants describe coercion and if they describe any additional or contrary meaning from the same prompt.
The coders first reviewed participants’ responses and together they identified common themes among their responses and wrote definitions for the themes. Then the coders individually qualitatively coded participants’ responses into the themes and adjusted the themes accordingly as coders developed code definitions together. The coders met to discuss any discrepancies in their codes and rationale for how they coded participants responses and then made a final decision together how to code participants responses. Participants responses could be placed in multiple themes. Participants’ responses to all three prompts were coded together, in context so that the natural tendency to reference within a conversation would not be misinterpreted. In other words, a participant might reference part of their response to prompt 1 in detailing prompt 3.
An auditor was assigned to review codes for raw data. The auditor received the same qualitative training as other coders but was not present for coding or data collection, to reduce biases. The question asked to participants and the themes derived from the data were reviewed before the auditing process. The auditor went through each participant’s response and reviewed the final themes for each response. The auditor suggested a few responses be coded in the “verbal coercion” theme. These responses had to do with the partner taking a long amount of time to convince the other person to have sex or the partner asking multiple times for sex. After review, the coders agreed with the auditor’s assessment and those responses were re-coded.
Items from the Post-Refusal Sexual Persistence Scale-Victimization (Struckman-Johnson et al., 2003) were presented to elicit consent ratings that could be compared to the insist item ratings. The PRSPS-V lists various tactics that a perpetrator could have used to have sex with someone after the individual indicated “no”. For example, one item from the enticement tactic is, “removed some of your clothing to arouse you.” Consentedness ratings can be defined as the extent to which participants viewed the item as consensual or nonconsensual.
Consent ratings were completed at a later, different point in the cognitive interviewing protocol. Participants were block randomized one tactic from each tactic type (total number of items possible) – verbal coercion (6), seduction (3), misuse of authority (2), and alcohol-involved tactics (2). Given that some tactic types had more items than others, participants all completed the same number of ratings (4) some items were rated less frequently than others. Participants rated how consensual the sexual activity would be using a scale from 1–7 with one being “Not consensual at all” and seven being “Completely consensual”. Physical force tactics were not included as prior research suggests these tend to be consistently perceived as non-consensual (Russell et al., 2011) and to limit response burden.
Please see the Supplemental Material for a description of the protocol.
Coders reviewed the qualitative data and together identified these six major themes using a thematic analysis (sub-themes are listed in parentheses): coercion (physical, verbal), consent, personal experiences, compliance, hesitancy to have sex, and intent of the perpetrator for a total of eight overall qualitative themes. Content validity was determined by examining the themes and how often themes occurred; the themes reflecting coercion and occurring more often would suggest content validity whereas non-coercion themes would suggest lack of validity and should be more rarely occurring. For brevity, theme definitions are provided in Table 1 and only briefly described in text below.
Participants responses feel under the first theme, coercion, 35 times (35/39; 89.74%). There were two sub-themes physical coercion and verbal coercion. The theme coercion emerged due to verbal and physical coercion often co-occurring. Coercion is verbal pressure of manipulation or physical harm to coerce someone.
Participants responses feel under the verbal coercion sub-theme 31 times (25/39 64.10%). One participant explained that the prompt, my partner insisted on sex, as meaning “They pressured you to have sex” while another participant said, “Just probably saying that the other person was wanting to have sex or urging to do it”.
Physical coercion was identified as a sub-theme four times (4/39; 10.26%). Although the prompt focused on verbal coercion, physical coercion emerged as a theme as verbal and physical coercion may occur together. For example, one participant stated that they do not think the prompt can refer to sex with consent because of the word “insist” implying force, “I think not really. Even if they eventually agreed but insisting implies some kind of force or manipulation.” This participant also stated that the prompt suggests that, “They were forced or manipulated into rather than something they wanted”.
Consent is a major theme identified 25 times (25/39; 64.10%) One participant responded to the prompt that insisting on sex could not refer to sex without consent, “So even if they did eventually agree it’s not really consensual.” Another participant responded, “If I were more in the mood, then it would be more consensual.”
The theme, compliance, was identified four times (4/39; 10.26%). Compliance is when the perpetrator insists on having sex and the victim may become interested and comply to having sex. Compliance differs from coercion because the victim does not “give in” to having sex, rather they were not in the mood to have sex and then they decide they do want to have sex. For example, one participant responded to the prompt, Could there be times when this could refer to sex with consent, “Like my partner was like yeah I really want to and I wasn’t in the mood at first but then I tried anyway and had a good time. Not ideal but no[t] bad”.
Hesitancy to have sex is a theme that was identified 111 times (11/39; 28.21%). One participant said the following in response to the prompt, Could there be times when this would refer to sex with consent?, “No, I don’t feel like having sex right now, I’m sorry.” Another participant described hesitancy to have sex as “the partner is more or less pressuring someone to have sex when they are hesitant to have sex” in response to prompt 1.
Personal experiences is a theme that was identified five times (5/39; 12.82%). One participant referenced their own partner and said, “It has never been forced, I trust my partner”, in response to prompt 1.
The theme, perpetrator’s intent, was identified twice (2/39; 5.13%). One participant responded to Could there be times when this would refer to sex with consent?, as, “It depends on the intent on the person who is saying [it].”
Most (37) participants provided quantitative ratings of how consensual the item, “your partner insisted on sex”. The average rating was 1.946 (SD = 1.70) with a range from “0-not at all consensual” to “7-completely consensual” indicating that while most participants perceived the phrase “insist” as not being consensual, a small number did think it could portray somewhat consensual scenarios by providing ratings greater than 0. Given that a significant minority of participants brought up other types of tactics during the study, and co-occurrence of multiple tactic types is common in real life, we also examined consent ratings for non-verbally coercive tactics, such as alcohol-related tactics. Participants consentedness ratings appeared to have moderate variability in that while Ms were generally between 0–3 the entire range of the rating scale was used for multiple items. See Supplemental Table 2 for consentedness ratings ranges, Ms and SDs.
The goal of the current study was to examine the content validity of a potentially ambiguous verbal coercion item, “my partner insisted on sex” in a series of proactive and reactive prompts. Thematic qualitative analysis revealed the following coercion (42.68%) (physical (4.88%) and verbal (37.80%)), consent (30.49%), compliance (4.88%), hesitancy to have sex (13.41%), personal experiences (6.10%), and perpetrator’s intent (2.44%). Overall, the themes revealed that most participants view the item, “my partner insisted on sex” as a verbally but not physically coercive tactic. Further, participants appeared to differentiate coercion from compliance in that compliance was rarely mentioned, and when compliance was identified, it was often used to indicate a contrast to coercive behavior. This suggests evidence for the content validity of the item as a measure of verbal coercion. As for consent ratings of the item, “my partner insisted on sex”, participants responses ranged from completely consensual to not at all consensual. Since participants responses were greater than 0 (M = 1.69, SD = 1.60), this shows that a small number of participants viewed the item as somewhat not consensual.
Several of the qualitative themes suggest evidence of content validity for this item due to the high percentage of participants responses falling under the following themes– verbal coercion (37.80%), consent (30.49%), and hesitancy to have sex (13.41%). Further, these themes were generated by participants far more frequently than contrary themes – such as compliance (4.88%). There was evidence of content validity via the verbal coercion theme – identified by 37.80% of participants and consent – identified by 30.49% of participants. This is notable in that reviewing the literature and Table 1 suggests that “your partner insisted on sex” is one of the more ambiguous ways of measuring verbal coercion; thus, content analysis of other verbal coercion items may generate a higher number of themes consistent with verbal coercion.
Quantitative ratings of the insist item suggest the majority of participants saw this item as generally nonconsensual (M = 1.69 on a scale of 0–7), consistent with the qualitative findings that most participants interpreted the item as coercive. We also asked participants to rate other tactic items to compare/contrast degrees of consentedness. The quantitative ratings show that the following items were generally viewed as clearly misuse of authority, some verbal coercion items (“Your partner insisted on sex”, “Tried to talk you into it”), and one of the alcohol items (“Gave you drugs or alcohol to get you high”). However, one of the alcohol items (“Took advantage of the fact you were drunk or high”), the findings were less clear.
The main limitation to this study is not having another verbal coercion item to have data to compare participants responses to. Another limitation of the study is that most participants were White, middle-class, and all were young adults. Older adults, senior citizens may have different perceptions. We had relatively few quantitative ratings for some of the items due to the randomization procedures.
As for the consentedness ratings, the range of consentedness ratings (0 – 7) suggest that when researchers are operationalizing seduction, verbal coercion, and alcohol-related violence they may need to more clearly describe the coercive behaviors and lack of consent so that these behaviors are more clearly perceived as non-consensual and coercive. The mixed consent ratings also suggest that focusing on these types of behaviors that were perceived as varying in consent could be beneficial in prevention/educational programs. It is likely that participants will have more questions and confusion around how to respond to verbal coercion and alcohol-related violence than physical force tactics.
Cognitive interviewing was used to better understand participants’ understanding of verbal coercion. Findings suggest most participants view “insisting on sex” to be verbally coercive even though this item is one of the more ambiguous in the current literature. Future research should examine how to more clearly operationalize the tactics given mixed consent ratings we found within tactics of the same type.