Authors: Mikael Selvin, Kjerstin Almqvist, Maria Fogelkvist, Lars-Olov Lundqvist, Agneta Schröder
Categories: Original Articles, Forensic psychiatric care, instrument development, patient participation
Source: Journal of Forensic Nursing
Patient participation is central in modern health care. However, it is a complex phenomenon that lacks a clear definition, and what constitutes participation varies depending on the context and theoretical perspective. It is known that patient participation in forensic psychiatric care is often rated as low by both patients and professionals, and it can be assumed that interventions to increase it are beneficial. In this process, management and staff could benefit from assessing perceived patient participation, and reliable and valid measurement instruments are essential. The aim of this study was to develop an instrument that could be used to measure experiences of participation in forensic psychiatric care from a patient perspective and test it for content validity. A definition of patient participation in forensic psychiatric care was formulated and operationalized in an instrument that an expert group, consisting of patients with ongoing care, evaluated for content validity. In total, 50 items were sorted into five different to have good communication, to be involved, to have mutual trust, to trust the care, and to take responsibility. After psychometric testing, the instrument has the potential to become a tool to use in research, clinical work, and development work in the field of forensic psychiatric care. In addition to being used as a measure, the Patient Participation in Forensic Psychiatric Care can also be used to encourage a dialogue about their care and to make both patients and staff more aware of patient participation.
KEY WORDS: Forensic psychiatric care, instrument development, patient participation
Patient participation is central in modern health care. Since the early 1960s, the patient's position in health care has been strengthened and gradually developed toward a more mutual patient–professional relationship, where both parties have a responsibility to take an interest or to act (Sahlsten et al., 2008; Thórarinsdóttir & Kristjánsson, 2014). The concept refers to the possibilities for patients to actively influence decision making about their care through an equal dialogue based on their own preferences combined with professional expertise (Castro et al., 2016). Patient participation is an important element in the patient–professional relationship; it has been highlighted in several theories and models (Chester et al., 2016; Ekman et al., 2011; Gruskin et al., 2007; Patel et al., 2008), and positive effects have been highlighted in research (Dwamena et al., 2012; Varming et al., 2015; Weingart et al., 2011). Because patient participation arguably plays an essential role in health care, it can be assumed that interventions to increase it are beneficial. However, as in all kinds of improvement work, it is important to know what to improve, how to do it, and when the desired results are achieved (Meyers et al., 2012). Management and staff could benefit in this process from methods for assessing perceived patient participation, and it is helpful to have reliable and valid measurement instruments (Kimberlin & Winterstein, 2008).
Forensic psychiatric care is a specialized field where the care involves treatment of patients with serious mental disorders who also have committed a crime (Nedopil et al., 2015). Compared with other forms of psychiatric care, the patients tend to show multiple disabilities, including antisocial behavior, poor insight, treatment adherence problems, and substance misuse (Gordon & Lindqvist, 2007). It is known that patient participation in forensic psychiatric care often is rated as low by both patients and professionals (Lundqvist et al., 2014; Lundqvist & Schröder, 2015; Selvin et al., 2019), and several aspects could explain this. The setting presents special challenges for professionals to establish and promote patient participation (Hörberg, 2008, Magnusson et al., 2020). They have to balance the paradoxical role of caring for the patients' interests and development, while simultaneously representing and adhering to the rules and regulations in the system in which they are employed (Söderberg et al., 2019). However, the complexity of the care and the severe disorders prevalent among the patients do not justify the exclusion of the patient perspective of participation in forensic care. On the contrary, the severity of psychiatric disorders together with long-lasting care times demand high-quality forensic care and the inclusion of the patient's perspective.
In addition to being challenging at a clinical level, patient participation is also a complex phenomenon, and what constitutes participation can vary depending on the context and theoretical perspective (Eldh et al., 2006; Jørgensen et al., 2018). In addition, there are a variety of factors such as staff and patients' attitudes, competence, environment, motivation, and health status, which directly or indirectly can affect patient participation and make the definition task even more complex (Sahlsten et al., 2008; Tobiano et al., 2015). Therefore, a specific definition of patient participation in forensic psychiatric care will contribute to the general knowledge of the concept, and it is recommended that instrument development be preceded by conceptual analysis, conceptual synthesis, or the derivation of concepts (Mishel, 1998).
Eldh et al. (2014) developed a tool for attempting to increase patients’ participation in care and to understand their preferences for participation—the 4Ps. The framework is based on patients' experiences, and it defines patient participation as having dialogue, partaking in planning, and managing self-care. The Patient Participation in Rehabilitation Questionnaire (Melin & Årestedt, 2020) and the Patient Participation Questionnaire (Berg et al., 2020) are other instruments developed for measuring patient participation. There are, however, to the authors' knowledge, no instruments that measure patient participation in forensic psychiatric care from the patient perspective. The instruments noted previously are concerned with participation in somatic care and are not necessarily directly transferable to forensic psychiatric care. Furthermore, even though the Quality in Psychiatric Care–Forensic Inpatient Instrument (Schröder et al., 2013) has a subscale with items relating to participation; it captures participation in a more general way. Therefore, the aim of this study was to develop an instrument that could be used to measure experiences of participation in forensic psychiatric care from a patient perspective and test it for content validity.
The study followed guidelines for instrument development as suggested by Devellis (2016). On the basis of these guidelines, this study was conducted in four main stages. The first stage was to formulate a definition of patient participation in forensic psychiatric care. The second stage was to operationalize the definition in an instrument and generate a pool of items. The third stage was to determine the format for measurement and to design the first version of the instrument. The fourth stage was to test the instrument for content validity with an expert group—in this case, patients in forensic psychiatric care.
On the basis of previous descriptions of the concept of patient participation in forensic psychiatric care from a patient perspective (Selvin et al., 2016), the following definitions were
In the original study (Selvin et al., 2016), the category “Own responsibility” had two subcategories, “To participate in activities” and “To take the initiative.” In this definition, we decided to merge these into only one category because of the relatively fewer number of statements.
The definition was operationalized in an instrument called Patient Participation in Forensic Psychiatric Care (PPFPC). The instrument was designed to measure the patient's own experience of participation in forensic psychiatric care.
To formulate an item pool, we used data from the previous phenomenographic interview study with 19 patients in forensic psychiatric care (Selvin et al., 2016). Statements from the results in that study formed the basis of the item pool, and we also went back to the original transcribed interviews to fully understand the context. On the basis of these data, items were derived, and it was important in this phase to keep the language in the items as close to that in the original interview statements as possible. The items were discussed in the research team several times regarding how well they captured the central ideas. Content as well as clarity has a significant bearing on item quality, in that each item should be unambiguous and easy to understand (Devellis, 2016). To achieve clarity, we considered factors such as reading difficulty level, number of words, and avoiding multiple negatives. We also decided to use only positively worded items to avoid misunderstandings, especially when considering that the patient group that is supposed to use the instrument might have cognitive disabilities.
During the process, we sorted all the items into dimensions and we used the same wordings as in the original categories. For example, an item formulated from a statement from the interviews belonging to the category “To be involved” was put in the dimension “To be involved” in the instrument. After consensus in the research group, we ended up with 50 items that we thought fully covered the subject and at the same time were a manageable amount to test for content validity.
The instrument included a number of background questions, such as gender, age, and how long they had been treated in forensic psychiatric care. All items were related to the statement “I experience that…,” and a 5-point Likert scale from 1 = strongly disagree to 5 = strongly agree was used (see Figure 1). At the end of the instrument, there was an open question if the respondent had something more to add.
FIGURE 1 Excerpts from the Patient Participation in Forensic Psychiatric Care instrument.
The instrument was evaluated for content validity by an expert group consisting of 16 patients having ongoing forensic psychiatric care.
Patients from two different forensic psychiatric clinics in Sweden were represented in the study. Both clinics had a medium security classification and were similar in size, each with around 25-bed sites and a staff of around 100. The patients treated at the clinics were admitted according to the Swedish Forensic Mental Care Act (1991:1129), which means that they had committed a crime that normally would have led to prison but, because of serious mental disorders, they were transferred to forensic psychiatric care. The most common diagnoses among the patients at the clinics were psychotic disorders, personality disorders, and substance misuse.
All patients in the forensic psychiatric clinics were informed about the study by a contact person at the clinic in collaboration with the first author. A poster and folders with written information were created and made available on the wards. All patients who were interested in participating in the study could sign up with the contact person. The only exclusion criteria were if the patient could not speak or read Swedish or were severely cognitively affected. In total, 16 patients wanted to participate in the study. They were all male, between 21 and 55 years old, and their current average treatment time was 26 months (9–54 months). Only men elected to participate in this study because there were no female patients treated at the wards during the data collection. The study was approved by the regional ethics committee in Uppsala (Reference Number 2019-05155), and all participants gave their written informed consent to participate before the start of the study.
The data collection was conducted in two steps. First, the patients filled in the PPFPC instrument individually at the wards. In this step, the patients answered the questions as if it had been a standard instrument and out of their own experiences as patients in forensic psychiatric care.
In the next step, they evaluated the PPFPC and discussed the instrument. The patients were divided into four groups and attended one meeting each that took around 90–120 minutes. The first author led these meetings, and the third author took notes during the discussions. Large rooms were used, which allowed us to comply with restrictions regarding the COVID-19 situation we had at this time (summer 2020). Each group session started with the patients filling in an adjusted version of the instrument (see Figure 2), where they rated the items according to relevance and comprehensibility. The relevance of each item was rated on a 4-point Likert-type 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, and 4 = very relevant. The comprehensibility was assessed on a 3-point Likert-type 1 = hard to understand and unclear, 2 = acceptable, or 3 = easy to understand and clear. Several patients needed clarifications of the concepts, which were given by the researchers, thus indicating the need for clarification of the questionnaire. When the patients had finished the rating, a group discussion led by the first author took place, guided by the following
FIGURE 2 Excerpts from the evaluation protocol of the Patient Participation in Forensic Psychiatric Care instrument.
The data from the first step of the data collection were analyzed to give the mean values for each item and for each dimension. The data from the evaluation of PPFPC were analyzed using the content validity index for items (I-CVI) and the average of all the I-CVIs on the overall scale (S-CVI/Ave). To calculate I-CVI, the number of experts who rated an item as quite relevant (3) or very relevant (4) was divided by the total number of experts. Thus, if 14 of 16 experts rated an item as “quite relevant” or “very relevant,” the I-CVI is 14/16 = 0.875. To calculate the S-CVI/Ave, the sum of all I-CVIs was divided by the total number of items. It is recommended that for a scale to be judged as having excellent content validity, it should be composed of items with a minimum I-CVI of 0.78 and it should have an S-CVI/Ave of 0.90 or higher (Polit & Beck, 2006). Cronbach's alpha was used to assess the internal consistency of the scale, and coefficients higher than 0.7 were considered acceptable (Nunnally & Bernstein, 1994). We investigated the instrument for possible ceiling and floor effects. Because of the specific and limited selection of patients, we considered a floor or ceiling effect if more than half of them scored a 1 (floor effect) or 5 (ceiling effect) on the PPFPC.
The results are presented in the following First, the data from the patients' experiences of participation and, second, the evaluation of the instrument. Finally, there is a summary of the group discussions.
The mean scores for each dimension are presented in Table 1. The To take responsibility dimension had the highest mean score (4.20), followed by To have mutual trust (3.82), To have good communication (3.78), To trust the care (3.60), and To be involved (3.49).
In total, 13 items from four of the five dimensions had a skewed response distribution, with a high maximum score (i.e., 5) indicating a ceiling effect. In the dimension To have good communication, Item 31 (I understand why I am being treated in the forensic psychiatric care), Item 36 (I am aware of the section's routines), and Item 41 (I understand the rules of the section) had a maximum score (5) with more than 50% of the patients. In the dimension To be involved, it was Item 22 (I regularly take part in meetings where my care is planned) that had this effect. In the dimension To have mutual trust, the following items had a ceiling Item 8 (the staff take me seriously as a person) and Item 38 (the staff treat me fairly). Among all the dimensions, the To take responsibility dimension had most items with a ceiling Item 5 (I stick to what we have agreed on in my care plan), Item 10 (I follow the directions for my medicine prescriptions), Item 25 (I stick to the rules in place in the section), Item 30 (I participate in my own care), Item 35 (I do whatever is needed for me to be discharged), and Item 40 (I take personal responsibility for my care). No item showed a floor effect.
The internal consistency was adequate with Cronbach's alpha coefficients greater than 0.70 in all To have good communication (0.85), To be involved (0.88), To have mutual trust (0.95), To trust the care (0.89), and To take responsibility (0.78). The Cronbach's alpha coefficient for the full instrument was 0.97.
The data from the second part of the data collection were analyzed with descriptive statistics, and these are presented in Table 1. Seven items were Item 26 (I know the reasons for the decisions made about my care), Item 12 (I have the possibility to choose between different treatment alternatives), Item 17 (I can influence decisions about my care as much as I choose), Item 37 (I am able to get across my opinion before meetings when I am not able to be present), Item 33 (The staff help me to take initiative when I myself cannot manage it), Item 44 (The things the staff and I agree upon actually happen), and Item 48 (I have confidence in the legal procedures). All these items had an I-CVI index below 0.78, which indicates less-than-excellent validity. Two items, Item 22 (I regularly take part in meetings where my care is planned) and Item 33, had a comprehensibility index below 0.78.
The instrument was described as useful and important by all groups, and the need for improvement in the quality of forensic psychiatric care was emphasized by the informants. The items were generally described as being easy to answer and neither too long nor too short. Single individuals had some difficulties in filling in the instrument and asked for a shorter version, and some patients suggested that questions concerning different activities should be added. In all groups, the instrument initiated a dialogue concerning the care and various aspects of it. The patients were keen on discussing the quality of the care and the different aspects of patient participation, and they expressed a wish that the use of the instrument would contribute to needed improvements of their care.
Most of the patients were positive about the instrument and believed it could be useful to help improve care. They also said that being included in the discussion about the development of the instrument made them more aware of patient participation, which they said was a good thing.
In this study, we described the initial development of the PPFPC instrument and tested it for content validity. PPFPC is the first instrument to be developed for measuring patients' experiences of participation in forensic psychiatric care. To be valid, it is necessary that the instrument is based on theoretical and empirical evidence and that it can represent the patients' views of patient participation (Beattie et al., 2015). It is also beneficial if patients are involved in decisions about the wording and design of instruments (Boyer et al., 2009). Therefore, it was important to keep the patient perspective throughout the entire process.
In the first step, the patients answered the questions in the instrument and the dimension with the highest mean value was To take responsibility (4.20), whereas the dimension with the lowest mean value was To be involved (3.49). It is interesting, although not surprising, that the To take responsibility dimension stands out as the dimension with most items with a ceiling effect because most of the items were about the patients' own efforts in the care. Generally, when designing an instrument, items are chosen that will not offend the responder, but if the statements lack the ability to differentiate, there is a risk that they elicit too much agreement (Devellis, 2016). There is, however, a well-documented bias that people's subjective confidence in judgments of their own efforts is often reliably greater than the objective accuracy of those judgments (Moore & Healy, 2008), and this case might be an example of such bias. In the second step, where the patients evaluated the items, the most interesting finding was that seven of the items had an I-CVI index below 0.78, which indicates less-than-excellent validity (Polit & Beck, 2006).
When we constructed the instrument, we chose to include 50 items sorted into five different To have good communication, To be involved, To have mutual trust, To trust the care, and To take responsibility. We decided to use only positively worded items because it can be confusing for respondents if there are reversals in item polarity, which could lead to the possibility of inconsistent responses and alternations in the internal consistency (Devellis, 2016). Two of the items had a comprehensibility index below 0.78, which indicates that those specific items were unclear but, overall, the wording of the instrument was clear and easy for the patients to understand. There is, however, a possibility of acquiescence bias when using only positive items, but it is probably easier to control for this by informing the subjects of the importance of their responses (Salazar, 2015).
It is interesting to compare the content with other instruments that measure patient participation in health care, and it might be arguable as to whether or not more instruments are needed. For example, the Patient Participation Questionnaire (Berg et al., 2020) has the dimensions Involvement, Information, Communication, and Relationship to the staff. The Patient Participation in Rehabilitation Questionnaire (Melin & Årestedt, 2020) measures Respect and integrity, Planning and decision-making, Information and knowledge, Motivation and encouragement, and Involvement of family. Even if those dimensions at first glance are similar with the dimensions in PPFPC, there are important differences. Particularly the content within the dimensions, that trust seems to be more prominent in this patient group and that participation correlates with motivation in forensic psychiatric care (Selvin et al., 2016). Thus, the PPFPC provides the opportunity to investigate the concept of participation among patients in forensic psychiatric care in more detail.
The next step will be to modify the items, undertake a pilot study, and evaluate the instrument in further psychometric testing. Therefore, we need to consider how the results in this study should be used in future revisions of the instrument. It can, for example, be argued that the items with an I-CVI index below 0.78 could be deleted because of low relevance. It is also possible that the dimension To take responsibility is redundant because of so many items having a ceiling effect. There is, however, a need for psychometric testing with a larger number of participants to determine if this would be worthwhile. For example, most of the items with an I-CVI index below 0.78 belong to what could be considered elementary in the definition of patient participation (Eldh et al., 2006; Sahlsten et al., 2008; Tobiano et al., 2016). It is also important to consider the difficulty for any person to evaluate the quality of items and how they relate to a concept in an instrument instead of answering the questions. It is perhaps even more important to consider this when we know that such patient groups often have some cognitive disabilities that relate to psychiatric diagnoses (National Forensic Psychiatric Quality Register, 2020).
A shortened version of the instrument is desirable under any circumstances. In this version with 50 items, the I-CVI/Ave was 0.86, which is considered to be less than excellent content validity, but with a smaller number of items with better quality, the overall quality of the instrument would increase. Some of the patients also told us in the group discussions that they wished for fewer items because of the challenge presented by needing to focus so long. This is an indication for the development of a shorter version and should be considered in future research.
A strength with this study was being able to keep the patient perspective during the whole process from choosing items to the phase where we relied on patients as our expert group for helping to determine content validity and in discussions regarding the instrument. It was advantageous to meet and discuss the instrument with patients in forensic psychiatric care because of their valuable input and positive responses. The researchers were not involved in the patients' care in any way, and we made clear in the instructions that it was important to be honest in their evaluations. It is possible, however, that the patients evaluated the instrument in a more positive way because they thought it could lead to some kind of advantage and that they told us what they believed we wanted to hear.
It would have been interesting to have some female patients evaluating the instrument, but it was not possible at the time this study was conducted. Data collection took place during the COVID-19 pandemic; it was difficult to access the clinics where we had an ongoing contact, and it was not a good time to reach out to other clinics.
One exclusion criterion was if the patient was severely cognitively affected, but the instruction to the contact persons at the clinics was not very specific, other than that it referred to ongoing psychosis, for example. There were no specific assessments of the participants' cognitive abilities for participating in this kind of study. It is therefore possible that some of the patients had difficulties in understanding the task, especially regarding the evaluation of the relevance of items in relation to patient participation. This can be quite challenging. In further psychometric testing, we will consider the inclusion of more female patients and have independent persons to conduct the data collection. We acknowledge that the research was conducted only in Sweden and the services available there may not be applicable in other jurisdictions.
Both patients and professionals have reported challenges regarding patient participation in forensic psychiatric care, which indicates a need for improvement. A valid and reliable instrument for measuring patients' experiences of patient participation could be useful for this purpose. Our findings indicate that the instrument, after further psychometric testing, has the potential to serve as a tool in clinical work to improve patient participation in their care. In addition to being used as a measure, the PPFPC can also be used to encourage dialogue about care and to make both patients and staff more aware of the value of patient participation.
This is the first instrument to be developed to specifically assess participation in forensic psychiatry based on the patient perspective. In this study, the initial development of the instrument was described, and this makes a solid base for a future study. After further psychometric testing, the instrument has the potential to become a useful tool to use in research, clinical work, and development work in the field of forensic psychiatric care, with the intention of enhancing clinical practice.
The patients contributed with their knowledge and experiences as experts in this study, and we would like to thank them all for that. We would also like to thank the contact persons at the clinics for helping with the logistics of the study and Lars Kjellin for valuable input in the writing process. Finally, thanks are due to Mervyn Gifford for the English language proofreading.
Kjerstin Almqvist, Email: kjerstin.almqvist@kau.se.
Maria Fogelkvist, Email: maria.fogelkvist@regionorebrolan.se.
Lars-Olov Lundqvist, Email: lars-olov.lundqvist@regionorebrolan.se.
Agneta Schröder, Email: agneta.schroder@regionorebrolan.se.