Experiences of being a control group: lessons from a UK-based randomized controlled trial of group singing as a health promotion initiative for older people

Abstract

Existing randomized controlled trials within the health field suggest that the concept of randomization is not always well understood and that feelings of disappointment may occur when participants are not placed in their preferred arm. This may affect a study's rigour and ethical integrity if not addressed. We aimed to test whether these issues apply to a healthy volunteer sample within a health promotion trial of singing for older people. Written comments from control group participants at two points during the trial were analysed, together with individual semi-structured interviews with a small sample (n = 11) of this group. We found that motivation to participate in the trial was largely due to the appeal of singing and disappointment resulted from allocation to the control group. Understanding of randomization was generally good and feelings of disappointment lessened over time and with a post-research opportunity to sing. Findings suggest that measures should be put in place to minimize the potential negative impacts of randomized controlled trials in health promotion research.

INTRODUCTION

Despite increasing life expectancy over the last 30 years, healthy life expectancy has not kept pace with this over the same period (ONS, 2010). This has given rise to a need for cost-effective and evidence-based ways of promoting and maintaining the health of older people. Recently, there has been growing interest in the potential value of participatory arts, such as singing, in addressing health and wellbeing (Clift, 2012), generating a subsequent need for well-designed studies, and particularly controlled trials, to evaluate the effectiveness of such programmes.

Randomized controlled trials (RCTs) are widely accepted as providing the definitive method of ascertaining treatment effects (Howard and Thornicroft, 2006). The crucial feature of random assignment to intervention and control arms is seen as the ‘gold standard’ (Corrigan and Salzer, 2003), which ensures there are no systematic factors differentiating treatment groups which may explain observed differences (Sibbald and Roland, 1998). Random allocation is also seen to be ethically justifiable using the principle of ‘clinical equipoise’ (Binik et al., 2011) meaning that there is no agreement within the research community as to the preferred treatment or the effectiveness of the intervention being compared with usual activities. As such, this approach is often the accepted method for evaluating health promotion interventions (for example Kolt et al., 2006; Yardley and Nyman, 2007; Ji et al., 2008).

Recently, however, a number of criticisms of the RCT as a design have emerged. Busby Grant et al. (Busby Grant et al., 2009) have argued that subjects of such studies have historically been assumed to be passive recipients of assigned conditions, whereas human participants are subject to psychological reactions, cognitive and behavioural influences, which may impact on a study's internal validity. In particular, there is concern regarding individuals' understanding and acceptance of randomization, giving rise to concern about the ethical principle of informed consent.

A review by Robinson et al. (Robinson et al., 2005) found that much recent work reports participants' failure to understand information about randomization. Subsequent studies have supported this in their finding that a minority of participants find the notion of randomization either too complex and difficult to comprehend or unacceptable and unfair within a clinical trial (Busby Grant et al., 2009). Although there is also evidence in many studies for positive recall and understanding of information, and acceptability of random allocation to treatments (Feathersone and Donovan, 1998; Kerr et al., 2004; Locock and Smith, 2011), most of the evidence for this has emerged from clinical intervention studies or analyses of hypothetical scenarios. Moreover, most research is not specifically concerned with older people, the focus of the present study and also the group for whom there is a need for rigorous evidence of health promotion interventions.

Studies where details of preference to allocation have been collected have reported that most of their participants, unlike researchers, lacked equipoise, that is, they stated a preferred treatment condition in unblinded studies (Snowdon et al., 1997; Leykin et al., 2007; Behrendt et al., 2011). This may have implications for both trial participants and research methodology. For participants there is a danger of ‘resentful demoralization’ among individuals who are aware that they are not in their treatment group of choice. For the research there is a potential for recruitment bias (King et al., 2005; Harrison et al., 2009) and retention problems (Corrigan and Salzer, 2003; Howard and Thornicroft, 2006), though evidence for the effect on outcomes is mixed (Hart, 2001; Thomas et al., 2004).

Between January 2010 and December 2011 we conducted a randomized controlled trial on the health benefits of group singing for older people in S.E. England (Skingley et al., 2011). Volunteers aged 60+ who consented and returned baseline questionnaires (n = 253) were randomized into one of five singing groups or corresponding ‘usual activities’ groupings. Follow-up self-report questionnaires were administered to participants at 3 months (immediately post-intervention) and 6 months. Approval was granted by an National Health Service research ethics committee (REC ref. 10/H1109/5). All participants were promised an opportunity to sing post-research.

Recruitment to the research was by self-selection in response to advertising widely, and the nature of randomization was explained both in information leaflets and at information sessions at each venue where potential participants were also able to experience a ‘taster’ of what being in a singing group would be like. Since our sample was composed of volunteers, we assumed that the practice would be acceptable to all those consenting. We were, therefore, concerned when written comments from some control group participants in the month 3 questionnaire (follow-up questionnaire 1) indicated a sense of considerable disappointment at their allocation. This phenomenon appears not to have been examined to any great extent in the health promotion literature with self-selecting, healthy participants, though Ji et al. (Ji et al., 2008) did encounter refusals to participate in their cluster RCT of a health promotion intervention for schools (but where refusal was at the level of the cluster, or school rather than the individual). We decided to explore this further and embed within the trial a small study of reactions to control group placement, using these initial data as a starting point.

AIMS

• To explore the reasons behind motivation to take part in the RCT.

• To establish whether the information provided about randomization was seen by participants to be adequate and understandable.

• To gauge participants' reactions to being placed in the control group and their subsequent experiences.

• To seek any suggestions for improving the experiences of the control group.

• To make recommendations for future research practice.

METHOD

In addition to the comments at month 3 (follow-up questionnaire 1), there were two further means of data collection: Written comments were analysed for indications of understanding and ‘resentful demoralization’ to establish broadly whether this was seen as an issue by individual participants. Interview data were transcribed and subjected to template analysis (Waring and Wainwright, 2008). This method makes use of codes and coding of data and can be used within a range of epistemological positions. Its defining feature is the development of a coding template, with pre-defined or ‘a priori’ initial codes being based on either the theoretical position of the research or specific questions which are being asked. These codes may be modified, or added to in a hierarchical manner (i.e. through sub-codes) as data are applied to the template, while also allowing for a quantification element. This makes it well suited to the current study, with its specific questions, while collecting subsidiary data provided by respondents around their experiences of trial participation was also integrated. The initial template was framed around the research questions and the existing literature (see Box 1).

• Interviews. A semi-structured interview schedule was developed, based on the aims of the research and the questionnaires found in the literature review (Behrendt et al., 2011; Locock and Smith, 2011) in order to expand on written comments and seek implications for research practice. Letters of invitation to be interviewed were sent to all control group participants and positive replies were received from 71 individuals (63 female: 8 male). It is known that men tend to use health services less frequently than women (Noone and Stephens, 2008), and are also outnumbered by women in choirs (Clift and Hancox, 2010) and so their views on participation in this health promotion research were considered important. In order to counter the preponderance of women among positive responders, our final interview sample included all the responding men in the control arm (n = 8) and a similar number of women (n = 11) drawn randomly and stratified across groups. Information sheets and letters of invitation were sent, outlining this phase of the research, together with consent forms. Of these, 12 positive responses and signed consent forms were returned and these individuals were contacted by phone by one of two researchers. One of these respondents was not able to be contacted. The rest (6 f, 5 m) were interviewed either face-to-face (n = 9) or by telephone (n = 2) between June and September 2011 and interviews were recorded with consent.

• Further written comments (follow-up questionnaire 2). Participants were again invited to comment at 6 months in order to ascertain whether any experiences had changed over time.

See Supplementary data, Appendix for a worked example of the analytical procedure.

FINDINGS

Questionnaire data (follow-up questionnaire 1)

A number (n = 20) of the total control group participants commenting (n = 71) expressed disappointment and for a minority this was very marked:

‘Because I was not chosen to participate in singing activity my self-worth feels zero … my confidence has taken quite a knock. This would have been such a confidence-booster for me. I felt used as a guinea pig’. (Female, age 68).

As a research team we were concerned at this level of disappointment, but also the frequent use of words like ‘chosen’, which cast doubt on the level of understanding on the part of participants.

Interview data

Motivation

Although a number of participants gave more than one reason for participating, the most frequent motivator was the singing, with 9 of the 11 respondents referring to this. Five individuals talked about how they had sung in the past, recalling childhood memories or more recent experiences of joining choirs which had been discontinued. Our publicity had flagged up the topic under the research title with the question: ‘is singing good for your health?’ and a subsequent brief description of the intervention/control nature of the project. It is clear that some participants had, consciously or not, misinterpreted this:

Other reasons given for responding included the fact that this offered something to do (something new, something interesting, meeting a need post-retirement or because individuals had been looking for a suitable activity—five responses), while three people mentioned the social element, in terms of joining in or meeting others. Although these reasons did not mention singing, there was an implicit assumption that respondents would be in an intervention group. The two people who participated because of persuasion by a husband or wife, and the two who mentioned the research itself (interest in findings; help for people in future), rather than the singing were clearly in a minority.

Information provision

The question about randomization was broadly consistent across respondents. Only one individual stated that she did not understand the notion of randomization, but later talked about the element of ‘chance’ in the allocation process, somewhat contradicting this and suggesting there was a grasp of the concept. It may be that she was referring here to the exact (computer-generated) method employed by the researchers. Most others appeared clear about both how and why randomization had taken place, talking in terms of it being ‘fair’ and that allocation being down to ‘chance’, ‘luck of the draw’, ‘50:50′ or ‘you win some, you lose some’:

‘It was just by picking names. The computer did it … . so that there wasn't any bias or anything' (Female, age 63)

That seemed fair enough. Obviously we wanted to sing but someone's got to be the control and I quite understood and that was fine'. (Female, age 62)

One exception was a participant who, on the face of it, understood the explanation, but whose further response demonstrated some confusion:

There is clearly an indication here that this individual thought that the researchers had some control over who would be in which group and that this was based on an individual's characteristics.

Reaction to control group allocation

Only one individual appeared unconcerned about the allocated group:

‘When I heard I was in the control group I thought ‘OK’ and got on with the next thing. So it didn't bother me at all'. (Male, age 76)

The main reaction voiced to being allocated to the control group was disappointment, with this word being mentioned by five of the respondents and others using similar sentiments such as ‘felt peeved’ or reiterating the desire to be singing. Others, however, qualified the sense of disappointment in terms of their understanding of the trial requirements (two), their interest in the research irrespective of allocation (one) or because of the promise made at the start that all participants would subsequently have the opportunity to sing (six):

‘I think we knew right from the start that we would be invited to sing eventually and so I always knew that was at the end of it, so that kept my interest and that was what I was aiming for really'. (Female, age 62)

Significantly, 9 of the 11 individuals interviewed subsequently joined a singing group, either for the extra sessions arranged post-research or the permanent groups which evolved from those.

Suggestions for improvement

Six respondents stated that they could not think of anything to make the control group experience better. Additional comments included expressing that they were kept informed by researchers or acknowledging that any greater involvement might have compromised the trial. The other five interviewees mentioned negative aspects of their experience in terms of ‘feeling in limbo’, ‘abandoned’ or missing contact or support and ‘going through without singing’. They suggested this could be helped by having a control group get-together, meeting, one-off singing session, more support or a newsletter, though some of the responses noted the potential effect this may have on the research:

‘I think it would have been nice if there had been one get-together of the control group, maybe half way through. On the other hand it may have given a bit of a skew, a bit of bias to the research, because it wouldn't be entirely a control group, because it was manipulated a little bit in the middle'. (Male, age 81)

The revised template, summarizing these responses, is illustrated in Box 2.

Questionnaire data (follow-up questionnaire 2)

Timing of the final questionnaire for participants was 3 months after the singing intervention had been completed. Comments from those in the control group reflected the sentiments expressed by our interviewees with respect to their reaction to control group allocation and to motivation to participate in the research. Only 10 out of 69 respondents commented on their disappointment at not being in the singing group (compared with 20 out of 70 in the previous questionnaire). Five of these also qualified their answer, in similar ways to the interviewees, finding some compensation to counter the disappointment. More importantly, a number commented favourably on their participation in the research in terms of: hoping to join in the continuing sessions (five), expressing interest, enjoyment or being pleased to have taken part (eight), joining another singing group (three), enjoying the taster session (two), expressing interest in the project and helping us (two) and finding it helpful to express feelings on paper (one). These later written comments were thus very much more positive than those in the earlier questionnaire.

DISCUSSION

Because an arts intervention, which appeals to the senses, is likely to elicit some emotional or psychological response, it is, perhaps, not surprising that the existence of treatment preferences, noted in the literature, was found to be such a strong motivator of trial participation in this research. Most of the work reviewed on preferences has been conducted in the context of unblinded clinical trials (comparing new vs. usual treatment) or testing interventions designed to promote a change in health behaviour. In such scenarios, participants are normally motivated by a need to improve their health, though opting for a particular treatment is rarely compatible with the researchers' position of equipoise. In our singing-based research intervention, participants were relatively healthy volunteers, whose preferences were based on what were perceived as ‘value-added’ activities, something which they had perhaps enjoyed in the past, or which held out a promise of social or affective benefit. As such, it may be that arts interventions should be seen as warranting specific considerations within the broader area of health promotion. However, findings also echoed the literature in the minority who saw trial participation as an altruistic activity, which might benefit others in the future, making the recommendation of Robinson et al. (Robinson et al., 2005), that this aspect should be emphasized in information leaflets, pertinent here.

The reaction of disappointment by research participants to control group or non-preferred treatment allocation is supported in much of the literature (Hart, 2001; Corrigan and Salzer, 2003; Howard and Thornicroft, 2006). The effect of this on sample recruitment, however, cannot be demonstrated since, like much of the existing research, interviews were only sought from those who actually agreed to participate in the trial. Also difficult to estimate is the effect of disappointment on the dropout rate, since only individuals who had completed all three questionnaires were interviewed. But, given that the overall retention rate was nearly 80%, it is unlikely that failure to allocate to treatment preference was a major disincentive, especially given that the feelings of disappointment appeared to reduce over time. It is perhaps of greater relevance to examine motivation for remaining in the trial. For 6 of the 11 interview participants, as well as those responding to the final questionnaire, a major influencing factor was the ‘carrot’ of having an opportunity to sing post-trial, which subsequently most of them took advantage of. Both of these findings have implications for how an arts and health trial may be designed and presented, with potential for sustainability in mind.

Adequacy of information giving in our research was generally positively evaluated. We found that understanding appeared generally good, though with some contradictory messages in some of the participant responses. This is supported in the literature (Heaven et al., 2006). Finally, suggestions for improvement revolved for our participants around counteracting any feelings of being abandoned or lacking support through being in the control group. Compensatory mechanisms suggested were largely based on contact with others in the same position, which would thus cater for the ‘attention-controlled’ element, but which, as some of the respondents realized, would add a degree of bias to the method. There is, however, useful information for researchers to be gleaned from these responses, and keeping research participants well informed of study progress, ensuring individuals are made aware that they are valued, etc. are important research tasks.

CONCLUSIONS

While there is evidence that issues of preference and disappointment are not uncommon in treatment trials where participants are aware of which arm they are in, the logic behind the lack of equipoise is usually based on a motivation to improve a clinical condition (often combined with the attraction of a ‘new’ treatment). Despite such preferences, the adoption of the RCT as a research method for such research may be justified both ethically and methodologically by virtue of its particular ability to measure treatment effects. Health promotion research into screening and prevention methods in ‘at-risk’ groups may largely be based on the same premises. However, the sense of disappointment which emerged in this research project, from a number of participants when allocated against preference to the control group, raises questions of the applicability of RCTs to this type of health promotion research, where the intervention is potentially seen as particularly attractive to a large proportion of the target population. We would argue that, in order to meet the need for convincing evidence for the effectiveness of innovative interventions for maintaining and promoting the health of older people, such approaches are indeed justifiable, for the same reasons as those applying to clinical trials. However, there is a need to maximize understanding and acceptability amongst potential participants in order to successfully conduct such research, as outlined below.

LIMITATIONS OF THE RESEARCH

• This was a small-scale study, based within a limited area of S.E. England, so its relevance to other populations and other arts-based interventions is unknown.

• We only interviewed individuals who had remained in the research to completion; therefore, we have no information from those whose objection to the possibility of control group placement was strong enough to deter them from entering the trial or remaining in it.

• While the template proved useful and allowed coding of participant responses at face value, there were occasions where contradictory statements meant that coding was based on inference (for example with the individual who claimed not to understand randomization but followed this up with evidence to suggest otherwise).

• Much of the data collection (i.e. the interviews and written comments from follow-up questionnaire 2) for this study was conducted retrospectively. It could be that, with hindsight, recall for participants was less accurate and so did not represent the reality of experience at the time of the original research.

RECOMMENDATIONS FOR FUTURE RESEARCH PRACTICE

From our findings, in order to counter potential feelings of resentful demoralization and minimize its effects on trial recruitment, retention and outcome we would suggest the following:

• It is important to stress the overall purpose of research, that is, to add to our knowledge, rather than to address individual needs. Encouraging a sense of altruism is likely to appeal to many people (Busby Grant et al., 2009).

• Ensure that, where possible, the preferred intervention is offered to all participants at the end of the research (a wait-list group). Where an intervention is likely to prove popular, plans for sustainability should be built into study protocols.

• It is important to ensure participants feel valued and not isolated. Frequent communication from the research team with updates and expressions of appreciation are appreciated.

• Where this is not possible, the collection of data on pre-randomization preferences, as in some studies (e.g. Leykin et al., 2007), may be useful in accounting for this factor in study analyses.

FUNDING

This work was supported by the National Institute for Health Research (NIHR) under its Research for Patient benefit (RfPB) programme (grant ref. PB-PG-0408-16038). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

References