- Split View
-
Views
-
Cite
Cite
F. Roncarolo, N. Lanati, M. Philpott, W. Drygas, J. Ruszkowska, R. Ireland, M. T. Tenconi, Process evaluation of European ‘Healthy Stadia’ program, Health Promotion International, Volume 30, Issue 4, December 2015, Pages 881–890, https://doi.org/10.1093/heapro/dau025
- Share Icon Share
Abstract
Healthy Stadia (HS) is a European public health pilot-program started in 2007 to support sports stadia in promoting the health of people who work and visit sports stadia, as well as inhabitants of the surrounding communities. The aim of this study is to describe the process evaluation of the program, from its beginning in July 2007 to December 2009, in order to assess the feasibility and sustainability of an HS network across Europe. The program involved nine associate partners involved in the coordination of activities at a local level, in the recruitment of stadia, in the development of specific program tasks and in the dissemination of the program at a national level. The activities of associate partners were evaluated through structured questionnaires administered every 6 months. The questionnaire response rate from associate partners was 77.8% for the first and third evaluations and 88.9% for the second and fourth evaluations. According to the evaluation's results, several good practices such as alcohol prevention policies and those supporting people with disabilities were implemented in stadia over the course of the program. Conversely, practices supporting mental health and green transport were generally not achieved. The implemented activities mainly involved staff and visitors. Lack of human and economic resources, especially toward the end of the program, was considered the principal challenge for program development. In conclusion, the process evaluation presented the feasibility of the HS program and the development of health promoting practices in sport stadia over time.
INTRODUCTION
For many years, the idea of using schools, universities, workplaces, hospitals, prisons and even cities to promote health has been developed by WHO and international agencies to improve community health by addressing lifestyle risk factors in a defined setting (Price and Tsouros, 1996; Chu et al., 2000; Kenzer, 2000; De Leeuw, 2001; St Leger, 2001; Dooris, 2002; Groene and Jorgensen, 2005; Baum et al., 2006; Donchin et al., 2006; Ginn, 2013). The influence of settings is central in health promotion, since health is determined not only by individual lifestyles and by health services, but also by economic, social,environmental and cultural factors (Dooris, 2012). The interactions between individual, environmental, cultural and social factors are dependent on the setting in question (Kokko et al., 2014); settings are therefore not only the place where interventions are implemented, but also are themselves important determinants of health (Dooris et al., 2007). In the last few years, the potential role of sports stadia as health promoting settings has been considered not only for possible benefits to local communities (Jackson et al., 2005; Dobbinson et al., 2006; Kokko et al., 2006, 2009), but also to help achieve the corporate objectives of the clubs and stadia involved through a possible increase in participation and membership (Eime et al., 2008). Stadia offer significant opportunities to reach large numbers of people, particularly young and middle aged men (Zhang et al., 1995; Hill and Christine, 2000; Dhillion, 2011), addressing a target population that is generally less engaged in health literacy and intervention programs than the general population (Robertson and Williamson, 2005; Pringle et al., 2011). In the UK, a wide range of projects to help sports stadia to promote public health and environmental initiatives targeting both fans and staff have been realized at local and national levels (Crabb and Ratinckx, 2005).
The Healthy Stadia (HS) European program was built upon the successes that Heart of Mersey (a regional health charity working to reduce the risk of cardiovascular diseases) achieved in Merseyside, UK, concerning the promotion of health in sport stadia. The main assumption of the HS program was that sports stadia can support health improvement and reduce health inequalities through interaction with local communities and with the people who use the stadia as a place to practice sport, for entertainment or work. The concept of HS is one that promotes the health of visitors, fans, players, employees and the surrounding community; they are places where people can go to have a healthy experience playing or watching sport. The main idea of the HS European Program was to create a network of stadia (HS network), able to share healthy practices to promote health among the visitors, the fans, the staff, the players and the local community. To assess the feasibility and the sustainability of a network involving stadia from numerous countries in Europe, the HS program and its process evaluation was setup between 2007 and late 2009. The HS program, hence, can be considered a pilot project that worked across eight European countries to promote the HS concept and to further develop a guidance framework for stadia to roll out healthy initiatives with the support of their local service providers. This paper presents the process evaluation of HS program that gave birth to HS network. The primary aim of the HS process evaluation was to evaluate the feasibility and the development of the program through investigation into the activities implemented, and to ensure that the program was on-track during its implementation.
To assess our evaluation process we considered four principles, as outlined by Issel in 2004 (Issel, 2004). First, the process evaluation should collect data about program delivery to determine whether pre-defined objectives are met. Secondly, process evaluation should provide operational information to allow the program to be replicated. Thirdly, it should respond and report on the requirements of funding agencies and fourthly, it should provide feedback to partners to enable midcourse corrections in the delivery of the program (Issel, 2004).
METHODS
The process evaluation of the HS program took place from September 2007 to December 2009. The evaluation was carried out through questionnaires sent out to associate and collaborative partners of HS program. Associate partners consisted of organizations from eight different countries: UK (2), Finland, Greece, Ireland, Italy, Latvia, Poland and Spain. Associate partners were involved in the program with specific tasks (coordination, evaluation, website creation, toolkit of best practices implementation, etc.), in addition to promoting the enrollment of stadia from their own countries. Collaborative partners (details of whom are given in the Acknowledgements section of this paper) were informed on the program's progress and supported the network through advocacy and communications, but did not have specific roles or duties inside the program. An audit of current practice was setup and sent to partners before the distribution of the evaluation questionnaires at the outset of the program (Drygas et al., 2013).
The questionnaires were distributed by e-mail every 6 months (September 2007, March 2008, September 2008 and March 2009) to associate partners, while the collaborative partners took part in just three evaluations (September 2007, September 2008 and March 2009) owing to the lesser role they played in the program. In the first evaluation, the same questionnaire was used for both associate and collaborative partners, while the third and fourth collaborative partners' evaluations were undertaken with a dedicated questionnaire.
The questionnaires had to be sent back to the evaluation team by email within 2 weeks; a reminder was sent twice (2 weeks and 4 weeks after the deadline) to non-responding partners.
Despite the dedicated questionnaire, the response rate of collaborative partners (i.e. organizations supporting the program but with no aligned budget) during process evaluation was low in all the evaluations. Just 3 partners out of 24 (12.5%) sent back the completed questionnaire during the first evaluation, despite reminders; the response rate was also low in the third and fourth evaluations (4 on 24 partners, 16.7%). Since the response rate was not sufficient enough to be representative of views offered by collaborative partners, the results of these partners are not shown in this paper, and only the associate partners' questionnaires will be considered.
The questionnaires were modeled through previous templates and experience of evaluating health setting programs and factored in the findings of the HS program's audit of current practices (Drygas et al., 2013). The questionnaires were designed in partnership with the evaluation work package lead organization (Department of Public Health and Neurosciences, Hygiene Section, University of Pavia) and Heart of Mersey's evaluation manager. On account of cultural differences among the countries where the HS program was implemented, a flexible questionnaire was developed that attempted to reflect a range of cultural backgrounds, taking into consideration differences in stadia ownership, operations and recognition of corporate and social responsibility issues. Periodic meetings (every 6 months) were organized with the associate partners and stakeholders to discuss and modify the questionnaire according to the suggestions of partners and needs of the program.
The questionnaire investigated the following indicators: An office excel database was created; the analysis of means and prevalence was performed through SPSS version 13. No statistical tests to compare partners’ answers were performed due to the low number of cases analyzed.
On-going activities: the activities implemented in the previous 6 months and the activities planned for the future were investigated through an open question.
Time and human resources devoted to the program: the number of hours dedicated to HS per month, and the number of people and the number of hours allocated to making new contacts, were investigated through closed questions.
Dissemination of the program: contact with other stadia, other organizations and governmental bodies were monitored through specific closed questions.
Population target reached: the number of stadia contacted or involved was investigated, as well as the type of sport played in the stadium.
Good practices implemented: the good practices implemented were investigated through three different questions—the first investigated the number of stadia that developed health promoting policies and practices; the second analyzed the type and the target of the practices (staff, visitor and local community); the third question investigated whether the good practice had been implemented owing to the HS program or other reasons (change of policies, other initiatives, etc.). The ‘set denominators’ that were used to describe examples of good practices were provided by the partners and investigated in the same question. The percentages concerning good practice targets and the role of HS were calculated by the number of partners and not on the number of stadia.
Media interest: partners were asked if the program had generated any media interest. In case of an affirmative answer, the respondents were invited to specify the kind of media (radio, television, paper media or Internet), the diffusion of the media (local and national), the number of columns for print media and the number of minutes of broadcasting for radio or television.
Limits and challenges of the program: the limits of the program were investigated through a semi-structured question.
RESULTS
Response rate to questionnaires
The results of the evaluation questionnaire are based on the answers of the nine associate partners involved in the program.
The respondents numbered seven (77.8%) in the first and in the third evaluation, and eight (88.9%) in the second and fourth evaluations. One partner did not respond to any of the four evaluations. Another partner missed the first and third evaluations.
Ongoing activities
In the first evaluation, the partners were mostly involved in promoting the program (86.7%), in developing their work packages (71.4%) and in engaging new sport stadia (70%). During the period of the final evaluation only 50% of the partners (four out of eight) were involved in developing work packages, 75% of partners were promoting the program, and 50% of them involved in engaging stadia.
Time and human resources devoted to the program
The proportion of partners who allocated >7 h a month in making contacts with relevant stakeholder organizations (i.e. Ministry of Health) increased over time: in the first evaluation this accounted for 57.1%, while they comprised 87.5% (seven out of eight respondents) in the fourth evaluation. Concerning the partners who allocated >7 h to the above activity, the mean time allocated was 28.7 h (±14.9) in Evaluation 1, 24.3 h (±25.1) in Evaluation 2, 14.3 h (±4.3) in Evaluation 3 and 18.3 (±7.8) in Evaluation 4. The mean number of people involved in making contacts with relevant organizations increased in associated partners’ units: there were 2.6 people per unit in the first evaluation, 2.9 in the second, 4.4 in the third and 4 people per unit in the last evaluation.
Dissemination of the program
Publicity for the program and dissemination of program deliverables increased steadily over time.
The interest generated by the program in other organizations (i.e. schools, non-governmental organizations, healthy networks) ranged between 50 and 62.5%. National and regional governmental bodies were contacted by each partner from the beginning to the end of the program. Contact with private stadia increased during the development of the program, from 71.4% of the first evaluation to 87.5% of the fourth evaluation, while contact with public sport stadia reached a peak in the third evaluation, when all the associate partners were involved in contacting stadia.
Population target reached
From the beginning of the program, the number of stadia involved in HS activities varied across the four different evaluations, specifically: 42 stadia in the first evaluation, 59 in the second, 16 in the third and 27 in the final evaluation. The type of sports played in stadia varied significantly, mainly depending on the country of origin. The type of sport represented the most was football, but basketball, rugby, volleyball, cricket and multi-sport stadia, along with minor sport stadia such as greyhound racing, also participated in the program.
Health promoting practices
The majority of stadia engaged by associate partners implemented health promoting practices with regard to smoking, alcohol, physical activity and disabilities; in the first evaluation only a few stadia implemented healthy transportation policies or activities supporting mental health (Table 1). The good practices initiated by stadia were generally targeted at staff and fans rather than the local community. The number of staff members involved in practices supporting people with disabilities, addressing alcohol policies and advocating healthier transport options increased overtime, while other practices (i.e. no-smoking policies or using professional players as health advocates) remained stable. Regarding fans and visitors, positive practices concerning alcohol and no-smoking policies were reported most often to have improved in partners' answers, while good practices in local communities, scarce at the time of first evaluation, steadily increased in promoting physical activity and supporting people with disabilities (Table 2). Many of the health promoting practices implemented by stadia increased over time as the program developed. 42.9% of respondent partners in the third evaluation said that some good practices were directly due to the development of the HS program, while the percentage increased to 50% in the fourth evaluation.
. | Evaluation 1 n = 17 (%) . | Evaluation 2 n = 33 (%) . | Evaluation 3 n = 25 (%) . | Evaluation 4 n = 26 (%) . |
---|---|---|---|---|
No-smoking policies | 14 (82.4) | 20 (60.6) | 17 (68) | 19 (73.1) |
Alcohol policies | 7 (41.2) | 15 (45.4) | 14 (56) | 20 (76.9) |
Healthy food policies | 7 (41.2) | 9 (27.3) | 7 (28) | 10 (38.5) |
Promoting physical activity | 11 (64.7) | 11 (33.3) | 13 (52) | 16 (61.5) |
Supporting mental health | 20 (11.8) | 2 (6.1) | 4 (16) | 4 (15.4) |
Supporting people with disabilities | 6 (35.3) | 8 (24.2) | 17 (68) | 19 (73.1) |
Healthy transport policies | 5 (29.4) | 2 (6.1) | 3 (12) | 6 (23.1) |
Sports players as healthy advocates | 8 (47.1) | 8 (24.2) | 9 (36) | 11 (42.3) |
. | Evaluation 1 n = 17 (%) . | Evaluation 2 n = 33 (%) . | Evaluation 3 n = 25 (%) . | Evaluation 4 n = 26 (%) . |
---|---|---|---|---|
No-smoking policies | 14 (82.4) | 20 (60.6) | 17 (68) | 19 (73.1) |
Alcohol policies | 7 (41.2) | 15 (45.4) | 14 (56) | 20 (76.9) |
Healthy food policies | 7 (41.2) | 9 (27.3) | 7 (28) | 10 (38.5) |
Promoting physical activity | 11 (64.7) | 11 (33.3) | 13 (52) | 16 (61.5) |
Supporting mental health | 20 (11.8) | 2 (6.1) | 4 (16) | 4 (15.4) |
Supporting people with disabilities | 6 (35.3) | 8 (24.2) | 17 (68) | 19 (73.1) |
Healthy transport policies | 5 (29.4) | 2 (6.1) | 3 (12) | 6 (23.1) |
Sports players as healthy advocates | 8 (47.1) | 8 (24.2) | 9 (36) | 11 (42.3) |
. | Evaluation 1 n = 17 (%) . | Evaluation 2 n = 33 (%) . | Evaluation 3 n = 25 (%) . | Evaluation 4 n = 26 (%) . |
---|---|---|---|---|
No-smoking policies | 14 (82.4) | 20 (60.6) | 17 (68) | 19 (73.1) |
Alcohol policies | 7 (41.2) | 15 (45.4) | 14 (56) | 20 (76.9) |
Healthy food policies | 7 (41.2) | 9 (27.3) | 7 (28) | 10 (38.5) |
Promoting physical activity | 11 (64.7) | 11 (33.3) | 13 (52) | 16 (61.5) |
Supporting mental health | 20 (11.8) | 2 (6.1) | 4 (16) | 4 (15.4) |
Supporting people with disabilities | 6 (35.3) | 8 (24.2) | 17 (68) | 19 (73.1) |
Healthy transport policies | 5 (29.4) | 2 (6.1) | 3 (12) | 6 (23.1) |
Sports players as healthy advocates | 8 (47.1) | 8 (24.2) | 9 (36) | 11 (42.3) |
. | Evaluation 1 n = 17 (%) . | Evaluation 2 n = 33 (%) . | Evaluation 3 n = 25 (%) . | Evaluation 4 n = 26 (%) . |
---|---|---|---|---|
No-smoking policies | 14 (82.4) | 20 (60.6) | 17 (68) | 19 (73.1) |
Alcohol policies | 7 (41.2) | 15 (45.4) | 14 (56) | 20 (76.9) |
Healthy food policies | 7 (41.2) | 9 (27.3) | 7 (28) | 10 (38.5) |
Promoting physical activity | 11 (64.7) | 11 (33.3) | 13 (52) | 16 (61.5) |
Supporting mental health | 20 (11.8) | 2 (6.1) | 4 (16) | 4 (15.4) |
Supporting people with disabilities | 6 (35.3) | 8 (24.2) | 17 (68) | 19 (73.1) |
Healthy transport policies | 5 (29.4) | 2 (6.1) | 3 (12) | 6 (23.1) |
Sports players as healthy advocates | 8 (47.1) | 8 (24.2) | 9 (36) | 11 (42.3) |
. | Evaluation 1 n = 7(%) . | Evaluation 2 n = 8 (%) . | Evaluation 3 n = 7(%) . | Evaluation 4 n = 8 (%) . |
---|---|---|---|---|
Staff | ||||
No-smoking policies | 6 (85.7) | 6 (75.0) | 5 (71.4) | 5 (62.5) |
Alcohol policies | 3 (42.9) | 4 (50.0) | 3 (42.9) | 8 (75.0) |
Healthy food policies | 2 (28.6) | 4 (50.0) | 3 (42.9) | 4 (50.0) |
Promoting physical activity | 4 (57.1) | 5 (62.5) | 5 (57.1) | 6 (75.0) |
Supporting mental health | 1 (14.3) | 3 (37.5) | 3 (42.9) | 4 (50.0) |
Supporting people with disabilities | 0 | 2 (25.0) | 3 (42.9) | 4 (50.0) |
Healthy transport policies | 3 (42.9) | 4 (50.0) | 3 (42.9) | 5 (62.5) |
Sports players as healthy advocates | 1 (14.3) | 1 (12.5) | 1 (14.3) | 1 (12.5) |
Visitors | ||||
No-smoking policies | 6 (85.7) | 6 (75.0) | 6 (85.7) | 8 (100) |
Alcohol policies | 3 (42.9) | 6 (75.0) | 6 (85.7) | 8 (100) |
Healthy food policies | 2 (28.6) | 5 (62.5) | 3 (42.9) | 5 (62.5) |
Promoting physical activity | 3 (42.9) | 6 (75.0) | 4 (57.1) | 5 (62.5) |
Supporting mental health | 0 | 3 (37.5) | 2 (28.6) | 4 (50.0) |
Supporting people with disabilities | 2 (28.6) | 3 (37.5) | 2 (28.6) | 5 (62.5) |
Healthy transport policies | 4 (57.1) | 4 (50.0) | 2 (28.6) | 5 (62.5) |
Sports players as healthy advocates | 2 (28.6) | 2 (25.0) | 2 (28.6) | 5 (62.5) |
Local community | ||||
No-smoking policies | 3 (42.9) | 1 (12.5) | 1 (14.3) | 2 (25.0) |
Alcohol policies | 1 (14.3) | 0 | 0 | 2 (25.0) |
Healthy food policies | 0 | 2 (25.0) | 1 (14.3) | 1 (12.5) |
Promoting physical activity | 3 (42.9) | 4 (50.0) | 4 (57.1) | 5 (62.5) |
Supporting mental health | 0 | 1 (12.5) | 0 | 2 (25.0) |
Supporting people with disabilities | 3 (42.9) | 3 (37.5) | 3 (42.9) | 5 (62.5) |
Healthy transport policies | 1 (14.3) | 3 (37.5) | 2 (28.6) | 4 (50.0) |
Sports players as healthy advocates | 2 (28.6) | 2 (25.0) | 3 (42.9) | 2 (25.0) |
. | Evaluation 1 n = 7(%) . | Evaluation 2 n = 8 (%) . | Evaluation 3 n = 7(%) . | Evaluation 4 n = 8 (%) . |
---|---|---|---|---|
Staff | ||||
No-smoking policies | 6 (85.7) | 6 (75.0) | 5 (71.4) | 5 (62.5) |
Alcohol policies | 3 (42.9) | 4 (50.0) | 3 (42.9) | 8 (75.0) |
Healthy food policies | 2 (28.6) | 4 (50.0) | 3 (42.9) | 4 (50.0) |
Promoting physical activity | 4 (57.1) | 5 (62.5) | 5 (57.1) | 6 (75.0) |
Supporting mental health | 1 (14.3) | 3 (37.5) | 3 (42.9) | 4 (50.0) |
Supporting people with disabilities | 0 | 2 (25.0) | 3 (42.9) | 4 (50.0) |
Healthy transport policies | 3 (42.9) | 4 (50.0) | 3 (42.9) | 5 (62.5) |
Sports players as healthy advocates | 1 (14.3) | 1 (12.5) | 1 (14.3) | 1 (12.5) |
Visitors | ||||
No-smoking policies | 6 (85.7) | 6 (75.0) | 6 (85.7) | 8 (100) |
Alcohol policies | 3 (42.9) | 6 (75.0) | 6 (85.7) | 8 (100) |
Healthy food policies | 2 (28.6) | 5 (62.5) | 3 (42.9) | 5 (62.5) |
Promoting physical activity | 3 (42.9) | 6 (75.0) | 4 (57.1) | 5 (62.5) |
Supporting mental health | 0 | 3 (37.5) | 2 (28.6) | 4 (50.0) |
Supporting people with disabilities | 2 (28.6) | 3 (37.5) | 2 (28.6) | 5 (62.5) |
Healthy transport policies | 4 (57.1) | 4 (50.0) | 2 (28.6) | 5 (62.5) |
Sports players as healthy advocates | 2 (28.6) | 2 (25.0) | 2 (28.6) | 5 (62.5) |
Local community | ||||
No-smoking policies | 3 (42.9) | 1 (12.5) | 1 (14.3) | 2 (25.0) |
Alcohol policies | 1 (14.3) | 0 | 0 | 2 (25.0) |
Healthy food policies | 0 | 2 (25.0) | 1 (14.3) | 1 (12.5) |
Promoting physical activity | 3 (42.9) | 4 (50.0) | 4 (57.1) | 5 (62.5) |
Supporting mental health | 0 | 1 (12.5) | 0 | 2 (25.0) |
Supporting people with disabilities | 3 (42.9) | 3 (37.5) | 3 (42.9) | 5 (62.5) |
Healthy transport policies | 1 (14.3) | 3 (37.5) | 2 (28.6) | 4 (50.0) |
Sports players as healthy advocates | 2 (28.6) | 2 (25.0) | 3 (42.9) | 2 (25.0) |
. | Evaluation 1 n = 7(%) . | Evaluation 2 n = 8 (%) . | Evaluation 3 n = 7(%) . | Evaluation 4 n = 8 (%) . |
---|---|---|---|---|
Staff | ||||
No-smoking policies | 6 (85.7) | 6 (75.0) | 5 (71.4) | 5 (62.5) |
Alcohol policies | 3 (42.9) | 4 (50.0) | 3 (42.9) | 8 (75.0) |
Healthy food policies | 2 (28.6) | 4 (50.0) | 3 (42.9) | 4 (50.0) |
Promoting physical activity | 4 (57.1) | 5 (62.5) | 5 (57.1) | 6 (75.0) |
Supporting mental health | 1 (14.3) | 3 (37.5) | 3 (42.9) | 4 (50.0) |
Supporting people with disabilities | 0 | 2 (25.0) | 3 (42.9) | 4 (50.0) |
Healthy transport policies | 3 (42.9) | 4 (50.0) | 3 (42.9) | 5 (62.5) |
Sports players as healthy advocates | 1 (14.3) | 1 (12.5) | 1 (14.3) | 1 (12.5) |
Visitors | ||||
No-smoking policies | 6 (85.7) | 6 (75.0) | 6 (85.7) | 8 (100) |
Alcohol policies | 3 (42.9) | 6 (75.0) | 6 (85.7) | 8 (100) |
Healthy food policies | 2 (28.6) | 5 (62.5) | 3 (42.9) | 5 (62.5) |
Promoting physical activity | 3 (42.9) | 6 (75.0) | 4 (57.1) | 5 (62.5) |
Supporting mental health | 0 | 3 (37.5) | 2 (28.6) | 4 (50.0) |
Supporting people with disabilities | 2 (28.6) | 3 (37.5) | 2 (28.6) | 5 (62.5) |
Healthy transport policies | 4 (57.1) | 4 (50.0) | 2 (28.6) | 5 (62.5) |
Sports players as healthy advocates | 2 (28.6) | 2 (25.0) | 2 (28.6) | 5 (62.5) |
Local community | ||||
No-smoking policies | 3 (42.9) | 1 (12.5) | 1 (14.3) | 2 (25.0) |
Alcohol policies | 1 (14.3) | 0 | 0 | 2 (25.0) |
Healthy food policies | 0 | 2 (25.0) | 1 (14.3) | 1 (12.5) |
Promoting physical activity | 3 (42.9) | 4 (50.0) | 4 (57.1) | 5 (62.5) |
Supporting mental health | 0 | 1 (12.5) | 0 | 2 (25.0) |
Supporting people with disabilities | 3 (42.9) | 3 (37.5) | 3 (42.9) | 5 (62.5) |
Healthy transport policies | 1 (14.3) | 3 (37.5) | 2 (28.6) | 4 (50.0) |
Sports players as healthy advocates | 2 (28.6) | 2 (25.0) | 3 (42.9) | 2 (25.0) |
. | Evaluation 1 n = 7(%) . | Evaluation 2 n = 8 (%) . | Evaluation 3 n = 7(%) . | Evaluation 4 n = 8 (%) . |
---|---|---|---|---|
Staff | ||||
No-smoking policies | 6 (85.7) | 6 (75.0) | 5 (71.4) | 5 (62.5) |
Alcohol policies | 3 (42.9) | 4 (50.0) | 3 (42.9) | 8 (75.0) |
Healthy food policies | 2 (28.6) | 4 (50.0) | 3 (42.9) | 4 (50.0) |
Promoting physical activity | 4 (57.1) | 5 (62.5) | 5 (57.1) | 6 (75.0) |
Supporting mental health | 1 (14.3) | 3 (37.5) | 3 (42.9) | 4 (50.0) |
Supporting people with disabilities | 0 | 2 (25.0) | 3 (42.9) | 4 (50.0) |
Healthy transport policies | 3 (42.9) | 4 (50.0) | 3 (42.9) | 5 (62.5) |
Sports players as healthy advocates | 1 (14.3) | 1 (12.5) | 1 (14.3) | 1 (12.5) |
Visitors | ||||
No-smoking policies | 6 (85.7) | 6 (75.0) | 6 (85.7) | 8 (100) |
Alcohol policies | 3 (42.9) | 6 (75.0) | 6 (85.7) | 8 (100) |
Healthy food policies | 2 (28.6) | 5 (62.5) | 3 (42.9) | 5 (62.5) |
Promoting physical activity | 3 (42.9) | 6 (75.0) | 4 (57.1) | 5 (62.5) |
Supporting mental health | 0 | 3 (37.5) | 2 (28.6) | 4 (50.0) |
Supporting people with disabilities | 2 (28.6) | 3 (37.5) | 2 (28.6) | 5 (62.5) |
Healthy transport policies | 4 (57.1) | 4 (50.0) | 2 (28.6) | 5 (62.5) |
Sports players as healthy advocates | 2 (28.6) | 2 (25.0) | 2 (28.6) | 5 (62.5) |
Local community | ||||
No-smoking policies | 3 (42.9) | 1 (12.5) | 1 (14.3) | 2 (25.0) |
Alcohol policies | 1 (14.3) | 0 | 0 | 2 (25.0) |
Healthy food policies | 0 | 2 (25.0) | 1 (14.3) | 1 (12.5) |
Promoting physical activity | 3 (42.9) | 4 (50.0) | 4 (57.1) | 5 (62.5) |
Supporting mental health | 0 | 1 (12.5) | 0 | 2 (25.0) |
Supporting people with disabilities | 3 (42.9) | 3 (37.5) | 3 (42.9) | 5 (62.5) |
Healthy transport policies | 1 (14.3) | 3 (37.5) | 2 (28.6) | 4 (50.0) |
Sports players as healthy advocates | 2 (28.6) | 2 (25.0) | 3 (42.9) | 2 (25.0) |
Media interest
Media interest generated by the program increased steadily over time: in the first evaluation only 42.9% of responding partners (three partners out of seven) supplied evidence of media interest in the program, while interest raised to 87.5% of respondents (seven partners out of eight) in the final evaluation. The most significant media channel in the diffusion of the program was print media. The interest of newspapers and magazines increased steadily over time, while television and radio, mostly local, reached a peak in the third evaluation (radio 60% and television 30%). In the third evaluation questionnaire, partners affirmed that a total of 25 columns were dedicated to the program in newspapers, while in the fourth evaluation partners declared that 10 pages and 7 column inches of newspaper copy were related to HS. Newspapers interested in the program were both local and national; radio and television media were mostly local.
Limits and challenges
The challenges and difficulties that associate partners encountered during the program changed over time, depending on the different phases of the program. In the first evaluation, just one partner considered the program budget to be a problem, but in the last questionnaire the lack of human or economic resources became the most important issue raised by partners (Table 3).
. | Evaluation 1 n = 7(%) . | Evaluation 2 n = 8 (%) . | Evaluation 3 n = 7(%) . | Evaluation 4 n = 8 (%) . |
---|---|---|---|---|
Cost | 1 (14.3) | 3 (37.5) | 4 (57.1) | 7 (87.5) |
Lack of human resources | 2 (28.6) | 4 (50.0) | 4 (57.1) | 7 (87.5) |
Lack of collaboration | 4 (57.1) | 6 (75.0) | 3 (42.9) | 4 (50.0) |
Communication problems | 2 (28.6) | 4 (50.0) | 3 (42.9) | 2 (25.0) |
Lack of organizational interest | 3 (42.9) | 3 (37.5) | 2 (28.6) | 3 (37.5) |
Lack of sport stadia interest | 4 (57.1) | 3 (37.5) | 3 (42.9) | 2 (25.0) |
. | Evaluation 1 n = 7(%) . | Evaluation 2 n = 8 (%) . | Evaluation 3 n = 7(%) . | Evaluation 4 n = 8 (%) . |
---|---|---|---|---|
Cost | 1 (14.3) | 3 (37.5) | 4 (57.1) | 7 (87.5) |
Lack of human resources | 2 (28.6) | 4 (50.0) | 4 (57.1) | 7 (87.5) |
Lack of collaboration | 4 (57.1) | 6 (75.0) | 3 (42.9) | 4 (50.0) |
Communication problems | 2 (28.6) | 4 (50.0) | 3 (42.9) | 2 (25.0) |
Lack of organizational interest | 3 (42.9) | 3 (37.5) | 2 (28.6) | 3 (37.5) |
Lack of sport stadia interest | 4 (57.1) | 3 (37.5) | 3 (42.9) | 2 (25.0) |
. | Evaluation 1 n = 7(%) . | Evaluation 2 n = 8 (%) . | Evaluation 3 n = 7(%) . | Evaluation 4 n = 8 (%) . |
---|---|---|---|---|
Cost | 1 (14.3) | 3 (37.5) | 4 (57.1) | 7 (87.5) |
Lack of human resources | 2 (28.6) | 4 (50.0) | 4 (57.1) | 7 (87.5) |
Lack of collaboration | 4 (57.1) | 6 (75.0) | 3 (42.9) | 4 (50.0) |
Communication problems | 2 (28.6) | 4 (50.0) | 3 (42.9) | 2 (25.0) |
Lack of organizational interest | 3 (42.9) | 3 (37.5) | 2 (28.6) | 3 (37.5) |
Lack of sport stadia interest | 4 (57.1) | 3 (37.5) | 3 (42.9) | 2 (25.0) |
. | Evaluation 1 n = 7(%) . | Evaluation 2 n = 8 (%) . | Evaluation 3 n = 7(%) . | Evaluation 4 n = 8 (%) . |
---|---|---|---|---|
Cost | 1 (14.3) | 3 (37.5) | 4 (57.1) | 7 (87.5) |
Lack of human resources | 2 (28.6) | 4 (50.0) | 4 (57.1) | 7 (87.5) |
Lack of collaboration | 4 (57.1) | 6 (75.0) | 3 (42.9) | 4 (50.0) |
Communication problems | 2 (28.6) | 4 (50.0) | 3 (42.9) | 2 (25.0) |
Lack of organizational interest | 3 (42.9) | 3 (37.5) | 2 (28.6) | 3 (37.5) |
Lack of sport stadia interest | 4 (57.1) | 3 (37.5) | 3 (42.9) | 2 (25.0) |
Further activities and developments
One of the outputs produced by HS was the European ‘Toolkit’ of best practice. The toolkit was developed as a hard copy product in nine European languages and in a digital format. A set of guidelines within the toolkit were aimed at stadia management and intermediary agencies (health services, local authorities, governing bodies of sport, etc.). The guidelines contained a step-by-step plan providing users with a walk-through of the basic steps needed to implement HS initiatives, such as putting an action plan together, locating partners, evaluating activities and the mainstreaming of healthy initiatives. It was supported by many detailed case studies drawn together from stadia participating in the program, grouped together under the three intervention themes of lifestyle, social and environmental.
At the end of the program, on 21 September 2009, a European Healthy Stadia Conference was held in Liverpool, UK, and officially launched the European Healthy Stadia Network that has carried on the preparatory work undertaken by the program. The website—www.healthystadia.eu—was developed for the purposes of the future European Healthy Stadia Network by a collaboration of partners in Spain and a sub-contracted web design company.
DISCUSSION
The results of HS process evaluation showed its feasibility through the goals reached during the established timetable.
The visibility of HS and mass media involvement in the program increased over time. Diffusion of the program through media involvement helped to obtain wider dissemination of program aims and outputs through stakeholder organizations, and in communities local to stadia. The conference held in Liverpool highlighted the favorable position of sports stadia in developing initiatives aimed at influencing healthier eating, increasing exercise, smoking cessation and men's health. Furthermore, it showed the feasibility of collaborations with local agencies involved in health, urban regeneration, transport and food and drink supplies.
The process evaluation of the HS program presents certain similarities to the evaluation adopted by the Healthy Cities network. Collaboration, participation and partnership, key principles adopted by the Healthy Cities network (Curtice et al., 2001; Lipp et al., 2013), are also nodal points in the HS program. Partnership, in particular, has to be considered at two levels for HS: at the organizational level including the steering group of associate partners, and at the local/national level with the involvement of governmental bodies and community organizations. To avoid misunderstanding among steering group partners and in the interpretation of questionnaires, our process evaluation took into consideration reported weak points of the Healthy Cities process. The differences between individual partners' settings and their working strategies in promoting health, a critical point in the Healthy Cities network (Curtice et al., 2001), were thoroughly analyzed before drafting the first evaluation questionnaire to ensure the questionnaire fitted all relevant settings. In addition, we paid attention to the individuality of a single country's needs, conscious that health policies, collaborative planning and the implementation of programs also depend not only on local priorities, but on the views and interests of stakeholders, and the context of local communities (Guba and Lincoln, 1989). Collaborative partnerships should ensure systematic sharing of their program's progress, results and weaknesses, but this is not always easy to achieve (Roussos and Fawcett, 2000).
HS process evaluations were consistently shared in associate partners' meetings; feedback and critical reflections were also used to improve the quality of the evaluation and the implementation of the program. Self-reflection and continuous appraisal of other partners’ points of view were helpful in improving the reliability of the evaluation (Genat, 2009).
In our program, associate partners of the project who were also involved in the practical activities of the program implemented the HS process evaluation internally. Concerning this point, it is debatable as to who should be responsible for a program evaluation: an internal evaluation conducted by participants could be accused of being insufficiently critical or objective, but on the other hand an internal evaluation can result in evaluation questions and methods that are more appropriate to the practice under scrutiny (Kennedy, 1995). The process evaluation of the HS program was always included in corporate decisions in order to allow for improvements and ideas that could be useful to the development of the program. Similar to the Healthy Cities Evaluation (Curtice et al., 2001), the HS process evaluation was organized in a way that moves away from rigid scientific methodologies to a more interpretative process with an emphasis on learning and sharing. This dynamic approach and the changes introduced within the program (and consequently in the evaluation) could allow for limitations in interpreting the results of community development approaches (Cruikshank, 1994). For example, the decrease in the number of stadia involved in the third and fourth evaluations could be interpreted as a drop in the number of stadia participating in the program. However, in reality, this decline can be traced to the splitting of the relevant question into two parts in the third and fourth evaluations: while in the first and the second evaluation the question measured ‘involved stadia’, the same question in the two subsequent evaluations asked to specify if the stadia were merely contacted or actively enrolled in the program. This specification was needed to investigate more clearly the role of stadia in the program. A second limitation is the low comparability of the results on health promoting practices between the first and second evaluations and the last two, owing to a difference in the denominator that was used: in the first two questionnaires we considered the number of practices in the stadia involved in the program, but from the third questionnaire onwards, to have a clearer overview on practices, we asked partners to indicate how many practices each individual stadium was involved in (the denominator of the ratio).
Finally, although associate partners accomplished their activity and reporting duties, a response rate of 100% was never reached. This was a consequence of personnel rearrangements within partner organizations, which in one case saw the dissolution of one partner organization toward the end of the project.
The limitations of our study described above clearly affected overall results and certain conclusions. Owing to both partner and survey changes over the course of the project, there are a number of questionnaire variables that can only be directly compared between the first two questionnaires and the final two questionnaires. This limits our conclusions, as we can neither make concrete assertions about the whole program development, nor about the number of stadia involved or number of health promotion practices developed. Given that the methods used in our research allowed only a descriptive analysis of the situation, these limitations should be taken into account in the analysis of the main results.
There is an overwhelming impetus to produce evidence for the effectiveness of health promotion and public health interventions (De Leeuw, 2009). However, evaluations demonstrating outcomes over and above predicted levels of achievement that are part of policy-level interventions are hard to achieve. This is mainly because intervention strategies that influence policy and only indirectly stimulate improvements in people's behavior (and consequently in health) are not always easy to demonstrate (Jackson et al., 2005; Clark and McLeroy, 1998). Moreover, health promotion interventions often involve actors from a variety of backgrounds and with different aims and objectives, which could mean that projects underpinned by pre-determined measured do not necessarily respond to the different requests of the actors involved in the study (Polland et al., 2001; Levy et al., 2003; Kegler et al., 2011). Despite the different competencies of the partners (sports clubs, academic, governmental organizations, NGOs, etc.), the process evaluation of the HS program demonstrated the feasibility and sustainability of the program and its further implementation. The results achieved in this process evaluation showed that sports stadia can represent a useful setting for health promotion interventions, while in some cases the uniqueness of using sport settings to engage target groups has also been recognized (Gray et al., 2013).
Since many health promotion settings (school, university, workplace, stadia and sport clubs) are at the same level, people may move from one setting to another many times per day (Kokko et al., 2014). The health promotion messages would be reinforced if health promoting networks including those representing moment of leisure and relaxation, where subjects might be more receptive to changes (Pringle et al., 2011), would collaborate (Kokko et al., 2014).
To substantiate the outcomes and influence of the HS program further, we propose that a follow-up analysis should be carried out to evaluate the effectiveness of the initiatives that were developed through the program. This outcome evaluation, although still at ‘submolecular’ level (De Leeuw, 2012), could be a first step forward toward a more global evaluation concerning process outcomes and impact of the network launched after the program finished (De Leeuw, 2012) and would also help to understand the interactions and synergies between different settings (Dooris, 2006).
CONCLUSION
The HS program included counties from north, east, west and south of Europe; it was useful for understanding how to implement an international network of stadia to promote health. A comparison based on analyses of appropriate health indicators of the communities where stadia are part of the European Healthy Stadia Network, alongside communities that are not involved in the network, could allow for an evaluation the efficacy of the HS program for health promotion and further policy work.
FUNDING
This work was supported by the European Union in the framework of the EU Public Health Program.
ACKNOWLEDGMENTS
The authors acknowledge the contributions and support made by all the partners who took part in this project. Associate partners: Heart of Mersey (England); University of Pavia (Italy); Medical University, Lodz (Poland); International Sports Projects Association, Oravais (Finland); Andalusian Centre for Sports Medicine (Spain); University of East London (England); Galway Healthy Cities (Ireland); Municipal Enterprise for Social Welfare, Trikala (Greece); State Agency, Public Health Agency (Latvia).
Collaborative partners: Tbilisi State Medical University (Georgia); European Public Health Alliance (Belgium); Union of European Football Associations (Belgium); Eurohealthnet (Belgium); Volos Municipal Enterprise for Urban Sites (Greece); Municipality of Sykies (Greece); Municipality of Rotterdam (the Netherlands); City of Solna (Sweden); Global Research and Financing (Israel); Sport England (UK); Bremen Institute for Prevention Research and Social Medicine (BIPS) (Germany); Ghent AHS, College of Higher Education (Belgium); National Centre of Preventive Medicine of Russian Ministry of Health and Social Protection (Russia); Department of Applied Health Sciences at the University of Szeged (Hungary); Yeniden Health and Education Society (Turkey); Sirius (Serbia); National University of Ireland (Ireland); Community Health Care Centre Ljubjana (Slovenia); Association for the Scientific Study into Health Ageing (AECES) (Spain); Sofia Municipality (Bulgaria); Public Health Institute, Madrid Health, Madrid City Council (Spain); Sports Area of the Provincial Council of Barcelona (Spain); Romtens Foundation (Romania); European Men's Health Forum (Belgium); City of Uppsala, Executive Office (Sweden).