- Split View
-
Views
-
Cite
Cite
Nina Weymann, Martin Härter, Jörg Dirmaier, Quality of online information on type 2 diabetes: a cross-sectional study, Health Promotion International, Volume 30, Issue 4, December 2015, Pages 821–831, https://doi.org/10.1093/heapro/dau019
- Share Icon Share
Abstract
Evidence-based health information is a prerequisite for patients with type 2 diabetes to engage in self-management and to make informed medical decisions. The Internet is an important source of health information. In the present study, we systematically assessed formal quality, quality of decision support and usability of German and English language websites on type 2 diabetes. The search term ‘type 2 diabetes’ was entered in the two most popular search engines. Descriptive data on website quality are presented. Additionally, associations between website quality and affiliation (commercial vs. non-commercial), presence of the HON code quality seal and website traffic were explored. Forty-six websites were included. Most websites provided basic information necessary for decision-making, while only one website also provided decision support. Websites with a HON code had significantly better formal quality than websites without HON code. We found a highly significant correlation between usability and website traffic and a significant correlation between formal quality and website traffic. Most websites do not provide sufficient information to support patients in medical decision-making. Our finding that usability and website traffic are tightly associated is consistent with previous research indicating that design is the most important cue for users assessing website credibility.
PRIMARY OBJECTIVES
Diabetes and its sequelae are a major and still growing cause of morbidity and mortality in many countries. In the last three decades, the number of people living with diabetes more than doubled (Danaei et al., 2011). In the year 2000, it was estimated that 2.8% of the world's population live with diabetes. Projections for the year 2030 expect the prevalence to rise to 4.4% (Wild et al., 2004). Type 2 diabetes accounts for 90–95% of diabetes cases (American Diabetes Association, 2012).
Patients, practitioners, scientists and politicians have called for more patient involvement in the making of medical decisions as well as in the management of their diseases in order to reduce the burden on patients and society. Patient involvement in diabetes management (especially in watching one's diet and exercising regularly) was shown to reduce fasting blood glucose levels, A1C and the need for diabetes medication (Deakin et al., 2005). Two main aspects of patient involvement are self-management and shared decision-making (SDM). Self-management means that the patient successfully copes with the challenges of living with and treating diabetes. This may involve that the patient autonomously sets goals, identifies barriers and challenges and monitors his or her health (Wagner et al., 1996, 2001). SDM means ‘(1) that at least two participants—physician and patient be involved; (2) that both parties share information; (3) that both parties take steps to build a consensus about the preferred treatment; and (4) that an agreement is reached on the treatment to implement’ (Charles et al., 1997). Both for SDM and for self-management patients need to be informed about their disease, its course and the treatment options at hand, including their advantages and disadvantages.
In times of rapidly growing Internet adoption and limited resources in the health care system, many patients turn to the World Wide Web in search for diabetes information: The Pew Internet and American Life Project found that 51% of people living with one or more chronic conditions search the Internet for health information (Fox and Purcell, 2010). Similarly, Wagner and colleagues (Wagner et al., 2004) found that 52% of patients with diabetes used the Internet for health information. For the chronically ill, the Internet offers the opportunity to gather health information at their own pace and become experts of their disease (Kerr et al., 2006). According to Shuyler and Knight (Shuyler and Knight, 2003), most visitors search websites for information about a condition, treatment or symptoms, or for advice about symptoms or treatment. According to Fox and Duggan (Fox and Duggan, 2013), the self-management topic that is searched most by people living with diabetes is weight loss/weight control. Topics that people with diabetes search significantly more often than people with no diabetes are information about drugs they saw advertised and medical test results. The information they find online have an impact on coping, health behaviour, decisions on whether a health professional is consulted or not, and choice of treatment and provider (Wagner et al., 2004; Fox and Purcell, 2010).
However, the correctness, usefulness and readability of online health information vary greatly (Eysenbach et al., 2002; Scullard et al., 2010; Patel and Cobourne, 2011; Buultjens et al., 2012). Only a minority of websites offer enough information to support patients in medical decision-making (Smart and Burling, 2001). For users, it can be difficult to distinguish high-quality from low-quality online health information: In a study by Eysenbach and Köhler (Eysenbach and Köhler, 2002), participants claimed that the source, a professional design, a scientific or official touch, the language used and ease of use were the main criteria when assessing the credibility of a website. When these participants were observed while searching the web for health information, they did not check the source of the information. These findings are consistent with other studies reporting that design/look are the most important clues for users when assessing website credibility and quality (Stanford et al., 2002; Sillence et al., 2007).
The latest evaluations of the quality and content of online patient information on diabetes date from the mid-2000s (Thakurdesai et al., 2004; Bull et al., 2005; van Esch et al., 2006). Two recent studies researched diabetes online social networks and also found variable quality (Greene et al., 2011; Weitzman et al., 2011). To our knowledge, there is no current study systematically assessing the quality of online patient information on type 2 diabetes in Germany, and there is no study assessing the content quality of web-based information on type 2 diabetes with a focus on the information needed for informed medical decision-making. Therefore, in our cross-sectional study, we aimed to systematically assess formal quality, quality of decision support and usability of web-based information on type 2 diabetes, using a set of instruments based on previously published work researching quality of online information (Abbott, 2000; Griffiths and Christensen, 2000; Khazaal, 2008a,b). We further explore whether website quality is associated with website affiliation, presence of HON code, a quality seal for health websites and higher website traffic. We hope that our work can make a small contribution to the elaboration and implementation of core criteria for online patient information. There are labels, codes and criteria that overlap in many aspects. Still, we believe that most patients or even health care professionals are not familiar with them, and that the transfer from science to practice is still in its infancy.
RESEARCH DESIGN
In January 2012, we systematically searched for English and German language online information on type 2 diabetes. Rating of websites was performed between February and December 2012. The present study thus is a cross-sectional study reflecting the situation in 2012.
Methods and procedures
We were interested in a sample that is representative for what the average user finds when performing an online search for diabetes information. Consequently, we aimed at evaluating websites across types (commercial, not-for-profit organization, government, etc.) that are accessible by anyone with Internet access.
The two most popular search engines of the year 2012 were determined using alexa.com which is, by the provider's own admission, ‘the leading provider of free, global web metrics’ (www.alexa.com). What we know about how ‘most popular’ is operationalized by alexa.com is the following: ‘The global traffic rank is a measure of how a website is doing relative to all other sites on the web over the past 3 months. The rank is calculated using a combination of the estimated average daily unique visitors to the site and the estimated number of pageviews on the site over the past 3 months. The site with the highest combination of unique visitors and pageviews is ranked #1.’ (http://www.alexa.com/help/traffic-learn-more). In the ‘top sites’ tab, the top 500 sites on the web are displayed. The first two search engines that are on the list are www.google.com (#1) and yahoo.com (#4).
The search term ‘type 2 diabetes’ was entered in English and German language in Google.de (German)/Google.com (English), and Yahoo.de (German)/Yahoo.com (English). Our goal was to imitate searches performed by lay users looking for health information. For this reason, each search engine was searched only once. All searches were conducted by the same person who was based in Germany. The search was conducted with the single search term ‘type 2 diabetes’ because according to Eysenbach and Köhler (Eysenbach and Köhler, 2002), 65% of the searches performed by users looking for health information consist of only one phrase. The same authors report that users primarily search among the first 20 hits of a keyword search. In order to imitate this user behaviour, we included the first 20 hits from each search engine. Hits that contained no information on type 2 diabetes, that were accessible only via login or requiring user fees, that were not accessible due to other (e.g., technical) reasons, that were duplicates of websites we had already included or that were not websites but YouTube videos, links, books, articles or online forums were excluded from the sample. Website affiliations were divided into seven categories: non-profit organization, commercial, governmental, university, private, insurance company and other according to the declaration of affiliation. If no affiliation was declared, we performed a WHOIS request on www.denic.de for German websites and on http://domains.whois.com/ for English websites. A WHOIS search provides information regarding a domain name, such as domain ownership, where and when registered, expiration date and the nameservers assigned to the domain. The information on domain ownership helped us determine the website's affiliation. In order to find out how much traffic each website has, we assessed the rank of the included websites on www.alexa.com.
Measures
Given the lack of a standard instrument or procedure that is applied for the evaluation of health websites, we turned to the literature and found that over the years a core set of quality criteria has been established: Around the turn of the century, three articles were published reviewing the quality criteria that had been applied in the literature on quality of health web sites so far. All three reviews state that page aesthetics/design/ease of use and content/currency and accuracy of information are major quality domains that are usually evaluated. There is also agreement that disclosure of authorship/sponsors are key criteria. These three quality domains are also repeatedly found in later publications on the quality of health websites, including a core set of criteria operationalizing them (Pealer and Dorman, 1997; Silberg et al., 1997; Kim et al., 1999; Abbott, 2000; Griffiths and Christensen, 2000; Eysenbach et al., 2002; Kisely et al., 2003; Murphy et al., 2004; Khazaal, 2008a, b). We reviewed this body of literature, extracted the core criteria and erased duplications. Additionally, the DISCERN instrument (www.discern.org.uk, 2012), the International Patient Decision Aid Standards Patient Decision Aid Checklist for Users (IPDAS, 2005) and the German Action forum for health information systems (Aktionsforum Gesundheitsinformationssystem (Afgis), 2012) were used as sources.
The DISCERN Project was originally funded by The British Library and the NHS Executive Research & Development Program. The DISCERN instrument can be used by consumers or producers of information on treatment choices. The International Patient Decision Aid Standards (IPDAS) Collaboration is an international group of researchers, practitioners and stakeholders aiming to establish an international standard for the evaluation of patient decision aids. The IPDAS Patient Decision Aid Checklist for Users is a summary of these standards. Afgis is a network that was originally funded by the German Federal Ministry of Health aiming at improving and assuring quality of health information. There is considerable overlap between these publications and instruments so we extracted a list of core criteria. Table 1 shows the quality criteria that were applied. They can be subsumed under the three quality domains formal quality criteria, usability and quality of decision support. In the following paragraphs, the contributions of all relevant sources are discussed by table column/quality domain. Items will be named according to quality domain (fq, formal quality; u, usability; qds, quality of decision support) and numeration as displayed in Table 1. For example, the first item in the column ‘formal quality’, ‘authors are identified’, will be named ‘fq1’.
Formal quality (fq) (Silberg et al., 1997; Griffiths and Christensen, 2000; Murphy et al., 2004; Khazaal, 2008a, b; Afgis, 2012; IPDAS, 2005; DISCERN, 2012) . | Usability (u) (Abbott, 2000; Kisely et al., 2003; Khazaal, 2008a, b; Afgis, 2012) . | Quality of decision support (qds) (DISCERN, 2012; IPDAS, 2005) . |
---|---|---|
|
|
|
Formal quality (fq) (Silberg et al., 1997; Griffiths and Christensen, 2000; Murphy et al., 2004; Khazaal, 2008a, b; Afgis, 2012; IPDAS, 2005; DISCERN, 2012) . | Usability (u) (Abbott, 2000; Kisely et al., 2003; Khazaal, 2008a, b; Afgis, 2012) . | Quality of decision support (qds) (DISCERN, 2012; IPDAS, 2005) . |
---|---|---|
|
|
|
Formal quality (fq) (Silberg et al., 1997; Griffiths and Christensen, 2000; Murphy et al., 2004; Khazaal, 2008a, b; Afgis, 2012; IPDAS, 2005; DISCERN, 2012) . | Usability (u) (Abbott, 2000; Kisely et al., 2003; Khazaal, 2008a, b; Afgis, 2012) . | Quality of decision support (qds) (DISCERN, 2012; IPDAS, 2005) . |
---|---|---|
|
|
|
Formal quality (fq) (Silberg et al., 1997; Griffiths and Christensen, 2000; Murphy et al., 2004; Khazaal, 2008a, b; Afgis, 2012; IPDAS, 2005; DISCERN, 2012) . | Usability (u) (Abbott, 2000; Kisely et al., 2003; Khazaal, 2008a, b; Afgis, 2012) . | Quality of decision support (qds) (DISCERN, 2012; IPDAS, 2005) . |
---|---|---|
|
|
|
Formal quality
Silberg et al. (Silberg et al., 1997) contributed to items fq1–4 and fq6–12. Kim et al. (Kim et al., 1999) contributed to Griffiths and Christensen (Griffiths and Christensen, 2000) contributed to fq1–3,fq6–7, fq9–10 and fq12–13. Murphy et al. (Murphy et al., 2004) contributed to items fq1–4 and fq6–12. Khazaal (Khazaal, 2008a,b) contributed to fq1–4, fq6–10 and fq12. Afgis (Afgis, 2012) contributed to fq7, fq9–10 and fq12. IPDAS (IPDAS, 2005) contributed to fq 3, fq5, fq10, fq14 and fq16–18. DISCERN (DISCERN, 2012) contributed to fq 9, fq12 and fq18–20.
Usability
Abbott (Abbott, 2000) contributed to u1, u3–7 and u11. Kisely et al. (Kisely et al., 2003) contributed to u2 and u4–6. Khazaal (Khazaal, 2008a,b) contributed to u2, u4–6 and u8–12. Afgis (Afgis, 2012) contributed to u8.
Quality of decision support
DISCERN (DISCERN, 2012) contributed to qd4–7 and qd18. IPDAS (IPDAS, 2005) contributed to all qd items.
We additionally assessed the presence of a Health on the Net (HON) code (HONcode, 2012). The HON foundation is a non-profit, non-governmental organization based in Switzerland that promotes reliable online health information. According to the foundation's web site www.hon.ch, the HON code of conduct is the oldest and most common quality seal for online health information. In order to be certified, site owners have to apply for a review of their site. The medical professionals of the HON review committee check if the site adheres to all eight of the HONcode ethical principles (According to www.hon.ch: ‘Authority—Give qualifications of authors,’ ‘Complementarity—Information to sup-port, not replace,’ ‘Confidentiality—Respect the privacy of site users,’ ‘Attribution—Cite the sources and dates of medical information,’ ‘Justi-fiability—Justification of claims/balanced and objective claims,’ ‘Transparency—Accessibility, provide valid contact details,’ ‘Financial disclosure—Provide details of funding,’ ‘Advertising—Clearly distinguish advertising from editorial content).’ A certified site receives a biennial review, beginning 1 year after the initial certification, or following user alert.
The resulting coding sheet has 44 items. 43 are numeric items, 1 is an open-ended question (string variable). All numeric items were rated as present (1) or absent (0).
The included websites were rated by four members of the research team under NWs supervision (see Acknowledgements). A codebook was developed by NW and one of the coders (IE). It contained operationalization of the items and the way the coders should search for the information. For example, for the item ‘Sponsoring is disclosed,’ the tabs ‘about’, ‘about us’, ‘imprint’, ‘editorial policy’ on English web sites and ‘Impressum’, ‘über uns’ on German web sites were searched. If there was an on-site search engine, the search terms ‘sponsor’, ‘partner’ and ‘fund(ing)’ were entered. The item was only achieved if it was either clearly stated that there was no sponsoring or if sponsoring was explicitly disclosed. The item was not achieved if it was stated that there was sponsoring but without disclosing by whom. The coders were trained in coding according to the codebook using three web sites that were results of the above-mentioned searches but not among the first 20 hits. If there were disagreements, they were discussed under NWs supervision, and the respective section in the codebook was modified accordingly (e.g. clearer wording). Inter-rater reliability was assessed based on full codings (all items) of a random sample of 10 sites. The median of Cohen's κ was 0.6 (interquartile range Q3–Q1 = 0.6, min = −0.4, max = 1). This corresponds to a good inter-rater reliability (Wirtz and Caspar, 2002). Due to a lack of variation in the ratings of one or both raters Cohen's κ could not be calculated for 19 items. Consequently, absolute correspondence of ratings (percentage of items that were rated correspondingly by both raters) was calculated as an additional measure of inter-rater reliability. The median was 84.5% (SD = 24.3%, min = 0%, max = 100%).
Analyses
For descriptive analysis, sum scores for total quality and quality domains (formal quality, usability, quality of decision support) were calculated. All criteria were weighted equally as in previous publications (Griffiths and Christensen, 2000; Kisely et al., 2003). The criterion ‘Does it provide probabilities of outcomes in an unbiased and understandable way?’ was operationalized by eight items (e.g. ‘Does it use event rates specifying the population and time period?’). The mean of these eight items constituted the value for the criterion. For each sum score, the percentage of quality criteria met was calculated (e.g. if a website met 9 out of 43 quality criteria, the percentage was 20%). The quality dimension (formal quality, usability, quality of decision support) that was achieved most often (highest mean percentage reached by websites) and most achieved least often (lowest mean percentage reached by websites) were identified. For each dimension, the two quality criteria that were achieved most often (met by the highest percentage of websites) and the two that were achieved least often (met by the lowest percentage of websites) were identified.
Additionally, associations between aspects of website quality (percentage reached on one or more of the quality domains) presence of HON code, affiliation (commercial vs. non-commercial) and website traffic (rank on alexa.com) are explored. Due to different scale levels of the variables, separate analyses are applied: Associations between website quality and presence of HON code as well as associations between website quality and affiliation are tested with a Mann–Whitney U-test. Associations of website quality and website traffic are tested with a Spearman's rank correlation. All analyses are two-tailed. For all analyses, a significance level of p ≤ 0.05 was used. Data analysis was performed using PASW Statistics 18 (SPSS Inc., Chicago, IL, USA).
MAIN OUTCOME AND RESULTS
Of the 80 hits, 26 were excluded because they were duplicates of already included websites. Two were not accessible or not accessible without login. Three were YouTube videos or articles. Another three were excluded because they had no content on type 2 diabetes. In total, 46 websites were included in the study. Of these, 23 were in German and 23 were in English language. All sites had been set up between 1995 and 2011. Most of them (83%) had been last updated in 2011 or 2012. Table 2 shows the URLs of the included websites.
Descriptive analyses
Forty-three numeric quality criteria were rated, one was a string variable (‘If decision aids are provided, which ones?’) and was therefore not included in the calculation. Since all criteria were weighted equally, a maximum score of 43 could be reached for total quality. The highest score that was reached by two websites was 27 (62.8% of quality criteria met, see Table 3). The lowest score was 7 (16.3%, http://www.diabetessymptome.com/). The quality subdomain where the most criteria were met at mean (49.6%) was quality of decision support. The quality domain where the least criteria were met at mean (35.0%) was formal quality. There were two quality criteria that were met by all of the included websites. One was ‘Presence of headings or subheadings’ (usability), and the other ‘Does the website describe the health condition?’ (quality of decision support). The quality of decision support criterion ‘Does it provide probabilities of outcomes in an unbiased and understandable way?’ was met by none of the included websites. Table 3 shows the 10 websites with the highest scores.
URL . | Affiliation . | Percentage of quality criteria that were met . |
---|---|---|
http://www.netdoktor.de/Krankheiten/Diabetes/Wissen/Diabetes-mellitus-Typ-2-Zuckerk-119.html | Commercial | 62.8% |
http://www.medicinenet.com/diabetes_mellitus/article.htm | Commercial | 62.8% |
http://www.nlm.nih.gov/medlineplus/ency/article/000313.htm | Non-commercial | 60.5% |
http://www.versorgungsleitlinien.de/themen/diabetes2/dm2_therapie/pdf/nvl-t2d-therapie-kurz-1.1.pdf | Non-commercial | 58.1% |
http://www.dlife.com/diabetes/type-2 | Commercial | 58.1% |
http://www.patient.co.uk/health/Diabetes-Type-2.htm | Non-commercial | 55.8% |
http://www.emedicinehealth.com/diabetes/article_em.htm | Commercial | 55.8% |
http://de.wikipedia.org/wiki/Diabetes_mellitus | Commercial | 53.5% |
http://www.ncbi.nlm.nih.gov/pubmedhealth/PMH0001356/ | Non-commercial | 53.5% |
http://diabetes.about.com/ | Commercial | 53.5% |
URL . | Affiliation . | Percentage of quality criteria that were met . |
---|---|---|
http://www.netdoktor.de/Krankheiten/Diabetes/Wissen/Diabetes-mellitus-Typ-2-Zuckerk-119.html | Commercial | 62.8% |
http://www.medicinenet.com/diabetes_mellitus/article.htm | Commercial | 62.8% |
http://www.nlm.nih.gov/medlineplus/ency/article/000313.htm | Non-commercial | 60.5% |
http://www.versorgungsleitlinien.de/themen/diabetes2/dm2_therapie/pdf/nvl-t2d-therapie-kurz-1.1.pdf | Non-commercial | 58.1% |
http://www.dlife.com/diabetes/type-2 | Commercial | 58.1% |
http://www.patient.co.uk/health/Diabetes-Type-2.htm | Non-commercial | 55.8% |
http://www.emedicinehealth.com/diabetes/article_em.htm | Commercial | 55.8% |
http://de.wikipedia.org/wiki/Diabetes_mellitus | Commercial | 53.5% |
http://www.ncbi.nlm.nih.gov/pubmedhealth/PMH0001356/ | Non-commercial | 53.5% |
http://diabetes.about.com/ | Commercial | 53.5% |
URL . | Affiliation . | Percentage of quality criteria that were met . |
---|---|---|
http://www.netdoktor.de/Krankheiten/Diabetes/Wissen/Diabetes-mellitus-Typ-2-Zuckerk-119.html | Commercial | 62.8% |
http://www.medicinenet.com/diabetes_mellitus/article.htm | Commercial | 62.8% |
http://www.nlm.nih.gov/medlineplus/ency/article/000313.htm | Non-commercial | 60.5% |
http://www.versorgungsleitlinien.de/themen/diabetes2/dm2_therapie/pdf/nvl-t2d-therapie-kurz-1.1.pdf | Non-commercial | 58.1% |
http://www.dlife.com/diabetes/type-2 | Commercial | 58.1% |
http://www.patient.co.uk/health/Diabetes-Type-2.htm | Non-commercial | 55.8% |
http://www.emedicinehealth.com/diabetes/article_em.htm | Commercial | 55.8% |
http://de.wikipedia.org/wiki/Diabetes_mellitus | Commercial | 53.5% |
http://www.ncbi.nlm.nih.gov/pubmedhealth/PMH0001356/ | Non-commercial | 53.5% |
http://diabetes.about.com/ | Commercial | 53.5% |
URL . | Affiliation . | Percentage of quality criteria that were met . |
---|---|---|
http://www.netdoktor.de/Krankheiten/Diabetes/Wissen/Diabetes-mellitus-Typ-2-Zuckerk-119.html | Commercial | 62.8% |
http://www.medicinenet.com/diabetes_mellitus/article.htm | Commercial | 62.8% |
http://www.nlm.nih.gov/medlineplus/ency/article/000313.htm | Non-commercial | 60.5% |
http://www.versorgungsleitlinien.de/themen/diabetes2/dm2_therapie/pdf/nvl-t2d-therapie-kurz-1.1.pdf | Non-commercial | 58.1% |
http://www.dlife.com/diabetes/type-2 | Commercial | 58.1% |
http://www.patient.co.uk/health/Diabetes-Type-2.htm | Non-commercial | 55.8% |
http://www.emedicinehealth.com/diabetes/article_em.htm | Commercial | 55.8% |
http://de.wikipedia.org/wiki/Diabetes_mellitus | Commercial | 53.5% |
http://www.ncbi.nlm.nih.gov/pubmedhealth/PMH0001356/ | Non-commercial | 53.5% |
http://diabetes.about.com/ | Commercial | 53.5% |
Quality of decision support
In the quality domain ‘Quality of decision support’, the criteria that were achieved most often were ‘Does the website describe the health condition?’ (100%) and ‘Does it describe what would happen if no treatment is used (natural course of the condition)?’ (82.6%), the criteria that were achieved least often were ‘Does it provide support for shared decision-making?’ (2.5%) and ‘Does it provide probabilities of outcomes in an unbiased and understandable way?’ (0%). Thus, within the domain quality of decision support the whole range from the item that was achieved most often to the one that was achieved least often is covered. There are items on both extremes of the range but hardly any in the middle. Only one website (https://www.edgepark.com/information/education/diabetes/diabetes-information/type-2-diabetes-symptom) provided a decision aid. The website owner is a company selling medical supplies, and the decision aid was on the decision whether or not to switch to insulin pump therapy.
Usability
The usability criteria that were achieved most often were ‘Presence of headings or subheadings’ (100%) and ‘Document has a distinguishable header, body, footer’ (90.7%), the ones that were achieved least often were ‘Audio or video support,’ ‘Presence of diagrams’ (20.5% respectively) and ‘Satisfaction and knowledge evaluation questionnaires for users’ (13.6%).
Formal quality
The formal quality criteria that were achieved most often were ‘Ownership of the site is disclosed’ (87.5%) and ‘Relevant copyright information is noted’ (80.0%), the ones that were achieved least often were ‘Patients were involved in website development’ (4.9%) and ‘Date of the next update of the site has been specified’ (4.3%).
Associations of website quality and website characteristics
Associations of website quality and website affiliation:
Almost half (48.8%) of the sites had commercial funding (e.g. pharmaceutical industry). Non-commercial sites were a heterogeneous group. Most non-commercial sites were government sites (12.2%) or run by private persons (9.8%). There was no significant difference between sites with commercial or non-commercial affiliation regarding any of the quality domains.
Associations of website quality and presence of HON code
23.1% of the included websites had a HON code. Websites with a HON code had significantly better formal quality (Sig. = 0.001). There was no significant difference regarding usability or quality of decision support.
Associations of website quality and website traffic
The traffic on the included sites ranged from highly frequented sites (http://en.wikipedia.org/wiki/Diabetes_mellitus_type_2, Alexa.com rank 6) to much less frequented sites (http://www.diabetesmellitus.net/, rank 8461523). There was a highly significant correlation (r = −0.47, p = 0.002) between usability and website traffic and a significant (r = −0.38, p = 0.014) correlation between formal quality and Alexa.com rank. Negative correlations are due to lower numbers signifying higher ranks. We also found a highly significant correlation (r = 0.45, p = 0.002) between two quality domains usability and formal quality. There was no significant correlation between presence/quality of decision support and website traffic.
CONCLUSION
We assessed formal quality, usability and quality of decision support in English and German language websites on type 2 diabetes. Most websites met basic formal quality criteria such as disclosure of ownership and copyright information. Still, 12.5% of websites did not disclose ownership. 83% of the websites had been updated 2011 or 2012, but only 4.3% specified a date for the next update.
Hardly any sites declared that patients had been involved in website development, and only 13.6% had evaluation questionnaires for users. This lack of user involvement might be due to the effort that is necessary when different stakeholders such as patients and experts are involved. Not involving patients harbours the risk of developing a website that does not meet the users' needs. The fact that Wikipedia, the embodiment of user generated content, was the website with the most traffic in this study, might be the harbinger of a trend towards more user involvement in the generation of online health information.
This study is the first to assess quality of decision support in German and English language diabetes websites. We found that most websites provide basic information necessary for decision-making (such as a description of the natural course of the disease), while only a minority provides specific information for SDM (e.g. 29% provide information on potential positive and 20% on potential negative outcomes of different treatment options). Content that is required for veritable decision aids [e.g. detailed risk communication using numbers and diagrams (Trevena et al., 2012)] is not present in any of the included websites. This might be due to the effort that is required to construct a veritable decision aid (Coulter et al., 2012) and to meet the quality criteria the IPDAS Collaboration put up for decision aids (IPDAS, 2005). In order to provide probabilities of outcomes in an unbiased and understandable way, one needs to have the numbers required to do so and the capacities to convert highly complex and contradictory data into helpful and understandable information. We found only one other study researching the quality of decision support in online health information on radiological procedures. The authors of that study also conclude that only few sites provide sufficient information for medical decision-making (Smart and Burling, 2001).
We found no significant difference regarding formal quality, usability or quality of decision support between sites with commercial or non-commercial affiliation. This finding is consistent with the literature (Khazaal, 2008a,b) and might be due to the heterogeneity of non-commercial sites. This category comprises websites that are run by private persons or non-profit organizations as well as government or university sites and is heterogeneous with respect to financial and personnel capacities, aims and scope. There was a significant difference between websites with and without HON code: Websites with a HON code had significantly better formal quality. There was no difference regarding usability and quality of decision support. This might be explained by the considerable overlap of the formal quality criteria used in this study and the HON code principles. Both put an emphasis on accountability and transparency (Boyer et al., 2011). Earlier studies found that the presence of HON code was associated with accountability, which is one aspect of formal quality. There were mixed results regarding the association between presence of HON code and content quality (Khazaal, 2008a, b).
In the present study, we also assessed whether there are associations between the three quality domains and website traffic as reported on Alexa.com. There is hardly any literature on associations of website quality and traffic. We found a highly significant correlation between usability and website traffic. This is consistent with findings that design and looks of a website are the most important cues for users (Stanford et al., 2002; Sillence et al., 2007) when assessing website quality. Our finding that formal quality and website traffic are significantly correlated might be seen as contrary to an earlier finding (Eysenbach and Köhler, 2002; Stanford et al., 2002; Sillence et al., 2007) that users do not consider the source of a health website when assessing its quality. Still, declaration of ownership and authorship are only 2 out of 20 criteria assessing formal quality in our study. Further research is needed in order to explore whether formal quality is taken into consideration by users and which aspects are important to their assessment of website quality. Since we also found a highly significant correlation between the two quality domains usability and formal quality, we cannot say how much each of these two domains contributes to the association with website traffic. The fact that they are associated might be due to common underlying principles such as clarity of the website, or both might be associated to a third factor like the degree of professionalism of the website author(s). We did not find an association between quality of decision support and website traffic. This might be interpreted as a hint that detailed information on probabilities of risks and benefits of different treatment options are, at least up to the present date, not a central concern of most users.
Various quality labels and instruments are used in practice. In the scientific community, a set of quality criteria has been developed and established that has been applied in various publications. There is considerable overlap between the labels and instruments used in practice and the quality criteria applied in research. We do not know if users are aware of these labels, instruments and criteria, and we do not know if quality in the sense of these labels and instruments play a role in users' approach to health websites. Further research is needed to address these questions.
There are some limitations to the work presented. First, the Internet is a dynamic tool that changes at a rapid pace. With the cross-sectional approach, we conveniently applied we can only capture one moment in the ever changing flow of information. Similarly, the location of the authors might have influenced the search results. All searches were based in Germany, and the results might have been different had we been based somewhere else on the globe. Both of these points limit replicability of our results.
AUTHORS' CONTRIBUTIONS
N.W. participated in the conception and design of the study and drafted the manuscript. M.H. participated in the conception and design of the study and revised the manuscript. J.D. participated in the conception and design of the study and revised the manuscript. All authors read and approved the final manuscript.
FUNDING
This study was funded by the German Federal Ministry of Education and Research, funding code 01GX0710.
Acknowledgements
The authors would like to thank the three research assistants who searched and rated the websites under NWs supervision: Iniobong Essien helped co-ordinate the search and the rating process and was responsible for data administration. Caroline Wüsten, Iris Extra and Franziska Dierks participated in the rating process.