Table 4 Checklist for Reporting Results of Internet E-Surveys (CHERRIES)
Checklist Item | Explanation | Manuscript section |
|---|---|---|
Describe survey design | Describe target population, sample frame. Is the sample a convenience sample? (In “open” surveys this is most likely.) | See “Methods” section, subheadings; ‘Study Design’, ‘Participants’ and ‘Sample Size.’ |
IRB approval | Mention whether the study has been approved by an IRB | See “Methods” section, subheading ‘Ethical Approval.’ |
Informed consent | Describe the informed consent process. Where were the participants told the length of time of the survey, which data were stored and where and for how long, who the investigator was, and the purpose of the study? | See “Methods” section, sub-heading ‘Final Survey’ paragraph 3 |
Data protection | If any personal information was collected or stored, describe what mechanisms were used to protect unauthorized access | See “Methods” section, sub-heading ‘Final Survey’ paragraph 2. All responses were anonymous |
Development and testing | State how the survey was developed, including whether the usability and technical functionality of the electronic questionnaire had been tested before fielding the questionnaire | See “Methods” section, sub-heading ‘Survey Design for question development and piloting process |
Open survey versus closed survey | An “open survey” is a survey open for each visitor of a site, while a closed survey is only open to a sample which the investigator knows (password-protected survey) | The survey was not password protected and screening questions at the beginning of the survey should prevent ineligible responses |
Contact mode | Indicate whether or not the initial contact with the potential participants was made on the Internet. (Investigators may also send out questionnaires by mail and allow for Web-based data entry.) | See “Methods” section, sub-heading ‘Recruitment/Distribution.’ |
Advertising the survey | How/where was the survey announced or advertised? Some examples are offline media (newspapers), or online (mailing lists—If yes, which ones?) or banner ads (where were these banner ads posted and what did they look like?). It is important to know the wording of the announcement as it will heavily influence who chooses to participate. Ideally the survey announcement should be published as an appendix | See “Methods” section, sub-heading ‘recruitment/distribution.’ |
Web/E-mail | State the type of e-survey (e.g., one posted on a Web site, or one sent out through e-mail). If it is an e-mail survey, were the responses entered manually into a database, or was there an automatic method for capturing responses? | See “Methods” section, sub-heading ‘Recruitment/Distribution’ the survey was hosted through an online platform that automatically recorded responses and was accessible through a weblink |
Context | Describe the Web site (for mailing list/newsgroup) in which the survey was posted. What is the Web site about, who is visiting it, what are visitors normally looking for? Discuss to what degree the content of the Web site could pre-select the sample or influence the results. For example, a survey about vaccination on a anti-immunization Web site will have different results from a Web survey conducted on a government Web site | NA—not hosted on a website |
Mandatory/voluntary | Was it a mandatory survey to be filled in by every visitor who wanted to enter the Web site, or was it a voluntary survey? | NA—not hosted on a website |
Incentives | Were any incentives offered (e.g., monetary, prizes, or non-monetary incentives such as an offer to provide the survey results)? | See “Methods” section, sub-heading ‘Incentive To Participate’ |
Time/Date | In what timeframe were the data collected? | See “Methods” section, sub-heading ‘Recruitment/Distribution.’ |
Randomization of items or questionnaires | To prevent biases items can be randomized or alternated | See “Methods” section, sub-heading ‘Final Survey’ paragraph 1. Response items were randomized |
Adaptive questioning | Use adaptive questioning (certain items, or only conditionally displayed based on responses to other items) to reduce number and complexity of the questions | See “Methods” section, sub-heading ‘Final Survey’ paragraph 1 |
Number of Items | What was the number of questionnaire items per page? The number of items is an important factor for the completion rate | See “Methods” section, sub-heading ‘Final Survey’ paragraph 1. This varied due to adaptive questioning being enabled |
Number of screens (pages) | Over how many pages was the questionnaire distributed? The number of items is an important factor for the completion rate | See “Methods” section, sub-heading ‘Final Survey’ paragraph 1 |
Completeness check | It is technically possible to do consistency or completeness checks before the questionnaire is submitted. Was this done, and if “yes”, how (usually JAVAScript)? An alternative is to check for completeness after the questionnaire has been submitted (and highlight mandatory items). If this has been done, it should be reported. All items should provide a non-response option such as “not applicable” or “rather not say”, and selection of one response option should be enforced | See “Methods” section, sub-heading ‘Final Survey’ paragraph 2 |
Review step | State whether respondents were able to review and change their answers (e.g., through a Back button or a Review step which displays a summary of the responses and asks the respondents if they are correct) | See “Methods” section, sub-heading ‘Final Survey’ paragraph 2 |
Unique site visitor | If you provide view rates or participation rates, you need to define how you determined a unique visitor. There are different techniques available, based on IP addresses or cookies or both | Not reported—data not collected |
View rate (Ratio of unique survey visitors/unique site visitors) | Requires counting unique visitors to the first page of the survey, divided by the number of unique site visitors (not page views!). It is not unusual to have view rates of less than 0.1% if the survey is voluntary | Not reported—data not collected |
Participation rate (Ratio of unique visitors who agreed to participate/unique first survey page visitors) | Count the unique number of people who filled in the first survey page (or agreed to participate, for example by checking a checkbox), divided by visitors who visit the first page of the survey (or the informed consents page, if present). This can also be called “recruitment” rate | Not reported—data not collected |
Completion rate (Ratio of users who finished the survey/users who agreed to participate) | The number of people submitting the last questionnaire page, divided by the number of people who agreed to participate (or submitted the first survey page). This is only relevant if there is a separate “informed consent” page or if the survey goes over several pages. This is a measure for attrition. Note that “completion” can involve leaving questionnaire items blank. This is not a measure for how completely questionnaires were filled in. (If you need a measure for this, use the word “completeness rate”.) | Completion rate for the full survey (includes data not reported in this study) was calculated as the number of respondents who completed/respondents who consented (554/764, 72.5%) |
Cookies used | Indicate whether cookies were used to assign a unique user identifier to each client computer. If so, mention the page on which the cookie was set and read, and how long the cookie was valid. Were duplicate entries avoided by preventing users access to the survey twice; or were duplicate database entries having the same user ID eliminated before analysis? In the latter case, which entries were kept for analysis (e.g., the first entry or the most recent)? | See “Methods” section, sub-heading ‘Final Survey’ paragraph 4 |
IP check | Indicate whether the IP address of the client computer was used to identify potential duplicate entries from the same user. If so, mention the period of time for which no two entries from the same IP address were allowed (e.g., 24 h). Were duplicate entries avoided by preventing users with the same IP address access to the survey twice; or were duplicate database entries having the same IP address within a given period of time eliminated before analysis? If the latter, which entries were kept for analysis (e.g., the first entry or the most recent)? | See “Methods” section, sub-heading ‘Final Survey’ paragraph 4 |
Log file analysis | Indicate whether other techniques to analyze the log file for identification of multiple entries were used. If so, please describe | NA |
Registration | In “closed” (non-open) surveys, users need to login first and it is easier to prevent duplicate entries from the same user. Describe how this was done. For example, was the survey never displayed a second time once the user had filled it in, or was the username stored together with the survey results and later eliminated? If the latter, which entries were kept for analysis (e.g., the first entry or the most recent)? | NA |
Handling of incomplete questionnaires | Were only completed questionnaires analyzed? Were questionnaires which terminated early (where, for example, users did not go through all questionnaire pages) also analyzed? | See “Methods” section, sub-heading ‘Data Analysis.’ |
Questionnaires submitted with an atypical timestamp | Some investigators may measure the time people needed to fill in a questionnaire and exclude questionnaires that were submitted too soon. Specify the timeframe that was used as a cutoff point, and describe how this point was determined | See “Methods” section, sub-heading ‘Data Analysis.’ The full survey required 15–20 min to complete |
Statistical correction | Indicate whether any methods such as weighting of items or propensity scores have been used to adjust for the non-representative sample; if so, please describe the methods | NA |