Public Opinion Research Standards and Disclosure Requirements

The dissemination of research into the public domain carries with it the potential for great consequence, especially when politics, policies and important issues are concerned. When done well, research that is shared publicly helps to inform the public of the views of Canadians on key issues and supports conversations in the development of sound policies. When done poorly, it can be misleading, erode public trust, and thus damage the reputation of our profession. The objectives of the CRIC Public Opinion Research Standards and Disclosure Requirements as applied to research released into the public domain are:

• To support sound and ethical practices in the disclosure of research;
• To ensure research is unbiased and supports decision-making in the public, private and not-for-profit sectors;
• To enhance public trust and improve the public's understanding of the use of research;
• To ensure the appropriate transparency and disclosure of research results and methods of studies.

While all CRIC members are required to follow these standards, we believe that all research, analytics and insights professionals have the obligation to follow these standards to uphold the credibility of the industry when releasing the results of research into the public domain. We recognize that there are multiple legitimate methods to conduct research. These standards cover the conduct and disclosure of quantitative research, qualitative research, and research based on secondary data sources. These standards are not intended to evaluate the merits of specific methods but to outline the disclosures that are required for each.

I. Principles of Professional Responsibility in Our Dealings with People

A. Research with Participants

1. We will avoid practices or methods that may harm, endanger, humiliate, or mislead participants and potential participants.
2. We will not misrepresent our research or conduct other activities (such as sales, fundraising, or political campaigning) under the guise of conducting research.
3. We recognize that participation in our research is voluntary except where specified by regulation, policy or law. Active participants have the right to make a free and informed decision to provide all, some or none of the information we request, without coercion, and to decline after initial agreement.
4. We will provide the name of the supplier conducting the research and a CRIC Research Verification Service (RVS) code to verify the legitimacy of the research and allow participants to share their feedback on the research.
5. We will be truthful and responsive to questions from participants and potential participants and will make no false or misleading claims about the research.
6. We recognize the critical importance of protecting the rights of minors and other vulnerable individuals when obtaining participation decisions and conducting our research.
7. We will act in accordance with Canadian laws, regulations, and data owners' (providers of research or administrative records previously collected for other purposes) rules governing the collection, use, and disclosure of information obtained from or about individuals, groups, or organizations.

B. Research Involving Personally Identifiable Information

1. We recognize the right of active participants to be provided with information about how personally identifiable information that we collect from them will be used.
2. We recognize the importance of preventing unintended disclosure of personally identifiable information. We will act in accordance with all relevant best practices, laws, regulations, and data owner rules governing the handling and storage of such information.
3. We will restrict access to identifiers and destroy them as soon as they are no longer required, in accordance with relevant laws, regulations, and data owner rules.
4. We will not disclose any information that could be used, alone or in combination with other reasonably available information, to identify participants with their data, without participant permission.
5. When disclosing personally identifiable data for purposes other than the current research, we will relay to data users any conditions of their use specified in the participant permission we have obtained.

Clients or Sponsors

1. When undertaking work for a client, we will hold confidential all proprietary information obtained about the client and about the conduct and findings of the research undertaken for the client, except when the client expressly authorizes the dissemination of the information.
2. We will inform those for whom we conduct publicly released research studies about the CRIC Public Opinion Research Standards and Disclosure Requirements and provide information on what should be disclosed in their releases.
3. We will be mindful of the limitations of our expertise and capacity to conduct various types of research and will accept only those research assignments that we can reasonably expect to accomplish within these limitations.

The Public

1. We will disclose to the public the methods and procedures used to obtain our own publicly disseminated research results in accordance with the ICC/ESOMAR code and section III of the CRIC Standards for the Conduct and Disclosure of Research
2. We will correct any errors in our own work that come to our attention that could influence the interpretation of the results, disseminating such corrections to all original recipients of our content.
3. We will correct factual misrepresentations or distortions of data or analysis, including those made by our research partners, co-investigators, sponsors, or clients. We will make good faith efforts to issue corrective statements to all parties who were presented with the factual misrepresentations or distortions, and if such factual misrepresentations or distortions were made publicly, we would correct them in a public forum to the best of our ability that is as similar as possible.
4. We recognize that differences of opinion in the interpretation of analysis are not necessarily factual misrepresentations or distortions and will exercise professional judgment in handling disclosure of such differences of opinion.

The Profession

1. We recognize the importance to the science of research to disseminate as freely as practicable the ideas and findings that emerge from our research.
2. We can point with pride to our membership in CRIC and adherence to CRIC standards as evidence of our commitment to high standards of ethics in our relations with participants, our clients or sponsors, the public, and the profession.

II. Principles of Professional Practice in the Conduct of Our Work

1. We will exercise due care in developing research designs, samples, and instruments, and in collecting, processing, and analyzing data, taking all reasonable steps to assure the reliability and validity of results.
2. We will recommend and employ only those tools and methods of analysis that, in our professional judgment, are fit for the research questions.
3. We will not knowingly select research tools and methods of analysis that yield misleading conclusions.
4. We will not knowingly make interpretations of research results that are inconsistent with the data available, nor will we tacitly permit such interpretations. We will ensure that any findings we report, either privately or for public release, are a balanced and accurate portrayal of research results.
5. We will not knowingly imply that interpretations should be accorded greater confidence than the data actually warrant. When we generalize from samples to make statements about populations, we will only make claims of precision and applicability to broader populations that are warranted by the sampling frames and other methods employed.
6. We will not engage in data fabrication or falsification.
7. We will accurately describe and attribute research from other sources that we cite in our work, in terms of its methodology, content, comparability, and source.
8. We will describe our methods and findings accurately and in appropriate detail in all research reports, adhering to the Standards of Disclosure specified in Section III of this document.

III. Standards of Disclosure

As noted previously, the release of findings from research can have a significant impact on the lives of Canadians, businesses, and public institutions. They can influence an issue or event and how the media covers it, how stakeholders, governments, and businesses react to it and how the public understands it. Good professional practice imposes a crucial obligation on all practitioners who release research into the public domain to disclose sufficient information about how the research was conducted or the data gathered to allow for independent review and peer verification of the research claims.

CRIC puts the onus on the individual and the company to comply with this responsibility, and each is singularly responsible for their respective action and failure to act. Where an individual represents a corporation, the individual and the corporation are singularly and jointly responsible. All parties involved in any infraction of the CRIC standards by commission or omission, will be held responsible because accountability is the cornerstone of every trusted structure.

Disclosure Requirements

1. We will disclose on our website details on how to obtain required disclosures for research released into the public domain.
2. We will inform those who wish to conduct research studies to be released into the public domain, including its use in whole or in part for advertising or other public displays, about the CRIC Public Opinion Research Standards and Disclosure Requirements and our professional obligations in advance of any undertaking.
3. We will disclose who sponsored the research and who conducted it. If different from the sponsor, the sources of funding will also be disclosed.
4. We will disclose an e-mail address and contact name for further enquiries. All other reasonable requests for data necessary for readers of the research to assess the validity of the research will be answered.
5. We will embed a clear URL on the release to the CRIC website that links to the CRIC Public Opinion Research Standards and Disclosure Requirements and a statement confirming compliance with the CRIC Standards.
6. We will disclose the following in a common, readily accessible format on a verifiable corporate or officially sponsored website, and make available to all when a report, results, or both from survey research are released into or appear in the public domain:
   1. The exact wording and presentation of questions and response options whose results are reported. These can be reported separately or on the data tables. This includes any preceding interviewer or respondent instructions and any preceding questions that might reasonably be expected to influence responses to the reported results. The percentage of respondents answering who are undecided or refuse to respond on voting questions should be reported.
   2. A definition of the population under study.
   3. The dates data were collected.
   4. A description of the sampling frame(s) and its coverage of the target population.
   5. The name of the sample supplier, if the sampling frame and/or the sample itself was provided by a third party always with an embedded URL to the supplier's web site home page.
   6. Whether the sample was drawn from a pre-recruited panel or pool of respondents and, if applicable, the methods used to recruit the panel or participants.
   7. Details of any strategies used to help gain cooperation (e.g., advance contact, compensation or incentives)
   8. A description of the sample design, giving a clear indication of the method by which the respondents were selected, recruited, intercepted, or otherwise contacted or encountered, along with any eligibility requirements and/or oversampling. The description of the sampling frame and sample design will include sufficient detail to determine whether the respondents were selected using probability or non-probability methods.
   9. The Method(s) and mode(s) used to administer the survey (e.g., CATI, CAPI, ACASI, IVR, mail survey, Web survey) and the language(s) offered.
   10. A statistic that expresses the amount of sampling error in a survey's results—such as a margin of error or a Bayesian credibility interval—if scientifically applicable.
   11. Whether weighting/calibration/normalization was used to adjust the results and the impact of the procedure on the data as measured through a weighting efficiency calculation and/or disclosure of the range (i.e., min and max) and variance of weights.
   12. For research released on public policy topics or election voting, the detailed tables or representations thereof by standard demographic questions with weighted and unweighted number of respondents so that the public can transparently see the original number of responses by standard demographic category and the weighted number of responses by standard demographic category. This data must be made available upon request for research released on other topics.
   13. For research released in to the public domain with a media partner, the required disclosures should be made available within 90 minutes of the release of the results by the media partner.
7. We will disclose the following in a common, readily accessible format on a verifiable corporate or officially sponsored website, and make available to all when a report, results, or both from qualitative research are released into or appear in the public domain:
   1. A definition of the population under study and its geographic location.
   2. The instrumentation used (e.g., questions, discussion guides), a description of the data collection strategies employed (e.g., focus groups, semi-structured interviews), and the language(s) used.
   3. A statement that the results of the research are not statistically projectable but rather are directional in nature, as they are based on the responses of a small selection of respondents recruited to specific criteria using qualitative recruiting practices. Reporting of the results should not include percentages, or proportions that might suggest that the results are projectable.
   4. A description of any relevant stimuli, such as visual or sensory exhibits or show cards.
   5. Dates of data collection.
   6. The physical location of all data collection activities (e.g., subject home, office/workplace, clinic, focus group facility, street corner, online, phone).
   7. A description of subject eligibility (e.g., age or gender requirements) and the procedures employed to screen and recruit research subjects.
   8. The number of research subjects, by data collection strategy.
   9. Duration of research participation (e.g., length of interviews, focus group sessions).
   10. Any compensation/incentives provided to research subjects
8. We will disclose the following in a common, readily accessible format on a verifiable corporate or officially sponsored website, and make available to all when a report, results, or both from the analysis of secondary data are released into or appear in the public domain:
   1. A description of how the content analyzed was collected or obtained. This discussion will state the source(s) used, how much content was analyzed (e.g., number and average length of articles, tweets or blog postings, news broadcasts), dates analyzed, language(s) included, and criteria or decision rules used to include or exclude elements of content.
   2. The sampling approach used. If a census of the target population of content was used, that will be explicitly stated.
   3. A discussion of any discernable threats or concerns about the validity or quality of the content (e.g., online commentary created by bots, fabricated social media profiles, relevant content missing from the source(s) used) and any steps taken to address them
   4. A description of how the analysis was conducted. This discussion will mention whether coding was done by software or human coders (or both) and whether quantitative or qualitative methods (or both) were used.
   5. The unit of analysis (e.g., a news article, broadcast, tweet, or blog posting).
   6. A description of how the weights were calculated, including the variables used, if weights were used to develop a coding scheme and/or produce final estimates.
   7. Other information that may need to be disclosed if the content analysis involves qualitative data collection (see Section III-7) or survey data collection (see Section III-6).
9. We will correct any error in our work that is brought to our attention which could influence the interpretation of the results, disseminating such corrections to all original recipients of the content.
10. Wherever possible, we will correct any factual misrepresentations or distortions of data or analysis, including those made by research partners, co-investigators, sponsors, or clients that are brought to my attention.
11. We understand that if any of our publicly released work becomes the subject of a formal review of an alleged violation of the CRIC Code, we will cooperate with any CRIC sponsored professional evaluation of such. The CRIC will only deal with those allegations of merit and will not act as a forum to settle scores between stakeholders and other parties who choose to use the review process as a tactic for their own outcome.
12. CRIC members who release market and/or opinion research into the public domain will have their corporate name and the URL link to their home page affixed to the CRIC website demonstrating that they are in compliance with the CRIC Standard of Disclosure. This posted list of CRIC firms will provide both an affirmation to members of the media and public that they are practicing the highest level of disclosure expected of the profession and a quick click directly to the firm engaged in public release activity so that more information can be found out about the firm and its release(s).
13. Reflecting the fundamental goals of transparency and replicability, CRIC members share the expectation that access to datasets and related documentation will be provided to allow for independent review and verification of research claims upon request. Datasets may be held without release for a period of up to one year after findings are publicly released to allow full opportunity for primary analysis. In order to protect the privacy of individual respondents, such datasets must be de-identified to remove variables that can reasonably be expected to identify a respondent. Those who commission publicly disseminated research have an obligation to disclose the rationale for why eventual public release or access to the datasets is not possible, if that is the case.

The CRIC Disclosure Declaration for Publicly Released Research

The following are provided for simplicity and guidance. If researchers wish to amalgamate this information into another format they may do so as long as the components listed below are contained therein.

Sample declaration of membership in CRIC and compliance with CRIC standards

XYZ Customer Research Inc. is a member of the Canadian Research Insights Council (CRIC) and confirms that this research fully complies with all CRIC Standards including the CRIC Public Opinion Research Standards and Disclosure Requirements (LINK TO STANDARDS)

A. Sample Disclosure Form for the release of Survey Research

BASIC DISCLOSURE ELEMENTS	DETAILS
Research sponsor (including all financial sponsors)	The ABC Magazine and XYZ Custom Research, Inc.
Research/Data collection supplier	XYZ Custom Research, Inc.
Population represented	Adults in Canada
Sample size	1,001
Mode of data collection	Telephone (RDD landline and RDD cell phones)
Source of sample	N/A
Type of sample (probability/non-probability)	Probability
Start and end dates of data collection	April 3, 2019 to April 7, 2019
Margin of sampling error for total sample	+/- 3.1 percentage points at the 95% confidence level
Margin of sampling error for key subgroups (if applicable)	+/- 4.3 percentage points at the 95% confidence level for people expecting or who already received a tax refund
Are the data weighted?	Yes, to the 2016 Canadian census by age, gender, region, and level of education.
Contact for more information	Susan G. Survey, Director of Polling The ABC Magazine (123) 456-7890, survey@magazine.com
Attach full text of survey questions asked, provide link to website where that information can be accessed or provide information about how the survey content can be obtained. Include any preceding questions that might reasonably be expected to influence responses to the reported results.

B. Sample Disclosure Form for the release of Qualitative

BASIC DISCLOSURE ELEMENTS	DETAILS
Research sponsor (including all financial sponsors)	The ABC Magazine and XYZ Custom Research, Inc.
Research/Data collection supplier	XYZ Custom Research, Inc.
Population under study	Voting Adults in selected cities
Number of participants, data collection strategy, length	Two 90 minutes focus groups with 8 participants in each in 5 cities (Halifax, Montreal, Toronto, Calgary, Vancouver). Groups in Montreal conducted in French. All others in English. All groups held at focus group facilities
Statement of non-projectability	The results of the research are not statistically projectable but rather are directional in nature, as they are based on the responses of a small selection of respondents recruited to specific criteria using qualitative recruiting practices.
Start and end dates of fieldwork	April 3, 2019 to April 7, 2019
Contact for more information	James Qual, Director of Qualitative Research The ABC Magazine (123) 456-7890, survey@magazine.com
Attach full text of questions and copies of any stimuli reported on or provide link to website where that information can be accessed. Include any preceding questions that might reasonably be expected to influence responses to questions reported on.