GLOBE 2020 : Country Co-Investigator (CCI) Handbook
1. ABOUT GLOBE 2020: A LARGE SCALE MULTICOUNTRY RESEARCH PROGRAM
This document broadly outlines the roles, responsibilities, benefits and task requirements for Country Co-Investigators (CCIs) who participate in the GLOBE 2020 study on societal culture. The sections below provide general guidelines and an overview for CCIs to be involved in various capacities in the GLOBE 2020 research project.
ABOUT THE RESEARCH PROJECT
The current research project is an extension of over 20 years of collaborative work conducted by the GLOBE research team. In the 1990s, the GLOBE (Global Leadership and Organizational Behavior Effectiveness) project examined the relationships among societal culture, leadership, and organizational practices in collaboration with more than 200 researchers from 62 societies. GLOBE has become one of the largest and most comprehensive studies of its kind in the social sciences. In this phase of GLOBE, called GLOBE 2020, we will build on the past two decades of findings and will focus on four important research questions:
To what extent have the cultures of the countries in the original GLOBE study changed? And what are the major societal drivers of culture change?
To answer this question, we will use the original GLOBE instruments with a few additional questions to survey primarily middle managers in the original GLOBE countries. The figure below shows a simplified theoretical framework driving the research on this question. Of course there are many possible drivers of culture change but these independent variables represent some of the larger ‘buckets’ of possible culture change drivers.
What is the relationship between national culture and antecedents of interpersonal trust in different countries?
We will use a newly designed and validated instrument and plan on conducting surveys in more than 100 countries. The figure below shows a simplified theoretical framework focused on this question.
What insights can be gained by expanding the number of countries mapped by the GLOBE project to include relatively unstudied countries in the Middle East and North Africa region?
In addition, GLOBE 2020 will collect cultural information from a large number of the countries in Africa and the Middle East that were not part of the original GLOBE study which enable more nuanced cross-comparative research for years to come. We are interested in assessing and examining GLOBE's cultural and leadership dimensions and the relationships between them with particular interest in societies that are experiencing a rapid cultural transition (MENA region).
How do leadership perceptions differ across genders and cultures?
In order to provide further understanding of the role of gender (and gender differences) in leadership behavior and effectiveness across cultures, our sample will include a larger percentage of female leaders than in earlier GLOBE research. We wish to determine if perceptions of outstanding leaders differ across genders and whether these perceived gender differences vary cross culturally. As with question #3 above, it will be instructive to examine the relatively unstudied MENA region with respect to perceptions of outstanding leadership.
The GLOBE research instruments were pilot tested with over 1000 managers in six countries (USA, Lebanon, India, China, Pakistan, Austria) and were finalized in February 2019. Data collection in participating countries is expected to be completed by the end of 2019.
The GLOBE 2020 team
The Principal Investigator (PI) for GLOBE 2020 is Dr. Ali Dastmalchian, Dean and Professor, Beedie School of Business, Simon Fraser University (SFU), Vancouver, Canada. Dr. Dastmalchian is the CEO of GLOBE Foundation and has primary responsibility for the intellectual direction of the research and research-related activities, and assumes administrative and financial responsibility for this project. The GLOBE team has already received substantial funding to conduct this research. The project will be headquartered at the Beedie School of Business, SFU.
The team has six Principal Co-Investigators. As a team, they will manage the intellectual direction of the research and research-related activities. Professors Carolyn Egri (Simon Fraser University), Rick Cotton (University of Victoria), Peter Dorfman (New Mexico State University, Past CEO of GLOBE Foundation), Amanda Bullough (University of Delaware), Mansour Javidan (Thunderbird School of Global Management and former CEO of GLOBE Foundation), and Mila Lazarova (Simon Fraser University) are the principal co-investigators of GLOBE 2020.
Project Director for GLOBE 2020 is Dr. Mansour Javidan, Garvin Distinguished Professor and Director of Najafi Global Mindset Institute at Thunderbird School of Global Management. He will manage the required activities to ensure a timely and smooth completion of the project.
Country Co-Investigators (CCIs)
CCI refers to the research collaborators participating in the project from various countries and regions in GLOBE 2020. CCIs play a significant role in the conduct of the on-site research activities such as data collection in their respective countries/regions, translation/ back translation of the instrument (if needed), and providing help in interpreting the findings from their own country. CCIs collecting data in a country are viewed as the primary researchers representing that country, and will be given the first opportunity to review and update existing GLOBE findings relating to that country.
Research Coordinators are post doctorate researchers and PhD students who work under the direction of the Project Director and Principal investigators. Research Coordinators provide support in research coordination and communication, assist in overseeing different teams, support local CCIs, and produce high-quality research resources and materials. At present, Dr. Medha Satish Kumar, Natasha Fox, and Dr. Anirban Kar are members of the research coordinating team. Dr. Avid Khamenehfar is the translation coordinator for this project.
The Research Team
Professor Ali Dastmalchian (Principal Investigator), Professor Mansour Javidan (Project Director), Professor Peter Dorfman, Professor Carolyn Egri, Professor Rick Cotton, Professor Amanda Bullough and Professor Mila Lazarova. The Research Team holds virtual and face to face meetings as needed to plan the theoretical models, design the instrument, pilot test the instrument, and conduct the required analyses of the results. The team has held several 3-4-day meetings in Vancouver over the past two years as well as meetings during Academy of Management annual meetings.
The Administrative Team
Professor Mansour Javidan (Project Director), Professor Peter Dorfman, Professor Ali Dastmalchian, Professor Rick Cotton, Dr. Medha Satish Kumar, Dr. Anirban Kar, Dr. Avid Khamenehfar, and (soon to be Dr.) Natasha Fox. The team manages the GLOBE 2020 project and meets every Friday for two hours. Many additional meetings take place on various topics on a regular and frequent basis.
EXPECTATIONS AND CONTRIBUTIONS
Contributions of the Principal Investigator Team
Supervise the intellectual direction of the GLOBE 2020 Project.
Meet at least 4 times each year (in person or virtually) for 2 to 3 days for think-tank type activities.
Develop yearly plans and research models and identify priorities for other research activities.
Oversee the development of measurement instruments and psychometric and statistical methodologies.
Oversee the data collection efforts of various teams and collaborate with CCIs by keeping an open channel of communication with different stakeholders and participants.
Develop conference forum proposals during the course of this project to reach out to various audiences for feedback and to share findings of the research study.
Oversee data analysis and produce summaries of research findings.
Collaborate in preparing journal articles, conference publications and book chapters based on the findings of this project.
Oversee knowledge mobilization activities other than publications such as development and delivery of practitioner-oriented workshops, course curriculums and updates, educational/training plans and training materials, etc.
Expected CCI Contributions
GLOBE 2020 CCIs play a critical role in GLOBE 2020 and as with previous GLOBE initiatives will can expect significant benefits including: co-authorship rights (specified below), increased and nuanced cultural understanding, enhanced research skills and the recognition associated with being part of one of the most wide-ranging, multi-country research endeavors yet devised. As such they are expected to make the following contributions to the project:
If needed, obtain necessary permissions or ethics approvals in their respective institutions. GLOBE 2020 study has already been approved by Simon Fraser University, Canada. These documents will be shared with CCIs as requested.
Provide accurate estimates of the number of respondents they expect to gather surveys from given the goal of gathering at least 300 (and ideally 500+) respondents per country and then being open to modifying their collection approach to achieve this goal as necessary.
Manage translation and back translation of the survey instruments in the major language (or in some cases languages) of their countries. In countries where the major language is shared by many countries (e.g. Spanish), coordinate the translation process with the administrative team and CCIs in these countries. A member of the GLOBE team, Dr. Avid Khamenehfar, will work directly with the CCIs to support them in this process.
Find qualified managers to participate in the survey in their country. Criteria and procedure for procuring research participants is provided in this document.
Manage timely, high quality data collection of surveys from their own country. Except in rare and unusual circumstances, data will be collected via internet-based surveys. In countries where this is not possible, we will use other means such as paper and pencil surveys. Either way, survey data must be of high-quality and as complete as possible.
If needed, participate in virtual meetings with the principal investigators team to share ideas, questions, concerns, and to receive updates.
If needed, provide interpretation of results that pertain to their specific country and to the overall meaningfulness of the project.
INTELLECTUAL PROPERTY OWNERSHIP
Ownership of Data
All data will be collected and housed on the servers at Simon Fraser University, Canada, for the duration of the project. The collected data in each country are owned by the GLOBE Foundation. Upon completion of GLOBE data collection in all countries, each CCI will receive a GLOBE approved and finalized copy of their own country data. They will be free to publish their own country data as a single country study. The data are not to be used in studies which compare the CCI country to other countries in the GLOBE dataset. In case there are multiple CCIs, all the data collected in that country will be sent to all the CCIs of that country.
The GLOBE team plans to publish four major articles in high quality journals within a two-year period after completion of data collection. These papers will be published jointly with all the participants including Principal Investigators, CCIs and research coordinators involved in the project. If past is prologue, these studies can expect wide recognition and use. (For example, the 2004 GLOBE book by House, R. J., Hanges, P. J., Javidan, M., Dorfman, P. W., & Gupta, V. (Eds.). entitled, “Culture, leadership, and organizations: The GLOBE study of 62 societies” by Sage Publications now has more than 9,000 citations on Google Scholar). The book received the 2005 M. Scott Myers Award for Applied Research in the Workplace from Society for Industrial and Organizational Psychology.
Upon publication of the 4 major papers, each CCI will receive a country level dataset of all participating countries and at that point, other papers/presentations may be proposed by Principal Investigators and CCIs for additional papers. These may include specific regions (e.g. MENA, Asia) or specific variables across countries. The GLOBE project team looks forward to the development of these papers/presentations.
As pointed out above, the GLOBE team plans to publish four major articles within a two-year period after completion of data collection and all CCIs will be listed as coauthors. The principal investigator and co-investigators acknowledge and agree to the importance of publishing articles in top rated academic journals and other highly respectable venues. Authorship credits for the jointly published papers will be determined in accordance to academic standards of contributions made by the members and then, alphabetically.
Timelines and compliance
The GLOBE 2020 data collection is expected to be completed by the end of 2019.
2. GLOBE 2020 SURVEY ADMINISTRATION PROTOCOL
This document lays out the protocol to administer the GLOBE 2020 survey. The document consists of the following sections:
- The survey instrument,
- Administration of the survey,
- The sample,
- Compliance with regulations, and
- The GLOBE research coordinators.
Country Co-Investigators (CCIs) are requested to closely follow this protocol. Doing so will ensure that the data collected across the countries are comparable and that the findings and interpretations are robust.
THE SURVEY INSTRUMENT
The GLOBE 2020 survey instrument consists of 182 items (demographic variables: 18, culture/trust/marker variables: 92 and leadership variables: 72). Pilot studies indicate that the survey takes between 20 and 30 minutes to complete. Considering the survey’s length, additions to the main survey are not feasible.
The GLOBE research team has prepared the English version of the survey instrument. The CCIs will assist in translating the survey instrument to their country’s major official language(s) as needed for their intended group of respondents. The translation must follow the translation protocol, available here (Section 4 in this handbook)
ADMINISTRATION OF THE SURVEY
The survey will be administered online. Once the instrument is ready in English or translated version, a member of the GLOBE research team will email the URL (web address) link of the survey instrument to the CCI. The CCI will share this URL link with the participating managers, who can then access the survey by clicking on the link or by copying and pasting the URL link on their internet browser. All the necessary permissions and instructions to fill out the survey are provided on the instrument.
The GLOBE team will have the survey instrument hosted on an online platform provided by Qualtrics (https://www.qualtrics.com/). Qualtrics will store the data in Canada. Only under rare and unique circumstances should the survey be administered with a hard-copy paper and pencil format. The reason for this is simple. The online format is much easier to administer with less cost and minimizes the chance of issues occurring (e.g. lost surveys, missing data). This decision was made for the CCIs’ and the GLOBE team’s benefit. However, if in a rare case, the CCI needs to use paper and pencil to meet the needs of their respondents, they need to contact the GLOBE team as soon as possible.
Once the survey is under way, a member of the GLOBE research team will update each CCI on a weekly basis (or at a shorter time interval if the CCI so desires), about the number of participating managers who completely filled out the survey instrument. No data will be gathered relating to the identity of the participating managers. So, it will not be possible to identify who filled out the survey.
The most preferred sample (participating managers) is mid-level managers working in domestic organizations, although the CCIs may need to include managers working in multinational firms to ensure a large sample size. Mid-level managers are those who manage at least one level of supervisors. Upper-level managers and first level managers may also be included in the sample; however, the preference is for mid-level managers. The sample may include those working in private or public (government owned/non-profit) organizations and those working in any industrial sector.
The sample needs to be from two or more organizations, in each country. Participating managers need to be citizens or long-term residents of the country for which the CCI is gathering the data. The ideal sample will represent different regions in the country and represents both genders. The ideal sample size of completed questionnaires is 500 in each country. The minimum sample size is 300 in each country; which is the main reason why we have multiple CCIs in several countries.
The CCIs need to send out a 'survey invitation letter' to participating managers. Please click here for a template for letter of invitation (Section # 3 in this handbook). The CCIs will keep a record of the number of participating managers that were contacted to complete the survey. The GLOBE research coordinators and the CCIs will need this information in order to be able to report the response rates in the manuscripts for publication.
COMPLIANCE WITH LOCAL REGULATIONS
The CCIs need to comply with the regulations in their respective country and institution. The GLOBE 2020 research project has received necessary approvals from Simon Fraser University and from the funding agencies in Canada and will make its documents available to CCI’s who request it. To preserve the confidentiality and privacy of the participating managers, as required by Simon Fraser University and Canadian regulations, no personal information such as names or contact information will be collected in the survey. The CCIs are requested to not collect any such information.
THE GLOBE RESEARCH COORDINATORSThe GLOBE research coordinators are Dr. Medha Satish Kumar (firstname.lastname@example.org), Natasha Fox (email@example.com), Dr. Anirban Kar (firstname.lastname@example.org), and Dr. Avid Khamenehfar (email@example.com) based in Vancouver, Canada. They will extend all possible support to the CCIs. They can all be reached at firstname.lastname@example.org. In addition, if required, the project director, Dr. Mansour Javidan (Mansour.Javidan@thunderbird.asu.edu) and the principal investigator, Dr. Ali Dastmalchian (email@example.com) can also be contacted.
3. SURVEY INVITATION LETTER TEMPLATE
This letter is a template that CCIs may use in an email invitation to target managers to complete the GLOBE questionnaire. If needed, the CCIs may slightly modify this letter to best fit their needs. Once you receive the Qualtrics survey link, please include it in this email to your target managers.
Dear Colleague, On behalf of Global Leadership and Organizational Behavior Effectiveness (GLOBE) project team, we invite you to fill out a survey on the topics of culture, trust and leadership. The survey can be accessed by clicking on this link or by copying and pasting the URL link on your internet browser and hitting enter: (CCI: please insert the survey link provided by the GLOBE team). The GLOBE project is a continuation of 20 years of collaborative work examining relationships among societal culture, leadership, and organizational practices around the world. You can find out more about GLOBE here: https://globeproject.com/. Our objective in this survey is to learn further about cultural and leadership practices in more than 100 countries. The findings of this research will be published in academic and managerial publications and will be presented at conferences. At the end of the survey we will also provide a summary of your preferred leadership profile. This survey will take about 20 to 30 minutes to complete. Your survey responses will be confidential and anonymous. Only aggregate information (no individual responses) will be published. The survey will not collect any information that discloses your identity nor the identity of your workplace. The Principal Investigator of this research is Dr. Ali Dastmalchian, Dean of the Beedie School of Business, Simon Fraser University, Canada. The research project director is Dr. Mansour Javidan, Garvin Distinguished Professor at Arizona State University, USA. This project is funded by the Social Sciences and Humanities Research Council of Canada (SSHRC) and Simon Fraser University, Canada. If you would like further information on this research project or the survey, please contact ________________________ (CCI name and affiliation) or the GLOBE research team (firstname.lastname@example.org). We thank you in anticipation of your participation in this worldwide research effort. Sincerely, CCI name CCI Affiliation and Address CCI Phone: _________________; Email address: ______________
4. TRANSLATION PROTOCOL
This document outlines the procedures involved in scientifically translating the GLOBE 2020 English survey to the local language(s) of all target countries. The goal of this translation protocol is to ensure that the translated surveys are conceptually equivalent and accurate for each of the target countries. Because the questionnaire has over 180 items, we have divided it into two parts: part 1 is required and covers items on culture, trust, and religiosity. Part 2 includes items on leadership expectations. To incentivize the respondents to complete part 2 which is voluntary, the survey system will provide an immediate feedback report to those who complete this section. The report shows the respondent’s profile of outstanding leadership. It is important that both the survey itself (part 1 and 2), and the feedback report are translated and back translated. The document provided to CCIs for translation includes both the questionnaire and the feedback report. From herein, for ease of communication, we will simply refer to both the survey and the report as “the survey”.
CONCERNS WITH TRANSLATION AND BACK-TRANSLATION QUALITY
Translation is a complex process! While the translation/back-translation procedure originally suggested by Brislin (1970) has often been used in cross-national research, this procedure as a single means to assure translation quality has been the subject of much concern and criticism. Empirical studies have shown that mistakes are common using this procedure alone with some studies showing upwards of one-third of survey questions may have subtle as well as glaring mistakes leading to misunderstandings and inappropriate translations. GLOBE 2020 has therefore chosen to employ the following state of the art procedures in translation/back translation and assessment of quality.
Identify four qualified bilingual individuals in the target country to assist in the following process: (1) two individuals for the translation, (2) one additional person to review and adjudicate, and (3) a final person for back-translation. All the CCIs are of course qualified and they may choose to be either one of the two translators or the reviewer/adjudicator. In any case the CCIs will need an additional three local individuals with strong proficiency in business English language. No individual can perform more than one of the three roles in this process.
The GLOBE 2020 team will finish the process by comparing the back translated English version of the survey with the original English version.
- Two individuals will independently translate the English survey version into the local language - they should not work together on this process to assure independence. After completing their separate translations (using the two provided Excel worksheets), the two individuals will submit their translated surveys to the reviewer/adjudicator.
- The reviewer/adjudicator is a third individual who will review both translated versions and if necessary, consult with both translators to create a final translated version. The agreed upon translated version will be sent to individual number 4, by the CCI, to translate the survey back into English language.
- Individual number 4 will back-translate the survey instrument into English. It is very important that this individual DOES NOT see the original English version. As explained earlier, the CCI cannot serve in the role of a back-translator.
- The CCI will send the back translated version to GLOBE 2020 research team. Notes concerning issues, potential problems and a general assessment of the translation should accompany the material sent to the GLOBE research team.
- The GLOBE 2020 research team will compare the original English version with the back-translated English version. They will inform the CCI if the questionnaire is ready in the local language or if it needs modification.
- Once the local language version is complete and finalized, the survey may begin in the target country.
NOTE: The Excel file going to Individual 4 for back-translation should not include any English items!
PROCEDURES FOR COUNTRIES WITH SHARED LOCAL LANGUAGES
There are some countries that have a local language in common with other countries, e.g., Arabic, Mandarin Chinese, French, German, Spanish. For these countries, the GLOBE research team will ask the local CCI to introduce two individuals who can independently verify the validity of the common language instrument in their local language and suggest any modification(s). The CCI could be one of these two individuals. If the changes are minimal, no back-translation is needed. If the changes are major, we need to follow the original translation back-translation protocol.
PROCEDURES FOR COUNTRIES WITH MULTIPLE CCIS
For countries in which there is more than one CCI, the responsibilities for the translation/back-translation process may be allocated to individuals at different institutions. For example, in the case of two CCIs for a country, one CCI may take responsibility for the initial translation of the survey into the local language, and the other CCI may take responsibility for the review/adjudication. A second translator would still be needed in this scenario as well as an additional person for the back translation. The GLOBE 2020 research team will coordinate with CCIs in these countries regarding the most appropriate allocation of translation/back-translation responsibilities.
Please notify the GLOBE research team if you believe there is a need to modify any part of the above protocol in your country.
5. GLOBE 2020 PROJECT PLAN
The table below lists all the activities involved in GLOBE 2020 to help the GLOBE team and CCIs have a better picture of all the required steps in this project up to the completion of data collection.
|No||Steps||Assigned to||Start Date||Completion Date|
|Setting up national CCI teams|
|1||CCIs will receive CCI handbook||GLOBE admin team|
|2||If agreed, CCIs will coordinate with other members in their country regarding splitting of tasks around translation or data collection||CCIs|
|3||CCIs will obtain necessary permission/ ethics approval/ funding from their institutions||CCIs|
|Translation and Back-translation procedure|
|4||Target languages for each country to be identified by CCIs||Avid and Natasha|
|5||Receive English version of survey, translation protocol and Excel worksheets||Avid and Natasha|
CCIs to send names, affiliation and email ids of translators, adjudicators and back translators to GLOBE admin team
CCIs and Avid
Survey forward translation (two independent translators)
Review and adjudication
Survey back translation
Review and approval of translation by comparing back translated English version and original English version of survey
GLOBE research team headed by Peter
|11||Prepare online links of translated surveys and review for errors||Anirban, Medha, Avid, and CCIs|
|Data collection process|
|12||CCIs will receive an online link to the translated survey||Medha and Anirban|
|13||CCIs will adapt and send survey invitation participation letters with survey link to participating managers||CCIs|
|14||GLOBE Admin team will send weekly updates on usable responses received to national CCI teams||GLOBE admin team|
|15||CCIs to send timely reminders to participants and improve response rates||CCIs|
|16||GLOBE admin team will update CCIs when sufficient data has been collected and when data collection is to be ended||GLOBE admin team|
|17||CCIs will report on the response rate; i.e. provide information on number of people approached, number of reminders sent and final usable data received||CCIs|
|18||To be conducted by the GLOBE research team|
6. COUNTRY CO-INVESTIGATOR CHECKLIST
Here is a list of all the important steps that you as a CCI need to take over the next few months. The checklist is prepared to be helpful to you and to ensure that all important steps are completed in this project. Please make sure to put a check mark in front of all the completed steps as you complete the project.
Received and reviewed CCI handbook.
- If needed, obtained necessary permissions/ethics approvals for my data collection.
- If needed, informed the GLOBE administration team of any concerns with survey administration and translation procedures.
- Identified language(s) in which survey will be administered in their country.
- Received English version survey and excel sheet templates for translation.
Identified 2 forward translators, 1 adjudicator and 1 back translator for the translation procedure.
- Back-translator is not a CCI of this project.
- The back-translator will never see the original English survey.
- Informed GLOBE administration team of the names, affiliations, and email IDs of all translators.
- Provided forward translators with the excel document that contains all questionnaire items in English.
- Received two independent forward translations (local language) of the instrument for adjudication.
- Adjudicated and prepared a unified forward translation (local language) instrument.
- Sent unified forward translation (local language) of the instrument (only) to back translator (making sure that none of the original English questions are shown and question numbers are appropriately maintained).
- Received back-translated English version from back translator.
- Sent back-translated version (English) and unified forward translation (local language) for approval to GLOBE administration team.
Received approval for final translation into local language.
- If questions had arisen, reviewed and revised accordingly.
- Received unique sharable link in local language from GLOBE administration team.
- Received survey invitation letter template (Document #3 the CCI handbook).
- Adapted survey invitation letter and included the unique link to the survey.
- Sent the invitation letter in an email to all participating managers.
- Received updates from GLOBE administration team regarding responses and sent reminders to participating managers.
- Tracked and reported (to GLOBE administrative team) on the number of participating. managers contacted for survey participation, as well as number of reminders sent.
- Completed data collection with sufficient sample size (300 – 500 usable data points in each country).
7.Protocol For the Use of the GLOBE 2020 Logo
In order to acknowledge your important contribution to the GLOBE 2020 research project, we have created a customized logo for all CCIs. As a CCI you are encouraged to use this logo in materials that may strengthen the visibility of GLOBE 2020. In using the logo we ask that all GLOBE 2020 CCIs observe the following protocol.
We encourage you to add the customized logo to your email signature, and you are especially encouraged to use this in any and all communications that pertain to GLOBE 2020. While email programs vary, the general method for adding a logo to an email signature is: 1. In your email program, go to "Preferences"; 2. Look for a section called "Signatures"; 3. You should be able to drag and drop a PNG file of the logo directly into your signature below the text.
In addition to using the logo in your email signature you are also welcome to create official letterhead using your custom logo. You are encouraged to use the logo in email, letters and other materials in a professional and scholarly manner to promote your role in GLOBE 2020 research and the GLOBE 2020 project in general.
Please note that manipulation of logo in any way, including the information contained in the logo, or its appearance, font or colour scheme is not permitted. Please contact the research administration team if you have any questions or concerns about the use of GLOBE 2020 logo.
8.Policy for Use of MTurk and Qualtrics Professional Panels for GLOBE 2020 Data Collection
We have some concerns about the use of MTurk and Qualtrics panels for data collection for the GLOBE 2020 project but if CCIs are willing to abide by the guidelines below, these panels may still be used. While some studies have suggested that data quality may be low when using such professional panels, other studies indicate the value of using such panels to address external validity concerns. If you are planning to use MTurk or Qualtrics panels for data collection, we request you to please contact the GLOBE team (at email@example.com). We also request you use the guidelines below which outline key steps to better ensure quality responses from GLOBE 2020 survey respondents.
Guidelines for MTurk and Qualtrics use as adapted from Walter, Seibert, Goering & O’Boyle Jr. (2019) and Holt & Laraas (2018):
1.Use pre-survey screening questions to select panel respondents for the GLOBE 2020 survey. The two most important points are that respondents need to be residents of the country for which the CCI is gathering data and that the respondents represent a wholly managerial (and preferably primarily middle manager) sample.
2.In the pre-survey questions, advise participants that their responses to the GLOBE 2020 survey will be quality checked and Include the following statement (bold font) within the first set of instructions.
The quality of the answers that you provide will be checked during this survey as well as after the survey responses are completed.
3.Select respondents to include in the final dataset such that each respondent has a different IP address.
4.Although an extreme measure, we may have to discard up to the first 150 respondents who may be professional respondents if there are systematic issues with their responses and therefore data collection size may need to be higher on a per country basis (500 +150).
5.Select respondents who have only a high quality rating on MTurk (90%+). For Qualtrics, back-end data checking is critical especially for respondents: not meeting screening criteria; submitting low quality responses (including unreasonably quick response time, e.g., total survey response time should be no shorter than 20 minutes); response oversimilarity (e.g., strong preponderance of the same question response); and rampant instruction inattention. These are all reasons for the CCI to request replacement respondents from Qualtrics.
6.The call for responses and the payment to respondents should be carefully calibrated for the country in which the CCI is collecting data from panels. This is to ensure that the payment amount motivates the right respondent to complete the survey and that such payment does not over-motivate low quality responses.
7.Use established missing data checks for data quality. We want the highest quality responses and will reject respondents who have too many skipped questions.
8.Use speeding checks (calculated post hoc by flagging respondents within the same country sample who take one-third of the median time or less) to complete the survey in order to further assess response quality. This may also lead to respondent rejection.
9.The marker variable questions for the GLOBE 2020 will also be used as an attention check with respondents who clearly did not pay attention to instructions for this section being flagged for replacement.
Please contact the research administration team if you have any questions or concerns about the use of MTurk/Qualtrics professional panels for data collection.
9. CONTACT INFORMATION
8888 University Drive, B.C. Canada V5A 1S6
The GLOBE books represent results from a twenty-year research program investigating the influence of culture on societal and organizational effectiveness. Our latest book showcases our examination of strategic leadership effectiveness for top-level management based on data from more than 1,000 CEOs and over 6,000 top-level managers in 24 countries.View