You are here

6 Online Research Platforms: Pros and Cons

by Barbara Toizer

You’ve made the decision to move your research study online, but perhaps you are not familiar or entirely comfortable with the various tools available. We’ve compiled a list of some of the more popular ones here and take a look at the pros and cons of each.

The first step to moving online is transferring the study materials to an online platform. The best platform to use depends on the type of study being conducted. If the study is a survey, then the questions can be pasted into a survey-hosting platform. Popular platforms include Qualtrics, SurveyGizmo, SurveyMonkey, and Google Forms. You may have access to at least one of these platforms through your institution.

If a study employs a more complex behavioral task, consider using an experiment-hosting platform. Popular platforms include Gorilla, Inquisit, and Pavlovia. These platforms allow researchers to put together experimental tasks, either from a library of common tasks or individually programmed by the user. These platforms also host the experiment on their servers so researchers do not have to set up their own websites to host studies.

Gorilla is an experiment-hosting platform for behavioral tasks that cannot be hosted in a survey. Gorilla has a graphical interface, which researchers can use to set up studies using a library of experimental tasks. Researchers pay per participant response.
No programming knowledge is required to use Gorilla.
Complicated tasks can require coding if they are not already included in the built-in libraries.

Inquisit similarly hosts complex behavioral tasks on its platform. Researchers pay for a license to conduct unlimited experiments for a certain amount of time.
It requires participants to download software on their own computer, increasing confidence that participants have a standardized experience.

Inquisit enables researcher to see if a participant has navigated off of the experiment window.

It can be cost effective for running a lot of experiments with a lot of participants.
Many participants will not be willing to download new software to complete a study.
Using Inquisit requires knowledge of computer programming, as researchers write their tasks in a language similar to JavaScript.
It can be expensive to use in a less active research lab.

Pavlovia is also an experiment-hosting site for complex behavioral tasks. It supports tasks written in Python or JavaScript using the libraries PsychoPy, jsPsych, or lab.js.  Researchers can choose between paying for a license or paying per participant.
It can be easy to use if you already have experience with Python or JavaScript.

Other researchers’ code is often public.

Uses open-source programming languages and libraries, which means researchers can save their code and host the experiments elsewhere in the future.
Pavlovia requires comfort with computer programming languages and tools.

Researchers have to do the bulk of coding themselves, which can be time consuming.

The second step is to recruit people to participate in the study. Researchers with funding to compensate participants will often recruit from online participant pools. Popular platforms include Amazon Mechanical Turk and Prolific. Researchers at academic institutions with an undergraduate participant pool can disseminate their online studies through platforms specific to their school, such as Sona Systems. Another option is to use social media to recruit individuals who are not signed up for a participant pool.

Most hosting platforms will store the data collected. Depending on the platform, data can be downloaded in a variety of formats, such as CSV and SAV.

Amazon Mechanical Turk (MTurk) is the most popular recruitment platform for academics. Amazon founded Mechanical Turk to establish a workforce to complete discrete tasks that computers are unable to do.
There are a lot of workers on MTurk, making it possible to collect hundreds of responses within hours.
MTurk does not have any restrictions on participant payment.
Researchers have raised concerns about bots on the platform.
MTurk offers limited prescreening options and researchers must pay extra to use prescreening criteria.
The interface can be difficult to use, so many researchers use a platform called Cloud Research (formerly TurkPrime) to recruit from MTurk.

Prolific is an online participant pool established to support academic research and used only for data collection.
The staff at Prolific closely manage the participant pool, including verifying the phone number on each account and strategically releasing studies to participant accounts.
Prolific includes an extensive prescreening questionnaire, which researchers can use to target populations at no extra cost.
Minimum compensation rate of £5.00 (or $6.50) per hour.
As participants only sign up for the platform if they’re interested in research studies, there may be concerns about participant non-naiveté.

Sona Systems is a participant management system. Schools or departments typically purchase and manage a site for its members to use. Institutions most frequently use Sona to manage undergraduate student participant pools, though some maintain a pool of paid participants as well.
Sona allows a lot of control over participant management.
Paid and student pools can be managed on the same platform.
With a license, unlimited studies can be run without paying commission for each participant.
Using Sona requires an on-site administrator to manage the website.
Finding participants to sign up for the Sona pool rather than individual studies can be time consuming. 
It is cost prohibitive for individual researchers to obtain a license.

Barbara Toizer is a PhD student at the University of Kansas.

Announcement Categories: