A teen on TikTok disrupts thousands of scientific studies with a single video

A 56-second TikTok video by a teenager caused thousands of scientific studies to juggle weeks of data.

The video for July 23 is short and simple. It opens with recent Florida high school graduate and self-described “teen author” Sarah Frank sitting in her bedroom and smiling at the camera.

Pointing to users on the website Prolific.co, she says in the video, “I recommend trying – welcome to Side Hustle.” “Basically, it’s a bunch of surveys for different amounts and different times.”

That video garnered 4.1 million views in the month after it was posted and thousands of new users flooded the Prolific platform. Prolific, a tool for scientists conducting behavioral research, had no screening tool to ensure that it distributed representative population samples for each study. Suddenly, as scientists began receiving a wide mix of subjects for their prolific studies, their surveys were flooded with responses from young women around Frank’s age.

For researchers relying on representative samples of the US population, demographic change was a major problem with no obvious cause and no immediately obvious way to fix it.

doing science on the internet

Although not particularly well-known, Prolific is part of a small collection of online tools that have changed the way corporations and scientists study the way people think and act. The first and largest of these research platforms is Amazon-owned Mechanical Turk, released in 2005 as a general-purpose platform for crowdsourcing work on repetitive tasks. Soon after its release, behavioral scientists realized its potential value for their research, and it rapidly revolutionized many research areas.

“Before Mechanical Turks existed, all social science research had to take place in the laboratory. You had to bring in college sophomores and teach them through questionnaires and surveys and what not,” said Nicholas Hall, director of the Behavioral Lab at the Stanford School of Business. have to keep.”

“It’s a huge time- and labor-intensive effort. Online research makes it so easy. You program a survey … you put it online, and within a day, you have 1,000 responses.” ,” Hall told ledge. “It changed the face of social science.”

Hall said the Behavioral Lab at Stanford primarily uses the newer, smaller Prolific platform for online studies. While many Mechanical Turk customers are large businesses doing corporate research, Prolific tailors its product to scientists.

The smaller platform offers greater transparency, promises to treat survey participants more ethically, and higher quality research topics than alternative platforms such as Mechanical Turk.

Scientists conducting this type of research in the United States typically want a pool of subjects who speak English as a first language, don’t have a lot of practice in taking psychological surveys, and together have a fair share of the American population. Representative demographics form the sample.

Vipul, most agreeable, did a good job in providing high quality themes. The sudden change in the platform’s demographics put that reputation in jeopardy.

after tiktok

In the days and weeks after Frank posted his video, researchers scrambled to figure out what was happening in their studies.

A member of the Stanford Behavioral Laboratory posted on the Prolific forum, “We’ve seen a huge jump in the number of participants on the forum in the US pool from 40k to 80k. Which is great, however, now that a lot of our studies have gender disparity.” Where probably 85% of the participants are women. Also the average age has been around 21.”

It seems that Wayne State psychologist Hannah Schechter was the first to settle the case.

“This may be far-fetched,” she tweeted, linking Frank’s video, “but given the timing of the video, the virality and the user’s follower demographics…”

Prolific survey takers have long complained on Reddit that Frank’s makes it difficult to find paid surveys to take on the Overrun platform.

“Now this is just another crap site to spend hours and make money,” wrote one user, who said he previously made $30 a week on the platform.

Frank, who “estimated” that he took a total of $80 to poll on Prolific prior to his video, pointed out ledge He also noticed a difference on stage.

“There is less study available to me and everyone else,” she said. ledge. “I’ve received some very mean comments accusing me of ruining the site and being selfish – even though I didn’t receive any compensation for that video.”

He said he hopes Prolific will be able to put in place a system to deal with its changed demographics.

“I also predict that many people who sign up after watching my video will forget it, and the boom will die out,” she said.

Prolific co-founder and CTO Felim Bradley explains ledge Looks like a lot of new users are turning off.

“Prior to TikTok, about 50% of the responses on our platform came from women,” he wrote in an email. “For a few days the surge raised it to 75%, but since then, that number has been going down, and we are currently getting ~60% of the responses from women.”

Courtesy of Vipul

According to Bradley, about 4,600 studies were intercepted by Frank’s TikTok, which was about a third of the total active on the platform during the boom. Of them, he said, the vast majority must be saved.

Prolific is back to researchers whose studies were heavily influenced by the increase in female survey takers and introduces a new suite of demographic screening tools. The company announced these moves a month after Frank posted his video. The company has now re-organised a team in charge of demographic balancing in order to more quickly identify and respond to this type of problem in the future.

“Honestly, we were somewhat surprised, and we didn’t predict how big the impact would be,” Bradley said.

The boom isn’t all bad. Refreshing the pool of surveyors probably has long-term benefits, says Vlad Chituk, a Yale graduate student in psychology who ran several pilot studies on Prolific when the boom occurred. When subjects take a lot of psychological surveys, they learn tricks scientists use to collect data, and this may affect the way they answer future survey questions. Fresh Topics provide high quality data.

“Young women who enjoy TikTok are people too,” he said.

As for Frank, she says that her Side Hustle video is by far the most popular TikTok she’s ever posted.

“It certainly didn’t occur to me that the video would blow up. I just posted it for my friends and followers, not for the reach it’s getting,” she said. Gone because the site is really cool, and people love efficient ways to make money.”

For the time being, much of his own favor has been withheld as Frank settles into his freshman year at Brown.