Safter scrolling: How algorithms popularise and gamify online hate and misogyny for young people

The research, conducted in partnership between UCL, the University of Kent and the Association of School and College Leaders (ASCL), found a fourfold increase in the level of misogynistic content in the “For You” page of TikTok accounts over just five days on the platform, in an algorithmic modelling study. Through interviews with young people and school leaders, the researchers also found that hateful ideologies and misogynistic tropes have moved off screens and into schools, becoming embedded in mainstream youth cultures.

The report authors stress the need for a 'healthy digital diet' approach to education to support young people, schools, parents and the community at large. They also say it is essential to champion the voices of young people themselves, particularly to include boys as part of discussions regarding online misogyny, and they suggest a 'peer-to-peer' mentoring approach.

The researchers began the study by interviewing young people engaging with and producing radical online content. This then informed the algorithmic study in the creation of archetypes, to represent typologies of teenage boys who may be vulnerable to becoming radicalised by online content. The researchers set up accounts on TikTok for each archetype, with distinct content interests typical of these archetypes (for example, seeking out content on masculinity or addressing loneliness), and researchers used these accounts to watch videos that TikTok suggested in its “For You” page, over a period of seven days.

Initial suggested content was in line with the stated interests of each archetype, such as with material exploring themes of loneliness or self-improvement, but then increasingly focused on anger and blame directed at women. After five days, the TikTok algorithm was presenting four times as many videos with misogynistic content such as objectification, sexual harassment or discrediting women (increasing from 13% of recommended videos to 56%).

The research team led roundtables and interviews with school leaders, who attested that misogynistic tropes are becoming normalised in how young people interact in person as well.

The researchers set out the following recommendations:

• Holding social media companies accountable, and applying pressure on them to address the harm caused by their algorithms and prioritise the wellbeing of young people over profit.
• Implementing “healthy digital diet” education, which considers different types of screen time and digital content young people are engaging with, akin to different food groups, considering how much of it is consumed, how it can become “ultra-processed” due to algorithms, and potential impacts on mental and physical health.
• Peer to peer mentoring, empowering older pupils to work with their younger peers, and helping to involve boys in discussions around misogyny.
• Promoting wider awareness of algorithmic processes among parents and the community at large.