- What is the actual reach of these programmes, among remote areas and different generations. And how is impact being measured?
- Do you work with media outlets in your target countries to produce content that combats gender stereotypes, and if so - how?
- I'm curious about the challenges faced by each projects and how PFI3/BBC/Girl Effect address them.
- Would be interested to hear any tips you have on evaluating norm change programmes - especially when dealing with barriers on social desirability bias and self-reporting - is there anything that works particularly well in your experience for objectively evaluating programmes?
- Do you have any rules of thumb for how long social norms take to change? Some say that it takes at least a generation before deep gendered norms to change. But some change can be quicker - for example recent changes on abortion in Ireland. What's the panel's view?
- What is the tipping point when individual attitude transformation becomes norms change?
- I would also be interested to understand how the panel considers the role social movements can play in influencing and changing gender norms?
1. What is the actual reach of these programmes, among remote areas and different generations. And how is impact being measured?
OB: One example shared by the panel is Girl Effect’s Malawi Brand, Zathu. 6.7m girls are exposed (estimated 64% of the population), and 4.5m annual reach.
- 700k girls
- 1m boys
- 2.8m adults
2.6m regular reach
We measure impact through measuring shifts in knowledge, attitudes and behaviours through both quantitative and qualitative methods. We also measure attitudes to gender norms that give us an indication of social norms shifting as well.
PM: Television is a powerful medium for social change, and its reach in India has been rapidly increasing. The number of households owning a television increased from 32% in 2001 to 47% in 2011 (Census). According to IRS 2017 data, overall household television ownership in India has risen from 47 percent in 2011 to 61 percent in 2017. TV accounts for more than half, or 2 hours 11 minutes, of daily time spent with all major media , and reaches 75 percent 12+ individuals across the country. Doordarshan, the national TV channel of Government of India operates 23 satellite channels providing TV coverage to about 92 percent of the country’s population besides providing free-to-air DTH service.
Launched in 2014, Main Kuch Bhi Kar Sakti Hoon – I, A Woman, Can Achieve Anything (MKBKSH), a transmedia edutainment intervention of Population Foundation of India (PFI) has completed two successful seasons and was broadcast on Doordarshan and All India Radio. PFI has launched Season 3 of MKBKSH on Doordarshan on January 26, 2019. The programme was also dubbed in 13 languages and aired on 16 regional centres of Doordarshan in first two seasons. The same content was aired on 216 All India Radio stations. The programme was re-telecast on Doordarshan subsequently. The programme was also aired by DD India in more than 50 countries in West Asia, Far East, Canada and Europe. The airing of the programme on Doordarshan and All India Radio has helped the programme to reach to remote and hard to reach areas.
MKBKSH has also leveraged the digital media platforms to reach out to youths across the country. Season 3 (first 26 episodes) has achieved the following impressive numbers already.
- 2.4 million video views on YouTube
- 25+ million impressions on Facebook
- 310,000+ impressions on Instagram
- Reach on Twitter about 100 million
- #MKBKSH reach 894 million
Season 3 of MKBKSH has received 281,000 + calls on Interactive Voice Response System (IVRS) platform so far. Nearly 2,000 calls have been received per day. The serial received approximately 1.7 million calls from viewers on IVRS platform in Season 1 and 2 together.
Advocating the issues showcased in the MKBKSH series, five films were produced and released as a series – ‘Reel to Real’ - from MKBKSH social media platforms during the year 2017. These films are based on inspiring role models, both women and men, who got inspired by MKBKSH and are leading change or have challenged social norms in their respective communities.
The stories integrated Dr. Sneha, the central character of MKBKSH and received very good response. By the end of September 2017, the five films together were viewed by more than 20, 31,500 and reached more than 5,750,000 digital users through Facebook, Twitter and YouTube.
The story of one of the Reel to Real heroes – ‘Nirma Devi’ – was chosen for inclusion in BBC’s 100 Women initiative. The initiative names 100 influential and inspirational women around the world every year. In 2017, it showcased those who tackle four of the biggest problems facing women today - the glass ceiling, female illiteracy, harassment in public spaces and sexism in sport. The story on Nirma Devi highlights how she was inspired by MKBKSH to spread the word on contraception within her community, going against established social norms: ‘100 Women: Breaking the contraception taboo in India’.
The other such story is from Chhatarpur in Madhya Pradesh (MP) - Men of Chhatarpur: Men must play an equal part in women's empowerment, and this group of men in Chhatarpur, MP, have done just that after watching MKBKSH. Habitual wife-beaters have turned into empathetic partners and those who only wanted boys have gotten sterilised even after one girl child. This 360 degree transformation occurred because local NGOs used the medium of MKBKSH to show a mirror to this group of men on their faulty behaviours. These men now sing their own songs and spread the message of women's empowerment and gender justice across villages. They have made small changes in their lives, which include helping their wives with daily chores. One of the men quips, "I may even cook better than her!"
The impact of the programme was measured for Season 1 and 2 through a third-party independent evaluation. The evaluation used mixed methodology, comprising quantitative and qualitative surveys. To complement the quantitative survey, a qualitative component was added, covering all the stakeholders through Focus Group Discussions (FGDs) and Key Informant Interviews (KII). The research design was retrospective, counterfactual and quasi-experimental in nature with Propensity Score Matching* (PSM). ie, an international agency known for impact evaluation provided the technical support for impact evaluation. A robust concurrent monitoring and evaluation plan is in place for Season 3. The most exciting piece is tracking of conversations around markers used in the serial on various digital platforms.
*The Propensity Score Matching (PSM) is a statistical matching technique that attempts to estimate the effect of a treatment, policy, or other intervention by accounting for the covariates that predict receiving the treatment.
2. Do you work with media outlets in your target countries to produce content that combats gender stereotypes, and if so - how?
CS: Partnering with broadcasters around the world who want to develop the way they approach gender issues - both within their organisations and across their schedules – is a key part of our work. Across the world we see that if broadcasters want to serve their diverse audiences well, they do best when they reflect that diversity in their own organisations. So we work with our partners on institutional issues related to women working in the media, such as management structures and decision-making, the allocation of roles and training and safeguarding and safety.
We also support them to plan, make and review content that that includes women’s voices, meets women’s needs for information, and reflects women’s lives, in all their diversity. In Afghanistan for instance we are currently supported by Global Affairs Canada to work long-term with FM radio station partners, and an Afghan organisation (AEPO) that produces a long-running and highly popular, radio drama.
PM: Population Foundation of India uses a 360 degree approach to Social and Behaviour Change communication. PFI adopted the approach to ensure that the series reaches to and makes an impact on as many people as possible. This is based on the idea that attitudinal and behavior change is more likely if individuals hear mutually reinforcing messages from a variety of different channels. So, the intervention used TV, radio, community radio, Interactive Voice Response System (IVRS), digital and social media and community outreach in two states.
PFI also had an 18-month pilot digital campaign, `Bas Ab Bahut Ho Gaya – Enough is Enough’ in partnership with director Feroz Abbas Khan and Farhan Akhtar’s initiative MARD, to mobilize the youth to address issues around gender-based violence (GBV), particularly violence against women and girls (VAWG). The campaign is part of the project `Combating Gender-based Violence among the Youth in India through a Celebrity Centered Digital Media Edutainment Intervention’.
#BasAbBahutHoGaya - #EnoughIsEnough was kicked off with a Facebook Live session by Farhan Akhtar and was aired on the Facebook pages of both PFI and MARD. The Facebook Live received more than one million views and an engagement with 25,000 people. The six films produced under the campaign have got 23 million impressions on Facebook and Twitter and more than 6 million views on YouTube and Facebook as of April 2018.
PFI has collaborated with media outlets, to create content that challenges the gender stereotypes. The op-eds, interviews, blogs and other compelling content (over 5000 articles) have been placed in online and print media in English and regional languages.
3. I'm curious about the challenges faced by each projects and how PFI3/BBC/Girl Effect address them.
CS: One of the biggest challenges is to really understand our audiences and carefully diagnose social norms so that we can address them in out work. We do this by investing in formative, qualitative research to help us get to the heart of what people are really experiencing – bearing in mind that on some topics people tend to give socially-desirable answers. We try to overcome this by using observational research, immersion visits with communities and qualitative research across reference groups. We also do a lot of work on building the understandings of our own staff and partners of norms, what they look like and why they matter. And the other challenge is to balance between tackling norms and keeping our programming and characters realistic and relatable.
OB: Some of the challenges that Girl Effect faces:
- Making content adequate for our target audience (usually adolescent girls between 15 and 19) but also ensure that messages are relevant to the overheard audience such as boys, parents, teachers, community leaders and that therefore can trigger broader community’s discussion. We address this issue including a wide range of characters in our dramas, shows, etc.…that can model behaviours we want people to emulate, as well as though some in-person engagement activities that target others beyond the girls.
- With an increased focus on digital channels, we find ensuring digital safety quite challenging. The digital space is quickly changing and we are currently investing in training all our safeguarding officers/focal points on digital safeguarding. Similarly we are putting a lot of effort in training and appointing moderators to moderate and monitor online conversations and comments.
- Another challenge in using digital channels, is to ensure that these allow us to reach a critical mass for social norms change. This is why Girl Effect ‘uses’ Girl Champions as mobilizers, that share content, new ideas and information with other girls who might or might not have access to the same digital channels.
4. Would be interested to hear any tips you have on evaluating norm change programmes - especially when dealing with barriers on social desirability bias and self-reporting - is there anything that works particularly well in your experience for objectively evaluating programmes?
CS: We start our work by investing in formative, qualitative research, which accounts for social desirability bias with observational research, immersion visits with communities and qualitative research across reference groups.
We take a mixed-methods approach to measuring impact – involving both quantitative and qualitative research. We regularly conduct nationally representative surveys to measure our impact, while controlling for age, gender, location, education and economic status. We’ve also experimented with using randomised control-trials, complemented by qualitative research.
The types of things we try to measure in our evaluations include shifts in norms themselves; attitudes towards norms / acceptance or rejection of norms; attitudes around the values that underpin norms; behaviours through which norms sometime manifest themselves, although we know that behaviours can sometime change without norms changing).
We’ve learnt a lot but establishing research approaches and tools that really work for social norms measurement in different contexts can remains a challenge that we need to work on together as a sector. We’ve found the work of Gerry Mackie and Holly Shakya at University of California, San Diego really helpful here. Specialist academic input is really valuable to us in this work.
In social norms research we’re trying to understand what people think others do and what people feel is expected of them. In our experience, both things can be hard for people to talk about. For example in Ethiopia women we talked to felt it was impolite to talk at all about what they thought other people did. So we need techniques to overcome barriers like this. And in social norms research we often want to carry out research across reference groups. So if we want to do this measurement properly we need to find the time/resources to map reference groups and carry out research across them.
PM: Measuring norm change programmes is challenging for two reasons: first, there is a perception that norms are unquantifiable; second, because data collection across various contexts is too patchy to be meaningful to be representative of all. Measuring norms is difficult, due to prevailing social barriers and taboos. The problem also lies in the under-reporting. It is also about providing safe and confidential environment for interview and asking the right questions. The change in norms and behaviour require sustained intervention over a longer period of time.
The evaluation of Main Kuch Bhi Kar Sakti Hoon brought out important data and insights on attitude and perception with regard to social norms, which were relevant and helpful in strengthening and revising programme and organizational strategies.
The systematic use of markers (popular phrases) represents an unprecedented “game-changing” opportunity in the design, implementation, monitoring, and evaluation of Entertainment Education programs. PFI has used the markers for the first time in an entertainment education programme (MKBKSH) under the guidance of Prof. Arvind Singhal. The conversations around identified markers were tracked on TV series, IVRS and digital platforms to measure the shift in social and cultural norms.
Markers such as Mastbandi—a new term and a new pleasurable norm for vasectomy—were introduced in MKBKSH-1 to shift the perception of nasbandi, the existing normative term referring to male vasectomy (literally, “closing of pipes”). Post-broadcast surveys after Season 1, showed 22% of the MKBKSH-1 audience had heard of mastbandi, and over 95% of those who had heard, knew exactly what the new marker meant.
The markers/phrases that denote new norms used in serial, IVRS campaigns and social media promotion in Season 3 (example).
- Lambi Sagai (Extended period of courtship: 7+ million reached; 2 million engaged
- Laadli Din (Daughter’s Day): 5+ million reached; 10000+ calls on IVRS
- Kahani Badlo (Flip the Story): 7.7 million reached: 100,000+ engaged
- Dono Barabar (Both are Equal): 7.2 million reached; 48,000 engaged
An independent evaluation of `Bas Ab Bahut Ho Gaya’, a 10 months long campaign of PFI on digital platforms on gender based violence, especially against girls and women, produced crucial data to understand the effectiveness of the campaign, primarily on knowledge, attitudes, beliefs, and intentions related to GBV and VAWG among the campaign’s target audience on digital platforms However challenges in recruiting online survey panel limited generalizability of the findings to a broader population.
Various target audiences have varying levels of openness and willingness to address certain issues, especially such as sex and family planning. Data from formative research on knowledge, attitude and perceptions of the people is therefore important and instrumental and policies and programmes should be designed bearing them in mind. Also, socio-economic category, education, gender, age etc. of the target audience should also be borne in mind.
While data is extremely crucial, it is also important to remember that measured changes are a combination of several factors (such as improvement in services, better trained staff etc.). It is therefore necessary to understand what works and what doesn’t in different contexts.
5. Do you have any rules of thumb for how long social norms take to change? Some say that it takes at least a generation before deep gendered norms to change. But some change can be quicker - for example recent changes on abortion in Ireland. What's the panel's view?
CH: We suspect norm change is indeed generational as it requires changes in both behaviours and attitudes to be sustained over time. Many achievements can be noted on the way. Laws may be ahead or behind the changing norms. For example laws may rule that women can vote, whilst many men and women continue to disagree with this. Conversely women may want fuller control over their bodies with the support of men, but the state influenced by important institutions, may lag behind. The building blocks for change depend on local contexts, but we do know that social movements and law makers play important roles, alongside individual actions. Look out for ALIGNS new series on history, generations and norm change to be published in July.
6. What is the tipping point when individual attitude transformation becomes norms change?
CH: There is a popular book called The Tipping Point by Malcolm Gladwell (2002) which draws on behaviour change theory to shift attitudes and to ‘nudge’ (also a book, by Richard Thaler, 2009) people into certain patterns of behaviour. These seem to work well in areas where individual actions can be taken without constraint from others. For example giving up smoking, wearing a seat belt in a car or not spitting in public places. Of course there is peer pressure or bravado, but there are not hotly contested and enduring power relations to be negotiated. In issues of gender norms personal power over others or assets or political space usually lies at the centre of resistance and changes require shifts in multiple layers of society. It’s a mistake therefore to only focus on what individuals can do to shift gender norms. Other institutions (formal and informal – church, tribe, clan, government, groups) have to visibly shift and allow individuals to change, alongside the individuals themselves learning and being supported by social movements and legal provisions.
OB: I would say that attitude transformation should generate some level of behavioural change. Then, when a critical mass takes up new behaviours and attitudes and there is public knowledge of this, that is when you have a norm change. The critical mass doesn’t need to be the majority of the community members but it’s of course difficult to precisely say how many people represent a critical mass.
7. I would also be interested to understand how the panel considers the role social movements can play in influencing and changing gender norms?
CH: Social movements are seen to play a huge role in changing gender norms, be they local small groups of people committed to change or global movements such as those emanating from Beijing in 1995 or the more recent #MeToo movement. It’s hard to judge the power of social movements as they work alongside powerful individuals and role models and law makers to advance gender equality. They also provide a supportive framework for individuals to make change in their own, and others, lives. We know that people change when able to talk through attitudes and behaviours with peers in a non- judgemental and non-threatening environment. Social movements can provide this context as well as a sometimes necessarily more confrontational approach.