6. Measuring changes in norms of masculinity

There is a lack of evidence on to what extent programmes that engage men and boys have an impact on their lives, or on the lives of women and girls at the household, community and structural level. The evidence that is available focuses largely on interpersonal issues and evaluation approaches tend to be quantitative, short-term and instrumental. There is a pressing need to test the effectiveness of programmes that engage men and boys for gender equality and equity (see EMERGE Evidence Report, 2015).

Programme evaluations do highlight a shift in attitudes as a result of activities around gender norms. However, they are unable to capture and highlight any resultant changes in behaviour – if any such changes occur. This is because of their short-term nature, as well as the challenge of measuring changes in patriarchal norms that are deeply entrenched. In addition, evaluations tend to be carried out immediately after a programme ends, and often fail to capture changes that occur over time as a result of its activities.

Programmes that work to engage men to change gender norms fail to measure outcomes for women. This is a major gap that makes it difficult to determine whether male engagement efforts change the lives of both men and women. Existing evaluations of outcomes for women have focused largely on outcomes related to health and violence reduction. It is important, therefore, to ask men and boys about their behaviour and then triangulate their responses with those from women and girls. This can be achieved through qualitative measurement.

Despite the lack of rigorous evaluations of male engagement programming, there is a growing body of expertise and knowledge about what works well and what does not in terms of outcomes for women and girls. The best-known tool for measuring gender attitudes is the Gender Equitable Men Scale (GEMS) developed by Population Council, Horizons and Promundo. This scale measures attitudes around gender norms in relation to violence, sexual relationships, homophobia, domestic chores and daily life, and reproductive health and disease prevention. It provides information about the prevailing norms in a community as well as any changes in these norms as a result of a programme.

The GEMS scale has been widely adapted for use in health, education and violence prevention programmes to measure attitude change in boys once a programme’s activities have been completed (see Singh et al., 2013). Initially developed for young men aged 18-29, the scale has also been adapted for use with different age groups between 10-59 years of age and has been used in middle- to high-income communities in various countries. The IMAGES survey was based on the GEMS scale, which was adapted to account for the contexts of different countries.

Another key tool is the Partners for Prevention toolkit for ‘Replicating the UN Multi-country Study on Men and Violence: Understanding Why Some Men Use Violence against Women and How We Can Prevent It’. This toolkit consolidates the methodology, learning and resources developed for the implementation of the original UN Multi-country Study on Men and Violence, conducted by Partners for Prevention between 2010-2013 in seven countries across Asia and the Pacific. It provides a comprehensive, step-by-step guide to using the methodology in any setting to conduct rigorous research with men on violence against women to inform prevention. The toolkit explains the importance of every step in each phase and, where relevant, directs the reader to the corresponding tool.

The randomized control trial (RCT) conducted in Rwanda to assess the impact of the gender-transformative Bandebereho intervention for couples on multiple behavioural and health-related outcomes is an important tool for the study of the outcomes of interventions that engage men for gender-norm transformation.