APPENDIX 1 – METHODOLOGY – ADDITIONAL INFORMATION
The approach taken in this project is described in the Methodology section [LINK]. This appendix provides additional information on two issues: taking account of overlapping and intersecting personal characteristics, and specific aspects of the scoring system.
Taking account of overlapping and intersecting personal characteristics
Given that people’s experiences of care are informed by their overlapping and intersecting personal characteristics – for example, being a carer and being a woman – the assessment process considered how policies affect marginalised groups. Depending on the policy area these groups include:
- Women
- Young people and older people
- Informally employed workers
- Single-parent households
- Minority religious, caste, racial, and ethnic groups
- Disabled people
- People on low incomes
- Migrants, asylum seekers and refugees
- Homeless people
- People living in rural areas
- LGBTQIA+ people.
For each policy area, the research team looked for:
- Information on how policies ensure reach and impact for marginalised groups
- Intersectional data across protected characteristics (which, in practice, proved to be very limited)
- Qualitative research capturing the perspectives of marginalised groups.
In some policy areas, certain groups are given more attention. For example, the assessment of childcare policies gave greater consideration to single-parent households as a marginalised group, in part in recognition of their status as an identified “priority group” within the Scottish Government’s efforts to reduce child poverty.
The assessment criteria set out in the global template also provides focus to those working in informal employment. Recognising the nature of paid employment in Scotland, we took a broad view on this to incorporate informal workers and those in precarious employment such as zero-hours contracts, or short-term or insecure contracts.
Scoring process
By design, a scorecard approach tends to seek binary numerical answers, and this can sometimes miss nuance. To address this, a consistent approach was applied to scoring across all indicators, although some assessment criteria required a specific approach, as outlined below.
The budgeting and administration assessment theme included a standing assessment criterion for all indicators relating to there being adequate human resources and technical capacity for implementation of the policy. As there was not always evidence of this, we often had to make an assumption that capacity did exist. As such, this criterion was scored in the following way: 1 if there was clear evidence that the area is well-resourced, 0.5 if an assumption was made or there was evidence of limitations to the capacity, and 0 if there was clear evidence of a lack of capacity.
The design and impact assessment theme asks whether women’s rights organisations and carers’ organisations were consulted about policies. This wording does not capture whether consultation was meaningful – that is, whether there was ongoing engagement and co-design with women’s and carers’ organisations. To reflect this, a score of 0 was given where there was no evidence of consultation, 0.5 was given where consultation took place, and 1 was given where there was evidence of meaningful consultation.
Also, within the design and impact assessment theme a criterion focuses on the level of women’s representation. Here, the assessment was based on publicly available information about women in senior management and governance roles. However, this does not indicate whether those in leadership positions have gender equality competence (the skills, attributes and behaviours that people need in order to mainstream gender concerns effectively into policies).
Assessment process
SWBG undertook the initial scoring across all indicators in the scorecard using publicly available evidence to justify the score.
Following this process, the scoring went through a sense-checking process with both the steering group of the A Scotland that Cares campaign and with organisational experts on the indicator topics. Three focus group discussions were also held with unpaid carers, young carers and parents, as part of the sense-checking process.
Through this process, the scores were refined and a review was undertaken to ensure consistency of approach across indicators.
While we recognise that the assessment process was necessarily limited by available resources and evidence we believe the sense-checking process increased the robustness and legitimacy of the Scorecard findings. That said, future iterations should seek to learn from this inaugural process to further strengthen the Care Policy Scorecard for Scotland.