Measuring digital inclusion in programs and initiatives

The term digital inclusion refers to our relative opportunities to access, use and contribute to digital resources and life online. It can be about: • access to, and accessibility for all of, digital devices, the Internet and digital technologies • online connectivity including the quality and speed of connection • affordability such as costs of devices, online connection and data usage • digital skills, digital ability, information literacy and digital capability • online safety and privacy • beneficial outcomes of using the Internet.

To help those working to support digital inclusion – particularly those seeking to track and measure the impact of their initiatives and activities – we have summarised commonly used key concepts below, to help understand, identify, select and develop relevant measurement tools.

The digital divide

The digital divide concept was introduced in the early years of the development of the internet to identify relative digital gaps for people using it. The pace of change of technology means the shape and characteristics of the digital divide are dynamic. The most commonly recognised gaps include: access to and accessibility of devices and the internet (usually labelled as ‘the first-level digital divide’), digital skills and use of the Internet (‘the second-level digital divide’), and beneficial outcomes of using the internet (‘the third-level digital divide’). While new forms and perspectives of the digital divide are emerging, such as differences in algorithmic awareness and data inequalities, common measures for digital inclusion or digital divide are centred around these three levels.

Digital skills, information literacy and digital capabilities

In practice, digital skills, information literacy and digital capabilities often are used interchangeably. Digital skills can be medium-related (operational and software skills), content-related (such has information skills, creative skills, and social skills), knowing how-to’s around online safety and security, and general internet skills and digital literacy.

An alternative framework groups internet skills to five categories: operational, information navigation, social, creative and mobile. The Australian Government has developed the Australian Digital Capability Framework, which contains 21 components for digital capability across five digital focus areas, to guide digital training activities. If digital skills development or capability building is a core impact area of the program you would like to assess, you may consider specific measures to assess what people learned from your training activities.

Digital benefits and digital outcomes

Digital benefits can be understood as offline benefits of online activities. This aspect highlights that digital skills and usage do not necessarily lead to equal outcomes for all. Consequently, differences in offline outcomes of using the internet is termed the ‘third-level digital divide’ as mentioned above. Digital benefits for people can be evident in a range of areas, from being more socially connected, to better access to work and opportunities to create business or cultural content online, civic participation, to improved educational and health outcomes. Digital outcomes can be negative or positive. While the third-level digital divide focuses on differences in benefiting from internet use, there can also be negative impacts on people and organisations from internet use (such as experiencing online bullying or cyber-attacks), related to relatively poorer digital skills or infrastructure.

What to consider when developing or selecting digital inclusion measures

Digital inclusion is usually measured at the population level drawing from large-scale household surveys or censuses (for example, the Australian Digital Inclusion Index), or using geographical data collected by infrastructure providers (such as the NSW Digital Connectivity Index). For organisations, communities, governments and academics working to close the digital divide, being able to measure and communicate digital inclusion impacts at a program level would help gain insights into implementation and outcomes of the change made.

When measuring digital inclusion at a program level, consider the following questions:

· What is your primary purpose of measuring digital inclusion?

· How would you like to use the findings? Who is your audience/s and what do they need to know?

· Do you have an impact framework or theory of change to articulate how to create digital inclusion impact through the program (see What is a theory of change and why does it matter?)?

· What mechanisms or interventions does your program or initiative apply to addressing the digital gap? For example, providing device-based supports, developing digital products or delivering digital training activities.

With support from the auDA Community Grant Program, Seedkit has developed a set of digital inclusion indicators for organisations to measure, track and communicate their digital inclusion activities and impacts. These indicators can be used to align with your existing theory of change.

If you are at the start of an impact measurement journey, you can explore Seedkit’s digital inclusion indicators in our Indicators Library to help you think through potential impact pathways. There are also other online resources to assist you in this journey, such as Steps to take to prepare for evaluation, and Tips for measuring and communicating Impact.