Using social media to track sustainable development

Many governments in the region struggle to keep their data up to date. Often, they are using paper-based systems that are too slow to upload into online systems or mash together with other pieces of information. But accurate data is crucial for measuring everything from food prices to social development.

Pulse Lab Jakarta, and their partners across the region, have created methods for agencies to use social media to gather accurate data - without investing in massive new IT systems or upgrading their existing processes. It's a short cut to gaining better understanding of national success.

Here are six methods that have proven successful:

1. Nowcasting food prices with Twitter

Early detection of food price anomalies is critical for timely action by governments.

Partnering with Bappenas (the Indonesian Ministry of Development Planning) and the World Food Programme, Pulse Lab Jakarta developed a statistical model to extract prices for four food commodities (beef, chicken, onion and chilli) in Indonesia from public discussions on Twitter.

When the modelled prices were compared with the official food prices from the Indonesian Government, released later, the figures were closely correlated.

These findings demonstrated that near real-time social media signals can function as a proxy for daily food price statistics, and an early warning mechanism for price fluctuations.

This approach is particularly useful to citizens and other stakeholders alike given that the official figures tend to be released to the public, and shared across government with some time lag.

2. Estimating socioeconomic indicators from mobile phone data

Building on past studies which show that data from mobile phones (in particular from call details records and airtime credit purchases) can be used to understand socio-economic conditions, we conducted research into the potential of using mobile phone data to produce a set of proxies for education and household characteristics.

Using anonymised mobile data from the Pacific island of Vanuatu, proxies for four types of statistical indicators were extracted from mobile phone data that was made available by a local carrier.

These indicators included education, household assets, household expenditure, and household income. The findings confirmed a relatively strong correlation between indicators from the mobile data and data from the official statistics provided by the National Statistics Office in Vanuatu.

3. Income mapping with financial data

Gender inequality is manifested in many aspects of life. We partnered with the UN Capital Development Fund (UNCDF)’s SHIFT Programme to analyse anonymised financial records from four financial service providers in Cambodia in order to investigate the factors affecting savings and loans mobilisation, with a focus on gender disaggregation.

Our data analysis suggests that despite the fact that men and women have equal access to credit and savings services in the region, the actual usage patterns in terms of the loans and savings amounts mobilised are much lower for most women.

The data also enables more powerful insights, not just breaking down data simply gender, but doing further gender disaggregated data analysis by age categories, marital status and geographies (rural, urban, and between provinces).

4. Proxy indicators of human development index based on Tweets

Gaining an up-to-date snapshot of development progress within countries is useful for tracking the effectiveness of national policies.

Pulse Lab Jakarta teamed up with Carlos III University of Madrid and UNICEF to explore the potential of utilising social media to produce a proxy indicator of the human development index.

For this research, we analysed tweets coming out of the islands of Java and Sumatra in Indonesia, and employed similar methods used for inferring unemployment rates in Spain.

The results from this research revealed that the predicted human development index fitted closely with the actual human development index, thus pointing to the precision of the model used.

This encouraged us to expand the approach to other countries including Brazil and Mexico, both of which showed relatively consistent results.

5. Commuting statistics from social media

Jakarta is renowned for its heavy traffic which makes daily commutes a lengthy task, and tracking the commutes of Greater Jakarta’s 30 million residents is an even more mammoth task.

To tackle this data problem, we decided to analyse GPS-stamped tweets. Along with Sekolah Tinggi Ilmu Statistik (STIS), we initiated a project to test whether location information from social media on mobile devices could reveal commuting patterns.

The project calibrated the initial result based on the population distribution and Twitter user distribution. Next, we verified the results with the official commuting statistics produced by the Indonesian Bureau of Statistics.

The results of the research confirmed that geo-located tweets have the potential to fill existing information gaps in the official commuting statistics.

6. Climate anomaly impact tracking

Our team has been analysing satellite imagery to map locations with climate and rainfall anomalies and to provide early warning alerts to policymakers.

Our VAMPIRE (Vulnerability Analysis Monitoring Platform for Impact of Regional Events) platform provides map-based visualisations and features three main layers: a baseline data layer (population data, socio-economic and food security surveys), a climate layer (rainfall anomaly, standardised precipitation index and vegetation health index) and an impact layer (economic vulnerability and exposure to drought).

The benefit is to provide real-time awareness of the evolving nature of slow onset climate phenomena, which can then be used to better channel assistance to vulnerable populations.

The working prototype is available here. The examples above represent a snapshot of our overall work related to the SDGs. To better address the challenges that we face today as a global community, we recognise that everyone needs to do their part.

For this reason, we are keen on fostering partnerships with governments, private sector and civil society entities on a range of topics linked to the Global Goals.

Furthermore, as new and powerful technologies continue to emerge with the potential to offer new insights, our mission is to continue leveraging these innovative tools in conjunction with traditional data in order to inform action to achieve each of the SDGs sub-targets by 2030.