How can governments make the most of data?
By Talend
Jimmy Kwang, Talend’s Regional Vice President of Sales Asia, discusses why data integration is necessary today.
The Gendarmerie Nationale set out to automate the processing of personal data to identify dangerous individuals. It integrated and cross-referenced security information collected by different agencies into a centralised platform, and sped up screening of identities from 300,000 identities each year to one million each month.
Being able to integrate and analyse from multiple sources is essential, not just for matters of national security. GovInsider spoke to Jimmy Kwang, Regional Vice President of Sales Asia at data management software company Talend, to find how governments can get started.
Why is it necessary?
We’ve heard of how data is generated in masses, but exactly how much is being produced every day? According to the International Data Corporation, 463 exabytes of data will be created every day globally - the equivalent of 212,765,957 DVDs per day.
“A consolidated perspective of various data sources” is key as the world’s data engine keeps churning, says Kwang. Aggregating data into a common format will help governments achieve better engagement with constituents - whether it’s citizens, agencies, or other governments.
“The ability to integrate into one common platform is crucial,” Kwang says. “Otherwise, the ability to drive value out of those data is going to be severely diminished.”
A key ingredient: trust
The key ingredient of trust cannot be missed out even as data integration occurs, says Kwang. That’s a “fundamental challenge” agencies face in sharing and integrating data, he adds.
If the data source is untrusted, a lot of validation is required, he says. Some transposing errors such as missing numbers or spelling errors may occur. There may also be differences in how data is formatted.
“If there is no common framework, then whoever receives the data has to work to transform it and check that the right fields are being populated in the right manner,” Kwang adds. “The ability to have a platform to help you integrate across the enterprise and multiple entities will help.”
Singapore has designated four trusted data centres to be data intermediaries for individual, business, geospatial, and sensor data, Quek Su Lynn, Director of Government Data Office at the Smart Nation Digital Government Office, told GovInsider.
The centres aggregate the data and distribute it securely to user agencies. Public officers can only access and analyse de-identified data via central platforms that have been designed with the requisite safeguards.
Integrate data into one platform
A “platform approach” gives agencies flexibility in managing data - whether it’s in a hybrid environment , or on public or private cloud platforms, says Kwang.
Data integration is at the heart of Talend’s Data Fabric platform. It enables governments to integrate data from any source and type, to any destination. The data fabric also has built-in data quality capabilities to ensure data is usable from day one.
In the data integration process, it’s key to integrate only the necessary data. “You don't want to integrate a whole lot of data for no reason,” Kwang says. Security teams, for instance, may integrate masses of logs for analysis - when only 1 per cent of it shows the anomaly.
“It’s about having a seamless way to help consumers of data look at what's relevant to them, in the right manner and in the right time,” he adds.
Create the right structures and roles
Data integration is not just about the tools and technology, but also about the people, says Kwang.
He recommends agencies to “take a holistic view” to understand what data management roles are needed and how to delegate the responsibilities. People can then work in harmony with the technologies to better integrate data, he adds.
Together with trust, a central platform, and the right roles in place, governments are well-positioned to make the most of data and create better policies for citizens.
To find out how your agency can enhance data management, download Talend’s guide here.