How to empower self-service analytics in government
By Informatica
Interview with Daniel Clarke, Head of Big Data & Hybrid Cloud APAC, Informatica.
Organisations often rely on IT teams to run their data analytics. But this takes more time, and could even result in inaccurate reports when communications go wrong.
The solution to this? Empower each public official to run their own analytics. Daniel Clarke, Head of Big Data, IoT and Emerging Products APAC at software and data company Informatica shares his vision for self-service analytics, and how tech can enable that.
Self-service analytics
Traditionally, the IT department produces all of an organisation’s analytics reports. “Before, there was the concept that someone had to be very IT-literate and had to know coding languages to churn and analyse big datasets,” Clarke says.
But that is changing with tech. Today, it is possible for even a “non techie to start mining and playing with data,” Clarke says.
This means faster, more accurate analytics reports. Employees don’t have to wait for someone else before getting on with their work, and there’s no risk of IT teams using the wrong datasets because of miscommunication, Clarke says.
There is huge potential for these self-service reports in government. In healthcare, for instance, public officials could combine all patient datasets, categorise them based on location or age groups, and find new insights.
“You can imagine the power of having data from five and a half million Singaporeans and a central repository of any trends of potential disease or virus,” Clarke says.
Running their own reports can encourage public officials to be more inquisitive about their own data, he adds. If gaining new insights about citizen sentiments, patient health or demographic data is convenient, they’re more likely to do it frequently.
Getting there
How can agencies achieve this level of self-service analytics?
They must first have a good data warehouse. Data warehouses store datasets specifically used for running analytics, as opposed to keeping a record of all of an organisation’s activities and transactions.
But data warehouses can be expensive to scale, and are not very good at handling the large volumes of data we see today. These often span multiple formats, such as videos or social media posts, and can be stored across cloud and on-premise environments.
Moving data warehouses onto the cloud can help. The cloud’s extensive compute power makes it faster and easier to run self-service analytics. It also takes all the weight off of IT teams, so organisations don’t have to worry about managing the IT system.
Beyond analytics, the cloud offers a lot more storage at a lower price. Developers have access to a whole range of built-in services, so building new apps can be a breeze.
How Informatica can help
Informatica’s cloud data warehouse can help individual employees run self-service reports easily. First, it’s able to pull data from any storage location, be it across different cloud providers or on-premise. Few organisations move all of their data onto one cloud platform; many prefer to keep some of their data on premise instead or spread their datasets across multiple cloud vendors, Clarke explains.
This can make analytics tricky, however. Informatica’s solution resolves this issue, as it can pull data from any location. “We are the Switzerland of data,” Clarke says.
Next, Informatica uses AI to help users comb through their vast databases and find relevant information. For instance, if a public official searches for the keyword “health”, the algorithm suggests useful datasets based on what other colleagues found helpful. “It's the same keyword search principles as Google. We're just doing this for data,” he explains.
AI also helps to automatically clean up the data. A common issue is the inconsistency in data formats, and different databases might record a date of birth as seventh of January when it’s really the first of July. AI would be able to change that quickly, Clarke says.
On top of that, AI can help with incomplete data. When someone fills in a tax application form and misses out on the address fields, the algorithm self-populates them. “All these menial tasks that could take years to clean up and centralise are now being automated,” he says.
Self-service analytics could mean more efficient and effective policymaking. “The future is enabling any business user to become a data scientist,” says Clarke, and the cloud data warehouse will be a big enabler for that.