When Covid-19 first started to spread, no one knew how to fight a problem they could not see. Over time, scientists understood the structure of the virus causing the disease, and developed the vaccines that we rely on today to defend against the virus.

Cybersecurity is no different. “Visibility is a key factor in cybersecurity. You can only protect what you can see,” says Wing Leong Ho, Solutions Engineering Director, Cloudera. With IT systems becoming increasingly complicated, it can be difficult to keep an eye on all the devices, users, and systems at work.

Ho shares how governments need to have visibility into all of their data and related activity to improve their cybersecurity posture, and increase positive threat detection and remediation outcomes in case of a breach.

Data-driven insights power cybersecurity 

Governments need to be able to rapidly process historical and real-time data to ensure timely responses to cyber threats, says Ho.

For example, the Internal Revenue Service (IRS) in the United States had compiled massive amounts of data over the years. They wanted to use this database to detect and prevent fraud and better protect taxpayers from risks like identity theft. However, they struggled to process the amount of data they held.

The IRS employed tools such as AI and machine learning that could rapidly detect attempts at fraud or identity theft. They worked with Cloudera and NVIDIA to quickly implement these technologies.

The Cloudera Data Platform (CDP) combined with NVIDIA’s processing capabilities allowed the IRS to create graphs that gave an overview of individuals and organisations’ actions over time. AI “bots” and machine learning algorithms could quickly and repeatedly analyse these graphs to detect anomalies in behaviours that signal potential fraud.

Datasets that used to take months or years to bring together and analyse can now be processed in days, hours, or even minutes. This led to a ten times increase in the engineering and data science departments’ speed of work.

Visible data allows for real-time analysis

Data needs to be as real-time as possible when dealing with preventive threat detection, says Ho.

This can be a challenge as governments often receive data from multiple sources such as the cloud, internal servers, and different applications. This causes data to be trapped in silos and makes data processing more complex, says Ho.

“These silos also result in blind spots that prevent IT security teams from practising proactive cybersecurity,” he continues.

Being able to analyse data in real time can help companies improve their cybersecurity response time, Ho suggests. For example, an oil and gas firm struggled to process its data as they had a rapid influx of data.

They employed Cloudera’s multi-cloud platform to perform data analytics in real time. The platform helped the cybersecurity department process data more efficiently by allowing them to manage their data from one location instead of across multiple premises.

For instance, the logs from the PCs of remote workers can directly be sent to Cloudera’s public cloud platform for analysis wherever they may be located.

The platform saw the company taking in log data from 130,000 PCs and different types of cloud platforms in real time. The data is then consolidated into a dataset that applications can use for analytics.

Cloudera’s platform reduced the average time to detect cybersecurity threats by 90 per cent, from seventy minutes to seven minutes. This gave the firm sufficient time to react and respond to any suspicious behaviour detected. It also reduced the rates of false positives being reported, detecting true positives and filtering out the false.

The importance of data visibility

Governments hold highly sensitive data of citizens, and will need strong security systems that can respond to and prevent cyber threats, highlights Ho. Better data visibility will allow governments to predict and respond effectively to cyberattacks.

Data extraction is inefficient without a consolidated platform, which may lengthen the time taken to detect a breach and assess the damage caused. The Cloudera Data Platform provides a common place for data analysts to access high-end tools across cloud environments. This gives them insight into both real-time and historic data, supported by AI and machine learning.

These combine with a security platform to allow governments and agencies to maintain command and control. IT teams can have visibility over all data and detect and respond quickly to threats.

Additionally, the Platform allows governments to keep their data independent of the programmes they are using and allows for easy transfer among cloud environments. This frees governments from being restricted to any one vendor or product.

Governments need to focus on achieving good end-to-end visibility and control of their data as databases become increasingly large and complex. This will prevent them from being blindsided by cyber threats and attacks when they happen, and ensure the data of citizens is kept safe and secure.