PARTNER

Using data for rapid threat detection

By Wing Leong Ho

Cloudera discusses the importance of data collection to root out harmful cyberthreats.

The 2016 Summer Olympics may have lasted just 16 days, but the athletes were preparing for it years in advance. To Jasmine Ser, Singaporean sports shooter, it was a culmination of five years of grueling training. According to the Singapore National Olympic Council, that was how long it took for her to train and qualify for the event.

Detecting a cyberthreat can be like competing in the Olympics. It’s a short burst of intense activity, but will require sufficient time and information to achieve peak performance. Cybersecurity teams need massive amounts of data to identify cybersecurity threats.

To improve cybersecurity, governments will need better storage, access, and analytics of their data. Adopting a platform to manage security-related data will improve the speed of cybersecurity response, and help keep citizen data more secure.

Data difficulties

Governments need access to both real-time and historic data to detect cybersecurity threats in their network. Having this data allows security teams to monitor the system and detect security anomalies, which could hint at a malicious attack.

Not only do governments need to be able to see this data, but they also need to store it. This storage needs to be secure whilst allowing security teams to view behaviour analytics, conduct threat hunting, and create a bot that will help close system vulnerabilities.

But with the increasing amount and variety of data available to governments, governments can find it challenging to store, access and analyse their data.

Security in action

To cross the data hurdles in their way, governments are adopting tools that help store data and analyse it for security. One example of this is the US Internal Revenue Service (IRS), which handles tax collection for the federal government.

The IRS wanted to make use of its petabytes of collected data (one petabyte being the equivalent data size as 2000 years of MP3-file music). But the sheer volume of the data proved too much for their existing systems to handle.

The organisation was able to analyse its data in days after adopting a cloud tool from Cloudera and NVIDIA. Cloudera’s cloud infrastructure coupled with NVIDIA’s processing units allowed the IRS to shave down the time taken to analyse its data from weeks and months to mere days.

Making use of AI and machine learning in the cloud also helped the IRS quickly identify behaviour or patterns that might be potential fraudulent activities.

Tests on the platform revealed up to 10 times faster engineering and data science workflows and a 50 per cent reduction in infrastructure costs. This greater capability for analysis also allows the IRS to detect cases of fraud, said Joe Ansaldi, a Technical Branch Chief at the organisation.

Another way governments can address the data challenges they face is by separating their data processing and storage. Separating these key areas means each one can expand or change without affecting the other. This flexibility simplifies the process of adapting to different cyber threats.

Additionally, governments should ensure that data is shareable, enabling real-time insights to inform threat hunting. This data should be open for different security teams no matter where it sits in the cloud.

Proactive and reactive defence

Organisations must develop their cybersecurity on two fronts: reactive and proactive. They should respond quickly to shut down attacks once a breach has happened. It is equally important for them to be able to quickly identify potential threats.

Cloudera provides a data platform to assist both areas. It enables government agencies to store and analyse years worth of data. This allows for extensive analysis of security patterns, making anomalies more visible and threat response faster.

The data tool enhances proactive security by processing real-time data with machine learning. This provides the ability to detect advanced threats 2.25 times faster, Cloudera research found.

This platform also provides flexible storage that stretches across the data center and cloud, allowing IT teams to store all the data needed for security investigations while having total ownership over it.

The ability to keep out hackers is built on a foundation of data-driven threat detection. Efficient storage and analysis of data equips governments with the tools for enhanced security response.

Wing Leong Ho is ASEAN Technical Lead at Cloudera