Is observability the key to explosive innovation?

The harnessing of nuclear power has historically been a controversial topic. While some may raise concerns about the environmental and political implications of nuclear power, most would agree that the ability to do so at all is a feat of human engineering and science. But what is less discussed is the equipment needed to power the research of such technology.

Within the United States’ National Ignition Facility (NIF), over 65,000 computer-controlled devices are using 40,000 optics to guide and amplify 192 laser beams, all centred at a target smaller than the size of a thumb.

This is the US’ answer to nuclear research without the need for full-scale testing. Instead of detonating catastrophic nuclear bombs in remote deserts in New Mexico, the facility allows for controlled testing to further the US’ nuclear deterrent capabilities.

But collecting and processing the data from these research is a Herculean task. The process of concentrating the 192 laser beams on a miniscule target at precisely the right energy levels requires more than 2,000 computers executing over 3.5 million lines of code.

This is where the NIF sought a solution that would enable them to collect, manage and analyse this data with utmost reliability. They needed observability.

Harnessing observability for innovation

“Scientists expect that data to be accurate and delivered quickly … we have to ensure that we make decisions with the right data to effectively resolve any issues that could affect the next [laser] shot,” wrote Philip Adams, the Chief Technology Officer of NIF.

They found this solution in software provider Splunk. Splunk helped the NIF ensure the smooth operation of their facility through its observability solutions, allowing them to further their research with ease.

Splunk provided the NIF team with a complete overview of all the events and processes of all their systems. This allowed them to quickly identify patterns and pinpoint potential anomalies with the laser beams, speeding up their troubleshooting process and even enabling the predictive maintenance of their systems.

These solutions helped improve the facility’s performance, ensuring that their systems were still performing well even when the NIF doubled the number of laser shots they were delivering from about 200 to 400, says Dhiraj Goklani, Vice President of Observability, APAC, at Splunk, during an online interview with GovInsider.

“In an R&D facility, there’s constant change,” Goklani says. “They see Splunk as this platform which is constantly learning and scaling with them, because they are also constantly trying new ways and means to improve their infrastructure – and Splunk has to keep up with that.”

Better observability, better services

Whether within research facilities or in citizen-facing services, government agencies are constantly looking for ways to innovate. With citizen-facing services, for instance, citizens are constantly looking for an engaging experience, Goklani says. Observability can help with that.

Goklani gave the example of personal banking. “Imagine if you were banking with your favourite bank, and you couldn’t look at your bank balance or pay your rent. That would be a problem,” he illustrated. Observability addresses this by taking all the information associated with the service both on the front end and the back end, and providing a real-time view of this data in a correlated fashion.

This allows organisations to easily make sense of the data and identify potential issues before they occur, he explains. “Observability makes sure that every consumer, every business can have a seamless experience with all their critical applications and services,” he says.

Crafting an observability strategy

Goklani proposes a plan of action organisations can adopt when trying to develop an observability strategy.

Organisations first need to identify what a seamless experience would look like for the citizens and businesses they support. They can then work backwards to determine what applications are needed to support that experience.

Splunk’s observability platform helps by ensuring that these agencies have a complete overview of all data and applications, regardless of whether they are on premise or on the cloud, Goklani says.

This is vital as hybrid architectures are still prevalent today, according to Splunk’s State of Observability 2023 report. The report, which surveyed 1,750 leaders in IT operations, application development and DevOps finds that nearly 90 per cent of respondents believe it is important to have observability solutions that cover hybrid architecture.

Artificial intelligence capabilities are also integrated within the platform. This enables the system to automatically learn about the data and identify patterns and correlations, giving organisations an automated way to identify how to troubleshoot, remediate, and address potential issues, Goklani adds.

Beyond just the technological platform, Goklani shares that Splunk is also able to help organisations customise their observability strategies and has already done so for numerous public sector agencies around the world.

For instance, Splunk has helped the US Census Bureau digitalise their operations. The Bureau is responsible for the implementation of a nationwide survey every five years of all the US states and territories. When the organisation wanted to digitalise this process, they realised that it would be extremely time-consuming and constantly to replace and modernise all their legacy tools.

Yet, the Bureau needed to deliver a digital experience that had no lag or outages for millions of people trying to submit their data, Goklani shares. To achieve this, Splunk’s observability platform helped to provide end-to-end visibility across all their siloed tools, giving them a single pane of glass across all these mission-critical applications.

This helped them to improve their overall mean time to detect and resolve issues quickly, Goklani explains. Additionally, Splunk’s predictive analytics capabilities could even identify potential service degradation and outages as they are happening, he adds.

Discover how observability can boost not just innovation, but also bolster the resilience of your organisations. Read Splunk’s State of Observability 2023 report here and hear from Dhiraj in person at Splunk's upcoming .conf GO event in Singapore on 19 September. Register here.