“In a candid conversation with India Technology News, Mr. Ajay Reddy Yeruva decodes the key benefits of Observability for next generation enterprises and how they can upskill their teams to unlock its true potential”
Below are the excerpts from the interaction-:
1) How observability can help enterprises to address the emerging business challenges and opportunities?
Modern applications based on microservices often run-in distributed environments across hundreds of servers, VMs, containers, serverless functions. So operational and development teams need massive amounts of data to process and help to identify the root cause, before they impact the customer experience and revenue.
So, observability plays a very critical role and acts as a key driver in solving these issues. Like understanding the top challenges faced by DevOps SRE teams, analysing the three pillars of observability, logs, metrics, and traces to provide more operational Insights.
Also using ML driven insights for faster root cause analysis, enabling developers to continuously optimize end to end application performance, connecting operational and business KPIs to deliver paths to customer experiences, and leveraging the benefits of open observability solutions like open telemetry to instrument the application without increasing the operational costs.
So, there are many ways that observability helps the businesses to overcome the operational challenges and sustain and provide greater customer experience.
2) How does observability complements conventional telemetry?
Monitoring is more of a react to kind of stuff where it provides overall systems application health. It doesn’t provide more details about discovering the unknowns, whereas observability on the contrary, it helps to discover the unknowns.
With correlation of logs and traces and with data across the systems and environments including hybrid environments and VMs, it provides more contextual information for the operational teams to troubleshoot the issues more, instead of waking up every single team to troubleshoot an issue in the middle of the night, you can send the alert to the right team.
And with AIOps and ML integration, you can automate some of the fixes which we are aware of, and you can automate it without any human intervention. So, in overall high-level summary, observability helps in faster troubleshooting of issues, automation of the issues using AI and ML.
There is a wide variety of data available with the latest paid platforms like Data Dog, like service mapping, logs, traces, metrics, database metrics, there are a lot of integrations available, which you can readily integrate. And within very few minutes, you can see the data. With this you can automate the configuration of alerts and see the real value out of it.
3) So, in short, can we say that if Telemetry has been a business enabler, then observability is more of a business enhancer?
Exactly, yes in my opinion. So, in terms of open telemetry, it’s a CNCF project enabled so you can instrument the application deeper deeply. And you can see the real value of performance of the API’s by instrumenting the right code, instead of just seeing the predefined data, like what you see in the real, the regular monitoring.
Observability helps you to instrument using Open Telemetry and wherever is the part of the code or system, and you can get the data out of it and visualize on the dashboards. Also, all of it happens in the real time.
4) How observability affects the CAPEX and OPEX of enterprises? Do enterprises have to invest a lot in implementing observability?
Because of the complex distributed systems, it’s very much complex to analyse massive amounts of data without observability. So, with the right tagging, you can get the data of the independent services in the monitoring platform with associated tags that might be across environments and any infrastructure layer or application layer.
So, with the right value of tagging, you can instantly see the data and visualize across the organization. Like the DORA metrics, which you see, which are the DevOps metrics, the mean time to recovery, change failure rate, lead time to changes, deployment frequency, the four Dora metrics are instantly visualized across the organization. And you can see the performance instantly.
If you see the latest research reports, most of the successful organizations have cut down their downtime cost by 90% by implementing Observability.
5) What kind of upskilling will observability require for enterprises to prepare existing human resources and tech teams?
Nowadays, there are a lot of training courses available on monitoring with hands-on exercises.
It’s not too hard for someone to learn and upskill them. So, companies can tie up with platforms like Linux Academy, Cloud Academy, to get the enterprise-wide plan and get their employees upskilled.
Also, companies can hire a training consultant who can train and guide the organisation-wide employees in enabling that holistic view right across the organisation. So, a good expert and tying up with some top training platform would help in this case to speed up the efforts and implement the observability quickly.
6) Talking about observability in India, then how do you see the future of this concept in the country?
As businesses across India are pressing on the needs of digital transformation efforts, cloud adoption has become imperative in this transformation journey. However, the move to cloud doesn’t just simply allow the business to quicken it’s time to market.
It’s also about making sense of business cloud data to innovate and deliver solutions that could help to future proof the business in this evolving business and consumer landscape.
The most successful organisations are optimising the cloud environments with observability to innovate faster and enable new business models to gain competitive advantage. Observability enables businesses to have full visibility of their data and helps to remove the barriers between data and action.
So according to the Splunk State of Observability report, business leaders can cut down the downtime cost to up to 90 percent.
Talking about India then only 26 percent of organisations are considered as beginners of Observability implementation as compared to an average 62 percent globally.
7) What are the compatibility challenges associated with the implementation of Observability?
I would say some of the challenges are data silos. There are multiple agents, data sources, and siloed enrolment tools which make it harder to understand the interdependencies across applications such as Web, Mobile, IoT etc.
The second challenge, I would say, is volume, velocity, variety, and complexity. So, it’s nearly impossible to get answers from the sheer amount of raw data collected from every component, such as AWS, Azure, GCP etc.
The other challenge is manual instrumentation configuration. When IT resources are forced to manually instrument and change code, they spend most of the time setting up the observability rather than innovating the stuff and lack of pre-production.
Even with low testing and pre-production, developers still don’t have a way to observe or understand how real users will impact before they push to production.
And the other challenge is wasting time troubleshooting applications. The development digital experience team is pulled into troubleshooting problems to try to understand the problems, wasting time guessing and trying to make sense of telemetry and come up with answers.
These are some of the challenges I would say but investing time into it and gaining upskilling themselves and making the environments more stable going forward, will help the organization to scale more, and hence to help implement the business models more successfully within a short time.
8) Please tell us about the self-healing application infrastructure in the observability domain, and how tech businesses leverage its maximum potential?
I would say understanding the context and topology is very important, while implementing and using observability. This involves instrumenting the code across the application and across the systems. And the second one is implementing continuous automation, whenever people identify, get notified.
So, when they see a proper fix with kinds of issues, they can automate it, like PagerDuty has its own Runbooks feature, which, based on the alert of the incident that got triggered, can automate the fix. So, it avoids manual intervention and businesses can leverage maximum potential.
9) A lot of enterprises go for a one size fits all approach. So, what will be your message to them?
I would say this depends on architectural reviews. The architecture team understands the system and design better, so design is very important while we are leveraging any observability tools.
There are a lot of interdependencies available. So, people who really understand including the leads of teams, and the architecture team who’s sitting on the enterprise-wide architecture team, can help to provide the right approach in deciding whether the observability is really needed or not.
But most of the companies moving towards digital transformation these days, and migrating towards microservices, we can probably assume even if they’re using a monolithic today, they will be migrating to micro services-based architecture in nearby future. So, they can start implementing Observability while they’re monolithic itself, and start trying to leverage the observability features. And then when they move to microservices, implementing it will become much easier. And of course, it depends on budget allocations.
10) Which industry verticals according to you can reap maximum benefit out of Observability?
I will say most of the industry verticals who have the technology department available, who have the services running public facing, will benefit with observability, because it’s more of data driven sites these days. So, they can identify and remediate the issues faster, even before they happen. Hence across the system, you don’t want to analyse huge amounts of data manually. This will help in greater customer experience and not to lose the customers and enhance the customer trust on the business.
11) What will be your message to enterprises who are a bit sceptical when it comes to the potential of Observability?
I would say enterprises who typically take the digital transformation as a serious need in the coming future or who are working towards it and who are already on it, who don’t have observability implemented end to end. I would strongly recommend them to take observability as a key driver in driving their business operations.
Because with investment into observability you get several profits. We should not see it as an investment but as a benefit to the organisation.
