Instana has announced the acquisition of BeeInstant (a real-time metrics analysis platform), StackImpact (a production polyglot application code profiler), and Signify a future solution to provide insight into complex system signals. The acquisitions of BeeInstant and StackImpact have a lot to say about the current and future states of the APM market.

StackImpact – All About Right Now

Microservices (small bits of code running in Docker containers, orchestrated by Kubernetes) are all the rage. The most advanced application teams have deployed microservices based applications at scale, and have proven that when combined with a proper CI/CD process and a proper DevOps culture that the application system can be advanced with more agility and speed than was the case with previous architectures leading to significant competitive advantage.

However it is the case that only a small part of the new applications that are being put into production are being built around a microservices architecture. This is because most applications do not require the dynamic scaling inherent in a microservices architecture orchestrated by Kubernetes, and because most development teams do not yet have the skills to build microservices based applications and support them in production. Therefore most of the new applications being put into production are still being built to either N-Tier or monolithic architectures.

There is also the minor issue of 15 years of previously built applications which are all built to monolithic and N-Tier architectures. What is important about all monolithic and N-Tier applications, is that you cannot monitor them in the same manner that you can a microservice based applications. Aficionados of the microservices architecture are fond of pointing out that since the amount of code in each service is minimal, that analyzing the code is not nearly as important as analyzing the interactions between the services. This is correct for microservices based applications, and completely incorrect for applications built to N-Tier and monolithic architectures.

Properly monitoring monoliths and N-Tier applications requires code analysis in production. This is the feature that put AppDynamics and New Relic on the map. Both have agents that get installed in the run time or application server supporting the application (the now ubiquitous Javaagent command).

Instana is the vendor with the leading product position in monitoring microservices. Instana offers an unmatched level of automation in the monitoring process, leading to fast time to initial value, and a product that is easy to maintain as the applications are evolved and new ones are introduced. So why does Instana need to “take a step back” and add support for code analysis? For the following simple and pragmatic reasons:

  • It dramatically broadens the set of applications that Instana can cover (Instana can now cover all of monoliths, N-Tier and microservices based applications)
  • When customers start to migrate existing applications that are monoliths or N-Tier, they do not start over, they start by carving off pieces of the existing applications into microservices, creating applications that are hybrids of monoliths and microservices and N-Tier and microservices.

Therefore the ability to understand what is happening in the code in the parts of the application that remains part of the monolith or the N-Tier architecture is crucial. With StackImpact Instana can now show you hot spots in the code just like AppDynamics and New Relic have done for years as shown below. What will be really exciting is when Instana integrates the right way to monitor microservices (traces) with the right way to monitor the code (code analytics) across these hybrid applications.

StackImpact Code Analysis (Click to Expand)

Now the interesting thing about Instana’s approach to microservices is that it is based upon a host agent that dynamically injects the required instrumentation into each container based upon the language and application run time in use. This is in dramatic contrast to the approach taken by all vendors who use the Javaagent command to load their agent – which is a step that has to be manually configured and maintained. At this moment in time, StackImpact requires the exact same manual type of installation that AppDynamics and New Relic require with their Java agents. It will be interesting to see if Instana can integrate and automate the StackImpact agents and make code instrumentation into something that is as automatic as the rest of the Instana product is today (the bet here would be yes). If Instana can pull this off, then Instana will be the only vendor with completely automated instrumentation for monoliths, N-Tier applications and microservices based applications.

BeeInstant – All about the Future of Observability Platforms

The acquisition of BeeInstant by Instana should serve as a wake-up call, and a very frightening one to the entire APM and more general monitoring industries. This is the case for the following reasons:

  • Instana already has the most real-time monitoring architecture in the APM space. Instana is today the only vendor that can collect metrics on a one-second basis and update the visualizations of those metrics on a one-second basis.
  • Instana already collects traces on a comprehensive basis, meaning that it already collects every trace and every span that is a part of every trace between all microservices that interact with each other.
  • Instana already integrates its one-second metric collection with its comprehensive trace collection. No one else can do this in the automated manner that Instana does.

So if all other vendors are stuck at one minute metric collection and Instana is already at one-second metric and trace collection, analysis and visualization, why is it necessary for Instana to acquire an even more real time and scalable metric analysis platform (which is what BeeInstant is)? An insight to this comes from the image below, which shows how BeeInstant can apply high cardinality analysis (percentiles to find outliers) to each individual metric, and to a set of metrics that span many objects (hundreds of thousands if necessary).

BeeInstant High Cardinality Analysis (click to expand)

The New Gauntlet for Modern Monitoring Vendors

Having Observability means that you have metric, trace, dependency, and log data for every unit of work of interest (a unit of work can be a call between two microservices, a sequence of many spans that comprise a transaction, a business transaction in an N-Tier application, an interaction between two objects in a monolith, or a batch job). Having Observability also means turning this deluge of data into something useful and actionable for the teams supporting these applications in production. This means apply high cardinality Analysis to the data to find the exceptions and the outliers that warrant attention.

The speculation here is that the ability to do high cardinality analytics at scale on real-time and comprehensive metric, relationship, and log data outstrips what the commodity open source databases that the monitoring vendors currently rely upon are capable of. If it turns out that whatever combination of Cassandra, OpenTSB, Influxdb, ElasticSearch, Postgres, MySQL, Hadoop, and Neo4J a vendor is using are incapable of dealing with the input, high cardinality analysis, and visualization of metric, relationship, trace, and log data for modern monitoring products at scale, then then all of the monitoring vendors have legacy back end data architectures. If this is true then Instana deserves credit for being one of the first monitoring vendors to do something about it.

This then gives rise to the question of which monitoring vendors actually have a back end architecture that can deal with the volume and diversity of the data that must be ingested and analyzed in real time. Again that fact that the most real-time of the APM vendors, Instana, felt the need to make this kind of an investment is telling as to the state of the monitoring world. Most other vendors operate at 1 minute intervals and sample metrics, and traces to manage the load upon their back ends. Going from sampling to get 1 minute data to getting all of the data all of the time and being able to analyze it instantly is going to require that the entire monitoring system get rearchitected to address these requirements. Therefore it seems that the requirements of Observability are about to create a lot of legacy monitoring vendors.

Summary

By acquiring StackImpact Instana has invested in the reality of being able to monitor the code the exists today. By acquiring BeeInstant, Instana has laid the foundation for a real-time Observability Platform and has distinguished itself with respect to all of the other APM vendors.