Data Integration 101: Using Metrics To Show Cross-Team Value

Jul 19, 2021

Data has become one of the most prized assets in a software delivery process. We can safely claim it to be the lifeline of any operation thereof. This is why data integration stands as one of the most vital tasks for IT leadership. As shown by the fact that 3 out of 4 of CIOs lament the pain of connecting data points from too many tools. All in the name of attaining a firm grip on the goings-on in their enterprise. When done well, data integration creates a stream of metrics that bring clarity and ease to managers. 

This post explores how you can leverage data integration to view metrics across teams actively building value into a software product. 

This is not to say you can't use the ideas suggested here for another product line. Software delivery is important in that teams use more products than in any other line. This is especially true for custom-made solutions to daily problems, which themselves ooze data worth integrating into the rest of the noise. 

Before things get complicated, let's spark our voyage by clearly laying out the topology from which data emanates across a multiteam workforce. 

Streamline your software delivery with Plutora!

Streamline your software delivery with Plutora!

Imagine a single dashboard managing all enterprise software delivery, boosting visibility, efficiency, and cutting costs. Experience Plutora's solutions today!

Imagine a single dashboard managing all enterprise software delivery, boosting visibility, efficiency, and cutting costs. Experience Plutora's solutions today!

Defining the Data Integration Portfolio

Unlike matter, data can be created—and sadly lost when not utilized. Losing data doesn't only happen when you misplace potentially useful zeros and ones. It can also occur when you take too long to use data. Think of the following systems when considering the channels that produce data along a value stream. 

  1. Procurement: Whenever your company adds capital into your ecosystem. This could be in the form of software or even talent.

  2. Operations: These individuals support the conveyer belt that produces your product. Think about the tools they use to ingest and create data.

  3. Development: These are the people on the belt. They design, code, and ship valuable products.

  4. Testing: An essential set of hands and tools that use the product exhaustively before pressing it out to users.

  5. Customer service: The feedback coming from intended users through support forums and emails.

You can already tell how there can't possibly be one tool to please the thirst for progress across this example setup. As such, you'll find that such a team can use the same communication tool. (Case in point, Slack.) Throw a project management tool like Jira and or GitHub into the fray, and things can really get out of control. 

The systems/platforms above create data that might fall through the cracks because you can't ensure everyone will use the platforms in the same ways. Developers tend to focus more on the version control tools and project management notifications offered by their tools, while customer service teams more frequently use job tracking tools without getting into the finer details. 

But these distinctions would quickly bore management. When broaching the subject of data integration with supervisors, recognize that everyone has different needs and preferences regarding the programs they use. 

A Typical Data Integration Approach

If you have capable developers and no technical debt to make up for, you could say, "Let's build a data collection system that everyone can work from!" Noble. Yet you can't possibly force a single login across the floor without negotiating quality and productivity. As such, it's wiser to let everyone work within their team-identified set of tools. 

A better approach would respect the tools ecosystem while creating a net around all your tools to catch data in real life. Think of a master key equivalent to APIs. Unlock streams of data from all tools and have an aggregation point that's optionally accessible. The less you enforce usage and the more you infuse value, the less resistance you'll get from all stakeholders. 

Make it such that customer support doesn't have to log in to GitHub to stay abreast with issues. Similarly, IT leadership shouldn't have to constantly watch out for notifications when following up on epics. 

What Data Matters When Integrating?

If you used the approach suggested above (creating a net for all tools), chances are your resulting system will get flooded by logs and metrics from all over the place. This is why it's important to have a high-level discussion on which data points make the most impact for decisions at the IT leadership level. 

For example, how many lines of code were committed in any time range matters when project managers determine their teams' productivity and morale. But the same metric won't improve decisions about the future of a product. Instead, management might want to know how many versions were released and how customers responded to each one. 

Therefore, you need something more than just a shell that gathers data from your tools' APIs. You need nodes along the path to analyze and add value to all streams of data. And you need to collect and analyze data quickly since some of it is time sensitive. 

Introduction To Data Visibility

Once you've ironed out and specified the need for particular sets of data, you'll have inched closer toward data visibility. This is defined as a state where you're not unknowingly ignoring potentially useful data around your value stream. 

Think of the lack of visibility as looking at a queue of data from a 2-D angle. While you're looking at everything, you can only see what's in front of the lot. With visibility, you sit in the director's chair, which raises you above the line of sight to give a 3-D perspective of all data points. 

For the most part, we experience data visibility through dashboards. Dashboards present all the valuable metrics in one viewport. The focus lies on awareness and the operability of the data pool that emerges. It's up to the observers to fish out useful ideas from any dashboard. However, the dashboard—and thus visibility—is the foundation for the use of metrics to alight business and IT. Visibility clarifies IT processes for all players across an enterprise.

How To Encourage Data Integration

You might already have some tools connected to seamlessly pass information around the floor. This might be through something like the native integrations from Slack to GitHub. In that case, GitHub has a learning curve, which can be steep for nontechnical users. As such, some teams may resort to printing out PDF files to keep leadership in the loop of things. That's problematic on two fronts. 

  1. Emails can slow down reaction times. Recipients must open the email and have time to come up with questions, write them down, and read the response.

  2. As soon as you export to PDF, the real-time aspect of data runs out the door. A PDF provides a snapshot of a historic point in the flow of data around the value stream. Making decisions based on this report ignores the most current state of the stream.

Propose an easier-to-understand and always up-to-date data visibility solution, like the kind provided by Plutora. This way, nobody has to sit through a class on how to read dashboards. Charts and diagrams are easier on the eye and remove the computation requirement when making sense of metrics. 

Request a demo and start running the platform that benefits every node along your value stream. 

Download our free eBook

Mastering Software Delivery with Value Stream Management

Discover the key to optimizing your software delivery process with our comprehensive eBook on Value Stream Management (VSM). Learn how leading organizations are streamlining their software pipelines, enhancing quality, and accelerating delivery.

Deliver Better Software Faster with Plutora

Deliver Better Software Faster with Plutora

Deliver Better Software Faster with Plutora