How Hybrid Integration is Changing: Trends and Questions You Need to Address

Changes in IT and integration in particular

Current trends emphasize performance, autonomy, delivery speed, and ease of knowledge. However, the problem is that centralized integration platforms cannot provide such comfort. This also applies to centralized teams. Nevertheless, there should still be some central components, such as API registries.

“In the digital IT world, the emphasis is on the ability to adapt and quickly deliver a given function to production. People in projects are under pressure, making it easier for them to opt for the first choice (even if it’s more expensive from an IT cost perspective). It allows them to deliver the product to production faster,” notes the architect, who has more than a decade of experience with integrations provided by Trask. The reasons for decentralization are pragmatic:

• Any centralization, whether of platforms or teams, can be perceived as a significant obstacle. The questions asked by the team mainly stem from the issue of responsibility: "When will you deliver the integration service? Why is it so expensive, and why do we have to wait 3 months? Why should we pay for the upgrade of your integration platform? Why did you transform the data this way? It doesn’t make business sense. What if we handled the integration ourselves?”
  ‍
• Decentralization, or rather, team autonomy, leads to them creating their own solutions. These are simpler and more purposeful. ”Clouds, for example, support decentralization. This development is happening even where teams do not seem so autonomous (where organizational verticals are not defined),” adds the architect about the phenomenon.
  ‍

[.infobox][.infobox-heading]How to cope with this fragmentation in your company?[.infobox-heading]1. Always keep in mind that the goal is conceptual integration. For everything to work, it is important to follow a set of functioning rules that define autonomy, among other things, and to have clearly defined boundaries of responsibility.

2. Offer teams a set of services, both in terms of integration products and tools (company stack), and regarding data access, including support for integration implementation. Create services that offer the highest degree of self-service and are transparent in terms of time and cost.

3. Focus on data – data governance and data discovery. Find a way to refer to a data object from integration analysis and central logging.[.infobox]

Do not compromise on data

We mentioned a focus on data. Yet, several pressing questions trouble us from the perspective of integration, which are important to understand. Here are some of the most critical ones.

Nothing is more annoying than searching for where I can get the necessary data, only to end up compromising.

1. ‍You need to know where your data is and in what quality.
   
   A fundamental element of successful integration is knowing where to find data. There are no compromises here. You must be able to rely on discovery APIs and data, self-service access, and subsequently quality. “Nothing is more annoying than wandering around the company looking for where (in which system) I can obtain the necessary data in the right quality, only to end up compromising by laboriously mining lower quality data from a non-primary system,” warns the architect.
   ‍
2. ‍Data does not belong to the application.
   
   How does data travel through your organization? If solution architects and integration analysts were to define the essence of data, it would almost always be as “pain." The absence of a data architect, data design, and tools for data storage (and their metadata, including affected systems) leads to repeated misunderstandings. Data does not belong to the application. It is processed and stored by the application. Integration analyses can then appropriately refer to the data.
   ‍
3. How to ensure data consistency in the Event stream.
   
   We differentiate between operational, analytical, and infrastructural data. Operational requires a significantly higher degree of consistency. This can be achieved either by the classic transactional approach – for example, using messaging technology, or in the event stream by introducing a series of error state treatments, consistency reports, and correction/reconciliation mechanisms.

Control and Dominate is a vain sentiment in a vertical-based organization

The integration department, in its effort to achieve conceptual integration, defines rules, offers services, and creates an integration platform package. Part of the effort includes obstacles. Some of these can be characterized as follows:

1. Who will pay for the shared infrastructure.
   
   Another unpleasant question related to integration remains who will pay for the platform’s construction. There are several options. “Simply say you want to use this platform? Pay for it. Often there are differences in expectations and a reluctance to contribute to the relatively high costs. Shared platforms are thus prepared but starved. A possible solution? Automation contributes to cost reduction. But remember, even that needs to be built and maintained,” adds Tomáš.
   ‍
2. Autonomy and independence of teams often lead to inconsistency in parts of the integration.
   
   How to prevent disorder and uncontrollability in all this? When you allow autonomy, products suddenly start accumulating, and you’re stuck with a “zoo.” Therefore, you should agree on rules that need to be met. Enforcing them, however, is worse. A way forward may be to convince teams of federalized governance. Rules will not be dictated to them; instead, autonomous teams can participate in their creation, maintenance, and enforcement.
   ‍

Integration is simply challenging, uncertain, and should be carried out by experienced professionals. Are you interested in learning more about the topic? Contact our experts, who are happy to help improve efficiency in your company as well.

Author

Tomas Rosa
Integration Technology Architect
trosa@thetrask.com