Streams of Events are Changing the Integration Paradigm. Companies Need their Systems to run Without Glitches.

Events Carry Information

Alongside long-standing trends in integration such as decentralization, containerization, the use of API management and the drive to create smaller and more flexible solutions, event-stream or event stream is another topic that needs attention. It is an architectural and technological concept that helps address the growing demands in the integration sphere, especially in the area of event-driven integration. For example, many Czech banks are already using event streams as a critical integration tool.

In this integration scheme, an event is defined as a description of something that is (a fact) – for example, a customer address or an accounting balance. However, it also includes notifications of a change in status, such as a change of customer address, notification of exceeding the allowed limit, or various instructions for action within a given system.

The Way Event Streams are Dealt With can be Apache Kafka

One way to manage and process events efficiently is by engaging a data streaming platform. The predominant technology in this regard has become Apache Kafka, an open-source distributed streaming platform that can be used to create applications and feeds for real-time streaming data.

Apache Kafka was created by developers at LinkedIn, who needed a tool for fast, secure, and scalable transfer of large amounts of data across the globe. Integration tools and practices at the time failed to adequately reflect these requirements. Today, Kafka is used in many industries and companies from mid-sized ones to global corporations.

When you use this streaming platform as a backbone technology for integration, you can implement service and microservice systems to exchange events in real time while creating new events, for example, in response to what the end user is currently doing in the application.

For example, in practice, an event after adding an item to the cart in the e-shop can lead to the creation of updated cart contents, setting a new corresponding price and updating the stock. Kafka also allows events to be handled in failover mode 24/7. Events are securely stored with their data in the Kafka cluster and are available to many applications and services.

Explains Petr Dlouhy, Integration, Client & Delivery Director at Trask

That's key these days, everything has gotten faster and clients now expect services to run without outages, be scalable and everything to be taken care of right away.

[.infobox][.infobox-heading]How can Event Streaming Based on Apache Kafka
Help Your Business Grow? Explore the Benefits:[.infobox-heading]- Realtime reaction to events across digital channels: e.g. active offers as e.g.
   insurance proposal as a pop-up in mobile banking at the moment of credit
   card purchase.
- Real-time Fraud Detection: by analyzing multiple streams of transaction
   data to identify unusual patterns or anomalies allows to swiftly take
   action to stop fraudulent activities.
- Personalized Customer Experiences: gather and process real-time
   customer data to deliver personalized experiences, such as targeted
   marketing offers, tailored product recommendations, and proactive
   customer service.
- Manage risk and maintain compliance with regulatory requirements
   by continuously monitoring transaction data and flagging potential issues
   for timely intervention and adherence to standards.
- Operational Efficiency and Performance Monitoring: track and analyze
   operational data across various banking functions, enabling proactive
   identification of inefficiencies and real-time performance monitoring
   to ensure smooth operations.[.infobox]

Getting to this requirement is possible because all communication between systems in Kafka is asynchronous. This means that the systems are loosely coupled and independent of each other. The benefits of this approach include the ability to easily update, scale and operate the systems in many replicas around the world to handle an ever-increasing number of client requests.

However, it is important that at the beginning of the whole project, there is first a data analysis, selection of integration patterns that Kafka allows, proper definition of transmission channels (topics) in Kafka and their schemas. It is also essential to treat errors that may occur when creating and reading messages - as always, proper design is crucial for success.  

When does an event-driven architecture come into play?  

Using Kafka makes sense wherever there is a need to process huge amounts of data in real time and without delay. Typically, event-driven integration thrives in environments that use microservice architecture, which it complements well in terms of scalability, fault tolerance, and agility.

Typical examples we encounter and help our clients address are log processing (application and audit), user activity monitoring and online evaluation of client offerings, or "classic" application integration. Especially in the last case, it is necessary to design the right approach to event distribution and consumption, as event streaming for existing systems can be more complex to implement than other types of integration.  

Proper implementation offers significant future potential – enabling faster responses to new client requests (often without downtime), effective scaling as the client base grows, and ensuring stable, consistently available services for customers.

If you want to enhance your real-time data processing and make your business more flexible, don’t hesitate to contact us. We will provide the best solution for your company’s environment.

Tip: How can better log management improve the business in a large bank? Read our case study from Raiffeisenbank.

Author

Martin Citron
Integration Leader
martin.citron@thetrask.com