Search the website

False: Marketers Should Only Use Real-time Data

True: They must combine real-time with batch data to achieve their dreams

The “marketing best practices” discussion in the last few years has been marked by a shift to trigger-driven marketing centered around real-time data usage. This has led to a fallacy that marketers should only use real-time data. This post will show you why this shift will only break your customers’ hearts and what you need instead.

Last year, a survey by research firm Gartner found that 66% of marketers indicate that event-triggered multichannel marketing is their primary approach for delivering marketing messages to customers and prospects. However, when compared to the 87% that back in 2017 had projected their organization’s multichannel marketing would be primarily event-triggered by 2019, the number comes up short.

Why the difference?

As marketers attempted to make their event-driven dreams a reality, vendors focused on providing real-time data solutions. However, customers aren’t as simple as “an abandoned cart with female activewear in it.” They are complex, have a history with the brand, and expect to be recognized by the brand they are purchasing from.

Slowly but surely, marketers discovered that event-driven marketing does not equal real-time data-based marketing. And to fulfill marketer demands and meet customer expectations, technologies need to adopt a hybrid approach.

This is where customer data comes into play – it is similar to the human brain, which has two categories. Professor Daniel Kahneman called them System 1 and System 2, but I prefer Mark Manson’s “feeling brain” and “thinking brain.” While the thinking brain is slow but conscientious, accurate, and impartial, the feeling brain arrives at conclusions quickly, and is impulsive, irrational, and inaccurate.

In CRM marketing, the thinking brain results from the transformation, processing, and unification of your historical customer data. While the feeling brain is the trigger set to react to real-time data streams. If you only rely on the latter, you are bound to be inaccurate. But if you combine both and strike the right balance, your marketing dreams could come true.

We are about to get a bit technical, so feel free to engage your thinking brain.

Putting the combined approach to the test

One of the most classic triggered campaigns in the eCommerce world is the post-purchase email that uses the purchase event to send customers immediate follow-up emails. Sometimes, these will include offers for complimentary products, and other times a list of how to maximize the product’s usage. But how do you know whom to send what email to?

You can ensure triggered campaigns differentiate between customers by leveraging a basket value model in conjunction with the trigger. By combining these two data points, the historical basket value with the real-time trigger can easily segment customers into multiple tiers or group different clusters together into tiers—for example, low, medium, and high. 

This will help you plan hyper-relevant triggered campaigns for each segment. You can then provide promotional and complimentary product information to the low tier and send more informative campaigns to the higher tier, for instance. By appending the data from the latest purchase to the historical data already available, you can ensure customers receive the correct campaign.

How Optimove combines batch and real-time data

Building an effective customer profile that is updated in real-time is a multi-step process at Optimove; first, a batch process analyzes all available data related to each customer. This historical data can go back years and includes data dimensions from multiple systems, fused and enriched by different algorithms such as clustering and ML.

One of the products of this deep analysis process is a customer profile containing hundreds of customizable attributes. When a fresh customer profile is created, it is encoded, timestamped, and streamed into Apache Kafka for integration with the real-time process.

An independent process is feeding customer-related events from multiple sources such as Optimove’s SDK or Segment.com into Apache Kafka. This raw event data is grouped by customer ID and fed into a very flexible aggregation framework driven by a graphical UI. It describes which events update which profile attributes, and in what way.

A simple example would be a “Total Purchase Amount” attribute that tracks “purchase” events, extracts the amount field from within it, and sums the amounts across events for the present day. This aggregation framework may update many such attributes, across all customers, and do it all in real-time.

Next, a smart combining mechanism receives both the batch-process produced profiles and the real-time attributes as a continuous stream of information to perfectly stitch the two data sources and calculate the exact correct values. It then matches the data based on customer ID and timestamp.

Finally, this data is made available as a live feed in Optimove that also feeds back into the Self-Optimizing Journey algorithm, allowing for a continuous reevaluation of decisions.

Final thoughts for the thinking brain

Using Optimove, marketers can easily leverage deep insights daily by looking into all available customer data. All while keeping an eye on live event data to maintain an up-to-date customer profile and improved orchestration decisions.

Effectively rolling out a contextually relevant, event-driven marketing plan with personalized offers requires a smooth integration of both the thinking and feeling brains. By combining real-time customer activity with historical data, marketers can ensure that each customer’s complexity is taken into account before any campaign is sent. And maybe the next time they are surveyed, the gap between expectations and reality will be closed.

Published on

Pini Yakuel

Pini co-founded Optimove in 2009 and has led the company, as its CEO, since its inception. With two decades of experience in analytics-driven customer marketing, business consulting and sales, he is the driving force behind Optimove. His passion for innovative and empowering technologies is what keeps Optimove ahead of the curve. He holds an MSc in Industrial Engineering and Management from Tel Aviv University.