For data-platform + CRM + marketing-ops leadership
Customer change events that reach every downstream system in seconds, not in tomorrow morning’s batch
A member updates their email at 9am at one location during checkout. The nightly Reverse-ETL job fires at 2am the next day. Eighteen hours of receipts, notifications, and marketing sends route to the old address. Real-time emission propagates the change to your ESP, CRM, loyalty platform, SMS provider, push channel, paid audiences, and contact center in seconds.
What this gets you
- Real-time customer change-event emission — member updates flow from the customer-graph to ESP, CRM, loyalty, SMS, push, paid audiences, contact center, and GBP listings in seconds. Sub-second target latency at the pub-sub layer; sub-minute end-to-end through downstream consumers.
- 7-axis customer-graph pipeline— Ingest + Resolve + Version + Compute-LTV + Cohort + Subscription-Ingest + Emit-Change. Emit-Change is the downstream propagation axis that sits on top of the other six.
- Per-banner topics + cross-banner reconciliation — each banner emits to its own topic; a reconciliation consumer maintains the parent-level cross-banner view; downstream consumers subscribe to per-banner or parent-level by intent.
- Per-jurisdiction PII handling + consent-state propagation — CCPA, GDPR, HIPAA, and per-state regulations encoded in the event schema. Consent state on every payload. Right-to-deletion events tombstone every downstream consumer.
- Event ordering + idempotency + retry + DLQ + audit trail — partition key per account ID guarantees ordering; unique event ID plus consumer-side dedup converts at-least-once to exactly-once; failed delivery routes to DLQ for replay; every emission audit-trailed for compliance review.
The nightly batch sync loses eighteen hours on every change
A multi-banner operator runs a centralized customer-graph that stitches identities across banners, computes per-banner LTV, tracks subscription billing, and maintains the canonical customer view. Downstream of the graph sit eight to twelve consumer systems — the ESP for transactional and marketing email, the CRM for sales and account management, the loyalty platform for points and tier status, the SMS provider for transactional and marketing texts, the push channel for mobile notifications, the paid-media audience platforms for retargeting suppression and lookalike generation, the contact center for agent context, and the Google Business Profile listings for the per-location reviewer-engagement workflow.
Every one of those systems needs to know when a customer changes. Email address change. Phone number change. Address change. Consent state change. Subscription state change. Tier change. Right-to-deletion request. Cross-banner merge event. Each event needs to land in every consumer that has a legitimate need for it.
The status-quo workflow runs nightly Reverse-ETL. The job pulls the day worth of changes from the customer-graph and pushes them to every consumer in sequence. The job fires at 2am the following day. A member who updates their email at 9am sees the change land in the ESP at 2am the next day. The receipt that fires immediately after the change uses the old address. The marketing send that goes out at 10am uses the old address. The loyalty notification at 2pm uses the old address. The SMS confirmation at 6pm uses the old phone number that the member also updated. Eighteen hours of customer touchpoints use stale data.
Real-time emission replaces the nightly batch with a pub-sub topic per customer-graph event type. The change publishes to the topic within seconds of the customer-graph write. Each downstream consumer subscribes to the topics relevant to its function and processes the event in order, idempotently, with retry and DLQ. The eighteen-hour latency drops to seconds.
What is in market — and what each category leaves to you
The pub-sub primitive layer is mature. The customer-graph orchestration layer that sits on top of it is operator-side wiring.
Enterprise CDPs — Segment (Twilio), mParticle, RudderStack, Tealium AudienceStream, Twilio Engage, Salesforce CDP, Adobe Real-Time CDP, Treasure Data
Excellent at single-tenant customer-data unification and event-streaming primitives. The multi-banner topic design, the cross-banner reconciliation, the per-jurisdiction PII policy engine, the cross-arc topic design that ties customer events to catalog events and order events, the 7-axis customer-graph integration — orchestration that wraps the CDP primitive.
Reverse-ETL — Census, Hightouch, Polytomic, Grouparoo (Airbyte), Workato, Tray.io
Strong at batch-to-batch syncs between warehouse and SaaS destinations. Run on schedules or on incremental triggers. Real-time emission with sub-minute end-to-end latency, per-banner topic design, and event ordering by partition key — closer to the pub-sub-streaming category than to the Reverse-ETL category. Reverse-ETL platforms can be configured to approximate real-time on tight schedules but the architecture is batch.
Event streaming platforms — Apache Kafka, Confluent, AWS EventBridge, Google Pub/Sub, Redpanda
Excellent at the underlying primitive — partitioning, ordering, retry, DLQ, schema registry, exactly-once semantics. Customer-graph-specific orchestration on top — the per-banner topic taxonomy, the cross-arc topic design, the per-jurisdiction policy engine, the consent-state-aware delivery filter — is the operator-side wiring this page describes.
MLOps + customer-event observability — Arize, Fiddler, WhyLabs, Evidently
Strong at ML-model monitoring and feature-drift detection. Adjacent to customer-data orchestration when the downstream consumers include ML models (next-best-action, propensity scoring). The customer-graph orchestration is upstream of the ML observability.
The 2am Reverse-ETL job
The status quo at most multi-location operators. A scheduled job runs every night, pulls changes from the warehouse, pushes them to every downstream SaaS, logs successes and failures, and sends a daily summary email to the data-platform lead. Failures get debugged the next morning. Latency between customer change and downstream consumption is 6-30 hours depending on when the change happened relative to the batch window.
The pipeline, end to end
- 7-axis customer-graph topology. Ingest (multi-channel behavioral ingestion), Resolve (deterministic plus probabilistic identity resolution), Version (versioned customer-history audit trail), Compute-LTV (per-location and per-channel and per-banner LTV math), Cohort (runtime-readable behavioral cohorts), Subscription-Ingest (subscription-billing data unification), Emit-Change (real-time downstream propagation). Emit-Change reads from the version axis and broadcasts to every consumer.
- Event-schema design and registry.One event type per customer-graph change category — customer.created, customer.email_changed, customer.phone_changed, customer.consent_changed, customer.tier_changed, customer.subscription_changed, customer.deletion_requested, customer.cross_banner_merged. Schemas live in a schema registry with backward-compat rules and per-version pinning for consumers.
- Per-banner topic taxonomy. Each banner emits to its own topic (customer.banner-a.email_changed, customer.banner-b.email_changed). The taxonomy encodes the multi-banner architecture so consumers can scope by intent without filtering at the payload level.
- Cross-banner reconciliation topic. A dedicated consumer subscribes to all per-banner topics, detects accounts that exist in two or more banners, and publishes reconciliation events to a parent-level cross- banner topic. Downstream consumers that need the parent view subscribe to the parent topic.
- Partition key per account ID. Every event routes to a partition by account-ID hash. Per-account event ordering is guaranteed by the pub-sub layer. Two events for the same account never process out of order.
- Event-ID idempotency. Every event carries a unique ID. Consumers track the highest event ID processed per account and discard duplicates received during retries. At-least-once delivery becomes exactly-once consumption per account.
- Per-jurisdiction PII policy engine. Event payloads tag PII fields with their regulatory tier (CCPA-California, GDPR-EU, HIPAA-healthcare, FERPA-edu). Consent state of the member is encoded on every event. A policy engine evaluates the event against the consent state and the consumer-authorization table and redacts fields before delivery.
- Right-to-deletion event type. A dedicated event type carries deletion requests. Every downstream consumer that processed prior events for the account processes the deletion event by tombstoning the record inside the consumer-system retention window. The audit trail captures the deletion across every consumer for compliance attestation.
- Retry plus DLQ plus replay. Failed deliveries route to a per-consumer dead-letter queue. A reconciliation job replays the DLQ on a schedule and surfaces patterns that indicate a consumer-side issue. Replay preserves event ordering within the affected account partition.
- Cross-arc topic design. Customer events share a topic taxonomy with catalog events and order events. A downstream consumer that needs the customer plus catalog plus order joined view subscribes to all three families and joins at the consumer side using shared keys. The architecture supports next-best-action engines and propensity-scoring models that depend on the joined event stream.
- Observability and SLA monitoring. Per-event-type latency tracked end-to-end. Per-consumer consumption lag tracked continuously. Alerts fire when a consumer falls more than the SLA threshold behind the topic head. Alert routes integrate with the operator incident-response process.
- ROI measurement. Latency from customer-graph write to downstream consumption per consumer. Downstream next-best-action quality measured against the latency baseline. Customer-revenue retention impact of reducing stale-data customer touchpoints. Signal feeds latency-target tuning and consumer prioritization per cycle.
Frequently asked
What is customer data orchestration?
Customer data orchestration is the real-time propagation of customer change events from the customer-graph source of truth to every downstream system that consumes customer data — the ESP, the CRM, the loyalty platform, the SMS provider, the push channel, the paid-media audience platforms, the contact center, and the Google Business Profile listings. The category leaders — Segment Functions, mParticle Audiences, RudderStack, Census, Hightouch, Polytomic, Twilio Engage — ship the pub-sub primitive. The per-banner customer-graph wiring + per-jurisdiction PII handling + cross-arc topic design with the catalog and order event streams is operator-side architecture.
Why does nightly batch sync of customer data fail multi-location operators?
A member updates their email address at 9am at one location during checkout. The status-quo workflow runs a nightly Reverse-ETL job that pulls the change from the customer-graph and pushes it to the ESP, the CRM, the loyalty platform, the SMS provider, and the push channel. The job fires at 2am the following day. Until 2am, every downstream system holds the old email. Receipts route to the old address. Loyalty notifications route to the old address. The marketing email scheduled for 10am goes to the old address. By the time the change propagates, eighteen hours of customer touchpoints have used stale data and at least one of them probably bounced or routed to spam.
How is this different from Segment Functions, mParticle Audiences, RudderStack, Census, Hightouch, or Twilio Engage?
Those platforms ship the pub-sub primitive — schema registry, partition keys, event delivery, retry, DLQ, observability. They are excellent at the primitive layer. The 7-axis customer-graph integration (Ingest + Resolve + Version + Compute-LTV + Cohort + Subscription-Ingest + Emit-Change), the per-banner customer-graph reconciliation, the per-jurisdiction PII handling (CCPA + GDPR per-state), the cross-arc topic design that ties customer events to catalog events to order events, the event-schema evolution policy across 5-15 downstream consumers — that orchestration layer is operator-side wiring that sits on top of the pub-sub primitive.
How do you handle event ordering and idempotency for customer change events?
Event ordering uses the customer account ID as the partition key on the pub-sub topic. Every change for a single member routes to the same partition, which guarantees ordered consumption per account. Idempotency uses a per-event unique ID plus consumer-side deduplication — every consumer tracks the highest event ID processed per account and discards any duplicate it receives during retries. The combination converts the at-least-once delivery guarantee of pub-sub into exactly-once consumption per account.
How do you handle customer change events across banners for a multi-banner operator?
Each banner emits to a per-banner topic (customer.banner-a.change, customer.banner-b.change, customer.banner-c.change). A cross-banner reconciliation consumer subscribes to all per-banner topics and maintains the parent-level customer-graph view. Cross-banner accounts that exist in two or more banners trigger a reconciliation event that publishes to a parent-level cross-banner topic. Downstream consumers can subscribe to either the per-banner topic (banner-specific behavior) or the parent topic (cross-banner behavior). The topic design encodes the multi-banner architecture in the event taxonomy.
How do you handle PII in customer change events under CCPA, GDPR, and per-state regulations?
PII fields are tagged in the event schema with the regulatory tier they belong to (CCPA-California, GDPR-EU, HIPAA-healthcare, FERPA-education). The consent-state of the member is encoded in the event payload. Downstream consumers receive the event with PII redacted to whatever the consumer is authorized to see — the ESP gets email and name, the SMS provider gets phone, the loyalty platform gets the full record. A per-jurisdiction policy engine evaluates the event against the consent-state and the consumer-authorization table before delivery. Right-to-deletion events propagate as a dedicated event type that triggers tombstoning in every downstream consumer.
Hire the agent that emits the change events
The customer-data-graph agent owns the 7-axis customer-graph pipeline — Ingest + Resolve + Version + Compute-LTV + Cohort + Subscription-Ingest + Emit-Change — sitting on top of whichever pub-sub layer (Kafka, Confluent, AWS EventBridge, Google Pub/Sub) and CDP (Segment, mParticle, RudderStack, Twilio Engage) you license downstream. Customer changes propagate to every consumer in seconds, with per-banner topics, per-jurisdiction policy enforcement, and cross-arc topic design.
We scope on the call and send a private checkout link after.
Related reading: Runtime customer cohorts · Per-location CLV