The Cloudflare team was so excited to hear how Twilio Segment solved problems they encountered with tracking first-party data and personalization using Cloudflare Workers. We are happy to have guest bloggers Pooya Jaferian and Tasha Alfano from Twilio Segment to share their story.
Introduction
Twilio Segment is a customer data platform that collects, transforms, and activates first-party customer data. Segment helps developers collect user interactions within an application, form a unified customer record, and sync it to hundreds of different marketing, product, analytics, and data warehouse integrations.
There are two “unsolved” problem with app instrumentation today:
Problem #1: Many important events that you want to track happen on the “wild-west” of the client, but collecting those events via the client can lead to low data quality, as events are dropped due to user configurations, browser limitations, and network connectivity issues.
Problem #2: Applications need access to real-time ( { const { settings: { writeKey }, }=context; const host=request.headers.get(“host”) || “”; return [ request, new HTMLRewriter().on(“head”, new ElementHandler(host, writeKey)) .transform(response), context, ]; };
Proxy SDK bundles and Segment API
The Edge SDK proxies the Segment CDN and API under the first-party domain. For example, when the browser loads a page with the injected analytics.js snippet, the snippet loads the full analytics.js bundle from https://example.com/seg/assets/sdk.js
, and the Edge SDK will proxy that request to the Segment CDN:
https://cdn.segment.com/analytics.js/v1//analytics.min.js
export const proxyAnalyticsJS: HandlerFunction=async (request, response, ctx)=> { const url=`https://cdn.segment.com/analytics.js/v1/${ctx.params.writeKey}/analytics.min.js`; const resp=await fetch(url); return [request, resp, ctx]; };
Similarly, analytics.js collects events and sends them via a POST request to https://example.com/seg/events/[method]
and the Edge SDK will proxy such requests to the Segment tracking API:
https://api.segment.io/v1/[method]
export const handleAPI: HandlerFunction=async (request, response, context)=> { const url=new URL(request.url); const parts=url.pathname.split("/"); const method=parts.pop(); let body: { [key: string]: any }=await request.json(); const init={ method: "POST", headers: request.headers, body: JSON.stringify(body), }; const resp=await fetch(`https://api.segment.io/v1/${method}`, init); return [request, resp, context]; };
First party server-side cookies
The Edge SDK also re-writes existing client-side analytics.js cookies as HTTPOnly cookies. When Edge SDK intercepts an identify
event e.g., analytics.identify('john')
, it extracts the user identity (“john”) and then sets a server-side cookie when sending a response back to the user. Therefore, any subsequent request to the Edge SDK can be associated with “john” using request cookies.
export const enrichResponseWithIdCookies: HandlerFunction=async ( request, response, context)=> { const host=request.headers.get("host") || ""; const body=await request.json(); const userId=body.userId; […] const headers=new Headers(response.headers); const cookie=cookie.stringify("ajs_user_id", userId, { httponly: true, path: "/", maxage: 31536000, domain: host, }); headers.append("Set-Cookie", cookie); const newResponse=new Response(response.body, { ...response, headers, }); return [request, newResponse, newContext]; };
Intercepting the ajs_user_id
on the Workers, and using the cookie identifier to associate each request to a user, is quite powerful, and it opens the door for delivering personalized content to users. The next section covers how Edge SDK can drive personalization.
Personalization on the Supercloud
The Edge SDK offers a registerVariation
method that can customize how a request to a given route should be fetched from the origin. For example, let’s assume we have three versions of a landing page in the origin: /red
, /green
, and /
(default), and we want to deliver one of the three versions based on the visitor traits. We can use Edge SDK as follows:
const segment=new Segment(env.SEGMENT_WRITE_KEY); segment.registerVariation("/", (profile)=> { if (profile.red_group) { return "/red" } else if (profile.green_group) return "/green" } }); const resp=await segment.handleEvent(request, env); return resp
The registerVariation
accepts two inputs: the path that displays the personalized content, and a decision function that should return the origin address for the personalized content. The decision function receives a profile object visitor in Segment. In the example, when users visit example.com/(root path)
, personalized content is delivered by checking if the visitor has a red_group
or green_group
trait and subsequently requesting the content from either /red
or /green
path at the origin.
We already explained that Edge SDK knows the identity of the user via ajs_user_id
cookie, but we haven’t covered how the Edge SDK has access to the full profile object. The next section explains how the full profile becomes available on the Cloudflare Workers platform.
How does personalization work under the hood?
The Personalization feature of the Edge SDK requires storage of profiles on the Cloudflare Workers platform. A Cloudflare KV should be created for the Worker running the Edge SDK and passed to the Edge SDK during initialization. Edge SDK will store profiles in KV, where keys are the ajs_user_id, and values are the serialized profile object. To move Profiles data from Segment to the KV, the SDK uses two methods:
- Profiles data push from Segment to the Cloudflare Workers platform: The Segment product can sync user profiles database with different tools, including pushing the data to a webhook. The Edge SDK automatically exposes a webhook endpoint under the first-party domain (e.g.,
example.com/seg/profiles-webhook
) that Segment can call periodically to sync user profiles. The webhook handler receives incoming sync calls from Segment, and writes profiles to the KV. - Pulling data from Segment by the Edge SDK: If the Edge SDK queries the KV for a user id, and doesn’t find the profile (i.e., data hasn’t synced yet), it requests the user profile from the Segment API, and stores it in the KV.
Figure 3 demonstrates how the personalization flow works. In step 1, the user requests content for the root path ( / ), and the Worker sends the request to the Edge SDK (step 2). The Edge SDK router determines that a variation is registered on the route, therefore, extracts the ajs_user_id
from the request cookies, and goes through the full profile extraction (step 3). The SDK first checks the KV for a record with the key of ajs_user_id
value and if not found, queries Segment API to fetch the profile, and stores the profile in the KV. Eventually, the profile is extracted and passed into the decision function to decide which path should be served to the user (step 4). The router eventually fetches the variation from the origin (step 5) and returns the response under the / path to the browser (step 6).
Figure 3- Personalization flow
Summary
In this post we covered how the Cloudflare Workers platform can help with tracking first-party data and personalization. We also explained how we built a Segment Edge SDK to enable Segment customers to get those benefits out of the box, without having to create their own DIY solution. The Segment Edge SDK is currently in early development, and we are planning to launch a private pilot and open-source it in the near future.
Source: cloudflare.com