Unifying Anonymous and Known Customer Profiles with AWS Entity Resolution
Discover how to implement an identity resolution strategy using AWS Entity Resolution to unify anonymous and known customer profiles across channels. Learn about ingesting click-stream data, deterministic and probabilistic matching, and storing unified views in Amazon DynamoDB for personalized customer experiences.
Published May 1, 2024
In the age of increasing data privacy regulations and walled gardens, it is more important than ever for brands to own their first party data. First-party data is the data that a company collects directly from its customers, with their consent, through interactions on its own platforms and channels.
First-party data helps companies understand their customers better, create more relevant and personalized experiences, and increase customer loyalty and retention. This type of data is the most valuable for businesses because it is the most accurate and reliable.
However, collecting and managing first-party data can be challenging, especially when customers interact with a company through multiple channels and devices. For example, a customer may browse a company's website on their laptop, make a purchase on their mobile app, and join a loyalty program at a physical store.
Identity resolution is a key component of any company’s first party data strategy. It allows brands to unify customer data from different sources and create a single, unified view of each customer. This is where AWS Entity Resolution comes in.
AWS Entity Resolution is a fully managed service that helps companies match, link, and enhance related records without the need to build custom solutions. The service can match and link records that belong to the same entity (such as a customer) across different disparate datasets, even if the records do not share common identifiers or have incomplete or inconsistent information. Companies can also use the service to link and enhance their records with data service providers.
AWS Entity Resolution can help companies build a first-party data strategy by connecting customer information into unified views from a variety of sources, such as:
- Website analytics data
- Customer relationship management (CRM) data
- Loyalty program data
- Point-of-sale data
- Social media data
Once unified customer views have been created, Companies can use these records in their CDP or audience segmentation tool of their choice to create audience segments that can receive personalized marketing campaigns and product recommendations.
AWS Entity Resolution uses following three methods to match anonymous customer records with known customer data:
- Deterministic rule-based matching: This type of matching uses unique identifiers, such as email addresses or phone numbers, to match anonymous customer data with known customer data.
- Probabilistic machine learning matching: This type of matching uses a variety of factors, such as name, address, and purchase history, to probabilistically match anonymous customer data with known customer data using ML.
- Data service providers enabled matching: This type of matching uses third-party data service providers, such as Unified ID 2.0, LiveRamp, and TransUnion, to enrich your records with additional attributes or identifiers from their databases. You can also use their matching algorithms to link records across different sources. With this matching workflow, you can enhance your records through column appends, or you can translate customer data into data service provider IDs to meet your business goals, such as delivering more relevant and complete customer experiences.
In this section we will explore how you can utilize AWS Entity Resolution to implement an identity resolution strategy to unify customer records views throughout the customer journey, enabling companies to associate disparate anonymous records with their known customer.
An anonymous customer, also known as a "guest" or "anonymous user," is a person who accesses and interacts with a website or application without creating an account or providing any personal information. They are not logged in to the system and do not have a unique identifier associated with their activity. On the other hand, a known customer, also referred to as a "registered user" or "authenticated user," is a person who has created an account on a website or application and has provided personal information, such as their name, email address, and password, to access the site's services or features. This allows them to log in and access their account information, purchase history, wish lists, and other personalized content.
In the proposed solution we will leverage first-party cookies to store a generated AWS Entity Resolution match ID and collect customer interactions (records) during web visits. First-party cookies are small text files that are stored on a user's device by a website. They can be used to store a variety of information, including anonymous user data. Then, once the customer provides personal information during the registration process, sign-in or log-on, or transaction (place order) activities, we will link the match ID stored in the first-party cookie to the personal information identifier (email address in this case).
The following diagram shows how the proposed identity resolution solution works when a customer visits a website from two different devices:
Figure 1 Anonymous to Known De-duplication
Day 1: The user visits the website from device A and interacted with its content to generate two events: “View Product” and “Add To Cart”. However, the user still Anonymous since no 1st party identifier and it is only identifier using a first-party cookie (1234).
Day 2: The user visits again the same website from the same device A and proceeds to register using email address (user@email.com). At this stage, the solution will link the first-party cookie (1234) with email address (user@email.com) since it belongs to the same user.
Day 3: The user receives a marketing email sent to user@email.com and clicks on the link within the email to land on the same website from a different device B. Since it is a new device the website tracking capability assign a new first-party cookie (5678) to the user in device B. Then, the user sign-in using email address user@email.com. The solution will link the first-party cookie set in device B (5678) to the used email address user@email.com.
As a result, the solution will be able to link all events from both devices from Anonymous to known stage using first-party cookies and email address first-party identifier.
Solution Architecture
The following diagram depicts a high-level technical architecture for using AWS Entity Resolution to unify anonymous and known customer profiles from your website:
Figure 2 Architecture Diagram
- Data ingestion: Businesses ingest their customer data into AWS Entity Resolution with following pattern:
- The user accesses the front-web web application via Amazon CloudFront.
- Amazon API Gateway is used to capture the click-stream and respond the API calls.
- Amazon API Gateway forward the click-stream to Amazon Kinesis Data Firehose process the data, apply transformation as needed and store the files in Amazon Simple Storage Service.
- AWS Glue is used to catalogue the click stream data hosted in the Amazon S3 into table.
- Data preparation:
- AWS Entity Resolution uses AWS Glue to define the schema from the created table
- S3 Events trigger matching workflow once a new clickstream file ingested in the Amazon S3.
- Entity matching:
- AWS Entity Resolution matches customer profiles across the different datasets. This is done using a combination of Deterministic Rule-based matching, or Probabilistic Machine learning matching.
- The AWS Entity Resolution workflow result is dropped in Amazon S3.
- Data output:
- AWS Lambda is triggered once the match workflow output is dropped in Amazon S3. The role of AWS Lambda is to process the matched records from Amazon S3 and ingest them in Amazon DynamoDB table
- Once the customer profiles have been matched, you can enrich the data with additional information, such as demographics, interests, and purchase history utilizing the AWS Entity Resolution data service provider matching workflow
- Unified Profile Lookup:
- Once the AWS Entity Resolution job is completed and the matched records are stored in Amazon DynamoDB table, the front-end web application can retrieve the unified record views using AWS Lambda invoked via API call from the Amazon API Gateway.
- The AWS Lambda function uses AWS SDK to query the Amazon DynamoDB table and return the unified record views to the front-end web application.
The full architecture deployment is available in our Git repository including deployment steps and demo screenshots.
AWS Entity Resolution can help business in solving customer experience problems:
- A retailer can use AWS Entity Resolution to create unified customer views across their website, mobile app, and physical stores. This can help them to better understand the customer journey and identify opportunities to improve the customer experience. For example, they could use AWS Entity Resolution to identify customers who are browsing products on their website but not abandoning their carts. They could then send these customers targeted emails or push notifications with personalized recommendations or offers.
- A hotel can use AWS Entity Resolution to create unified customer views across their website, mobile app, and customer service center. This can help them to better understand the customer experience and identify areas for improvement. For example, they could use AWS Entity Resolution to identify customers who are calling customer service to change their reservations. They could then investigate why these customers are changing their reservations and make changes to their website or mobile app to make it easier for customers to book rooms and manage their reservations.
- An airline can use AWS Entity Resolution to create unified customer views across their website, mobile app, and airport kiosks. This can help them to provide a more personalized customer experience. For example, they could use AWS Entity Resolution to identify customers who are waiting in line at an airport kiosk. They could then send these customers targeted messages with information about their flight status, gate information, and boarding time.
Conclusion
AWS Entity Resolution is a valuable service for businesses that want to build a first-party data strategy and leverage the resolved records to improve customer segmentation and personalization. By creating unified views of their customers across devices and sources, AWS Entity Resolution can help businesses better understand their customers and create more personalized experiences. This can lead to increased customer satisfaction, loyalty, and revenue.
Further Reading