Now you can see me

Overview 

As customers, we shop with various brands (i.e. labels, retailers and ecommerce marketplaces). Once, we grow comfortable with certain bands, we tend to interact more frequently with them. In today's channel-less world, this interaction takes various forms - visiting stores, following the social media updates for inspiration, searching/browsing online in the apps and last but not the least talking to customer service.

However, despite all the "breadcrumbs" we leave behind with the brand, the experience often leaves us wanting more. On the other hand, brands are increasingly wanting to know and understand about their customers. Various attempts are made at "personalizing" the experience. This is mainly along three pillars below -

1. Elevating service in physical touchpoints such as stores
2. Loyalty programs to incentivize repeat purchases
3. Seeking out patterns in data from products, services, channels and customers

Out of the the above 3, #1 and #2 can be expensive, prohibitively so sometimes. Often the more reliable way is #3 - to get the insights from data. Historically, this has been easier said than done as it involves investment in creating large data science and analytics teams who are constantly monitoring customer behavior. The more detailed and granular the monitoring, it helps brands to take proactive and corrective actions.

Brands who are unable to fully execute the 3 pillars have a gap. Let's call this the "satisfaction gap". As you can see in the model below, these gaps exist at multiple levels. The gaps start with understanding the intent and need of customer to executing the relevant experience for the same customer. 

Credits: Howard Tiersky @ CMSWire

Bridging the Satisfaction Gap

If we were to go down pillar #3, in the last couple of decades - technologies, frameworks, design and architecture patterns and engineering practices have evolved to a great extent.

1. Databases have evolved from having rigid data structure to flexible schemas
2. Cloud has been a huge enabler to store and process data at scale
3. Business Intelligence tools have evolved from being able to just "view" dashboards to "actioning" insights
4. Multiple tools have emerged to manage data lineage and governance
5. Data engineering has finally borrowed the best practices from software engineering around topics like - infrastructure as code, test automation and data meshes. 

Art of the Possible, Today

This is where a combination of analytics, AI and Generative AI can help understand customers better. Before we get into the technology, let's look at the constraints that brands normally have. These generally tend to be -

1. Limited data, especially when it comes to the brick-and-mortar stores
2. Paucity of talent that is required to build and maintain data platforms

Let's face it, these are not challenges that are easy to solve. So, what can a brand do? Here's a quick guide -

1. Control the Controllable. Look at the above constraints and work with them for progressing further.
2. Small data, big data are just labels. More important question is - Is it "good enough" data? Is it available at the right time to the right users? Are decisions taken on "gut feeling" or based on data? We get more value on ensuring the business processes and IT systems capture and maintain the existing data better.
3. Build insights over the existing data, some of these can be deterministic (based on facts), some of these can be probabilistic (derived from facts).
4. Brands often obsess over insight that tell "WHAT" happened in the past. No doubt, this is required. Especially to measure and adjust business performance and operations.
5. Brands also need to obsess over "WHY" something happened and "HOW" do we improve it in future. This comes from insights that are actionable. 

All these might sound very high level. So, let's look at some concrete examples of how actionable insights can be generated even from the most basic data that is available today. 

Capturing Intent Signals

Brands already capture customer intent in various forms. The diagram below illustrates the typical customer journey. As the customer progresses down the journey, the strength of the signal gets stronger.

Credits: Rajan Manickavasagam

All of the above steps in the customer journey are already captured by a brand. All it needs is stitching them together in a data platform and extracting the insights. The most crucial insights would be customer attributes and segments.

Understanding the Intent

Now comes the interesting part. Broadly, we could divide the customer attributes into two types - deterministic and probabilistic. The basis for creating these attributes all come from a few forms of economics namely - microeconomics, behavioral economics and consumer theory. Of these, perhaps the most fascinating is behavioral economics which is a combination of economics and psychology.

The deterministic part of understanding intent could be captured by following attributes. These are calculated base on business rules, statistics and plain mathematics.

1. Frequency of purchase
2. Recency of purchase
3. Basket Mix
4. Channel Affinity
5. Loyalty
6. Transaction Value
7. Time between purchases
8. Locations of purchase

The probabilistic part of understanding intent is captured by the following attributes. And, this is where we can start leveraging Generative AI.

1. Demographics
2. Lifestyle
3. Behaviors and Tendencies
4. Habits
5. Interests

When we combine the deterministic and probabilistic attributes, we can create almost unlimited customer segments down to that elusive "Segment of 1". This is where each customer is identifiable individually. Something like how each viewer gets a very personalized homepage when they open Netflix. Here is a wonderfully concise article on the personalization journey at Netflix by Gibson Biddle, former VP/CPO of Netflix.

Coming back to retail brands, here is what a typical retail store transaction looks like. Most of the below data comes from the point of sale systems augmented by other sources like merchandising system as well external sources like weather.

Transaction
{ 
 time: "28-10-2023",
 weather: "30 celsius, humidity 60%",
 store: "flagship store, orchard road, singapore",
 amount: 1000,
 currency: "singapore dollar",
 loyalty_points_earned: 100,
 loyalty_points_redeemed: 300,
 items : {
   item_1: {
    product_name: “t-shirt long sleeve (black color)”,
    product_category: “womens - casual - t-shirt”,
    quantity: 3,
    price: 300 
   },
   item_2: {
    product_name: “womens mid rise jeans (medium size)”,
    product_category: “womens - casual - bottoms”,
    quantity: 2,
    price: 600 
   },
   item_3: {
    product_name: “cashmere scarves (black and white)”,
    product_category: “womens - accessories”,
    quantity: 1,
    price: 100 
   },
 }
}

Let's use the PaLM model from Google Cloud Vertex AI platform. We pass the above transaction to the text-bison large language model and prompt it with the below instruction.

Segments
Season - Unknown, Spring, Summer, Autumn, Winter
Loyalty - Unknown, Casual, Committed
Purpose of Shopping - Unknown, Travel, Weekend, Office, Mixed
Demographics - Male, Female, Transgender, Gender neutral, 
Non-binary, Agender, Pangender, Genderqueer, Two-spirit, 
Third gender, All, None, Unknown
Age - 20-30, 30-40, 40-50, 50-60, Unknown

For the above transaction, provide possible segments and values in 
the following json format -
{segment name: {[multiple segment values], [explanation of segment]}}

The response from the model is below. Of course, more prompt engineering can be done to further customize the response.

{
  "Season": ["Unknown", "Summer"],
  "Loyalty": ["Committed"],
  "Purpose of Shopping": ["Casual", "Office"],
  "Demographics": ["Female"],
  "Age": ["20-30"],
  "Segments": {
    "Season": "The transaction took place in October, 
    which is the summer season in Singapore. 
    However, the season cannot be determined 
    conclusively from the transaction data alone.",
    "Loyalty": "The customer redeemed 300 loyalty points, 
    which suggests that 
    they are a committed customer.",
    "Purpose of Shopping": "The customer purchased a mix of 
    casual and workwear items. The purchase of three 
    long-sleeved t-shirts suggests that the customer may be shopping 
    for the cooler autumn weather. Additionally, the purchase 
    of two pairs of mid-rise jeans and one cashmere scarf 
    suggests that the customer may also be shopping for workwear.",
    "Demographics": "The customer purchased women's clothing, 
    suggesting that they are female.",
    "Age": "The customer spent SGD 1000, which is a 
    relatively high amount for clothing. This suggests that 
    they are likely to be in the 20-30 age group, when people are 
    typically more likely to have disposable income."
  }
}

Let's look at another sample transaction, this time from the online channel.

Transaction
{ 
 time: "29-10-2023",
 weather: "autumn",
 channel: "mobile app",
 amount: 500,
 currency: "singapore dollar",
 loyalty_points_earned: 0,
 loyalty_points_redeemed: 0,
 items : {
   item_1: {
    product_name: “formal shirt short sleeve (white color)”,
    product_category: “mens - formal - shirt”,
    quantity: 1,
    price: 100 
   },
   item_2: {
    product_name: “mens linen trouser - black (small size)”,
    product_category: “mens - formal - bottoms”,
    quantity: 2,
    price: 150 
   },
   item_3: {
    product_name: “pin stripe tie - blue and red”,
    product_category: “mens - accessories”,
    quantity: 1,
    price: 100 
   },
 }
}

We pass the above online transaction to the text-bison large language model and prompt it with the same instruction as before.

Segments
Season - Unknown, Spring, Summer, Autumn, Winter
Loyalty - Unknown, Casual, Committed
Purpose of Shopping - Unknown, Travel, Weekend, Office, Mixed
Demographics - Male, Female, Transgender, Gender neutral, 
Non-binary, Agender, Pangender, Genderqueer, Two-spirit, 
Third gender, All, None, Unknown
For the above transaction, provide possible segments and values 
in the following json format -
{segment name: {[multiple segment values], [explanation of segment]}}

The response from the model is below. This time, the response is tailored to the transaction as well.

{
  "Season": ["Unknown", "Summer", "Autumn"],
  "Loyalty": ["Unknown"],
  "Purpose of Shopping": ["Office", "Rainy Day"],
  "Demographics": ["Male"],
  "Age": ["20-30", "30-40"],
  "Segments": {
    "Season": "The transaction occurred in October, 
	which is autumn in Singapore.",
    "Loyalty": "The customer did not redeem or earn 
    any loyalty points, which suggests that they are 
    either a new customer or a customer who does not 
    shop at the store often.",
    "Purpose of Shopping": "The customer purchased a 
    formal shirt, trousers, and tie, which suggests 
    that they are shopping for work or a special occasion. 
    The customer may also be purchasing these items for a 
    rainy day, as they are all made of materials that 
    are water-resistant or quick-drying.",
    "Demographics": "The customer's age is unknown, but 
    the average age of customers who purchase formal wear 
    is typically between 20 and 30 years old.",
    "Age": "The customer purchased items that are 
    typically popular with young adults and adults in 
    their early 30s, such as a formal shirt and linen 
    trousers. This suggests that they are in the 20-30 
    or 30-40 age range."
  }
}

Imagine if we could run this on all store and online transactions. Even with the most basic data available with a brand, we are able to derive customer insights into - demographic, lifestyle, interests and habits. And, all of this is done in a privacy friendly way without the need for personal information like - name, phone and email.

Now, brands can get more insight into not just what is selling well or not well, but also get a better understanding of who is buying the products and why. The prompts could be further optimized by using other prompt engineering strategies like - few shot, chain of thought, tree of through etc.

The prompt could be customized to the type of retail brand (convenience, grocery, department store, fashion, luxury, marketplace etc.) as well as the attributes that the brand is interested in. Of course, these attributes would have to be calculated/re-calculated on a weekly or fortnightly basis to keep capturing the latest customer "zeitgiest'.

Creating Experience from the Intent

Once the brand has derived the customer insights from the various data points like - purchase transactions, customer feedback, product reviews, this can be leveraged across the organization. The potential applications of the customer attributes and segments can be -

1. Optimize assortment by store / location
2. Additional signal for refining pricing and promotions (especially quantum and timing)
3. Personalize the digital experience (homepage, search ranking and product recommendations)
4. Decide format and location of stores
5. Contextual digital and store marketing campaigns

Summary

In the next post, we will see what are some of the best practices to close the "satisfaction gap". 

Dear brands, now that you can see me clearer, you can serve me better. Mind the Gap!