Market Basket Analysis

Market basket analysis uses purchase transaction data to identify associations between products or combinations of products and services that occur together frequently. A market basket analysis can be applied on past customer transactions at the store or on the website to understand customer buying habits.

Instructor Note: The following video is a great overview of market basket analysis and help support the lecture below and it is up to you as the instructor to determine which parts of the video you share and when.

Decision Making using market basket analysis

There are many practical applications of market basket analysis, and both online and brick-and-mortar businesses, particularly retailers, apply market basket analysis. Once a product is placed in a shopping basket, market basket analysis enables companies to influence a customer’s selection of other products. While some of these products may be on the customer’s shopping list, other product selections could be influenced by marketers using different strategies to remind or introduce customers to additional products or services. While market basket analysis is useful for in-store recommendations, it is even more valuable for online purchasing because the information on product selection is in real time – once the item is placed in an online shopping cart, the retailer can provide immediate suggestions of other items customers are likely to purchase. Amazon uses collaborative filtering to offer customized recommendations to customers making online purchases. Target has been successful using transaction data to engage customers. They use customer sales patterns to influence and assist with product selections. Traditional supermarkets use the analysis to plot item location in their stores to increase sales. Market basket analysis enables marketers to identify what products are being purchased together. Uncovering relationships between products and service bundle purchases is important for understanding customer purchase patterns and maximizing sales.

Affinity Analysis

Market basket analysis is an unsupervised learning algorithm that explores patterns between two or more items using association rules, or affinity analysis. An association rule helps define relationships in a transaction dataset using if-then statements. By analyzing transaction data, we discover dependencies between items purchased in the form of IF [item A] THEN [item b] statements. For example, if soda (antecedent) is purchased, then milk (consequent) will also likely be purchased. Thus, the antecedent corresponds to the IF item, and the consequent corresponds to the THEN item in the association rule. Association analysis identifies that the antecedent and consequent of the rule share a co-occurrent relationship. These rules are commonly generated using the Apriori algorithm. The Apriori algorithm identifies combinations of items in datasets that are associated with each other. Associations are identified based on the frequency in which the products occur together in the basket.

Concepts of support, confidence, and lift.

When using market basket analysis, there are three measures that indicate the reliability of the associations: support, confidence, and lift. Support measures the frequency of the specific association rule and shows the number of transactions that include the items of interest divided by the total number of transactions. (Using Exhibit 9-5) For example, the itemset [Soda; Milk] appears six out of ten times. Support can be calculated as 6/10 or 0.60.  This means, 60 percent of baskets contain both soda and milk.

Confidence measures the conditional probability of the consequent occurring given that the antecedent occurs. It is calculated as the count of purchases consisting of both items in the association rules divided by the total number of times the antecedent item is purchased. This measure indicates the percentage of times the association rules is correct. Here, using the association rule IF [soda] THEN [milk], confidence is 6/8 or 0.75.  When a grocery basket contains soda, 75 percent of them also contain milk.

Finally, lift enables us to evaluate the strength of the association. In a lift ratio. We divide the confidence with a benchmark score called ‘expected confidence.’  Calculate expected confidence as the number of total transactions that includes the consequent item divided by the total number of transactions. So, using the association rule IF [soda] THEN [milk], expected confidence would be 6/10 or 0.6.  Then lift ratio is calculated as the ratio of the confidence divided by the expected confidence, here 0.75/0.60 or 1.25.  Lift validates that a transaction containing both products is not random. A lift value of 1 indicates no relationship within the itemset. A lift value higher than 1 indicates a relationship between the products or services that is higher than would be expected if they were independent. A lift value below 1 indicates a negative relationship that the products or services in the itemset are unlikely to be purchased together. If soda is purchased, then with a confidence of 75 percent milk will also be purchased. The lift of 1.25 suggests that the purchase of milk is 1.25 times more likely when soda is purchased compared to shoppers just purchasing milk.

Collaborative filtering and association rule

In association rules, the goal is to find items that are frequently purchased together and to make a customer recommendation every time one of those items is added to the basket. This application is useful when there are products or services frequently purchased together repeatedly. But what is a company wants to offer personalized recommendations for the user when items are not purchased as frequently? In cases like this, collaborative filtering can be used which enables us to analyze data on user preferences, then recommend products based on their similarity to other customers. The goal of collaborative filtering is to recommend new products and services to a customer who has not purchased them before. The goals differ from collaborative filtering to association rules. While association rules use if-then statements, collaborative filtering has two main types: item to item filtering and user to item filtering.