Shifting Profiles

Shifting Profiles

A specialty insurance provider offers auto, home and umbrella insurance coverage exclusively to educators in a few states. The company primarily attracts new prospects through direct marketing and by networking with education associations, touting below-average insurance rates due to the attractive risk profile that educators represent. Since the company only offers insurance to educators, their rates are fine-tuned just to that audience. Recently the company expanded geographically into two new states.

The problem:

The company noticed their core metrics begin to dip, specifically average premium and retention rates.  Like any subscription-based product, insurers need customers who buy several insurance products and then stay with the company for several years. This is especially the case for direct marketers since their marketing cost is front-loaded. What caused the shift?

The approach:

We received a multi-year dataset of customers and then appended third-party data to provide explanatory attributes for segmentation modeling. Using principal component analysis (PCA) and a K-means clustering algorithm, we identified the optimal number of distinct clusters for segmentation. I like this process because it doesn’t arbitrarily split the audience using a pre-determined number of splits. Instead, this approach completes segmentation using two clusters, then three, then four, all the way up to around 200, measuring the statistical improvement in BIC (Bayesian Information Criterion) through each iteration. The goal of this step is simply to determine the optimal number of clusters. Once this step is done, we follow-up with agglomerative hierarchical clustering, which builds a tree dendrogram from the ground up rather than a divisive process which is a top-down approach. The segments are derived by minimizing the variance of all explanatory attributes within a cluster and maximizing the variance across clusters.


The result:

Three distinct customer profiles emerged from the segmentation. The customer profile began shifting two years prior to one emerging segment, slowly driving down average premiums and retention rates.


Interestingly, most of the profile shift could be explained by demographics and spending behavior. The company’s recent growth efforts attracted younger teachers vs the higher-paid administrators and tenured teachers. This new profile skewed younger, with a much lower length of residence and a much higher presence of children. This segment purchased fewer insurance products on average (typically auto insurance only), and had fewer cars to insure and a lower policy retention rate. The segment also skewed female, rented and had a lower household income.

Segmentation modeling is not predictive in nature. The goal is simply to create distinct profiles based on similar attributes and behaviors. This information can then inform strategy, such as creative design, offer, messaging, and even what the marketing allowable should be considering the disparity in lifetime revenue of this segment vs the others.