Case study  ·  Data science  ·  Health analytics

Who is costing the plan the most? Predicting healthcare cost

My master's capstone. Using 10,578 adults from the federal MEPS 2022 survey, I modeled what each chronic condition adds to a person's annual healthcare cost and built a classifier that ranks people by their risk of landing in the top 20% of spenders, for a self-insured employer deciding where wellness money goes.

ProgramM.S. Data Analytics (Data Science), WGU
DataMEPS 2022 (AHRQ, public)
ModelsOLS, logistic regression, random forest benchmarks
CompletedJune 2026

Self-insured employers pay every claim, and a few people drive most of it

A self-insured employer pays employee healthcare claims straight out of operating funds, so every claim dollar is a budget dollar. Spending is heavily concentrated in a small share of people, yet wellness budgets usually get spread evenly across the whole workforce.

The question was whether public data could show which conditions drive cost, by how much and for whom, and whether a model could pick out likely high-cost people well enough to aim outreach at them.

Pre-registered targets

The cost model had to explain more than 10% of the variation in annual cost, and the high-cost classifier had to beat 70% accuracy on held-out data. Both were met, and section 05 covers why accuracy turned out to be the wrong headline for the second one.

A nationally representative survey, cut to clean adult records

22,431
Survey respondents
MEPS HC-243, the 2022 full-year consolidated file from AHRQ.
10,578
Adults analyzed
Age 18+, valid BMI and diagnosis codes, complete cases only.
9
Study variables
Diabetes, hypertension, high cholesterol, BMI, age, sex, race, income and total annual expenditure.

Cost is heavily right-skewed (a lot of people spend almost nothing and a few spend enormous amounts), so the regression models predict log annual expenditure. That turns each condition's coefficient into a cost multiplier, which is easier for a benefits team to read than a log-dollar figure.

Two models, each answering a different half of the question

01
Baseline vs full cost model
OLS on log expenditure with demographics only, then with the four metabolic conditions added. Demographics alone explained 20.0% of the variation and the conditions added 5.3 points on top (ΔAIC of −713), so the conditions carry real signal beyond age, sex, race and income.
02
Stratified and interaction models
The full model was refit separately by sex and by age band (18 to 40, 41 to 60, 61+), plus a version with age × condition interactions to test whether a diagnosis costs more at some ages than others.
03
High-cost classifier
Logistic regression predicting whether a person lands in the top 20% of annual spending ($12,879 or more), trained on 80% of the data and scored on a stratified 20% holdout.
04
Random forest benchmarks for both
The random forest regressor hit an R² of 0.89 on training data and 0.21 on the holdout, a clear overfit. OLS held at 0.24 on the holdout. The simpler models generalized better on both tasks, and they're the ones a benefits team can actually read.

Diabetes drives cost, and hits hardest in younger adults

ConditionCost multiplier
Diabetes2.77×
High cholesterol1.88×
Hypertension1.83×
BMI (per unit)1.01×

Expected annual cost against a similar person without the condition, controlling for age, sex, race and income.

6.0×
Diabetes, ages 18 to 40
The effect shrinks with age (3.7× at 41 to 60, 2.0× at 61+), and the age × diabetes interaction is significant.
8×
Three conditions vs none
Median annual cost of $11,347 with all three diagnoses against $1,422 with none.
29.5%
Variance explained, men
Against 19.2% for women. BMI was significant for women and not for men.
Diagnosed conditionsPeopleMedian annual cost
None4,555$1,422
One2,692$3,566
Two2,289$5,699
All three1,042$11,347

80% accuracy sounds better than it is

The logistic model scored 80.4% accuracy on the holdout, which cleared the 70% target. But only 20% of people are high-cost by definition, so a model that predicts "not high-cost" for everyone scores 80% without learning anything. Accuracy was the wrong yardstick for a target this imbalanced, and I'd pick a different one if I set the targets again.

The numbers that matter are how well it ranks people and what you get when you act on that ranking:

0.71
AUC
Logistic regression. The random forest scored 0.67.
39%
Hit rate, top-scored 20%
Of the people it ranks highest, 39% really are high-cost.
~2×
Lift over random
Random selection would hit 20%.
10%
Recall at default cutoff
Left at 0.5, it flags very few people. Use it as a ranker.

Used the way an employer would actually use it (score everyone, sort and reach out to the top fifth first), outreach lands on a high-cost person about twice as often as spreading it evenly. That's a useful screening tool. It doesn't predict any one person's cost, and nine demographic and condition variables were never going to.

The ranking and recall figures come from re-running the submitted model after the capstone. The submitted presentation reported accuracy and AUC.

What these models can and can't support

Limits

  • One year of cross-sectional data, so it shows association and not cause. Treating diabetes may or may not lower cost.
  • Diagnoses are self-reported, which undercounts conditions in people without regular care.
  • Survey weights weren't applied, so results describe this sample and not precise national estimates.

Next steps

  • Add insurance type, visit counts and prescription spend, all in MEPS and unused here.
  • Use the two-year MEPS panel to follow the same people and see whether a new diagnosis raises their cost.
  • Tune the classifier threshold for the budget an employer actually has, and report precision at that cutoff.

Standard Python data science, start to finish

Language
Python
Data
pandas
Statistics
statsmodels
ML
scikit-learn
Charts
matplotlib / seaborn
Ingest
pyreadstat
Source
MEPS HC-243
Program
WGU D606