01 / DECISION · EXECUTIVE SUMMARY
Keep the simple benchmark when prioritizing slow cases.
An operations leader needs to decide which active applications deserve scarce review time. The detailed workflow model makes slightly better forecasts than a simple stage-and-age model, but the gain is small. It supports testing a prioritization process; it does not justify a claim of faster service or lower staffing cost.
What a leader can take from this
Use stage and age as a credible starting point. At event 20 and 20% review capacity, the detailed model identifies 218 late applications versus 215 for the simple benchmark. Evaluate whether that small targeting difference helps a real review team before investing in a complex deployment.
Forecasting improvement is not process improvement.
The paired error difference is -0.139 days (95% interval -0.181 to -0.093). That means roughly 3.3 hours less forecasting error, not applications completed 3.3 hours sooner. Predictions end at a defined recorded status and cap remaining time at 30 days.
02 / IMPLICATION
Investigate the handoff; test the intervention.
Among common transitions visible in early application prefixes, the gap from A_Complete to A_Validating has a median of 6.86 elapsed days across 266 observations. That suggests a concrete handoff to investigate. The timestamps alone do not distinguish waiting, customer response, batching or active employee work.
A sensible pilot would compare the existing review queue with model prioritization at the same capacity and measure actual downstream outcomes. This historical forecast study does not estimate that intervention. Its staffing controls expose assumptions so visitors can see how strongly a scenario depends on them.
03 / EVIDENCE
Follow the time cutoff, the simple alternatives and the uncertainty.
The final evaluation contains 2,365 distinct applications and 6,247 eligible prefixes. All prefixes from the same application stay together. Each application receives total weight one across its available prefixes.
| Model | MAE, days | 95% interval | 14-day Brier |
|---|---|---|---|
| Pooled survival baseline | 9.326 | 9.190–9.462 | 0.2455 |
| Current stage and age baseline | 8.722 | 8.597–8.869 | 0.2262 |
| Workflow hazard boosting | 8.583 | 8.459–8.734 | 0.2251 |
| Age and current stage only | 8.672 | 8.551–8.815 | 0.2245 |
Lower error and Brier scores are better. The reduced-history hazard model has a slightly better Brier score than the detailed model, so extra history does not win every comparison. The selected model’s nominal 80% prediction bands cover 88.5% of restricted outcomes: conservative coverage, not exact calibration.
WORKFLOW EVIDENCE EXPLORER
Where does a forecast help the review queue?
Explore saved December results. Stage and prefix filters change the evaluated cohort; no control refits a model or simulates new applications.
1,588 applications · 1,588 prefixes in this selection. Nominal 80% band coverage: 84.9%.
Average individual 80% prediction-band endpoints: 4.01–24.28 days. These averages are neither an interval for the group mean nor a guarantee for a particular application. Times are capped at 30 days. Subsets retain application-prefix weights; cells below 30 prefixes are unavailable.
Observed application-stage transitions
Each card is a recorded link in the process. Select an available destination stage to inspect its forecasts above. The map uses the longest eligible prefix per application, independently of the filters.
Edges with fewer than 30 observations are omitted. Disabled destinations lack an eligible diagnostic cell at this prefix. Elapsed time is not active work time; early prefixes omit later loops.
Prioritize late applications at a fixed review capacity
This comparison includes all stages at the selected prefix. “Late” means no target status within 14 days after that prefix. Stage filtering above does not change these capacity totals.
218 of 623late applications identified among 317 prioritized applications
Late-case capture: 35.0%; conditional 95% interval 31.1%–38.8%. This measures identification, not improvement caused by review.
| Model | Late identified | Capture | 95% interval |
|---|---|---|---|
| Pooled survival baseline | 116 / 623 | 18.6% | 15.9%–21.6% |
| Current stage and age baseline | 215 / 623 | 34.5% | 31.2%–38.1% |
| Workflow hazard boosting | 218 / 623 | 35.0% | 31.1%–38.8% |
| Age and current stage only | 208 / 623 | 33.4% | 29.9%–37.0% |
ASSUMED CAPACITY / SCENARIO ONLY
Separate a staffing assumption from measured evidence.
Assume some share of the selected cohort’s forecast duration varies inversely with capacity. The dataset does not identify that share or the effect of staffing.
13.50 dayshypothetical duration versus 13.50 forecast days
Formula: forecast × [(1 − share) + share ÷ multiplier]. No causal confidence interval is available. Values above 30 days extrapolate beyond the fitted horizon. These assumptions do not alter observed error, measured coverage or the review comparison.
Probability calibration and diagnostic limits
| Probability bin | Prefixes | Predicted | Observed |
|---|---|---|---|
| 0–10% | 221 | 8.0% | 5.9% |
| 10–20% | 388 | 14.4% | 23.7% |
| 20–30% | 82 | 23.1% | 42.7% |
| 30–40% | 207 | 34.1% | 41.1% |
| 40–50% | 168 | 46.3% | 30.3% |
| 50–60% | 1,230 | 57.3% | 52.0% |
| 60–70% | 3,271 | 65.7% | 64.5% |
| 70–80% | 680 | 72.2% | 70.1% |
Bins use predictions, without final outcomes. Counts are prefixes, not independent applications. Stage and prefix diagnostics retain the application weights and are withheld below 30 prefixes. Full model comparisons are available in the aggregate download.
500 paired application-cluster bootstrap draws retain all evaluated prefixes of each application. Conditional on fitted models and one institutional/time cohort; no retraining uncertainty. Capacity intervals hold the selected ranking fixed. These are not simultaneous guarantees across every subgroup or slider setting.
04 / DATA
A real workflow with a carefully defined endpoint.
The original BPI Challenge 2017 file contains 31,509 applications and 1,202,267 events. Applications opened in 2016; the last recorded timestamp is February 1, 2017. Offers remain attached to their application, and start, suspend, resume and complete lifecycle events remain distinct.
The target is the first recorded A_Pending, A_Denied or A_Cancelled status. It is not loan disbursement or the last event in the trace. At each of the first 5, 10 or 20 events, features use only information already observed. Resource identities, applicant identifiers and financial attributes do not enter the model or the public tables.
Development uses January–August arrivals; September supplies validation. Final training uses January–September arrivals with eligible prefixes before November and follow-up through November 30. The untouched final cohort consists of December arrivals with prefixes before January 1. That cutoff provides at least 30 days of final follow-up.
Of the final prefixes, 183 lack a recorded endpoint by the source cutoff. Their unrestricted durations are censored, but their 30-day restricted outcome is fully observed. The data audit checks trace order, duplicated event IDs and cross-application offer references; none of those conflicts were found.
van Dongen, B. (2017): BPI Challenge 2017. Eindhoven University of Technology / 4TU.ResearchData. https://doi.org/10.4121/uuid:5f3067df-f10b-45da-b98b-86ae4c7a310b Publisher record ↗
Aggregate research retains attribution and the dataset’s 4TU General Terms of Use (2016), including noncommercial use. No raw records are redistributed. The publisher does not endorse the analysis.
05 / METHOD & LIMITATIONS
Model remaining survival without borrowing future events.
The baselines estimate survival from all training cases or cases with the same stage and age band. A discrete-time boosted hazard model adds event history and elapsed gaps. It learns the probability of reaching the target status in each of the next 30 days.
Validation selects 15 leaves from a fixed 7-versus-15 comparison. The selected model and a reduced stage/age-only model are then fitted on final training. Category mappings are learned within training, and censored partial days contribute only completed risk intervals. Remaining-time means sum the saved survival probabilities; they do not assume an unobserved tail.
- One institution and one final arrival month; temporal generalization is not established.
- Early prefixes may omit later loops; unavailable prefixes and already-dispositioned cases are excluded with counts.
- All predictions and bands refer to a30-day restricted endpoint, not unrestricted remaining duration.
- Repeated prefixes are dependent and treated as application clusters.
- No resource-efficiency, causal automation, financial-saving or lending-decision claim is made.
- Independent technical review is pending.
The public interaction uses saved predictions and explicitly assumed proportional capacity arithmetic. It is not a queueing model or evidence that automation improves throughput.
Frozen endpoint, split and evaluation protocol ↗06 / CODE
Trace every finding to a saved run.
Run S58-9dd9fde5-183c5e51
Analysis commit 9dd9fde5837bd032b6f3a13c5ae1e2ba7d768a05
git clone https://github.com/mpgibb/michael-gibb-research.git
cd michael-gibb-research
uv sync --frozen
uv run python -W error studies/S58/study.py
uv run python scripts/report_s58.py
uv run python -W error -m unittest discover -s tests -vFixed seeds · locked dependencies · complete source checksum. Checks cover future-event exclusion, censored risk intervals, probability identities, cohort accounting and independent reconstruction of final metrics. Independent technical review is pending.