ICU Mortality Prediction: Why Feature Selection Matters More Than AUC in Singapore Hospitals

ICU Mortality Prediction: Why Feature Selection Matters More Than AUC in Singapore Hospitals

When Singapore hospital teams evaluate ICU mortality prediction models, vendor presentations focus on AUC scores and sensitivity metrics. But the real deployment barrier isn't accuracy—it's interpretability. A model that achieves 0.92 AUC using 847 diagnosis codes is clinically useless if intensivists can't understand why it flagged a patient, and governance teams can't audit its logic when outcomes diverge from predictions.

This post is for clinical informatics teams, ICU directors, and AI deployment leads in Singapore hospitals who need to move beyond benchmark performance and build mortality prediction systems that clinicians will actually trust and use.

Key takeaways

• Feature selection is a governance decision, not just a technical optimization: High-dimensional EHR diagnosis codes create sparse, noisy input spaces that make models fragile and uninterpretable [1]
• Recent research shows feature selection methods vary dramatically in clinical settings: A 2026 comparative study on opioid use disorder prediction found that different feature selection approaches produce models with similar accuracy but vastly different clinical interpretability [1]
• Singapore hospitals need validation protocols that test feature stability, not just predictive performance: Models trained on complete EHR data fail when deployed against incomplete, inconsistent ward documentation [2]
• Trust in predictive AI depends on whole-system factors: The National Academy of Medicine's 2026 framework emphasizes coordinated action across communication, community engagement, and information access—not just model performance [3]

Why high-dimensional diagnosis codes break ICU mortality models

Electronic health records in Singapore hospitals contain thousands of potential diagnosis codes. A typical ICU patient might have 15-40 ICD-10 codes documented across their admission. When you train a mortality prediction model on historical data, you're working with a feature space where:

• Most codes appear rarely: The long tail of diagnosis codes means sparse data—many codes appear in fewer than 1% of cases
• Codes are redundant and hierarchical: ICD-10's structure means "septic shock" and "sepsis" are both present, creating collinearity
• Documentation is inconsistent: Coding practices vary by clinician, shift, and documentation burden

A recent preprint examining feature selection for EHR-based prediction highlights these challenges: "Large feature sets not only increase computational burden and overfitting risk, but also make model interpretation difficult, leading to limited usefulness in clinical settings" [1]. The study compared multiple feature selection methods and found that while predictive performance remained similar, the features selected varied significantly—meaning different methods surface different clinical narratives.

For Singapore hospital teams, this matters because your model's feature set becomes your audit trail. When a mortality prediction triggers an escalation protocol, the clinical team needs to understand which patient characteristics drove the alert. If your model relies on 200+ diagnosis codes, that explanation becomes meaningless.

What feature selection methods actually do in practice

The 2026 comparative study [1] examined multiple approaches:

• Filter methods (chi-squared, mutual information): Fast, model-agnostic, but ignore feature interactions
• Wrapper methods (recursive feature elimination): Account for model behavior but computationally expensive and prone to overfitting
• Embedded methods (LASSO, tree-based importance): Balance interpretability and performance but depend on model architecture

In our deployment work with Singapore health systems, we've seen teams default to LASSO regularization because it's familiar and produces sparse models. But LASSO's feature selection is unstable—small changes in training data can flip which features are retained. For ICU mortality prediction, this means your model's clinical logic changes subtly with each retrain, making longitudinal governance difficult.

The practical implication: feature selection method is a design decision that affects clinical trust, not just a hyperparameter to tune for AUC.

Why incomplete data makes feature selection critical

A concurrent 2026 preprint on emergency triage prediction addresses a problem every Singapore ICU team knows: "available electronic health record (EHR) data, including structured data and clinical text, are often incomplete, unreliable, and inconsistent" [2]. The CRS-Triage framework they propose uses confidence- and reliability-aware selective prediction—essentially, the model abstains when input data quality is poor.

This connects directly to feature selection. If your ICU mortality model relies on 150 diagnosis codes, and 40% of those codes are missing or inconsistently documented in real-time ward data, your model's predictions become unreliable. Feature selection that prioritizes robustness to missing data over raw predictive power produces models that degrade gracefully in production.

We recommend Singapore hospital teams test feature selection approaches against:

1. Temporal stability: Do selected features remain consistent across quarterly retrains?
2. Missingness robustness: How does model performance degrade when 20%, 40%, 60% of features are missing?
3. Clinical coherence: Can an intensivist explain why these features matter for mortality risk?

This aligns with our earlier work on early warning score ML calibration drift, where we emphasized that temporal validation protocols must test feature stability, not just aggregate metrics.

How to build interpretable patient similarity explanations

Another 2026 preprint introduces a "Patients-like-me" framework that combines language models with graph neural networks to provide reference-patient attribution [4]. The core insight: instead of explaining predictions through feature importance scores, show clinicians which historical patients the model considers similar to the current case.

For ICU mortality prediction in Singapore hospitals, this approach has practical appeal:

• Clinicians already think in case-based reasoning: "This patient reminds me of a case last month" is natural clinical logic
• Reference patients provide audit trails: You can review the historical cases the model referenced and verify their outcomes
• Similarity explanations surface feature interactions: Instead of isolated feature importance, you see how combinations of characteristics define risk

The challenge is computational cost and data governance. Graph-based patient similarity requires linking across historical admissions, which raises PDPA considerations for Singapore hospitals. You need clear data governance protocols that define:

• How long historical patient data remains in the similarity graph
• Whether clinicians see anonymized or pseudonymized reference cases
• How to handle edge cases where the "similar" patient is identifiable

This intersects with our LLM-driven health data interoperability work, where we've emphasized that data linkage for AI must follow the same governance standards as clinical data sharing.

Why trust in predictive AI requires whole-system thinking

A 2026 JAMA Perspective from the National Academy of Medicine argues for a "whole-system model of trust in science, medicine, and public health" [3]. The framework emphasizes that trust isn't built through individual interventions—it requires coordinated action across communication, community engagement, and information access.

For ICU mortality prediction in Singapore hospitals, this means:

• Communication: How do you explain model predictions to families during goals-of-care discussions?
• Community engagement: Have you involved ICU nurses and junior doctors in model design, or only senior intensivists?
• Information access: Can clinical teams audit historical predictions and outcomes, or is the model a black box?

We've seen Singapore hospital teams focus exclusively on model performance metrics during procurement, then struggle with adoption because they didn't build trust infrastructure. Feature selection is part of this trust infrastructure—it determines whether your model's logic is auditable and clinically coherent.

This connects to our commercial sustainability vs. ethical governance analysis, where we argued that governance isn't a constraint on deployment—it's a prerequisite for sustainable adoption.

Why this matters in Singapore

Singapore's hospital clusters are deploying ICU mortality prediction models as part of broader clinical deterioration pathways. The HSA's 2026 AI-SaMD sandbox exemption pathway (covered in our earlier analysis) creates regulatory space for experimentation, but also raises expectations for post-market surveillance.

Feature selection determines whether your mortality prediction model can meet those surveillance requirements:

• Temporal validation: Can you demonstrate that selected features remain stable across retrains?
• Subgroup performance: Do selected features generalize across ethnic groups, age ranges, and comorbidity profiles in Singapore's diverse population?
• Clinical coherence: Can you document why selected features align with clinical understanding of mortality risk?

Singapore hospital teams that treat feature selection as a governance decision—not just a technical optimization—will build models that survive regulatory scrutiny and clinical adoption barriers.

What to do next

If you're deploying ICU mortality prediction models in Singapore hospitals:

1. Audit your feature selection method for temporal stability: Retrain your model on rolling 6-month windows and track which features are selected. If feature sets change dramatically, your model's clinical logic is unstable.
2. Test robustness to missing data: Simulate real-world documentation incompleteness by randomly masking 20-40% of features and measuring performance degradation. Prioritize feature selection methods that maintain performance under missingness.
3. Build clinical review protocols for selected features: Before deployment, have intensivists review the final feature set and document clinical rationale. This creates an audit trail for governance and builds trust.
4. Consider patient similarity explanations as an alternative to feature importance: Explore frameworks like Patients-like-me [4] that provide reference-patient attribution instead of isolated feature scores.
5. Integrate feature selection into your MLOps pipeline: Track selected features as metadata in your model registry, and alert when retrains produce significantly different feature sets.

For teams building clinical AI deployment infrastructure, our clinical AI services include feature selection audits, temporal validation protocols, and governance documentation templates. Start a project if you need support moving from vendor benchmarks to production-ready mortality prediction systems.

FAQ

How many features should an ICU mortality prediction model use?

There's no universal answer, but clinical interpretability typically breaks down above 20-30 features. We recommend starting with domain-expert feature selection (intensivists choose clinically relevant codes), then using data-driven methods to refine within that constrained space. A model using 15 well-justified features will gain more clinical trust than one using 150 algorithmically selected codes.

Should we use the same feature selection method for all predictive models?

No. Feature selection method should match your deployment context. For high-stakes ICU mortality prediction where interpretability is critical, prefer methods that produce sparse, stable feature sets (e.g., LASSO with stability selection). For operational forecasting where accuracy matters more than explanation, ensemble methods with larger feature sets may be appropriate. Document your rationale as part of model governance.

How do we validate that selected features generalize across Singapore's ethnic diversity?

Perform subgroup analysis during validation: stratify by ethnicity, age, and comorbidity burden, then measure whether selected features maintain predictive power across strata. If certain features only predict mortality in specific subgroups, document this as a model limitation and consider separate feature sets or interaction terms. This aligns with HSA's expectations for AI-SaMD post-market surveillance.

What's the relationship between feature selection and model calibration drift?

Feature selection affects calibration stability. Models with many features are more sensitive to distribution shift—if documentation practices change or patient mix evolves, high-dimensional models recalibrate poorly. Sparse feature sets produced by careful feature selection tend to maintain calibration longer, reducing the frequency of retrains. See our calibration drift analysis for temporal validation protocols.

Sources

[1] A Comparative Study of Feature Selection Methods for EHR Diagnosis Codes in Opioid Use Disorder Prediction. arXiv preprint, August 2026. https://arxiv.org/abs/2608.04180v1

[2] CRS-Triage: Confidence- and Reliability-Aware Selective Triage under Incomplete Clinical Evidence. arXiv preprint, August 2026. https://arxiv.org/abs/2608.03862v1

[3] Whole-System Trust in Science, Medicine, and Public Health. JAMA Network, August 2026. https://jamanetwork.com/journals/jama/fullarticle/2850957

[4] Patients-like-me: A Variational LM–GNN Framework for Explainable Clinical Prediction. arXiv preprint, August 2026. https://arxiv.org/abs/2608.04193v1