Integrating Health Behaviour and AI/ML Theories: A Case for Pre-Screening Prediction in Industry-Sponsored Clinical Trials

2.1 Clinical trial theories

RCTs are widely regarded as the gold standard in clinical research methodology, offering the most rigorous framework for evaluating the safety and efficacy of medical interventions. According to seminal works by Friedman et al. [3], RCTs are not only essential for medical advancement but also represent “the most definitive tool for evaluation of the applicability of clinical research.” In addition to randomisation, these trials are designed with strict methodological safeguards, including blinding and intention-to-treat analysis, to protect against bias and ensure internal validity. This approach also demonstrates sufficient statistical power to enhance external validity.

Success in clinical trials relies significantly on effective recruitment strategies, as engaging participants is a vital component of the process. A notable increase in the number of preclinical and early phase (Phase I and II) drugs suggests a strong innovation pipeline. Still, it also draws attention to stagnating Phase III numbers [1]. This raises concerns about the progression of drug development, particularly around recruitment, retention, and the overall trial feasibility.

Ethical trial design theory emphasises the importance of fairness, representativeness and transparency in participant selection [2]. Fundamental principles such as equipoise underscore the ethical imperative of genuine uncertainty in treatment allocation [3]. Methodological frameworks, such as the CONSORT guidelines [4], ensure transparency in reporting, while adaptive trial designs provide flexibility in addressing emerging evidence as the trial progresses [5]. Therefore, recruitment failures that result in demographic imbalances have implications for the generalisability of trials and for healthcare equity. The call for methodologically rigorous and ethically sound recruitment models is not new; rather, it is foundational to the discipline [6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]. Despite decades of advancement, many trials still suffer from poor external validity due to homogenous recruitment pools, suggesting that methodological rigour alone does not guarantee representativeness or public health impact. [17, 18, 19, 20, 21, 37].

The tangible consequences of weak recruitment infrastructures necessitate a shift toward scalable, adaptive recruitment models, particularly in industry-sponsored trials, where operational efficiency and participant diversity often conflict. This emphasises the urgent need for innovative, data-driven recruitment methodologies that align with established theoretical frameworks.

Resources such as ClinicalTrials.gov [38] provide researchers and sponsors with real-world data for benchmarking and planning [34, 39, 40]. In unison, initiatives like the Clinical Trials Transformation Initiative (CTTI) emphasise the importance of patient engagement in ethical trial design. Together, they play a pivotal role in shaping the field of patient engagement by addressing barriers and developing solutions to improve clinical trials, aiming for greater collaboration between patient groups and trial sponsors [41].

However, structural assumptions embedded within clinical protocols frequently overlook psychosocial variables, for example, perceived burden, trust, and motivational alignment that influence enrolment [35, 36, 42, 43, 44, 45, 46, 47]. Simultaneously, AI/ML-driven modelling approaches are being proposed to predict trial success and improve recruitment outcomes [27, 30, 48, 49, 50]. Thus, the integration of behavioural theory and AI/ML techniques guided by established trial design principles and governance is a logical evolution in patient recruitment methodology.

This research focuses on clinical trial theories and challenges in patient recruitment, emphasising the need for an interdisciplinary strategy that merges behavioural insights with predictive modelling. This concept will be further explored in the next section.

2.2 Health behaviour theories

Effective patient engagement hinges on understanding how individuals perceive the risks, benefits, and burdens associated with participating in clinical trials. Studies have identified multiple barriers, including social behaviours, cultural norms, and logistical constraints [9, 10, 12, 28, 51]. Despite this, the theoretical frameworks that underpin patient decision-making remain underutilised in patient recruitment models. Signorell et al. [22] emphasise the importance of consistent and detailed methodologies for gathering participant perspectives, highlighting that neglecting participant insights compromises the internal validity and reliability of clinical trials.

To address these gaps, seminal health behaviour theories provide foundational insights into individual decision-making regarding clinical trial participation. The Health Belief Model (HBM) elucidates individual health behaviours through subjective perceptions or concepts of risks, benefits, and barriers to action. The HBM posits that health behaviours are shaped by personal beliefs (perceptions) regarding health threats, benefits and/or barriers to action [23]. As such, individuals are more likely to engage in health-promoting behaviours when they perceive a significant health threat, recognise the substantial benefits of taking action, and encounter fewer barriers [47].

The HBM also offers a structured approach to identifying individual-level barriers and motivators, aligning closely with motivational factors critical when designing trial pre-screening questionnaires and during early patient recruitment phases [24, 36, 43]. The HBM therefore provides valuable guidance for crafting recruitment materials that emphasise perceived benefits, such as access to novel treatments, and reduce perceived barriers, including safety concerns and logistical inconvenience [25]. However, the HBM also has notable limitations, particularly its abstract nature, limited predictive power, and the omission of constructs related to social influence. Critics argue that the model overemphasises rational decision-making, neglecting the often irrational and emotional factors that influence health decisions [24, 26, 36, 52, 53].

In response, alternative models such as the Theory of Planned Behaviour (TPB), the Theory of Reasoned Action (TRA), the Integrated Behavioural Model (IBM), and the Transtheoretical Model (TTM) explicitly address social norms, behavioural intentions, and perceived behavioural control [29, 36]. The TPB and TRA both emphasise intention as a robust predictor of behaviour, significantly influenced by attitudes and subjective norms, providing comprehensive frameworks for addressing social contexts and individual capabilities. The TTM offers additional utility by conceptualising behaviour change as a progression through distinct stages, enabling interventions that can be tailored to participant readiness [31]. The assumption that patient referrals occur in a strict sequence may not accurately reflect real-world decision-making, which could result in inefficiencies when implementing interventions. While subjective norms may be less important during the initial stages of patient recruitment, they become vital in later phases, particularly when implementing targeted interventions and securing informed consent.

Given this limitation, the HBM proves particularly effective during the initial recruitment and pre-screening stages, as it emphasises individuals’ perceptions of risk and benefit. Recognising these advantages, the HBM was adopted as the primary behavioural framework for guiding feature engineering and model development. The HBM constructs presented in Table 1 can be successfully translated into structured, patient-reported features through Named Entity Recognition (NER) tasks. This approach enables the development of scalable and dynamic AI/ML applications. The structured domains of the HBM enable a direct mapping of patient-generated language into features, allowing ML techniques, such as XGBoost, to leverage nuanced behavioural indicators in predicting referral quality.

The following section examines how AI/ML principles facilitate the translation of behavioural insights into predictive models that enhance patient recruitment, responsiveness, and scalability.

2.3 Artificial intelligence and machine learning theories

Despite rapid advancements, AI/ML have yet to achieve widespread clinical impact, particularly in patient recruitment for clinical trials [32, 48, 50]. As noted by Balagopalan et al. [54], this limited success is primarily attributed to a persistent focus on technical performance rather than on outcomes that tangibly improve patient engagement or trial efficiency. The authors argue for a realignment of ML efforts toward patient-centred objectives, calling for collaborative approaches that integrate the expertise of AI/ML researchers, clinicians and regulatory stakeholders to advance both health equity and operational relevance.

Within the clinical trial context, AI/ML offers significant potential to address recruitment inefficiencies. Predictive models, particularly ensemble methods such as Bootstrap Aggregation, Random Forest, and XGBoost, have consistently demonstrated robust performance across various domains [30, 33, 55]. However, these methods, despite their efficiency, must be carefully managed to avoid perpetuating existing biases as highlighted by Obermeyer et al., [56, 57]. Mitigation strategies, such as the Synthetic Minority Over-sampling Technique (SMOTE) and Adaptive Synthetic Sampling (ADASYN), are commonly used to address class imbalances, though they have the potential to reinforce existing structural inequalities in healthcare data and introduce artefacts that lead to overfitting and distorting representativeness. [58].

Ensuring the ethical deployment of AI/ML systems requires more than algorithmic accuracy; it also necessitates models that are grounded in patient diversity and regulatory compliance. Integrating AI/ML into patient recruitment must be accompanied by deliberate efforts to train models on diverse and representative datasets, supported by rigorous bias detection and correction strategies [54]. Moreover, the ethical deployment of such systems requires strict ethical compliance with regulatory frameworks, for example, the Helsinki Declaration [2] and the General Data Protection Regulation (GDPR) [59], which ensures the protection of patient autonomy and public trust.

A key challenge in operationalising these models lies in the extraction and interpretation of structured and unstructured clinical text, specifically trial pre-screening questions and responses. Recent advancements in Natural Language Processing (NLP), particularly Medical Named Entity Recognition (NER) within Large Language Models (LLMs), have enhanced the classification of clinical texts. LLMs such as BERT and BioBERT utilise transformer architectures to extract medical entities with contextual sensitivity [60, 61, 62, 63]. While these models contribute to accurately extracting relevant information, several limitations persist, such as the variability in clinical language, including colloquialisms and non-standard terminology, posing a challenge to model accuracy. Although models trained on domain-specific corpora address some of these issues, as noted by Alsentzer et al. [60] and Peng et al. [62], performance disparities across demographic and linguistic subgroups remain a concern [64].

Thus, AI/ML can operationalise these constructs by detecting behavioural patterns across large datasets. For example, AI/ML models can predict which individuals view themselves as being at higher risk and therefore more inclined to participate in trial initiatives. By utilising cues to action, such as reminders or customised messaging, AI/ML algorithms can refine outreach strategies and improve referral quality. Leveraging these insights, clinical recruiters can tailor their communication, address concerns, and enhance the likelihood of enrolment.

However, mapping extracted NLP entities onto abstract HBM constructs introduces challenges, necessitating consideration of various theoretical and practical dimensions, as well as clearly defined operational rules [23, 53]. Additionally, the interpretability of these models is crucial, as stakeholders need to understand how predictions are made and their relation to patient behaviour and decision-making. Interpretability is crucial; stakeholders must understand and trust the model’s outputs to make informed patient referrals. Therefore, techniques such as SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME) are essential for elucidating predictions [65].

In addition, time-to-event (TTE) or survival analysis provides insights into patient recruitment dynamics that complement behavioural and ensemble models by quantifying the temporal progression from initial contact to consent or randomisation.

The Kaplan-Meier (KM) estimator is a non-parametric method for estimating survival probabilities in clinical settings, particularly where censoring is present [66, 67, 68]. Its transparency and robustness render it highly suitable for assessing the effectiveness of patient referrals to clinical sites. However, KM estimates rely on assumptions such as independent censoring and population homogeneity, which may not hold in real-world patient recruitment scenarios. For instance, KM estimates can become biased if dropouts or delayed responses are related to specific patient characteristics. Additionally, the step nature of KM curves may not support the smooth, dynamic predictions needed for real-time decision-making.

Alternatives such as Cox proportional hazards models or survival forests offer more nuanced predictions by incorporating covariates and handling heterogeneity [69]. As such, while KM estimators provide a valuable foundation, they should be paired with advanced methodologies, such as accelerated failure time models or ML-based survival models, to accommodate complex relationships and improve prediction accuracy.

Therefore, integrating AI/ML with behavioural theory and temporal modelling offers a robust, interdisciplinary framework for optimising patient recruitment. When aligned with established behavioural frameworks and ethical safeguards, AI/ML can meaningfully enhance both the quality and equity of clinical trial recruitment strategies. Success, however, hinges on transparent and interpretable models, equitable data representation, and rigorous methodological validation, as below.

3.1 Design

This study adopts a pragmatic research paradigm to address the complex operational and behavioural challenges associated with patient recruitment in industry-sponsored clinical trials. Grounded in a deductive, quantitative framework, the research employs a sequential case study design that enables granular, context-specific insight while preserving methodological robustness.

Although the study is fundamentally quantitative, its phased structure supports a stepwise examination of associations between behavioural constructs and recruitment outcomes, as well as the development and validation of predictive models. This alignment across data integration, sampling, and modelling enhances both coherence and interpretability.

3.2 Techniques and procedures

3.3 Ethical considerations

Ethical integrity in this research is governed by established frameworks, including Good Clinical Practice (GCP) [22], the Health Insurance Portability and Accountability Act (HIPAA), and the General Data Protection Regulation (GDPR) [59, 74, 75, 76, 77]. All data used in the study were fully anonymised prior to analysis. Common identifiers—such as names, residential addresses, and dates of birth—were excluded to ensure that no individual patient could be re-identified, even when linked with trial-specific data.

Data handling protocols adhered to data minimisation and role-based access principles, ensuring that only the necessary data was used for analysis and that access was restricted to authorised personnel. Moreover, ethical considerations were central to the integration of AI/ML methods, particularly regarding model fairness, interpretability, and the responsible handling of features related to race, ethnicity, and socioeconomic vulnerability.

3.4 Analytical workflow overview

The primary goal was to create a “Patient Referral Scoring” (PRS) model to predict the likelihood that a patient will complete the site phone screening, as illustrated in Figure 2.

The analysis involved key stages: descriptive analysis for feature engineering using large language models and health behaviour theory; univariate analysis for missing values and outliers; bivariate analyses to explore relationships between the unit of analysis and categorical variables; multivariate logistic regression to understand interactions among factors; feature engineering specific to the HBM; predictive analysis using a Gradient-Boosted Ensemble (XGBoost) Model; and dynamic risk adjustment via Kaplan-Meier survival analysis. Insights from each stage informed previous phases, enabling continuous refinement of features and model structure.

3.4.1 Feature engineering using large language models and health behaviour theory

With contextual sensitivity, medically relevant terms, symptoms, diagnoses, treatment, and behavioural concerns were extracted by operationalising the HBM through Natural Language Processing (NLP) by leveraging Named Entity Recognition (NER) within large language models (LLMs) [60, 62]. Terms, derived from the pre-screening criteria, were categorised by applying the HBM constructs via a multi-class classification label (ensemble label), accommodating diagnoses, symptoms (perceived susceptibility and severity), and treatment (perceived benefits) as illustrated in Figure 3.

Diversity and equity are critical considerations in trial participation [56]. The HBM does not reliably address these without introducing bias. This concern is addressed by the AI/ML modelling layer, which applies the Social Vulnerability Index (SVI) and minority group indicators, ensuring they are accounted for without compromising theoretical integrity.

4.1 Data description

The final dataset comprises 5,436,136 observations and 69 variables, spanning demographic, clinical, socioeconomic, and site-level attributes across 543 distinct counties in the United States of America (US). Each patient referral record was categorised by trial identifier, disease classification, self-reported ethnicity, and structured responses to trial-specific pre-screening criteria, representing 398,879 unique patient trial referral IDs. This indicates that some patients were eligible for, and referred to, multiple studies over time.

Of the referrals, 23.3% successfully passed the site phone screening. The remaining 76.7% comprised unprocessed referrals, identified by missing finalisation dates; unsuccessful contact attempts, identified by missing contact dates but populated finalisation dates, and referrals that failed the phone screening. To support subsequent analyses of referral screening progression and attrition, these distinctions were preserved in the dataset.

4.2 Descriptive analysis and discussion

4.3 Exploratory analysis and discussion

4.4 Predictive analysis and discussion

Following feature engineering, a gradient-boosted ensemble (XGBoost) model with cross-validation is applied to estimate the probability that a patient will successfully pass the phone screening phase. This XGBoost model capitalises on the nuanced feature space created by the LLM to provide a robust, non-linear prediction of screening success [92]. Model performance was evaluated using standard classification metrics, including AUC and a confusion matrix. Feature importance was assessed using SHAP values [93] to enhance model interpretability and support stakeholder confidence in model decisions.

4.4.2 Model evaluation and interpretability

Evaluation of the test set (n = 96,281) confirmed the model’s moderate discriminative ability, achieving an AUC of 0.7398 and an overall accuracy of 67.04% (95% CI: 66.74–67.33%). The model significantly outperformed the no information rate (51.16%) with a p-value <2e–16. Sensitivity (66.19%) and specificity (67.84%) were well-balanced, indicating consistent classification across positive and negative outcomes. The balanced accuracy of 67.02% accounts for class distribution imbalances, while the Kappa statistic of 0.3403 reflects fair agreement beyond chance. McNemar’s test yielded a p-value of 0.7448, suggesting no significant prediction bias across the two outcome classes.

The model’s performance, disaggregated by disease classification, reveals substantial heterogeneity in predictive performance across disease areas, underscoring the interaction between behavioural constructs and disease typology as observed in Figure 4.

An evaluation of the confusion matrix by disease classification (Appendix G) reveals an overall accuracy ranging from as low as 58% for musculoskeletal diseases to as high as 90% for chemically induced disorders, with corresponding variability in sensitivity (1–100%) and specificity (2–100%).

Notably, high sensitivity was observed in domains such as behaviour and behaviour mechanisms (100%), diagnosis (99%), and congenital, hereditary, and neonatal disorders (95%).

In contrast, stomatognathic diseases and virus diseases yielded critically low sensitivity (≤3%) and exhibited near-perfect specificity. Domains such as digestive system diseases and skin and connective tissue diseases exhibited strong specificity (≥92%) but poor sensitivity (≤20%), reflecting a conservative model bias that favours minimising false positives at the expense of false negatives.

4.4.4 SHAP-based feature evaluation and clustering

SHAP values were computed for all features using the final XGBoost model to quantify the contribution of each variable to the predicted probability of passing the phone screening. For each patient referral, local SHAP values were calculated and summarised across features using mean absolute values to indicate global importance. To interpret the contribution of individual predictors to the model’s classification output, SHAP values were computed and summarised in a global SHAP beeswarm plot (Figure 5). Feature importance was ranked by the mean absolute SHAP value across all instances, and the plot displays both the magnitude and direction of features’ effects on the predicted probability that a patient will pass the phone screening stage.

Figure 5 shows that perceived barriers outperform all other features with an average SHAP contribution of 0.398. Disease classification and overall social vulnerability were followed by mean SHAP values of 0.215 and 0.155, respectively. All remaining HBM construct variables, e.g., perceived susceptibility, severity, and/or perceived benefits, contributed to lower average impact values, ranging from 0.022 to 0.046. Notably, the minority indicator had minimal standalone predictive contribution, confirming its limited direct influence on the trained model.

In addition, the distribution of SHAP values across individual patients also highlighted heterogeneity in feature effect: perceived barriers and overall social vulnerability exhibited wide SHAP ranges (up to ±2.5). In contrast, features such as perceived severity and benefits showed tightly bound distributions near zero.

To investigate the interaction between disease classification, overall social vulnerability and HBM constructs, a two-way matrix of mean absolute SHAP values was constructed. The resulting matrix was scaled column-wise and visualised using a dendrogram heatmap (Figure 6), with hierarchical clustering applied to both rows (disease classifications) and columns (features). This clustering grouped disease classifications with similar referral behaviour patterns, and the columns reflected shared SHAP value profiles among features. Clustering was based on the Euclidean distance and the complete linkage method.

Figure 6 also shows that disease classifications, such as musculoskeletal and neural physiological phenomena, psychological phenomena, and bacterial infections, cluster together. Similarly, chemically induced disorders, hemic and lymphatic, as well as eye diseases, clustered together. Some leaf nodes are distantly clustered, as is the case with female urogenital diseases and congenital disorders, suggesting unique SHAP patterns.

5.1 Construct validity: Generalisability across conditions

Construct validity was assessed by testing whether HBM construct distributions differed meaningfully across disease classifications (known-groups differentiation). The nonparametric Kruskal-Wallis rank-sum test [96] was used to evaluate differences in HBM construct distributions across 27 disease classifications. The test yielded highly significant results across all constructs (p < 2.2e–16), demonstrating that perceptions of barriers, susceptibility, severity, and benefits varied meaningfully by condition. These findings (Table 4) support the construct differentiation and contextual sensitivity of the underlying behavioural features, bolstering both the ecological and theoretical validity of the model.

5.2 Reliability: Cross-validation (stability and reproducibility)

Reliability was assessed through five-fold cross-validation, yielding a mean training AUC of 0.7403 (SD = 0.0008) and a mean test AUC of 0.7348 (SD = 0.0022). The narrow standard deviations indicate high performance consistency and negligible variance across folds, suggesting robust reproducibility. The optimal number of boosting rounds was identified as 500, reflecting the model’s stable learning trajectory and convergence.

5.3 Structural validity: Robustness across data representations

To assess model robustness across feature architectures, predictions were compared between wide- and long-format representations of HBM domains. While overall accuracy remained comparable, key differences in sensitivity and specificity were observed, reflecting trade-offs between parsimony and granularity. The wide-format favoured specificity, while the long-format improved sensitivity, particularly in low-prevalence conditions. These results underscore the structural validity of the behavioural feature set and support context-sensitive deployment. Full benchmarking results are provided in Appendix I.

5.4 Bias and fairness: Class imbalance and interpretability checks

To ensure equitable performance across outcome classes, McNemar’s test was applied to assess symmetry in misclassifications. The resulting p-value (0.7448) indicated no statistically significant prediction bias, affirming parity in error rates between positive and negative cases. Further fairness validation was conducted via SHAP value decomposition, which revealed alignment between high-impact features and theoretically grounded HBM domains, rather than spurious or proxy variables. The model was also trained using a stratified sampling strategy based on disease classification and minority group status to preserve equity in representational learning.

Overall, the model demonstrates strong internal validity, domain-general external validity, construct sensitivity, and high reliability, underpinned by consistent statistical performance and interpretable feature contributions. Importantly, these findings reflect the conceptual integration of behavioural theory into an ML pipeline capable of supporting operational improvements in clinical recruitment. However, to ensure clinical utility, the model must undergo continuous validation against site-level outcomes, periodic retraining with real-world data and sensitivity analyses to balance referral volume with predictive precision [97].

5.5 Time-to-event validity: Survival model discrimination

To evaluate the discriminative performance of the time-to-event component, the Concordance Index (C-index) was used. The C-index assesses the model’s ability to correctly order pairs of patients by risk, while accounting for censored observations [98]. Theoretically, this measures the likelihood that, for any randomly selected pair of people, the individual with the higher predicted risk will experience the event (such as loss of contact eligibility) before the other patient. A C-index of 0.5 indicates random chance, whereas a value of 1.0 denotes perfect discrimination [99].

The KM model achieved a C-index of 0.987, indicating high concordance between predicted and observed referral outcomes over time. This result supports the model’s validity in capturing time-sensitive degradation in referral quality.

6.1 The primary research question

This research aimed to explore the primary research question: How can integrating HBM constructs into AI/ML models enhance the prediction of successful clinical trial site screening?

The results show that integrating HBM constructs with AI/ML techniques meaningfully improved both the quality and equity of patients recruited for clinical trials. From a quality perspective, the inclusion of behavioural features enabled nuanced segmentation of patients based on motivational, perceptual, and structural influences, moving beyond static demographic or clinical eligibility filters. SHAP value analysis revealed that contextual variables, particularly perceived barriers, play a pivotal role in shaping referral predictions, providing actionable insights for refining recruitment strategies.

From an equity perspective, the model demonstrated consistent performance across underserved subgroups, thereby reducing the risk of algorithmic exclusion. This suggests that embedding behavioural theory into ML pipelines can counterbalance the systemic biases often inherent in historical clinical trial datasets.

Finally, the differential performance across disease categories underscores the need for condition-specific behavioural profiling. This opens new avenues for ethical, adaptive recruitment practices that are both data-driven and person-centred, supporting broader clinical trial participation and better alignment with public health objectives.

6.2 The predictive value of HBM pre-screening features

Which behavioural and contextual factors most strongly predict successful progression to the site screening phase within a clinical trial? Hierarchical clustering of SHAP values reveals two dominant behavioural feature groupings, directly addressing the questions as to which factors, derived from patient responses, best predict successful progression through the recruitment process (Research question RQ1.1).

First, perceived barriers, overall social vulnerability, and perceived severity cluster tightly, suggesting a shared influence across clinical domains and are likely tied to protocol burden or screening logistics. In contrast, perceived susceptibility and benefits cluster separately, indicating that they operate through distinct perceptual pathways and may capture more subjective, or belief-oriented variance.

The interpretability of the model’s behaviour was explored using both global SHAP summary plots and a condition-stratified heatmap of mean SHAP values by disease classification. Together, these maps offer insights into both the overall structure of feature influence and how feature importance shifts across different clinical domains. This dual analysis reinforces the behavioural and contextual heterogeneity in trial recruitment and provides empirical support for applying the HBM framework.

The global SHAP summary plot confirms the theoretical and operational dominance of perceived barriers, which exhibited the highest average contribution to model predictions (mean SHAP: 0.398). Barriers with high values consistently suppressed the likelihood of phone screening success, underscoring the primacy of logistical, emotional or procedural obstacles in shaping patient recruitment.

The second and third-ranked features, that is, disease classification and overall social vulnerability, highlight the complementary importance of clinical context and linguistic framing in the model. Overall social vulnerability suggests that how patients articulate their interest or concerns, as captured through NER and thematic analysis, meaningfully shapes algorithmic assessments of patient referral quality.

In contrast, features representing HBM constructs, such as internal belief systems like perceived susceptibility, severity, and benefits, offered only limited predictive power. This may indicate either a restricted expression of these constructs in the pre-screening context or a more general trend where structural and procedural factors have a greater influence than internal health beliefs during the referral stage. The minority indicator showed negligible predictive contribution, which reduces concerns of disparate model impact by ethnicity and affirms the model’s focus on behavioural rather than demographic signals.

To further explore these dynamics across medical domains, a SHAP heatmap was constructed, mapping the mean absolute SHAP values for each feature against corresponding disease classification. Hierarchical clustering applied to both rows (disease classification) and columns (features) revealed two major patterns. First, perceived barriers, overall social vulnerability, and perceived severity formed a tightly clustered feature group, suggesting that perceived barriers, thematic content of responses, and perceived severity often co-occur as dominant signals across various conditions. In contrast, perceived susceptibility and benefits clustered separately, reflecting less consistent influence across clinical contexts and reinforcing their subordinate role in model behaviour.

The disease classification row identified clinically meaningful groupings of disease categories with similar SHAP value profiles through clustering. For instance, the neurological, psychological, and infectious disease categories formed a coherent cluster characterised by a more substantial influence from perceived barriers and severity, suggesting a heightened concern over procedural burden or clinical seriousness. Conversely, female urogenital diseases and congenital disorders exhibited more idiosyncratic SHAP patterns, pointing to unique contextual or demographic influences not captured by other domains. These findings suggest that trial recruitment dynamics vary substantially across indications, reinforcing the need for disease-specific referral strategies.

In summary, the combined SHAP visual analyses offer a multidimensional understanding of model decision-making. They confirm that perceived barrier-related perceptions and language-based response patterns dominate predictive behaviour, while traditional health beliefs play a secondary role. Moreover, the variability of SHAP influence across clinical conditions highlights the importance of stratified recruitment strategies.

These findings validate the integration of the HBM into the XGBoost model and suggest that perceived burden and disease classification framing are central levers for improving patient referral conversion and equity in industry-sponsored trials.

6.2.1 Correlation analysis of HBM constructs across disease domains

Spearman’s rank-order correlation was employed to examine monotonic relationships among HBM constructs across disease classifications (Figure 7). This analysis revealed consistently strong and statistically robust inverse correlations between perceived benefits and barriers, as well as severity and susceptibility, with coefficients often exceeding −0.95. This suggests a cognitive trade-off when patients evaluate risks versus benefits during pre-screening.

Conversely, perceived severity and susceptibility were highly positively correlated (often >0.98), indicating strong convergence in patients’ perceived disease threat. The perceived barriers dimension exhibited weaker and more variable associations, notably when correlated with perceived benefits and severity, where coefficients rarely exceeded ±0.25 in specific disease categories (e.g., mental and musculoskeletal disorders).

These findings indicate construct-specific differentiation and support the theoretical distinction between threat appraisal (severity and susceptibility) and action-modulating factors (benefit and barriers), with potential implications for feature independence in behavioural modelling pipelines.

A closer examination of the Spearman correlation coefficients, barriers, and other HBM constructs revealed a heterogeneous but theoretically meaningful pattern across disease domains. In the context of bacterial infections and mycoses, barriers demonstrated a strong inverse correlation with perceived benefits (ρ = −0.71), suggesting that patients who identified more barriers to participation were markedly less likely to perceive the potential benefits of clinical trial enrolment. This reflects a cognitive suppression effect, wherein a high logistical or emotional burden attenuates the perceived value of engagement. This dynamic is likely intensified by the acute and transmissible nature of bacterial diseases.

This finding is consistent with results from a trial using the HBM to assess the behaviour of mammography patients. [24]. Notably, barriers in this domain also showed moderate positive associations with perceived severity (ρ = 0.51) and susceptibility (ρ = 0.64), indicating that those who perceived the disease as severe and themselves as susceptible still felt encumbered by participation barriers.

By contrast, in categories such as mental disorders, otorhinolaryngologic diseases, and female urogenital conditions, perceived barriers-benefit correlations were weaker (ρ ranging from −0.22 to −0.25), and associations with perceived severity and susceptibility remained low to moderate. This attenuated inter-construct linkage suggests that in these contexts, perceived barriers function more independently of benefit appraisal and threat perception. The strong negative perceived barriers-benefit coupling in bacterial infections may thus represent a condition-specific motivational bottleneck, underscoring the need for targeted strategies during trial recruitment in infectious disease settings.

Correlations between perceived severity and susceptibility reveal a consistently strong and positive association across all disease categories. Correlation coefficients frequently exceeded 0.95, with several nearing unity (e.g., stomatognathic diseases: ρ = 0.999, behaviour and behaviour mechanisms: ρ = 0.999, congenital diseases: ρ = 0.999), indicating an almost linear monotonic relationship. This suggests that individuals who regard a condition as severe also tend to perceive themselves as highly susceptible, confirming the HBM theory that threat appraisal emerges from the joint perception of severity and personal risk (detailed in Appendix H).

These findings indicate that structured behavioural constructs derived through LLMs can effectively inform AI/ML models, and that perceived health threats with access concerns serve as crucial motivators in early trial engagement. This emphasises the theoretical consistency of HBM threat components while also highlighting the conditional independence of factors such as barriers, especially in behavioural models for participation in RCTs.

6.3 The effect of timing on pre-screening outcomes

Kaplan-Meier survival estimates revealed a consistent and progressive decline in the likelihood of successful patient contact as time elapsed from the initial referral. This speaks directly to the influence of the time lag between initial screening and site contact on the likelihood of a patient potentially moving beyond the initial site phone screening stage (research sub-question RQ1.2).

Figure 8 represents the mean site-level probabilities derived from an average cohort of 1891 clinical trial sites, offering robust insight into system-wide patterns of temporal degradation in patient engagement. On day 1, the average contact probability was 0.80 (95% CI: 0.67–0.88), suggesting a high initial likelihood of successful recruitment. However, this probability declined steadily, falling to 0.60 (CI: 0.50–0.69) by day 11. Significantly, the lower confidence limit (LCL) dropped below the critical 0.50 threshold on day 12, signalling that in at least half of all observed scenarios across sites, the probability of successful contact fell below chance if the patient was not engaged within this window.

This downward trend continued, with the mean contact probability declining to 0.46 (CI: 0.37–0.54) by day 31, representing a relative reduction of approximately 42.5% from the baseline. These findings underscore the narrow and operationally significant window of opportunity for referral conversion. The most rapid decline occurred during the first 7 to 10 days, during which the contact probability dropped by 14 percentage points, from 0.80 to 0.66, reflecting a nonlinear risk function. After day 10, the rate of decay attenuated, with probabilities declining more gradually, indicative of a saturation point in outreach efficacy.

Following dynamic adjustment using Kaplan-Meier (KM) survival probabilities, substantial shifts in the predicted likelihood of successful patient contact were observed across clinical disease classification domains. The unadjusted scores, generated via an XGBoost classifier, represented the baseline probability of referral conversion independent of elapsed time. However, when scaled by time-sensitive survival probabilities, the recalibrated cumulative probabilities, as shown in Figure 9, more accurately reflect the real-world engagement potential within each disease area.

Across the full sample, the mean static probability was 0.59. In contrast, the KM-adjusted cumulative mean dropped to 0.26, a 55.9% reduction, which highlights the systemic overestimation of referral viability when time decay is not considered. This effect was not uniform across disease classifications. Under the static model, high-performing categories, such as male urogenital diseases (0.82), cardiovascular diseases (0.80), and female urogenital diseases (0.73), retained relatively higher adjusted probabilities (0.42, 0.38, and 0.37, respectively), indicating more durable referral quality over time. In contrast, lower-performing categories such as digestive system diseases (0.33) and musculoskeletal diseases (0.39) exhibited substantial post-adjustment attenuation, with cumulative probabilities falling to 0.11 and 0.12, respectively.

This suggests that referral decay curves differ meaningfully by disease classification, potentially due to underlying differences in trial complexity, patient burden, symptom acuity, or pre-screening ambiguity. For instance, the steep decline in domains such as neoplasms (from 0.40 to 0.13) and endocrine disorders (from 0.49 to 0.17) may reflect delayed outreach combined with clinical complexity or informational overload during screening. Notably, certain domains with moderate static scores, such as diagnosis (0.54) and mental disorders (0.55), maintained intermediate adjusted probabilities (0.30 and 0.25, respectively), suggesting some resilience to time decay.

Static classification models alone mask these temporal vulnerabilities, leading to inefficient prioritisation and potential patient loss. Incorporating time-aware adjustments enables precision targeting and aligns referral efforts with empirically observed conversion windows. Consequently, these stratified findings reinforce the clinical and operational relevance of survival-adjusted scoring. They demonstrate that domain-specific referral decay dynamics must be considered in triage logic, resource allocation, and follow-up protocols.

7.1 The integration of artificial intelligence and machine learning techniques

In response to calls for process-oriented HBM research [52, 53], this research demonstrates that the HBM can serve as both a conceptual and operational scaffold for structuring ML inputs. LLM-derived features mapped to HBM constructs enabled patient responses to be transformed into predictive variables, aligning health behaviour with computational modelling.

The integration of behavioural constructs, such as perceived barriers, benefits and severity into the pre-screening dataset revealed both their analytical potential and limitations when treated as post hoc explanatory variables. The opportunity to reverse this logic, rather than applying behavioural theories retrospectively to interpret patient decisions, is a key insight emerging from the synthesis of SHAP value distributions and clustering patterns. The minimal contribution of perceived severity and benefits may not necessarily indicate that these constructs lack predictive relevance, but rather that the structure and content of pre-screening interactions fail to surface these dimensions, warranting further attention to address them effectively.

Pre-screening criteria are often operational, focusing on inclusion and exclusion thresholds that may not prompt participants to reflect on, or disclose, deeper motivational factors. As such, the low SHAP influence of perceived severity and benefits may reflect an information asymmetry between what the constructs represent and what is practically expressed in the data. This suggests a methodological gap: current screening protocols may under-capture rich behavioural indicators critical to understanding patient motivation for engaging in RCTs.

Future recruitment frameworks could proactively embed these constructs into pre-screening design. Specifically, operationalising behavioural readiness, motivational intent, and perceived trial burden at the point of initial screening may enable more nuanced and accurate stratification of patient referrals. This approach represents a shift from eligibility-based screening to behavioural risk stratification, with the potential to improve recruitment efficiency and enhance patient engagement.

7.2 Temporal modelling and operational relevance

The observed shifts in referral probability across disease classifications following the Kaplan-Meier (KM) adjustment confirm the critical need to incorporate temporal survival modelling into patient scoring pipelines. While static classifiers, such as XGBoost, effectively capture behavioural, clinical, and demographic predictors, they systematically overestimate referral viability when time-dependent engagement decay is not considered. This underscores temporal vulnerability as an independent risk dimension, not detectable through static modelling alone.

To address this, integrating KM survival probabilities into the scoring architecture provides a methodologically rigorous and operationally responsive enhancement. The end-to-end framework, which merges NLP for structured feature extraction, ensemble classification (XGBoost) for probabilistic labelling, and non-parametric survival analysis (Kaplan-Meier) for time-aware recalibration, enables dynamic, risk-adjusted scoring that reflects both the initial likelihood of contact and its temporal resilience.

This multidisciplinary approach supports real-time triage of patient referrals, aligning model outputs with the cadence and constraints of clinical trial workflows. The resulting framework functions not merely as a predictive engine but as a decision-support mechanism, prioritising outreach based on cumulative probability degradation and clinical urgency at the disease classification level. By contextualising risk within site-level operations, contact timelines and disease-specific referral behaviour, this model/framework bridges analytical performance with recruitment feasibility.

The findings provide empirical support for integrating Kaplan-Meier-derived survival probabilities into dynamic risk scoring models. Traditional static classification methods fail to account for the temporal decline in site contact and patient responsiveness. By incorporating time-to-event adjustments, the referral model can prioritise cases during the peak period for contact, minimising operational inefficiencies and reducing site burden.

Future work will focus on monitoring calibration drift arising from shifts in recruitment strategies, campaign responsiveness or systemic outreach delays. Evaluating model robustness across evolving referral pathways will be critical to ensuring sustained accuracy, equity and clinical utility.

7.3 Addressing bias and enhancing equity

While the HBM remains a robust behavioural framework, it does not explicitly accommodate structural inequities or population-level disparities. Equity was considered not by altering the HBM to include elements of diversity or structural factors which could compromise its behavioural focus, but rather by integrating SVI features into the model’s architecture. Similarly, fairness was addressed procedurally through model design. This method preserved the behavioural integrity of the HBM while accommodating the structural inequities that affect patient recruitment.

Model bias revealed through heterogeneous predictive performance in disease classification highlights the need for domain-specific calibration. Potential adjustments to decision thresholds could mitigate asymmetric performance and optimise operational utility in referral scoring workflows.

7.4 Theoretical and practical contribution

This research introduces an innovative interdisciplinary approach that combines theoretical models with practical applications. It uses a hybrid modelling framework that retains the strengths of individual-level cognitive concepts (e.g. perceived barriers and benefits) while also incorporating population-level fairness considerations directly into its design. In doing so, it represents a methodological advancement in which ethical safeguards can be built into models through careful feature selection and validation, rather than by expanding the underlying theoretical concepts [43]. Fairness was reinforced through stratified performance checks across SVI-defined subgroups, confirming the model’s stability and equity across varying levels of structural vulnerability.

This research seeks to contribute to the growing body of work that emphasises equity-informed AI/ML in patient recruitment, without compromising the integrity of domain-specific theoretical frameworks. It presents a blueprint for incorporating fairness into behavioural modelling pipelines by using structural covariates and performance stratification, rather than relying on post-hoc corrections or theoretical overreach.

9.1 Prospective validation and deployment evaluation

This research reports retrospective model development and internal validation. To establish prospective performance under live operating conditions, future work will evaluate the PRS model in a forward-looking, time-split design using newly accrued referrals after a model freeze date. In an initial silent-mode deployment, predictions will be generated without influencing outreach decisions, enabling unbiased estimation of discrimination (AUC), calibration (Brier score; calibration slope/intercept), and operational utility (precision/recall at action thresholds and yield per outreach effort). Performance will be monitored at predefined intervals and disaggregated by disease classification and minority indicator to detect drift and inequitable degradation. Recalibration or retraining will be triggered when calibration or subgroup performance falls below pre-specified tolerances.

9.2 Perform deployment utility and decision-threshold analysis (non-monetary cost: Benefit)

Because monetary costing parameters were not available in this retrospective dataset, deployment value will be quantified using operational proxies: successful phone screenings per unit outreach effort (e.g., per recruiter-hour or per 1000 outreach attempts), effort per success (attempts/minutes per successful screen), and, where available, downstream progression. Decision thresholds will be selected using an expected-utility approach, based on plausible relative cost ratios for false positives (wasted outreach) versus false negatives (missed eligible referrals), evaluated across disease classifications and the minority indicator, to ensure that efficiency gains do not introduce inequitable performance. Where trial-level cost inputs become available, these operational metrics can be translated into monetary net benefit.

Appendix A: Patient referral dataset attributes

(See Table A1).

Appendix B1: Patient referral univariate analysis

5,436,136 Observations 69 Variables, Phone_Screen_Date 2020-2104-21 to 2025-2104-15 (Table B1).

Appendix C: Bivariate analysis, Pearson correlation, and Spearman rho

(See Figure C1).

Appendix D: Feature engineering, encoding and interaction terms

To support predictive modelling, a structured feature engineering approach was adopted. This process transformed the patient referral data into analytically meaningful features, designed to capture both behavioural and protocol-related dimensions relevant to screening outcomes (Table D1).

Appendix E: Multivariate analysis, GLM model odds ratios (OR)

In interpreting the logistic regression results, odds ratios (ORs) were used to assess the direction and magnitude of each predictor’s effect on the likelihood of a patient completing the phone screening. An OR greater than 1 indicates that the predictor increases the likelihood of progressing to the phone screening stage, whereas an OR less than 1 suggests a decreased likelihood. An OR close to 1 implies that the predictor has minimal or no discernible effect. Predictors were considered statistically significant and thus meaningful in the model when the associated p-value was less than 0.05 (Table E1).

Appendix F.1: HBM constructs feature engineering

To theoretically ground the analysis within the Health Belief Model (HBM), four core constructs were derived from structured patient data and integrated as explanatory features. These theoretically informed constructs were foundational to the predictive modelling framework and aligned with established behavioural science literature to enhance interpretability and construct validity (Table F1).

Appendix G: Patient referral scoring confusion matrix by mesh heading

(See Table G1).

Appendix H: Patient referral scoring HBM constructs Spearman CHBM

(See Table H1).

Appendix I: Modelling approaches

Two modelling approaches were tested: a wide-format structure, where each HBM domain was represented as a distinct feature in a single-row record per patient and trial referral, and a long-format structure, where each HBM domain contributed a separate row per patient per trial. Both approaches were evaluated to assess the trade-offs between model simplicity, granularity, and the extent of required imputation due to missing domain-level information.

To assess how data structure influences classification performance across clinical domains, confusion matrix metrics (accuracy, sensitivity, and specificity) were compared between wide- and long-format models. While overall accuracy remained broadly comparable, with minimal divergence in most mesh headings, there were critical differences in sensitivity and specificity that reflect trade-offs in granularity versus model conservatism. The wide-format model demonstrated higher specificity in most domains (e.g., digestive system diseases: 99.7 vs. 97.1%, chemically induced disorders: 100 vs. 5.6%), indicating a bias toward minimising false positives. However, this came at the cost of substantially lower sensitivity, particularly in underrepresented conditions such as stomatognathic diseases (3.1 vs. 11.5%) and Bacterial Infections and Mycoses (10.9 vs. 19.3%).

In contrast, the long-format model consistently improved sensitivity, especially in domains with variable HBM coverage. Notably, immune system diseases saw a gain from 68.4 to 76.2%, and skin and connective tissue diseases rose from 76.3 to 84.9%. Physiological phenomena achieved 100% sensitivity under the long format, albeit with a complete loss in specificity (0%). These shifts highlight the long-format model’s strength in recovering true positives, likely due to its finer-grained representation of domain-level constructs. However, this approach also introduced increased noise and overfitting in some categories, particularly where positive cases are rare and imputation rates were higher.

Overall, the wide-format model offered greater parsimony and better control of false positives, making it advantageous for high-prevalence categories or operational use-cases requiring high specificity. In contrast, the long-format model was superior for recall-oriented applications, particularly in exploratory or safety-critical contexts where false negatives must be minimised. These findings support a context-sensitive deployment strategy, selecting format structures aligned with domain prevalence, data sparsity, and end-use priorities (Table I1).