A bank or credit union can use models throughout its business without building a single one. Fraud scoring, anti-money laundering alert ranking and credit decisions may all rely on quantitative methods embedded in vendor products. The institution still needs to understand how those methods influence its decisions and customer outcomes.
Buying the software can limit access to the underlying methodology, but it does not remove the need for oversight. A proportionate approach starts by identifying the models in use, assessing their significance and deciding what evidence is needed to support their continued use.
For a practical guide to connecting risk, controls and evidence across your institution, download Protecht’s Risk in motion eBook.
Understand the current guidance and its scope
In April 2026, US banking agencies issued revised model-risk guidance that superseded SR 11-7. It is expected to be most relevant to banks with more than $30 billion in assets, although it may also be relevant to smaller banks with significant model exposure. Generative and agentic AI are outside its scope. Smaller institutions still need internal governance suited to their activities and risk profile; the revised guidance does not justify treating every vendor feature as a model or applying large-bank validation arrangements indiscriminately.[1]
In Canada, OSFI’s revised E-23 guideline takes effect on May 1, 2027. It covers federally regulated financial institutions and includes AI and machine-learning models, with oversight proportionate to model risk. Its scope should not be presented as a blanket requirement for every provincially regulated credit union. Institutions need to establish which framework applies before planning implementation.[2]
For vendor models within the applicable framework, the institution needs enough understanding to assess suitability, limitations and performance. That responsibility includes deciding when missing evidence makes a proposed use unacceptable. Third-party oversight and model governance should support the same decision rather than produce separate assessments with conflicting conclusions.
Work with the evidence available
Proprietary methodology and confidential training data can restrict what a vendor discloses. Those limitations should be recorded and assessed. Outcome testing, independent expertise and appropriate restrictions on use can help, but cannot automatically replace all the evidence needed for a material model.
Build an inventory that distinguishes quantitative models from deterministic rules and other tools. For each model, identify the owner, purpose, vendor, version, decisions influenced and materiality. Separately record generative or agentic tools under the governance arrangements that apply to them. Procurement, technology and business owners can help find capabilities that a central risk team would otherwise miss.
Request development documentation, validation results, performance information, relevant bias testing and change records. Assess whether the evidence covers the version and use deployed at your institution. A vendor’s validation can inform your assessment, but does not establish that the model is suitable for your customer population or configuration.
For example, a fraud model tested on a different transaction mix may produce an unsuitable balance between detection and customer disruption. Compare the evidence with your own portfolio and explain any limits on the conclusion. Where data or methodology remains unavailable, document the uncertainty, the compensating controls and the decision-maker accepting the residual risk.
Monitor outcomes against agreed expectations. Approval rates, fraud detection performance, overrides and complaints can reveal changes that aggregate accuracy measures conceal. Assign owners and thresholds, and define the action that follows a breach. Where outcomes emerge slowly, such as credit losses, use appropriate interim indicators alongside longer-term analysis.
A useful monitoring record links the indicator to its source data, threshold, owner and response. If false positives rise, the response may include checking data quality, reviewing a vendor change and assessing staff workload. Those actions should be traceable to the model and business process so that later reviews can determine whether performance improved.
Require notice of material changes and establish a review process proportionate to their consequences. A change to fraud scoring may alter declined transactions, customer contacts and manual-review workloads. Assess those effects before deployment where possible, and compare results with the baseline afterward.
Record overrides and investigate recurring patterns. Frequent overrides may indicate a model limitation, a data-quality problem or inconsistent staff practice. Where users cannot override a result, define an escalation route and a way to contain harm while the issue is reviewed.
Make proportionality a documented decision
Rank models by their purpose, exposure and potential consequences. A model influencing credit access or financial-crime decisions warrants different oversight from a low-risk productivity tool. Document the basis for each tier and review it when use expands. Specialist external support may be needed for material models; proportionality should guide resources without becoming a reason to leave significant risks unassessed.
Proportionality also depends on access to expertise. A small team may coordinate the inventory and monitoring while obtaining independent specialist review for a material model. Define the question that external support must answer, the evidence it can access and how findings will be resolved. Outsourcing the review still leaves the institution responsible for deciding whether use is acceptable.
Connect model oversight with vendor and compliance work
Link the model inventory to vendor records, relevant obligations, monitoring controls and incidents. Validation evidence then informs due diligence, change notices feed the review workflow, and complaints can be considered alongside performance indicators. A connected record makes it easier to see whether an issue affects one model, several models from the same vendor or a shared source of data. The useful management question is whether every consequential model has an owner, evidence supporting its use and a response when performance deteriorates.
Start with a small set of consequential models and make the governance cycle work before widening coverage. Confirm that owners can obtain the required evidence, monitoring produces usable results and escalations lead to decisions. This gives the institution a practical foundation for broader coverage while making any remaining gaps visible to management.
Next steps for your organization
Vendor model oversight depends on bringing validation evidence, monitoring results and change decisions together. Protecht connects vendor assessments with risks, controls, obligations, incidents and actions, helping teams track evidence gaps, assign follow-up and report material issues alongside the business processes those models support.
Request a Protecht demo to see how connected risk and compliance workflows can support the decisions your institution needs to make.
References
[1] US banking agencies, Revised Guidance on Model Risk Management, April 17, 2026. https://www.federalreserve.gov/frrs/guidance/supervisory-guidance-on-model-risk-management.htm
[2] OSFI, Guideline E-23, Model Risk Management, effective May 1, 2027. https://www.osfi-bsif.gc.ca/en/guidance/guidance-library/guideline-e-23-model-risk-management-2027