Model Governance and Controls Requirements
Model Governance and Controls Requirements
Model governance establishes the framework for developing, validating, and monitoring quantitative models used in derivatives pricing, risk measurement, and valuation. Regulatory guidance, particularly SR 11-7 and OCC 2011-12, requires financial institutions to manage model risk through robust governance, independent validation, and ongoing monitoring.
Definition and Key Concepts
Model Risk Management
Model risk: The potential for adverse consequences from decisions based on incorrect or misused model outputs.
| Risk Type | Description |
|---|---|
| Model error | Incorrect specification or implementation |
| Input error | Inaccurate or inappropriate data |
| Misuse | Applying model outside intended scope |
| Limitation risk | Failure to understand model constraints |
Governance Framework
| Component | Description |
|---|---|
| Model inventory | Catalog of all models in use |
| Model owners | Designated accountability |
| Validation | Independent review and testing |
| Documentation | Technical specifications, assumptions |
| Change control | Process for model modifications |
| Monitoring | Ongoing performance tracking |
Model Tiers
| Tier | Criteria | Validation Frequency |
|---|---|---|
| Tier 1 (High) | Material financial impact, regulatory capital | Annual full validation |
| Tier 2 (Medium) | Moderate impact, internal risk | Every 2 years |
| Tier 3 (Low) | Limited impact, operational | Every 3 years |
How It Works in Practice
Model Development Standards
Development lifecycle:
| Phase | Activities |
|---|---|
| Conceptual | Define scope, select methodology |
| Development | Implement, code, test |
| Validation | Independent review |
| Implementation | Deploy to production |
| Monitoring | Ongoing performance tracking |
Documentation requirements:
| Document | Content |
|---|---|
| Model specification | Mathematical framework, assumptions |
| Development report | Implementation details, testing results |
| User guide | Operating procedures, inputs/outputs |
| Validation report | Independent assessment, findings |
| Monitoring plan | Metrics, thresholds, escalation |
Validation Process
Validation scope:
| Area | Activities |
|---|---|
| Conceptual soundness | Review theory, assumptions |
| Implementation testing | Code review, unit tests |
| Outcomes analysis | Backtesting, benchmarking |
| Sensitivity analysis | Parameter stress testing |
| Limitation assessment | Identify constraints, document risks |
Independence requirements:
| Validation Type | Who Performs |
|---|---|
| Tier 1 models | Independent model validation group |
| Tier 2 models | Qualified independent party |
| Tier 3 models | Self-validation with oversight |
Model Inventory
Required fields:
| Field | Description |
|---|---|
| Model ID | Unique identifier |
| Model name | Descriptive name |
| Model owner | Responsible individual |
| Business use | Pricing, risk, capital |
| Tier classification | High, medium, low |
| Last validation | Date of most recent review |
| Next validation due | Scheduled date |
| Status | Active, development, retired |
Worked Example
Validating an Options Pricing Model
Model details:
- Model: Black-Scholes for European equity options
- Use: Pricing and risk management
- Tier: Tier 1 (material valuation impact)
- Owner: Derivatives Trading Desk
Validation scope:
| Validation Area | Scope |
|---|---|
| Conceptual soundness | BSM assumptions appropriateness |
| Implementation | Code review, numerical accuracy |
| Outcomes analysis | Comparison to market prices |
| Sensitivity | Greeks accuracy |
| Limitations | Volatility smile not captured |
Step 1: Conceptual Review
| Assumption | Assessment |
|---|---|
| Constant volatility | Limitation: vol smile not modeled |
| No dividends | Adjustment needed for dividend stocks |
| European exercise | Appropriate for index options |
| Log-normal returns | Standard market convention |
Finding: Model appropriate for liquid, short-dated options; limited for long-dated or exotic products.
Step 2: Implementation Testing
| Test | Method | Result |
|---|---|---|
| Unit test | Compare to analytic solutions | Pass |
| Boundary test | Test at extreme parameters | Pass |
| Numerical precision | Verify to 8 decimal places | Pass |
| Greeks calculation | Compare to finite difference | Pass |
Step 3: Outcomes Analysis
| Metric | Benchmark | Result |
|---|---|---|
| Average pricing error | <0.5% | 0.3% |
| Max pricing error | <2% | 1.8% |
| Delta accuracy | <1% | 0.5% |
| Vega accuracy | <2% | 1.2% |
Backtesting results:
| Period | Trades | Within Tolerance | Exception Rate |
|---|---|---|---|
| Q1 2024 | 15,000 | 14,850 | 1.0% |
| Q2 2024 | 16,200 | 16,038 | 1.0% |
| Q3 2024 | 14,800 | 14,652 | 1.0% |
Step 4: Validation Conclusion
| Category | Rating |
|---|---|
| Conceptual soundness | Satisfactory |
| Implementation | Satisfactory |
| Outcomes | Satisfactory |
| Overall | Approved with conditions |
Conditions:
- Document limitation for volatility smile
- Implement quarterly backtesting
- Review for long-dated option use
Risks, Limitations, and Tradeoffs
Model Risk Categories
| Risk | Description | Mitigation |
|---|---|---|
| Specification error | Wrong model for product | Careful model selection |
| Calibration error | Incorrect parameter estimation | Independent calibration review |
| Implementation bug | Coding errors | Code review, testing |
| Data quality | Bad inputs produce bad outputs | Data validation |
| Misapplication | Using model outside scope | Clear use guidelines |
Governance Failures
| Failure | Consequence |
|---|---|
| Missing validation | Regulatory finding, capital add-on |
| Inadequate documentation | Unable to explain model |
| No monitoring | Degradation goes undetected |
| Weak oversight | Model errors persist |
Common Pitfalls
| Pitfall | Description | Prevention |
|---|---|---|
| Stale validation | Validation not current | Enforce schedule |
| Scope creep | Model used beyond intent | Clear use restrictions |
| Over-reliance | Model treated as truth | Acknowledge limitations |
| Undocumented changes | Modifications not recorded | Change control process |
| Insufficient testing | Limited test cases | Comprehensive test plan |
Regulatory Expectations
SR 11-7 / OCC 2011-12 Requirements
| Requirement | Description |
|---|---|
| Board oversight | Senior management accountability |
| Model inventory | Complete catalog of models |
| Validation | Independent, qualified review |
| Documentation | Comprehensive specifications |
| Ongoing monitoring | Performance tracking |
| Internal audit | Periodic framework review |
Examination Focus
| Area | Examiner Questions |
|---|---|
| Governance | Is there board-level oversight? |
| Inventory | Are all models cataloged? |
| Validation | Is validation independent and qualified? |
| Documentation | Are specifications complete? |
| Monitoring | Is performance tracked? |
| Findings | Are issues addressed timely? |
Checklist and Next Steps
Model development checklist:
- Define model scope and use
- Document theoretical framework
- Implement with quality controls
- Conduct developer testing
- Prepare validation package
- Submit for independent validation
Validation checklist:
- Review conceptual soundness
- Test implementation accuracy
- Conduct outcomes analysis
- Perform sensitivity testing
- Document limitations
- Issue validation opinion
Ongoing monitoring checklist:
- Track model performance metrics
- Compare outputs to benchmarks
- Review exception reports
- Assess model usage patterns
- Escalate material issues
- Update validation as needed
Governance checklist:
- Maintain complete model inventory
- Assign model owners
- Enforce validation schedule
- Review findings and remediation
- Report to senior management
- Conduct periodic framework review
Related articles:
- For sales practices, see Regulation Best Interest and Derivative Sales
- For accounting, see Accounting Standards ASC 815 Overview