Report on Plans and Priorities
January 2008
Table of contents
Executive Summary
Introduction
Audit Objective
Audit Scope
Overall Audit Opinion
Statement of Assurance
Summary of Internal Control Strengths
Summary of Internal Control Weaknesses
Detailed Report
Audit Scope
Audit Methodology and Criteria
Findings and Recommendations
Appendix A - Audit Criteria and Conclusions
Appendix B - TBS and Corporate Linkages to the RPP
Executive Summary
Introduction
The internal audit of the Report on Plans and Priorities is part of the Risk-Based Annual Internal Audit Plan 2007-08 approved by the Canadian Institutes of Health Research (CIHR) Governing Council.
Each year, federal government departments and agencies prepare Estimates in support of their requests to Parliament for authority to spend public monies. Part III of the Estimates includes Reports on Plans and Priorities (RPP) and Departmental Performance Reports (DPR). RPPs provide planned spending information on a strategic outcome and program activity basis per the Program Activity Architecture (PAA), and describe departmental priorities, expected results, and the associated resource requirements, covering a three-year period. The target audience for the RPP is parliamentarians who rely on these reports to perform their role of holding the government accountable for the public funds entrusted to it.
Audit Objective
The audit objective was to assess the adequacy and effectiveness of internal controls over the integrity of information in the CIHR RPP. The integrity of information is defined by its relevance, reliability, comprehensiveness, consistency, and balance.
Audit Scope
The audit focused on internal controls over the integrity of information in the 2007-08 Report on Plans and Priorities and not on the information itself. Therefore, this was not an attest audit of whether the information is correctly presented. The audit was based on the reporting principles contained in the Treasury Board Secretariat (TBS) Guide for the Preparation of Part III of the 2007-2008 Estimates: Reports on Plans and Priorities and Departmental Performance Reports.
Overall Audit Opinion
The process for developing CIHR's Report on Plans and Priorities (RPP) is well controlled overall and has only one area containing moderate issues related to the linking of resources to results. The underlying cause of these issues is the CIHR Program Activity Architecture (PAA) approved by the Treasury Board (TB) for reporting on plans and priorities. This PAA is not adequately aligned with the mandate and operations of CIHR to optimize the linking of resources to results. However, the overall risk exposure is limited because of other, compensating internal controls.
Statement of Assurance
In my professional judgement as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinion provided in this report. The audit of the internal controls supporting the development of CIHR's RPP 2007-2008 was conducted in accordance with the Federal Government Policy on Internal Audit and related professional standards. The audit opinion is based on a comparison of conditions that existed at the time of the audit against established audit criteria that were agreed upon with management. The evidence is sufficient to provide senior management with proof of the opinion.
Summary of Internal Control Strengths
Strategic Planning and Priority Setting
- CIHR has an established, formal strategic planning and priority setting process that is updated on an annual basis.
- The strategic planning process feeds the development of the RPP and confirms that all strategic outcomes are included in the PAA and RPP.
- The strategic planning and priority setting process, along with the RPP development process, is fully integrated into CIHR's Planning, Budgeting and Reporting Cycle 2007-08. CIHR's strategic plans clearly cascade into a set of operational and tactical plans describing resources required to achieve outcomes.
- CIHR's summer strategic planning and priority setting retreat of its Governing Council and its operational planning process both consider risk management (at organizational and program levels) and potential consequences for CIHR.
- CIHR's RPP process ensures the RPP is based on the program activities and strategic outcomes contained in the PAA.
- CIHR's plans and priorities link to government-wide priorities.
Roles, Responsibilities, and Accountabilities
- Roles, responsibilities, and accountabilities for the RPP are clearly defined, including the drafting, review, and approval phases as well as oversight and quality control, and are communicated to all relevant CIHR employees and governance bodies.
Oversight and Quality Assurance
- Quality control review is performed by appropriate governance bodies (Standing Committees) to challenge the key assumptions and related resource allocations, and to ensure the RPP is developed in accordance with TBS reporting requirements.
- The established set of procedures and oversight mechanisms result in internal controls that ensure the RPP is complete, accurate, appropriate, and reasonable.
- Appropriate approvals (Director, VP levels) are evidenced and RPP final approval is provided by authorized signatories (including the President of CIHR and the Minister).
TBS Reporting Principles and Requirements
- Within the confines of the current PAA, the RPP information is based on the reporting principles contained in TBS guidelines, with meaningful linkages to financial information, and a focus on outcomes and some expected measurable results that provide a foundation for related performance documents (e.g., the Departmental Performance Report).
- The RPP focuses on the benefits for Canadians, explaining the critical aspects of planning and performance, and setting them in context. Specifically, benefits for Canadians are well explained in terms of initiatives, partnerships, and government-wide priorities. The operating environment and collaborative and strategic efforts are well explained.
- The RPP presents credible, reliable, and balanced information. In particular, strategic, programmatic, and administrative risks that have been identified are adequately addressed; sources are stated to verify information. Information provided is consistent with the TBS Management, Resources and Results Structure (MRRS), specifically the Program Activity Architecture (PAA) component.
- The RPP associates performance with plans, priorities, and expected results, explains changes, and applies lessons learned. The Strategic Outcomes (Research, Researchers, and Knowledge Transfer) contained in the RPP are those contained in the TBS approved PAA. Application of lessons learned is evidenced and adequate and proposed changes focus on performance measurement and evaluation.
- Reporting is consistent from year to year to enable comparisons. The 2006 International Review Panel Report and the 2006 Survey of Funded and Non-Funded Researchers provide independent performance measurement indicators of results achieved. Financial controls are in place to provide accurate financial information in the RPP.
Summary of Internal Control Weaknesses
TBS Reporting Principles and Requirements
-
The RPP does not adequately address TBS Reporting Principle 4: Link Resources to Results because of the very broad nature of the PAA's Strategic Outcomes (SOs) and Program Activities (PA). The PAA as approved by TBS does not support the desired linkages between (1) CIHR Mandate, Research Themes, Strategic Initiatives, and Strategic Plan, and (2) the current CIHR PAA including Strategic Outcomes, Strategic Priorities, and Program Activities.
SOs are defined at a very high level and are, therefore, general in nature. The high-level nature of SOs and PAs does not facilitate "managing for outcomes." Consequently, reporting to parliament may not be as informative and relevant as it could be because it is not possible to determine whether resources are being used efficiently and effectively with respect to the outcomes and the priorities.
Minor Issues
Some minor issues were identified that could be addressed to strengthen CIHR's RPP internal controls. The level of risk associated with these issues is considered low in likelihood, with a low level of consequence.
Strategic Planning and Priority Setting
- CIHR's business planning is not always completed before the RPP must be produced. Completing the RPP before completing CIHR's annual business planning exercise could increase the risk of misalignment between the corporate business plan and the RPP.
- Risk information could be strengthened by the development of an approved Corporate Risk Profile (CRP) and Risk Management Strategy and Plan. A CRP would provide management with a comprehensive list of risks, their causes, the existing controls, the consequences if the risks materialize, an objective determination of risk severity, and action plans to target interventions better and to enhance the likelihood of achieving outcomes.
Oversight and Quality Assurance
- RPP-related files and documentation are not consistently maintained in a central location. The result is an increased risk of inability to retrieve information that supports the RPP, increased risk of corporate memory loss, and risk of overlooking potential lessons learned.
Dev Loyola-Nazareth
Chief Audit Executive
Canadian Institutes of Health Research
Detailed Report
Audit Scope
The audit scope covered the RPP information related to the Management Representation Statement that is signed by the CIHR President:
"I submit for tabling in Parliament the 2007-2008 Report on Plans and Priorities (RPP) for the Canadian Institutes of Health Research.
This document has been prepared based on the reporting principles contained in Guide for preparation of Part III of the 2007-2008 Estimates: Reports on Plans and Priorities and Departmental Performance Reports:
- It adheres to the specific reporting requirements outlined in the TBS guidance;
- It is based on the department's Strategic Outcomes and Program Activity Architecture that were approved by the Treasury Board;
- It presents consistent, comprehensive, balanced and reliable information;
- It provides a basis for accountability for the results achieved with the resources and authorities entrusted to it; and
- It reports finances based on approved planned spending numbers from the Treasury Board Secretariat."
The audit addresses the risk that the Management Representation Statement signed by the President will not be a true reflection of the RPP.
Audit Methodology And Criteria
The approach included interviews, documentary and transaction review, and analysis. The fieldwork was conducted between September and November 2007.
The audit criteria were derived from the TBS Guide for the Preparation of Part III of the 2007-2008 Estimates: Reports on Plans and Priorities and Departmental Performance Reports; Management, Resources and Results Structure (MRRS) and related Program Activity Architecture (PAA); Risk Management Policy; Performance Reporting - Good Practices Handbook; Policy on Transfer Payments (Grants & Contributions); and TBS: A Review of Business Planning.
Appendix A contains detailed criteria and conclusions.
Appendix B illustrates the linkages between TBS and corporate requirements and demonstrates how the RPP is integrated into these relationships.
Findings And Recommendations
| Observation | Impact | Recommendation |
|---|---|---|
| 1. TBS Reporting Principles and Requirements | ||
|
The RPP does not adequately address TBS Principle 4: Link Resources to Results because of the very broad nature of the PAA's Strategic Outcomes (SOs) and Program Activities (PA). The PAA as approved by TBS does not support the desired linkages between (1) CIHR Mandate, Research Themes, Strategic Initiatives, and Strategic Plan, and (2) the current CIHR PAA including Strategic Outcomes, Strategic Priorities, and Program Activities. SOs are defined at a very high level and are, therefore, general in nature. The high-level nature of SOs and PAs does not facilitate "managing for outcomes." Consequently, reporting to parliament may not be as informative and relevant as it could be because it is not possible to determine whether resources are being used efficiently and effectively with respect to the outcomes and the priorities. |
|
The Chief Financial Officer should work with TBS to revise the PAA to define strategic outcomes that allow for increased alignment to the CIHR mandate and business activities, in order to facilitate the tracking of progress and provide information to parliament that more clearly demonstrates the achievement of progress. |
| 2. Minor Issues | ||
| 2.1 Strategic Planning and Priority Setting | ||
| 2.1 CIHR's business planning is not always completed before the RPP must be produced. The strategic planning and priority setting process, along with the RPP development process, is fully integrated into CIHR's Planning, Budgeting and Reporting Cycle 2007-08. CIHR's strategic plans clearly cascade into a set of operational and tactical plans describing resources required to achieve outcomes. However, RPPs are usually tabled on or before March 31, and submission timelines as dictated by TBS can require their completion and approval before the end of the CIHR business planning process. | This increases the risk that the RPP will not accurately reflect CIHR's planned business activities and resource allocations. | The Chief Financial Officer should consider raising the issue of misalignment of RPP timelines as part of the discussion in recommendation 1. |
| 2.2 Risk information could be strengthened by the development of an approved Corporate Risk Profile (CRP) and Risk Management Strategy and Plan. CIHR's summer strategic planning and priority setting retreat of its Governing Council and its operational planning process both consider risk management (at organizational and program levels) and potential consequences for CIHR. A CRP would provide management with a comprehensive list of risks, their causes, existing controls, the consequences if the risks materialize, an objective determination of risk severity, and action plans to target interventions better and enhance the likelihood of achieving outcomes. | Incomplete risk information can impact the achievement of corporate objectives. | The Chief Financial Officer should develop and maintain (an annual refresh) a Corporate Risk Profile and Risk Management Strategy and Plan for CIHR as part of its risk management framework. |
| 3. Oversight and Quality Assurance | ||
| RPP-related files and documentation are not consistently maintained in a central location. | The result is an increased risk of our inability to retrieve information, increased risk of corporate memory loss, and a risk of overlooking potential lessons learned. | The Chief Financial Officer should ensure that all RPP-related files and documentation are stored centrally. |
Appendix A - Audit criteria and conclusions
| Assessment Ranking | Description of Assessment |
|---|---|
| Well Controlled | Well managed, no material weaknesses noted or only minor improvements are needed. |
| Moderate Issues | Control weaknesses, but exposure is limited because either the likelihood or impact of the risk is not high. |
| Significant Improvements Required | Requires significant improvements in the area of material financial adjustments, or control deficiencies represent serious exposure. |
Overall Conclusion: The process for developing CIHR's Report on Plans and Priorities (RPP) is well controlled overall and has only one area containing moderate issues related to the linking of resources to results. The underlying cause of these issues is the CIHR Program Activity Architecture (PAA) approved by the Treasury Board (TB) for reporting on plans and priorities. This PAA is not adequately aligned with the mandate and operations of CIHR to optimize the linking of resources to results. However, the overall risk exposure is limited because of other, compensating internal controls.
| Control Element | Audit Criteria | Conclusion/ Assessment | Observation Number |
|---|---|---|---|
|
Strategic Planning and Priority Setting CIHR's strategic planning and priority-setting process feeds the development of the RPP. |
A formal strategic planning process exists. | Well Controlled | |
| The strategic planning process feeds the development of the RPP. | Well Controlled | ||
| Strategic plans clearly cascade into a set of operational and tactical plans describing resources required to achieve outcomes. | Well Controlled, with minor issue | 2.1 | |
| Risk Management is in place. | Well Controlled, with minor issue | 2.2 | |
| The RPP is based on CIHR's approved PAA. | Well Controlled | ||
| The organization's plans and priorities link to government-wide priorities. | Well Controlled | ||
|
Roles, Responsibilities, and Accountabilities Roles, responsibilities, and accountabilities for the development of the RPP are clearly defined and well. |
Roles, responsibilities, and accountabilities are clearly defined, including the drafting, review, and approval phases of the RPP, as well as oversight and quality control, and are communicated to all relevant CIHR employees and governance bodies, and are well understood. | Well Controlled | |
|
Oversight and Quality Assurance Review and oversight mechanisms are in place to ensure the RPP is complete, accurate, appropriate, and reasonable. |
Established set of procedures and oversight mechanisms result in internal controls that ensure the RPP is complete, accurate, appropriate, and reasonable. | Well Controlled | |
| Director level and VP level RPP approvals are evidenced and authorized signatories provide final RPP approval. | Well Controlled | ||
| Appropriate governance bodies perform quality control review. | Well Controlled | ||
| Information is in place, with a thorough, complete, and centralized project file relating to each RPP. | Well Controlled, with minor issue | 3 | |
|
TBS Reporting Principles and Requirements RPP information is based on the reporting principles contained in TBS guidelines, including relevance, balance, comprehensiveness, consistency, completeness, with meaningful linkages to financial information and a focus on outcomes and expected (measurable) results which provide a foundation for related performance documents (i.e., DPR). |
Principle 1: Focus on the benefits for Canadians, explain the critical aspects of planning and performance, and set them in context. | Well Controlled | |
| Principle 2: Present credible, reliable, and balanced information. | Well Controlled | ||
| Principle 3: Associate performance with plans, priorities, and expected results, explain changes, and apply lessons learned. | Well Controlled | ||
| Principle 4: Link resources to results. | Moderate Issues | 1 | |
| Principle 4: Reporting is consistent from year to year to enable comparisons. Financial controls are in place to provide accurate financial information in the RPP. | Well Controlled |
Appendix B - TBS and Corporate Linkages to the RPP
The figure below illustrates the linkages between TBS and corporate requirements and demonstrates how the RPP is integrated into these relationships. This is the context that was used for analyzing the alignment of CIHR's strategic plan, PAA, operational business plans, and RPP.
Linkages between TBS and Corporate Requirements
Management Action Plan
December 2007
| Observation | Recommendation | Action Plan | Timeline |
|---|---|---|---|
| 1. TBS Reporting Principles and Requirements | |||
|
The RPP does not adequately address TBS Principle 4: Link Resources to Results because of the very broad nature of the PAA's Strategic Outcomes (SOs) and Program Activities (PA). The PAA as approved by TBS does not support the desired linkages between (1) CIHR Mandate, Research Themes, Strategic Initiatives, and Strategic Plan, and (2) the current CIHR PAA including Strategic Outcomes, Strategic Priorities, and Program Activities. SOs are defined at a very high level and are, therefore, general in nature. The high-level nature of SOs and PAs does not facilitate "managing for outcomes." Consequently, reporting to parliament may not be as informative and relevant as it could be because it is not possible to determine whether resources are being used efficiently and effectively with respect to the outcomes and the priorities. |
The Chief Financial Officer should work with TBS to revise the PAA to define strategic outcomes that allow for increased alignment to the CIHR mandate and business activities, in order to facilitate the tracking of progress and provide information to parliament that more clearly demonstrates the achievement of progress. |
Responsibility: Action: CIHR has initiated its Strategic Planning exercise for 2008-2012 (Blueprint II). One of the objectives of this process is to refine the SOs so that they are more specific and can be more easily managed against. Once the strategic direction and the SOs have been confirmed, the PAA will be revised accordingly. Management acknowledges that managing for outcomes is a challenge for all public institutions and particularly a research-based organization like CIHR where all impacts of research are long-term and multi-factorial. Nevertheless, management is of the opinion that CIHR's SOs, while they can be improved, are highly relevant to its mandate and do facilitate meaningful reporting of results. |
September 2008 |
| Minor Issues | |||
| 2. Strategic Planning and Priority Setting | |||
| 2.1 CIHR's business planning is not always completed before the RPP must be produced. The strategic planning and priority setting process, along with the RPP development process, is fully integrated into CIHR's Planning, Budgeting and Reporting Cycle 2007-08. CIHR's strategic plans clearly cascade into a set of operational and tactical plans describing resources required to achieve outcomes. However, RPPs are usually tabled on or before March 31, and submission timelines as dictated by TBS can require their completion and approval before the end of the CIHR business planning process. | The Chief Financial Officer should consider raising the issue of misalignment of RPP timelines as part of the discussion in recommendation 1. |
Responsibility: Action: |
October 2008 |
| 2.2 Risk information could be strengthened by the development of an approved Corporate Risk Profile (CRP) and Risk Management Strategy and Plan. CIHR's summer strategic planning and priority setting retreat of its Governing Council and its operational planning process both consider risk management (at organizational and program levels) and potential consequences for CIHR. A CRP would provide management with a comprehensive list of risks, their causes, existing controls, the consequences if the risks materialize, an objective determination of risk severity, and action plans to target interventions better and enhance the likelihood of achieving outcomes. | The Chief Financial Officer should develop and maintain (an annual refresh) a Corporate Risk Profile and Risk Management Strategy and Plan for CIHR as part of its risk management framework. |
Responsibility: Action: |
April 2008 |
| 3. Oversight and Quality Assurance | |||
| RPP-related files and documentation are not consistently maintained in a central location. | The Chief Financial Officer should ensure that all RPP-related files and documentation are stored centrally. |
Responsibility: Action: |
September 2008 |
Supplemental content (right column)
- Modified: