from four grant programs
In addition to unrestricted operating grants, the Department has in recent years begun to target specific initiatives within the advanced education sector by allocating conditional grants to post-secondary education institutions. During 1998-99, the Department issued conditional grants
amounting to approximately $85 million from four funding envelope grant programs. The Department expended approximately $28 million from the Access Fund grant program for the purposes of increasing the number of students accessing Alberta’s post-secondary system.
Another $10 million was spent to support the integration of technology into the sector through the Learning
Enhancement Envelope program and $3.5 million was allocated to assist universities in the recruitment and
retention of quality faculty and graduate students through the Research Excellence Envelope program. A total of
$43 million was granted to institutions to help meet facility
and infrastructure maintenance needs via the Infrastructure Renewal Envelope program.
My staff conducted a review of the effectiveness of the processes followed by the Department to approve, monitor and evaluate the results of the institutions’ spending of conditional grants for each of these four grant programs.
Conditional grant processes Recommendation No. 14
It is recommended that the Department of Learning improve the processes used to collect and verify conditional grant information from the public post-secondary education institutions to facilitate the monitoring and evaluation of each conditional grant program.
The Department should expand the methods used to obtain assurance that the institutions have systems in place to ensure the projects are
cost-effective and the accountability reports are accurate and reliable
As a condition to each grant program, the institutions are required to provide annual accountability reports to the Department. Given the significant dollars involved, the Department should enhance the methods used to obtain assurance that the institutions have processes in place to ensure the projects undertaken are cost-effective. The Department should also expand the processes used to ensure the institutions have systems that compile reliable and accurate information when completing the accountability reports. For example, one method the Department currently follows to evaluate the cost-effectiveness of the projects and the reasonableness of the submitted accountability reports is to conduct annual site visits to the Infrastructure program projects. Periodic site visits to Learning Enhancement projects, especially for projects where significant funding has been granted, should also be conducted. Observations obtained and conclusions reached during site visits should be documented.
Furthermore, the Department could assess the accuracy of submitted financial information by agreeing amounts to the institutions’ audited financial statements or by periodically examining the institutions’ systems used to compile the information. Specifically, the Department could verify that the institutions have processes in place to properly monitor expenditures for individual projects. Moreover, wherever possible, non-financial information within the accountability reports could be corroborated with other information
available within the Department. For example, since Access
funding seeks to expand the number of students, as measured by Full-Load Equivalents (FLEs), who are able to access the post-secondary educational system, institutions are required to report the number of additional FLEs entering the
institutions. This information should be agreed to the data submitted by institutions for KPI reporting purposes to confirm that overall, the number of FLEs has increased at the institution.
In addition to gaining more assurance regarding the accuracy of accountability reports, supplementary disclosure within the reports would facilitate the Department’s evaluation of progress within the sector and identification of where problems may arise. For example, institutions should be required to clearly indicate within the accountability reports where projects have deviated from the institutions’ original intentions or timelines.
The assessment of the overall impact of each grant program should be improved
Finally, the assessment of the overall impact of each grant program should be improved. The Department should evaluate the progress first in terms of achieving the
individual grant program goals, and then on a higher level in terms of achieving the Departmental goals of accessibility, affordability, responsiveness, effectiveness and research excellence. The results of the latter comprehensive analyses should be included in the Department’s annual report as a measure of the sector’s performance towards achieving the Department’s goals.
Comprehensive analyses of the achievement of individual grant program goals and overall
Departmental goals have not been prepared for each of the grant programs
While the Department has recently prepared high level analyses for the Access Fund and Research Excellence grant programs, comprehensive analyses on both levels described above have not been prepared for each of the grant programs.
For example, since a main goal of the Access Fund is to increase the number of FLEs at the institutions, the
Department has determined and reported how many new FLE
placements have been created. However, the sector’s progress towards achieving a second goal of the Access Fund, which is to encourage innovation in creating learning opportunities, has not been measured and evaluated. To facilitate the preparation of such analyses, it is essential that the institutions submit to the Department all relevant data supporting the criteria used to evaluate the success of the projects. Additionally, certain grant program analyses could be made more meaningful by relating outcomes of the grant program to performance measures in the Department’s
annual report. For example, research excellence is indicated in the Department’s annual report by the average amount of sponsored research funding per faculty member received by each institution. However, these averages are only shown for the current year. Increasing the disclosure to include comparative figures would enable readers of the
Department’s annual report to appreciate the impact of the program over time.
Contract Management It is recommended that the Department of Learning ensure that contract manager verification and follow-up procedures as established in the Department’s contract procedures manual are satisfied before payments are made to contract service providers.
The Department has developed a
comprehensive contract procedures manual
During 1997-98, the Department developed a contract procedures manual which includes detailed policies regarding contract tendering and negotiating procedures, managing on-going contracts and post-payment verification and monitoring. In my 1997-98 annual report (page 47), I stated I would continue to monitor the progress of the Department in implementing the new contract procedures manual. This year, my examination focussed on the latter stages of post-payment verification and monitoring of results.
The manual requires both contractors and
departmental contract managers to follow-up on the results obtained from contract services
Contractor payments are based on services delivered and results obtained. An example of a result to be reported to the Department by contractors is the employment status of the clients served by labour market development programs. The manual therefore establishes two phases in the monitoring process to address the risk that payments may be made to contractors when the contract requirements have not been fulfilled. The first is a follow-up of the client to be
performed and documented by the contractor and the second is a follow-up of the contractor and selected clients,
performed by the Department contract manager.
Documentation of both follow-up procedures and invoice verification is not being prepared
consistently by the Department’s contract managers
My staff found that documentation of the follow-up by the Department’s contract project managers is not being prepared consistently as required by the manual. The Department also conducted an internal review of the administration of program contracts. In summary, it was noted by the Department that a number of the contract files lacked sufficient documentation regarding contract manager reviews of the contractor’s client files, on-site visits by the contract manager and communications made with the clients
or their employers. Although adequate documentation was not kept in these files, the Department maintains that in some cases follow-up work was completed. The Department also found that contract managers were not consistently
documenting their verification of the appropriateness of invoice amounts.
Without documented follow-up information, the Department cannot substantiate whether contractors have fulfilled their obligations as stated in the contracts
Without documented follow-up information, the Department cannot substantiate whether contractors have fulfilled their obligations as stated in the contracts. As a result, there is a risk that contractors are receiving payment for services not fully rendered. Additionally, obtaining sufficient follow-up documentation is essential to the evaluation of Departmental programs such as for Labour Market Development
initiatives.