JCPSG | TRAC Guidance | Part II TRAC Principles

Section C: Quality assurance and validation

1. TRAC processes are designed to be robust, and to produce quality, fit for purpose, cost data. These include techniques for:
   • verifying time allocation data – see Part III Chapter C.1;
   • attributing estates costs robustly – see Part III Chapter D.5;
   • attributing library costs robustly – see Part III Chapter D.4;
   • using appropriate cost drivers in other areas – see Part III Chapter D.1;
   • avoiding double-charging – for example for TRAC fEC see Annex 18 (for equipment) and Part V Chapter C.4 (for academic salaries);
   • providing audit trails by which institutions can demonstrate the validity of their cost calculations and of their charges to projects – for example, see Part V Section F.

2. Quality assurance processes are used to confirm that an institution is using methods that meet TRAC requirements. They include processes to validate the figures produced. There are five types of quality assurance and validation process:
   • Internal review
   • Institutional committee confirming compliance
   • Internal audit
   • benchmarking
   • Sponsors' QA procedures

3. Costing standard 7 covered two of these quality assurance procedures:
   • a review by the internal audit service to advise the institution on their compliance with the TRAC costing standards and the full set of minimum requirements;
   • an appropriate Committee of the Board, or equivalent, to confirm this compliance.

4. Costing standard 8 allows institutions whose research activity is not material to apply dispensation if they wish.
5. TRAC fEC introduced a new requirement for institutions to participate in an external quality assurance process, and to provide benchmarking data.

Chapter C.1 Internal review

1. Institutions should understand the data and any unexpected or outlying results. Tests for reasonableness should be carried out. The results of these should be presented to the Finance Committee or appropriate Committee of the Board, or equivalent, to give assurance that the outcome is fair and reasonable.
2. Examples of these comparisons and tests are:
   • study the results – do they match the records, observations and expectations of heads of department?
   • compare with results from previous comparable periods – are the data consistent?
   • match with income – are the results understandable, explainable and fair and reasonable?
   • benchmark – compare with data from other (peer) institutions; or with sector data;
   • plot results – prepare a graph of time or costs of each activity by department against student numbers, and against income;
   • compare with information produced by external bodies – calculate an average pay cost per FTE student and compare with relativities for each subject type with those used in the relevant Funding Council’s funding methodologies for Teaching;
   • carry out a small number of surveys or interviews to test the validity of results;
   • carry out a sensitivity analysis on key cost drivers and cost pools – test equally appropriate but different cost drivers or weightings in the cost drivers. (An example here might be to use weighted cost drivers in the estates cost pool which reflect actual, or standard, costs of each building or campus);
   • convene a small group to discuss the profile of academic activity in a year – they could consider what activity took place during the various periods (Easter, term, vacation etc) or for different purposes (library services used by students; estates used by PGR students).

3. ‘Outliers’ can be investigated through surveys, interviews or workshops. It is important to give heads of department, the TRAC implementation project group, and the senior management team an opportunity to understand and comment on the figures. When they do so, their comparability with other figures from the management accounts, the resource allocation model and/or the strategic planning model, needs to be understood.
4. The results of these tests and checks should be given to the relevant institutional committee to provide assurance that the outcome is fair and reasonable.

Chapter C.2 Institutional committee confirming compliance

1. An appropriate Committee of the Board (or equivalent) should confirm compliance with TRAC requirements. Compliance is the responsibility of institutional managers and institutions would generally wish to involve their Finance Committee in ensuring that this is achieved. The Audit Committee can, independently, on the advice of the internal audit service, confirm this compliance.
2. The Audit Committee should oversee the programme of internal audit and should receive reports from the internal auditor. The Audit Committee should report to the Governing Body that it has done this and whether it is satisfied on the extent of the compliance with the TRAC requirements. The Audit Committee may advise the Finance Committee (or other appropriate committee).
3. TRAC reporting to the relevant Funding Council is made by the head of institution as accounting officer. He/she would satisfy him/herself that the institution has complied with the TRAC requirements, reassured by advice from internal audit. Depending on the committee structure and governance relationships, the Finance Committee or other appropriate committee should receive a report on the compliance and maintain a strategic overview of the development of costing and other financial management initiatives in the institution.
4. To give the Committee assurance that the outcomes are ‘fair and reasonable’ it should also be provided with information on:
   • key assumptions, with a rationale that indicates that their use is fair and reasonable;
   • supporting evidence that shows that the cost drivers for the larger items of cost represent the use of or benefits from those resources;
   • the results of tests for reasonableness and other checks on the time allocation and cost attribution figures;
   • the results from the time allocation method.
5. Management should present these to the Finance or other appropriate committee along with the information to be reported under TRAC. A reconciliation with the audited financial statements should be included as part of this.
6. It is worth reiterating a key principle of TRAC requirements. Institutions are free to use alternative methods to those suggested in the costing standards, and discussed in the Guidance. However, to do this, they must be able to demonstrate that the information reported is at least as robust as that under the methods suggested in the Guidance.
7. The Audit Committee (informed by the internal audit service) should satisfy itself that this is the case.

Chapter C.3 Internal audit

1. Internal audit can provide assurance on the systems used for ensuring compliance with TRAC standards and minimum requirements, as set out in this Guidance. A review to assess the extent of this compliance should be built into their audit programme, which will be agreed by the Audit Committee. It should be undertaken periodically.
2. It is not a TRAC requirement that external auditors scrutinise or audit the calculations or methods used.
3. The internal audit programme should help to:
   • ascertain that accounting and other information is reliable as a basis for producing accounts, and financial, statistical and other returns;
   • ascertain the integrity and reliability of financial and other information provided to management, including that used in decision-making.
4. The internal audit review should cover all TRAC systems, and in particular the annual TRAC process, and TRAC fEC. Any additional information included on a TRAC report (e.g. income allocation on the annual TRAC return) should also be covered.
5. The internal audit review could be planned around the TRAC minimum requirements:

   principles and conventions

   annual TRAC

   income allocation

   indirect cost rate and estates charges

   TRAC fEC

6. A full systems audit should be carried out every three years (the end of the second TRAC cycle relates to 2007/08 data and should be carried out before 31 January 2009). A full systems audit covers a review of the controls in systems designed to meet TRAC minimum requirements.
7. Following assurance that new TRAC systems have been robustly introduced, or previous systems are operating effectively, the reviews should then be planned on an on-going basis following institutions’ normal risk analysis. This should be planned in the knowledge that a considerable amount of implementation of TRAC fEC will still be taking place up to 2007 – and that by 1 August 2007 full robustness should have been achieved. However, full implementation of TRAC fEC will only be possible by December 2009.
8. The work to be carried out should depend on the institution’s view of:
   • the risks;
   • the controls and checks needed and those that are in place;
   • the tests of these controls and checks that are necessary to assess their effectiveness;
   • whether the methods being used are ‘tried and tested’ (e.g. described in the Guidance and in wide use in the sector) or developed and used by only a small number of HEIs;
   • the quality of TRAC project management during the development and implementation stages;
   • other objectives that the institution is seeking to satisfy, which rely on TRAC data or systems, e.g. pricing decisions and other internal decision-making.
9. The particular risks to be considered under TRAC would include:
   • the risk of not reporting (perception of a poorly managed institution that is unable to provide information);
   • reporting results that are not robust (not in compliance with the costing standards) – and asking the head of the institution to underwrite these results;
   • the inability to provide information that would be considered to be sufficiently robust by external sponsors (loss of stakeholder confidence; potentially lower recovery rates on work);
   • using inadequate or misleading information for internal decision making;
   • implementation of an overzealous or inappropriate approach to implementation (including towards internal audit work in this area) – diverting institutional time from other priorities.
10. The TRAC internal audit plan should be designed in a way that respects the principles which underpin the design of the TRAC requirements. These principles require the TRAC process to be minimally burdensome and as helpful to institutions as possible. Institutions are allowed flexibility in the systems and methods they use, and can improve and adapt the guidance within the overall standards and requirements set out.
11. The methods should focus attention on important costs and do not require repeated measurement of factors which do not change year-to-year, or undue precision over small cost elements. Materiality is an important principle, along with recognition that there will always be elements of judgement in cost allocation.
12. Audit does not require precise judgement in all areas. Several areas in costing involve judgement and discretion. Perhaps the most obvious of these is the allocation of staff effort, where even the most costly methods are not easily auditable in the sense of being externally verifiable.
13. Modern approaches to audit relate to a judgement based on risk, materiality, and the appropriateness of the methods used, in other words, a normal systems-based audit. This can be interpreted to imply that the costs produced under TRAC should fairly represent the full economic costs of the activity. The principles to be adopted here are:
    • focusing particular effort on the most significant costs;
    • ensuring that the method is defensible and is signed by the head of institution as showing results that are commensurate with the real cost of the activities reported.
14. The number of days required for these reviews will depend on several factors:
    • the risks identified;
    • the types of system being introduced;
    • the approach and level of assurance identified for the reviews;
    • the quality of the systems and project management arrangements found to be in place.
15. The internal audit approach to TRAC audit is expected to rely largely upon management assurances, supported by limited testing, as opposed to carrying out detailed compliance testing.
16. The internal audit process is in addition to external QA, and sponsors’ QA processes.

Chapter C.4 External QA process

1. The methods used by institutions to calculate the indirect cost rates, and estates charges for Research, are to be periodically reviewed by an external quality assurance team. Approval of institutions’ methods may be conditional upon the institution addressing any action points raised by that team. All institutions should be involved in this QA process.
2. In years one and two of implementation of TRAC fEC, this process was designed to focus particularly on aspects of the methods that materially affect indirect cost rate calculations or other aspects of project costing. The process resulted in the production of constructive advice to institutions covering any changes required.
3. The external QA process carried out in 2004/5 is described in HEFCE circular letter 05/2004 – accessible from Annex 1. It comprised:
   • the completion of a 'self-assessment checklist' by institutions
   • their participation in three benchmarking exercises
   • a visit by HEFCE’s internal audit team, which resulted in the identification of any ‘significant issues’ that could be materially affecting the robustness of the estates charge or the indirect cost rate calculations, as well as other developmental issues
   • resolution of any significant issues, followed by a review by the institutions’ internal auditors and confirmation by an appropriate committee to the QA team, that this had been done.
4. The QA process had to take place before an institution’s indirect cost rate or estate charges could be accepted as a basis for including indirect or estates costs on Research projects for cost-based pricing.
5. Until an institution is able to satisfy the QA team about any identified outstanding areas of robustness, then a non-compliant default indirect cost rate (determined by the OST/Research Councils) has to be used as the basis for cost-based pricing (if this is lower than the institution’s own rate). If the lack of robustness affects the estates charge, then no estates charge can be applied (the non-compliant default rate on estates is zero). However, when determining the total cost (fEC) on each of their projects, the institution’s own indirect cost rates are still to be applied, not the non-compliant default rate. For Research Council and OGD projects, the price recovered on these projects is likely to be significantly less than a price calculated on the fEC.

Benchmarking

6. The external QA process includes benchmarking. Institutions should calculate the following for benchmarking purposes:
   1. a single £/academic and research staff FTE indirect cost rate for Research;
   2. two £/academic and research staff FTE estates charges for Research: for both laboratory and non-laboratory (with clinical departments included in the laboratory group);
   3. a cost per square metre of each of the four (or more) categories of space (optional before August 2007).

   (a) and (b) do not mean that institutions have to use these FTE rates to charge estates costs to Research projects – alternative methods could be used.

7. Other information should be provided for the QA process, similar to the benchmarking carried out in the sector in 2004 (for example, including income and costs by type of activity, and an analysis of academic staff time and of total Support costs).
8. If an institution’s indirect rate or estates charges fall above the upper quartile (UQ) of the sector rates, then they should be prepared to justify these higher rates, or should use the non-compliant default rate.
9. Justification can be provided:
   1. by the rate being below the sector UQ issued from the most recent benchmarking exercise;

      or

   2. by the institution providing assurance that their processes are robust i.e.
      1. all other significant issues raised by the QA team have been addressed; and
      2. they have:
         • carried out a sensitivity analysis on their cost allocation model to identify possible reasons for the differential from benchmarking data, and have reviewed the appropriateness of this;
         • understood why their rates are above the UQ (e.g. very low direct time, or high Support time; or generally acknowledged through other exercises that they are a higher or lower cost institution); and
      3. by the institution’s head of Audit or Finance Committee signing a statement to the effect that their rates are above the sector UQ; that they understand that because they are being funded from public funds it is important that they are satisfied that it is a true cost; that, having considered other information (e.g. the nature of the estates, academic workload/productivity, and EMS statistics – see Annex 1 on the efficiency of the estate) they are satisfied that these are real costs incurred on Research and are a legitimate basis for funding through the public purse.

Chapter C.5 Sponsors’ QA processes

1. Sponsors rely on the TRAC quality assurance procedures, and sometimes augment this with their own quality assurance procedures.
2. The Research Councils operate a programme of dipstick testing – refer to their website accessible at Annex 1.
3. OGDs and charities generally rely on TRAC and Research Council procedures to provide the required reassurance.
4. The EU does not rely on TRAC procedures or systems. They operate various types of procedure on EU-funded Research, ESF and ERDF projects – see references to information provided at Annex 1.

Chapter C.6 Dispensation

1. Institutions should always have regard to materiality. Some institutions’ volume of Research activity is not of a level that, taken with the sector as a whole, requires the same degree of verification and validation for either the annual TRAC process or TRAC fEC. Dispensation allows for this.
2. Institutions with a Research income of less than £500k have dispensation on the testing and validation requirements. This means that although they should report costs, and meet TRAC requirements, they do not need to introduce a robust method of time allocation, nor a fully robust set of cost drivers. However, they may wish to use these for their own reasons.
3. Research income is defined for this purpose as total income for publicly funded research activity (Funding Councils, Research Councils, OGDs), calculated as an average over the last five years.
4. Institutions may wish to consider whether they actually want to take up dispensation, as it is likely that they will wish to use robust costing information internally. Funding Councils are also interested in the application of TRAC to costing Teaching. Although dispensations are currently allowed, this is likely to change as HEFCE develops a new funding method for Teaching (which is to be informed by TRAC).
5. A dispensation does not mean exemption from annual TRAC reporting (Part III of the Guidance), or from the requirement to bid for research projects on an fEC basis (Part V). TRAC requirements still apply to these institutions. However, they do not have to use fully robust methods. This means that they do not have to:
   • obtain time allocation data from academics (heads of department could, for example, provide this information);
   • use more than four to six cost drivers as they allocate indirect costs;
   • identify space use across the whole institution (as opposed to just that used for Research);
   • commission a full systems audit;
   • calculate an indirect cost rate or estates charges;
   • meet any of the requirements required under TRAC fEC for 1 August 2007, specifically:
     • a robust calculation of FTEs
     • taking into account the type of space when allocating space costs
     • directly allocating laboratory technicians and research facility costs, separately from estates charges.
6. Under TRAC fEC, institutions that are applying their right to dispensation do not have fully robust systems, and therefore:
   • should only apply the lower of their own indirect cost rate; or a dispensation default indirect cost rate, to research projects;
   • should only apply the lower of their own estates charge; or a dispensation default sector estates charge, to research projects.
7. The dispensation default rates are set by the OST/Research Councils.
8. The situation of institutions eligible for dispensation is therefore different from that of other institutions with higher levels of Research activity. More Research-intensive institutions without TRAC-compliant systems can only apply a non-compliant default rate for indirect costs. The non-compliant default estates charge for these institutions is zero.