Guideline Twelve | Evaluate

12. Think about, plan and use evaluation effectively from early on.

Evaluation can serve a number of purposes and result in very important benefits for projects. Increasingly, sponsoring agencies (e.g. the ALTC) have evaluation requirements as part of the conditions of funding for some projects.
Traditionally the two principal purposes of project evaluation have been formative and summative, with the greatest emphasis being on summative. Both of these purposes, along with others, are explored in the paper by Boyle & Griffiths (2010) (see below).
Good evaluation hinges on having clear and appropriate Evaluation Questions (EQs). They focus evaluation work and in addition help to keep Project Leaders’ and teams’ eyes on what’s important in a project. EQs will vary with the evaluation purpose. Those used for formative evaluation need to focus on matters that will enable improvement of aspects of a project in progress (e.g. communication), whereas questions to guide summative evaluation are usually broader in focus and concern dimensions of merit for the project overall (e.g. how well the project objectives were achieved).  More details on EQs are provided in the paper cited above and the ALTC resource Evaluating Projects (Chesterton & Cummings (2007). Link

The best piece of general advice we can give on evaluation is to view it in an integrative way. To get the most from evaluation for a project, see and use it as:

(a)    an integral strand of project planning and implementation (operations); and (b)    processes that facilitate project success, and not simply as something done towards the end to report on achievements (e.g. provide data or findings)
Related to this, is the idea that evaluation activity should be happening from early in a project and throughout its life. This doesn’t mean spending a lot of time on it, but there are important things that can be done early that facilitate better evaluation and project implementation. Examples include: asking key stakeholders to suggest their project success criteria; setting up ways to monitor the project’s critical success factors; developing understandings (and/or a plan) with stakeholders to enable them to provide feedback on important stage-based project outcomes; and thinking ahead about possible key evaluation questions for summative evaluation.

Supplementary Comments
The historical norm for evaluation of funded projects in Australia, whether in universities or elsewhere, has been to have quite limited expectations. Typically, some kind of end-of-project summative evaluation might be required. This is changing, with a trend towards increased expectations that projects have a clear evaluation strategy as an integral part of the project design. More funding agencies are insisting on explicit information about evaluation as part of project proposals, and evaluation is being expected to serve at least the formative purpose, in addition to the summative. An increasingly common requirement is provisions for systematic stakeholder engagement with evaluation during the project (e.g. to get their feedback and judgments; and to keep them involved in decision making about the project).

Some Guidance for Systematic Project Evaluation Patrick Boyle & Mark Griffiths.pdf225.6 KB