A tool-kit for the evaluation of sustainability processes and
sustainability levels of public health programs and projects

 

How to use tool 2

Tool 2 helps you to evaluate the sustainability level of programs/projects. This evaluation is accomplished in two steps: information collection, and information synthesis.

First step: Information collection

In order to identify information corresponding to sustainability levels, you can use one or any combination of the following procedures: a) observation of the program/project, b) analysis of documents related to the program/project, c) individual interviews with key people, and d) group interviews with key people (focus groups). You may collect information from all observations in one “Tool 2” form, and collect information from all documents analyzed in another form, while you may need to complete one form per interview.The 15 questions included in the table below can be used to obtain indicator-related information that addresses the four characteristics of organizational routines (references).

 

Sustainability level: Guide for information collection (e.g. interview guide)
Memory – resources that preserve lessons learned

1 – Are financial resources used to accomplish program/project activities integrated in the organization’s (name the organization) budget?

  • Are these costs part of the budget of the organization (name the organization)?
  • If so, are these costs a permanent or temporary part of this budget?

2 – Who are the people hired by the organization (name the organization) to accomplish these activities?

  • Are these people hired full time or part time?
  • Are these people hired permanently or temporarily?

3 – What are the material resources provided by the organization (name the organization) (rooms, photocopies, medication, etc.)?

  • To which budget do these material resources correspond?

4 – Are there other resources allocated to these activities?

Adaptation and barriers to adaptation

5 – How are these activities adapted to the context of the organization (name the organization)?

6 – Are these activities organized in a way that we can evaluate their effects (e.g. according to follow-up reports, annual statement of accounts or evaluation outcomes)?

7 – Among these activities, which ones are still organized because “we like it” or for historical or any other reason, but for which there is doubt concerning the relevance (barrier to adaptation)?

Values – beliefs and codes

8 – Do these activities correspond to written objectives formalized by the organization (name the organization) (e.g. in the form of a logical frame for “project management”) (Ask to see them).

9 – What are the symbols attached to these activities such as logos (or any other symbol)?

10 – What are the rituals established concerning these activities such as periodic meetings (or any other ritual)?

11 – Did a specific language emerge around these activities, some sort of jargon?

Rules and acting decision

12 – Did the organization (name the organization) formally assign a supervisor for these activities?

13 – Are these activities included in the official planning of the organization (name the organization)?

14 – Do task descriptions exist for the organization (name the organization) workforce assigned to these activities?

15 – Are certain activities related to written rules as in a manual of procedure?

 

Second step: Information synthesis

The second step of the sustainability level evaluation consists of the analysis of collected information. First, make a synthesis of completed “Tool 2” forms. The synthesis-table summarizes all indicators according to the four characteristics of organizational routines (memory, adaptation, values, rules). An example of a synthesis-table is presented in “Example of Tool 2 utilization”.

This synthesis-table allows you to establish a sustainability level. The following five scenarios correspond to the five levels of sustainability.

5 levels of sustainability
High sustainability. Standardized activities stem from the program/project; they possess the four characteristics of organizational routines and are carried out in accordance with public policy.
Moderate sustainability. Routinized activities stem from the program/project; they possess the four characteristics of organizational routines; these activities are durable.
Low sustainability. Official activities stem from the program/project but they don’t possess all characteristics of organizational routines and so are not routinized; these activities can change in a radical way on a short term basis.
Precarious sustainability. Residual activities from the program/project are pursued unformally by members of the organization within a function that has nothing to do with the program/project; the continuation of these activities rely on these people.
No sustainability. No activities stem from the program/project in the organization.

 

You can validate the synthesis-table and its interpretation with people involved in the program/project in group meetings with key people. Finally, you can organize periodic meetings with key people to follow up on the sustainability level.