Example of Tool 2 utilization

A tool-kit for the evaluation of sustainability processes and
sustainability levels of public health programs and projects

Example of TOOL 2 utilization.

This tool was used in different contexts, specifically in Canada and Haiti. The following is an example of its utilization in a health project in Haiti.

Centre for nutritional recuperation (Haiti)

« Terre des hommes » Foundation is a Swiss nongovernmental organization (NGO) that has been working in Haiti since 1989. This NGO is concerned with nutrition and providing direct aid to children. In 1997, the NGO implemented a unit to manage severe acute malnutrition in children. Five years later, after an evaluation conducted by the NGO and local health authorities, the Unit was altered to focus on issues of moderate malnutrition.

In 2003, the NGO was responsible for the Unit, and began to transfer this responsibility to the paediatrics service of the departmental hospital. To aid in this transition the NGO organized activities to increase the sustainability processes of the Unit in the hospital. At the end of 2004, the Unit was functioning within the hospital. Between 2005 and 2007, the NGO’s goal was to pass on sole responsibility of the Unit to the hospital. It is in this context that the NGO decided to perform an initial evaluation of the sustainability level.

Tool 2. Sustainability level of the Nutritional care unit
Concerning the program/project Yes No Indicators (Justify your yes/no answers)
Memory (specific indicators)
Financial resources of the organization No resources supplied by the hospital; very low budgetary allocation from the State to the hospital.
Human resources of the organization No nomination by the State of civil servants required for the functioning.
FMaterial resources of the organization No resources supplied by the hospital.
Other resources of the organization No resources supplied by the hospital.
Adaptation (specific indicators)
Adaptation to the context The management of severely underfed children is the responsibility of a referred hospital.
Adaptation to the effects Tools are adapted to local capacities and to the context for follow-up and evaluation of the effects of the Unit.
Barriers to adaptation Relevance of the Unit’s activities was not challenged.
Values (specific indicators)
Coherence with the organization’s objectives Implicit sharing of objectives but absence of an established plan mentioning the Unit.
Symbols in the organization Presence of a symbol for the Unit but it was the NGO’s logo.
Rituals in the organization No formal meetings organized.
Specific language or jargon Existence of specific technical jargon but limited to a few people with a low degree of familiarity.
Rules (specific indicators)
Nomination of a supervisor from the organization No supervisor assigned to the coordination of all of hospital’s activities.
Inclusion in the organization’s planning No real inclusion in the planning, but in the memorandum of understanding.
Tasks description Very good task description for unit staff.
Written procedures Existence of a practical guide for nutrition monitoring and many therapeutic protocols.

With the collected information presented in the above synthesis-table, it was estimated that the sustainability level of the Unit in the hospital was low (see the five levels table at the How to use Tool 2 section). Official hospital activities are derived from the Unit, but they don’t posses all of the characteristics required to qualify as routinized. They are not routinized because they can drastically change on a short term basis. A large number of activities derived from the NGO’s program are still carried out by the Unit. However, they are far from integrated in the hospital’s functioning.

There were notable achievements in relation to some characteristics of sustainability: adaptation to context, nutrition management procedures, tasks description, follow-up system (monitoring). However, the low level of sustainability can partly be explained by the Unit’s implementation and partly by the lack of activities or events favourable to routinization (see the Example of Tool 1 utilization section). The implementation of the Unit was the responsibility of the NGO. Activities were carried out in the hospital’s facilities but were financed, staffed and supplied by the NGO. In order to insure a higher level of sustainability, the hospital needs to take more organizational risks.

How to use tool 2

A tool-kit for the evaluation of sustainability processes and
sustainability levels of public health programs and projects

 

How to use tool 2

Tool 2 helps you to evaluate the sustainability level of programs/projects. This evaluation is accomplished in two steps: information collection, and information synthesis.

First step: Information collection

In order to identify information corresponding to sustainability levels, you can use one or any combination of the following procedures: a) observation of the program/project, b) analysis of documents related to the program/project, c) individual interviews with key people, and d) group interviews with key people (focus groups). You may collect information from all observations in one “Tool 2” form, and collect information from all documents analyzed in another form, while you may need to complete one form per interview.The 15 questions included in the table below can be used to obtain indicator-related information that addresses the four characteristics of organizational routines (references).

 

Sustainability level: Guide for information collection (e.g. interview guide)
Memory – resources that preserve lessons learned

1 – Are financial resources used to accomplish program/project activities integrated in the organization’s (name the organization) budget?

  • Are these costs part of the budget of the organization (name the organization)?
  • If so, are these costs a permanent or temporary part of this budget?

2 – Who are the people hired by the organization (name the organization) to accomplish these activities?

  • Are these people hired full time or part time?
  • Are these people hired permanently or temporarily?

3 – What are the material resources provided by the organization (name the organization) (rooms, photocopies, medication, etc.)?

  • To which budget do these material resources correspond?

4 – Are there other resources allocated to these activities?

Adaptation and barriers to adaptation

5 – How are these activities adapted to the context of the organization (name the organization)?

6 – Are these activities organized in a way that we can evaluate their effects (e.g. according to follow-up reports, annual statement of accounts or evaluation outcomes)?

7 – Among these activities, which ones are still organized because “we like it” or for historical or any other reason, but for which there is doubt concerning the relevance (barrier to adaptation)?

Values – beliefs and codes

8 – Do these activities correspond to written objectives formalized by the organization (name the organization) (e.g. in the form of a logical frame for “project management”) (Ask to see them).

9 – What are the symbols attached to these activities such as logos (or any other symbol)?

10 – What are the rituals established concerning these activities such as periodic meetings (or any other ritual)?

11 – Did a specific language emerge around these activities, some sort of jargon?

Rules and acting decision

12 – Did the organization (name the organization) formally assign a supervisor for these activities?

13 – Are these activities included in the official planning of the organization (name the organization)?

14 – Do task descriptions exist for the organization (name the organization) workforce assigned to these activities?

15 – Are certain activities related to written rules as in a manual of procedure?

 

Second step: Information synthesis

The second step of the sustainability level evaluation consists of the analysis of collected information. First, make a synthesis of completed “Tool 2” forms. The synthesis-table summarizes all indicators according to the four characteristics of organizational routines (memory, adaptation, values, rules). An example of a synthesis-table is presented in “Example of Tool 2 utilization”.

This synthesis-table allows you to establish a sustainability level. The following five scenarios correspond to the five levels of sustainability.

5 levels of sustainability
High sustainability. Standardized activities stem from the program/project; they possess the four characteristics of organizational routines and are carried out in accordance with public policy.
Moderate sustainability. Routinized activities stem from the program/project; they possess the four characteristics of organizational routines; these activities are durable.
Low sustainability. Official activities stem from the program/project but they don’t possess all characteristics of organizational routines and so are not routinized; these activities can change in a radical way on a short term basis.
Precarious sustainability. Residual activities from the program/project are pursued unformally by members of the organization within a function that has nothing to do with the program/project; the continuation of these activities rely on these people.
No sustainability. No activities stem from the program/project in the organization.

 

You can validate the synthesis-table and its interpretation with people involved in the program/project in group meetings with key people. Finally, you can organize periodic meetings with key people to follow up on the sustainability level.

Tool 2: Sustainability level

A tool-kit for the evaluation of sustainability processes and
sustainability levels of public health programs and projects

TOOL 2. Sustainability level

Step 1 (collection): Complete as many Tool 2 forms as necessary (e.g. one form per interview).
Step 2 (synthesis): Complete one Tool 2 form (synthesis-table) per organization or organization type).

See the How to use Tool 2 section.

Concerning the program/project Yes No Indicators (Justify your yes/no answers)
Memory (specific indicators)
Financial resources of the organization
Human resources of the organization
Material resources of the organization
Other resources of the organization
Adaptation (specific indicators)
Adaptation to the context
Adaptation to the effects
Barriers to adaptation
Values (specific indicators)
Coherence with the organization’s objectives
Symbols in the organization
Rituals in the organization
Specific language or jargon
Rules (specific indicators)
Nomination of a supervisor from the organization
Inclusion in the organization’s planning
Tasks description
Written procedures
Institutional standards (specific indicators)
Accordance with public policy

Printable version of Tool 2