Example of Tool 1 utilization

A tool-kit for the evaluation of sustainability processes and
sustainability levels of public health programs and projects

Example of TOOL 1 utilization.

This tool was used in different contexts, specifically in Canada and Haiti. The following is an example of its utilization in a health project in Haiti.

Centre for nutritional recuperation (Haiti)

« Terre des hommes » Foundation is a Swiss nongovernmental organization (NGO) that has been working in Haiti since 1989. This NGO is concerned with nutrition and providing direct aid to children. In 1997, the NGO implemented a unit to manage severe acute malnutrition in children. Five years later, after an evaluation conducted by the NGO and local health authorities, the Unit was altered to focus on issues of moderate malnutrition.

In 2003, the NGO was responsible for the Unit, and began to transfer this responsibility to the paediatrics service of the departmental hospital. To aid in this transition the NGO organized activities to increase the sustainability processes of the Unit in the hospital. At the end of 2004, the Unit was functioning within the hospital. Between 2005 and 2007, the NGO’s goal was to pass on sole responsibility of the Unit to the hospital. It is in this context that the NGO decided to perform an initial evaluation of the sustainability processes started in 2003. This evaluation was undertaken to promote necessary adjustment for the Unit’s sustainability.

Concerning the program/project Events (e.g. actions taken) Favourable Unfavourable
Specific sustainability events
Stabilization of organizational resources
  • The actual capabilities of the State and its failure to keep its promises during a project in 1999 are not taken into account.
  • Unlike what was planned, the NGO still pays the Unit salaries instead of giving the hospital a grant before official agents are nominated.

 

Organizational risk-taking
  • Two entities function separately in only one building.
  • Staff reduction in the Unit and asking parents to compensate.

Joint sustainability and implementation events
Incentives or benefits for people involved
Adaptation of the activities  Good reputation of the NGO concerning the quality of nutrition projects via past activities in this area of the country.
Objectives fit with those of the organization  Signing of a memorandum of understanding.
Transparent communication between people involved
  • Consultation for the writing of the protocol.
  • Setting up of a committee (board).
  • Lack of follow-up with the newly assigned Minister of Health.
  • Not enough meetings with all the hospital staff.

 

 

 

 

Sharing of culture and artefacts with the organization
Integration of rules into those of the organization  Hospital routines not taken into account (staff turnover).
Specific implementation events
Investment of adequate resources to complete the activities
  • Construction of a building; material supplies.
  • Training and practical training of nurses in two entities (NGO and hospital).
  • No practical training for hospital support staff.

 

 

 

Technical or practical compatibility of the activities with those of the organization  Addition of a joint building to the hospital with its staff.

 

With the information presented in the above synthesis-table, it was estimated that the sustainability process was moderate to low. In fact, more unfavourable events than favourable ones were mentioned during the evaluation process. The synthesis-table also shows that no “specific sustainability events” identified as favourable were mentioned.

It was also important to know the perspective of key people in relation to what immediate actions may be taken to promote the sustainability process of the Unit. The stabilization of resources and the operational imbrications of the Unit and the hospital were mentioned. The importance of collaboration between members of the Unit and the hospital was also stressed. This type of integration could be accomplished by sharing staff and material resources with greater transparency and confidence possible. Also, the NGO provided significantly more human and material resources to the Unit in comparison to the hospital. In order to make the Unit sustainable, the hospital needs to take more risks.

How to use tool 1

A tool-kit for the evaluation of sustainability processes and
sustainability levels of public health programs and projects

 

How to use tool 1

Tool 1 helps you to evaluate the sustainability process of programs/projects. This evaluation is accomplished in two steps: collection, and information synthesis.

First step: Information collection

Information collection takes place in two phases.

1- Creation of an events and activities log

First, all of the program/project’s activities (e.g. a weekly specific physical exercise in school) must be listed as well as events (e.g. the implementation of these specific physical exercises) that happened in relation to the program/project. To identify these activities and events, you can use one or any combination of the following procedures: a) observation of the program/project, b) analysis of documents related to the program/project, c) individual interviews with key people, and d) group interviews with key people (focus groups). You may collect information from all observations in one Tool 1 form, and collect information from all documents analyzed in another form, while you may need to complete one form per interview.

The objective is to create an events and activities log for the program/project. It is possible to update this log at any time by documenting changes in activities as well as new events.

2 – Classification of program/project-linked events

The next step is to classify events that are related to the program/project by reviewing the log with the help of Tool 1. You can classify events a) by yourself, and/or b) by interviewing key people separately, and/or c) by interviewing key people in focus groups. These events should be classified as either favourable or unfavourable to the sustainability of the program/project.

In an interview you can ask the following question in relation to each event listed in the log:
–  Was this event favourable or unfavourable to the sustainability of the program/project (name the program/project)?

People often need prompting to describe the important aspects of these events. The nine questions included in the table below can help to illicit the desired information. The information gained from this process can be used to fill out the “Tool 1” form.

Sustainability process: Guide for information collection (e.g. interview guide)
  1. Were actions taken to stabilize organizational resources for program/project activities (financial, human, material, training)?
  2. Were organizational risks taken in favour of the program/project?
  3. Were the people involved encouraged to carry out program/project activities?
  4. Were program/project activities adapted to local context according to their effectiveness and to population needs?
  5. Were program/project activities designed with the goals of the organization in mind (name the organization)?
  6. Was there transparent communication between the people involved (enumerate people)?
  7. Was there a cultural sharing between the organization (name the organization) and the program/project (artefacts, myths, symbols, metaphors, rituals, jargon, values)?
  8. Were the rules related to the organization’s (name the organization) activities integrated to those of the program/project, or vice versa?
  9. Were the resources invested by the organization (name the organization) adequate to accomplish program/project activities (financial, human, material, training)?
  10. Were practices and techniques related to program/project activities compatible with those of the organization (name the organization)?

 

Additional questions can be used to gain a deeper understanding of events. For each event, try to answer the following questions:

  • What led to this event?
  • How did this happen?
  • Why?
  • With whom?
  • What was particularly positive or negative in relation with program/project activities?
  • How did the concerned parties feel?
  • How did concerned parties react?
  • What strategies were used?
  • What were the effects?
  • What happened next or what were the perceived consequences?

Second step: Information synthesis

The second step of the sustainability process evaluation consists of the analysis of collected information. First, a synthesis-table including all completed “Tool 1” forms must be created. For example, six individual interviews were conducted during the evaluation process, and six “Tool 1” forms were completed. These forms must then be synthesized into one form. An example of a synthesis-table is presented in “Example of Tool 1 utilization”.

The objective of the creation of a synthesis-table is twofold: a) to compare the amount of favourable and unfavourable events associated with the program/project, and b) to identify what types of actions may be taken to make the program/project (more) sustainable.

When examining a synthesis-table, three scenarios are possible: 1) more favourable events indicates a strong sustainability process, 2) an equal amount of favourable and unfavourable events indicates a moderate sustainability process, and 3) more unfavourable events indicates a low sustainability process.

You can validate the synthesis-table and its interpretation with people involved in the program/project in group meetings with key people for example. Meetings can also allow you to identify future events that would be favourable to the sustainability process of the program/project by asking the question:
– What actions may be taken in the next few weeks/months to make the program/project (more) sustainable?

Finally, you can periodically update the synthesis-table by integrating changes from the events and activities log for the program/project. Updating the synthesis-table will allow you to adjust the actions that may be taken.

Tool 1: Sustainability process

A tool-kit for the evaluation of sustainability processes and
sustainability levels of public health programs and projects

TOOL 1: Sustainability process

 

Step 1 (collection): Complete as many Tool 1 forms as necessary (e.g. one form per interview)Step 2 (synthesis): Complete one Tool 1 form (synthesis-table) per organization or organization type). See the How to use Tool 1 section.

 

Concerning the program/project Events (e.g. actions taken) Favourable Unfavourable
Specific sustainability events
Stabilization of organizational resources
Organizational risk-taking
Joint sustainability and implementation events
Incentives or benefits for people involved
Adaptation of the activities
Objectives fit with those of the organization
Transparent communication between people involved
Sharing of culture and artefacts with the organization
Integration of rules into those of the organization
Specific implementation events
Investment of adequate resources to complete the activities
Technical or practical compatibility of the activities with those of the organization

 

Printable version of Tool 1