How to use tool 1

Feb 23, 2017 | Uncategorized

A tool-kit for the evaluation of sustainability processes and
sustainability levels of public health programs and projects

 

How to use tool 1

Tool 1 helps you to evaluate the sustainability process of programs/projects. This evaluation is accomplished in two steps: collection, and information synthesis.

First step: Information collection

Information collection takes place in two phases.

1- Creation of an events and activities log

First, all of the program/project’s activities (e.g. a weekly specific physical exercise in school) must be listed as well as events (e.g. the implementation of these specific physical exercises) that happened in relation to the program/project. To identify these activities and events, you can use one or any combination of the following procedures: a) observation of the program/project, b) analysis of documents related to the program/project, c) individual interviews with key people, and d) group interviews with key people (focus groups). You may collect information from all observations in one Tool 1 form, and collect information from all documents analyzed in another form, while you may need to complete one form per interview.

The objective is to create an events and activities log for the program/project. It is possible to update this log at any time by documenting changes in activities as well as new events.

2 – Classification of program/project-linked events

The next step is to classify events that are related to the program/project by reviewing the log with the help of Tool 1. You can classify events a) by yourself, and/or b) by interviewing key people separately, and/or c) by interviewing key people in focus groups. These events should be classified as either favourable or unfavourable to the sustainability of the program/project.

In an interview you can ask the following question in relation to each event listed in the log:
–  Was this event favourable or unfavourable to the sustainability of the program/project (name the program/project)?

People often need prompting to describe the important aspects of these events. The nine questions included in the table below can help to illicit the desired information. The information gained from this process can be used to fill out the “Tool 1” form.

Sustainability process: Guide for information collection (e.g. interview guide)
  1. Were actions taken to stabilize organizational resources for program/project activities (financial, human, material, training)?
  2. Were organizational risks taken in favour of the program/project?
  3. Were the people involved encouraged to carry out program/project activities?
  4. Were program/project activities adapted to local context according to their effectiveness and to population needs?
  5. Were program/project activities designed with the goals of the organization in mind (name the organization)?
  6. Was there transparent communication between the people involved (enumerate people)?
  7. Was there a cultural sharing between the organization (name the organization) and the program/project (artefacts, myths, symbols, metaphors, rituals, jargon, values)?
  8. Were the rules related to the organization’s (name the organization) activities integrated to those of the program/project, or vice versa?
  9. Were the resources invested by the organization (name the organization) adequate to accomplish program/project activities (financial, human, material, training)?
  10. Were practices and techniques related to program/project activities compatible with those of the organization (name the organization)?

 

Additional questions can be used to gain a deeper understanding of events. For each event, try to answer the following questions:

  • What led to this event?
  • How did this happen?
  • Why?
  • With whom?
  • What was particularly positive or negative in relation with program/project activities?
  • How did the concerned parties feel?
  • How did concerned parties react?
  • What strategies were used?
  • What were the effects?
  • What happened next or what were the perceived consequences?

Second step: Information synthesis

The second step of the sustainability process evaluation consists of the analysis of collected information. First, a synthesis-table including all completed “Tool 1” forms must be created. For example, six individual interviews were conducted during the evaluation process, and six “Tool 1” forms were completed. These forms must then be synthesized into one form. An example of a synthesis-table is presented in “Example of Tool 1 utilization”.

The objective of the creation of a synthesis-table is twofold: a) to compare the amount of favourable and unfavourable events associated with the program/project, and b) to identify what types of actions may be taken to make the program/project (more) sustainable.

When examining a synthesis-table, three scenarios are possible: 1) more favourable events indicates a strong sustainability process, 2) an equal amount of favourable and unfavourable events indicates a moderate sustainability process, and 3) more unfavourable events indicates a low sustainability process.

You can validate the synthesis-table and its interpretation with people involved in the program/project in group meetings with key people for example. Meetings can also allow you to identify future events that would be favourable to the sustainability process of the program/project by asking the question:
– What actions may be taken in the next few weeks/months to make the program/project (more) sustainable?

Finally, you can periodically update the synthesis-table by integrating changes from the events and activities log for the program/project. Updating the synthesis-table will allow you to adjust the actions that may be taken.

Logo_REALISME_201505

Main Toolkit Page

Tool 1
How to use tool 1
Example of Tool 1 utilization

Tool 2
How to use tool 2
Example of Tool 2 utilization

Glossary

References

Your experience interests us!

You can complete the attached printable version and send it back to us to share your experience.

Fax: 514-398-8483
E-mail:

How to use tool 2

Feb 23, 2017 | Uncategorized

A tool-kit for the evaluation of sustainability processes and
sustainability levels of public health programs and projects

 

How to use tool 2

Tool 2 helps you to evaluate the sustainability level of programs/projects. This evaluation is accomplished in two steps: information collection, and information synthesis.

First step: Information collection

In order to identify information corresponding to sustainability levels, you can use one or any combination of the following procedures: a) observation of the program/project, b) analysis of documents related to the program/project, c) individual interviews with key people, and d) group interviews with key people (focus groups). You may collect information from all observations in one “Tool 2” form, and collect information from all documents analyzed in another form, while you may need to complete one form per interview.The 15 questions included in the table below can be used to obtain indicator-related information that addresses the four characteristics of organizational routines (references).

 

Sustainability level: Guide for information collection (e.g. interview guide)
Memory – resources that preserve lessons learned

1 – Are financial resources used to accomplish program/project activities integrated in the organization’s (name the organization) budget?

  • Are these costs part of the budget of the organization (name the organization)?
  • If so, are these costs a permanent or temporary part of this budget?

2 – Who are the people hired by the organization (name the organization) to accomplish these activities?

  • Are these people hired full time or part time?
  • Are these people hired permanently or temporarily?

3 – What are the material resources provided by the organization (name the organization) (rooms, photocopies, medication, etc.)?

  • To which budget do these material resources correspond?

4 – Are there other resources allocated to these activities?

Adaptation and barriers to adaptation

5 – How are these activities adapted to the context of the organization (name the organization)?

6 – Are these activities organized in a way that we can evaluate their effects (e.g. according to follow-up reports, annual statement of accounts or evaluation outcomes)?

7 – Among these activities, which ones are still organized because “we like it” or for historical or any other reason, but for which there is doubt concerning the relevance (barrier to adaptation)?

Values – beliefs and codes

8 – Do these activities correspond to written objectives formalized by the organization (name the organization) (e.g. in the form of a logical frame for “project management”) (Ask to see them).

9 – What are the symbols attached to these activities such as logos (or any other symbol)?

10 – What are the rituals established concerning these activities such as periodic meetings (or any other ritual)?

11 – Did a specific language emerge around these activities, some sort of jargon?

Rules and acting decision

12 – Did the organization (name the organization) formally assign a supervisor for these activities?

13 – Are these activities included in the official planning of the organization (name the organization)?

14 – Do task descriptions exist for the organization (name the organization) workforce assigned to these activities?

15 – Are certain activities related to written rules as in a manual of procedure?

 

Second step: Information synthesis

The second step of the sustainability level evaluation consists of the analysis of collected information. First, make a synthesis of completed “Tool 2” forms. The synthesis-table summarizes all indicators according to the four characteristics of organizational routines (memory, adaptation, values, rules). An example of a synthesis-table is presented in “Example of Tool 2 utilization”.

This synthesis-table allows you to establish a sustainability level. The following five scenarios correspond to the five levels of sustainability.

5 levels of sustainability
High sustainability. Standardized activities stem from the program/project; they possess the four characteristics of organizational routines and are carried out in accordance with public policy.
Moderate sustainability. Routinized activities stem from the program/project; they possess the four characteristics of organizational routines; these activities are durable.
Low sustainability. Official activities stem from the program/project but they don’t possess all characteristics of organizational routines and so are not routinized; these activities can change in a radical way on a short term basis.
Precarious sustainability. Residual activities from the program/project are pursued unformally by members of the organization within a function that has nothing to do with the program/project; the continuation of these activities rely on these people.
No sustainability. No activities stem from the program/project in the organization.

 

You can validate the synthesis-table and its interpretation with people involved in the program/project in group meetings with key people. Finally, you can organize periodic meetings with key people to follow up on the sustainability level.

Logo_REALISME_201505

Main Toolkit Page

Tool 1
How to use tool 1
Example of Tool 1 utilization

Tool 2
How to use tool 2
Example of Tool 2 utilization

Glossary

References

Your experience interests us!

You can complete the attached printable version and send it back to us to share your experience.

Fax: 514-398-8483
E-mail:

Glossary

Feb 23, 2017 | Uncategorized

A tool-kit for the evaluation of sustainability processes and
sustainability levels of public health programs and projects

 

Glossary

Activities: An activity is defined as the understandable behavior of individuals with respect to the behavior of others. Programs prescribe the activities required to achieve a set of objectives that direct the behavior of people involved. In turn, these activities consist of tasks to be completed while drawing on financial, human and material resources.

Adaptation: The notion of adaptation refers to a modification or adjustment aiming at coherence and harmony. Routinized activities are adapted to programs/projects’ and organizations’ contexts or environments. Adaptation is one of the four characteristics of organizational routines.

Event (e.g. actions taken): Events are relevant observation units for assessing processes. Events are defined as elements in sequences (ordered samples of phenomena, order being usually temporal). Events and their consequences may contribute to explaining organizational processes. Events are distinguished by what is set off in time by a before and an after, and there are multiple types of events (mergers, decision-makings, meetings, etc.).

Evaluation: Evaluation is a judgement process that takes a critical approach and utilizes systematic data collection in order to make decisions.

Indicator: An indicator is a significant parameter used to measure outcomes, resource utilization, works progression status, or context.

Institutional standards: Institutional standards are defined as social norms established by state-level authorities (professional corporations or public administration). Neo-institutionalists emphasize the legitimacy of institutional standards upon which organizations are based over power and coercion. These standards are public and exist legally for the social good. Institutional standards directly constrain organizations or people. State-level measures, principles, policies or regulations that constrain organizations and people make institutional standards operational. Examples of institutional standards include federal, state level, county level, and municipal government rules and policies, as well as professional guidelines and accreditation standards that might be set by universities, hospitals, or school districts.

Memory: Memories refer to shared interpretations of past experiences that influence present activities. Memory requires stable resources and includes three major components: social networks, paper-based resources, and electronic resources. Memory is one of the four characteristics of organizational routines.

Organizational routines: Organizational routines are defined in terms of memory, adaptation, values, and rules. A routine is a typical procedural operation. Routines are integrated in organizations like the memory of actions or procedures shared by the actors. Routines are adapted to suit their contexts. Routines reflect the values, beliefs, codes, or cultures by means of symbols, rituals, and language. Routines adhere to rules that govern action and decision-making such as manuals of procedure, rules of information transmission, or plans.

Program/project: A program/project is a coherent, organized and structured set of objectives, activities and resources (human, financial and material), which are carried out by people involved for achieving the objectives. It is justified on the basis of needs of individuals, communities or a society. It is usually controlled by one or many people who are responsible for the quality of its design and operation.

Program/project implementation: A program/project’s implementation consists of a mobilization of resources to set up activities to reach an objective.

Routinized program/project: A routinized program/project is defined by the presence of routinized activities, i.e. activities presenting the four characteristics of organizational routines: memory, adaptation, values, and rules. To be considered routinized, at least one activity stemming from a program/project must possess all four of these characteristics. Routinization is considered to be the first sustainability process, while standardization is considered to be the second.

Rules: Routinized activities follow rules that regulate decision making and action. Rules are one of the four characteristics of organizational routines.

Sustainability level: The notion of sustainability level refers to a ‘snapshot’ of the sustainability process at a certain moment in time. The presence of organizational routines makes it possible to evaluate the sustainability level of a program. The four characteristics of organizational routines are: memory, adaptation, values, and rules.

Sustainability process: The notion of sustainability process refers to the processes that allow for the continuation of effects and activities relating to programs and projects. This process begins with the first events that outline a program or a project, i.e. their planning and implementation.

Standardized program/project: A standardized program/project is defined by the presence of routinized activities in accordance with public policy. Program/project standardization is considered to be the second sustainability process. Standardized programs/projects are even more durable than routinized programs/projects.

Values: Values and collective beliefs are manifested through cultural artefacts such as codes, symbols, rituals, or jargon. The formulation of objectives is also based on values and collective beliefs. Values are one of the four characteristics of organizational routines.

 

Logo_REALISME_201505

Main Toolkit Page

Tool 1
How to use tool 1
Example of Tool 1 utilization

Tool 2
How to use tool 2
Example of Tool 2 utilization

Glossary
References

Your experience interests us!

You can complete the attached printable version and send it back to us to share your experience.

Fax: 514-398-8483
E-mail:

References

Feb 23, 2017 | Uncategorized

A tool-kit for the evaluation of sustainability processes and
sustainability levels of public health programs and projects

 

References

(1) Pluye, P., Potvin, L. & Denis, J.-L. (2004). Making public health programs last : conceptualizing sustainability. Evaluation and Program Planning, 27, 2, 121-133. (DOWNLOAD)

(2) Pluye, P., Potvin, L. & Denis, J.-L. (2004). Corrigendum [Making public health programs last: conceptualizing sustainability]. Evaluation and Program Planning, 27, 4, 453. (DOWNLOAD)

(3) Pluye, P., Potvin, L., Denis, J.-L. & Pelletier, J. (2004). Program sustainability : focus on organizational routines. Health Promotion International, 19, 4, 489-498. (DOWNLOAD)

(4) Pluye, P., Potvin, L., Denis, J.-L., Pelletier, J. & Mannoni, C. (2005). Program sustainability begins with the first events. Evaluation and Program Planning, 28, 2, 123-137. (DOWNLOAD)

(5) Ridde, V. & Bailat, S. (2005). Projet de lutte contre la malnutrition infantile dans le département du Sud 2003-2005. Rapport d’évaluation. Terre des Hommes. 56 pages. (DOWNLOAD)

(6) Ridde, V., Pluye, P., & Queuille, L. (2006). Évaluer la pérennité des programmes de santé publique : un outil et son application en Haïti. Revue d’Épidémiologie et de Santé Publique, 54(5), 421–431. (DOWNLOAD)

(7) Ridde, V., Pluye, P. & Queuille, L. (2006, Juin). A tool-kit for program sustainability evaluation. Communication présentée à la 5th Nordic Health Promotion Research Conference, Esbjerg, Danemark. (DOWNLOAD)

Others :

Samb, O. M., Ridde, V., & Queuille, L. (2013). Quelle pérennité pour les interventions pilotes de gratuité des soins au Burkina Faso ? Tiers Monde, 215, 73–91. Download
Bélanger, M. (2014). Comment améliorer la pérennisation des projets de Desarrollo? Des lignes directrices pour cet organisme oeuvrant en Bolivie [Mémoire]. Université de Sherbrooke. Download
Beaudet, N. (2016, Janvier). Soutenir les mamans immigrantes. Perspective infirmière, 13 (1). 52-55. (DOWNLOAD)

Logo_REALISME_201505

Main Toolkit Page

Tool 1
How to use tool 1
Example of Tool 1 utilization

Tool 2
How to use tool 2
Example of Tool 2 utilization

Glossary

References

Your experience interests us!

You can complete the attached printable version and send it back to us to share your experience.

Fax: 514-398-8483
E-mail:

A tool-kit for the evaluation of sustainability processes and sustainability levels of public health programs and projects

Feb 22, 2017 | Uncategorized

A tool-kit for the evaluation of sustainability processes and
sustainability levels of public health programs and projects

 

This tool-kit helps you to evaluate, plan, and better understand the sustainability processes and sustainability levels of public health programs and projects.

Why should you use this tool-kit?

In the public health domain, program sustainability has become an increasing concern for all. However, the knowledge concerning programs sustainability is still contradictory and fragmented. This tool kit offers a simple and convenient analytic framework to globally evaluate the sustainability processes and levels of public health programs and projects.

Who is this tool kit intended for?

This tool kit is intended for many types of stakeholders in the field of public health: coordinators, evaluators, managers, managers of public health organizations, teachers, researchers, decision makers.

Definitions

The notion of sustainability process refers to the processes that allow for the continuation of effects and activities relating to programs and projects. This process begins with the first events that outline a program or a project, i.e. their planning and implementation.

The notion of sustainability level refers to a ‘snapshot’ of the sustainability process at a certain moment in time. The presence of organizational routines makes it possible to evaluate the sustainability level of a program.

Specific terms used in this tool kit are defined in the glossary section.

 Tool kit

This tool kit can help you evaluate the sustainability process (Tool 1) and the sustainability level (Tool 2) of a program or a project.

Tool 1
How to use tool 1
Example of Tool 1 utilization

 

Tool 2
How to use tool 2
Example of Tool 2 utilization

Glossary
References

Logo_REALISME_201505

Main Toolkit Page

Tool 1
How to use tool 1
Example of Tool 1 utilization

Tool 2
How to use tool 2
Example of Tool 2 utilization

Glossary
References

Your experience interests us!

You can complete the attached printable version and send it back to us to share your experience.

Fax: 514-398-8483
E-mail: