ACET, Inc. Blog

One of the things that I like best about evaluation is that there’s no specific formula to get the job done,  and although there are tried-and-true methods and best practices (like the National Institutes of Health’s Best Practices for Mixed Methods Research in the Health Sciences), evaluators have the ability to be flexible and work with stakeholders to design an efficacious evaluation that meets everyone’s needs.

Even as I enjoy the freedom to tailor evaluations to the needs of clients, I find it helpful to have a set of guidelines or a framework to fall back on. Because I’m trained in public health, my favorite framework – based on familiarity and ease of use – is the Centers for Disease Control and Prevention’s (CDC) Framework for Program Evaluation. The framework organizes the basic steps and standards of program evaluation into an easy to use form:

Image source: http://www.cdc.gov/eval/framework/index.htm

What I really like about this graphic is the illustration of the program evaluation process as a cyclical process, with the standards at the center of everything. The visual fits my personal beliefs about evaluation: that it should be ongoing and should always revolve around these set standards of practice.

As far as the listed standards, these groupings include 30 sub-standards set in place by the Joint Committee on Standards for Educational Evaluation.

I strive to keep them in mind throughout the evaluation process and while connecting with clients and program stakeholders:

1. Utility standards help to ensure that evaluation results will be useful to stakeholders, and that they fulfill the needs of users.
2. Feasibility standards are in place to help evaluators design an evaluation that is “realistic, prudent, diplomatic and frugal.”
3. Propriety standards concern legal and ethical aspects of evaluation and research, as well as ensure that the welfare of those involved in the evaluation (or who stand to be affected by the results) is protected.
4. Accuracy standards ensure that evaluation results will present an accurate picture of the program and its operations, especially any features that help to determine the “worth or merit” of the program.

While there’s no real benefit to using this framework over another, and many organizations develop their own framework or set of guidelines for evaluation, I find the CDC framework incredibly useful in guiding my evaluation work. Additionally, I believe that the focus on the standards of evaluation helps to promote transparency, which is a fundamental method of operation here at ACET.

What do you think of the CDC’s framework? Is it too simple? Too convoluted? Does it help you better visualize the evaluation process? We’d love to hear your reactions in the comments!

Elizabeth

Brainstorming creative and useful ways to display data and results can be a challenge. A recent discussion on the LinkedIn page of the American Evaluation Association has turned my attention to a useful tool called the “Periodic Table of Visualization Methods, which can be used to aid and/or inspire this creative process. The table, as suggested by the name, is designed to mirror the periodic table of the elements, but instead of describing the many basic elements and their components, this table is based on the many ways to visually display common evaluation information.

Though the table itself is quite detailed and may look daunting at first glance, you can easily begin to explore its usefulness through its interactive features. Please click on the link and I can further explain this tool: http://www.visual-literacy.org/periodic_table/periodic_table.html.

Once you have activated the link, take a moment and hold your cursor over one of the “elements.” A little graphic should pop-up that will give you an example of that particular visualization method. Each element has its own unique example. Though these are just examples, one can begin to imagine their own information in such displays.

Along with the great examples of each visualization type, the table provides various color coding and symbols to highlight what each type of visualization communicates. For example, yellow indicates a visual display of data while blue-green indicates a visualization of a concept. The two-letter acronyms are described within each box; the symbols used within the boxes are described in the key. Again, hold your cursor over the box for information.

For a more detailed explanation of the development and rationale behind the table, follow this link: http://ow.ly/wk7d

One of the challenges presented by this table is the actual creation of these displays. Though the displays are very visually appealing and often complex, the appeal and complexity demands software that can do such. However, this table does provide some useful guidance and inspiration for how to grab readers’ attention and communicate complex information effectively. What visualization methods stand out the most for you? Which do you think are most appealing? Any personal favorites?

Joseph