ACET, Inc. Blog

We are pleased to announce that Joseph Curiel has been promoted to Evaluation Associate at ACET. When we interviewed Joseph in 2008, we knew that he would be an excellent fit for our company so Joseph was recruited to join ACET as an Evaluation Specialist. We were delighted that he agreed to move from Texas to Minnesota and join our firm. We were impressed with his passion for quantitative data and experience doing multi-level statistical modeling as part of his master’s thesis. Our positive reaction to Joseph’s work has also been shared by our clients. They appreciate the thorough approach Joseph takes when analyzing their data and his ability to succinctly communicate technical findings and patterns in the results to a wide range of audiences. We would like to take this opportunity to share one of many compliments received on Joseph’s work: “[he] does an excellent job” and “this is a strong statement of thanks.”

So we will end this blog with thanks to Joseph for all his hard work!

In the Opinion section of today’s (12/23/11) Star Tribune, Joe Nathan of the Center for School Change at Macalester College described the positive impact of the LEAD for Charters project on student achievement: www.startribune.com/opinion/letters/136111683.html. LEAD for Charters is funded by the Cargill Foundation and consists of a collaboration between the Center for School Change, LarsonAllen, and eight area charter schools (LEAD has also added three new schools for the 2011-2012 school year). LEAD focuses support for schools in the areas of academic achievement, organizational management, family involvement, and financial performance. ACET has been the principal evaluator of the project since its inception.

Congratulations to all involved in achieving the project’s goals!

The Minneapolis-based General Mills Foundation recently announced they will be accepting applications for their Celebrating Communities of Color grants program. Non-profits in the seven-county metropolitan Twin Cities area can apply to receive a one-time, $10,000 grant. Schools and school districts, theaters, nonprofits, health departments, government agencies, and Native American Tribes are eligible to apply for the grant.

Please be aware that the Celebrating Communities of Color program is a one-time award only. If your agency has received a Celebrating Communities of Color grant in the past, you are not eligible to apply in this cycle.

The deadline for applications is February 1, 2012. To learn more about the Celebrating Communities of Color grant program and the application packet please visit: http://generalmills.com/en/Responsibility/community_engagement/Grants/Minneapolis_area/Communities_of_color.aspx

Good luck!

Kirsten

Do you remember your first interview? How about your last interview? Although almost everyone in the workforce has had to complete an interview to gain their position, employers have many different interviewing formats and styles. Last month, I had the opportunity to perform mock interviews with graduate students at my alma mater. The mock interviewing experience gave students the opportunity to practice their interviewing skills and provided an opportunity to gain feedback on their performance without the consequences of a ‘real’ interview. Reflecting upon my own experiences as a student completing the mock interview process last year, I can truly say that the experience was beneficial for students – they received feedback in order to be more successful during their next interview. From the viewpoint of the interviewer, I gained insight into elements that lead to a good interview. Throughout the process, I observed three behaviors that I felt made the difference between a successful and not-so-successful interview. Here are the behaviors that I feel contributed to the success of the interviews.

Take a few seconds to think through your answer before you respond

After a question was asked, interviewees who were successful took a few seconds to think about their response. After a few seconds to think, they were also able to provide greater detail about specific events and were able to better articulate their discussion points. It was more apparent when an interviewee did not think through their response because they often stumbled through their words and their response did not make as much logical sense.

Provide specific examples that demonstrate your experience related to the position

For many of the interview questions, providing specific examples was beneficial by adding evidence to interviewee’s claims. Interviewees who provided specific examples demonstrated their experience and, in some cases, were able to connect their experience to a future work environment. In contrast, interviewees who did not provide examples were less successful because they were unable to back up their claim with evidence.

Be aware of your nonverbal actions that you may not normally notice

During the interviews, some interviewees provided great eye contact, were attentive, and did not fidget; their attentiveness and poise contributed to a successful interview. However, other interviewees performed distracting behaviors like pencil clicking or foot tapping. For these interviewees their distracting nonverbal behaviors led to a less successful interview.

Do you have any strategies for successful interviewing? If so, please share!

Dan

The 2001 No Child Left Behind (NCLB) law set a new standard of proficiency rates for all students across the United States. The law specifies that proficiency proportions for mathematics and reading are required to be 100% by 2014 across all 50 states and territories.

The NCLB law also stipulated that each state could determine which test would be used to measure students’ proficiency. And the NCLB law ensured states that they would determine academic standards, minimum proficiency for graduation, and how to test for student proficiency. Because states can set their own academic standards and how to test for student proficiency, one would expect the difficulty of obtaining proficiency to vary from state-to-state. In fact, under NCLB it is possible that 100% of a group of students might be proficient in State A, but only 70% of the same group of students might be proficient in State B because State B has higher academic standards or a more difficult proficiency test.

So how can we identify differences in states’ academic standards if each state sets their own standards and has their own proficiency test? One way is to test students in different states on the same test and compare the results to proficiency results. One such test is the Measures of Academic Progress (MAP) which is published by the Northwest Evaluation Association. Many states use the MAP to measure students’ progress towards proficiency. Staff at Northwest Evaluation Association’s Kingsbury Center examined potential differences in proficiency measures for 36 states. Kingsbury Center staff identified the MAP percentile scores that were needed to pass each state’s test and compared the differences in difficulty on mathematics and reaching proficiency across states. The higher the percentile needed to pass the test, the more difficult the test.

So where does Minnesota stand? As can be seen in the table below, with all grades aggregated into a single state average, Minnesota’s mathematics standards were the 4^th most difficult of the 36 states studied. Students in Minnesota had to score just above the 45^th percentile on the mathematics MAP in order to be proficient, but the average score for all 36 states was around the 35^th percentile to be proficient in mathematics. Across states, mathematics proficiency varied – some students had to score at the 65^th percentile to be proficient while students in another state had to score at the 10^th percentile. In reading, Minnesota’s reading standards were the 14^th most difficult out of the 36 states studied. In order to be proficient in reading, students in Minnesota had to score at the 35^th percentile, but the average score to be proficient among the 36 states was at the 30^th percentile. Again, across states there was a wide range of proficiency for reading – some students had to score at the 50^th percentile to be proficient while other students had to score only at the 8^th percentile.

*The higher the percentile, the more difficult it is to for a student to score proficient on a state test.

The current Obama administration recognizes the challenges that the existing NCLB law presents for establishing proficiency. The Obama administration’s blueprint for reauthorizing the NCLB Act emphasizes college and career readiness rather than proficiency standards. Perhaps, moving forward with new college readiness standards, the reauthorization might yield some means of standardizing results between states so we can remove some of the big gaps between standards.

To learn more, please click this link for the recent MAP report from the Kingsbury Center: http://kingsburycenter.org/sites/default/files/State%20of%20Proficiency%202011_0725.pdf

Joseph

Imagine that you’ve hired an evaluator to distribute a survey and report the findings back to you. You and your evaluator work hard to develop key questions for your constituents and distribute the survey in such a way to maximize the response rate. Your evaluator has crunched all the numbers and written a report, but you still have some questions. You wonder if looking at some individual survey responses might shed more light on your program and improvements which could be made, or you want to examine the responses of clients who fit specific demographic characteristics (e.g., how did women in outstate-Minnesota respond vs. men in the Twin Cities metro?).

At a glance, those follow-up questions may not seem problematic but from an evaluator’s perspective they can be ethically challenging. Evaluators have a responsibility to protect the anonymity of participants’ responses, especially if anonymity was promised at any stage of the survey process. In surveys, anonymity often comes from combining multiple responses together so that no individual’s response can be identified. In general, the more responses that are combined together the more anonymity participants have. So when clients want to examine an individual’s response or examine the responses of a small number of individuals, anonymity can be jeopardized.

There really are no “hard and fast” rules on how the minimum number of individuals needed to report disaggregated outcomes, but many agencies use 10 as the break point. In other words, if disaggregating outcomes results in 10 or more people in each group, the agency will report on the data. But if there are 9 or fewer people in each group, no data would be reported for that group. But each agency—and evaluator—has their own break point for reporting data; some use 10, others may go as low as 5, while others will not disaggregate data over concerns that any disaggregation could cause harm.

While planning an evaluation it is important for clients and evaluators to discuss all phases of the evaluation, including reporting. Clients can facilitate the reporting process by explaining their needs, including any needs or wishes for disaggregation. Doing so will give the evaluator time to adequately plan for collecting enough data so that the clients’ disaggregation needs are addressed. And evaluators should share their guidelines for break points so that clients are aware of any reporting limitations they might experience.

If you have questions about reporting, especially reporting small numbers of clients, please feel free to comment below and we’ll get back to you!

Stella