September 9, 2011
Intergage Consulting Group Inc. (Intergage) was contracted by Elections Canada (EC) to conduct the Voting Rules! civic education program formative evaluation. Voting Rules! is a joint pilot project developed by Elections Ontario (EO) in association with EC for Ontario educators.
The evaluation was based on the following data collection methodologies:
The survey results show that over one-third of respondents had used the toolkit by the time the survey was conducted (see Table 5 of the Report). More specifically, 5% had used most or all of the material provided in the toolkit and 30% had used some of the material. Of the remaining respondents, 40% had only reviewed the material and 25% had ordered the material but did not have a chance to review or utilize the material in the classroom.
The level of usage was attributed to the April distribution date of the toolkits. Of those who didn't use the toolkit:
Both the late distribution of the toolkits and the timing of the survey (which was sent to teachers in late May and open into June 2011) had an impact on the overall response rate. That being said, the overall response rate to the survey was 21%.
Overwhelmingly, the rationale and relevance of the Voting Rules! program was supported by both grade 5 and 10 teachers participating in the interview consultations. The program is viewed as providing an engaging resource that teachers can integrate into existing civics curriculum. Those consulted indicated that the "enquiry based, experiential learning" approach to the toolkits and lessons helped students make the connection between theory and their daily lives.
By and large, those consulted were quite satisfied with the resources and material available in the toolkit, as overall, 37% felt the material is "excellent" and another 54% felt it was "good". Grade 10 respondents were slightly less positive in their assessment than Grade 5 respondents.
Specific results show that nearly all respondents agree that the:
Among those who responded, one-third (35%) specifically alluded to the graphic, comic or cartoon format of the toolkit. One-third of these respondents (33%) indicated that they liked various aspects of the activities, including creating a superhero candidate, the true/false activity, the short and easy nature of the activities as well as the variety available in the toolkit. Thirty-one percent also specifically noted the election simulation activity and the "genuine nature" of the material related to this activity as a strong feature of the toolkit. Another common theme related to the general format of the toolkit, where close to 20% felt the toolkit was in an "easy-to-use" format and clear.
Among those who responded, one-quarter commented on the student-friendly, clear and interactive nature of the activities presented in the toolkit (28%). Other common themes raised by respondents include the election simulation and its materials (19%), the handouts and worksheets (17%), the graphics and "cartoons" (16%) and the general layout of the lesson plans (16%).
The majority of respondents, when asked, did not feel that anything had to be done to improve the toolkits – that they were excellent the way they were. Otherwise, feedback provided by the other respondents does not point to a potential weakness or an overarching issue. Among the few comments provided, Grade 5 respondents commented that the handouts were too text-intensive or that the page was too cluttered to write anything down. A few commented about font size (12%), that the graphic organizer of government needed improvement (12%) and that the toolkit should be available in digital format (12%). The most common issue raised by Grade 10 respondents (at 15%) was that the activities in general needed to be more discovery-based.
While the lessons contained in each toolkit were ranked quite positively, there were some recommendations to improve a few of the lessons in order to make them more relevant and engaging to students (please see the Evaluation Findings and the Conclusions and Recommendations sections).
The evaluation indicates that the program is achieving its intended outcomes. When respondents were asked what they consider the most important outcome of the program, over a quarter of respondents indicated: the knowledge it develops about the voting process followed by involving and engaging students in the issues of democracy and voting (36% and 24% respectively). Those who ordered the Grade 5 toolkit were more likely to identify the increased knowledge about voting and elections as the most important outcome, noted by 47% of those respondents.
Both the Grade 5 and Grade 10 toolkit were seen as useful resources for respondents concerning teaching aspects of democracy, elections, voting and government. The toolkits seem particularly constructive in terms of teaching: the rights and responsibilities of citizens (87%); the three levels of government (86%); why it is important to vote (86%); and in terms of increasing students' awareness of how democracy, elections, voting and government are connected to their everyday lives (86%).
An overwhelming majority of survey respondents and all interview respondents indicated that they would and plan to use the toolkit in the future to support their civics teachings.
Respondents are in widespread agreement that material from Elections Canada and Elections Ontario is trustworthy, accurate and politically neutral (74%). Furthermore, 90% agree that they consider Elections Canada and Elections Ontario authoritative sources for material related to voting and democratic rights and responsibilities for their students and 89% agree that their material is relevant to their teaching needs and requirements.
What follows are the recommendations from the formative evaluation of the Voting Rules! program. More detailed recommendations on specific changes to selected lessons in each toolkit are presented in the Conclusions and Recommendations section of this report.
The positive results from this evaluation suggest greater promotion of the toolkits to Grade 5 and Grade 10 educators in Ontario is warranted. It is clearly viewed as a useful and valuable resource that can benefit civics educators. Expanding its use across the province would help to achieve one of the sponsoring organizations' key objectives of contributing to increased awareness and understanding of the electoral process among students in Ontario.
EC and EO should review the findings from this evaluation, share them with their provincial partners and colleagues and collaboratively work with educators and school boards across Canada to either develop similar toolkits to be used in the remaining jurisdictions in Canada or to expand the use of this program nationally.
To ensure the continued success and relevance of the Voting Rules! program, EC should undertake a future evaluation of the program once it has been fielded for at least 1-year (April 2012) and teachers have been given enough time to integrate the program/toolkits into their teachings. Alternatively EC and EO could put in place ongoing data collection (likely through surveying) to monitor implementation and further enhancements and changes could be applied to the program. It is further recommended that the program track impact on participating students by conducting pre and post-program participation assessments.
Lessons that may require further review and attention, to help raise the quality of the program more towards an excellent rating in the future, are identified in the tables below under Grade 5 and Grade 10 respectively.
The popularity of the resource sheets and the activities and the relative lack of importance of the time estimates suggest further investment could be made in developing a greater variety of activities, resource sheets, and extensions to provide educators greater range and flexibility within each lesson plan. Educators clearly appreciate creative and innovative material, especially the kind they would not be able to easily develop themselves (e.g. crossword puzzles).
Additionally, if quizzes are to remain in the toolkit, additional efforts might be warranted in terms of further assessing the relevance and usefulness of the quizzes given how much of the toolkit is dedicated to this feature.
EC and EO should consider tailoring the toolkit to Grade 5 and 10 students that are in a non-traditional classroom environment (i.e. students with learning disabilities that are being educated through health treatment centres, those located in a correctional facility or others that may have special development needs, etc.). This would likely require some consultations with teachers and developmental experts to determine how the lesson plans can be simplified to better engage students in these special circumstances. This evaluation has demonstrated that the toolkits are being used by teachers in these special circumstances; however, those consulted have to make significant adjustments. The benefits of reaching youth in these circumstances is that they tend to be higher "risk" – who are often harder to reach and have a higher likelihood of being more marginalized from society, democratic issues and voting as they age.
Educators are not averse to leveraging information and material from multiple sources. It is recommended that EC and EO continue to develop and refine the toolkits to include additional complementary resources (i.e. online and print).
EC and EO should review and analyze the findings from the Voting Rules! program formative evaluation and incorporate "lessons learned" into future programs on civics education to ensure teacher and student satisfaction with the use of EC and EO programs is exceptional and that the program remains unique.
Educators overwhelmingly agreed that EC and EO are authoritative sources of impartial, fact-based and credible resources in support of civics education programs. Intergage recommends that EC and EO continue building their presence as authoritative sources for civics education by further developing existing and new materials to fill the need of educators both in Ontario and the other provinces and territories.
Intergage recommends that a PDF version be developed, deployed (either by email or in a CD format), periodically updated and remain accessible electronically to all educators for their continued ease of access. MSWord versions of the resource sheets would also empower educators to easily modify the documents to better suit their needs.