Assignment 4 - Logic Model

    Project Logic Model:  Nelson Literacy Reading Program
      The Nelson Literacy Reading Program provides teachers with a wide variety of text forms and lessons to help teach students reading strategies.  The program provides teachers with detailed lessons, opportunity for differentiated instruction and assessment tools.  Nelson Literacy emphasizes the need for whole class, small group and individual instruction. The primary goal of the program is to equip students with reading strategies that they can apply to different text forms to help them learn new information. 

GOAL	OBJECTIVES	ACTIVITIES	OUTPUTS	OUTCOMES	IMPACTS
Statement of the overall purpose of the project	Specific statements of what the project sets out to accomplish	Specific tasks to complete through implementation of the project	Immediate results (direct products of project activities)	Intermediate results (1 to 3 years after project starts)	Long-term results (3 to 10 years after project starts)
To develop a wide variety of appealing and well – written text forms which will be used to teach students various reading strategies to help them develop their reading comprehension skills and be successful in comprehending new information in all subject areas.	Students will question what they read to make personal connections to the text Students will learn how to summarize the main ideas of a text Students will make inferences and synthesize to evaluate texts	Teacher provides instruction using a reading strategy Teacher models how to use the strategy Teacher and students use the strategy together Students apply strategy, with some support from teacher Student applies the strategy independently	Students are able to apply the learned reading strategies to the performance tasks at the end of instruction	Students are able to apply the learned reading strategies to various subject areas, depending on the text type which is being used to present new information Students benchmark results (reading comprehension test) will improve as they practice the learned reading strategies	Students will have learned various reading strategies which they can use to help them learn new material in their academic futures  Students will think more critically about what they read
Goals and Objectives represent the Strategic Directions of your Project		Activities comprise the Project Work Plan, which should include details for each activity (Who?, What?, When?, Where? How?)	Outputs link directly to project activities; activities are what is done…outputs are the expected results of what is done	Outcomes relate to your objectives; objectives are desired, outcomes are the expected results	Impacts relate to goals; a goal is desired, impacts are the expected end-results

Assignment 3 - Program Evaluation

Program Evaluation:  Nelson Literacy Reading Program

Engage Stakeholders

 Who should be involved?

Teachers and students should be involved in the program evaluation.  

How might they be engaged?

They will be engaged by answering a questionnaire and rating scale.

Focus the Evaluation

 What are you going to evaluate? 

The evaluation will determine if the Nelson Literacy program is successful in teaching students various reading strategies which they can use to help themselves learn in all subject areas.

What is the purpose of the evaluation?

The purpose of the evaluation is to determine how effective and user-friendly the program is.  Is it set up to ensure teachers are able to effectively teach students the different reading strategies?

Who will use the evaluation and how will they use it?

Nelson Literacy Developers – Will use the evaluation to consider improvements to their program.

School Divisions – Will use the evaluation to decide if the program should be implemented in their schools.

What questions will the evaluation seek to answer?

Do teachers find the resource easy to use?  Are the lessons set up to ensure students are able to be successful in learning the various reading strategies?

Are students able to apply their reading strategies across all subject areas?

What information do you need to answer the questions?

What I wish to know:	Indicators – How will I know it?
How useful teachers find the resource to be	Results from survey
Does the program succeed in teaching students various reading strategies?	Results from student reading comprehension testing (benchmark results) and performance tasks from each reading strategy (from the program)
Are students able to apply the reading strategies to help them learn new content in all subject areas?	Results from survey

When is the evaluation needed?

The evaluation is needed during and after the implementation of the Nelson Literacy program.

What evaluation design will be used?

Scriven – Summative and Formative

Collect the Information

What sources of information will be used?

Existing Information:  The Nelson Literacy Resource Overview Booklet

People:  Teachers and students currently using the program

What data collection methods will be used?

Surveys and Tests

Instrumentation:  What is needed to record the information?

Questionnaire for teachers and students, Benchmark testing results and Performance Task results

When will data be collected for each method?

Survey (Teachers) – During program

Survey (Students) – During program

Tests – Benchmark – Immediately after program

Demonstration Task Results – Immediately after the teaching of each literacy strategy (During program)

Analyze and Interpret

How will the data be analyzed?

Data analysis methods:  Responses will be read and answers compiled using the computer.

Who responsible:  I will read the teacher and student survey responses.  I will compile the responses into similar categories on the computer.  Teachers will be responsible for providing information on their students’ benchmark results and performance task results for the different reading strategies.  This information will also be categorized using the computer.

How will the information be interpreted – by whom?

Have students’ benchmark results improved since the Nelson Literacy Program has been implemented?

Nelson Literacy program developers will use the results to determine how effective their program is in improving students’ reading strategies and what teachers like / dislike about the program.

Use of the Evaluation

What was learned from the evaluation?  What are the limitations?

What action will be taken (if any), to help make improvements towards the Nelson Literacy program?

To Whom	When / Where / How to present
Teachers	Email teachers (who participated), the results from teacher surveys and student testing results – will help teachers recognize common areas of strengths and weaknesses
Nelson Literacy Developers	Written report, including charts / graphs displaying the results from surveys and student achievement

Manage the evaluation :  Human subject’s protection

                                           Timeline

                                           Responsibilities

                                           Budget

Assignment 2 - Simulated Program Case Study

The program for this assignment focused on a prenatal exercise program for Saskatoon Aboriginal women.  The program’s goal was to promote physical activity amongst pregnant Aboriginal women, with the hope that rates of Type 2 Diabetes would decrease. 

I would choose to use both the Scriven – Summative and Formative, and the Provus (DIPPC) models of evaluation for this program.  I would start with evaluating the content of the program and the plans for installing it.  During the program I would evaluate the “process” by questioning the women about their enjoyment of the program, what future activities they would like to see and what additional supports they may need to help them reach their personal goals.  This information could then be used to adapt the program if necessary, to help meet the needs of the participants.  The “product” of this program could be evaluated by gathering data in the health care system to see if the goals were met.  A “cost benefit analysis” would help to recognize if the money spent on this program was worth it and if the preventative measures of this program were of any benefit to our health care system. 

I would conduct “formative” evaluations during this program to help make improvements in advertising of the program and recruiting participants. Formative evaluations would also help to make changes (if any were necessary) to the physical activities, supports and resources that were offered to the women.  The roles of the Aboriginal Project Facilitator, Elder and other program facilitators should be evaluated during the process to determine how efficient they are in helping the participants to reach the goals of the program.  Since this program ran for a two year period, “summative” evaluations should be completed to determine if the goals of the program were met:  Did promoting regular physical activity among pregnant Aboriginal women help to reduce rates of type 2 diabetes in future generations?  Did promoting the regular physical activity help the Aboriginal women to have healthier pregnancies?  What improvements could be made to the program?

Assignment 1

ECUR 809 – Assignment One           

            

 The program evaluation I chose for this assignment was completed in November, 2012, for the YWCA Saskatoon:  A Turning Point for Women.  The article is available at:  

 http://www.homelesshub.ca/Library/Saskatoon-YWCA-Turning-Points-Program-Evaluation-55408.aspx

Shufflebeam’s CIPP model was used for this evaluation.  This model can be broken down into the following:

C – The context of this program was clearly stated.  This program is for homeless women who make the personal choice to come into the Turning Points Program (TPP).  These women must be ready to address their personal weaknesses and be willing to accept assistance from various supports that the program offers.  The goal of this program is to provide “transitional housing” which means that women are fully supported by the TPP program, provided with a place to stay and individualized supports, then gradually able to become more independent and be successful in mainstream society.

I – The evaluation identified who needed to be interviewed, how the interviews were conducted and from what sample they were taken from (specified in the process).

P – The method used by the evaluators was a “qualitative participatory evaluation process”.  The process involved questions being constructed by a team of the evaluators, the Executive Director of TPP, the Housing Manager of TPP, the Housing Supervisor of TPP and the TPP Coordinator.  A total of eight Community Partners (mental health, addictions, street outreach and employment programs), were involved in an interviewing process.  The partners selected had either made a referral to the TPP program or was currently involved in helping a woman in the TPP program.  10 out of 52 current and past clients were randomly selected to be interviewed.  Interviews were conducted in the summer of 2012.  

P – The success of the project was presented in a Logic Model.  This model clearly outlined the resources that the program uses, the activities offered to the women, the objectives of these activities, along with the short term and longer term program goals.  

The strengths of the evaluation include:   

-          Evaluators worked cooperatively with the staff at TPP to formulate the interview questions.

-          A pilot interview was conducted to help ensure questions were appropriate for gathering the required information.

-          The evaluation included personal statements from community partner workers and the women who were interviewed.

-          The description of TPP was very well explained and included many examples of what types of supports and activities this program offers.

-          Evaluators used data and research results from other programs around Canada that were similar to TPP, to help support the positive impacts that these programs have for women. 

-          The Salvation Army report (2010) indicated that 1 in 10 Canadians live in poverty today and continue to be at risk - This indicates that there is a great need for programs like TPP to help make improvements in the Saskatoon area.

-          Positive connections were made between transitional housing programs and how they prevent the health care system from spending significant amounts of money towards caring for homeless people.

The weaknesses of the evaluation include:

-          The role of the Program Facilitator was not fully explained.  It seemed unreasonable that one “Program Facilitator” was able to oversee “individualized programs” for each woman’s specific needs and situation.  The evaluation did not mention how many employees work for TPP.

-          The evaluation stated that the Program Facilitator met with the clients as often as is necessary to keep connected with their progress.  It is unclear as to how the facilitator determines how often they need to meet with each client (Progress Reports?  Checklists?  Goals for the clients?)

-          The program speaks of being very “flexible” to help meet the needs of different women, but there were no examples provided on how this was done.

-          The program claims that there is 24/7 support for women, but how this is attainable is not clarified.

-          More than 10 current and past clients of TPP should have been involved in the interview process.

-          It is stated that the location of TPP makes it easy for clients to access all the services they need – the “location” is not clarified, nor are examples of services available around this location.

-          “Group Support” meetings were held once a week.  I think that these meetings should be held more frequently since they would encourage the women to make more personal connections with others who are sharing their struggles.

For more valid results, I feel that a questionnaire or rating scale could have been given to all women who have been or currently are involved in the TPP program.  Evaluators could have also conducted on-site observations to help them understand more about how the program works and identify any areas for improvement.