Report #5
Hamburg, 19–22 February 2007
5th Meeting of the “QualiFLY” Project
“Monitoring and evaluating family literacy programmes”
The fifth meeting of the QualiFLY project was held from 19 to 22 February 2007 in Hamburg. The following participants attended the meeting:
Prof. Greg Brooks, University of Sheffield, England
Mr Eric Nedelec, Agence nationale de lutte contre l’illettrisme, France
Dr. Peter May, State Institute for Teacher Training and School Development,
Hamburg, Germany
Dr. Gabriele Rabkin, State Institute for Teacher Training and School Development,
Hamburg, Germany
Ms Carla Barozzi, Università Popolare di Roma (UPTER), Italy
Mr Luciano Daina, Università LUISS di Roma, Italy
Marthese Cini, Foundation for Educational Services, Malta
Sandro Spiteri, Foundation for Educational Services, Malta
Ms Snoeks Desmond, The Family Literacy Project, South Africa
Prof. (Ms) Ayhan Aksu-Koç, Bogaziçi University, Istanbul,
Turkey
Prof. (Ms) Sevda Bekman, Bogaziçi University, Istanbul, Turkey
Ms Ceren Lordoglu Tuz, Mother-Child Education Foundation (AÇEV),
Turkey
Ms Maren Elfert, UNESCO Institute for Lifelong Learning
The following participated temporarily in the meeting:
Ms Anne Bock, University of Bremen, Germany
Peter Daschner, Director, State Institute for Teacher Training and School
Development, Hamburg, Germany
Ms Anna Fricke, Germany
Dr. Sven Nickel, University of Bremen, Germany
Ms Marissa Pablo-Dürr, Pedagogical Institute Nürnberg, Germany
Ms Yvonne Zirra, University of Bremen
Ms Derya Akalin, Mother-Child Education Foundation (AÇEV), Turkey
Ms Imke Behr, UNESCO Institute for Lifelong Learning
Ms Bettina Bochynek, UNESCO Institute for Lifelong Learning
Dr. Ulrike Hanemann, UNESCO Institute for Lifelong Learning
Ms Lisa Krolak, UNESCO Institute for Lifelong Learning
Dr. Adama Ouane, Director, UNESCO Institute for Lifelong Learning
Teachers and facilitators working in the FLY project: Ms Songül Ala, Ms Tami Alon, Ms Aysen Ciker, Ms Andrea Dünnwald, Ms Sonja Gringel, Ms Marianne Heidbruch, Ms Annette Huber, Ms Tatjana Köhne, Ms Mechthild Langemeier, Ms Anna Lies Liedtke, Ms Ilona Ozimek, Mr Christian Schulz, Ms Ute Stather
The meeting started with a welcome by the Director of the UNESCO Institute
for Lifelong Learning, Adama Ouane, an introduction to the QualiFLY project
by the coordinator, Maren Elfert, and a presentation of a TV documentary
on the family literacy pilot project “FLY” in Hamburg, commented
by Gabriele Rabkin, coordinator of the FLY project. In the course of the
meeting evaluations of several family literacy projects from different countries
were presented:
the Family Literacy Project (FLP) in South Africa;
the Turkish Mother Child Education Program and Preschool Parent-Child Education
Program;
the English family literacy programmes which began in the 1990ies;
the family literacy pilot project FLY in Hamburg;
the Maltese HILTI and NWAR family literacy programmes.
Eric Nedelec from France presented new family-oriented initiatives being
implemented in France, the “local family programmes” and the
“Educative Achieving Programmes” without addressing evaluation.
The major findings of the evaluations can be resumed as:
- limited quantitative evidence of benefit to parents’ literacy;
- better evidence of boosting parents’ ability to help their children;
- some evidence of benefit to children’s language and literacy.
All in all there is not enough controlled comparative research. However, it seems a very promising approach and needs to be explored further.
Discussions addressed the aims, results and main challenges of evaluating family literacy programmes as well as the impact, challenges and limitations of family literacy programmes as they are reflected by the evaluations. All presented evaluations used qualitative methods (questionnaires and interviews), some also quantitative methods (literacy tests with children) and quasi experimental designs involving control groups. The evaluation of the South African project used an adaptation of the Photo Voice technique and the Most Significant Change approach. Both evaluations seem to have promoted a sense of ownership of the evaluation process and observation and reflection in the facilitators and learners.
The main aspects addressed during the discussions among the group in the course of the meeting were the following:
What works in family literacy programmes:
Talking about things that matter to the parents.
Do learning contracts with parents: The parents say what they expect from
the programme, these expectations are negociated and aims are set (NWAR).
In Malta the children are always asked what they thought about the programme.
They get more involved by signing the contract (together with their parents).
What is family literacy about?
Is family literacy a response to multicultural integration?
Or is family literacy a response to social class education?
It cannot be divided. If you address ethnicity, you address the social class
at the same time. Social class is not necessarily a predictor of educational
achievement.
Is it a middle class approach?
It shouldn’t be either or. We need family literacy programmes and
adult literacy programmes to target different people with different needs.
In many cases, family literacy is a women’s empowerment programme.
It’s a refuge for women from a man’s world. So it can be seen
as positive that these programmes are mainly attended by women.
What can family literacy do?
Family literacy needs to set manageable goals, it cannot do everything.
Family literacy can diminish the gaps between the educational backgrounds.
Family literacy is about breaking the cycle of educational failure.
To involve the “difficult”, “hard-to-reach” people
is a long-term process and needs a very close contact. Family literacy programmes
do not always reach the target group they want. To change this, programmes
have to be sustained over a long period.
Avoid top-down “power relationship”
Should FL be an entitlement for parents?
FL cannot be an obligation. Preschool education should be an obligation.
In Malta, the focus goes back to the work done in the schools. After-school
programmes should not compensate what can be done in the schools. In France
there is awareness that the school has failed, therefore investments are
currently being made in out-of-school activities.
For center-based programmes: How can it be achieved that the activities
continue at home? Give homework, parents can borrow books and games and
take them home (parents’ suitcase).
Challenges of evaluations:
Fluctuations, lack of control groups, too many conditions, too few effects
to analyze.
Not reaching the target group that you really want.
Use the evaluation to bring the change you want.
Photovoice and The Most Significant Change are ways of assessing that avoid
the power relationship. It is a way of assessing what is meaningful to the
the parents, not to evaluate them.
The impact of family literacy programmes seems promising, but there are
not enough data actually proving significant effects on reading and writing
skills of parents and children.
There are very little long-term evaluations except some data in the U.S.
and a long-term survey carried out by Turkey which showed that a higher
proportion of children who had participated in family literacy programmes
studied at university.
Assessment depends on the context.
Family literacy is multidimensional. If only the literacy skills are tested,
the other effects and dimensions of the programme are lost out of sight.
Evaluations are needed to prove that the programme achieves what it is aiming
for.
What evaluations are needed?
Stories with numbers. For the funders, the numbers are needed. For the learners
and the more holistic effects, the stories are essential as well.
What would an ideal evaluation of a family literacy programme look
like:
1. There should be an intervention group and a control group, put together
by random selection. Randomized control trials are rare in literacy assessments.
2. People get lost along the way, so the cohort should be big enough, e.g.
150 participants in the intervention group, 150 in the control group. The
methods used should involve a pre-test and a post-test.
3. Preferably standardized tests should be used. Standard tests (e.g. phonological
awareness tests, vocabulary tests, vocabulary language use, discourse competence)
have the advantage that they can be generalized. The Clay test tests the
reading culture at home through indicators, e.g. does the child have a favourite
book?
4. The assessment should be dynamic, meaning that the child is being helped
so that it can give the correct answer at a second stage. Not only pointing
at something and if the child fails it gets a score of 0. The score should
be more differentiated. Such an evaluation would be time-consuming and require
well-trained people.
5. There should be a follow-up test after 1 year and maybe another one after
4 years. If the effect washes out over the years the effect in the first
place needs to be questioned.
6. The core characteristics of family literacy evaluations is that they
look both at the children and the parents.