Field trial
Pisa 2006 Field Trial was carried out in April 2005.
Sample
In November 2004 a judgment sample was drawn, comprising 63 Secondary schools representing all Italian education tracks and geographic regions. Moreover, Bolzano province participated in the Field Trial with a separately adjudicated sample of 23 schools, whose language of instruction was either German or Ladin. The school selection was based on the rolls provided by the Italian Ministry of Education. The number of sampled schools was determined so that each item might be administered to at least 200 students and each booklet to at least 100 students.
| Number of schools | Number of students planned for Field Trial sample | Number of respondents | |
|---|---|---|---|
| North Eastern Region | 7 | 245 | 222 |
| Bolzano Province | 23 | 805 | 653 |
| North Western Region | 8 | 280 | 258 |
| Central Region | 8 | 280 | 253 |
| Southern Region | 9 | 315 | 268 |
| South & Islands | 8 | 280 | 232 |
| Total | 63 | 2205 | 1886 |
Sample distribution per geographic region
| Number of schools | |
|---|---|
| Lyceés | 15 |
| Vocational Institutes | 10 |
| Technical Institutes | 15 |
| Total | 40 |
Sample distribution per school type (National sample)
Within each school, a random sample of 35 students is drawn from 15-year-old students.
Preparing and conducting the assessment session
In January 2005, sampled schools were contacted by the PISA National Centre. Each school participating in PISA was asked to appoint a School Associate - meeting the Consortium requirements - who were to be held responsible for all assessment activities within the school. In February, PISA National Centre provided School Associates with five training session. Training sessions were held in Foggia, Padua, Palermo, Rome, and Turin. During the training sessions assessment procedures were thoroughly illustrated and School Associates were given the School Associate Manual. Similarly, training sessions were organized for the Associates of German/Ladin schools of the Bolzano Province.
Materials to be used in the Main Study were all field trialled between April 4th and April 30th 2005. Assessment materials comprised: 12 Test Booklets containing scientific literacy items (test item rotating within each booklet and booklets rotating within student groups), Student Questionnaires (also including ICT familiarity questions), a Parent Questionnaire, and a School Questionnaire.
Only four schools - within the national sample - had to schedule a follow-up session, as more than 15% of the students sampled from their schools (not counting student or parent refusals, or excluded or ineligible students) were absent from the assessment session.
Each assessment session was divided into two sections: section 1 was devoted to the Cognitive Test and section 2 to the Student Questionnaire. Students were given two hours to complete the Test Booklet (with a short break after the first hour) and then, after a 10 minutes break, they were given another 50 minutes to complete the Student Questionnaire. The whole Assessment session took little more than three hours.
| Nationwide | Bolzano Province | |
|---|---|---|
| Booklets(Respondents) | 1233 | 653 |
| Student questionnaire(Respondents) | 1232 | 652 |
| Completed Parent Questionnaires | 1082 | 536 |
Quality Monitoring
On the day of the assessment, quality monitors appointed by the National Centre visited some randomly selected schools without announcing their visit, in order to verify that assessment procedures were duly followed.
Answer coding and data entry
Marking and coding procedures started when all materials were back from the schools. Subject experts were hired for coding student responses to both the tests and the questionnaires, as well as for coding the answers to the Parent Questionnaire. As to the coding of the questions concerning occupation in the Student and Parent Questionnaires, the International Standard Classification of Occupations (ISCO-88) was used as a reference. Data entry was contracted out to a reliable firm.
Field Trial data were sent out to the International Consortium at the end of July to undergo data analysis, as Pisa National Centre is only responsible for national-level data analysis.
