What follows is a summary of the order of difficulty of questions, percentage of students with the correct answer, the pre- and post-test stems, and my classification of the questions in the four knowledge categories - factual, procedural, conceptual and metacognitive.*
       
Rank Order of Difficulty Per Students' Post-test Answers
% of Class with Correct Answer: Pretest - Posttest
Pre- and Post-test Stems
My Classfication of the Question
Hydrologic Cycle Test (Direct Instruction w/ Online PowerPoint & Review w/ Online Crossword for Vocabulary)

1 - Easiest

 57 - 91 The per capita water usage is highest in Metacognitive Knowledge
2 75 - 88 When water evaporates it Conceptual Knowledge
3 50 - 71 The movement of water from the oceans to the atmosphere is called Factual Knowledge
4 50 - 68 Evaporation rates increase with Conceptual Knowledge
5 59 - 66 Earth is made up of approximately Factual Knowledge
6 34 - 66 A common method to desalinate water is Factual Knowledge
7 25 - 64 In the event of a contamination in the water reservoir, a good plan of action would be to call Procedural Knowledge
8 36 - 63 Condensation rates increase with Conceptual Knowledge
9 41 - 55 The movement of water from plants to the atmosphere is called Factual Knowledge
9 14 - 55 Of the 70% water on Earth, the available fresh water is Factual Knowledge
11 39 - 48 The driver for the hydrologic cycle is Conceptual Knowledge
12 23 - 45 A major drawback with water desalination is Conceptual Knowledge
13 38 - 36 The movement of water from the atmosphere to earth is called Factual Knowledge
14 25 - 34 Evaporation rates increase with Conceptual Knowledge
15 46 - 30 The process of water vapor becoming liquid water is called Factual Knowledge
16 20 - 25 Desalination is a process that removes Factual Knowledge
17 32 - 23 A common method to desalinate water is Factual Knowledge
18 - Hardest 18 - 14 Evaporation rates increase with Conceptual Knowledge
       
Water Test (D. I. w/ Online PowerPoint, Review w/ Online Crossword for Vocabulary, & Follow up w/ Hands-on Activities)
1- Easiest 41 - 86 In any water treatment plant, where does the dirty water enter the filter? Procedural Knowledge
1 54 - 86 Any design can be improved by testing and using Metacognitive Knowledge
3 79 - 79 In a water treatment plant, the largest chlorine is used to kill Factual Knowledge
4 39 - 77 PPB stands for what? Factual Knowledge
4 48 - 77 Pure water has a pH of Procedural Knowledge
4 54 - 77 PPM stands for what? Factual Knowledge
7 48 - 71 A simple system has Metacognitive Knowledge
8 64 - 75 The property of carbon that makes it useful for designing your test-filter is its Procedural Knowledge
8 18 - 75 What are the three reservoirs that supply water to the 27,000 residents in Lafayette? Factual Knowledge
8 45 - 75 In water treatment plants, filters commonly used are Procedural Knowledge
11 45 - 73 How do you know if a water source, a reservoir for example, is contaminated? Procedural Knowledge
12 48 - 71 At school, the flow rate of your test-filters is calculated in Conceptual Knowledge
12 50 - 71 In water treatment plants, six layer filters using anthracite, sand and gravel are used because they are Conceptual Knowledge
14 41 - 68 Conductivity measures which of the following? Procedural Knowledge
15 57 - 66 An average human hair is how many microns (micrometers)? Factual Knowledge
15 32 - 66 Coke has a pH of Factual Knowledge
17 61 - 64 What is the primary focus of water treatment? Factual Knowledge
17 34 - 64 Household cleaning products typically have a pH of Procedural Knowledge
17 52 - 64 In a water treatment plant in the United States, flow rate is calculated in Conceptual Knowledge
17 18 - 64 In a water treatment plant, water is filtered through Conceptual Knowledge
21 43 - 63 High density cotton (HDC) filters are Conceptual Knowledge
21 34 - 63

Turbidity if a measure of water's

Procedural Knowledge
23 38 - 59 The property of cotton that makes it useful for designing your test-filter is its Conceptual Knowledge
24 25 - 55 For designing your filters, a dixie cup of carbon will cost $100.00 and a dixie cup of sand will cost only $10.00. The concept that helps you purchase appropriate quantities is Conceptual Knowledge
24 14 - 55 When the flow rate in a water treatment plant decreases below 10% of the designed flow rate. what does the water treatment plant operator do? Procedural Knowledge
26 23 - 54 How do you remove chlorine from the tap water easily? Procedural Knowledge
27 38 - 52 Commonly transported hazardous materials are Factual Knowledge
28 34 - 50 When the water supply is contaminated, what is the a water plant operators priority? Procedural Knowledge
28 46 - 50 During what time of year would backup storage, supply water for four days (Total storage capacity is 13 MG) Procedural Knowledge
30 34 - 48 What, if any, separated the different layers in a water treatment plant filter? (include reason) Conceptual Knowledge
31 50 - 46 What, if any, separated the different layers in a water treatment plant filter? Conceptual Knowledge
31 32 - 46 PPM and PPB are used to describe what? Conceptual Knowledge
33 30 - 43 In a water treatment plant, the largest dose of chlorine is used in the Procedural Knowledge
34 52 - 41 Does a water treatment facility reuse the anthracite (carbon)? Procedural Knowledge
34 34 - 41 The property of sand that makes it useful for designing your test-filter is its Conceptual Knowledge
36 63 - 39 In a water treatment plant, giardia is removed using Procedural Knowledge
37 25 - 38 What is the flow rate of the Lafayette Water Treatment Plant in winter? Factual Knowledge
38 20 - 36 In a water treatment plant filter, the three typical materials used are Factual Knowledge
39 16 - 34 Flow rate refers to Procedural Knowledge
40 32 - 32 In a water treatment plant in the United Kingdom, flow rate is calculated in Conceptual Knowledge
41 30 - 29 What is the flow rate of the Lafayette Water Treatment Plant in summer? Factual Knowledge
41 - Hardest 16 - 29 If the allowable amount of lead in drinking water is 15 PPB, what does this mean? Conceptual Knowledge
*It was a grueling task to manually compile and analyze students' individual test scores - over 7500 unique data points. To make matters worse, when I set these tests, I did not think far enough about the retrieval process and had randomized the order of questions in the pre- and post-tests. Gradually, I retrieved students' individual records online, recoded them on a chart paper for the individual questions for both pre- and post-tests, and entered this data in Microsoft EXCEL and the Statistical Package for the Social Sciences (SPSS) for further analysis.