Improving Comprehension Online Project, 2005-08 Designing and testing

Improving Comprehension Online Project, 2005-08 Designing and testing

Improving Comprehension Online Project, 2005-08 Designing and testing a universally designed strategic digital literacy environment for diverse learners Bridget Dalton, Vanderbilt University A goal 2 development award Patrick Proctor, Boston College to CAST, Inc.

IES Research Conference Washington, DC ~ June 11, 2008 Research team Bridget Dalton (Co-PI), Vanderbilt University, and Elaine Mo, Kristin Robinson, Ge Vue, Mary OMalley, & Boris Goldowski, CAST, Inc. Patrick Proctor (Co-PI), Yi-Chien Li, & Kevin OConnor, Boston College Catherine Snow (Co-PI), Paola Uccelli, Sabina Neugebauer, Lorena Landeo Schenone, Harvard Graduate School of Education

School partners: 3 semi-urban and 1 urban school in northeastern Massachusetts Project goal To develop and test a universally designed (Rose & Meyer, 2002) strategic digital reading approach (Dalton & Proctor, 2007) to improving reading achievement of 5th grade students, including

bilingual students and struggling readers Multiple perspectives required Reading comprehension Bilingualism 2LA Vocabulary icon Digital literacy environments

New literacies Universal Design for Learning Universal design for learning (Rose & Meyer, 2002) Design for the broadest range of learners from the beginning; avoid retrofitting Provide multiple means of Representation Expression Engagement

Assumptions Shift to universal design perspective Attention to diversity and individual difference benefits individual and society New literacies, while more complex, are more flexible and inclusive potential to level the playing field for those who have not fared well with print literacy Rand Reading Study Groups (2002) reading comprehension heuristic

reader text activity comprehension Sociocultural context Strategic Digital Reading (Dalton & Proctor, 2007) reader text

activity comprehension Sociocultural context Comprehension in a new literacies landscape: Strategic digital reading reader text activity comprehension Sociocultural context

How does ICON support diverse learners in relation to Representation? Expression? Engagement? What is unique for ELLs? What features/supports are essential for some; good for many/all? Iterative design, formative feedback and testing Y1. Develop Vocabulary

Y2. Compare Vocabulary, Strategies & Combo Versions Y3. Compare Combo Vs. Control ICON optimal prototype (Yr. 3) Embedded Strategies Spanish language support

Coaches Level 1 coaches provide text-specific models and think alouds. As skill increases, students select strategies and coaches provide generic think-alouds. Vocabulary: Connect It!

Vocabulary: Language Alert More than 60% of the power words are Spanish English cognates Vocabulary: Web It! Vocabulary: Caption It! All 3 years/studies: Feasibility, appeal & usability

Teachers and students view ICON as a helpful reading tool, easy to use, & engaging Technical support required; bandwidth issue Variation in teacher enactment of ICON suggests need for additional study English proficiency levels influence ways in which students use ICON and extent to which additional support is needed Peer collaboration one means of support Increased sensitivity to learner (needs, use of system, performance) is likely to benefit all Y1 Study of Semantic Depth 35 students, 24 bilinguals (Spanish and other lowincidence languages), 11 English monolinguals

Oral language skills (WJ picture vocab+listening comprehension) Reading skills (WJ passage comp + MCAS ELA score) Average semantic depth score for 8 target words (Anxiously, Bitter, Dense, Grasp, Ignore, Menacing, Powerless, Relieved) Yr. 1 study of semantic depth (Proctor, Uccelli, Dalton, & Snow, in press) Effective teaching and learning activities targeted for further analysis: Caption-It Semantic depth was a significant predictor of

reading performance, mediated by interaction with English oral language proficiency Figure 1. Effect of semantic depth predicting reading latent variable as a function of oral language proficiency 2.5 Reading Latent S core 2 1.5 1 10% Oral Lg

0.5 50% Oral Lg 90% Oral Lg 0 -0.5 -1 -1.5 0 1 2

3 4 5 Sema ntic De pth Score *No effect for language status (bilingual vs. monolingual) What did we learn from Y1 vocabulary study? Caption It: Encouraging but preliminary As both an

activity and an assessment, it appeared to reduce the monolingual/bilingual gap in students performance, though oral language was heavily implicated Semantic Depth: Promising but far from final Semantic depth showed a positive association with reading comprehension, beyond the contribution of decoding and oral language skills Semantic depth seems to play a more prominent role as oral language skills improve Semantic depth refers to a cluster of skills: associations among these and with other dimensions of vocabulary depth need to be

explored. Year 2 study: Strategies vs. vocabulary vs. combo Very hard to find research that compares effects of vocabulary versus comprehension instruction Likely because the two are so strongly intertwined Given that vocabulary is a primary focus of instruction for ELL students, we found this question intriguing, and

asked 2 basic questions: 1. 2. Does assignment to condition (Vocabulary-only, Strategy-only, Combo) affect students performance on standardized and researcher-developed measures of vocabulary and comprehension? Do the effects vary by language status (monolingual, SpanishEnglish bilingual, other bilingual)? Our hypotheses For standardized and researcher-developed vocabulary:

Combo > Vocabulary > Strategy For standardized and researcher-developed comprehension: Combo > Strategy > Vocabulary Y2 study of vocab vs strategies: What matters and for whom? (Dalton, Proctor, Uccelli, Mo, & Snow, in preparation) 106 students, 21 Spanish-English bilinguals, 17 other-English bilinguals, 68 monolinguals in 6 classrooms, 3

districts Random assignment to condition (vocabulary, strategy, combination) 14-week intervention Pre-post standardized testing, embedded vocabulary and comprehension quizzes Y2 effect size overview What did we learn from Y2 study of vocabulary vs. reading strategies vs. combined? Overall, hypotheses held, and combination version showed strongest results across standardized and researcher measures

Did the small sample size mask learner by treatment interactions? Theoretically, an interaction between student characteristics (reader type and/or language status) would make sense Sample size may be too small Student controls access to support and may not be making good decisions about when and how to use support Thresholds of language proficiency Year 3 study: Combo vs control Having established general effectiveness, time to move to a comparison between treatment and control using optimal

version of ICON Quasi-experimental study 12 classrooms, classrooms assigned randomly to treatment or control condition, n = 227 (108 control, 119 intervention; 10.5% other bilinguals, 48.5% Spanish-English bilinguals, 41% English monolinguals)

For intervention group, 2 x 50 minutes per week, for 16 weeks For control group, across the three districts, standard literacy curriculum included reading strategies focus, but limited vocabulary instruction Initial training of teachers and students by research team, gradual release of ICON prototype teaching responsibility

Analyses conducted at student level, randomization at teacher level Y3 measures Gates-MacGinitie reading vocabulary and comprehension subtests pre- and postintervention Aprenda reading vocabulary - pre Researcher-designed breadth of vocabulary (targeted words) post-intervention 20-item multiple choice assessment Reseacher-designed depth of vocabulary, post-intervention 5-item definition, drawing, & captioning assessment

Y3 results General results: No effect of condition on standardized measures; significant voc and comp gain for both groups Strong effect of condition on researcher developed measures Condition Depth/Breadth Standard

Vocab and comp Condition effects on ICON vocabulary breadth Significant effect of condition on ICON voc. Breadth F(1,205) = 56.62, p < .001 Significant difference between Spanish bilinguals and English monolinguals (t = 5.1, p < .001) Strong readers significantly outperform average (t = 5.1, p

< .001) and struggling (t = 12.6, p < .001) No interactions by language or reader status and condition 80 70 60 50 Control 40

Experimental 30 20 10 0 Mono BilSp BilOth Condition effects on ICON vocabulary depth

Exp. significantly outperform Control on vocabulary depth F(1,224) = 101.4, p < .001 English monolinguals significantly outperform Spanish bilinguals (t = 5.3, p < .001) and non-Spanish bilinguals (t = 2.2, p < .05) Strong readers significantly outperform average (t = 5.0, p < .001) and struggling (t = 9.4, p < .001) readers No interactions by language status, BUT average-reader X condition

interaction (p = .048) Sum English Vocabulary Depth Score Spanish-English bilinguals and ICON depth of vocabulary 8 7 6 5 4 3

control vocdepth intervention vocdepth 2 1 0 400 500 600 700

Aprenda Vocabulary Score For intervention Spanish-English bilinguals, Spanish Vocabulary scores explained English Vocabulary Depth performance, after controlling for condition and prior English proficiency. Conclusions and next steps Continue to analyze Year 3 data Worklog responses, multimodal retellings, student feature use, teacher use of feedback support. For whom does this intervention work best? Goal 3: effects for Spanish-English bilinguals and struggling readers are intriguing

Goal 2: Work for transfer. Design for increased sensitivity to learner characteristics, especially language proficiency. Distal effects on standardized measures

Recently Viewed Presentations

  • Humanistic Personality Theory

    Humanistic Personality Theory

    * * HUMANISTIC PERSONALITY THEORY WHAT IS IT? EXAMPLE: Life's goal is to become the best you can be! : biological push to fulfill one's potential Carl Rogers and Humanism Often, we form images of ourselves…
  • Chapter 4-3

    Chapter 4-3

    Present Worth. Future Worth. Annual Worth. Internal Rate of Return . External Rate of Return. Benefit/Cost Ratio. Capitalized worth yields the same recommendation if the planning horizon is infinitely long or equal to a least common multiple of lives of...
  • Welcome to Back to School Night

    Welcome to Back to School Night

    Important Reminders. Friday Folders. On Friday of each week, please look for a folder to come home. I will use it to send home important notices from the office, graded work, and any class news.
  • Savoir-Être


    Étienne Marchand Cégep François-Xavier Garneau Mars 2006 [email protected] À la question: « Qu'est-ce que l'intelligence émotionnelle » posée à John Mayer par la revue Psychology Today, « L'intelligence émotionnelle, c'est un ensemble d'habiletés mentales qui vous aident à reconnaître et...
  • Today&#x27;s Program -

    Today's Program -

    e.g. Phillips Exeter Academy, March 2, 2017 report. See also: Revil v. Coleman. July 19, 2016, Appeals Court of Massachusetts. Factually truthful press release that school employee was on leave pending investigation of sexual misconduct when ultimately no charges were...
  • Section 11.1: Solving Linear Systems by Graphing

    Section 11.1: Solving Linear Systems by Graphing

    Create a dot plot for the data set using an appropriate scale for the number line. Determine whether the extreme value is an outlier. Suppose that the list of salaries from the Explore is expanded to include the owner's salary...
  • On-Orbit Propulsion System Performance of ISS Visiting Vehicles

    On-Orbit Propulsion System Performance of ISS Visiting Vehicles

    Developed in a joint effort by MOD, ISS engineering specialists, and vehicle owners. Integrate relevant engineering constraints. ... MLM (scheduled for launch in Dec 2013) Pitch/Yaw Attitude Control. Service Module. Progress at SM-Aft. ATV at SM-Aft.
  • Darren Strange -

    Darren Strange -

    "Green IT has reached critical mass. Virtually. ... energy-efficient, less-carbon-intensive business models, enterprises, value chains, products and services with reduced environmental impact." ... Centralize client/server power management with Group Policy. Desktop power management.