Score Like A ProAZELLA Writing Sample Tests Stages I-III

Score Like A Pro AZELLA Writing Sample Tests Stages I-III Jonathan Wolfe Marlene Johnston Gabriela Finn OELAS Conference December 9-10, 2015 1

Todays Agenda Introduction Fundamentals of Scoring Rubrics: Applying criteria within holistic scoring Stage III Stage II Insider Tips Connect to the Classroom

2 Relationship to PearsonPerspective 3 Finding the Sample Tests on the

AZELLA Homepage AZELLA Homepage Resources Tab Stage I V Sample Tests http://www.azed.gov/standards-development-assessment/arizona-english-language-learner-assessment-azella/ 4

Stage III Prompt 5 Stage III Students write responses in Answer Documents 6

Score Using the AZELLA Rubrics 7 Extended Response Writing Stages III V

0-5 points 8 Extended Response Writing Stages III V (continued) 9

Score Based on AZELLA Rubrics A rubric provides criteria for assigning scores to student responses. Adhering exclusively to the criteria of the rubric, provides fair, consistent, and predictable scores. Rubrics are based on the ELP Standards & AZELLA Performance

Level Descriptors. 10 Score Based on AZELLA Rubrics Official AZELLA Reassessment scoring uses AZELLA rubrics. AZELLA rubrics are different than AIMS or AzMERIT. AZELLA rubrics are posted on the Assessment webpage and in the

Teachers Edition of Sample Tests. AZELLA rubrics are used in the PearsonPerspective training for scoring Placement Tests. 11 Holistic Scoring Consider all writing elements without focusing on any single criteria. Key elements must be present in the

writing. The overall score is the one that best describes the response as a whole. The whole is more than the sum of its parts. 12 Scoring Criteria (III-V) During training, we look at each of the criterion to better understand them.

When scoring student responses, we look at the criteria as a whole and weigh and balance them in the student response. Complexity (Standards 2 and 4 of the ELPS) Sentence Structure/Grammar Capitalization & Punctuation (Standard

2 of the ELPS) (Standard 2 of the ELPS) Spelling (Standard 2 of the ELPS) Organization (Standard 4 of the ELPS) Word Choice (Standard 4 of the ELPS)

Connection to Prompt 13 Complexity Varied sentence structures, simple and complex An idea supported with relevant reasons, examples, and/or details; characters and setting as appropriate. An increased number of sentences representing different ideas is an

indication of complexity. Grammatical structures, both simple and complex, are used with accuracy. With the addition of words and phrases (e.g., relative clauses and prepositional phrases), simple sentences take on some complexity. Advanced structures such as subordinate clauses (beginning with because, when, if, etc.) may also be found in upper-level ELL writing. 14

Sentence Structure Grammar and Usage Command of grammar (including syntax) and usage with few or no errors. Typical sentence markers, a capital letter at the beginning and a period at the end, can be absent in ELL writing. It is therefore necessary to look for the elements of a basic English sentencea subject, a verb, and maybe an object. When these elements are present and in the correct order (subject-verb-object), the student has produced a sentence. When looking for subject-verb agreement, consider the whole verb phrase,

which could include a helping verb + a main verb. 15 Capitalization, & Punctuation Correct capitalization and end punctuation; infrequent errors in internal punctuation. A sentence that is recognizable without capitalization and punctuation is considered a sentence (e.g., dad sed I am

Bize [Dad said, I am busy]). However, capitalization and punctuation remain important considerations in scoring. In upper-level writing (score points 3 and 4), there are usually fewer spelling and mechanics errors than at lowerlevel writing (score points 1 and 2), but these types of errors may still be found at all levels of ELL writing. 16 Spelling

Infrequent and/or minor spelling errors that do not impede the readers comprehension. One of the most visible and pervasive characteristics of ELL writing at all ability levels is phonetic spelling. Spelling, when it does not interfere with comprehension, and when a word is easily recognizable as an English word, is only one component in the total evaluation of student writing and should not take on excessive importance. 17

Word Choice Consistent use of descriptive words and phrases that address the purpose, topic, and audience using academic language as appropriate. Another feature that differentiates upper- from lower-level ELL writing is word choice. In upper-level writing, choice of vocabulary will become more precise and descriptive. For example, a student in the fifth grade might be used instead of a girl.

As a students writing ability develops, a few idiomatic words and phrases may be used. It is not uncommon therefore for words like cool and stuff, which are more appropriate for spoken language, to be used even in upper-level ELL writing. 18 Organization Idea(s) supported with relevant reasons and details; organization uses transition

words and guide reader from beginning to end Recognizable organization that allows the reader to follow the text from beginning to end. Logical progression of ideas, linking words (so, then, next, therefore, while, finally, although, because, since, due to, for instance, etc.), contribute to organization of ELL writing. In upper-level writing, students are capable of expressing more ideas. The ability to organize the ideas becomes one of the characteristics which can

help differentiate between score point 3 and 4. 19 Connection to Prompt Prompt/question addressed either in a statement or with clear implication. A response with no connection to the prompt is not acceptable. A response that addresses and develops only one aspect of the prompt is acceptable.

20 Applying the Criteria Holistically Score point above Bullets may match some elements in the response. Score point below Bullets may match some elements in the

response. Score assignment Which score point best fits the response? Most elements supported in the score point May satisfy bullets from other score points 21 What Are Annotations?

An annotation explains the score of each anchor paper using language from the rubric. 22 Scoring Activity 1 23

Stage III Prompt 24 Extended Response Writing Stages III-V

(score points 3-5) 25 Extended Response Writing Stages III-V continued

26 Stage III Response #1 27 Score and Annotation (#1) Score Point 2 The response contains ideas that are vague and simplistic. Vocabulary is limited and repetitive.

The response is not a score point 1 because ideas are clear and organized. The response is not a score point 3 due to repetitive vocabulary and sentence structure. 28 Stage III Response #2 29

Score and Annotation (#2) Score Point 3 Response exhibits some of the traits of a score point 4; including a main idea with significant details. However, communication is impeded by errors in grammar, spelling, punctuation, sentence structure, and capitalization. The response is not a score point 2 as the ideas are beyond vague and simplistic and there is clear evidence of an organizational structure. The response is not a score point 4 due to the density and variety of errors in standard English conventions.

30 Stage III Response #3 31 Score and Annotation (#3) Score Point 4 Main idea is evident and development is present. There are minor errors in spelling and grammar that do not impede communication (decises for

decides, Than for then, ben for bin, etc). The response is not score point 3 because details are connected and developed; and errors do not impede communication. The response is not a score point 5 because there is a limited use of descriptive words and phrases that address the audience and purpose. The number of errors in conventions, while minimal, does not display command of grammar. 32 Scoring

Activity 2 33 Stage II Prompt 34 Stages I-II Students write responses in Test Books

35 Extended Response Writing Stages I II 0-3 points

36 Scoring Criteria (Stages I-II) Ideas and Details are Present Sentence Structure Subject/Verb Agreement Capitalization & Punctuation Spelling Connection to Prompt

37 Stage II Response #1 38 Score and Annotation Score Point 2 This response contains ideas and details. Capitalization and punctuation are correct. There is inconsistent

subject-verb agreement (The boy and the girl is playing a game). Spelling errors begin to impede the meaning of words (talking instead of taking). Not a 3 as subject-verb agreement errors are evident and spelling errors begin to impact meaning. 39 Stage II Response #2 40

Score and Annotation Score Point 3 This response presents ideas and details that directly relate to the prompt. Sentences show subject-verb agreement. There is sentence variety that includes the use of transition words. Use of phonetic spelling does not impede the meaning of words. Not a 2 as the response has subject-verb agreement, ideas and details are not vague, and the errors in

conventions are minimal. 41 What Are Anchor Papers? Student responses used to define the range of performance levels within each score point on the AZELLA rubric. Anchor sets contain exemplar student responses that clarify the rubric. From the lowest possible score to the highest one,

they demonstrate different approaches and levels of achievement within a score point. Anchors do not represent every possible way to achieve each score point. 42 Where are the Sample Test Anchor Papers and Annotations? The Anchor Set can be found

on the AZELLA webpage with the associated Stages Sample Test materials. 44 Insider Tips 45 The Pros Practice Bias-free Scoring

Do not allow your scores to be influenced by preconceived ideas you may have about what constitutes good writing or what an ELL student should be able to demonstrate in a writing assessment. Do not be influenced by your personal history with a student. Do not be influenced by the gender, socioeconomic status, personality, political or religious viewpoints, or the emotional content evidenced in the students writing responses. Do not be influenced by the response appearance such as style, tidiness and size.

46 Connect to the Classroom 47 Using the Writing Responses to the AZELLA Sample Test to Improve Learning

Remember! The scoring rubrics are Standards-based To score: look at the responses using a holistic approach To improve instruction: take discrete elements into consideration to Identify strengths (e.g., minimal errors in capitalization) Isolate weaknesses (e.g., spelling errors impede comprehension - homophones) Give feedback and set instructional goals

48 49 AZELLA Writing Rubric Checklist Wheel Located at http://www.azed.gov/assessment/azella/ under the Resources tab 50

Questions? Thank you!

Recently Viewed Presentations

  • PASTORAL DA SOBRIEDADE Assemblia Nacional Campina Grande do

    PASTORAL DA SOBRIEDADE Assemblia Nacional Campina Grande do

    PASTORAL DA SOBRIEDADE Assembléia Nacional Campina Grande do Sul, PR - 23/09/2006 Mobilização de recursos para projetos sociais Rogerio Dardeau de Carvalho
  • 8.4 - Co-dominance & Multiple Alleles

    8.4 - Co-dominance & Multiple Alleles

    In the last couple of lessons, we dealt mainly with simple situations where the alleles were either: Dominant. Or recessive. Today's lesson will focus on more 'complex' situations of inheritance, where we will deal with: Co-dominance. Multiple alleles.
  • Revisiting "The Evolution of Reciprocity in Sizable Groups"

    Revisiting "The Evolution of Reciprocity in Sizable Groups"

    Revisiting "The Evolution of Reciprocity in Sizable Groups" Masanori Takezawa (Tilburg University, the Netherlands) Michael E. Price (Brunel University, UK) Boyd & Richerson (1988) "The Evolution of Reciprocity in Sizable Groups" Argued that reciprocity per se cannot explain the evolution...
  • (Non)Action Has Consequences: Reasons and Ways for Young

    (Non)Action Has Consequences: Reasons and Ways for Young

    Malthusian something (correction, crisis, disaster) Slow adjustment to a new equilibrium. Right now - neither here, nor there. Rigid pension systems. Modigliani's life-cycle hypothesis. Retirement. Older population => retirement costs. Retirement age.
  • Structural and Social Action Theories

    Structural and Social Action Theories

    Human behaviour is largely determined by society One sees society as being the product of human activity. Human behaviour is seen as critical in the development of society. Examples of Structural Theories Despite being very different, Functionalism and Marxism examine...
  • Washington Sales Tax - Financial Management

    Washington Sales Tax - Financial Management

    Retail sales tax is Washington's principal tax source. Businesses making retail sales in Washington collect sales tax from their customer. Retail sales tax includes the state and local components of the tax. State portion is 6.5 percent and local varies,...
  • Automatic Wrapper Induction: Look Mom, no hands! Thomas

    Automatic Wrapper Induction: Look Mom, no hands! Thomas

    Semi-structured web pages are laid out in a pattern that is regular (repeating multiple times) and contiguous (few gaps among fields or records). A limited number of patterns are consistent with each web site and can be discovered automatically.
  • The SAL Integrated Hybrid Cognitive Architecture

    The SAL Integrated Hybrid Cognitive Architecture

    The SAL Integrated Hybrid Cognitive Architecture Christian Lebiere, Niels Taatgen & John Anderson - Carnegie Mellon University Randall O'Reilly - University of Colorado David Jilk - eCortex 4/7/08 Neuromorphic Computing Workshop * Hybrid Cognitive Architectures Cognitive invariants ...