Writing center assessment

Writing center assessment refers to a set of practices used to evaluate writing center spaces. Writing center assessment builds on the larger theories of writing assessment methods and applications by focusing on how those processes can be applied to writing center contexts. In many cases, writing center assessment and any assessment of academic support structures in university settings builds on programmatic assessment principles as well.^[1] As a result, writing center assessment can be considered a branch of programmatic assessment, and the methods and approaches used here can be applied to a range of academic support structures, such as digital studio spaces.

History

While writing centers have been prominent features in university settings dating back to the 1970s in American higher education, questions remain about the role of the writing center in improving student writing ability.^[2] In discussing the lack of discussion about writing center assessment, Casey Jones compares writing centers to the group Alcoholics Anonymous, claiming that "both AA and writing labs have similar features" yet "the structure of AA complicates empirical research, the desired outcome, sobriety, can be clearly defined and measured. The clear-cut assessment of writing performance is a far more elusive task".^[2] Between 1985 and 1989 the Writing Lab Newsletter, a popular publication among writing center directors, lacked discussion of hard evaluation of writing centers, illustrating the early lack of discussion about assessment in the context of writing centers, instead focusing primarily on advice and how-to guides.^[3] In many cases, writing center directors or writing program administrators (WPAs) are responsible for assessing writing centers, and must communicate these results to academic administration and various stakeholders.^[4] Assessment is seen as beneficial for writing centers because it leads us to assume the professional and ethical behaviors important not just for writing centers but for all higher education.^[5]

Methods

One of the major sources of methods and approaches to writing center assessment comes from writing assessment at large, and programmatic assessment. James Bell argues that directors of writing centers should "turn to educational program evaluation and select general types of evaluations most appropriate for writing centers".^[6] Writing center assessment methods can largely be divided into two major forms of methods: qualitative and quantitative. Qualitative methods are predicated on the desire to understand teaching and learning from the actions and perspectives of teachers and learners, and has largely dominated knowledge making in composition studies, particularly in the last twenty years.^[7] Quantitative methods, meanwhile, stem from the belief that the world works in predictable patterns, ones that might be isolated in terms of their causes and effects or the strengths of their relationships (i.e., correlation).^[7] The use of quantitative methods in writing center contexts leaves room for issues to arise, however, such as data being interpreted incorrectly to support the work of the writing center,^[8] or not choosing appropriate data to measure student success like ACT writing test scores or course grades in first-year composition courses.^[9][10] Some writing scholars endorse quantitative methods more thoroughly than others, and see them as most helpful when reframed in a postmodern epistemology since most writing center directors subscribe to a theory of epistemology that sees knowledge as constructed, tenuous, and relative.^[11] Writing center scholars such as Stephen North group these methodologies into three larger approaches: Reflections on Experience, or looking back of writing center events to help others out in similar situations; Speculation, or a theory of how writing centers should work; and Surveys, or what he champions as enumeration.^[12] Fitting into and blending these methods, several writing studies scholars have published articles on methods used in assessing different elements of writing centers which can be seen in the sections below.

Focus Groups

One method of assessment used in writing center contexts is focus groups. In writing centers, using this method allows writing center directors to collect responses to specific questions and to use the social dynamics of the group to allow for the participants to play off one another's answers, resulting in changes that can be implemented rapidly to make their organization or product more productive.^[13] For writing center assessment, focus groups should be about 7-12 people.^[13]

Surveys

Another common method of assessing writing centers is use of surveys, one of the most common quantitative methods used to gather data in writing centers.^[12] This fits into the notion of enumeration mentioned by North above. Surveys are commonly used to determine information such as student satisfaction with tutoring sessions in the form of a post-session survey, or the confidence of students as writers following their sessions in the writing center.^[11] Due to the nature of tutoring sessions, collecting this type of data in the middle of sessions may prove difficult, and as such, while writing in 1984, North claimed that "there is not a single published study of what happens in writing center tutorials".^[12] Typically, surveys determine the number of students seen, number of hours tutored, reaction of students to center, reaction of teachers to center, and so on.^[12]

Recording Sessions

Recording sessions are seen by some writing center scholars as a viable method of data gathering that answers critiques from the likes of Stephen North about the lack of research regarding what happens during tutoring sessions.^[12] To accomplish this, writing center directors using this method explicitly study what happens during a tutoring session using audio or video tapes and analyzing the transcripts.^[5]

Assessment Plans

Assessment plans are encouraged by some writing center scholars as a means of planning and enacting improvement of centers. Discussed in many forms, several writing center scholars advise directors to develop assessment plans, and provide a series of approaches for doing so. These typically begin with figuring out what to measure, validating these plans, and presenting these findings to the relevant stakeholders.

Developing Assessment Plans

One prominent example of an assessment plan can be seen in the Virginia Commonwealth Assessment Plan.^[5] In discussing the VCAP, Isabelle Thompson lists six general heuristics of program assessment that fit into this context. According to her, program assessment and improvement should be:^[5]

• Pragmatic, intending to be informative and, hence, improve conditions for student learning as well as summative and, hence, justify a program or service.
• Systematic, orderly, and replicable.
• Faculty-designed and led.
• Multiply measured and sourced.
• Mission-driven.
• Ongoing and cumulative.

According to Thompson, in order to develop an assessment plan, writing center directors should:^[5]

1. Prepare a mission statement for the writing center based on the services the center provides and aspires to provide.^[7][14]
2. Develop goals, objectives, or intended educational outcomes for the center^[1][7][15]
3. Determine appropriate assessment methods for the writing center.^[10]
4. Conduct the assessment of the writing center's services.
5. Analyze the results of the assessment and draw conclusions about the results in terms of outcomes and the current strengths and weaknesses of the writing center.
6. Use the results to bring about improvements in the center's services.^[7]

Others like Neal Lerner endorse frameworks for assessment plans for writing centers that consist of heuristics such as determining: who participates in the writing center, what students need from the writing center, how satisfied students are with the writing center, identifying campus environments, outcomes, finding comparable institution assessments, analyzing nationally accepted standards, and measuring cost-effectiveness.^[15]

Validating Assessment Plans

Assessment of writing relies on the concept of validity, or insuring that you measure what you intend to measure.^[16] Chris Gallagher supports developing writing assessments locally, something that many scholars in writing assessment firmly support,^[17][18][19] but adds that we should validate our assessment methods and choices on a larger scale^[20] and suggests the following heuristics for doing so in his Assessment Quality Review Heuristic:

1. Briefly describe the writing program, including curricular and instructional goals, institutional constraints and opportunities (e.g. resources issues, labor conditions, professional development offerings), and student and teacher demographics. Append relevant documentation.
2. Briefly describe the assessment and its relationship, if any, to other assessments conducted in the program. If this assessment is part of an overall assessment plan, append the plan.
3. Answer the following questions about the assessment under review:
   • Meaningful:
     • What are the purposes of this assessment? What are its intended uses? How were these purposes arrived at? Who formulated them? Why and to whom are those purposes significant? How were these purposes made known to students and teachers? How does the content of the assessment match its purpose?
   • Appropriate
     • How is the assessment suitable for this context, these participants, and its intended purposes and uses? How does the assessment reflect the values, beliefs, and aspirations of the participants and their immediate communities?
   • Useful
     • How does the assessment help students learn and help teachers teach? How does the assessment provide information that may be used to improve teaching and learning, curriculum, professional development, program policies, accountability, etc.? Who will use the information generated from this assessment and for what purposes?
   • Fair
     • How does the assessment ensure that all students are able to do and demonstrate their best work? How does the assessment contribute to the creation or maintenance of appropriate working conditions for teachers and students? How does it ensure adequate compensation and/or recognition for the labor required to produce it?
   • Trustworthy
     • How are the assessment results arrived at and by whom? How does the assessment ensure that these results represent the best professional judgment of educators? How does the assessment ensure that the results derive from a process that honors articulated differences even as it seeks common ground for decisions?
   • Just
     • What are the intended and unintended consequences of this assessment—for students, teachers, administrators, the program, the institution, etc.? How does the assessment ensure that these consequences are in the best interest of participants, especially students and teachers?
4. In light of this review, what changes, if any, do you plan to make to this assessment?^[20]

Presenting Findings to Stakeholders

After designing and implementing an assessment plan in writing center contexts, assessment experts advise considering how this information is provided to the rest of the administrators in the university setting.^[21][22] Writing center practitioners recommend that directors of these spaces balance the usefulness of findings in writing center assessments to the improvement of the space itself, and rhetorically appealing to the intended audience.^[7] Some administrators advise using quantifiable data, and connecting that data to important concepts in a given university, like retention, persistence, and time-to-degree, though the important factors to assess and present may vary depending on what a given university administration values.^[22]

In their book Building Writing Center Assessments that Matter, Ellen Schendel and William J. Macauley Jr. provide a set of heuristics for presenting information to stakeholders in the university setting:

• Carefully presenting "good news" and "bad news" in the report, and discussing them ethically.
• Articulating what we do in ways that non-experts will understand.
• Creating a story about the writing center that is supported by the data and that is told clearly in the report.
• Planning and articulating trajectories for our work and using the assessment reports to collaborate with others.
• Using the report to inform our public communications about the writing center.^[7]

Some of this advice, such as the desire to tell a story about the writing center space, clashes directly with advice from administrators like Josephine Koster, who claims that "administrators don't want to read essays. Directors should use bulleted lists, headings, graphs, and charts, and executive summaries in documents sent to administrators".^[22] These clashes appear to support the larger importance placed on local writing assessment practices^[17][18][19] in determining what local administrators may expect.

See also

• Academic administration
• Assessing Writing
• Communication studies
• Composition studies
• Concurrent validity
• Construct validity
• Content validity
• Digital studio
• Educational assessment
• External validity
• Face validity
• Predictive validity
• Rhetoric
• Validity (logic)
• Writing center

References

1. Bell, James H. (2001). "When Hard Questions Are Asked: Evaluating Writing Centers". The Writing Center Journal. 21 (1): 7–28.
2. Jones, Casey (2001). "The relationship between writing centers and improvement in writing ability: An assessment of the literature". Education: 3–20.
3. Bell, James (March 1989). "What are We Talking About?: A Content Analysis of the Writing Lab Newsletter" (PDF). Writing Lab Newsletter. Retrieved 30 October 2015.
4. Gallagher, Chris (Fall 2009). "What Do WPAs Need to Know about Writing Assessment? An Immodest Proposal". WPA: Writing Program Administration. Retrieved 30 October 2015.
5. Thompson, Isabelle (2006). "Writing center assessment: Why and a little how". Writing Center Journal. 26 (1): 33–54.
6. Bell, James H. (1998). "When Hard Questions Are Asked: Evaluating Writing Centers". {{cite journal}}: Cite journal requires |journal= (help)
7. Schendel, Ellen; Macauley, William J. (2012-01-01). Building Writing Center Assessments That Matter. Logan: Utah State University Press. ISBN 9780874218343.
8. Enders, Doug (2005). "Assessing the writing center: A qualitative tale of a quantitative study" (PDF). Writing Lab Newsletter. Retrieved 30 October 2015.
9. Lerner, Neal (September 1997). "Counting beans and making beans count" (PDF). Writing Lab Newsletter. Retrieved 30 October 2015.
10. Lerner, Neal (September 2001). "Choosing beans wisely" (PDF). The Writing Lab Newsletter. Retrieved 30 October 2015.
11. Carino, Peter; Enders, Doug (2001). "Does Frequency of Visits to the Writing Center Increase Student Satisfaction? A Statistical Correlation Study--or Story". Writing Center Journal. 22 (1): 83–103. ISSN 0889-6143.
12. North, Stephen (1984-09-01). "Writing Center Research: Testing our Assumptions". Writing Centers: Theory and Administration. Urbana, Ill.: Natl Council of Teachers. pp. 24–35. ISBN 9780814158784.
13. Cushman, Tara; Marx, Lindsey; Brower, Carleigh; Holahan, Katie; Boquet, Elizabeth (March 2005). "Using focus groups to assess writing center effectiveness" (PDF). Writing Lab Newsletter. Retrieved 30 October 2015.
14. Building Writing Center Assessments That Matter (1 ed.). Logan, Utah: Utah State University Press. 2012-09-06. p. 40. ISBN 9780874218169.
15. Lerner, Neal (2003-10-01). "Writing Center Assessment: Searching for the "Proof" of Our Effectiveness". Center Will Hold (1 ed.). Logan, Utah: Utah State University Press. pp. 58–73. ISBN 9780874215700.
16. Yancey, Kathleen Blake (1999-02-01). "Looking Back as We Look Forward: Historicizing Writing Assessment". College Composition and Communication. 50 (3): 487. doi:10.2307/358862. JSTOR 358862.
17. O'Neill, Peggy; Moore, Cindy; Huot, Brian (2009-03-20). Guide to College Writing Assessment (1 ed.). Logan, Utah: Utah State University Press. p. 57. ISBN 9780874217322.
18. Broad, Bob (2003-03-01). What We Really Value: Beyond Rubrics in Teaching and Assessing Writing (1 ed.). Logan: Utah State University Press. ISBN 9780874215533.
19. Adler-Kassner, Linda; O'Neill, Peggy (2010-08-01). Reframing Writing Assessment to Improve Teaching and Learning (1 ed.). Logan, Utah: Utah State University Press. p. 2. ISBN 9780874217988.
20. Gallagher, Chris (2010). "Assess locally, validate globally: Heuristics for validating local writing assessments" (PDF). WPA: Writing Program Administration. 34 (1): 10–32. Retrieved 30 October 2015.
21. Simpson, Jeanne (2006-03-23). "Managing Encounters with Central Administration". The Writing Center Director's Resource Book. Mahwah, N.J.: Routledge. pp. 199–214. ISBN 9780805856088.
22. Koster, Josephine (2003-10-01). "Administration Across the Curriculum: On Practicing What We Preach". Center Will Hold (1 ed.). Logan, Utah: Utah State University Press. pp. 151–165. ISBN 9780874215700.