Expected Learning Outcomes
Upon completion of the Ph.D. Program in Second Language Acquisition and Teaching, students will be able to:
- Demonstrate broad knowledge of applied linguistics as related to Second Language (L2) teaching and learning.
- Think critically and independently about topics in Second Language Acquisition and Teaching in all areas of SLAT which are: Sociocultural Dimensions of L2 learning (L2 use), Linguistic Dimensions of L2 learning (L2 analysis), Cognitive Dimensions of L2 learning (L2 processes), and Instructional Dimensions of L2 learning (L2 pedagogy and program administration).
- Define and design a research area and plan.
- Conduct independent research using all acceptable and ethical considerations including adherence to all human subjects regulations.
- Communicate research findings orally and in writing.
These assessment activities were developed by the SLAT Chair and staff, with approval by the SLAT Executive Council and the University of Arizona Office of Instruction and Assessment. SLAT requires that students complete a first year qualifying exam, comprehensive examinations, Dissertation Proposal, and a defense of their Ph.D. dissertation. Students also fill out a final self-report of their skills as part of an exit survey. Assessment data is collected for each learning objective as outlined in the matrix below:
| Assessment Activities | Outcome 1: Knowledge of Applied Linguistics | Outcome 2: Think critically and independently | Outcome 3: Define and design a Research Plan | Outcome 4: Conduct independent research | Outcome 5: Communicate research findings |
| First Year Qualifying Exam | X | X | |||
| Written and Oral Comps Exam | X | X | X | ||
| Dissertation Proposal Presentation and Defense | X | X | X | X | X |
| Written Dissertation and Oral Defense | X | X | X | X | X |
| Exit Survey | X | X | X | X | X |
As of the close of summer 2020, SLAT has collected data for five academic years: AY 2016, 2017, 2018, 2019, and now 2020. Upon initiation of assessment data collection in AY 2016, SLAT anticipated having measurable data and outcomes by the fifth year of data collection. Not only do we now have a robust number of responses collected over time, but since SLAT is a five-year program, this fifth year of data collection gives us our first glimpse into a data set that measures a student cohort that entered and exited SLAT during the time we were collecting data. A total of nine students who began the Ph.D. program in 2015-2016 graduated in 2019-2020.
To collect assessment data, faculty are emailed a survey with the rubric questions as students reach each milestone: Qualifying Review, Comprehensive Exams, Proposal Defense, and Ph.D. Defense. The students were asked to complete an exit survey at the completion of their degree. The participants gave a score of 1 to 5 based on the following criteria:
1-Needs improvement
2-Developing
3-Progressing
4-Achieving
5-Excellent
We then compile these scores into a table that allows us to observe two measures: 1) average score across all students for each student learning outcome at each milestone, 2) the number of individual responses where at least 80% of the responses scored at 4 or higher. The average score allows us to observe trends in how our students are performing overall in the measured skills at each milestone. For the second measure, SLAT defines success for any skill as 80% of responses rated 4 or higher. This measure allows us to see if there are any areas where we do not have at least 80% of responses indicating that our students are achieving that skill at that particular milestone. If our average score is low in any area, or if there are milestones where our faculty have not assessed 80% of our students as achieving that skill, then the SLAT Executive Council will meet to determine how to change our preparation and teaching to better prepare our students to meet that milestone.
All files (images) are available for download
Below: Numbers highlighted in yellow indicate where fewer than 80% of responses came in at 4 or higher (achieving or excellent) for these measures. See discussion below for more details.
Our first year of assessment data collection took place during the 2015-2016 academic year. At the end of that cycle, it was determined that our results were too preliminary, and that It would require at least 5 years of data collection to adequately determine what changes are required in response to our findings. Now, at the end of the 2019-2020 academic year, we have completed a full five years of data collection and can make some observations.
Notes and Observations for the 2019-2020 Assessment Year
COVID-19: SLAT students were affected by COVID-19, as were students around the world. Many advanced international students chose to return to their home countries for personal and health reasons. Others faced funding insecurity due to the reduced university budget. Far more faced the reality that their data collection plans had to be interrupted, revised, or completely re-envisioned. These responses to the coronavirus pandemic slowed the progress to degree for a number of our students. SLAT does not expect the pandemic to change the performance of our students in any of these measures of success upon completion of the program. The skills measured here are required for a Ph.D. graduate, and, on an individual level, students must demonstrate these skills to graduate.
Student confidence: Across the board, students score themselves very highly in all skills measured in the SLAT assessment rubrics. SLAT collects this exit survey immediately after students post their degree, so some of this confidence may be due to completion euphoria. One way to correct for this would be to attempt to collect a follow-up self-assessment 1 or more years after graduation. Since post-graduation success is beyond the purview of in-program assessment, and since job placement is a measure of post-graduation success SLAT already collects, the decision was made not to create a follow-up survey. We are happy to be graduating skilled, confident Ph.Ds.
Data Averages from 2016-2020: In exploring the average numbers collected for all SLO measures at all milestones for the past five years, the SLAT Executive Council found no alarming numbers. There are some expected trends such as an increase in overall knowledge of applied linguistics at the comprehensive exam phase, and some surprising trends such as a slightly slower rating for a student’s ability to conduct independent research at the dissertation phase compared to the proposal phase, but the numbers are not statistically significant. All scores average out to between 4-5, which is acceptable to the committee.
Specific Measures of Success: Based on our updated success measure which defined success as 80% of students scoring four or higher on the rubric, SLAT did not meet our definition of success on the following SLOs:
- Qualifying Review Rubric question #2: “The paper is well-written,” and #3: “The paper demonstrates the student’s ability to convey ideas in a concise manner. “
- Dissertation Proposal Rubric question #3: “The research design for studying the problem is appropriate and data collection is appropriate.”
Response to the Qualifying Review Data: Upon discussion, the SLAT Executive Committee determined that the Qualifying Review rubric responses will not require a specific programmatic response. This measure is collected at the end of the student’s first semester in the Ph.D. program, upon completion of the introductory survey class. It is not surprising that students cannot yet display Ph.D.-level writing skills at the end of the first semester in the program. Since students do meet these measure in all future milestones, this indicates learning is taking place, and the SLAT program is teaching students appropriate writing skills at the doctoral level. Some explicit writing instruction may be included in the first-semester SLAT Proseminar, but the SLAT Executive Committee was not concerned that these milestones had not yet been met at the end of the students’ first semester in the program.
Response to the Dissertation Proposal Data: Upon discussion, the SLAT Executive Council determined that these data are problematic. Students are not meeting the program’s definition of success in developing an appropriate research design and determine the appropriate data to collect at the stage when they are presenting their dissertation proposal. It should be noted that students do have an editing and revision process to complete after the presentation of their proposal, so students have the opportunity to address these problems before dissertation data collection begins. Nonetheless, ExCo members decided it was problematic that fewer than 80% of faculty members determined that students could perform this task at the time of their dissertation proposal presentation.
Future steps: The SLAT Executive Council will meet in Spring 2021 to discuss specific plans to address student preparation in research methodology, planning, and design. Possible responses include changing research methodology requirements, adding another research class, creating one or more workshops for SLAT students on the topic, etc. Changes will be implemented as soon as reasonably possible, hopefully by the 2021-2022 academic year.
Assessment data collection for the 2020-2021 academic year is ongoing.