This essay aims to provide an overview of the challenges of accounting for students with disabilities (SWDs) and English learners (ELs) in the evaluation of mainstream teachers. The authors focus on the two prominent indicators of teaching quality—classroom observations and value-added scores. The authors conclude with recommendations for states and districts to ensure that teacher evaluation systems adequately and fairly account for these students.