
{"id":6928,"date":"2009-09-03T12:05:18","date_gmt":"2009-09-03T20:05:18","guid":{"rendered":"http:\/\/www.carrborocitizen.com\/main\/?p=6928"},"modified":"2009-09-03T12:05:18","modified_gmt":"2009-09-03T20:05:18","slug":"test-scores-are-up-kind-of-maybe","status":"publish","type":"post","link":"https:\/\/www.ibiblio.org\/carrborocitizen\/main\/2009\/09\/03\/test-scores-are-up-kind-of-maybe\/","title":{"rendered":"Test scores are up? Kind of? Maybe?"},"content":{"rendered":"<p>By Steve Peha<\/p>\n<p>This past school year, almost every school in North Carolina made test score gains in reading, math and science. So what accounted for this? Did we:<br \/>\n(A) implement new instructional methods?<br \/>\n(B) lengthen the school day or the school year?<br \/>\n(C) buy more computers?<br \/>\n(D) none of the above.<br \/>\n<!--more--><br \/>\nThe correct answer is \u00e2\u20ac\u0153D: none of the above.\u00e2\u20ac\u009d But something important did change, and it accounted for a significant percentage of the gains.<\/p>\n<p>On Aug. 6, 2009, the Department of Public Instruction put out a press release entitled \u00e2\u20ac\u0153ABCs Results Released For 2008-09 School Year;<br \/>\nRetesting Benefits K-8 Schools.\u00e2\u20ac\u009d Whoa there! What\u00e2\u20ac\u2122s this retesting? And how much did K-8 schools benefit from it?<\/p>\n<p>Retesting is just what it says it is: Kids who don\u00e2\u20ac\u2122t pass the test the first time get to take it again. The state uses only the highest of the two scores for official measurement purposes. So how big was the retesting effect? Here\u00e2\u20ac\u2122s what the North Carolina Department of Public Instruction said in the Aug. 6 release:<\/p>\n<p>\u00e2\u20ac\u0153In mathematics, the retest moved the percent proficient from 72 percent to 79.9 percent. In reading, the percent proficient went from 58.5 percent to 67.5 percent. In science, the change was 10 percentage points \u00e2\u20ac\u201c from 54 percent proficient before retesting to 64 percent with retesting included.\u00e2\u20ac\u009d<\/p>\n<p>Looks like the state improved by roughly eight points in math, nine points in reading and 10 points in science, simply as a result of giving kids a second shot at the test. With state scores in the 50s, 60s and 70s, this increase seems quite significant to me. So what does this mean?<\/p>\n<p>It could mean that the tests are not very reliable. It could mean that prior familiarity with the test produces a significant advantage. It could mean that test takers were more comfortable or more diligent the second time around. It could mean a little bit of all three. <\/p>\n<p>One thing these retest increases do not mean, however, is that kids got any smarter between test #1 and test #2. So what\u00e2\u20ac\u2122s going on here?<\/p>\n<p>When the tests were first constructed over a decade or ago, they were intentionally created to be very easy to pass. In fact, according to a variety of comparison studies, North Carolina\u00e2\u20ac\u2122s tests were almost the easiest in the nation. As a result, kids passed them easily and scores zoomed up. <\/p>\n<p>Then, over the last three years, the reading and math tests were made more difficult. This caused scores to go down.<\/p>\n<p>Now retesting has been introduced. Obviously, allowing retesting makes the tests easier to pass. What does all this mean? It means we now have harder tests that are easier. What does that mean?<\/p>\n<p>I have no idea.<\/p>\n<p>At this point, it is virtually impossible, even with detailed test score data, for anyone to know if or by how much students have improved in reading and math since testing began. And now that retesting will be included this coming year, and probably in all years thereafter, it seems that we will be forever wondering whether our kids are getting smart or just getting smart about taking tests.<\/p>\n<p>When all this testing began, we were promised two important things: that we\u00e2\u20ac\u2122d know whether our kids were getting smarter and what to do about it if they weren\u00e2\u20ac\u2122t. When tests start out easy, then are made harder, then are made easier again, instead of getting these two things, what we get is confusion.<\/p>\n<p>And yet, DPI thinks just the opposite:<\/p>\n<p>\u00e2\u20ac\u0153State Board of Education Chairman Bill Harrison said that this year\u00e2\u20ac\u2122s data give North Carolinians a clear picture of how well schools are performing.<br \/>\nBy including retest results, we can see the number of students who are able to score proficient and demonstrate they are ready for the next grade level. I am pleased that we can give schools credit for this accomplishment.\u00e2\u20ac\u009d<\/p>\n<p>A clear picture of how well schools are performing? This situation seems more than a little muddy to me. And since I couldn\u00e2\u20ac\u2122t find out why the retesting option was offered this year, I\u00e2\u20ac\u2122m forced to use my own student-achievement sleuthing skills.<\/p>\n<p>It seems to me that the kids who would benefit most from retesting would be those who just missed passing by a few points. How many points exactly? Probably the number of points in the test\u00e2\u20ac\u2122s Standard Error of Measurement, or SEM. I can imagine a savvy statistician noticing after the tests got harder that many kids fell just below the passing level but within the SEM. Why not give those kids a second shot, when we know that their scores are likely to go up just enough to have them pass?<\/p>\n<p>So what do we know? Obviously, retesting inflates scores. And obviously, the state feels this somehow gives \u00e2\u20ac\u0153a clear picture of how well schools are performing\u00e2\u20ac\u009d? But does it? Really? As someone who has followed state test scores pretty closely for the last five years, I don\u00e2\u20ac\u2122t have a clear picture of how well schools are performing. What I have is a clear picture of how easily test scores can be manipulated, whether kids are learning or not.<br \/>\n<em><br \/>\nSteve Peha is founder and president of Carrboro-based Teaching That Makes Sense Inc.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>By Steve Peha This past school year, almost every school in North Carolina made test score gains in reading, math and science. So what accounted for this? Did we: (A) implement new instructional methods? (B) lengthen the school day or the school year? (C) buy more computers? (D) none of the above.<\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[27,361],"class_list":["post-6928","post","type-post","status-publish","format-standard","hentry","category-opinion","tag-education-matters","tag-steve-peha"],"_links":{"self":[{"href":"https:\/\/www.ibiblio.org\/carrborocitizen\/main\/wp-json\/wp\/v2\/posts\/6928","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ibiblio.org\/carrborocitizen\/main\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ibiblio.org\/carrborocitizen\/main\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ibiblio.org\/carrborocitizen\/main\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ibiblio.org\/carrborocitizen\/main\/wp-json\/wp\/v2\/comments?post=6928"}],"version-history":[{"count":1,"href":"https:\/\/www.ibiblio.org\/carrborocitizen\/main\/wp-json\/wp\/v2\/posts\/6928\/revisions"}],"predecessor-version":[{"id":6929,"href":"https:\/\/www.ibiblio.org\/carrborocitizen\/main\/wp-json\/wp\/v2\/posts\/6928\/revisions\/6929"}],"wp:attachment":[{"href":"https:\/\/www.ibiblio.org\/carrborocitizen\/main\/wp-json\/wp\/v2\/media?parent=6928"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ibiblio.org\/carrborocitizen\/main\/wp-json\/wp\/v2\/categories?post=6928"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ibiblio.org\/carrborocitizen\/main\/wp-json\/wp\/v2\/tags?post=6928"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}