Page 2 of 7 FirstFirst 1234567 LastLast
Results 16 to 30 of 93

Thread: Elementary School MCAS scores

  1. #16
    Join Date
    Nov 2005
    Posts
    726

    Default

    Alan,

    The "rankings" are, as rankings often seem to be, not a failsafe way of ensuring the "best" school gets the best ratings.

    So let's look, for example at 3rd Grade English:

    Claypit Hill School
    3rd English 14 60 22 5 125 201 of 980

    Happy Hollow School
    3rd English 27 44 29 0 79 259 of 980

    Claypit's ranking is 201. Happy Hollow's is 259. But the other numbers you see are % advanced, proficient, warning and failing, followed by the total number of students, then the ranking.

    While Happy Hollow's ranking is lower, I think its distribution is better. If you look at the CPI scores (which provide different ratings depending on the score, and are another flawed, but probably better measure), HH outperformed Claypit on this test, with a CPI of 90.8 (v. 90.6 for Claypit). Pretty similar, and probably not statistically significantly different. But definitely higher for HH.

    Similarly, for third grade math, HH's CPI is 90.2 to 89.8 for Claypit.

    Across all grades, CH's CPI is higher: 92.9 to 90.3 for the ELA, and 90.3 to 87.2 for the math.

    Here are the distributions:

    ELA: Claypit: 24/56/18/3
    HH: 24/50/24/2

    Math: Claypit: 33/43/20/4
    HH: 29/38/27/5

    These differences look a lot smaller to me than the "rankings", which only look at the sum total of advanced+proficient, and ignore the overall distribution.

    Whether these differences are statistically significant...? My facility with stats is a little rusty. It might be significant, but it's not immediately obvious - at least not for this one year. Did you run statistical tests on the numbers to establish significance? Exactly which numbers did you compare?

    BTW, I'm not implying that I don't see that typically Claypit's numbers are generally better than HH's. Alan, if you have a theory, you ought to share it so we can discuss it.
    Last edited by Kim Reichelt; 10-06-2009 at 06:45 PM. Reason: to add last sentence

  2. #17
    Join Date
    Dec 2005
    Posts
    165

    Lightbulb Anne Elk

    Ahem! My theory - by Anne Elk. What is my theory that it is? Well, you may well ask what is my theory. I have a theory. Well, this theory, that I have, that is to say, which is mine,... is mine. (if you want to know how this ends, check out http://www.orangecow.org/pythonet/sketches/anneelk.htm)

    Actually, I have no theory. Not a big surprise, but I don't really feel qualified to comment on why one area of town might score higher or lower on a standardized test.

    If you try to look at this from a distance, there are at least 3 general areas where test scores may be influenced. There are probably more, but these 3 cover a pretty wide swath:

    1) The instructor (ie: the inputs)
    2) The student (ie: the processing of the inputs providing outputs)
    3) The environment

    Not a popular political place to go, but one could hypothesize that either the teachers and/or the students at HH are somehow inferior to those found at CH. Likely? Not really. But certainly possible. If the tests are really designed to measure teaching effectivity, then if the students are performing poorer than their counterparts in CH, wouldn't it follow that the teachers are superior at CH? Of course, the tests could be flawed, but according to both Alan and Jeff, they are "generally accepted" (the test results that is). So, ruling out the inputs and the processors, this leaves us with the environment. This is a general term, and could refer to the classical environment (including but not limited to pollutants which of course would open a whole new can of worms related to what we're exposing our poor HH students to that our lucky CH students are not exposed to) or to the environment that the test is administered in. By this, I mean it could be noisy, uncomfortable, hot, cold, smelly, or a host of other things that would distract the student from the test at hand, and cause lower grades. As you can see from where this is headed, I've conclusively proved my earlier hypothesis that I am not qualified to comment on this odd dichotomy. So, dear Alan, thanks for asking. But just as you have declined to offer up your mysterious theory, I fear that I must do the same.

    (Maybe I should go to law school too? )

  3. #18
    Join Date
    Mar 2008
    Location
    Wayland MA 463 Old Conn Path
    Posts
    382

    Default Raw percentages were my numbers

    Quote Originally Posted by Kim Reichelt View Post
    BTW, I'm not implying that I don't see that typically Claypit's numbers are generally better than HH's. Alan, if you have a theory, you ought to share it so we can discuss it.
    I saw the CPI statistics and they are one level removed from the raw data. CPI is a composite statistic and it can be useful to quantify an entire group but lacks the precision to see it at its most basic level.

    One way to explain this is by the median property example which is the 50% ranked property in town which only represents one property and says nothing about all the other properties. But it you want to track tax changes or valuation changes then it makes sense. Still another example is where there are 100 kids in a classroom and 1/2 get 100% on a test while the other 1/2 get 0% and the average is 50% and nobody actually got 50% but the average is a valid statistic.

    In the case of MCAS scores I would prefer to look and compare directly at the level of percentage of kids vs. performance vs. grade/subject across two co-horts of schools.

    This is why I choose to use the raw percentages which are directly associated with the actual number of kids who placed in one group of performance vs. another.

    These raw percentages are available under profiles.doe.mass.edu and you can hunt around for various tables to get different levels of slicing through the raw data.

    As for my theory or theories... I'm still formulating them but I did provide some insight to what I'm thinking under my reply here

    On another note, I would like to propose a puzzle but you have to pretend a little...

    Chart SchoolAvsSchoolB.jpg is attached below.
    And using this chart...

    Pretend...and envision...
    1. You are a new family in town
    2. You get to choose which elementary school your child goes to (independent of where you live)
    3. One school is called BLUE Elementary
    4. One school is called RED Elementary
    5. The vertical axis is Percentage of Kids who took MCAS in a given group.
    6. Advanced plus Proficient has been summed together on the left chart
    7. Needs Improvement plus Fails has been summed together on the right chart.

    Question: Which school would you prefer your kid attend...
    BLUE or RED?
    Attached Images Attached Images  

  4. #19
    Join Date
    Dec 2005
    Posts
    165

    Default Interesting chart

    Is it me, or does something seem to happen in the RED school in Grade 4 that worsens significantly in Grade 5, and recovers a bit, but not much in Grade 6 on the "bad" chart? The "good" chart doesn't seem terribly different between BLUE and RED (although granted BLUE always seems to do a little better), but there is a definite departure in how many RED students fall into the hole starting in Grade 4 (grade 3 again doesn't seem terribly different between BLUE and RED on the "bad" side).

    Weird, no?

  5. #20
    Join Date
    Nov 2005
    Location
    Wayland MA
    Posts
    1,431

    Default

    The charts have a bit of a visual disconnect because they don't use the same Y-axis. Alan, could you re-post with both using 0-100 on the vertical axis?

  6. #21
    Join Date
    Nov 2005
    Location
    Wayland MA
    Posts
    1,431

    Default

    It occurs to me that even the Advanced+Proficient vs. Needs Improvement/Warning/Failing grouping is a composite of the raw data, not the raw data itself. The attached chart shows the raw data with a full y-axis representation.
    Attached Images Attached Images  

  7. #22
    Join Date
    Mar 2008
    Location
    Wayland MA 463 Old Conn Path
    Posts
    382

    Default I've looked at your two graphs

    Quote Originally Posted by Jeff Dieffenbach View Post
    It occurs to me that even the Advanced+Proficient vs. Needs Improvement/Warning/Failing grouping is a composite of the raw data, not the raw data itself. The attached chart shows the raw data with a full y-axis representation.
    As a parent I would want my kid in the Advanced+Proficient and I would NOT want my kid in the Needs Improvement/Warning Failing so summing the pairs of the groups meets this need.

    The data is still baseline raw because its summing the percentages of each of the two groups... again as a parent I am pretty sure I would want one pair and not the other.

    Your graph expects me to do a mental calculation of color verticals to see which pair is greater. This would not serve my purpose as a parent who does not want my kid in on pair of groups and not the other.

    My two graphs have different verticals because they don't need to be compared absolutely side by side... only that BLUE vs. RED is consistently different and with an absolute 100% correlation of result.

    So which one is it Jeff... you want your kid at BLUE elementary or RED elementary... thats the puzzle?
    Last edited by AlanJReiss; 10-07-2009 at 10:52 AM.

  8. #23
    Join Date
    Nov 2005
    Location
    Wayland MA
    Posts
    1,431

    Default

    Quote Originally Posted by AlanJReiss View Post
    My two graphs have different verticals because they don't need to be compared absolutely side by side... only that BLUE vs. RED is consistently different and with an absolute 100% correlation of result.
    Alan, your two graphs aren't even necessary, since one is simply the inverse of the other, a situation articulately outlined by Edward Tufte of "the Visual Display of Quantitative Information" fame. Your first chart alone is factually sufficient.

    But, but magnifying the y-axis of the Needs Improvement/Warning/Failing chart, and placing that chart to the right of the Advanced/Proficient chart, you present a visual effect that highlights the negative.

    I would have absolutely no qualms about my children attending any of Wayland's elementary schools.

  9. #24
    Join Date
    Mar 2008
    Location
    Wayland MA 463 Old Conn Path
    Posts
    382

    Default Elementary School Mathematics

    Quote Originally Posted by Jeff Dieffenbach View Post
    Alan, your two graphs aren't even necessary, since one is simply the inverse of the other...
    They are not the 'inverse' of the other. If the first graph is f(x) an the second graph is g(x) and they were inverse then the relationship between the two graphs would be f(x) = 1 / g(x) and this is just not numerically the case.

    The two graphs are matched correlations of each other. You could make a judgement on BLUE vs. RED with just one graph no doubt but both graphs show the details of the magnitude of the problem. ie. the differences between BLUE vs. RED are not constant and change in a non smooth manner and with a specific signature. Both graphs have their own specific signatures.

    But who cares if you could have made a judgement with just one graph?
    Thats not the point.

    The question of this exercise was not whether you would have any qualms of your children attending any of Wayland's elementary schools... BLUE or RED.

    The question is, if you could make a choice based on these two charts, which school BLUE or RED, would be your choice?

    Now here is the second part of the puzzle.
    Why is this happening?

  10. #25
    Join Date
    Nov 2005
    Location
    Wayland MA
    Posts
    1,431

    Default

    Quote Originally Posted by AlanJReiss View Post
    They are not the 'inverse' of the other. If the first graph is f(x) an the second graph is g(x) and they were inverse then the relationship between the two graphs would be f(x) = 1 / g(x) and this is just not numerically the case.
    You are correct, they are not 1/x mathematical inverses, which I did not intend to suggest. What I should have said was that the first chart shows a percentage out of 100 and the second chart simply shows the remaining percentage. That is, the second chart adds no new information, since you could derive it entirely from the first chart. This is a form of Tufte's "chart junk." [grin]

  11. #26
    Join Date
    Nov 2005
    Posts
    726

    Default

    Methinks the word you wanted was "complement", which is the "additive inverse". So inverse wasn't really wrong, it just wasn't precise.

  12. #27
    Join Date
    Mar 2008
    Location
    Wayland MA 463 Old Conn Path
    Posts
    382

    Default Junk?

    Quote Originally Posted by Jeff Dieffenbach View Post
    This is a form of Tufte's "chart junk." [grin]
    I had this overwhelming feeling that you would find some way to be dismissive of these charts and thats disappointing.

    Let me try to be very serious here about what the charts are saying...

    1. The chart on the left which sums the raw data into the best two outcomes is always BLUE > RED
    2. The chart on the right which sums the raw data into the worse two outcomes is always RED > BLUE
    3. 100% = RED(mcas) + BLUE(mcas)
    4. The key here is the word ALWAYS.
    5. ALWAYS means 100% positive correlation where the equation in (3) above does not guarantee that.
    6. Each graph has its own specific signature which carries additional information.

    I would have expected, if the BLUE school and the RED school were more closely matched in the delivery of learning as measured by MCAS then you would have seen a more random behavior of sometimes BLUE > RED or RED > BLUE on the same chart... but this is not the case.

    This is a 100% correlation which points to something more systemic.

    Now with all due respect to you Mr. SC Sir... your answers to this puzzle have been a form of "answer junk" [FROWN]. (as noted by the Steven Martin school of useless retorts)

    If you don't want to choose a BLUE vs. RED school or come up with a reason why this is happening or launch a project where discovery can lead to remedy of such an MCAS mismatch in two elementary schools in Wayland which are only a few miles apart... then just say so and I'll stop talking about it in this forum.

    Else...
    Which school for your kids BLUE or RED ? and
    Why is this happening?

  13. #28
    Join Date
    Nov 2005
    Location
    Wayland MA
    Posts
    1,431

    Default

    I wasn't being dismissive of your charts--rather, I suggested an improvement to one of them and proposed an alternative.

    As I said early on in this thread, I don't have a theory as to why there's a difference. More importantly, I went on to say that our professional educators are working to understand this difference. Doing so is beyond my skill set and the scope of my position.

    I don't understand what you mean when you say "100% = RED(mcas) + BLUE(mcas)". Isn't it 100% = Advanced + Proficient + Needs Improvement + Warning/Failing?

  14. #29
    Join Date
    Mar 2008
    Location
    Wayland MA 463 Old Conn Path
    Posts
    382

    Default The Equation

    Quote Originally Posted by Jeff Dieffenbach View Post
    I wasn't being dismissive of your charts--rather, I suggested an improvement to one of them and proposed an alternative.
    The chart I proposed clearly and simply shows the comparison. I feel that your alternative charts increase the complexity of what the data is showing.

    Quote Originally Posted by Jeff Dieffenbach View Post
    As I said early on in this thread, I don't have a theory as to why there's a difference. More importantly, I went on to say that our professional educators are working to understand this difference. Doing so is beyond my skill set and the scope of my position.
    Who are the participants? How long have they been working on it? Are they accountable to the SC? Are they accountable to the parents? When would we expect an answer? Is this something the SC would be discussion at a future meeting or is this a subject for www.WaylandSchoolCommittee.org ?

    Quote Originally Posted by Jeff Dieffenbach View Post
    I don't understand what you mean when you say "100% = RED(mcas) + BLUE(mcas)". Isn't it 100% = Advanced + Proficient + Needs Improvement + Warning/Failing?
    Advanced + Proficient + Needs Improvement + Warning/Failing always = 100% for any given category (of mcas) is what that equation is saying.

    OK, you've answered the second question...
    You don't know why its happening. But you say some group (yet to be named) is working on it.

    But what about the first question...?
    Which school; BLUE or RED for your kid?

    Or are you saying that those two schools as measured by the MCAS would have no effect on your decision as to where you would want your kid(s) to go? (Re: you comment on 'no qualms' from before)

    I can answer the first question for me...
    I would take BLUE.

    My answer to the second question is more of a list of possibilities of which each may have more or less effect on the outcome and they are listed here.

    Would you like to comment on my list of possibilities or provide any data to show that there are differences (in real life) to account for this?

  15. #30
    Join Date
    Apr 2009
    Posts
    117

    Default

    In response to Jeff's ongoing criticisms “... This is a form of Tufte's "chart junk." [grin]” Alan noted:

    Quote Originally Posted by AlanJReiss View Post
    I had this overwhelming feeling that you would find some way to be dismissive of these charts and thats disappointing.
    Alas, I must agree. Let me fantasize about Jeff's position. The problem with the two sets of test scores is obvious (even without all the chart stuff), and Jeff can't refute it with the usual “quote” and “criticism” technique, and he also knows that the School Committee should have an intelligent response to identify the problem and propose a solution, but doesn't. This leaves his arguments struggling and him refusing to pick.

    Seems I would pick the school with more “advanced” student scores (by now I can't remember whether it's red or blue).

    donBustin@verizon.net

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •