Illinois joins the procession

The Chicago Tribune on Halloween 2013 reported “Illinois grade school test scores plunge — especially in poor communities”.  Following on the heels of New York’s testing debacle, in which performance gaps widened as well, we’re beginning to detect a pattern.  

The Illinois State Board of Education, on its student assessment page, reports that it had to “adjust the performance levels on the ISAT for Reading and Mathematics to better align with the more rigorous standards of the Common Core”, and a separate document entitled “2013 ISAT Mathematics Assessment”, dated December 5, 2012, and authored by “Rachel Jachino, ISBE Mathematics Principal Consultant”, states that “[a]pproximately twenty percent (20%) of the operational items on the Reading and Mathematics ISATs were written to Common Core Standards and will be included as part of students’ scores/results for the 2013 ISAT,” but it remains unclear what portion of the drop in 2013 scores is attributed to when the “state increased the scores required to pass ISAT math and reading tests by 13 to 30 points, depending on the test and grade” (Tribune), or to an actual change of test content.

Is any improvement in store for this year?  Cut scores are not set by teachers, but we’re well into the 2013-2014 school year, and the 2014 ISAT Mathematics “Roadmap” (pdf), intended to guide teachers in their instruction, has yet to be released.

The ISBE recently removed sample questions for the 2013 tests on the premise that the “sample items displayed were not necessarily representative of the material that will appear on the 2014 ISAT”, which, incidentally, will be a one-year deal, because Illinois will “replace the ISAT with the Partnership for Assessment of Readiness for College and Careers (PARCC) assessments during the 2014-2015 school year.”

It’s a foggy road ahead indeed for Illinois teachers and their students.

We’ll leave alone these various issues be sorted out, but since we snagged the ISAT 2013 sample items before the ISBE removed them, let’s have a look at some of them.  We're not convinced that next year's exam questions won't be similar to the samples, in particular, the questions aligned to Common Core.

Here’s how ISAT exams were assembled for each of grades 3 through 8:

It’s not surprising to see the Pearson name pop up, and they're involved with PARCC, too.  ISAT questions are of three types: multiple-choice, short-response, and extended-response.  Short- and extended-response questions are graded via rubrics.  All of the 2013 ISAT short-response and extended-response samples were aligned to the sunsetting Illinois Learning Standards, not Common Core.

Although we’ll focus mainly on questions that the ISBE pegs to Common Core standards, some of ISAT’s questions aligned to the Illinois standards are worth looking at as well.

Grade 3

We here at ccssimath.blogspot.com despise rote memorization tasks, especially those that simply test the memorization of definitions.  This non-Common Core-aligned question for Grade 3 references one of three commonly taught mathematical terms in analyzing data distributions, namely the mean, the median, and the mode.  Why mode has endured in any curriculum is beyond us; it’s fairly useless, especially as a skill applicable to solving problems.  If you didn’t memorize the definition, you’re sunk; otherwise, count all the numbers and determine which occurs the most times.  Besides the definition, as no more than a counting exercise, it’s not appropriately Grade 3 mathematics either way.

Concepts of data distribution are important, though, so if students are going to learn the meaning of all three terms, mean and median being the more useful and commonly encountered concepts, why not learn them all together, so that the topic need not be unnecessarily revisited and rehashed?  In particular, mean doesn’t belong in the curriculum until students have mastered calculations involving division, fractions and decimals, which are only beginning in Grade 3, and median and mode can be introduced as comparable and contrastable companions when learning the mean becomes timely as a useful application of division.

We’re further baffled by ISBE thinking when ISAT offers this simpler “mode” question in Grade 4:

Grade 3

This ISAT Common-Core aligned question…
 

…looks eerily similar to this New York question…

If you think they’d be assessing the same Common Core standard, you’d be right.  And thus raises an important issue:

This standard, like others in Common Core, does not lend itself to creative problem posing, but rather is restrictively worded to the extent that independent exam-writing committees reading the standard are likely to pose essentially identical questions.

Note, however, the difference in wording of the two questions.

Our takeaway is that for teachers and students, the way many of Common Core’s standards are written will lead teachers to pose to students numerous problems of the same sort, i.e., use drills, or in modern parlance, "apps".

Grade 4

We mentioned above, and have discussed repeatedly in earlier blog posts, that an effective way to abstract a concept—the highest form of understanding—is for students to be exposed to numerous and seemingly unrelated applications of the concept, and let them make the connections.

One obvious and distinct application of fractions, a topic that, pre- or post-Common Core, expands significantly in Grade 4, is probability.  Thus, while the form of the answers in this ISAT question is lousy (the answer choice should be written 4/9), this probability question is grade-appropriate for Grade 4.

However, in Common Core, the definition of probability does not appear until 7.SP.5,  “Understand that the probability of a chance event is a number between 0 and 1 that expresses the likelihood of the event occurring.”  We don’t understand the delay in formally introducing what is appropriately a primary school concept.  Grade 7, where this standard appears, is the year for working extensively with signed numbers, a far more advanced topic than fractions or decimals between 0 and 1, and thus 7.SP.5 is absurdly misplaced.

Pertinent questions like this ISAT example are off the board for the Common Core future, and it’s a shame.

Grade 4

This ISAT Common Core-aligned question…

…is similar to a NYSED question we reviewed previously…

…and assesses the same standard, 4.NF.4a, which reads “Understand a fraction a/b as a multiple of 1/b. For example, use a visual fraction model to represent 5/4 as the product 5 × (1/4), recording the conclusion by the equation 5/4 = 5 × (1/4).”

Another Common Core standard that does not lend itself to creative problem posing.  Since the problem is so predictable, there will be teaching to this question.

Worse still, the problem itself is terrible.  It exposes a shortcoming we’ve explained before: an overuse of “fraction models” instead of concentrating on the mathematics will cause students to rely on the pictures—and get wrong answers.

Thus, in the ISAT question above, we can predict with confidence that a significant number of students will choose incorrect answer A, because 5 out of 15 rectangles are shaded purple.  It will be difficult for students (and perhaps their teachers) to reconcile from the model that the shaded regions are 5/3 (or 1 ⅔) “wholes”, where each of the five original shapes represents one whole.

To compensate for an incomplete understanding of the mathematics involved, teachers are likely going to drill this question.

Grade 5

Common Core loves line plots.  Common Core loves line plots so much that they appear explicitly in Grades 2, 3, 4, 5, 6, 7, and high school.  Until reading the Standards, no one at ccssimath.blogspot.com had ever heard of or seen a line plot.  Line plots (or dot plots), for the uninitiated, are a “method of visually displaying a distribution of data values” involving making little X’s (for older students, dots) over a number line and counting or otherwise recognizing patterns in the distribution of the X’s.  Although line plots may progress from tallying whole numbers in Grade 2 to fractions in Grades 3, 4 and 5, and looking at the distributions in high school, the essential underlying mathematical skill, counting, was learned in kindergarten.  Years and years of line plots, while we’ve previously written that Common Core doesn’t include circle graphs or pie charts anywhere.  What were the authors thinking?

Here is a Grade 5 (thus mid-level) ISAT Common-Core aligned line plot problem::

This problem confused everyone here.  After reading the problem, we had to confirm that there were indeed ten X’s on the line plot, corresponding to the “10 small bags of birdseed” in the setup.

The first confusion arose when we realized that the amount of birdseed in each bag is measured in fractions of a cup, a volume in this sense, but the birdseed is not in physical cups.  The line plot adds to the confusion by putting “bags” at the top and “cups” below.

Are the “small” bags different sizes, since there are three different amounts?  No idea.  If so, wouldn’t they be “small”, “medium” and “large” bags?  Also, incidentally, birdseed is sold by weight, not volume.

The bags are distinct from the ½-cup containers mentioned later, the physical form of which is not made clear, but they are also not specifically described as cups in the kitchen cupboard sense.  So “cups” of birdseed are moving from bags to containers without ever being in literal cups.  Some students might confuse the two meanings of “cups” and think of the “containers” as “cups”.

A dual meaning of one word in the English language manifests itself in this problem without being recognized by the problem’s authors.  Getting into trouble with word problems is an issue we discussed in our analysis of NYSED’s exams.

Now we get to the actual mathematics.  In doing the problem, we realized that in one instance, a bag with ½ cup of birdseed will be poured into a ½-cup container without being commingled with the birdseed from any other bags.  Um, why does the pet store need to do this, anyway?  In the line plot, the X over the designation “½ cup” implies that that birdseed is already in a “½-cup container” of a sort, in this instance, a bag, so do we need to tally it?  That's a critical question because two answer choices hinge on the decision.  The remaining bags of birdseed can be combined (e.g., ⅛ + ⅛ + ¼ = ½ is one container) into four ½-cup containers, so it seems students might choose either 4 or 5, depending on how they approach the problem, especially those who confuse the meanings of “cups”.

Finally, and this is indefensible, the spacing between ⅛ and ¼, and ¼ and ½, on a purported number line, is indicated as the same.  That’s bad mathematics.

Either way, the simple fractions question underlying this Common Core-aligned problem is obscured by poor wording and terms with multiple or ambiguous meanings.  In all of the ISAT questions we reviewed, we could find no pre-Common Core question with such weaknesses.

Grade 7

“Proportional relationship” is a catch phrase which will become commonplace in the American middle school classroom in the Common Core regime.  The question remains open: will there be understanding of the concept or will students be drilled on how to answer the questions?

7.RP.2a states, “Decide whether two quantities are in a proportional relationship, e.g., by testing for equivalent ratios in a table or graphing on a coordinate plane and observing whether the graph is a straight line through the origin.”  The way this standard is framed, the analysis is reduced to a yes or no question (read: drillable).

We might also comment that the standard as it seems intended is written wrong mathematically; it should say, at minimum, "Decide whether two sets of quantities are in a proportional relationship…"

Here’s ISAT’s example:

From the drills, students who are paying attention will recognize that choice A is a “proportional relationship” because it shows a “straight line through the origin”.  Deeper understanding?  Not necessary.

How about SBAC?  For some reason, they put a question assessing this standard into their Grade 8 questions:

Plot a point that is $2 on the vertical axis, 1 pound on the horizontal axis, and connect that point to the origin, and the answer is complete.  Drillable.

Then there is PARCC’s version:

This PARCC example shows how it is possible to blow up with hot air the same, simple mathematics question, without adding substance.  There’s going to be a lot of this kind of fluff in Common Core in the guise of “complex problems” (SMP1).

It bears repeating: this middle school standard, which always presents students with a binary choice, yes or no, is not so complex in any form that it cannot be drilled.

Grade 8

We conclude with comments on what is likely to become commonplace: Common Core phony “real-world” questions, such as…

…which, if we understand it correctly, has no actual ice cream because the problem is only about an empty cone (really sad), and requires nothing more than plugging numbers into the formula for the volume of a cone.  This particular question tests 8.G.9: "Know the formulas for the volumes of cones, cylinders, and spheres and use them to solve real-world and mathematical problems."  Common Core, which is supposed to engender more complex thinking processes, still has “plug the numbers into the formula” standards.

***

While Illinois’ students’ scores may have plummeted (along with New York’s), the devil remains in the details, into which we only have small peephole to view.

Gaps may widen during the transition in the name of tougher standards, to the detriment of many and the benefit of few, but teaching to the test?  Likely to remain alive and well under Common Core.