Posted On: Monday - April 30th 2018 9:34AM MST
In Topics: Curmudgeonry Educational Stupidity
OK, from the get-go, the post title, this one may seem boring if you're here for the political stuff. However for 2 reasons, I would like to write a bit about exams, boards, or placements tests.
1) This relates to the material in the last post on HR, scourge of the business world, involving software-based applications*. I had promised at the end to make a post on this, since it's something I'd been thinking about writing on.
2) A family member has been studying for a big one of these multiple choice exams, and I started to see some of the problems.
To get to it, multiple-choice tests are in use everywhere for medical boards, professional engineer licences, ham radio, etc. In addition, they are the test of choice for teachers and professors for the very simple reason that the grading is MUCH easier. If it's not computerized (and why not, by this point?) it's a mindless chore that can be done while listening to music, cooking, what-have-you.
I say this as a comparison to the types of tests that must be given in math and engineering especially, where the problems are long, and involve multiple steps. You just can't have a multiple choice test to look for the one correct numerical answer that results from an applied thermodynamics problem that may be 1 of 2 that make up the test. A problem will have mulitple steps that can't be tested separately. Yet, the numbers from one part will be used in the next part of the problem. Therefore, the grader must put lots of thinking into the grading. Sure, he can check the numbers at each step at a time, but if the eariler ones are wrong, he must now use the wrong numbers to check the work of the latter steps. To just count the rest wrong is unfair. It's a real problem and makes grading a big chore that involves concentration and not a little good judgement. As good as computer software is, a paper test like this cannot be graded by computer. (That's not to say the whole test couldn't be done via software that does take initial errors into account.)
OK, the studying my family member was doing was for a mutilple-choice test. I started realizing some of the difficulties in taking, and flaws in, these tests as she was taking the practice ones.
As for the difficulties, many of these questions could have multiple correct answers of the (a), (b), (c), and (d) (sometimes (e)). That's not what I've been used to. In the old way, with only one correct, a couple of fairly nonsensical answers would result in a 50/50 chance on the answer left. Not good, but let's say there are 20 questions like this you aren't at all sure of, 10 more that you have no idea of any of the answers on, with the 70 rest of the 100 known. Probability-wise, you've got an 82.5 score (70 pts. + 1/2 x 20 + 1/4 x 10) vs. < 80 if the possible answers were all reasonable. With any number of good answers that could all be correct, the test becomes much harder.
The "all of the above" and "none of the above" possible answers always helped out too, in the old tests. There is more logic available for the test-taker to winnow out which is right, even when unsure. I don't there was to be much of this in this upcoming test. Now, the factors just discussed don't make the newer tests unfair, but just more difficult.
As for the flaws in multiple-choice exams. the real problem is the implied knowledge of the test-MAKER. Especially for true/false questions, but even for the normal pick-one-out-of-four, it's often the case that the test-taker must guess how smart or knowledgeable the test-maker is. It's the words "almost" "always" "never", "often", etc. that can make the question hard for someone who knows the subject matter very well. In fact, knowing it well often makes these questions harder!
Maybe it's physics and the question is (just an example mind you) "A particle in motion will ....". (four choices). Hey, you know Newton's law very well, but did the professor want you to remember the exeptions for things moving near the speed of light, or did he just expect an answer correct in the normal realm? At least you may know the professor's ways. "Yeah, he drilled this into us one day a week back, so he must want this answer." Who knows for sure, yet you damn well know the material.
On some type of medical board test, let's say the question is something like "These 3 symptoms often indicate the patient has blahblah-itis, true or false?" Well, in class we learned that sometimes this 4th symptom is a better way to figure out if he's got this nasty disease, but it's hard to look for. Did the test-makers even know this fact? How much the test-makers know can effect which is the correct answer, and that means the question sucks.
This kind of thing is worse in any kind of nationwide board test or on a test involved in a job application, as there is usually no recourse (not even a way to find out what they wanted in the end - this leads to the subject of the next post). At least in your own school classroom, you may bring up the question and why you answered "wrongly" and have it thrown out. Back on the other hand, you figure those board questions have been thorougly tested as legitimate and fair, so that you should be getting good ones. One bad one, however, may be the difference between practicing law next year or not (probably just as well not - we have more than enough lawyers!).
The making up of good multiple-choice test questions takes some skill, but also testing of the questions. That leads right into the subject of a follow-up post on the strangeness of the new way of computerized testing. This may be neither interesting nor stupid, but it's my blog, and feel free to skip. I'll be back into politics as soon as the next time I get on the internet, I'm sure - lots of stupid in that realm!
* I mean "applications" as one of its REAL defintiions here, as in applying for a job, versus this silly Apple "app" talk, in which an application is a piece of software (what idiot started that use?).
Comments: