2023-12-28
Assessments are crucial to teaching classes. Assessments include the following:
It is difficult to format a Google Doc with programmatically generated equations. As a result, it is better to distribute questions that my involve formats that do not render easily in Google Doc as PDFs.
A prerequisite of this approach is the modeling of common (or otherwise) misunderstanding of concepts and/or steps to perform a task. With such a model, a completed work but with flaws intentionally injected can be presented to a student to locate and identify the flaw, and possibly following up with a correction of the flaw.
This method is resistant to cheating because of the possibility of variations. The variations can be due to the type of flaw(s), and/or the location of the flaw(s).
This can be difficult with a traditional open-ended question where a question asks a student to construct or derive an answer. However, in the case of a flaw-finding type of question, the answer format can be more structured.
This approach is also possible when the format of an answer can be machine-parsed.
Google Sheet (along with most of the Google office apps) has several advantages over using Canvas to capture answers. All the editing are time-stamped. This can be useful for many purposes, including serving as evidence of collaboration.
A Google Sheets is essentially a collection of tables, where each table (sheet) is a tab of the overall “document”. A table is rigid, but it also has a definite structure. This makes it easier to capture answers in a structured way.
To facilitate automated grading, the answers should not be embedded into the Google Sheets files distributed to students. A Google Sheets object may embed “hooks”, but what they hook to should be stubs that do not reveal answers. Alternatively, a separate Google app access a Google Sheets object for grading purposes.
Any type of open-ended explanation is difficult to grade using an automated tool. As a result, it is best to avoid questions where answers consist of open-ended questions.
A question may present alternative explanations, and the answer can be a selection of the available explanations. In order to improve resistance to cheating, this approach should be combined with a variation of questions sharing the same format but not the same explanation. These questions should be randomized so that collaborators cannot associate answers to questions based on question ordering.
Depender and Dependee
If A depends on B, then A is the depender, and B is the dependee in the “depends on” relation.
Have the scoring of a depender question conditional to the scoring of related dependee questions.
For example, have set operator questions (depender) as depending question on “element-of” (dependee) questions.
The idea is to rule out lucky answers and potential results of collaboration.
If this kind of scoring is used, the questions should reflect the scoring mechanism
In multiple choice questions, the wrong answer should have a negative score associated so that randomly guessing all the question should still end up with a score of zero.
If this kind of scoring is used, the questions should reflect the scoring mechanism
This kind of question provides a list of constraints, and the answer should meet all of the constraints. The question generator should parametrize and randomize parameters of the constraints.
The evaluation of an answer is to pass it through all the constraints. Scoring can be a little tricky because it may or may not be the summation of the constraints met. Some constraints may be considered more difficult.
\(A=\{b,a,c\}\), \(B=\{a,c,d\}\), consider the expression \(R = (d \in A \cup B)\)
In cell A1, type the result of expression \(R\).
In cell A2, explain the result of expression \(R\) using the following choices:
Then have variations based on the set operator (“union” versus “intersection” versus “difference”) as well as the membership of the result of the set operator. Variations that are possible:
The ordering (and numbering) of the multiple choice options should also be randomized.
This essentially tests the understanding of how values propagate in a circuit that has “loops”. The propagation of values can potentially lead to states that are not stable, meaning that without changes of the input pins, some internal values can alter before “settling”.
A question can involve a known state in terms of both the input pins and the current output states of all the gates. Then an input is changed. The question is to test the whether a student understand the connectivity of components as well as how values are propagated.
An open format this kind of question is very difficult to grade for several reasons. Need to read open-ended explanations is one, and the second one is what if an error is made, causing additional incorrect conclusions?
A more efficient way to ask this kind of question is to produce a faulty trace of state changes and ask students to spot the incorrect step(s) of the trace. A question generator can inject flaws into the wrong answer. The following are some examples of categories of flaws: