# Multiple choice inputs

The whole point of STACK is not to use multiple-choice questions, but instead to have the student enter an algebraic expression!  That said there are occasions where it is very useful, if not necessary, to use multiple-choice questions in their various forms.  STACK's use of a CAS is then very helpful to generate random variants of multiple-choice questions based on the mathematical values.

This can also be one input in a multi-part randomly generated question. E.g. you might say "which method do you need to integrate \( \sin(x)\cos(x) \)?" and give students the choice of (i) trig functions first,
(ii) parts, (iii) substitution, (iv) replace with complex exponentials.  (Yes, this is a joke: all these methods can be made to work here!)  Another algebraic input can then be used for the answer.

If you are new to STACK please note that in STACK MCQs are *not* the place to start learning how to author questions.  Please look at the [authoring quick-start guide](../../AbInitio/Authoring_quick_start_1.md).

Please read the section on [inputs](../../Authoring/Inputs/index.md) first.

STACK provides a number of multiple choice input types including:
* Checkbox (students select one or more options from a displayed list)
* Radio (students select *one* option from a list)
* Drop down list (a drop-down menu box is added to the question text, from which students select a single option)

Multiple choice input types return a CAS object which is then assessed by the potential response tree.  For this reason, these inputs do not provide "feedback" fields for each possible answer, as does the Moodle multiple choice input type.

* Checkbox inputs return a *list* of the CAS objects selected;
* Dropdown inputs and radio inputs return a single *expression*, which is the CAS object selected.

The goal of these input types is to provide *modest* facilities for MCQ.  An early design decision was to restrict each of the possible answers to be a CAS expression.  In particular, we decided *NOT* to make each possible answer [CASText](../../Authoring/CASText.md).  Adopting CASText would have provided more flexibility but would have significantly increased the complexity of the internal code. If these features are extensively used we will consider a different input type.

Example questions are in the stack library under `Features\MCQ_....`.

It is possible to include [plots within MCQ](Multiple_choice_plots.md).

## Model answer ##

This input type uses the "model answer" both to input the teacher's answer and the other options.
In this respect, this input type is unique, and the "model answer" field does *not* contain just the teacher's model answer.  Constructing a correctly formed model answer is complex, and so this input type should be considered "advanced".  New users are advised to gain confidence writing questions with algebraic inputs first, and gain experience in using Maxima lists.

The "model answer" must be supplied in a particular form as a list of lists `[[value, correct(, display)], ... ]`.

* `value` is the value of the teacher's answer.
* `correct` must be either `true` or `false`.  If it is not `true` then it will be considered to be `false`!
* (optional) `display` is another CAS expression to be displayed in place of `value`.  This can be a string value here, but it will be passed through the CAS if you choose the LaTeX display option below.  `display` is only used in constructing the question.  STACK will take `value` as the student's answer internally, regardless of what is set here.

For example

     ta:[[diff(p,x),true],[p,false],[int(p,x),false]]

At least one of the choices must be considered `correct`.  However, the `true` and `false` values are only used to construct the "teacher's correct answer".   You must still use a [potential response tree](../../Authoring/Potential_response_trees.md) to assess the student's answer as normal.

Note the `value` expression must be constructed with noun functions where a noun alternative exists.  E.g. use `%union()` (the noun form) not `union()`.  STACK has to match up what the student selects, and this is an expression they might well type into an algebraic input.  In this situation functions in students' expressions are turned to noun forms.  Hence the teacher must also use noun forms in setting up the MCQ input options.

STACK provides some helper functions

1. `mcq_correct(ta)` takes the "model answer" list and returns a list of values for which `correct` is true.
2. `mcq_incorrect(ta)` takes the "model answer" list and returns a list of values for which `correct` is false.

Note, that the optional `display` field is *only* used when constructing the choices seen by the student when displaying the question.  The student's answer will be the `value`, and this value is normally displayed to the student using the validation feedback, i.e. "Your last answer was interpreted as...". A fundamental design principal of STACK is that the student's answer should be a mathematical expression, and this input type is no exception.

In situations where there is a significant difference between the optional `display` and the `value` which would be confusing, the only current option is to turn off validation feedback.  After all, this should not be needed anyway with this input type.  In the example above when a student is asked to choose the right method the `value` could be an integer and the display is some kind of string.

An example which includes the `display` option is

    tacp:[[A, false, "A. Direct proof"],  [B, false, "B. Definition-chasing"], [C, false, "C. If and only if"], [D, false, "D. Exhaustive cases"], [E, false, "E. Induction"], [F, false, "F. Contrapositive"], [G, true, "G. Contradiction"]];

Note in this example the `value` of the student's answer will be a letter which is literally a Maxima variable name.  In this situation you can't really randomize the letters used easily.  (Not impossible with some cunning code, but a lot of work....)

When using checkbox or radio input types, the `display` value can include LaTeX code. (See below for including mathematics in a drop down input type.) Any `\` in the LaTeX code needs to be *escaped* to `\\`. For example a model answer might be

     tacp:[[0, true, "\\( \\sin(x) \\)"], [1, false, "\\( \\cos(x) \\)"], [2, false, "\\( e^x \\)"]]


If you choose to use an integer, and randomly shuffle the answers then the validation feedback would be confusing, since an integer (which might be shuffled) has no correspondence to the choices selected.  *This behaviour is a design decision and not a bug! It may change in the future if there is sufficient demand, but it requires a significant change in STACK's internals to have parallel "real answer" and "indicated answer". Such a change might have other unintended and confusing consequences.*

Normally we don't permit duplicate values in the values of the teacher's answer.  If the input type receives duplicate values STACK will throw an error.  This probably arises from poor randomization.  However it may be needed.  If duplicate entries are permitted use the display option to create unique value keys with the same display. *This behaviour is a design decision may change in the future.*

When STACK displays the "teacher's answer", e.g. after a quiz is due, this will be constructed from the `display` fields corresponding to those elements for which `correct` is `true`.  I.e. the "teacher's answer" will be a list of things which the student could actually select.  Whether the student is able to select more than one, or if more than one is actually included, is not checked.  The teacher must indicate at least one choice as `true`.

If you need "none of these" you must include this as an explicit option, and not rely on the student not checking any boxes in the checkbox type.  Indeed, it would be impossible to distinguish the active selection of "none of these" from a passive failure to respond to the question.

## Model answer: nouns ##

If one of the responses is \(x=1 \text{ or } x=2\) then use `nounor` which is commutative and associative.  Do not use `or` which always simplifies its arguments.  In this example `x=1 or x=2` evaluates to `false`.

Functions `diff` and `int` will evaluate, so you don't have displayed calculus operation.  If one of the responses is a _displayed_ derivative or integral then construct your answer with Maxima's inert forms `'diff` and `'int`.

## Model answer: mathematics display in dropdowns ##

HTML dropdowns cannot display LaTeX within the options.  This is a restriction of HTML/MathJax (not of STACK).  You can use HTML-entities within a string field.  For example

    ta1:[[0,false,"n/a"],[1,true,"&ge;"],[2,false,"&le;"],[3,false,"="],[4,false,"?"]];

Note here that an integer will returned internally.

Similarly, you can include logical symbols. For example

    ta1:[[0, false, "&#8658;"], [1, true, "&#8656;"], [2, false, "&#8660;"]];

will give a choice, e.g. a dropdown, from &#8658;, &#8656; and &#8660; and an integer will returned internally.

    ta1:[[0, false, "&#8704;"], [1, true, "&#8707;"]];

will give a choice, e.g. a dropdown, from &#8704; and &#8707;.

    ta1:[[0, true, "c &#8712;"], [1, false, "c &#8713;"]];

will give a choice, e.g. a dropdown, from "c &#8712;" and "c &#8713;".

    ta1:[[N, false, "&#8469"], [Z, true, "&#8484"], [Q, false, "&#8474"], [R, false, "&#8477"], [C, false, "&#8450"]];

will give a choice between sets of numbers &#8469;, &#8484;, &#8474;, &#8477;, and &#8450;.

### Example: degree of a polynomial ###

Here is an example where the teacher would like the student to state the degree of a polynomial using the adjectives, rather than a number.

    pol:sum(rand_with_prohib(-9,9,[0])*x^k,k,0,1+rand(5));
    deg:hipow(pol,x);
    /* Use strings, and not keywords. */
    l1:["constant", "linear", "quadratic", "cubic", "quartic", "quintic"];
    /* Create a matching list of true/false values as to whether each option is correct. */
    a1:maplist(lambda([ex], is(ex=deg)), makelist(k,k,0,length(l1)));
    /* Basic answer list for MCQ in the correct format. This returns the string to the PRT. */
    ta1:zip_with("[",l1,a1);
    /* Since lists index at one, you need this for the correct answer! */
    tac:l1[deg+1];
    /* Returns the degree as the student's answer, not the word. */
    ta2:maplist(flatten, zip_with("[",makelist(k,k,0,length(l1)), zip_with("[",a1,l1)));
    /* Feedback can turn this into a word using indexing, e.g. l1[ans1+1] in the PRT. */

Either `ta1` or `ta2` can be used with the MCQ inputs, in this case dropdown probably makes most sense and checkbox least sense!  With the `ta2` option you probably want to hide the validation feedback.

## Internals ##

The dropdown and radio inputs return the `value`, but the checkbox type returns the student's answer as Maxima `list`,
even if they have only chosen one option.

The function `mcq_correct` returns a list, so for dropdown and radio you might need to take the `first` element when using this function to extract values from the list of teacher's answers.

If, when authoring a question, you switch from radio/dropdown to checkboxes or back, you will probably break a PRT because of mismatched types.

For the select and radio types the first option on the list will always be "Not answered".  This enables a student to retract an answer and return a "blank" response.

For the checkbox type there is a fundamental ambiguity between a blank response and actively not selecting any of the provided choices, which indicates "none of the others".  Internally STACK has a number of "states" for a student's answer, including `blank`, `valid`, `invalid`, `score` etc.  A student who has not answered will be considered `blank`, which is different from invalid.  Potential response trees which rely on this input type will not activate until the state is `score`.

To enable a student to indicate "none of the others", the teacher must add this as an explicit option in all MCQ types.  If you add an option "none of the others" then it will return the `value` of that selection: you could give this the value of `null`, for example, which is a Maxima atom.  We did not add support for a special internal "none of the others" because the teacher still needs to indicate whether this is the true or false answer to the question.  To support randomisation, this needs to be done as an option in the teacher's answer list.

The radio and dropdown types always add a "not answered" option as the first option.  This allows a student to retract their choice, otherwise they will be unable to "uncheck" a radio button, which will be stored, validated and possibly assessed (to their potential detriment).  If you want to remove this then use the extra option `nonotanswered`, but keep in mind the possible effect when using the penalty scheme.

If one of the items in the teacher's answer list is is the special variable name `notanswered`, and then default message `(Clear my choice)` will be replaced by the `display` value.  If no `display` value is given (and it is optional) then the original message will remain.  `notanswered` will not appear in the list of valid choices for a user and `value` for this input is ingored.


## Extra options ##

These input types make use of the "Extra options" field of the input type to pass in options.  These options are not case sensitive.
This must be a comma-separated list of values as follows.

The way the items are displayed can be controlled by the following options.

* `LaTeX` The default option is to use LaTeX to display the options, using an inline maths environment `\(...\)`.  This is probably better for radio and checkboxes.  It sometimes works in dropdowns, but not always and we need to test this in a wider variety of browsers.
* `LaTeXdisplay` use LaTeX to display the options, using the display maths environment `\[...\]`.
* `LaTeXinline` use LaTeX to display the options, using the inline maths environment `\(...\)`.
* `LaTeXdisplaystyle` use LaTeX to display the options, using the inline maths environment and the displaystyle option `\(\displaystyle...\)`.
* `casstring` does not use the LaTeX value, but just prints the casstring value in `<code>...</code>` tags.
* `nonotanswered` removes the ``Not answered'' option from radio and dropdown.  This is _not recommended_ as it means a student has no opportunity to "uncheck" a radio button once selected.  They may wish not to answer, rather than save an incorrect answer.
* `allowempty` is supported by MCQ inputs.


## Randomly shuffling the options ##

To randomly shuffle the options create the list in the question variables and use the Maxima command `random_permutation` in the question variables.

For example, the question variables might look like the following.

    /* Create a list of potential answers. */
    p:sin(2*x);
    ta:[[diff(p,x),true],[p,false],[int(p,x),false],[cos(2*x)+c,false]];
    /* The actual correct answer.    */
    tac:diff(p,x)
    /* Randomly shuffle the list "ta". */
    ta:random_permutation(ta);
    /* Add in a "None of these" to the end of the list.  The Maxima value is the atom null. */
    tao:[null, false, "None of these"];
    ta:append(ta,[tao]);

These commands ensure (1) the substantive options are in a random order, and (2) that the `None of these` always comes at the end of the list. Note, the value for the `None of these` is the CAS atom `null`. In Maxima `null` has no special significance but it is a useful atom to use in this situation.

As the Question Note, you might like to consider just taking the first item from each list, for example:

    {@maplist(first,ta)@}.  The correct answer is {@tac@}.

This note stores both the correct answer and the order shown to the student without the clutter of the `true/false` values or the optional display strings.
Remember, random variants of a question are considered to be the same if and only if the question note is the same,
so the random order must be part of the question note if you shuffle the options.

## Constructing MCQ arrays in Maxima ##

It is not easy to construct MCQ arrays in Maxima.  This section contains some tips for creating them, using Maxima's `lambda` command.
Below is an example of a correctly constructed teacher's answer.

    ta:[[x^2-1,true],[x^2+1,false],[(x-1)*(x+1),true],[(x-i)*(x+i),false]]

To create a list of correct answers you could use the function `mcq_correct(ta)`.  This essentially consists of the following code.

    maplist(first, sublist(ta, lambda([ex], second(ex))));

The function `sublist` "filters" out those entries of `ta` for which the second element of the list is true.
We then "map" first onto these entries to pull out the value.
It is relatively simple to modify this code to extract the incorrect entries, the displayed forms of the correct entries etc.

To go in the other direction, the first list `ta1` is considered "correct" and the second `ta2` is considered incorrect.

    ta1:[x^2-1,(x-1)*(x+1)];
    ta1:maplist(lambda([ex],[ex, true]), ta1);
    ta2:[x^2+1,(x-i)*(x+i)];
    ta2:maplist(lambda([ex],[ex, false]), ta2);
    ta:append(ta1,ta2);
    /* If you want to shuffle the responses then use the next line. */
    ta:random_permutation(ta);

Also, you can use STACK's `rand_selection(L, n)` to select \(n\) different elements from the list \(L\).
Say you have the following list of wrong answers and you want to take only 3 out of 5.

    ta2:[x^2,w^2,w^6,z^4,2*z^4];
    ta2:rand_selection(ta2, 3);
    /* Then, as before. */
    ta2:maplist(lambda([ex],[ex, false]), ta2);


Another way to create an MCQ answer list is to have Maxima decide which of the answers are true.
For example, in this question the student has to choose which of the answers are integers.

    L:[1,4/2,3.0,2.7,1/4,%pi,10028];
    /* Map the appropriate predicate to the list to create the true/false list. */
    A:maplist(integerp,L);
    /* Note the use of zip_with together with the list constructing function "[". */
    ta:zip_with("[",L,A);
    /* If you want to shuffle the responses then use the next line. */
    ta:random_permutation(ta);

## MCQ helper functions ##

STACK has helper functions to create MCQ arrays in Maxima.

    [ta, variant] = multiselqn(corbase, numcor, wrongbase, numwrong)

This function takes two lists `corbase` and `wrongbase` and two integers `numcor` and `numwrong`.
It randomly selects `numcor` from `corbase`, and `numwrong` from `wrongbase` and then creates the MCQ list `ta` with these selections,
and an answernote `variant`.

The function returns a list with two arguments.
The first argument of the list is the MCQ array, the second is just the list of answers which is useful for the answer note.
Note, this function does use `random_permutation` internally to randomly order the random selections.

For example, the following generates random expressions for an MCQ calculus question.  Note the use of `ev(...)` later to evaluate the derivative.

    trg:rand([sin(p), cos(p)]);
    dtrg:diff(trg, p);
    wrongbase:[a*trg, 2*a*x*trg, -2*a*x*trg, ev(dtrg, p=2*a*x), 2*a*x*ev(dtrg, p=2*a*x)];
    p:a*x^2+b;
    wrongbase:ev(wrongbase); /* Now we have a value for p, the extra evaluation will use it. */
    ans:diff(ev(trg), x);
    multisel:multiselqn([ans], 1, wrongbase, 3);
    ta:multisel[1];
    variant:multisel[2];

In the above example there is only one correct answer, so we just select `1` from `[ans]`.
This is fine, and we then choose three randomly generated wrong answers.

This returns (for example) the values

    ta = [[-2*a*x*cos(a*x^2+b),false],[-sin(2*a*x),false],[a*cos(a*x^2+b),false],[-2*a*x*sin(a*x^2+b),true]];
    variant = [-2*a*x*cos(a*x^2+b),-sin(2*a*x),a*cos(a*x^2+b),-2*a*x*sin(a*x^2+b)];

The following function does a similar job when we have MCQ display strings.

    [ta, variant] = multiselqndisplay(corbase, numcor, wrongbase, numwrong)

For example, here the return values could be

    ta = [[3,false,2*a*x*sin(a*x^2+b)],[2,false,a*sin(a*x^2+b)],[5,false,cos(2*a*x)],[1,true,2*a*x*cos(a*x^2+b)]]
    variant = [3,2,5,1]

The function `multiselqndisplay` automatically assigns numbers \(1,\cdots, k\) to the `corbase` entries, and then \(k+1,\cdots, n\) to the `wrongbase` entries so that the numbers returned by the input type uniquely map to the entries in the two lists regardless of which random variant is generated.  These numbers are useful internally, but not for students.

The function `multiselqnalpha` automatically selects the appropriate number of correct and incorrect entries, permutes the list and then assigns strings `"(a)"`, `"(b)"`, `"(c)"` etc. in the correct order. The student's answer will thus be a list of strings (somewhat subverting the whole purpose of STACK in returning a genuine mathematical expression, but this has its place...).

    [ta, variant] = multiselqnalpha(corbase, numcor, wrongbase, numwrong, [dispstyle])

For example, here the return values (with inline maths) could be

    ta = [["(a)",false,"<b>(a)</b> \\(-2\\,a\\,x\\,\\sin \\left( a\\,x^2+b \\right)\\)"],["(b)",true,"<b>(b)</b> \\(2\\,a\\,x\\,\\cos \\left( a\\,x^2+b \\right)\\)"],["(c)",false,"<b>(c)</b> \\(\\cos \\left( 2\\,a\\,x \\right)\\)"],["(d)",false,"<b>(d)</b> \\(2\\,a\\,x\\,\\cos \\left( 2\\,a\\,x \\right)\\)"]]
    variant = [-2*a*x*sin(a*x^2+b),2*a*x*cos(a*x^2+b),cos(2*a*x),2*a*x*cos(2*a*x)]

Notes

1. The fifth optional argument to ` multiselqnalpha` controls the display style of the LaTeX.  Use `"i"` (a string containing `i`) for inline, `"d"` for displayed and `"id"` for inline "displaystyle" (which is the default).
2. Note that the "student's" answer will now be a _string_ such as `"(a)"`.  When you construct the PRT you will not need to check against these strings, not the original CAS expressions.
3. The `variant` retains the CAS statements, in order.
4. Do not use `multiselqnalpha` in conjunction with the `shuffle` option.  There is no need as the selection are permuted, and it messes up the order of the choices for the student.


## Dealing with strings in MCQ ##

A likely situation is that a teacher wants to include a language string as one of the options for a student's answer in a multiple-choice question.

Recall: *A fundamental design principal of STACK is that the student's answer should be a mathematical expression which can be
manipulated by the CAS as a valid expression.* Students are very limited in the keywords they are permitted to use in an input type.
It is very likely that strings will contain keywords forbidden in student expressions.

One option to overcome this is to do something like this as one option in the teacher's response:

    [C, false, "(C) None of the other options"]

The optional display part of this input is displayed to the student.  Their answer is the (valid) CAS atom `C` which the PRT will deal with appropriately.

To construct appropriate arrays use the `multiselqnalpha` function.

The quotation marks will be removed from strings, and the strings will not be wrapped `<code>...</code>` tags or LaTeX mathematics environments.

Question authors should consider using the Moodle MCQ question type in addition to these facilities for purely text-based answers.

You must protect characters within strings.  E.g. in

    [A, true, "A. There exists \\(M\\) such that \\( |a_n| &lt; M \\)."]

We have protected the backslash, and the inequality.

The language strings are not CAStext, they are simply raw strings.  It is possible to construct strings which include variable values using the `stack_disp` function.

    [oc(-inf,a), false, sconcat("The half interval: ", stack_disp(oc(-inf,a),"i"))]

The argument `"i"` here displays the expression "inline", other options are `""` (you are responsible for maths environments), `"d"` (displayed), and `"di"` (inline but using displaystyle).  If you construct strings in this way the display of any equations will not respect the display options in the particular input since variables are typically defined in the question variables and the input options are not available at that point in the code base.

For checkbox and radio inputs you could add modest HTML style to your strings.  E.g.

    ta:[[A, false, "(A) This is <b>bold</b>"], [B, true, "(B) this is <em>emphasis</em>"]];

Dropdown lists are created with the HTML `<select>` and `<option>` elements.  There is no support for HTML (or LaTeX) within dropdown lists.  You can use HTML-entities within a string field: see the separate documentation on this. 

## Inline CASText as MCQ labels ## {#castextlabels}

Since 4.4 it has been possible to write more complex labels using inline CASText. Inline CASText is basically a static string value wrapped in special function call and how one would use it is as follows:

    /* The old way of constructing a string: */
    [oc(-inf,a), false, sconcat("The half interval: ", stack_disp(oc(-inf,a),"i"))]
    /* Same using a inline CASText */
    [oc(-inf,a), false, castext("The half interval: {@oc(-inf,a)@}")]

You may write normal CASText syntax inside that string and it should behave exactly like it does in question-text or PRT feedback etc.. The only limitation at this time is that the list that includes these labels must be defined in the question-variables, you may not write inline CASText directly to the model answer field of the input. The `castext()`-function is not a real CAS-function it is converted to more complex logic at compile time and therefore it must receive a static string as its argument.

The most obvious use case for inline CASText is to provide localisation inside MCQ labels in situations where the mlang2-filter is not an option:

    [true, true, castext("[[lang code='en']]Yes[[/lang]][[lang code='fi']]Kyllä[[/lang]]")]

## Partial Marking for MCQs ##

Due to the wide variety of partially correct options in a typical MCQ, the usual method of creating a node in a PRT for each partially correct option and then traversing the tree is both brittle and unwieldy. To get around this, you can use the fact that the Score in a PRT can be defined by a Maxima value between 0 and 1 to dynamically generate the score.

The following example (written in the feedback variables of a PRT with student solution called `ans1` and the MCQ called `MCQ`) checks if the student has checked any incorrect solutions, and if they have assigns them 0. If they haven't, it assigns them the fraction they got correct:

     incorrectInputs : sublist(ans1, lambda([x], member(x, mcq_incorrect(MCQ))));
     mark : if emptyp(incorrectInputs) then length(ans1) / length(mcq_correct(MCQ)) else 0;

In this example, the output value `mark` can be placed into the Score of the false route in the MCQ PRT to get the desired marking.

## Writing question tests ##

Quality control of questions is important.  See the notes on [testing](../../STACK_question_admin/Testing.md) questions.

When entering test cases the question author must type in the CAS expression they expect to be the `value` of the student's answer (NOT the optional `display` field!).  For example, if the teacher's answer (to a checkbox) question is the following.

     ta:[[x^2-1,true],[x^2+1,false],[(x-1)*(x+1),true],[(x-i)*(x+i),false]]

Then the following test case contains all the "true" answers.

     [x^2-1,(x-1)*(x+1)]

There is currently minimal checking that the string entered by the teacher corresponds to a valid choice in the input type.  If your test case returns a blank result this is probably the problem.
