# Analyze responses 

Basic reporting is undertaken through the Moodle quiz.  For example, lists of scores, etc. are available there.  The Moodle quiz also calculates basic statistics based on the numerical information.

To review work in more detail we need to use two important parts of the question. Please read the following two entries before continuing with this article.

* The [question note](../Authoring/Question_note.md).
* The [answer note](../Authoring/Potential_response_trees.md#Answer_note).

## Moodle quiz features ##

Most of the day to day reporting is done through the Moodle quiz features.

For detailed reporting go to

    Quiz administration -> Results -> Reporting

The "response" column shows the raw Maxima input value entered by the student and the validity status, e.g. "blank" (not reported), "invalid", "valid", "score".  The "response" column also shows the value of the [answer note](../Authoring/Potential_response_trees.md#Answer_note) for each potential response tree.

An example note is shown below.  In this question a student has answered with `(x-4)^6+c` for the input `ans1`.
The potential response tree `prt1` has executed and the note is `ATInt_generic. | 1-0-F`.

    ans1: (x-4)^6+c [score]; prt1: ATInt_generic. | 1-0-F

Notes:

1. If the potential response tree does not execute it will be reported as `prt1: #` to indicate it exists but was not used.
2. Reporting is stored in the database, so you may need to "regrade" attempts if you update the answer notes.
3. You can download the responses for offline analysis.  Split over `;` to separate inputs and separate PRT answer notes.  Split the individual answer note over `|` to get the route through the tree, and the notes added by the answer tests.

## Individual STACK item analysis ##

STACK has a bespoke basic reporting mechanism for analysing responses to questions.  To access the report you need to navigate to the STACK question dashboard and choose "Analyze responses".  Choose the quiz in which the question is used.

The report makes use of the response summary.  (For developers, this is generated by `qtype_stack_question::summarise_response` in `question.php`.)

The response summary is a string consisting of the following information.

* The "seed" for the question used, enabling attempts to be grouped by question variant.
* The Maxima string value of each input, and its status (blank, invalid, valid, score).
* The answernote generated by each potential response tree. `!` denotes a blank note, i.e. the PRT did not execute.

Here is a raw response summary.

     Seed: 254530683; ans1: 15180 [score]; ans2: 24360 [score]; prt1: # = 1 | prt1-1-T; prt2: # = 0 | prt2-1-F

Here we see the following.

* The report can be split over `;` into separate fields.
  Within the PRTs the data can be split over `|`.
* The seed used in this example was 254530683.
* There are two inputs, each at the "score" state.
  * `ans1: 15180 [score]`
  * `ans2: 24360 [score]`
* There are two potential response trees
  * `prt1: # = 1 | prt1-1-T`  The `prt1` was activated, the score `#` returned was `1` and we see the answer note.
  * `prt2: # = 0 | prt2-1-F`  The `prt2` was activated, the score `#` returned was `0` and we see the answer note.

The response summary contains a lot of information in a condensed way, and can only be understood with an appreciation of the question structure itself in STACK (inputs, prts, variants) and the individual question.

The report splits this information into the following, text-based, reports.

* Frequency of answer notes, for each PRT, regardless of which variant was used.  Useful to know the route through the tree.
* Frequency of answer notes, for each PRT, split by |, regardless of which variant was used.   Useful to understand the outcome at each node.
* Raw inputs, regardless of which variant was used.
* Raw inputs and PRT answer notes by variant.
* Raw data.

Note, the question summary is stored in a database and will not be updated if you subsequently edit the question.  If you wish to improve your question you will need to do a quiz "regrade" to update the basic question use report.

## Download JSON data

Rather than provide complex analysis online, we provide download of data in JSON format for offline analysis.  Choose the tab "JSON data".

The output has two parts.

1. Minimal metadata about the question.  E.g. `{"id":"1729","name":"3.2.1: Completing the square","reporttime":"2025-08-28T10:18:42+01:00"}`.  The metadata has only three fields.
    1. The `id` of the question from the Moodle database.
    2. The `name` of the question (which should be human-readable)
    3. The `reporttime` which is the time the _report is created_ for download.
2. JSON objects, one response per line.

Here is an example line of data.

````
{"userid":"736","timecreated":"1756387076","inputs":{"ans1":{"status":"score","note":"","value":"4*(x-5)^2-4"}},"prts":{"prt1":{"score":1,"penalty":0,"note":["prt1-1-T","ATCompSquare_true.","prt1-2-T"],"errors":[]}},"Seed":1569710461}
````

which encodes the following student's single response:

````
{
    "userid": "736",
    "timecreated": "1756387076",
    "inputs": {
        "ans1": {
            "status": "score",
            "note": "",
            "value": "4*(x-5)^2-4"
        }
    },
    "prts": {
        "prt1": {
            "score": 1,
            "penalty": 0,
            "note": [
                "prt1-1-T",
                "ATCompSquare_true.",
                "prt1-2-T"
            ],
            "errors": []
        }
    },
    "Seed": 1569710461
}
````

If you save your data in a file `data.txt` then the following is minmal Python script to read in and decode the data.

````
# Example Python code to load in JSON data from the STACK response analysis download.
import json

def load_json_lines(file_path):
    dataset = []
    with open(file_path, "r", encoding="utf-8") as f:
        for line_number, line in enumerate(f, start=1):
            line = line.strip()
            if not line:
                continue  # skip empty lines
            try:
                data = json.loads(line)
                pretty = json.dumps(data, indent=4)
                print(f"Line {line_number}:\n{pretty}\n")
                dataset.append(data)
            except json.JSONDecodeError as e:
                print(f"Error parsing line {line_number}: {e}")
    return dataset

# Example usage:
if __name__ == "__main__":
    file_path = "data.txt"  # Replace with your filename
    dataset = load_json_lines(file_path)
````


## Site-wide usage ##

Moodle Admins can get stats on [site-wide STACK usage](../STACK_question_admin/Advanced_reporting.md). 