This document details initial planned data checks of the cognitive task data. These data checks are planned prior to the submission of the registered report. See below and the Open Science Workflow for more information on how we minimized the chance of bias.
To investigate whether the cognitive tasks that we identified as being suitable for Drift Diffusion Modeling (DDM) indeed have the required properties. These are 1) the NIH Flanker Task, 2) the NIH Pattern Comparison Processing Speed Task, 3) the NIH Picture Vocabulary Task, 4) The NIH Dimensional Change Card Sort Task, and 5) the Little Man Task. Suitability will be investigated in the following ways:
These data checks are performed prior to submitting the registered report. To prevent researcher bias, we developed an automated workflow that is based on automated tracking via Github. In short, the workflow is as follows (for more details, see here):
Note that in this phase, we will only read in and process cognitive task data. We will not look at any other variables that will play a role in the final data analyses - such as demographics and adversity measures.
On a task trial-level, we will exclude:
Response times < 300 ms and log-transformed response times > 3 SD from the mean of each participant;
Trials with missing response times or accuracy data.
On a participant-level, we will exclude participants who:
Have < 20 valid trials on a particular task (after trial-level exclusions);
Did not complete one or more cognitive tasks included in this preregistration.
Did not perform above chance-level on a particular task. Using a binomial distribution, we will calculate the expected accuracy (with 97.5% probability) that a participant would get when purely guessing.
At some point in the past suffered possible mild traumatic brain injury (TBI) or worse.
We will fit DDM to each task separately in a hierarchical Bayesian framework as implemented in the Python package HDDM (Wiecki et al., 2013). In each model, parameter z will be fixed to the mid-point (assuming no bias for a particular response option) and the inter-trial variability parameters are fixed to 0 in order to reduce model complexity. Parameters boundary separation (a), drift rate (v) and non-decision time (t0) are freely estimated. The model specification will look as follows:
For the Flanker Task and the Dimensional Card-Sorting Task, we compare the fit of two model versions: one in which a single value for a and v is estimated across task conditions (incongruent vs. congruent for the Flanker Task and switch vs. repeat trials for the Dimensional Card-Sorting Task) and one in which these parameters are estimated separately in each condition. The latter model specification will look as follows:
In the case of small differences, we favor model versions with one estimate across conditions given the low number of trials per condition.
For each task, we initially draw 5,000 samples from the posterior distribution while discarding the first 1,000 samples as burn-in. Model convergence is first investigated visually by plotting the trace, autocorrelation, and the marginal posterior (following Wiecki et al. (2013)). If a model does not converge properly, we will first try increasing the number of drawn samples to 10,000 (discarding the first 5,000 samples), 50,000 (discarding the first 45,000 samples) and if necessary to 100,000 samples (discarding 95,000 samples). In case high autocorrelation remains, we will apply thinning (taking 100,000 samples, discarding 50,000 and keeping every 10th sample). If none of these steps improve convergence, DDM might not be suitable for the task at hand.