ACES: Automated Correlation of Electric field strength and Stimulation effects for non-invasive brain stimulation

Abstract

Multiple sources of (protocol, inter-individual) variability contribute to the limited reliability of non-invasive brain stimulation (NIBS) findings [1]. Meta-analytical techniques could potentially even out such variability, but are hampered by the large parameter space involved [2]. Wischnewski and colleauges [3] recently proposed a partial solution, suggesting a novel approach to aggregate transcranial direct current stimulation (tDCS) studies using various montages and stimulation parameters. They simulated the electric field in a common head model using SimNIBS [4], an open source software which allows the user to model the fields induced by a specific NIBS protocol in a particular head model. The stimulation protocols were hence transformed to sets of merely quantitatively differing values in a common brain space, rendering them directly comparable. Subsequently, the electric field magnitude (|E|) values from these simulations were correlated to the effect sizes across studies, identifying loci where electric field magnitude was associated with an impact on the outcome of interest (i.e., working memory performance). An analogous method could in principle handle inter-subject variability, i.e., overcome morphological differences or aggregate data from different montages at the within-study level (see below). To facilitate adoption of this approach, we introduce Automated Correlation of Electric field strength and Stimulation effect (ACES), a MATLAB algorithm enabling the aggregation of NIBS findings on the meta-analytical or within-study level. To foster easy adoption, all input can be entered through the MATLAB GUI; no coding skills are required. Apart from user-friendly automatization and minor features, ACES incorporates three principal methodological advancements. First, ACES allows to weight studies for meta-analytical purposes. Second, ACES incorporates a cluster-based method [5] which can handle the spatial contiguity that typically characterises |E|. This approach can retrieve small areas featuring strong associations between |E| and stimulation effect as well as large areas where they are only moderately associated. Third, the cluster-based permutation test implemented in ACES features adequate multiple comparison control. This is critical, given that, typically , tens of thousands of correlations are involved. A detailed practical manual and the algorithm itself can be found here: https://osf. io/5rswh/. Fig. 1 gives a general overview of the procedure. In a first step, ACES correlates |E| with a quantification of the stimulation effect, across studies or participants. This correlation is computed for each of the elements making up the SimNIBS output mesh. The results is a mesh with a correlation per element (performance-electric field correlation or PEC [3]). For example, a large PEC at a given site reflects that studies featuring higher intensity stimulation at that location tended to report larger effect sizes. As NIBS effects can be nonlinear [6], ACES supports both Pearson and Spearman's rank correlation, although Pearson might still be preferable in such circumstances [7]. The Spearman correlation is implemented as a Pearson correlation on value ranks with fractional ranking in case of ties. ACES uses the studentized correlation coefficient [8]. For meta-analytical purposes, the precision of individual studies may be important to consider, approximated by sample size, or through more advances weighting schemes. Therefore, ACES can weight studies equally, by sample size, or another precision measure, by means of the weighted Pearson correlation between variables x and y given study weights N: m(x; N) = ∑ i N i x i ∑ i N i cov(x, y; N) = ∑ i N i (x i m(x; N))(y i m(y; N)) ∑ i N i corr(x, y; N) = cov(x, y; N) ̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅ cov(x, x; N)cov(y, y; N) √ As |E| of neighbouring mesh elements differs only gradually, PECs of neighbouring mesh elements are not independent. Cluster-based permutation tests can handle significance testing in the context of such spatial correlation [5], identifying contiguous clusters of mesh elements with significant PECs. Thus, in a second step, an arbitrary threshold is used to filter out small correlations and demarcate clusters. Negative correlations are set to zero-if desired, negative correlations can be investigated by inverting the sign of the input effect sizes. Next, t-values of above-threshold PECs are summed for each contiguous cluster, to be used for significance testing. Finally, for the permutation test, effect sizes are shuffled and PECs recomputed, using the same thresholding and clustering procedure as on the observed true PECs. In case of weighting, the correspondence between effect sizes and sample sizes is maintained throughout the permutations. The maximal cluster score and maximal individual t-value across mesh elements is stored for each permutation. The position of observed (peak or cluster) values in the ordered list of the permuted values is used as a criterion for statistical significance, thus controlling for multiple comparisons [5,8]; e.g., 95th percentile corresponds to a one-sided α = 0.05. ACES outputs a table with cluster size, peak and cluster t and p, and 3D coordinates of cluster peaks. Analyses on surface meshes can be easily visualized, and we provide example code to automatize this procedure. While originally developed for meta-analytical purposes, the logic of ACES can be expanded to handle inter-subject variability at the within-study level. If individual anatomical data form participants are available , SimNIBS allows the user to first simulate fields in each unique