Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable computation of probabilities by default for MKDAChi2 #856

Merged
merged 17 commits into from
Jan 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions docs/decoding.rst
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ The correlation-based decoding is implemented in NiMARE's `CorrelationDecoder` c
decoder = CorrelationDecoder(
frequency_threshold=0.001,
meta_estimator=mkda.MKDAChi2,
target_image='z_desc-specificity',
target_image='z_desc-association',
)
decoder.fit(ns_dset)
decoding_results = decoder.transform('pain_map.nii.gz')
Expand Down Expand Up @@ -140,7 +140,7 @@ of the database in a confusion matrix. For each label in the ontology, studies a
four groups: selected and label-positive (:math:`S_{s+l+}`), selected and label-negative
(:math:`S_{s+l-}`), unselected and label-positive (:math:`S_{s-l+}`), and unselected and
label-negative (:math:`S_{s-l-}`). Each method then compares these groups in order to evaluate
both consistency and specificity of the relationship between the selection criteria and each label,
both uniformity and association of the relationship between the selection criteria and each label,
which are evaluated in terms of both statistical significance and effect size.


Expand All @@ -162,10 +162,10 @@ individual peak, and so weighting based on the number of foci would be inappropr

This decoding method produces four outputs for each label. First, the distribution of studies in
the sample with the label are compared to the distributions of other labels within the sample.
This consistency analysis produces both a measure of statistical significance (i.e., a p-value)
This uniformity analysis produces both a measure of statistical significance (i.e., a p-value)
and a measure of effect size (i.e., the likelihood of being selected given the presence of the
label). Next, the studies in the sample are compared to the studies in the rest of the database.
This specificity analysis produces a p-value and an effect size measure of the posterior
This association analysis produces a p-value and an effect size measure of the posterior
probability of having the label given selection into the sample. A detailed algorithm description
is presented below.

Expand Down Expand Up @@ -261,17 +261,17 @@ those in a larger database, but, unlike the BrainMap method, does not take foci
this reason, the Neurosynth method would likely be more appropriate for selection criteria not
based on regions of interest (e.g., for characterizing meta-analytic groupings from a
meta-analytic clustering analysis). However, the Neurosynth method requires user-provided
information that BrainMap does not. Namely, in order to estimate probabilities for the consistency
and specificity analyses with Bayes' Theorem, the Neurosynth method requires a prior probability of
information that BrainMap does not. Namely, in order to estimate probabilities for the uniformity
and association analyses with Bayes' Theorem, the Neurosynth method requires a prior probability of
a given label. Typically, a value of 0.5 is used (i.e., the estimated probability that an
individual is undergoing a given mental process described by a label, barring any evidence from
neuroimaging data, is predicted to be 50%). This is, admittedly, a poor prediction, which means
that probabilities estimated based on this prior are not likely to be accurate, though they may
still serve as useful estimates of effect size for the analysis.

Like the BrainMap method, this method produces four outputs for each label. For the consistency
Like the BrainMap method, this method produces four outputs for each label. For the uniformity
analysis, this method produces both a p-value and a conditional probability of selection given the
presence of the label and the prior probability of having the label. For the specificity analysis,
presence of the label and the prior probability of having the label. For the association analysis,
the Neurosynth method produces both a p-value and a posterior probability of presence of the label
given selection and the prior probability of having the label. A detailed algorithm description is
presented below.
Expand Down
4 changes: 2 additions & 2 deletions examples/02_meta-analyses/01_plot_cbma.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,13 +84,13 @@
cres = corr.transform(results)

plot_stat_map(
results.get_map("z_desc-consistency"),
results.get_map("z_desc-uniformity"),
draw_cross=False,
cmap="RdBu_r",
threshold=0.1,
)
plot_stat_map(
cres.get_map("z_desc-consistencySize_level-cluster_corr-FWE_method-montecarlo"),
cres.get_map("z_desc-uniformitySize_level-cluster_corr-FWE_method-montecarlo"),
draw_cross=False,
cmap="RdBu_r",
threshold=0.1,
Expand Down
4 changes: 2 additions & 2 deletions examples/02_meta-analyses/07_macm.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,9 @@
corr = FWECorrector(method="montecarlo", n_iters=10000)
cres = corr.transform(results)

# We want the "specificity" map (2-way chi-square between sel and unsel)
# We want the "association" map (2-way chi-square between sel and unsel)
plotting.plot_stat_map(
cres.get_map("z_desc-consistency_level-voxel_corr-FWE_method-montecarlo"),
cres.get_map("z_desc-uniformity_level-voxel_corr-FWE_method-montecarlo"),
threshold=3.09,
draw_cross=False,
cmap="RdBu_r",
Expand Down
2 changes: 1 addition & 1 deletion examples/02_meta-analyses/11_plot_cbmr.py
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,7 @@

###############################################################################
# Now that we have done group comparison tests with the specified contrast matrix,
# we can plot the z-score maps indicating consistency in activation regions among
# we can plot the z-score maps indicating uniformity in activation regions among
# all four groups.

plot_stat_map(
Expand Down
2 changes: 1 addition & 1 deletion nimare/decode/continuous.py
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ def __init__(
features=None,
frequency_threshold=0.001,
meta_estimator=None,
target_image="z_desc-specificity",
target_image="z_desc-association",
n_cores=1,
):
meta_estimator = (
Expand Down
6 changes: 3 additions & 3 deletions nimare/decode/discrete.py
Original file line number Diff line number Diff line change
Expand Up @@ -325,7 +325,7 @@ def brainmap_decode(
n_selected_term - np.mean(n_selected_term)
).ravel() # pylint: disable=no-member

# Two-way chi-square test for specificity of activation
# Two-way chi-square test for association of activation
cells = np.array(
[
[n_selected_term, n_selected_noterm], # pylint: disable=no-member
Expand Down Expand Up @@ -605,14 +605,14 @@ def neurosynth_decode(
prior = p_term

# Significance testing
# One-way chi-square test for consistency of term frequency across terms
# One-way chi-square test for uniformity of term frequency across terms
chi2_fi = one_way(n_selected_term, n_term)
p_fi = special.chdtrc(1, chi2_fi)
sign_fi = np.sign(
n_selected_term - np.mean(n_selected_term)
).ravel() # pylint: disable=no-member

# Two-way chi-square test for specificity of activation
# Two-way chi-square test for association
cells = np.array(
[
[n_selected_term, n_selected_noterm], # pylint: disable=no-member
Expand Down
4 changes: 2 additions & 2 deletions nimare/diagnostics.py
Original file line number Diff line number Diff line change
Expand Up @@ -310,9 +310,9 @@ def _transform(self, expid, label_map, sign, result):
cluster_masker.fit(label_map)

# CBMAs have "stat" maps, while most IBMAs have "est" maps. ALESubtraction has
# stat_desc-group1MinusGroup2" maps, while MKDAChi2 has "z_desc-specificity" maps.
# stat_desc-group1MinusGroup2" maps, while MKDAChi2 has "z_desc-association' maps.
# Fisher's and Stouffer's only have "z" maps though.
target_value_keys = {"stat", "est", "stat_desc-group1MinusGroup2", "z_desc-specificity"}
target_value_keys = {"stat", "est", "stat_desc-group1MinusGroup2", "z_desc-association"}
avail_value_keys = set(result.maps.keys())
union_value_keys = list(target_value_keys & avail_value_keys)
target_value_map = union_value_keys[0] if union_value_keys else "z"
Expand Down
4 changes: 2 additions & 2 deletions nimare/meta/cbma/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -372,7 +372,7 @@ def _p_to_summarystat(self, p, null_method=None):
null_distribution /= np.max(null_distribution)
null_distribution = np.squeeze(null_distribution)

# Desired bin is the first one _before_ the target p-value (for consistency
# Desired bin is the first one _before_ the target p-value (for uniformity
# with the montecarlo null).
ss_idx = np.maximum(0, np.where(null_distribution <= p)[0][0] - 1)
ss = self.null_distributions_["histogram_bins"][ss_idx]
Expand All @@ -382,7 +382,7 @@ def _p_to_summarystat(self, p, null_method=None):
assert "histweights_corr-none_method-montecarlo" in self.null_distributions_.keys()

hist_weights = self.null_distributions_["histweights_corr-none_method-montecarlo"]
# Desired bin is the first one _before_ the target p-value (for consistency
# Desired bin is the first one _before_ the target p-value (for uniformity
# with the montecarlo null).
ss_idx = np.maximum(0, np.where(hist_weights <= p)[0][0] - 1)
ss = self.null_distributions_["histogram_bins"][ss_idx]
Expand Down
Loading
Loading