Researchers sometimes use informal judgment for statistical model diagnostics and assumption checking. Informal judgment might seem more desirable than formal judgment because of a paradox: Formal hypothesis tests of assumptions appear to become less useful as sample size increases. We suggest that this paradox can be resolved by evaluating both formal and informal statistical judgment via a simplified signal detection framework. In 4 studies, we used this approach to compare informal judgments of normality diagnostic graphs (histograms, Q-Q plots, and P-P plots) to the performance of several formal tests (Shapiro-Wilk test, Kolmogorov-Smirnov test, etc.). Participants judged whether or not graphs of sample data came from a normal population (Experiments 1-2) or whether or not from a population close enough to normal for a parametric test to be more powerful than a nonparametric one (Experiments 3-4). Across all experiments, participants\' informal judgments showed lower discriminability than did formal hypothesis tests. This pattern occurred even after participants were given 400 training trials with feedback, a financial incentive, and ecologically valid distribution shapes. The discriminability advantage of formal normality tests led to slightly more powerful follow-up tests (parametric vs. nonparametric). Overall, the framework used here suggests that formal model diagnostics may be more desirable than informal ones.

The primary aim of this study was to document the developmental course of distinctiveness effects throughout childhood. Specifically, we examined whether the reduction in false recognition rates that is traditionally observed in children after distinctive encoding could be explained not only by enhanced discrimination between studied and new items but also by the implementation of a conservative response criterion resulting from the use of metacognitive expectations about the quality of memories (i.e., distinctiveness heuristic). Two experiments were conducted in which children in three age groups-4-5, 6-7, and 8-9 years-were asked to study a set of items presented in either pictorial (distinctive) or word (less distinctive) form. In Experiment 1, pictures and words were displayed in two separate lists, a design that is supposed to favor reliance on the distinctiveness heuristic. In Experiment 2, the two types of stimuli were presented within the same list, a design that is supposed to make using the metacognitive heuristic ineffective. Overall, Experiments 1 and 2 provide evidence that children as young as 4 years rely on the distinctiveness heuristic to guide their memory decisions, resulting in a reduction in the false recognition rate when items are presented using a pure-list design (Experiment 1) but not when they are presented using a mixed-list design (Experiment 2). The implications of these findings for our understanding of the development of metacognition and the involvement of metacognitive skills in children\'s memory performance are discussed.

Previous studies provided contradicting results regarding metacognitive sensitivity estimated from subjective reports of confidence in comparison to subjective reports of visual experience. We investigated whether this effect of content of subjective reports is influenced by the statistical method to quantify metacognitive sensitivity. Comparing logistic regression and meta-d in a masked orientation task, a masked shape task, and a random-dot motion task, we observed metacognitive sensitivity of reports regarding decisional confidence was greater than of reports about visual experience irrespective of mathematical procedures. However, the relationship between subjective reports and the logistic transform of accuracy was often not linear, implying that logistic regression is not a consistent measure of metacognitive sensitivity. We argue that a science of consciousness would benefit from the assessment of both visual experience and decisional confidence, and recommend meta-d as measure of metacognitive sensitivity for future studies.