LoboVault Home

Examining the advantages and disadvantages of pilot studies : Monte-Carlo simulations


Please use this identifier to cite or link to this item: http://hdl.handle.net/1928/17492

Examining the advantages and disadvantages of pilot studies : Monte-Carlo simulations

Show full item record

Title: Examining the advantages and disadvantages of pilot studies : Monte-Carlo simulations
Author: Nakazawa, Masato
Advisor(s): Delaney, Harold
Committee Member(s): Bryan, Angela
Goldsmith, Timothy
Parkes, Jay
Department: University of New Mexico. Dept. of Psychology
Subject: power, estimation of population effect size, pilot studies, economic performance
LC Subject(s): Experimental design.
Sampling (Statistics)
Degree Level: Doctoral
Abstract: Estimating population effect size accurately and precisely plays a vital role in achieving a desired level of statistical power as well as drawing correct conclusions from empirical results. While a number of common practices of effect-size estimation have been documented (e.g., relying on one’s experience and intuition, and conducting pilot studies), their relative advantages and disadvantages have been insufficiently investigated. To establish a practical guideline for researchers in this respect, this project compared the accuracy and precision of effect-size estimation, resulting power, and economic implications across pilot and non-pilot conditions. Furthermore, to model the potential advantages of pilot studies in finding and correcting flaws before main studies are run, varying amounts of random error variance and varying degrees of success at its removal – often neglected aspects in simulation studies – were introduced in Experiment 2. The main findings include the following. First, pilot studies with up to 30 subjects were utterly ineffective in achieving the desired power of 0.80 at a small population effect size even under the best-case scenario. At this effect size intuitive estimation without pilot studies appears to be the preferred method of achieving the desired power. Second, the pilot studies performed better at medium and large population effect sizes, achieving comparable or even greater power to that in the non-pilot condition. The relative advantages of pilot studies were particularly evident when moderate to large error variances were present, and a portion of it had been removed through conducting pilot studies. These broad findings are discussed in the context of flexible design: study design can be modified flexibly in accordance with the researcher’s particular goals.
Graduation Date: December 2011
URI: http://hdl.handle.net/1928/17492

Files in this item

Files Size Format View Description
Dissertation Nakazawa.pdf 3.342Mb PDF View/Open pilot studies

This item appears in the following Collection(s)

Show full item record

UNM Libraries

Search LoboVault


My Account