The mitochondrial genome (mtDNA) is required for normal cellular function; inherited and somatic mutations in mtDNA lead to a variety of diseases. Saccharomyces cerevisiae has served as a model to study mtDNA integrity, in part because it can survive without mtDNA. A measure of defective mtDNA in S. cerevisiae is the formation of petite colonies. The frequency at which spontaneous petite colonies arise varies by ∼100-fold between laboratory and natural isolate strains. To determine the genetic basis of this difference, I applied quantitative trait locus (QTL) mapping to two strains at the opposite extremes of the phenotypic spectrum – the widely studied laboratory strain S288C and the vineyard isolate RM11-1a. Four main genetic determinants explain their phenotypic difference. An S288C-specific allele of MKT1 compromises the growth of petite cells. Moreover, strain-specific alleles of CAT5, MIP1, and SAL1 contribute to the high petite frequency of S288C and its derivatives by affecting the formation of petite colonies. The latter three alleles are present in the EM93 strain, the founder that contributed ∼88% of the S288C genome. Nearly all of the phenotypic difference between S288C and RM11-1a was reconstituted by introducing the common alleles of these four genes into the S288C background. In addition to the nuclear gene contribution, the source of the mtDNA influenced its stability. These results demonstrate that a few rare genetic variants with individually small effects can have a profound phenotypic effect in combination. Moreover, the polymorphisms identified in this study open new lines of investigation into mtDNA maintenance.