segfault on reload

[petrelharp]

On my machine, running this script:

initialize()
{
    setSeed(23);
    initializeTreeSeq(timeUnit="generations");
    initializeSex();
    initializeMutationType("m1", 0.5, "f", 0.0);
    initializeGenomicElementType("g1", m1, 1.0);
    initializeGenomicElement(g1, 0, 999999);
    initializeMutationRate(1e-3);
    initializeRecombinationRate(0.0);
}

1 early() { 
    sim.addSubpop("p1", 10);
}

2 late() {
    sim.treeSeqOutput("out.trees");
    sim.simulationFinished();
}

followed by this script:

initialize()
{
   defineConstant("L", 1e2);
   initializeTreeSeq(timeUnit="generations");
   initializeSex();
   initializeMutationType("m1", 0.5, "f", 0.0);
   initializeGenomicElementType("g1", m1, 1.0);
   initializeGenomicElement(g1, 0, 999999);
   initializeMutationRate(1e-3);
   initializeRecombinationRate(0.0);
}

1 late() { 
    sim.readFromPopulationFile("out.trees");
}

results in a segfault. I haven't tried to track down where the segfault is (this happened in the middle of finding another bug).

My only possible clue is that this script involves a lot of stacked mutations.

[bhaller]

Thanks, I'll look into it!

[bhaller]

Hmm, on my machine I get "ERROR (Species::__CreateMutationsFromTabulation): mutation type m38161 has not been defined for this species." This check does not appear to be debug-only. Interesting that you get a crash rather than this error. Anyhow, obviously "m38161" is a garbage value, so something is clearly very wrong here; I'll investigate further. [EDIT: Ah, now I'm getting EXC_BAD_ACCESS. :->]

[bhaller]

(Notes to myself:) OK, this turns out to be a bit subtle, and at quite a low level. When NewMutationFromBlock() is called, to allocate a new mutation, it potentially calls IncreaseMutationBlockCapacity() to expand the mutation block capacity, and that reallocs the mutation block's mutation_buffer_, refcount_buffer_, and trait_info_buffer_. It turns out there is at least one place in the code that caches the pointer to one of those buffers while doing a bunch of work, and the cached pointer goes invalid across the realloc. Pointers to those buffers cannot be kept across allocation of new mutations, for this reason. I think bugs were introduced in this area when I added the MutationBlock class, probably, and haven't been hit until now because (a) a realloc only gets triggered when the initial capacity of 16384 mutations is exceeded, and (b) a realloc often doesn't need to move the pointer, if you get lucky; if the heap happens to have space for more capacity, it can expand the existing buffers into the unused memory without doing a memory move, and then everything is fine. So, it is an occasional crasher that occurs only in relatively large models. Yikes. I need to check every place in the code that locally caches a pointer to one of those buffers, and make sure a pointer is never held across a potential realloc.