Papa Bear likes his chair hard, his porridge hot and his browser written in a memory safe language that helps engineers avoid memory bugs like buffer overruns and use after frees.

But even Papa Bear has to compromise, part of Firefox is written in a memory safe language and the rest is written in C++. When using C++ there are a variety of defenses programmers can take to help catch memory errors. One of those is called memory poisoning.

mozjemalloc the memory allocator built into Firefox will poison memory by calling memset(aPtr, 0xE5, size); before freeing it. Any memory containing the pattern 0xE5E5E5E5 is therefore very likely to be memory that’s already been freed. This has two and a half benefits: If some code were to free and then dereference some memory (a use after free bug) it would most likely cause the browser to crash, which is much better than a potentially exploitable bug allowing Goldilocks to steal Papa Bear’s banking credentials! The other benefit is that when Firefox does crash due to such a use-after-free, the presence of this pattern in the crash report allows engineers to see the type of error that occurred and hopefully fix the mistake.

Note that back in March 2023 we moved the poison operation outside of the arena lock’s critical section; which improved performance in some tests.

You probably figured out by now that I’m going to persist with this metaphor. Mama Bear likes her chair soft, her porridge cold (and congealed (yuck)), and her browser fast.

But how much faster is Mama Bear’s experience? This is the question that was raised recently when Randell Jesup was benchmarking various memory allocators in Firefox. He noted that while mozjemalloc performs poisoning, many of the other allocators do not and to compare the performance of the allocators more fairly they should either all perform poisoning or none of them should.

And so Randell noted that, depending on the test, Firefox could be between 0.5% and 4% faster with poisoning disabled.

There are some results I collected. The "sp2" (Speedometer 2) and "sp3" (Speedometer 3) tests are browser benchmarks - larger numbers indicate better performance. The amazon and instagram tests are pageload tests measured in seconds with the ContentfulSpeedIndex metric - smaller numbers indicate better performance.

The speedometer figures are pretty close and these are the best pageload figures (the others showed very little difference but nothing regressed, yes I’m aware I’ve cherry-picked data).

This means that if it weren’t for the lack of security and debugability Mama Bear would have the right approach.

Baby Bear loves a compromise, they want their computer to be safe from Goldilocks' hacking attempts but also love performance improvements.

One compromise may be to probabilistic poison memory some of the time, e.g. a roughly 5% chance of poisoning. That’s more complex and involves a memory write anyway to keep the "time until poison" counter updated. We didn’t investigate it. But it’s worth noting that it would be similar in spirit to the Probabilistic Heap Checker (PHC) that’s rolling out in Firefox or the similar GWP-ASan capability in Chrome.

Instead we tested "what if we poison only the first cache line of a memory cell". Andrew McCreight and Olli Pettay pointed out that Element, a common DOM structure, is 128 bytes long and poisoning it is useful to detect memory errors in DOM code, as a lot of DOM code will involve Element.

We tested poisoning the first 64, 128 and 256 bytes of each structure. We assume that management of cache and writing cache lines back to RAM is going to be the dominant cost. Therefore we round-up our writes to the next cache line boundary..

For example, on a computer with 64-byte cache lines, if a 96-byte object is allocated so that the first 32-bytes is in one cache-line, while the next 64-bytes is in another. Our 64-byte write would cover two halves of different cache lines. In this case we will poison all 96-bytes because doing so writes to the same number of cache lines as the original 64-byte write.

Let’s add these options to our table of results.

As above, sp2 and sp3 are scores - bigger numbers are better. While amazon and instagram are page load tests where smaller numbers are better.

As expected the partial poisoning results fall between full and no poisoning. But what’s a little bit surprising is that in some tests (sp2 and amazon) poisoning a larger amount of memory made things faster. This could be because the memset() routine or the hardware itself is able to optimise larger writes more effectively. That said it’s important to acknowledge that the standard deviation is fairly high and doing the right statistical analysis is beyond this blog post.

Since poisoning more memory isn’t much slower and in some cases is faster than poisoning a little memory, then we might as well choose to poison 256 bytes which comfortably covers the Element object and most others and for the others it likely covers many of their most-often accessed fields. We’re confident that this is enough to help us catch many errors that can be caught with poisoning. While also performing well enough, especially for the pageload tests where it is closer to the performance available with poisoning disabled. We think that Baby Bear would agree, it is Just Right.

With the Probablistic Heap Checker (PHC) rolling out soon we will have an even greater ability to catch information related to memory errors. I’ll be writing about this in the future.

In some ways it feels more natural to lean in to (negative) gender stereotypes where Papa Bear wants things fast and Mama Bear is the cautious one. I considered this however to make comprehension easier it’s easier to explain poisoning before explaining turning poisoning off and the nursery tale describes Papa Bear’s preferences first, so that’s the order I introduced them here. Flipping the script on gender stereotypes was accidental.