Setting MallocDebugReport=crash once the program is started

Question

Created Jun ’20

Replies 1

Boosts 0

Views 1.1k

Participants 2

I have dubious error message on stderr because of memory error that are correctly handled

✗ python3                               
 

Python 3.10.0a0 (heads/bpo-40937-universal-dirty:838a92a848, Jun 10 2020, 16:02:22) 
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> [0] * 10000000000000000                                                                                                                                               
python(41625,0x108c70dc0) malloc: can't allocate region

:*** mach_vm_map(size=80000000000004096, flags: 100) failed (error code=3)
python(41625,0x108c70dc0) malloc: *** set a breakpoint in malloc_error_break to debug
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError

The error is actually handled so this message should not be displayed. Setting the environment variable MallocDebugReport=crash correctly suppress the message but this is only checked once at the start of the program and this requires to change the environment in which the program is ran. Is there a possibility to set this once the program has started?

Boost

Answer 1

Developer Tools Engineer OP

Apple

Jun ’20

The system allocator (libmalloc) doesn't currently support that, and as you probably already know it reads the environment variables at startup and if stderr is a tty, it will by default print these warnings when asked to allocate too much memory. Please request an enhancement on this via Feedback Assistant.

In the meantime, the best workaround would probably be to artificially cap any allocation on the python side with a check whether the size is "reasonable" against some very large limit (e.g. 256 TB).

0