micropython/py
Damien George 9883d8e818 py/persistentcode: Maintain root ptr list of imported native .mpy code.
On ports where normal heap memory can contain executable code (eg ARM-based
ports such as stm32), native code loaded from an .mpy file may be reclaimed
by the GC because there's no reference to the very start of the native
machine code block that is reachable from root pointers (only pointers to
internal parts of the machine code block are reachable, but that doesn't
help the GC find the memory).

This commit fixes this issue by maintaining an explicit list of root
pointers pointing to native code that is loaded from an .mpy file.  This
is not needed for all ports so is selectable by the new configuration
option MICROPY_PERSISTENT_CODE_TRACK_RELOC_CODE.  It's enabled by default
if a port does not specify any special functions to allocate or commit
executable memory.

A test is included to test that native code loaded from an .mpy file does
not get reclaimed by the GC.

Fixes #6045.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-02 22:34:09 +10:00
..
argcheck.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
asmarm.c py/asm: Add funcs/macros to emit machine code for logical-shift-right. 2020-06-27 00:24:04 +10:00
asmarm.h py/asm: Add funcs/macros to emit machine code for logical-shift-right. 2020-06-27 00:24:04 +10:00
asmbase.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
asmbase.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
asmthumb.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
asmthumb.h py/asm: Add condition codes for signed comparisons. 2020-06-27 00:24:04 +10:00
asmx64.c py/asm: Add funcs/macros to emit machine code for logical-shift-right. 2020-06-27 00:24:04 +10:00
asmx64.h py/asm: Add condition codes for signed comparisons. 2020-06-27 00:24:04 +10:00
asmx86.c py/asm: Add funcs/macros to emit machine code for logical-shift-right. 2020-06-27 00:24:04 +10:00
asmx86.h py/asm: Add condition codes for signed comparisons. 2020-06-27 00:24:04 +10:00
asmxtensa.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
asmxtensa.h py/asm: Add funcs/macros to emit machine code for logical-shift-right. 2020-06-27 00:24:04 +10:00
bc.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
bc.h all: Format code to add space after C++-style comment start. 2020-04-23 11:24:25 +10:00
bc0.h py/bc0.h: Shift comment to start of line to improve format consistency. 2020-02-28 10:29:32 +11:00
binary.c all: Fix implicit floating point to integer conversions. 2020-04-18 22:42:24 +10:00
binary.h
builtin.h extmod/uasyncio: Add optional implementation of core uasyncio in C. 2020-03-26 01:25:45 +11:00
builtinevex.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
builtinhelp.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
builtinimport.c all: Format code to add space after C++-style comment start. 2020-04-23 11:24:25 +10:00
compile.c py/compile: Don't await __aiter__ special method in async-for. 2020-07-25 00:58:18 +10:00
compile.h
dynruntime.h py/dynruntime.h: Make mp_obj_str_get_str raise if arg not a str/bytes. 2020-06-08 09:05:03 +02:00
dynruntime.mk py/dynruntime.mk: Set MICROPY_ENABLE_DYNRUNTIME instead of per module. 2020-04-05 14:13:02 +10:00
emit.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
emitbc.c all: Format code to add space after C++-style comment start. 2020-04-23 11:24:25 +10:00
emitcommon.c
emitglue.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
emitglue.h
emitinlinethumb.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
emitinlinextensa.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
emitnarm.c
emitnative.c py/emitnative: Implement binary operations for viper uint operands. 2020-06-27 00:24:04 +10:00
emitnthumb.c
emitnx64.c
emitnx86.c py/nativeglue: Add new header file with native function table typedef. 2019-12-12 20:15:28 +11:00
emitnxtensa.c
emitnxtensawin.c
formatfloat.c all: Fix implicit conversion from double to float. 2020-04-18 22:42:24 +10:00
formatfloat.h
frozenmod.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
frozenmod.h
gc.c all: Format code to add space after C++-style comment start. 2020-04-23 11:24:25 +10:00
gc.h
grammar.h py/compile: Implement PEP 526, syntax for variable annotations. 2020-06-16 23:18:01 +10:00
lexer.c py/compile: Implement PEP 572, assignment expressions with := operator. 2020-06-16 22:02:24 +10:00
lexer.h py/compile: Implement PEP 572, assignment expressions with := operator. 2020-06-16 22:02:24 +10:00
makecompresseddata.py py/makecompresseddata.py: Make compression deterministic. 2020-04-20 10:32:49 +10:00
makemoduledefs.py all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
makeqstrdata.py all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
makeqstrdefs.py py: Implement "common word" compression scheme for error messages. 2020-04-05 14:20:57 +10:00
makeversionhdr.py all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
malloc.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
map.c py/objdict: Fix popitem for ordered dicts. 2020-04-27 23:53:17 +10:00
misc.h py/misc.h: Add missing semi-colon in mp_float_union_t for big-endian. 2020-06-22 13:42:24 +10:00
mkenv.mk py/mkenv.mk: Move usage of 32-bit flags to py.mk. 2020-01-12 10:34:10 +11:00
mkrules.mk py: Always give noop defines when MICROPY_ROM_TEXT_COMPRESSION disabled. 2020-04-14 22:13:11 +10:00
modarray.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
modbuiltins.c py/modbuiltins: Fix getattr to work with class raising AttributeError. 2020-06-02 15:42:20 +10:00
modcmath.c all: Format code to add space after C++-style comment start. 2020-04-23 11:24:25 +10:00
modcollections.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
modgc.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
modio.c py/modio: Allow uio.IOBase streams to return errno for read/write error. 2020-04-27 23:58:46 +10:00
modmath.c py/modmath: Work around msvc float bugs in atan2, fmod and modf. 2020-05-28 09:54:54 +10:00
modmicropython.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
modstruct.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
modsys.c py/modsys: Use consistent naming pattern for module-level const objects. 2020-05-28 10:02:14 +10:00
modthread.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
moduerrno.c all: Format code to add space after C++-style comment start. 2020-04-23 11:24:25 +10:00
mpconfig.h py/persistentcode: Maintain root ptr list of imported native .mpy code. 2020-08-02 22:34:09 +10:00
mperrno.h
mphal.h
mpprint.c all: Fix implicit conversion from double to float. 2020-04-18 22:42:24 +10:00
mpprint.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
mpstate.c
mpstate.h py/persistentcode: Maintain root ptr list of imported native .mpy code. 2020-08-02 22:34:09 +10:00
mpthread.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
mpz.c all: Format code to add space after C++-style comment start. 2020-04-23 11:24:25 +10:00
mpz.h tools/codeformat.py: Eliminate need for sizeof fixup. 2020-03-11 14:34:40 +11:00
nativeglue.c all: Fix implicit floating point promotion. 2020-04-18 22:36:14 +10:00
nativeglue.h py/nativeglue.h: Rename "setjmp" entry to "setjmp_" to avoid any clash. 2020-05-14 21:48:05 +10:00
nlr.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
nlr.h all: Add *FORMAT-OFF* in various places. 2020-02-28 10:31:07 +11:00
nlrpowerpc.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
nlrsetjmp.c
nlrthumb.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
nlrx64.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
nlrx86.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
nlrxtensa.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
obj.c py/objcomplex: Add mp_obj_get_complex_maybe for use in complex bin-op. 2020-06-27 01:03:10 +10:00
obj.h py/obj.h: Fix mp_seq_replace_slice_no_grow to use memmove not memcpy. 2020-07-21 01:05:59 +10:00
objarray.c py/objarray: Fix sign mismatch in comparison. 2020-04-18 22:42:19 +10:00
objarray.h
objattrtuple.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objbool.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objboundmeth.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objcell.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objclosure.c py: Rework mp_convert_member_lookup to properly handle built-ins. 2020-06-30 23:55:32 +10:00
objcomplex.c py/objcomplex: Add mp_obj_get_complex_maybe for use in complex bin-op. 2020-06-27 01:03:10 +10:00
objdeque.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
objdict.c py/obj.h: Add public mp_obj_is_dict_or_ordereddict() helper macro. 2020-06-24 12:05:00 +10:00
objenumerate.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objexcept.c py: Always give noop defines when MICROPY_ROM_TEXT_COMPRESSION disabled. 2020-04-14 22:13:11 +10:00
objexcept.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objfilter.c
objfloat.c all: Fix implicit floating point promotion. 2020-04-18 22:36:14 +10:00
objfun.c py: Rework mp_convert_member_lookup to properly handle built-ins. 2020-06-30 23:55:32 +10:00
objfun.h
objgenerator.c py: Rework mp_convert_member_lookup to properly handle built-ins. 2020-06-30 23:55:32 +10:00
objgenerator.h
objgetitemiter.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objint.c py/objint: Do not use fpclassify. 2020-04-18 22:42:24 +10:00
objint.h py/modsys: Use consistent naming pattern for module-level const objects. 2020-05-28 10:02:14 +10:00
objint_longlong.c py/modsys: Use consistent naming pattern for module-level const objects. 2020-05-28 10:02:14 +10:00
objint_mpz.c py/modsys: Use consistent naming pattern for module-level const objects. 2020-05-28 10:02:14 +10:00
objlist.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
objlist.h
objmap.c
objmodule.c extmod/uasyncio: Add optional implementation of core uasyncio in C. 2020-03-26 01:25:45 +11:00
objmodule.h py/objmodule.h: Remove obsolete mp_builtin_module_weak_links_map decl. 2020-02-11 15:43:13 +11:00
objnamedtuple.c all: Format code to add space after C++-style comment start. 2020-04-23 11:24:25 +10:00
objnamedtuple.h
objnone.c py/obj: Add MICROPY_OBJ_IMMEDIATE_OBJS option to reduce code size. 2020-01-13 01:01:45 +11:00
objobject.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
objpolyiter.c
objproperty.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objrange.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
objreversed.c
objset.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
objsingleton.c py/objsingleton: Use mp_generic_unary_op for singleton objects. 2019-12-27 12:53:36 +11:00
objslice.c all: Clean up error strings to use lowercase and change cannot to can't. 2020-04-13 22:19:37 +10:00
objstr.c all: Format code to add space after C++-style comment start. 2020-04-23 11:24:25 +10:00
objstr.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objstringio.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
objstringio.h
objstrunicode.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
objtuple.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
objtuple.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objtype.c py/objtype: Support passing in an OrderedDict to type() as the locals. 2020-06-24 12:05:40 +10:00
objtype.h py/runtime: Don't allocate iter buf for user-defined types. 2019-12-27 12:34:22 +11:00
objzip.c
opmethods.c py: Make mp_obj_get_type() return a const ptr to mp_obj_type_t. 2020-01-09 11:25:26 +11:00
pairheap.c py/pairheap: Properly unlink node on pop and delete. 2020-03-26 01:21:04 +11:00
pairheap.h py/pairheap: Add helper function to initialise a new node. 2020-03-26 01:21:04 +11:00
parse.c py/parse: Make mp_parse_node_extract_list return size_t instead of int. 2020-05-09 00:55:44 +10:00
parse.h py/parse: Make mp_parse_node_extract_list return size_t instead of int. 2020-05-09 00:55:44 +10:00
parsenum.c all: Fix implicit floating point promotion. 2020-04-18 22:36:14 +10:00
parsenum.h
parsenumbase.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
parsenumbase.h
persistentcode.c py/persistentcode: Maintain root ptr list of imported native .mpy code. 2020-08-02 22:34:09 +10:00
persistentcode.h py/persistentcode: Make ARM Thumb archs support multiple sub-archs. 2019-12-12 20:15:28 +11:00
profile.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
profile.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
py.mk py/py.mk: Use additional CFLAGS to compile string0.c. 2020-05-27 23:10:23 +10:00
pystack.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
pystack.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
qstr.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
qstr.h py: Implement "common word" compression scheme for error messages. 2020-04-05 14:20:57 +10:00
qstrdefs.h all: Add *FORMAT-OFF* in various places. 2020-02-28 10:31:07 +11:00
reader.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
reader.h
repl.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
repl.h
ringbuf.c
ringbuf.h extmod/modbluetooth: Make modbluetooth event not a bitfield. 2020-06-05 14:04:20 +10:00
runtime.c py/persistentcode: Maintain root ptr list of imported native .mpy code. 2020-08-02 22:34:09 +10:00
runtime.h py/scheduler: Convert mp_sched_full and mp_sched_num_pending to macros. 2020-05-08 23:20:45 +10:00
runtime0.h py/nativeglue: Add new header file with native function table typedef. 2019-12-12 20:15:28 +11:00
runtime_utils.c
scheduler.c py/scheduler: Convert mp_sched_full and mp_sched_num_pending to macros. 2020-05-08 23:20:45 +10:00
scope.c py/scope: Add assert to check that low numbered qstrs do fit in uint8_t. 2020-04-13 22:27:27 +10:00
scope.h py/compile: Convert scope test to SCOPE_IS_COMP_LIKE macro. 2020-06-16 21:42:37 +10:00
sequence.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
showbc.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
smallint.c
smallint.h
stackctrl.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
stackctrl.h
stream.c py/stream: Remove mp_stream_errno and use system errno instead. 2020-04-27 23:58:46 +10:00
stream.h py/stream.h: Include sys/types.h to get size_t and off_t for POSIX API. 2020-03-25 01:00:52 +11:00
unicode.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
unicode.h
vm.c py/scheduler: Fix race in checking scheduler pending state. 2020-04-13 21:55:47 +10:00
vmentrytable.h all: Add *FORMAT-OFF* in various places. 2020-02-28 10:31:07 +11:00
vstr.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
warning.c