all: Remove MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE.

This commit removes all parts of code associated with the existing
MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE optimisation option, including the
-mcache-lookup-bc option to mpy-cross.

This feature originally provided a significant performance boost for Unix,
but wasn't able to be enabled for MCU targets (due to frozen bytecode), and
added significant extra complexity to generating and distributing .mpy
files.

The equivalent performance gain is now provided by the combination of
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE (which has
been enabled on the unix port in the previous commit).

It's hard to provide precise performance numbers, but tests have been run
on a wide variety of architectures (x86-64, ARM Cortex, Aarch64, RISC-V,
xtensa) and they all generally agree on the qualitative improvements seen
by the combination of MICROPY_OPT_LOAD_ATTR_FAST_PATH and
MICROPY_OPT_MAP_LOOKUP_CACHE.

For example, on a "quiet" Linux x64 environment (i3-5010U @ 2.10GHz) the
change from CACHE_MAP_LOOKUP_IN_BYTECODE, to LOAD_ATTR_FAST_PATH combined
with MAP_LOOKUP_CACHE is:

diff of scores (higher is better)
N=2000 M=2000       bccache -> attrmapcache      diff      diff% (error%)
bm_chaos.py        13742.56 ->   13905.67 :   +163.11 =  +1.187% (+/-3.75%)
bm_fannkuch.py        60.13 ->      61.34 :     +1.21 =  +2.012% (+/-2.11%)
bm_fft.py         113083.20 ->  114793.68 :  +1710.48 =  +1.513% (+/-1.57%)
bm_float.py       256552.80 ->  243908.29 : -12644.51 =  -4.929% (+/-1.90%)
bm_hexiom.py         521.93 ->     625.41 :   +103.48 = +19.826% (+/-0.40%)
bm_nqueens.py     197544.25 ->  217713.12 : +20168.87 = +10.210% (+/-3.01%)
bm_pidigits.py      8072.98 ->    8198.75 :   +125.77 =  +1.558% (+/-3.22%)
misc_aes.py        17283.45 ->   16480.52 :   -802.93 =  -4.646% (+/-0.82%)
misc_mandel.py     99083.99 ->  128939.84 : +29855.85 = +30.132% (+/-5.88%)
misc_pystone.py    83860.10 ->   82592.56 :  -1267.54 =  -1.511% (+/-2.27%)
misc_raytrace.py   21490.40 ->   22227.23 :   +736.83 =  +3.429% (+/-1.88%)

This shows that the new optimisations are at least as good as the existing
inline-bytecode-caching, and are sometimes much better (because the new
ones apply caching to a wider variety of map lookups).

The new optimisations can also benefit code generated by the native
emitter, because they apply to the runtime rather than the generated code.
The improvement for the native emitter when LOAD_ATTR_FAST_PATH and
MAP_LOOKUP_CACHE are enabled is (same Linux environment as above):

diff of scores (higher is better)
N=2000 M=2000        native -> nat-attrmapcache  diff      diff% (error%)
bm_chaos.py        14130.62 ->   15464.68 :  +1334.06 =  +9.441% (+/-7.11%)
bm_fannkuch.py        74.96 ->      76.16 :     +1.20 =  +1.601% (+/-1.80%)
bm_fft.py         166682.99 ->  168221.86 :  +1538.87 =  +0.923% (+/-4.20%)
bm_float.py       233415.23 ->  265524.90 : +32109.67 = +13.756% (+/-2.57%)
bm_hexiom.py         628.59 ->     734.17 :   +105.58 = +16.796% (+/-1.39%)
bm_nqueens.py     225418.44 ->  232926.45 :  +7508.01 =  +3.331% (+/-3.10%)
bm_pidigits.py      6322.00 ->    6379.52 :    +57.52 =  +0.910% (+/-5.62%)
misc_aes.py        20670.10 ->   27223.18 :  +6553.08 = +31.703% (+/-1.56%)
misc_mandel.py    138221.11 ->  152014.01 : +13792.90 =  +9.979% (+/-2.46%)
misc_pystone.py    85032.14 ->  105681.44 : +20649.30 = +24.284% (+/-2.25%)
misc_raytrace.py   19800.01 ->   23350.73 :  +3550.72 = +17.933% (+/-2.79%)

In summary, compared to MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE, the new
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE options:
- are simpler;
- take less code size;
- are faster (generally);
- work with code generated by the native emitter;
- can be used on embedded targets with a small and constant RAM overhead;
- allow the same .mpy bytecode to run on all targets.

See #7680 for further discussion.  And see also #7653 for a discussion
about simplifying mpy-cross options.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
pull/7704/head^2
Jim Mussared 1 year ago committed by Damien George
parent 60c6d5594f
commit b326edf68c
  1. 2
      docs/reference/mpyfiles.rst
  2. 1
      examples/embedding/mpconfigport_minimal.h
  3. 5
      mpy-cross/README.md
  4. 6
      mpy-cross/main.c
  5. 2
      mpy-cross/mpconfigport.h
  6. 1
      ports/cc3200/mpconfigport.h
  7. 1
      ports/nrf/mpconfigport.h
  8. 1
      ports/stm32/mpconfigport.h
  9. 5
      ports/unix/Makefile
  10. 3
      ports/unix/mpconfigport.h
  11. 1
      ports/unix/variants/minimal/mpconfigvariant.h
  12. 1
      ports/windows/Makefile
  13. 1
      ports/windows/mpconfigport.h
  14. 2
      ports/windows/msvc/genhdr.targets
  15. 14
      py/bc.c
  16. 2
      py/dynruntime.mk
  17. 6
      py/emitbc.c
  18. 9
      py/mpconfig.h
  19. 1
      py/mpstate.h
  20. 11
      py/persistentcode.h
  21. 12
      py/profile.c
  22. 12
      py/showbc.c
  23. 101
      py/vm.c
  24. 10
      tests/cmdline/cmd_parsetree.py.exp
  25. 24
      tests/cmdline/cmd_showbc.py.exp
  26. 12
      tests/cmdline/cmd_verbose.py.exp
  27. 4
      tests/micropython/import_mpy_native_gc.py
  28. 6
      tests/micropython/import_mpy_native_x64.py
  29. 4
      tests/run-tests.py
  30. 28
      tools/mpy-tool.py
  31. 5
      tools/mpy_cross_all.py
  32. 9
      tools/mpy_ld.py

@ -66,8 +66,6 @@ If importing an .mpy file fails then try the following:
print('mpy flags:', end='')
if arch:
print(' -march=' + arch, end='')
if sys_mpy & 0x100:
print(' -mcache-lookup-bc', end='')
if not sys_mpy & 0x200:
print(' -mno-unicode', end='')
print()

@ -45,7 +45,6 @@
#define MICROPY_LONGINT_IMPL (MICROPY_LONGINT_IMPL_NONE)
#define MICROPY_STREAMS_NON_BLOCK (0)
#define MICROPY_OPT_COMPUTED_GOTO (0)
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0)
#define MICROPY_CAN_OVERRIDE_BUILTINS (0)
#define MICROPY_BUILTIN_METHOD_CHECK_SELF_ARG (0)
#define MICROPY_CPYTHON_COMPAT (0)

@ -17,10 +17,7 @@ by the target MicroPython runtime (eg onto a pyboard's filesystem), and then
imported like any other Python module using `import foo`.
Different target runtimes may require a different format of the compiled
bytecode, and such options can be passed to the cross compiler. For example,
the unix port of MicroPython requires the following:
$ ./mpy-cross -mcache-lookup-bc foo.py
bytecode, and such options can be passed to the cross compiler.
If the Python code contains `@native` or `@viper` annotations, then you must
specify `-march` to match the target architecture.

@ -108,7 +108,6 @@ STATIC int usage(char **argv) {
"Target specific options:\n"
"-msmall-int-bits=number : set the maximum bits used to encode a small-int\n"
"-mno-unicode : don't support unicode in compiled strings\n"
"-mcache-lookup-bc : cache map lookups in the bytecode\n"
"-march=<arch> : set architecture for native emitter; x86, x64, armv6, armv7m, armv7em, armv7emsp, armv7emdp, xtensa, xtensawin\n"
"\n"
"Implementation specific options:\n", argv[0]
@ -205,7 +204,6 @@ MP_NOINLINE int main_(int argc, char **argv) {
// set default compiler configuration
mp_dynamic_compiler.small_int_bits = 31;
mp_dynamic_compiler.opt_cache_map_lookup_in_bytecode = 0;
mp_dynamic_compiler.py_builtins_str_unicode = 1;
#if defined(__i386__)
mp_dynamic_compiler.native_arch = MP_NATIVE_ARCH_X86;
@ -264,10 +262,6 @@ MP_NOINLINE int main_(int argc, char **argv) {
return usage(argv);
}
// TODO check that small_int_bits is within range of host's capabilities
} else if (strcmp(argv[a], "-mno-cache-lookup-bc") == 0) {
mp_dynamic_compiler.opt_cache_map_lookup_in_bytecode = 0;
} else if (strcmp(argv[a], "-mcache-lookup-bc") == 0) {
mp_dynamic_compiler.opt_cache_map_lookup_in_bytecode = 1;
} else if (strcmp(argv[a], "-mno-unicode") == 0) {
mp_dynamic_compiler.py_builtins_str_unicode = 0;
} else if (strcmp(argv[a], "-municode") == 0) {

@ -57,8 +57,6 @@
#define MICROPY_COMP_TRIPLE_TUPLE_ASSIGN (1)
#define MICROPY_COMP_RETURN_IF_EXPR (1)
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0)
#define MICROPY_READER_POSIX (1)
#define MICROPY_ENABLE_RUNTIME (0)
#define MICROPY_ENABLE_GC (1)

@ -53,7 +53,6 @@
#define MICROPY_LONGINT_IMPL (MICROPY_LONGINT_IMPL_MPZ)
#define MICROPY_FLOAT_IMPL (MICROPY_FLOAT_IMPL_NONE)
#define MICROPY_OPT_COMPUTED_GOTO (0)
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0)
#define MICROPY_READER_VFS (1)
#ifndef DEBUG // we need ram on the launchxl while debugging
#define MICROPY_CPYTHON_COMPAT (1)

@ -66,7 +66,6 @@
#endif
#define MICROPY_OPT_COMPUTED_GOTO (0)
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0)
#define MICROPY_OPT_MPZ_BITWISE (0)
// fatfs configuration used in ffconf.h

@ -59,7 +59,6 @@
#ifndef MICROPY_OPT_COMPUTED_GOTO
#define MICROPY_OPT_COMPUTED_GOTO (1)
#endif
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0)
#ifndef MICROPY_OPT_LOAD_ATTR_FAST_PATH
#define MICROPY_OPT_LOAD_ATTR_FAST_PATH (1)
#endif

@ -269,7 +269,6 @@ ifneq ($(FROZEN_MANIFEST)$(FROZEN_MPY_DIR),)
CFLAGS += -DMICROPY_QSTR_EXTRA_POOL=mp_qstr_frozen_const_pool
CFLAGS += -DMICROPY_MODULE_FROZEN_MPY
CFLAGS += -DMPZ_DIG_SIZE=16 # force 16 bits to work on both 32 and 64 bit archs
MPY_CROSS_FLAGS += -mcache-lookup-bc
endif
ifneq ($(FROZEN_MANIFEST)$(FROZEN_DIR),)
@ -285,9 +284,7 @@ endif
CXXFLAGS += $(filter-out -Wmissing-prototypes -Wold-style-definition -std=gnu99,$(CFLAGS) $(CXXFLAGS_MOD))
ifeq ($(MICROPY_FORCE_32BIT),1)
RUN_TESTS_MPY_CROSS_FLAGS = --mpy-cross-flags='-mcache-lookup-bc -march=x86'
else
RUN_TESTS_MPY_CROSS_FLAGS = --mpy-cross-flags='-mcache-lookup-bc'
RUN_TESTS_MPY_CROSS_FLAGS = --mpy-cross-flags='-march=x86'
endif
ifeq ($(CROSS_COMPILE),arm-linux-gnueabi-)

@ -79,9 +79,6 @@
#endif
#define MICROPY_STREAMS_POSIX_API (1)
#define MICROPY_OPT_COMPUTED_GOTO (1)
#ifndef MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (1)
#endif
#ifndef MICROPY_OPT_LOAD_ATTR_FAST_PATH
#define MICROPY_OPT_LOAD_ATTR_FAST_PATH (1)
#endif

@ -55,7 +55,6 @@
#define MICROPY_LONGINT_IMPL (MICROPY_LONGINT_IMPL_NONE)
#define MICROPY_STREAMS_NON_BLOCK (0)
#define MICROPY_OPT_COMPUTED_GOTO (0)
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0)
#define MICROPY_OPT_LOAD_ATTR_FAST_PATH (0)
#define MICROPY_OPT_MAP_LOOKUP_CACHE (0)
#define MICROPY_CAN_OVERRIDE_BUILTINS (0)

@ -60,7 +60,6 @@ SRC_QSTR_AUTO_DEPS +=
ifneq ($(FROZEN_MANIFEST),)
CFLAGS += -DMICROPY_QSTR_EXTRA_POOL=mp_qstr_frozen_const_pool -DMICROPY_MODULE_FROZEN_MPY=1 -DMPZ_DIG_SIZE=16
MPY_CROSS_FLAGS += -mcache-lookup-bc
endif
include $(TOP)/py/mkrules.mk

@ -59,7 +59,6 @@
#define MICROPY_STREAMS_NON_BLOCK (1)
#define MICROPY_STREAMS_POSIX_API (1)
#define MICROPY_OPT_COMPUTED_GOTO (0)
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (1)
#define MICROPY_MODULE_WEAK_LINKS (1)
#define MICROPY_CAN_OVERRIDE_BUILTINS (1)
#define MICROPY_VFS_POSIX_FILE (1)

@ -123,7 +123,7 @@ using(var outFile = System.IO.File.CreateText(OutputFile)) {
<PreprocessorDefinitions>MICROPY_MODULE_FROZEN_MPY=1;MICROPY_QSTR_EXTRA_POOL=mp_qstr_frozen_const_pool;%(PreprocessorDefinitions)</PreprocessorDefinitions>
</ClCompile>
</ItemGroup>
<Exec Command="$(PyPython) $(PyBaseDir)tools\makemanifest.py -v MPY_DIR=$(PyBaseDir) -v MPY_LIB_DIR=$(PyBaseDir)../micropython-lib -v PORT_DIR=$(PyWinDir) -f&quot;-mcache-lookup-bc&quot; -o $(PyBuildDir)frozen_content.c -b $(PyBuildDir) $(FrozenManifest)"/>
<Exec Command="$(PyPython) $(PyBaseDir)tools\makemanifest.py -v MPY_DIR=$(PyBaseDir) -v MPY_LIB_DIR=$(PyBaseDir)../micropython-lib -v PORT_DIR=$(PyWinDir) -o $(PyBuildDir)frozen_content.c -b $(PyBuildDir) $(FrozenManifest)"/>
<WriteLinesToFile File="$(TLogLocation)frozen.read.1.tlog" Lines="$(FrozenManifest)" Overwrite="True"/>
</Target>

@ -304,24 +304,10 @@ void mp_setup_code_state(mp_code_state_t *code_state, size_t n_args, size_t n_kw
// The following table encodes the number of bytes that a specific opcode
// takes up. Some opcodes have an extra byte, defined by MP_BC_MASK_EXTRA_BYTE.
// There are 4 special opcodes that have an extra byte only when
// MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE is enabled (and they take a qstr):
// MP_BC_LOAD_NAME
// MP_BC_LOAD_GLOBAL
// MP_BC_LOAD_ATTR
// MP_BC_STORE_ATTR
uint mp_opcode_format(const byte *ip, size_t *opcode_size, bool count_var_uint) {
uint f = MP_BC_FORMAT(*ip);
const byte *ip_start = ip;
if (f == MP_BC_FORMAT_QSTR) {
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE_DYNAMIC) {
if (*ip == MP_BC_LOAD_NAME
|| *ip == MP_BC_LOAD_GLOBAL
|| *ip == MP_BC_LOAD_ATTR
|| *ip == MP_BC_STORE_ATTR) {
ip += 1;
}
}
ip += 3;
} else {
int extra_byte = (*ip & MP_BC_MASK_EXTRA_BYTE) == 0;

@ -46,7 +46,6 @@ ifeq ($(ARCH),x86)
# x86
CROSS =
CFLAGS += -m32 -fno-stack-protector
MPY_CROSS_FLAGS += -mcache-lookup-bc
MICROPY_FLOAT_IMPL ?= double
else ifeq ($(ARCH),x64)
@ -54,7 +53,6 @@ else ifeq ($(ARCH),x64)
# x64
CROSS =
CFLAGS += -fno-stack-protector
MPY_CROSS_FLAGS += -mcache-lookup-bc
MICROPY_FLOAT_IMPL ?= double
else ifeq ($(ARCH),armv7m)

@ -560,9 +560,6 @@ void mp_emit_bc_load_global(emit_t *emit, qstr qst, int kind) {
MP_STATIC_ASSERT(MP_BC_LOAD_NAME + MP_EMIT_IDOP_GLOBAL_GLOBAL == MP_BC_LOAD_GLOBAL);
(void)qst;
emit_write_bytecode_byte_qstr(emit, 1, MP_BC_LOAD_NAME + kind, qst);
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE_DYNAMIC) {
emit_write_bytecode_raw_byte(emit, 0);
}
}
void mp_emit_bc_load_method(emit_t *emit, qstr qst, bool is_super) {
@ -596,9 +593,6 @@ void mp_emit_bc_attr(emit_t *emit, qstr qst, int kind) {
}
emit_write_bytecode_byte_qstr(emit, -2, MP_BC_STORE_ATTR, qst);
}
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE_DYNAMIC) {
emit_write_bytecode_raw_byte(emit, 0);
}
}
void mp_emit_bc_store_local(emit_t *emit, qstr qst, mp_uint_t local_num, int kind) {

@ -423,10 +423,8 @@
// Configure dynamic compiler macros
#if MICROPY_DYNAMIC_COMPILER
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE_DYNAMIC (mp_dynamic_compiler.opt_cache_map_lookup_in_bytecode)
#define MICROPY_PY_BUILTINS_STR_UNICODE_DYNAMIC (mp_dynamic_compiler.py_builtins_str_unicode)
#else
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE_DYNAMIC MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
#define MICROPY_PY_BUILTINS_STR_UNICODE_DYNAMIC MICROPY_PY_BUILTINS_STR_UNICODE
#endif
@ -520,13 +518,6 @@
#define MICROPY_OPT_COMPUTED_GOTO (0)
#endif
// Whether to cache result of map lookups in LOAD_NAME, LOAD_GLOBAL, LOAD_ATTR,
// STORE_ATTR bytecodes. Uses 1 byte extra RAM for each of these opcodes and
// uses a bit of extra code ROM, but greatly improves lookup speed.
#ifndef MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0)
#endif
// Optimise the fast path for loading attributes from instance types. Increases
// Thumb2 code size by about 48 bytes.
#ifndef MICROPY_OPT_LOAD_ATTR_FAST_PATH

@ -44,7 +44,6 @@
#if MICROPY_DYNAMIC_COMPILER
typedef struct mp_dynamic_compiler_t {
uint8_t small_int_bits; // must be <= host small_int_bits
bool opt_cache_map_lookup_in_bytecode;
bool py_builtins_str_unicode;
uint8_t native_arch;
uint8_t nlr_buf_num_regs;

@ -41,16 +41,15 @@
#define MPY_FEATURE_ENCODE_ARCH(arch) ((arch) << 2)
#define MPY_FEATURE_DECODE_ARCH(feat) ((feat) >> 2)
// The feature flag bits encode the compile-time config options that
// affect the generate bytecode.
// The feature flag bits encode the compile-time config options that affect
// the generate bytecode. Note: position 0 is now unused
// (formerly MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE).
#define MPY_FEATURE_FLAGS ( \
((MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) << 0) \
| ((MICROPY_PY_BUILTINS_STR_UNICODE) << 1) \
((MICROPY_PY_BUILTINS_STR_UNICODE) << 1) \
)
// This is a version of the flags that can be configured at runtime.
#define MPY_FEATURE_FLAGS_DYNAMIC ( \
((MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE_DYNAMIC) << 0) \
| ((MICROPY_PY_BUILTINS_STR_UNICODE_DYNAMIC) << 1) \
((MICROPY_PY_BUILTINS_STR_UNICODE_DYNAMIC) << 1) \
)
// Define the host architecture

@ -540,9 +540,6 @@ STATIC const byte *mp_prof_opcode_decode(const byte *ip, const mp_uint_t *const_
instruction->qstr_opname = MP_QSTR_LOAD_NAME;
instruction->arg = qst;
instruction->argobj = MP_OBJ_NEW_QSTR(qst);
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
instruction->argobjex_cache = MP_OBJ_NEW_SMALL_INT(*ip++);
}
break;
case MP_BC_LOAD_GLOBAL:
@ -550,9 +547,6 @@ STATIC const byte *mp_prof_opcode_decode(const byte *ip, const mp_uint_t *const_
instruction->qstr_opname = MP_QSTR_LOAD_GLOBAL;
instruction->arg = qst;
instruction->argobj = MP_OBJ_NEW_QSTR(qst);
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
instruction->argobjex_cache = MP_OBJ_NEW_SMALL_INT(*ip++);
}
break;
case MP_BC_LOAD_ATTR:
@ -560,9 +554,6 @@ STATIC const byte *mp_prof_opcode_decode(const byte *ip, const mp_uint_t *const_
instruction->qstr_opname = MP_QSTR_LOAD_ATTR;
instruction->arg = qst;
instruction->argobj = MP_OBJ_NEW_QSTR(qst);
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
instruction->argobjex_cache = MP_OBJ_NEW_SMALL_INT(*ip++);
}
break;
case MP_BC_LOAD_METHOD:
@ -618,9 +609,6 @@ STATIC const byte *mp_prof_opcode_decode(const byte *ip, const mp_uint_t *const_
instruction->qstr_opname = MP_QSTR_STORE_ATTR;
instruction->arg = qst;
instruction->argobj = MP_OBJ_NEW_QSTR(qst);
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
instruction->argobjex_cache = MP_OBJ_NEW_SMALL_INT(*ip++);
}
break;
case MP_BC_STORE_SUBSCR:

@ -208,25 +208,16 @@ const byte *mp_bytecode_print_str(const mp_print_t *print, const byte *ip) {
case MP_BC_LOAD_NAME:
DECODE_QSTR;
mp_printf(print, "LOAD_NAME %s", qstr_str(qst));
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
mp_printf(print, " (cache=%u)", *ip++);
}
break;
case MP_BC_LOAD_GLOBAL:
DECODE_QSTR;
mp_printf(print, "LOAD_GLOBAL %s", qstr_str(qst));
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
mp_printf(print, " (cache=%u)", *ip++);
}
break;
case MP_BC_LOAD_ATTR:
DECODE_QSTR;
mp_printf(print, "LOAD_ATTR %s", qstr_str(qst));
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
mp_printf(print, " (cache=%u)", *ip++);
}
break;
case MP_BC_LOAD_METHOD:
@ -270,9 +261,6 @@ const byte *mp_bytecode_print_str(const mp_print_t *print, const byte *ip) {
case MP_BC_STORE_ATTR:
DECODE_QSTR;
mp_printf(print, "STORE_ATTR %s", qstr_str(qst));
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
mp_printf(print, " (cache=%u)", *ip++);
}
break;
case MP_BC_STORE_SUBSCR:

@ -180,23 +180,6 @@
#define TRACE_TICK(current_ip, current_sp, is_exception)
#endif // MICROPY_PY_SYS_SETTRACE
#if MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
static inline mp_map_elem_t *mp_map_cached_lookup(mp_map_t *map, qstr qst, uint8_t *idx_cache) {
size_t idx = *idx_cache;
mp_obj_t key = MP_OBJ_NEW_QSTR(qst);
mp_map_elem_t *elem = NULL;
if (idx < map->alloc && map->table[idx].key == key) {
elem = &map->table[idx];
} else {
elem = mp_map_lookup(map, key, MP_MAP_LOOKUP);
if (elem != NULL) {
*idx_cache = (elem - &map->table[0]) & 0xff;
}
}
return elem;
}
#endif
// fastn has items in reverse order (fastn[0] is local[0], fastn[-1] is local[1], etc)
// sp points to bottom of stack which grows up
// returns:
@ -361,55 +344,20 @@ dispatch_loop:
goto load_check;
}
#if !MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
ENTRY(MP_BC_LOAD_NAME): {
MARK_EXC_IP_SELECTIVE();
DECODE_QSTR;
PUSH(mp_load_name(qst));
DISPATCH();
}
#else
ENTRY(MP_BC_LOAD_NAME): {
MARK_EXC_IP_SELECTIVE();
DECODE_QSTR;
mp_map_elem_t *elem = mp_map_cached_lookup(&mp_locals_get()->map, qst, (uint8_t*)ip);
mp_obj_t obj;
if (elem != NULL) {
obj = elem->value;
} else {
obj = mp_load_name(qst);
}
PUSH(obj);
ip++;
DISPATCH();
}
#endif
#if !MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
ENTRY(MP_BC_LOAD_GLOBAL): {
MARK_EXC_IP_SELECTIVE();
DECODE_QSTR;
PUSH(mp_load_global(qst));
DISPATCH();
}
#else
ENTRY(MP_BC_LOAD_GLOBAL): {
MARK_EXC_IP_SELECTIVE();
DECODE_QSTR;
mp_map_elem_t *elem = mp_map_cached_lookup(&mp_globals_get()->map, qst, (uint8_t*)ip);
mp_obj_t obj;
if (elem != NULL) {
obj = elem->value;
} else {
obj = mp_load_global(qst);
}
PUSH(obj);
ip++;
DISPATCH();
}
#endif
#if !MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
ENTRY(MP_BC_LOAD_ATTR): {
FRAME_UPDATE();
MARK_EXC_IP_SELECTIVE();
@ -436,28 +384,6 @@ dispatch_loop:
SET_TOP(obj);
DISPATCH();
}
#else
ENTRY(MP_BC_LOAD_ATTR): {
FRAME_UPDATE();
MARK_EXC_IP_SELECTIVE();
DECODE_QSTR;
mp_obj_t top = TOP();
mp_map_elem_t *elem = NULL;
if (mp_obj_is_instance_type(mp_obj_get_type(top))) {
mp_obj_instance_t *self = MP_OBJ_TO_PTR(top);
elem = mp_map_cached_lookup(&self->members, qst, (uint8_t*)ip);
}
mp_obj_t obj;
if (elem != NULL) {
obj = elem->value;
} else {
obj = mp_load_attr(top, qst);
}
SET_TOP(obj);
ip++;
DISPATCH();
}
#endif
ENTRY(MP_BC_LOAD_METHOD): {
MARK_EXC_IP_SELECTIVE();
@ -513,7 +439,6 @@ dispatch_loop:
DISPATCH();
}
#if !MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
ENTRY(MP_BC_STORE_ATTR): {
FRAME_UPDATE();
MARK_EXC_IP_SELECTIVE();
@ -522,32 +447,6 @@ dispatch_loop:
sp -= 2;
DISPATCH();
}
#else
// This caching code works with MICROPY_PY_BUILTINS_PROPERTY and/or
// MICROPY_PY_DESCRIPTORS enabled because if the attr exists in
// self->members then it can't be a property or have descriptors. A
// consequence of this is that we can't use MP_MAP_LOOKUP_ADD_IF_NOT_FOUND
// in the fast-path below, because that store could override a property.
ENTRY(MP_BC_STORE_ATTR): {
FRAME_UPDATE();
MARK_EXC_IP_SELECTIVE();
DECODE_QSTR;
mp_map_elem_t *elem = NULL;
mp_obj_t top = TOP();
if (mp_obj_is_instance_type(mp_obj_get_type(top)) && sp[-1] != MP_OBJ_NULL) {
mp_obj_instance_t *self = MP_OBJ_TO_PTR(top);
elem = mp_map_cached_lookup(&self->members, qst, (uint8_t*)ip);
}
if (elem != NULL) {
elem->value = sp[-1];
} else {
mp_store_attr(sp[0], qst, sp[-1]);
}
sp -= 2;
ip++;
DISPATCH();
}
#endif
ENTRY(MP_BC_STORE_SUBSCR):
MARK_EXC_IP_SELECTIVE();

@ -78,11 +78,11 @@ arg names:
45 STORE_NAME g
48 LOAD_CONST_OBJ \.\+
50 LOAD_METHOD format
53 LOAD_NAME b (cache=0)
57 CALL_METHOD n=1 nkw=0
59 STORE_NAME h
62 LOAD_CONST_NONE
63 RETURN_VALUE
53 LOAD_NAME b
56 CALL_METHOD n=1 nkw=0
58 STORE_NAME h
61 LOAD_CONST_NONE
62 RETURN_VALUE
mem: total=\\d\+, current=\\d\+, peak=\\d\+
stack: \\d\+ out of \\d\+
GC: total: \\d\+, used: \\d\+, free: \\d\+

@ -119,11 +119,11 @@ Raw bytecode (code_info_size=\\d\+, bytecode_size=\\d\+):
\\d\+ UNARY_OP 3
\\d\+ STORE_FAST 10
\\d\+ LOAD_DEREF 14
\\d\+ LOAD_ATTR c (cache=0)
\\d\+ LOAD_ATTR c
\\d\+ STORE_FAST 11
\\d\+ LOAD_FAST 11
\\d\+ LOAD_DEREF 14
\\d\+ STORE_ATTR c (cache=0)
\\d\+ STORE_ATTR c
\\d\+ LOAD_DEREF 14
\\d\+ LOAD_CONST_SMALL_INT 0
\\d\+ LOAD_SUBSCR
@ -233,7 +233,7 @@ Raw bytecode (code_info_size=\\d\+, bytecode_size=\\d\+):
\\d\+ LOAD_DEREF 16
\\d\+ POP_TOP
\\d\+ JUMP \\d\+
\\d\+ LOAD_GLOBAL y (cache=0)
\\d\+ LOAD_GLOBAL y
\\d\+ POP_TOP
\\d\+ JUMP \\d\+
\\d\+ LOAD_DEREF 14
@ -418,13 +418,13 @@ arg names:
(N_EXC_STACK 0)
bc=0 line=1
########
bc=13 line=150
00 LOAD_NAME __name__ (cache=0)
04 STORE_NAME __module__
07 LOAD_CONST_STRING 'Class'
10 STORE_NAME __qualname__
13 LOAD_CONST_NONE
14 RETURN_VALUE
bc=12 line=150
00 LOAD_NAME __name__
03 STORE_NAME __module__
06 LOAD_CONST_STRING 'Class'
09 STORE_NAME __qualname__
12 LOAD_CONST_NONE
13 RETURN_VALUE
File cmdline/cmd_showbc.py, code block 'f' (descriptor: \.\+, bytecode @\.\+ bytes)
Raw bytecode (code_info_size=\\d\+, bytecode_size=\\d\+):
########
@ -434,8 +434,8 @@ arg names: self
(N_EXC_STACK 0)
bc=0 line=1
bc=0 line=157
00 LOAD_GLOBAL super (cache=0)
\\d\+ LOAD_GLOBAL __class__ (cache=0)
00 LOAD_GLOBAL super
\\d\+ LOAD_GLOBAL __class__
\\d\+ LOAD_FAST 0
\\d\+ LOAD_SUPER_METHOD f
\\d\+ CALL_METHOD n=0 nkw=0

@ -8,12 +8,12 @@ arg names:
(N_EXC_STACK 0)
bc=0 line=1
bc=0 line=3
00 LOAD_NAME print (cache=0)
04 LOAD_CONST_SMALL_INT 1
05 CALL_FUNCTION n=1 nkw=0
07 POP_TOP
08 LOAD_CONST_NONE
09 RETURN_VALUE
00 LOAD_NAME print
03 LOAD_CONST_SMALL_INT 1
04 CALL_FUNCTION n=1 nkw=0
06 POP_TOP
07 LOAD_CONST_NONE
08 RETURN_VALUE
1
mem: total=\\d\+, current=\\d\+, peak=\\d\+
stack: \\d\+ out of \\d\+

@ -48,8 +48,8 @@ class UserFS:
# Pre-compiled examples/natmod/features0 example for various architectures, keyed
# by the required value of sys.implementation.mpy.
features0_file_contents = {
# -march=x64 -mcache-lookup-bc
0xB05: b'M\x05\x0b\x1f \x84b\xe9/\x00\x00\x00SH\x8b\x1ds\x00\x00\x00\xbe\x02\x00\x00\x00\xffS\x18\xbf\x01\x00\x00\x00H\x85\xc0u\x0cH\x8bC \xbe\x02\x00\x00\x00[\xff\xe0H\x0f\xaf\xf8H\xff\xc8\xeb\xe6ATUSH\x8b\x1dA\x00\x00\x00H\x8b\x7f\x08L\x8bc(A\xff\xd4H\x8d5\x1f\x00\x00\x00H\x89\xc5H\x8b\x05-\x00\x00\x00\x0f\xb78\xffShH\x89\xefA\xff\xd4H\x8b\x03[]A\\\xc3\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x90\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x84@\x12factorial\x10\x00\x00\r \x01"\x9f\x1c\x01\x1e\xff',
# -march=x64
0xA05: b'M\x05\x0a\x1f \x84b\xe9/\x00\x00\x00SH\x8b\x1ds\x00\x00\x00\xbe\x02\x00\x00\x00\xffS\x18\xbf\x01\x00\x00\x00H\x85\xc0u\x0cH\x8bC \xbe\x02\x00\x00\x00[\xff\xe0H\x0f\xaf\xf8H\xff\xc8\xeb\xe6ATUSH\x8b\x1dA\x00\x00\x00H\x8b\x7f\x08L\x8bc(A\xff\xd4H\x8d5\x1f\x00\x00\x00H\x89\xc5H\x8b\x05-\x00\x00\x00\x0f\xb78\xffShH\x89\xefA\xff\xd4H\x8b\x03[]A\\\xc3\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x90\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x84@\x12factorial\x10\x00\x00\r \x01"\x9f\x1c\x01\x1e\xff',
# -march=armv7m
0x1605: b"M\x05\x16\x1f \x84\x12\x1a\xe0\x00\x00\x13\xb5\nK\nJ{D\x9cX\x02!\xe3h\x98G\x03F\x01 3\xb9\x02!#i\x01\x93\x02\xb0\xbd\xe8\x10@\x18GXC\x01;\xf4\xe7\x00\xbfj\x00\x00\x00\x00\x00\x00\x00\xf8\xb5\tN\tK~D\xf4X@hgi\xb8G\x05F\x07K\x07I\xf2XyD\x10\x88ck\x98G(F\xb8G h\xf8\xbd6\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x1c\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x80\x00\x00\x00\x00\x00\x00\x00\x01\x84\x00\x12factorial\x10\x00\x00\r<\x01>\x9f8\x01:\xff",
}

@ -52,11 +52,11 @@ class UserFS:
# fmt: off
user_files = {
# bad architecture
'/mod0.mpy': b'M\x05\xff\x00\x10',
'/mod0.mpy': b'M\x05\xfe\x00\x10',
# test loading of viper and asm
'/mod1.mpy': (
b'M\x05\x0b\x1f\x20' # header
b'M\x05\x0a\x1f\x20' # header
b'\x20' # n bytes, bytecode
b'\x00\x08\x02m\x02m' # prelude
@ -78,7 +78,7 @@ user_files = {
# test loading viper with additional scope flags and relocation
'/mod2.mpy': (
b'M\x05\x0b\x1f\x20' # header
b'M\x05\x0a\x1f\x20' # header
b'\x20' # n bytes, bytecode
b'\x00\x08\x02m\x02m' # prelude

@ -742,9 +742,7 @@ the last matching regex is used:
cmd_parser.add_argument(
"--via-mpy", action="store_true", help="compile .py files to .mpy first"
)
cmd_parser.add_argument(
"--mpy-cross-flags", default="-mcache-lookup-bc", help="flags to pass to mpy-cross"
)
cmd_parser.add_argument("--mpy-cross-flags", default="", help="flags to pass to mpy-cross")
cmd_parser.add_argument(
"--keep-path", action="store_true", help="do not clear MICROPYPATH when running tests"
)

@ -132,14 +132,6 @@ def mp_opcode_format(bytecode, ip, count_var_uint):
ip_start = ip
f = (0x000003A4 >> (2 * ((opcode) >> 4))) & 3
if f == MP_BC_FORMAT_QSTR:
if config.MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE:
if (
opcode == MP_BC_LOAD_NAME
or opcode == MP_BC_LOAD_GLOBAL
or opcode == MP_BC_LOAD_ATTR
or opcode == MP_BC_STORE_ATTR
):
ip += 1
ip += 3
else:
extra_byte = (opcode & MP_BC_MASK_EXTRA_BYTE) == 0
@ -440,10 +432,7 @@ class RawCodeBytecode(RawCode):
"// frozen bytecode for file %s, scope %s%s"
% (self.source_file.str, parent_name, self.simple_name.str)
)
print("STATIC ", end="")
if not config.MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE:
print("const ", end="")
print("byte fun_data_%s[%u] = {" % (self.escaped_name, len(self.bytecode)))
print("STATIC const byte fun_data_%s[%u] = {" % (self.escaped_name, len(self.bytecode)))
print(" ", end="")
for i in range(self.ip2):
print(" 0x%02x," % self.bytecode[i], end="")
@ -798,7 +787,6 @@ def read_mpy(filename):
raise Exception("incompatible .mpy version")
feature_byte = header[2]
qw_size = read_uint(f)
config.MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE = (feature_byte & 1) != 0
config.MICROPY_PY_BUILTINS_STR_UNICODE = (feature_byte & 2) != 0
mpy_native_arch = feature_byte >> 2
if mpy_native_arch != MP_NATIVE_ARCH_NONE:
@ -836,14 +824,6 @@ def freeze_mpy(base_qstrs, raw_codes):
print('#include "py/nativeglue.h"')
print()
print(
"#if MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE != %u"
% config.MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
)
print('#error "incompatible MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE"')
print("#endif")
print()
print("#if MICROPY_LONGINT_IMPL != %u" % config.MICROPY_LONGINT_IMPL)
print('#error "incompatible MICROPY_LONGINT_IMPL"')
print("#endif")
@ -940,11 +920,7 @@ def merge_mpy(raw_codes, output_file):
header = bytearray(5)
header[0] = ord("M")
header[1] = config.MPY_VERSION
header[2] = (
config.native_arch << 2
| config.MICROPY_PY_BUILTINS_STR_UNICODE << 1
| config.MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
)
header[2] = config.native_arch << 2 | config.MICROPY_PY_BUILTINS_STR_UNICODE << 1
header[3] = config.mp_small_int_bits
header[4] = 32 # qstr_win_size
merged_mpy.extend(header)

@ -6,14 +6,11 @@ import os.path
argparser = argparse.ArgumentParser(description="Compile all .py files to .mpy recursively")
argparser.add_argument("-o", "--out", help="output directory (default: input dir)")
argparser.add_argument("--target", help="select MicroPython target config")
argparser.add_argument(
"-mcache-lookup-bc", action="store_true", help="cache map lookups in the bytecode"
)
argparser.add_argument("dir", help="input directory")
args = argparser.parse_args()
TARGET_OPTS = {
"unix": "-mcache-lookup-bc",
"unix": "",
"baremetal": "",
}

@ -48,7 +48,6 @@ MP_CODE_NATIVE_VIPER = 4
MP_SCOPE_FLAG_VIPERRELOC = 0x10
MP_SCOPE_FLAG_VIPERRODATA = 0x20
MP_SCOPE_FLAG_VIPERBSS = 0x40
MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE = 1
MICROPY_PY_BUILTINS_STR_UNICODE = 2
MP_SMALL_INT_BITS = 31
QSTR_WINDOW_SIZE = 32
@ -118,9 +117,7 @@ class ArchData:
ARCH_DATA = {
"x86": ArchData(
"EM_386",
MP_NATIVE_ARCH_X86 << 2
| MICROPY_PY_BUILTINS_STR_UNICODE
| MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE,
MP_NATIVE_ARCH_X86 << 2 | MICROPY_PY_BUILTINS_STR_UNICODE,
2,
4,
(R_386_PC32, R_386_GOT32, R_386_GOT32X),
@ -128,9 +125,7 @@ ARCH_DATA = {
),
"x64": ArchData(
"EM_X86_64",
MP_NATIVE_ARCH_X64 << 2
| MICROPY_PY_BUILTINS_STR_UNICODE
| MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE,
MP_NATIVE_ARCH_X64 << 2 | MICROPY_PY_BUILTINS_STR_UNICODE,
2,
8,
(R_X86_64_GOTPCREL, R_X86_64_REX_GOTPCRELX),

Loading…
Cancel
Save