micropython/py
Damien George 1fb01bd6c5 py/emitnative: Put a pointer to the native prelude in child_table array.
Some architectures (like esp32 xtensa) cannot read byte-wise from
executable memory.  This means the prelude for native functions -- which is
usually located after the machine code for the native function -- must be
placed in separate memory that can be read byte-wise.  Prior to this commit
this was achieved by enabling N_PRELUDE_AS_BYTES_OBJ for the emitter and
MICROPY_EMIT_NATIVE_PRELUDE_AS_BYTES_OBJ for the runtime.  The prelude was
then placed in a bytes object, pointed to by the module's constant table.

This behaviour is changed by this commit so that a pointer to the prelude
is stored either in mp_obj_fun_bc_t.child_table, or in
mp_obj_fun_bc_t.child_table[num_children] if num_children > 0.  The reasons
for doing this are:

1. It decouples the native emitter from runtime requirements, the emitted
   code no longer needs to know if the system it runs on can/can't read
   byte-wise from executable memory.

2. It makes all ports have the same emitter behaviour, there is no longer
   the N_PRELUDE_AS_BYTES_OBJ option.

3. The module's constant table is now used only for actual constants in the
   Python code.  This allows further optimisations to be done with the
   constants (eg constant deduplication).

Code size change for those ports that enable the native emitter:
   unix x64:   +80 +0.015%
      stm32:   +24 +0.004% PYBV10
    esp8266:   +88 +0.013% GENERIC
      esp32:   -20 -0.002% GENERIC[incl -112(data)]
        rp2:   +32 +0.005% PICO

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 16:44:49 +10:00
..
argcheck.c py: Add option to compile without any error messages at all. 2021-04-27 23:51:52 +10:00
asmarm.c py/emitglue: Always flush caches when assigning native ARM code. 2021-06-05 11:03:04 +10:00
asmarm.h py/emitglue: Always flush caches when assigning native ARM code. 2021-06-05 11:03:04 +10:00
asmbase.c py: Rework bytecode and .mpy file format to be mostly static data. 2022-02-24 18:08:43 +11:00
asmbase.h py: Rework bytecode and .mpy file format to be mostly static data. 2022-02-24 18:08:43 +11:00
asmthumb.c py/emitglue: Always flush caches when assigning native ARM code. 2021-06-05 11:03:04 +10:00
asmthumb.h py/emitglue: Always flush caches when assigning native ARM code. 2021-06-05 11:03:04 +10:00
asmx64.c py/asmx64: Support full range of regs in asm_x64_lea_disp_to_r64. 2022-05-17 14:25:51 +10:00
asmx64.h py/asm: Add condition codes for signed comparisons. 2020-06-27 00:24:04 +10:00
asmx86.c py/asm: Add funcs/macros to emit machine code for logical-shift-right. 2020-06-27 00:24:04 +10:00
asmx86.h py/asm: Add condition codes for signed comparisons. 2020-06-27 00:24:04 +10:00
asmxtensa.c py/asmxtensa: Fix use of l32i/s32i when offset won't fit in encoding. 2022-03-30 16:32:17 +11:00
asmxtensa.h py/asmxtensa: Fix use of l32i/s32i when offset won't fit in encoding. 2022-03-30 16:32:17 +11:00
bc0.h py: Change jump-if-x-or-pop opcodes to have unsigned offset argument. 2022-03-28 15:43:09 +11:00
bc.c py/bc: Provide separate code-state setup funcs for bytecode and native. 2022-05-17 14:25:51 +10:00
bc.h py/bc: Provide separate code-state setup funcs for bytecode and native. 2022-05-17 14:25:51 +10:00
binary.c all: Fix signed shifts and NULL access errors from -fsanitize=undefined. 2021-06-24 23:01:04 +10:00
binary.h py/binary: Change mp_uint_t to size_t for index, size, align args. 2019-09-02 13:14:27 +10:00
builtin.h extmod/moduos: Create general uos module to be used by all ports. 2022-03-09 10:03:23 +11:00
builtinevex.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
builtinhelp.c py: Only search frozen modules when '.frozen' is found in sys.path. 2021-12-18 00:01:59 +11:00
builtinimport.c py/builtinimport: Alias sys to usys if import weak links aren't enabled. 2022-03-28 16:09:58 +11:00
compile.c py/emitnative: Put a pointer to the native prelude in child_table array. 2022-05-17 16:44:49 +10:00
compile.h py: Rework bytecode and .mpy file format to be mostly static data. 2022-02-24 18:08:43 +11:00
dynruntime.h py/obj: Introduce mp_obj_malloc macro to allocate, and set object type. 2022-05-03 22:23:46 +10:00
dynruntime.mk all: Remove MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE. 2021-09-16 16:04:03 +10:00
emit.h py: Change jump opcodes to emit 1-byte jump offset when possible. 2022-03-28 15:41:38 +11:00
emitbc.c py/emitbc: Assert that a small int fits its encoding when emitting one. 2022-03-31 23:59:10 +11:00
emitcommon.c py/emitcommon: Don't implicitly close class vars that are assigned to. 2022-05-03 16:38:43 +10:00
emitglue.c py: Rework bytecode and .mpy file format to be mostly static data. 2022-02-24 18:08:43 +11:00
emitglue.h py: Rework bytecode and .mpy file format to be mostly static data. 2022-02-24 18:08:43 +11:00
emitinlinethumb.c py/emitinlinethumb: Use 16 bit encodings for PUSH LR and POP PC. 2022-04-11 15:35:39 +10:00
emitinlinextensa.c py: Use unsigned comparison of chars. 2020-10-22 11:47:36 +02:00
emitnarm.c py/emitnative: Add support for return/break/continue in try and with. 2018-09-04 14:31:28 +10:00
emitnative.c py/emitnative: Put a pointer to the native prelude in child_table array. 2022-05-17 16:44:49 +10:00
emitnthumb.c py/emitnative: Add support for return/break/continue in try and with. 2018-09-04 14:31:28 +10:00
emitnx64.c py/emitnative: Add support for return/break/continue in try and with. 2018-09-04 14:31:28 +10:00
emitnx86.c py: Rework bytecode and .mpy file format to be mostly static data. 2022-02-24 18:08:43 +11:00
emitnxtensa.c py/emitnative: Add support for return/break/continue in try and with. 2018-09-04 14:31:28 +10:00
emitnxtensawin.c py/emitnative: Put a pointer to the native prelude in child_table array. 2022-05-17 16:44:49 +10:00
formatfloat.c all: Fix implicit conversion from double to float. 2020-04-18 22:42:24 +10:00
formatfloat.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
frozenmod.c py: Rework bytecode and .mpy file format to be mostly static data. 2022-02-24 18:08:43 +11:00
frozenmod.h py: Only search frozen modules when '.frozen' is found in sys.path. 2021-12-18 00:01:59 +11:00
gc.c py/gc: Update debug code to compile with changes to qstr pool types. 2022-02-17 11:17:21 +11:00
gc.h py/gc: Change include of stdint.h to stddef.h. 2021-02-05 15:46:56 +11:00
grammar.h py/compile: Implement PEP 526, syntax for variable annotations. 2020-06-16 23:18:01 +10:00
lexer.c py/persistentcode: Remove unicode feature flag from .mpy file. 2022-05-17 12:51:54 +10:00
lexer.h py: Implement partial PEP-498 (f-string) support. 2021-08-14 16:58:40 +10:00
makecompresseddata.py py/makecompresseddata.py: Make compression deterministic. 2020-04-20 10:32:49 +10:00
makemoduledefs.py all: Update Python code to conform to latest black formatting. 2020-08-29 15:18:01 +10:00
makeqstrdata.py py/qstr: Separate hash and len from string data. 2022-02-11 22:52:32 +11:00
makeqstrdefs.py py/makeqstrdefs: Cleanup and extend source file classification. 2022-04-01 15:03:21 +11:00
makeversionhdr.py py/makeversionhdr: Add --tags arg to git describe. 2021-07-05 10:41:31 -05:00
malloc.c py/malloc: Introduce m_tracked_calloc, m_tracked_free functions. 2022-05-05 10:31:50 +10:00
map.c py: Add wrapper macros so hot VM functions can go in fast code location. 2021-10-15 23:31:19 +11:00
misc.h py/malloc: Introduce m_tracked_calloc, m_tracked_free functions. 2022-05-05 10:31:50 +10:00
mkenv.mk py/mkrules: Automatically build mpy-cross if it doesn't exist. 2021-08-07 20:25:32 +10:00
mkrules.cmake py/mkrules.cmake: Set frozen preprocessor defs early. 2021-12-18 00:05:18 +11:00
mkrules.mk all: Remove support for FROZEN_DIR and FROZEN_MPY_DIR. 2021-12-17 23:54:05 +11:00
modarray.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
modbuiltins.c py/modbuiltins: Add additional macro for extending builtins. 2022-01-07 11:36:52 +11:00
modcmath.c all: Format code to add space after C++-style comment start. 2020-04-23 11:24:25 +10:00
modcollections.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
modgc.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
modio.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
modmath.c py/modmath: Add math.tau, math.nan and math.inf constants. 2022-01-23 09:28:33 +11:00
modmicropython.c py/gc: Make gc_lock_depth have a count per thread. 2021-05-10 13:07:16 +10:00
modstruct.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
modsys.c py/modsys: Introduce sys.implementation._machine constant. 2022-04-28 17:23:03 +10:00
modthread.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
moduerrno.c all: Format code to add space after C++-style comment start. 2020-04-23 11:24:25 +10:00
mpconfig.h py/emitnative: Put a pointer to the native prelude in child_table array. 2022-05-17 16:44:49 +10:00
mperrno.h py/mperrno: Add MP_ECANCELED error code. 2021-06-24 23:14:01 +10:00
mphal.h all: Rename absolute time-based functions to include "epoch". 2020-09-18 17:20:34 +10:00
mpprint.c py/mpprint: Fix length calculation for strings with precision-modifier. 2020-12-07 23:32:06 +11:00
mpprint.h extmod/modujson: Add support for dump/dumps separators keyword-argument. 2021-08-07 13:52:16 +10:00
mpstate.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
mpstate.h py/persistentcode: Remove unicode feature flag from .mpy file. 2022-05-17 12:51:54 +10:00
mpthread.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
mpz.c py/mpz: Fix bugs with bitwise of -0 by ensuring all 0's are positive. 2021-12-21 18:00:05 +11:00
mpz.h py/mpz: Fix bugs with bitwise of -0 by ensuring all 0's are positive. 2021-12-21 18:00:05 +11:00
nativeglue.c py/bc: Provide separate code-state setup funcs for bytecode and native. 2022-05-17 14:25:51 +10:00
nativeglue.h py/bc: Provide separate code-state setup funcs for bytecode and native. 2022-05-17 14:25:51 +10:00
nlr.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
nlr.h all: Reformat remaining C code that doesn't have a space after a comma. 2022-05-05 13:30:40 +10:00
nlraarch64.c py/nlraarch64: Add underscore prefix to function symbols for Darwin ABI. 2021-05-18 11:46:30 +10:00
nlrpowerpc.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
nlrsetjmp.c py/nlr: Factor out common NLR code to macro and generic funcs in nlr.c. 2017-12-28 16:46:30 +11:00
nlrthumb.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
nlrx64.c py/nlrx64: Correct the detection of Darwin ABI. 2021-05-18 11:52:00 +10:00
nlrx86.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
nlrxtensa.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
obj.c py/obj: Introduce mp_obj_malloc macro to allocate, and set object type. 2022-05-03 22:23:46 +10:00
obj.h py/obj: Introduce mp_obj_malloc macro to allocate, and set object type. 2022-05-03 22:23:46 +10:00
objarray.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objarray.h py/objarray.h: Add mp_obj_memoryview_init() helper function. 2020-09-25 12:23:11 +10:00
objattrtuple.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objbool.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objboundmeth.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objcell.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objclosure.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objcomplex.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objdeque.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objdict.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objenumerate.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objexcept.c py/modsys: Add optional sys.tracebacklimit attribute. 2022-03-10 10:43:21 +11:00
objexcept.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objfilter.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objfloat.c py/objfloat: Explain why mp_obj_malloc isn't used. 2022-05-03 22:25:40 +10:00
objfun.c py/bc: Provide separate code-state setup funcs for bytecode and native. 2022-05-17 14:25:51 +10:00
objfun.h py: Rework bytecode and .mpy file format to be mostly static data. 2022-02-24 18:08:43 +11:00
objgenerator.c py/emitnative: Put a pointer to the native prelude in child_table array. 2022-05-17 16:44:49 +10:00
objgenerator.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objgetitemiter.c py: Support single argument to optimised MP_OBJ_STOP_ITERATION. 2021-07-15 00:12:41 +10:00
objint_longlong.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objint_mpz.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objint.c all: Fix signed shifts and NULL access errors from -fsanitize=undefined. 2021-06-24 23:01:04 +10:00
objint.h py/modsys: Use consistent naming pattern for module-level const objects. 2020-05-28 10:02:14 +10:00
objlist.c extmod/modujson: Add support for dump/dumps separators keyword-argument. 2021-08-07 13:52:16 +10:00
objlist.h py/obj.h: Move declaration of mp_obj_list_init to objlist.h. 2018-03-13 14:03:15 +11:00
objmap.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objmodule.c py/objmodule: Move stray #include to top of file. 2022-05-05 11:02:38 +10:00
objmodule.h py/objmodule: Support delegating failed attr lookups. 2022-03-10 10:35:44 +11:00
objnamedtuple.c py: Add option to compile without any error messages at all. 2021-04-27 23:51:52 +10:00
objnamedtuple.h py/objnamedtuple: Allow to reuse namedtuple basic functionality. 2017-11-20 09:30:06 +02:00
objnone.c py/obj: Add MICROPY_OBJ_IMMEDIATE_OBJS option to reduce code size. 2020-01-13 01:01:45 +11:00
objobject.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objpolyiter.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objproperty.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objrange.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objreversed.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objset.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objsingleton.c py/objsingleton: Use mp_generic_unary_op for singleton objects. 2019-12-27 12:53:36 +11:00
objslice.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objstr.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objstr.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objstringio.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objstringio.h py/objstringio: If created from immutable object, follow copy on write policy. 2017-06-09 17:33:01 +03:00
objstrunicode.c py/mpprint: Fix length calculation for strings with precision-modifier. 2020-12-07 23:32:06 +11:00
objtuple.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objtuple.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objtype.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
objtype.h py/runtime: Don't allocate iter buf for user-defined types. 2019-12-27 12:34:22 +11:00
objzip.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
opmethods.c py: Make mp_obj_get_type() return a const ptr to mp_obj_type_t. 2020-01-09 11:25:26 +11:00
pairheap.c py/pairheap: Properly unlink node on pop and delete. 2020-03-26 01:21:04 +11:00
pairheap.h py/pairheap: Add helper function to initialise a new node. 2020-03-26 01:21:04 +11:00
parse.c py/parse: Add MICROPY_COMP_CONST_TUPLE option to build const tuples. 2022-04-14 23:52:12 +10:00
parse.h py/parse: Factor obj extract code to mp_parse_node_extract_const_object. 2022-04-14 22:44:56 +10:00
parsenum.c py: Add option to compile without any error messages at all. 2021-04-27 23:51:52 +10:00
parsenum.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
parsenumbase.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
parsenumbase.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
persistentcode.c py/emitnative: Put a pointer to the native prelude in child_table array. 2022-05-17 16:44:49 +10:00
persistentcode.h py/persistentcode: Remove unicode feature flag from .mpy file. 2022-05-17 12:51:54 +10:00
profile.c py: Rework bytecode and .mpy file format to be mostly static data. 2022-02-24 18:08:43 +11:00
profile.h py: Rework bytecode and .mpy file format to be mostly static data. 2022-02-24 18:08:43 +11:00
py.cmake py/py.cmake: Introduce MICROPY_INC_CORE as a list with core includes. 2021-04-09 13:08:35 +10:00
py.mk samd/moduos: Convert module to use extmod version. 2022-03-09 10:03:23 +11:00
pystack.c py: Introduce and use mp_raise_type_arg helper. 2021-07-15 00:12:41 +10:00
pystack.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
qstr.c py/qstr: Use const consistently to avoid a cast. 2022-02-11 22:55:02 +11:00
qstr.h py: Rework bytecode and .mpy file format to be mostly static data. 2022-02-24 18:08:43 +11:00
qstrdefs.h py/modsys: Add optional mutable attributes sys.ps1/ps2 and use them. 2022-03-10 10:58:33 +11:00
reader.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
reader.h py: Allow lexer to raise exceptions during construction. 2017-03-14 11:52:05 +11:00
repl.c shared/readline: Make tab insert an indent when it follows whitespace. 2022-04-22 17:00:16 +10:00
repl.h py/modsys: Add optional mutable attributes sys.ps1/ps2 and use them. 2022-03-10 10:58:33 +11:00
ringbuf.c py/ringbuf: Add peek16 method. 2019-11-21 12:04:53 +11:00
ringbuf.h extmod/modbluetooth: Make modbluetooth event not a bitfield. 2020-06-05 14:04:20 +10:00
runtime0.h py/nativeglue: Add new header file with native function table typedef. 2019-12-12 20:15:28 +11:00
runtime_utils.c py: mp_call_function_*_protected(): Pass-thru return value if possible. 2017-12-05 00:38:41 +02:00
runtime.c all: Use mp_obj_malloc everywhere it's applicable. 2022-05-03 22:28:14 +10:00
runtime.h py/scheduler: Add support for scheduling static C-based callbacks. 2022-04-14 12:31:53 +10:00
scheduler.c py/scheduler: Add support for scheduling static C-based callbacks. 2022-04-14 12:31:53 +10:00
scope.c py: Rework bytecode and .mpy file format to be mostly static data. 2022-02-24 18:08:43 +11:00
scope.h py/emitcommon: Don't implicitly close class vars that are assigned to. 2022-05-03 16:38:43 +10:00
sequence.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
showbc.c py: Change jump-if-x-or-pop opcodes to have unsigned offset argument. 2022-03-28 15:43:09 +11:00
smallint.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
smallint.h py/smallint: Introduce MP_SMALL_INT_BITS macro. 2022-03-25 12:23:43 +11:00
stackctrl.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
stackctrl.h py/stackctrl: Prevent unused-var warning when stack checking disabled. 2021-06-05 11:03:09 +10:00
stream.c py/stream: Remove mp_stream_errno and use system errno instead. 2020-04-27 23:58:46 +10:00
stream.h py/stream.h: Include sys/types.h to get size_t and off_t for POSIX API. 2020-03-25 01:00:52 +11:00
unicode.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
unicode.h py/objstr: Add check for valid UTF-8 when making a str from bytes. 2017-09-06 16:43:09 +10:00
usermod.cmake rp2: Add support for USER_C_MODULES to CMake build system. 2021-03-31 00:26:01 +11:00
vm.c py/runtime: Allow multiple *args in a function call. 2022-03-31 16:59:30 +11:00
vmentrytable.h py/vmentrytable: Ignore GCC -Woverride-init. 2020-10-22 11:47:36 +02:00
vstr.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
warning.c py: Update my copyright info on some files. 2019-02-06 00:19:00 +11:00