Commit Graph

182 Commits

Author SHA1 Message Date
Damien George
596f41da04 py: Reuse value stack in VM WITH_CLEANUP opcode to reduce C-stack size.
Saves 8 bytes C-stack on stmhal and 16 bytes on unix x86.
2015-02-10 13:21:42 +00:00
Damien George
7d414a1b52 py: Parse big-int/float/imag constants directly in parser.
Previous to this patch, a big-int, float or imag constant was interned
(made into a qstr) and then parsed at runtime to create an object each
time it was needed.  This is wasteful in RAM and not efficient.  Now,
these constants are parsed straight away in the parser and turned into
objects.  This allows constants with large numbers of digits (so
addresses issue #1103) and takes us a step closer to #722.
2015-02-08 01:57:40 +00:00
Damien George
dab1385177 py: Add load_const_obj to emitter, add LOAD_CONST_OBJ to bytecode.
This allows to directly load a Python object to the Python stack.  See
issue #722 for background.
2015-01-13 15:55:54 +00:00
Damien George
7ee91cf861 py: Add option to cache map lookup results in bytecode.
This is a simple optimisation inspired by JITing technology: we cache in
the bytecode (using 1 byte) the offset of the last successful lookup in
a map. This allows us next time round to check in that location in the
hash table (mp_map_t) for the desired entry, and if it's there use that
entry straight away.  Otherwise fallback to a normal map lookup.

Works for LOAD_NAME, LOAD_GLOBAL, LOAD_ATTR and STORE_ATTR opcodes.

On a few tests it gives >90% cache hit and greatly improves speed of
code.

Disabled by default.  Enabled for unix and stmhal ports.
2015-01-07 21:07:23 +00:00
Damien George
b4b10fd350 py: Put all global state together in state structures.
This patch consolidates all global variables in py/ core into one place,
in a global structure.  Root pointers are all located together to make
GC tracing easier and more efficient.
2015-01-07 20:33:00 +00:00
Damien George
51dfcb4bb7 py: Move to guarded includes, everywhere in py/ core.
Addresses issue #1022.
2015-01-01 20:32:09 +00:00
Damien George
f89d659e3b py: In VM, for selective ip saving, store 1 byte past last opcode.
This is for efficiency, so we don't need to subtract 1 from the ip
before storing it to code_state->ip.  It saves a lot of ROM bytes on
unix and stmhal.
2014-12-29 00:29:59 +00:00
Paul Sokolovsky
749575097f vm: Record exception ip only for instructions where exceptions may happen.
Mirroring ip to a volatile memory variable for each opcode is an expensive
operation. For quite a lot of often executed opcodes like stack manipulation
or jumps, exceptions cannot actually happen. So, record ip only for opcode
where that's possible.
2014-12-28 07:37:04 +02:00
Damien George
74eb44c392 py: Reduce size of VM exception stack element by 1 machine word.
This optimisation reduces the VM exception stack element (mp_exc_stack_t)
by 1 word, by using bit 1 of a pointer to store whether the opcode was a
FINALLY or WITH opcode.  This optimisation was pending, waiting for
maturity of the exception handling code, which has now proven itself.

Saves 1 machine word RAM for each exception (4->3 words per exception).
Increases stmhal code by 4 bytes, and decreases unix x64 code by 32
bytes.
2014-12-22 12:49:57 +00:00
Damien George
7764f163fa py: Fix label printing in showbc; print sp in vm trace. 2014-12-12 17:18:56 +00:00
Damien George
cd97a43f8d py, vm: Make unum a local variable for each opcode that uses it.
This makes no change to the generated code, but it's now easier to
understand since unum is not a "global" variable anymore.
2014-12-02 19:25:10 +00:00
Damien George
b0b0012fd8 py: Fix VM dispatch following a pending exception check. 2014-10-26 00:33:23 +01:00
Damien George
124df6f8d0 py: Add mp_pending_exception global variable, for VM soft interrupt.
This allows to implement KeyboardInterrupt on unix, and a much safer
ctrl-C in stmhal port.  First ctrl-C is a soft one, with hope that VM
will notice it; second ctrl-C is a hard one that kills anything (for
both unix and stmhal).

One needs to check for a pending exception in the VM only for jump
opcodes.  Others can't produce an infinite loop (infinite recursion is
caught by stack check).
2014-10-25 23:37:57 +01:00
Damien George
8456cc017b py: Compress load-int, load-fast, store-fast, unop, binop bytecodes.
There is a lot potential in compress bytecodes and make more use of the
coding space.  This patch introduces "multi" bytecodes which have their
argument included in the bytecode (by addition).

UNARY_OP and BINARY_OP now no longer take a 1 byte argument for the
opcode.  Rather, the opcode is included in the first byte itself.

LOAD_FAST_[0,1,2] and STORE_FAST_[0,1,2] are removed in favour of their
multi versions, which can take an argument between 0 and 15 inclusive.
The majority of LOAD_FAST/STORE_FAST codes fit in this range and so this
saves a byte for each of these.

LOAD_CONST_SMALL_INT_MULTI is used to load small ints between -16 and 47
inclusive.  Such ints are quite common and now only need 1 byte to
store, and now have much faster decoding.

In all this patch saves about 2% RAM for typically bytecode (1.8% on
64-bit test, 2.5% on pyboard test).  It also reduces the binary size
(because bytecodes are simplified) and doesn't harm performance.
2014-10-25 20:23:13 +01:00
Damien George
b534e1b9f1 py: Use variable length encoded uints in more places in bytecode.
Code-info size, block name, source name, n_state and n_exc_stack now use
variable length encoded uints.  This saves 7-9 bytes per bytecode
function for most functions.
2014-09-04 14:44:01 +01:00
Damien George
b427d6ae86 py: Fix line number printing for file with 1 line.
With a file with 1 line (and an error on that line), used to show the
line as number 0.  Now shows it correctly as line number 1.

But, when line numbers are disabled, it now prints line number 1 for any
line that has an error (instead of 0 as previously).  This might end up
being confusing, but requires extra RAM and/or hack logic to make it
print something special in the case of no line numbers.
2014-08-26 23:35:57 +01:00
Damien George
4747becc64 py: Improve encoding scheme for line-number to bytecode map.
Reduces by about a factor of 10 on average the amount of RAM needed to
store the line-number to bytecode map in the bytecode prelude.

Using CPython3.4's stdlib for statistics: previously, an average of
13 bytes were used per (bytecode offset, line-number offset) pair, and
now with this improvement, that's down to 1.3 bytes on average.

Large RAM usage before was due to some very large steps in line numbers,
both from the start of the first line in a function way down in the
file, and also functions that have big comments and/or big strings in
them (both cases were significant).

Although the savings are large on average for the CPython stdlib, it
won't have such a big effect for small scripts used in embedded
programming.

Addresses issue #648.
2014-07-31 16:12:01 +00:00
Damien George
40f3c02682 Rename machine_(u)int_t to mp_(u)int_t.
See discussion in issue #50.
2014-07-03 13:25:24 +01:00
Damien George
df896eceef Merge branch 'alloca' of github.com:marcusva/micropython into marcusva-alloca 2014-06-08 13:18:14 +01:00
Damien George
aabd83ea20 py: Merge mp_execute_bytecode into fun_bc_call.
This reduces stack usage by 16 words (64 bytes) for stmhal/ port.

See issue #640.
2014-06-07 14:16:08 +01:00
Marcus von Appen
0c90eb1658 - FreeBSD provides alloca() via stdlib.h, in contrast to Linux and Windows
- Move the includes for alloca() intp mpconfigport.h
2014-06-07 09:36:04 +02:00
Paul Sokolovsky
411732e93b vm: If there's no lineno info, set lineno in traceback to 0, not 1.
To clearly signify that lineno is not known.
2014-06-03 12:32:59 +03:00
Damien George
3ebd4d0cae py: Add option to disable set() object (enabled by default). 2014-06-01 13:46:47 +01:00
Damien George
fb510b3bf9 Rename bultins config variables to MICROPY_PY_BUILTINS_*.
This renames:
MICROPY_PY_FROZENSET -> MICROPY_PY_BUILTINS_FROZENSET
MICROPY_PY_PROPERTY -> MICROPY_PY_BUILTINS_PROPERTY
MICROPY_PY_SLICE -> MICROPY_PY_BUILTINS_SLICE
MICROPY_ENABLE_FLOAT -> MICROPY_PY_BUILTINS_FLOAT

See issue #35 for discussion.
2014-06-01 13:32:54 +01:00
Damien George
c60a261ef0 py, vm: Replace save_ip, save_sp with code_state->{ip, sp}.
This may seem a bit of a risky change, in that it may introduce crazy
bugs with respect to volatile variables in the VM loop.  But, I think it
should be fine: code_state points to some external memory, so the
compiler should always read/write to that memory when accessing the
ip/sp variables (ie not put them in registers).

Anyway, it passes all tests and improves on all efficiency fronts: about
2-4% faster (64-bit unix), 16 bytes less stack space per call (64-bit
unix) and slightly less executable size (unix and stmhal).

The reason it's more efficient is save_ip and save_sp were volatile
variables, so were anyway stored on the stack (in memory, not regs).
Thus converting them to code_state->{ip, sp} doesn't cost an extra
memory dereference (except maybe to get code_state, but that can be put
in a register and then made more efficient for other uses of it).
2014-06-01 12:32:28 +01:00
Damien George
1b87d1098a Merge branch 'vm-alloca' of github.com:pfalcon/micropython into pfalcon-vm-alloca
Conflicts:
	py/vm.c

Fixed stack underflow check.  Use UINT_FMT/INT_FMT where necessary.
Specify maximum VM-stack byte size by multiple of machine word size, so
that on 64 bit machines it has same functionality as 32 bit.
2014-06-01 12:06:17 +01:00
Damien George
5b5562c1d1 py: Fix stack underflow with optimised for loop. 2014-05-31 17:59:11 +01:00
Paul Sokolovsky
b4ebad3310 vm: Factor out structure with code execution state and pass it around.
This improves stack usage in callers to mp_execute_bytecode2, and is step
forward towards unifying execution interface for function and generators
(which is important because generators don't even support full forms
of arguments passing (keywords, etc.)).
2014-05-31 18:22:01 +03:00
Paul Sokolovsky
b16523aa95 vm: Don't unconditionally allocate state on stack, do that only if needed.
This makes sure that only as much stack allocated as actually used, reducing
stack usage for each Python function call.
2014-05-31 18:19:33 +03:00
Paul Sokolovsky
ff8da0b835 vm: Detect stack underflow in addition to overflow. 2014-05-31 18:14:54 +03:00
Paul Sokolovsky
1f07b7e3c3 py: Reformat few long functions argument lists for clarity. 2014-05-31 03:36:37 +03:00
Damien George
25c84643b6 py: Fix break from within a for loop.
Needed to pop the iterator object when breaking out of a for loop.  Need
also to be careful to unwind exception handler before popping iterator.

Addresses issue #635.
2014-05-30 15:20:41 +01:00
Damien George
503d611033 py: Implement long int parsing in int(...).
Addresses issue #627.
2014-05-28 14:07:21 +01:00
Damien George
d8675541a9 py, vm: Where possible, make variables local to each opcode.
This helps the compiler do its optimisation, makes it clear which
variables are local per opcode and which global, and makes it consistent
when extra variables are needed in an opcode (in addition to old obj1,
obj2 pair, for example).

Could also make unum local, but that's for another time.
2014-05-25 22:58:04 +01:00
Paul Sokolovsky
afaaf535e6 objslice: Support arbitrary objects start, stop, and step.
Older int-only encoding is not expressive enough to support arbitrary slice
assignment operations.
2014-05-25 01:42:24 +03:00
Damien George
ee3fd46f13 Rename configuration variables controling Python features.
Now of the form MICROPY_PY_*.  See issue #35.
2014-05-24 23:03:12 +01:00
Damien George
58ebde4664 Tidy up some configuration options.
MP_ALLOC_* -> MICROPY_ALLOC_*
MICROPY_PATH_MAX -> MICROPY_ALLOC_PATH_MAX
MICROPY_ENABLE_REPL_HELPERS -> MICROPY_HELPER_REPL
MICROPY_ENABLE_LEXER_UNIX -> MICROPY_HELPER_LEXER_UNIX
MICROPY_EXTRA_* -> MICROPY_PORT_*

See issue #35.
2014-05-21 20:32:59 +01:00
Paul Sokolovsky
da9f0924ef py, unix: Add copyright for modules I worked closely on. 2014-05-13 18:41:25 +03:00
Paul Sokolovsky
c48d6f7add py: Don't expect that type->getiter() always returns iterator, check for NULL.
This is better than forcing each getiter() implementation to raise exception.
2014-05-11 20:51:31 +03:00
Damien George
3417bc2f25 py: Rename byte_code to bytecode everywhere.
bytecode is the more widely used.  See issue #590.
2014-05-10 10:36:38 +01:00
Damien George
a32c1e41cc py: Improve native emitter; now supports more opcodes. 2014-05-07 18:30:52 +01:00
Damien George
04b9147e15 Add license header to (almost) all files.
Blanket wide to all .c and .h files.  Some files originating from ST are
difficult to deal with (license wise) so it was left out of those.

Also merged modpyb.h, modos.h, modstm.h and modtime.h in stmhal/.
2014-05-03 23:27:38 +01:00
Paul Sokolovsky
f54bcbf099 py, unix: Make "mpconfig.h" be first included, as other headers depend on it.
Specifically, nlr.h does.
2014-05-02 17:48:40 +03:00
Damien George
968bf34c4c py: Remove unnecessary LOAD_CONST_ID bytecode.
It's the same as LOAD_CONST_STR.
2014-04-27 19:12:05 +01:00
Damien George
db12891918 py: Eliminate 'op' variable in VM dispatch loop.
Remembering the last op is rarely needed, and when it is, can simply use
*save_ip.
2014-04-27 18:19:06 +01:00
Damien George
c0dc7c3dfa py, vm: Fix recent bug where state is freed too early. 2014-04-24 15:42:05 +01:00
Damien George
d5e8482c4a py, vm: Free heap-allocated state if it was allocated on the heap. 2014-04-24 13:52:06 +01:00
Paul Sokolovsky
c5e32c6995 vm: Add rudimentary bytecode execution tracing capability. 2014-04-23 03:46:00 +03:00
Damien George
5f6a25fc50 py: Wrap #if's around emitter functions that are used only by emitcpy.
3 emitter functions are needed only for emitcpy, and so we can #if them
out when compiling with emitcpy support.

Also remove unused SETUP_LOOP bytecode.
2014-04-20 18:02:27 +01:00
Damien George
3558f62fb5 py: Making closures now passes pointer to stack, not a tuple for vars.
Closed over variables are now passed on the stack, instead of creating a
tuple and passing that.  This way memory for the closed over variables
can be allocated within the closure object itself.  See issue #510 for
background.
2014-04-20 17:50:40 +01:00