Commit Graph

2867 Commits

Author SHA1 Message Date
Tom Collins
6f56412ec3 py/lexer: Process CR earlier to allow newlines checks on chr1.
Resolves an issue where lexer failed to accept CR after line continuation
character.  It also simplifies the code.
2017-05-12 15:14:24 +10:00
Damien George
a1f254028d py/mkrules.mk: Add dependency of .mpy files upon mpy-cross.
This ensures that mpy-cross is automatically built (and is up-to-date) for
ports that use frozen bytecode.  It also makes sure that .mpy files are
re-built if mpy-cross is changed.
2017-05-11 23:40:16 +10:00
Paul Sokolovsky
edc02bd952 unix/main: Implement -m option for packages. 2017-05-09 14:22:21 +03:00
Tom Collins
2998647c4e py/lexer: Simplify lexer startup by using dummy bytes and next_char().
Now consistently uses the EOL processing ("\r" and "\r\n" convert to "\n")
and EOF processing (ensure "\n" before EOF) provided by next_char().

In particular the lexer can now correctly handle input that starts with CR.
2017-05-09 14:43:23 +10:00
Damien George
6cfa61a4cc py/binary: Handle storing big-ints to all arrays types.
Prior to this patch only 'q' and 'Q' type arrays could store big-int
values.  With this patch any big int that is stored to an array is handled
by the big-int implementation, regardless of the typecode of the array.
This allows arrays to work with all type sizes on all architectures.
2017-05-09 10:41:00 +10:00
Paul Sokolovsky
4a4490ffcc py/modio: resource_stream: Implement "package" param handling. 2017-05-06 18:42:35 +03:00
Damien George
58bb73e010 py/objint: In int.from_bytes, only create big-int if really needed.
This patch ensures that int.from_bytes only creates a big-int if necessary,
by checking the value for a small-int overflow as it's being parsed.
2017-05-06 10:29:09 +10:00
Paul Sokolovsky
d7da2dba07 py/modio: Implement uio.resource_stream(package, resource_path).
The with semantics of this function is close to
pkg_resources.resource_stream() function from setuptools, which
is the canonical way to access non-source files belonging to a package
(resources), regardless of what medium the package uses (e.g. individual
source files vs zip archive). In the case of MicroPython, this function
allows to access resources which are frozen into the executable, besides
accessing resources in the file system.

This is initial stage of the implementation, which actually doesn't
implement "package" part of the semantics, just accesses frozen resources
from "root", or filesystem resource - from current dir.
2017-05-03 01:47:08 +03:00
stijn
2f0ce2a6f5 py: Cleanup use of global DEBUG preprocessor definition
The standard preprocessor definition to differentiate debug and non-debug
builds is NDEBUG, not DEBUG, so don't rely on the latter:
- just delete the use of it in objint_longlong.c as it has been stale code
  for years anyway (since commit [c4029e5]): SUFFIX isn't used anywhere.
- replace DEBUG with MICROPY_DEBUG_NLR in nlr.h: it is rarely used anymore
  so can be off by default
2017-04-30 14:28:37 +03:00
Damien George
f85fd79c6c py/mpz: In mpn_sub, use existing function to remove trailing zeros. 2017-04-25 12:22:04 +10:00
Damien George
c7aa86ce6f py/mpz: Strip trailing zeros from mpz value when set from bytes. 2017-04-25 12:06:10 +10:00
Damien George
dd11af209d py: Add LOAD_SUPER_METHOD bytecode to allow heap-free super meth calls.
This patch allows the following code to run without allocating on the heap:

    super().foo(...)

Before this patch such a call would allocate a super object on the heap and
then load the foo method and call it right away.  The super object is only
needed to perform the lookup of the method and not needed after that.  This
patch makes an optimisation to allocate the super object on the C stack and
discard it right after use.

Changes in code size due to this patch are:

   bare-arm: +128
    minimal: +232
   unix x64: +416
unix nanbox: +364
     stmhal: +184
    esp8266: +340
     cc3200: +128
2017-04-22 23:39:20 +10:00
Damien George
5335942b59 py/compile: Refactor handling of special super() call.
This patch refactors the handling of the special super() call within the
compiler.  It removes the need for a global (to the compiler) state variable
which keeps track of whether the subject of an expression is super.  The
handling of super() is now done entirely within one function, which makes
the compiler a bit cleaner and allows to easily add more optimisations to
super calls.

Changes to the code size are:

   bare-arm: +12
    minimal:  +0
   unix x64: +48
unix nanbox: -16
     stmhal:  +4
     cc3200:  +0
    esp8266: -56
2017-04-22 21:46:32 +10:00
Damien George
0dd6a59c89 py/compile: Don't do unnecessary check if iter parse node is a struct.
If we get to this point in the code then pn_iter is guaranteed to be a
struct.
2017-04-22 21:43:42 +10:00
Damien George
ae54fbf166 py/compile: Add COMP_RETURN_IF_EXPR option to enable return-if-else opt.
With this optimisation enabled the compiler optimises the if-else
expression within a return statement.  The optimisation reduces bytecode
size by 2 bytes for each use of such a return-if-else statement.  Since
such a statement is not often used, and costs bytes for the code, the
feature is disabled by default.

For example the following code:

    def f(x):
        return 1 if x else 2

compiles to this bytecode with the optimisation disabled (left column is
bytecode offset in bytes):

    00 LOAD_FAST 0
    01 POP_JUMP_IF_FALSE 8
    04 LOAD_CONST_SMALL_INT 1
    05 JUMP 9
    08 LOAD_CONST_SMALL_INT 2
    09 RETURN_VALUE

and to this bytecode with the optimisation enabled:

    00 LOAD_FAST 0
    01 POP_JUMP_IF_FALSE 6
    04 LOAD_CONST_SMALL_INT 1
    05 RETURN_VALUE
    06 LOAD_CONST_SMALL_INT 2
    07 RETURN_VALUE

So the JUMP to RETURN_VALUE is optimised and replaced by RETURN_VALUE,
saving 2 bytes and making the code a bit faster.
2017-04-22 14:58:01 +10:00
Damien George
40b40ffc98 py/compile: Extract parse-node kind at start of func for efficiency.
Otherwise the type of parse-node and its kind has to be re-extracted
multiple times.  This optimisation reduces code size by a bit (16 bytes on
bare-arm).
2017-04-22 14:23:47 +10:00
Damien George
fa03bbf0fd py/compile: Don't do unnecessary check if parse node is a struct.
PN_atom_expr_normal parse nodes always have structs for their second
sub-node, so simplify the check for the sub-node kind to save code size.
2017-04-22 14:13:37 +10:00
Damien George
4df013c8cc py/objtype: mp_obj_new_super doesn't need to be public, so inline it.
Saves code size (20 bytes on bare-arm) and makes it a tiny bit more
efficient.
2017-04-22 12:14:04 +10:00
Paul Sokolovsky
9e8f316392 extmod/moductypes: Fix bigint handling for 32-bit ports. 2017-04-21 16:43:21 +03:00
Damien George
7a72c0db5a py: Reduce str/repr precision of float numbers when floats are 30-bit.
With 30-bit floats there aren't enough bits to faithfully print 7 decimal
digits, so reduce the precision to 6 digits.
2017-04-21 16:21:56 +10:00
Damien George
bbb4b9822f py/modmicropython: Add micropython.kbd_intr() function.
It controls the character that's used to (asynchronously) raise a
KeyboardInterrupt exception.  Passing "-1" allows to disable the
interception of the interrupt character (as long as a port allows such a
behaviour).
2017-04-18 17:24:30 +10:00
Damien George
c7e8c6f7de py/gc: Execute finaliser code in a protected environment.
If a finaliser raises an exception then it must not propagate through the
GC sweep function.  This patch protects against such a thing by running
finaliser code via the mp_call_function_1_protected call.

This patch also adds scheduler lock/unlock calls around the finaliser
execution to further protect against any possible reentrancy issues: the
memory manager is already locked when doing a collection, but we also don't
want to allow any scheduled code to run, KeyboardInterrupts to interupt the
code, nor threads to switch.
2017-04-12 13:52:04 +10:00
Damien George
08242eed26 py/nlrsetjmp: Add check for failed NLR jump.
Also optimise the function so it only needs to call the MP_STATE_THREAD
macro once (following how other nlr code is written).
2017-04-12 13:50:31 +10:00
Damien George
a73501b1d6 py/objfloat: Add implementation of high-quality float hashing.
Disabled by default.
2017-04-12 13:38:17 +10:00
Damien George
816413e4b2 py: Optimise types for common case where type has a single parent type.
The common cases for inheritance are 0 or 1 parent types, for both built-in
types (eg built-in exceptions) as well as user defined types.  So it makes
sense to optimise the case of 1 parent type by storing just the type and
not a tuple of 1 value (that value being the single parent type).

This patch makes such an optimisation.  Even though there is a bit more
code to handle the two cases (either a single type or a tuple with 2 or
more values) it helps reduce overall code size because it eliminates the
need to create a static tuple to hold single parents (eg for the built-in
exceptions).  It also helps reduce RAM usage for user defined types that
only derive from a single parent.

Changes in code size (in bytes) due to this patch:

    bare-arm:       -16
    minimal (x86): -176
    unix (x86-64): -320
    unix nanbox:   -384
    stmhal:         -64
    cc3200:         -32
    esp8266:       -108
2017-04-12 13:22:21 +10:00
Damien George
fc710169b7 py/obj: Clean up and add comments describing mp_obj_type_t struct. 2017-04-12 13:20:26 +10:00
Damien George
e31fbd9b41 py/objint: Use unsigned arithmetic when formatting an integer.
Otherwise the edge case of the most negative integer value will not convert
correctly.
2017-04-11 15:18:35 +10:00
Damien George
f66df1efc8 py/objint: Extract small int value directly because type is known. 2017-04-11 15:16:09 +10:00
Damien George
79ce664952 py/runtime: When init'ing kbd intr exc, use tuple ptr instead of object. 2017-04-10 17:07:26 +10:00
Damien George
ee86de1f1a py: Make sure that static emg-exc-buffer is aligned to size of mp_obj_t.
This buffer is used to allocate objects temporarily, and such objects
require that their underlying memory be correctly aligned for their data
type.  Aligning for mp_obj_t should be sufficient for emergency exceptions,
but in general the memory buffer should aligned to the maximum alignment of
the machine (eg on a 32-bit machine with mp_obj_t being 4 bytes, a double
may not be correctly aligned).

This patch fixes a bug for certain nan-boxing builds, where mp_obj_t is 8
bytes and must be aligned to 8 bytes (even though the machine is 32 bit).
2017-04-10 16:02:56 +10:00
Damien George
81d2ca2b12 py/objtuple: Add support for inplace add (same as normal add). 2017-04-05 12:38:02 +10:00
Damien George
de9b53695d py: Raise a ValueError if range() step is zero.
Following CPython.  Otherwise one gets either an infinite loop (if code is
optimised by the uPy compiler) or possibly a divide-by-zero CPU exception.
2017-04-05 10:50:26 +10:00
Damien George
fc245d1ca4 py/objint: Consolidate mp_obj_new_int_from_float to one implementation.
This reduces code duplication and allows to make mp_classify_fp_as_int
static, which reduces code size.
2017-04-04 16:45:49 +10:00
Damien George
19f2e47d59 py: Add very simple but correct hashing for float and complex numbers.
Hashing of float and complex numbers that are exact (real) integers should
return the same integer hash value as hashing the corresponding integer
value.  Eg hash(1), hash(1.0) and hash(1+0j) should all be the same (this
is how Python is specified: if x==y then hash(x)==hash(y)).

This patch implements the simplest way of doing float/complex hashing by
just converting the value to int and returning that value.
2017-04-04 11:57:21 +10:00
Paul Sokolovsky
9a973977bb py/objstr: Use MICROPY_FULL_CHECKS for range checking when constructing bytes.
Split this setting from MICROPY_CPYTHON_COMPAT. The idea is to be able to
keep MICROPY_CPYTHON_COMPAT disabled, but still pass more of regression
testsuite. In particular, this fixes last failing test in basics/ for
Zephyr port.
2017-04-02 21:20:07 +03:00
Damien George
a5500a8aad py/obj.h: Make sequence grow more efficient and support overlapping.
The first memmove now copies less bytes in some cases (because len_adj <=
slice_len), and the memcpy is replaced with memmove to support the
possibility that dest and slice regions are overlapping.
2017-04-02 17:28:24 +10:00
Damien George
4c307bfba1 all: Move BYTES_PER_WORD definition from ports to py/mpconfig.h
It can still be overwritten by a port in mpconfigport.h but for almost
all cases one can use the provided default.
2017-04-01 11:39:38 +11:00
Damien George
b6c7e4b143 all: Use full path name when including mp-readline/timeutils/netutils.
This follows the pattern of how all other headers are now included, and
makes it explicit where the header file comes from.  This patch also
removes -I options from Makefile's that specify the mp-readline/timeutils/
netutils directories, which are no longer needed.
2017-03-31 22:29:39 +11:00
Damien George
4b753515c7 py/objzip: Convert mp_uint_t to size_t. 2017-03-30 22:58:08 +11:00
Paul Sokolovsky
29dd92c82a zephyr/Makefile: Rework to use modern, official build integration.
Build happens in 3 stages:

1. Zephyr config header and make vars are generated from prj.conf.
2. libmicropython is built using them.
3. Zephyr is built and final link happens.
2017-03-30 00:08:27 +03:00
Damien George
6b34107537 py: Change mp_uint_t to size_t for mp_obj_str_get_data len arg. 2017-03-29 12:56:45 +11:00
Damien George
6213ad7f46 py: Convert mp_uint_t to size_t for tuple/list accessors.
This patch changes mp_uint_t to size_t for the len argument of the
following public facing C functions:

mp_obj_tuple_get
mp_obj_list_get
mp_obj_get_array

These functions take a pointer to the len argument (to be filled in by the
function) and callers of these functions should update their code so the
type of len is changed to size_t.  For ports that don't use nan-boxing
there should be no change in generate code because the size of the type
remains the same (word sized), and in a lot of cases there won't even be a
compiler warning if the type remains as mp_uint_t.

The reason for this change is to standardise on the use of size_t for
variables that count memory (or memory related) sizes/lengths.  It helps
builds that use nan-boxing.
2017-03-29 12:56:17 +11:00
Damien George
f9b0e644e5 py/compile: Provide terse error message for invalid dict/set literals. 2017-03-29 12:44:27 +11:00
Damien George
18c059febf py: Shorten a couple of error messages. 2017-03-29 12:36:46 +11:00
Damien George
f55a059e7a py/compile: Simplify syntax-error messages for illegal assignments.
With this patch all illegal assignments are reported as "can't assign to
expression".  Before the patch there were special cases for a literal on
the LHS, and for augmented assignments (eg +=), but it seems a waste of
bytes (and there are lots of bytes used in error messages) to spend on
distinguishing such errors which a user will rarely encounter.
2017-03-29 12:28:33 +11:00
Damien George
5010d1958f py/lexer: Simplify and reduce code size for operator tokenising.
By removing the 'E' code from the operator token encoding mini-language the
tokenising can be simplified.  The 'E' code was only used for the !=
operator which is now handled as a special case; the optimisations for the
general case more than make up for the addition of this single, special
case.  Furthermore, the . and ... operators can be handled in the same way
as != which reduces the code size a little further.

This simplification also removes a "goto".

Changes in code size for this patch are (measured in bytes):

bare-arm:       -48
minimal x86:    -64
unix x86-64:   -112
unix nanbox:    -64
stmhal:         -48
cc3200:         -48
esp8266:        -76
2017-03-29 10:56:52 +11:00
Damien George
94c41bb06f py: Use mp_raise_TypeError/mp_raise_ValueError helpers where possible.
Saves 168 bytes on bare-arm.
2017-03-28 22:37:26 +11:00
Damien George
54507f78ee py/objmap: Convert mp_uint_t to size_t. 2017-03-27 12:19:24 +11:00
Damien George
40c1272e55 py/compile: When compiling super(), handle closed-over self variable.
The self variable may be closed-over in the function, and in that case the
call to super() should load the contents of the closure cell using
LOAD_DEREF (before this patch it would just load the cell directly).
2017-03-27 11:27:08 +11:00
Damien George
a0973b09ce py/vm: Fix VM opcode tracing to print correct stack pointer.
Also const_table is now moved to the code_state->fun_bc structure.
2017-03-27 10:52:04 +11:00