Commit Graph

3168 Commits

Author SHA1 Message Date
Damien George
6b4b6d388b py/obj.h: Fix math.e constant for nan-boxing builds.
Due to a typo, math.e was too small by around 6e-11.
2018-05-01 23:25:18 +10:00
Ayke van Laethem
d43c737756 py/stream: Use uPy errno instead of system's for non-blocking check.
This is a more consistent use of errno codes.  For example, it may be that
a stream returns MP_EAGAIN but the mp_is_nonblocking_error() macro doesn't
catch this value because it checks for EAGAIN instead (which may have a
different value than MP_EAGAIN when MICROPY_USE_INTERNAL_ERRNO is enabled).
2018-05-01 15:54:50 +10:00
Damien George
96740be357 py/mperrno: Define MP_EWOULDBLOCK as EWOULDBLOCK, not EAGAIN.
Most modern systems have EWOULDBLOCK aliased to EAGAIN, ie they have the
same value.  But some systems use different values for these errnos and if
a uPy port is using the system errno values (ie not the internal uPy
values) then it's important to be able to distinguish EWOULDBLOCK from
EAGAIN.  Eg if a system call returned EWOULDBLOCK it must be possible to
check for this return value, and this patch makes this now possible.
2018-05-01 15:53:25 +10:00
Ayke van Laethem
deaa46aa66 py/nlrthumb: Fix Clang support wrt use of "return 0".
Clang defines __GNUC__ so we have to check for it specifically.
2018-04-27 15:10:42 +10:00
Damien George
ef12a4bd05 py: Refactor how native emitter code is compiled with a file per arch.
Instead of emitnative.c having configuration code for each supported
architecture, and then compiling this file multiple times with different
macros defined, this patch adds a file per architecture with the necessary
code to configure the native emitter.  These files then #include the
emitnative.c file.

This simplifies emitnative.c (which is already very large), and simplifies
the build system because emitnative.c no longer needs special handling for
compilation and qstr extraction.
2018-04-10 15:06:47 +10:00
Jeff Epler
cbf981f330 py/objgenerator: Check stack before resuming a generator.
This turns a hard crash in a recursive generator into a 'maximum recursion
depth exceeded' exception.
2018-04-10 14:06:26 +10:00
Damien George
cf31d384f1 py/stream: Switch stream close operation from method to ioctl.
This patch moves the implementation of stream closure from a dedicated
method to the ioctl of the stream protocol, for each type that implements
closing.  The benefits of this are:

1. Rounds out the stream ioctl function, which already includes flush,
   seek and poll (among other things).

2. Makes calling mp_stream_close() on an object slightly more efficient
   because it now no longer needs to lookup the close method and call it,
   rather it just delegates straight to the ioctl function (if it exists).

3. Reduces code size and allows future types that implement the stream
   protocol to be smaller because they don't need a dedicated close method.

Code size reduction is around 200 bytes smaller for x86 archs and around
30 bytes smaller for the bare-metal archs.
2018-04-10 13:41:32 +10:00
Jeff Epler
d6cf5c6749 py/objstr: In find/rfind, don't crash when end < start. 2018-04-05 16:14:17 +10:00
Damien George
f1df86a017 py/objint: Simplify LHS arg type checking in int binary op functions.
The LHS passed to mp_obj_int_binary_op() will always be an integer, either
a small int or a big int, so the test for this type doesn't need to include
an "other, unsupported type" case.
2018-04-05 01:11:26 +10:00
Damien George
3f420c0c27 py: Don't include mp_optimise_value or opt_level() if compiler disabled.
Without the compiler enabled the mp_optimise_value is unused, and the
micropython.opt_level() function is not useful, so exclude these from the
build to save RAM and code size.
2018-04-04 14:24:03 +10:00
Damien George
323b5f7270 py/modsys: Don't compile getsizeof function if feature is disabled. 2018-04-04 14:23:25 +10:00
Damien George
bc36521386 py/vm: Optimise handling of stackless mode when pystack is enabled.
When pystack is enabled mp_obj_fun_bc_prepare_codestate() will always
return a valid pointer, and if there is no more pystack available then it
will raise an exception (a RuntimeError).  So having pystack enabled with
stackless enabled automatically gives strict stackless mode.  There is
therefore no need to have code for strict stackless mode when pystack is
enabled, and this patch optimises the VM for such a case.
2018-04-04 00:51:10 +10:00
Damien George
c7f880eda3 py/vm: Don't do unnecessary updates of ip and sp variables.
Neither the ip nor sp variables are used again after the execution of the
RAISE_VARARGS opcode, so they don't need to be updated.
2018-04-04 00:46:31 +10:00
Damien George
f50b64cab5 py/runtime: Be sure that non-intercepted thrown object is an exception.
The VM expects that, if mp_resume() returns MP_VM_RETURN_EXCEPTION, then
the returned value is an exception instance (eg to add a traceback to it).
It's possible that a value passed to a generator's throw() is not an
exception so must be explicitly checked for if the thrown value is not
intercepted by the generator.

Thanks to @jepler for finding the bug.
2018-03-30 12:43:38 +11:00
Damien George
3280788195 py/runtime: Check that keys in dicts passed as ** args are strings.
Prior to this patch the code would crash if a key in a ** dict was anything
other than a str or qstr.  This is because mp_setup_code_state() assumes
that keys in kwargs are qstrs (for efficiency).

Thanks to @jepler for finding the bug.
2018-03-30 11:13:32 +11:00
Damien George
5edce4539b py/objexcept: Make MP_DEFINE_EXCEPTION public so ports can define excs. 2018-03-17 00:31:40 +11:00
Damien George
f6a1f18603 py/makeqstrdefs.py: Optimise by using compiled re's so it runs faster.
By using pre-compiled regexs, using startswith(), and explicitly checking
for empty lines (of which around 30% of the input lines are), automatic
qstr extraction is speed up by about 10%.
2018-03-16 23:54:06 +11:00
Damien George
e0bc438e4b py/obj.h: Move declaration of mp_obj_list_init to objlist.h.
If this function is used then objlist.h is already included to get the
definition of mp_obj_list_t.
2018-03-13 14:03:15 +11:00
Damien George
9f811e9096 py/obj.h: Clean up by removing commented-out inline versions of macros. 2018-03-13 14:01:55 +11:00
Damien George
d4b55eff44 py/misc.h: Remove unused count_lead_ones() inline function.
This function was never used for unicode/utf8 handling code, or anything
else, so remove it to keep things clean.
2018-03-13 13:23:30 +11:00
Damien George
9884a2c712 py/objint: Remove unreachable code checking for int type in format func.
All callers of mp_obj_int_formatted() are expected to pass in a valid int
object, and they do:

- mp_obj_int_print() should always pass through an int object because it is
  the print special method for int instances.

- mp_print_mp_int() checks that the argument is an int, and if not converts
  it to a small int.

This patch saves around 20-50 bytes of code space.
2018-03-02 11:01:24 +11:00
Damien George
955ee6477f py/formatfloat: Fix case where floats could render with negative digits.
Prior to this patch, some architectures (eg unix x86) could render floats
with "negative" digits, like ")".  For example, '%.23e' % 1e-80 would come
out as "1.0000000000000000/)/(,*0e-80".  This patch fixes the known cases.
2018-03-01 17:00:02 +11:00
Damien George
7b050fa76c py/formatfloat: Fix case where floats could render with a ":" character.
Prior to this patch, some architectures (eg unix x86) could render floats
with a ":" character in them, eg 1e+39 would come out as ":e+38" (":" is
just after "9" in ASCII so this is like 10e+38).  This patch fixes some of
these cases.
2018-03-01 16:02:59 +11:00
Damien George
bc12eca461 py/formatfloat: Fix rounding of %f format with edge-case FP values.
Prior to this patch the %f formatting of some FP values could be off by up
to 1, eg '%.0f' % 123 would return "122" (unix x64).  Depending on the FP
precision (single vs double) certain numbers would format correctly, but
others wolud not.  This patch should fix all cases of rounding for %f.
2018-03-01 15:51:03 +11:00
Damien George
a9f6d49218 py/vm: Simplify handling of special-case STOP_ITERATION in yield from.
There's no need to have MP_OBJ_NULL a special case, the code can re-use
the MP_OBJ_STOP_ITERATION value to signal the special case and the VM can
detect this with only one check (for MP_OBJ_STOP_ITERATION).
2018-02-27 15:48:09 +11:00
Damien George
22ade2f5c4 py/vm: Fix case of handling raised StopIteration within yield from.
This patch concerns the handling of an NLR-raised StopIteration, raised
during a call to mp_resume() which is handling the yield from opcode.

Previously, commit 6738c1dded introduced code
to handle this case, along with a test.  It seems that it was lucky that
the test worked because the code did not correctly handle the stack pointer
(sp).

Furthermore, commit 79d996a57b improved the
way mp_resume() propagated certain exceptions: it changed raising an NLR
value to returning MP_VM_RETURN_EXCEPTION.  This change meant that the
test introduced in gen_yield_from_ducktype.py was no longer hitting the
code introduced in 6738c1dded.

The patch here does two things:

1. Fixes the handling of sp in the VM for the case that yield from is
   interrupted by a StopIteration raised via NLR.

2. Introduces a new test to check this handling of sp and re-covers the
   code in the VM.
2018-02-27 15:39:31 +11:00
Damien George
9d8347a9aa py/mpstate.h: Add repl_line state for MICROPY_REPL_EVENT_DRIVEN. 2018-02-26 16:08:58 +11:00
Damien George
f75c7ad1a9 py/mpz: In mpz_clone, remove unused check for NULL dig.
This path for src->deg==NULL is never used because mpz_clone() is always
called with an argument that has a non-zero integer value, and hence has
some digits allocated to it (mpz_clone() is a static function private to
mpz.c all callers of this function first check if the integer value is zero
and if so take a special-case path, bypassing the call to mpz_clone()).

There is some unused and commented-out functions that may actually pass a
zero-valued mpz to mpz_clone(), so some TODOs are added to these function
in case they are needed in the future.
2018-02-25 22:59:19 +11:00
Damien George
c0bcf00ed1 py/asm*.c: Remove unnecessary check for num_locals<0 in asm entry func.
All callers of the asm entry function guarantee that num_locals>=0, so no
need to add an explicit check for it.  Use an assertion instead.

Also, the signature of asm_x86_entry is changed to match the other asm
entry functions.
2018-02-24 23:10:20 +11:00
Damien George
7dfa56e40e py/compile: Adjust c_assign_atom_expr() to use return instead of goto.
Makes the flow of the function a little more obvious, and allows to reach
100% coverage of compile.c when using gcov.
2018-02-24 23:03:17 +11:00
Damien George
638b860066 extmod/vfs_fat: Merge remaining vfs_fat_misc.c code into vfs_fat.c.
The only function left in vfs_fat_misc.c is fat_vfs_import_stat() which
can logically go into vfs_fat.c, allowing to remove vfs_fat_misc.c.
2018-02-23 17:24:57 +11:00
Damien George
6af4515969 py: Use "GEN" consistently for describing files generated in the build. 2018-02-22 12:48:51 +11:00
Damien George
65ef59a9b5 py/py.mk: Remove .. path component from list of extmod files.
This just makes it a bit cleaner in the output of the build process:
instead of "CC ../../py/../extmod/" there is now "CC ../../extmod/".
2018-02-22 12:48:51 +11:00
Damien George
8ca469cae2 py/py.mk: Split list of uPy sources into core and extmod files.
If a port only needs the core files then it can now use the $(PY_CORE_O)
variable instead of $(PY_O).  $(PY_EXTMOD_O) contains the list of extmod
files (including some files from lib/). $(PY_O) retains its original
definition as the list of all object file (including those for frozen code)
and is a convenience variable for ports that want everything.
2018-02-22 12:48:15 +11:00
Damien George
6e675c1baa py/objdeque: Use m_new0 when allocating items to avoid need to clear.
Saves a few bytes of code space, and is more efficient because with
MICROPY_GC_CONSERVATIVE_CLEAR enabled by default all memory is already
cleared when allocated.
2018-02-21 23:36:46 +11:00
Damien George
160d670868 py/objdeque: Protect against negative maxlen in deque constructor.
Otherwise passing -1 as maxlen will lead to a zero allocation and
subsequent unbound buffer overflow in deque.append() because i_put is
allowed to grow without bound.
2018-02-21 23:34:17 +11:00
Damien George
6c3faf6c17 py/objdeque: Allow to compile without warnings by disabling deque_clear. 2018-02-21 22:52:58 +11:00
Paul Sokolovsky
970eedce8f py/objdeque: Implement ucollections.deque type with fixed size.
So far, implements just append() and popleft() methods, required for
a normal queue. Constructor doesn't accept an arbitarry sequence to
initialize from (am empty deque is always created), so an empty tuple
must be passed as such. Only fixed-size deques are supported, so 2nd
argument (size) is required.

There's also an extension to CPython - if True is passed as 3rd argument,
append(), instead of silently overwriting the oldest item on queue
overflow, will throw IndexError. This behavior is desired in many
cases, where queues should store information reliably, instead of
silently losing some items.
2018-02-21 22:39:25 +11:00
Damien George
fe3e17b026 py/objint: Use MP_OBJ_IS_STR_OR_BYTES macro instead of 2 separate ones. 2018-02-21 00:20:46 +11:00
Damien George
8769049e93 py/objstr: Remove unnecessary check for positive splits variable.
At this point in the code the variable "splits" is guaranteed to be
positive due to the check for "splits == 0" above it.
2018-02-20 19:19:02 +11:00
Damien George
7e2a48858c py/modmicropython: Allow to have stack_use() func without mem_info().
The micropython.stack_use() function is useful to query the current C stack
usage, and it's inclusion in the micropython module doesn't need to be tied
to the inclusion of mem_info()/qstr_info() because it doesn't rely on any
of the code from these functions.  So this patch introduces the config
option MICROPY_PY_MICROPYTHON_STACK_USE which can be used to independently
control the inclusion of stack_use().  By default it is enabled if
MICROPY_PY_MICROPYTHON_MEM_INFO is enabled (thus not changing any of the
existing ports).
2018-02-20 18:30:22 +11:00
Damien George
209936880d py/builtinimport: Add compile-time option to disable external imports.
The new option is MICROPY_ENABLE_EXTERNAL_IMPORT and is enabled by default
so that the default behaviour is the same as before.  With it disabled
import is only supported for built-in modules, not for external files nor
frozen modules.  This allows to support targets that have no filesystem of
any kind and that only have access to pre-supplied built-in modules
implemented natively.
2018-02-20 18:00:44 +11:00
Damien George
6e7819ee2e py/objmodule: Factor common code for calling __init__ on builtin module. 2018-02-20 17:56:58 +11:00
Damien George
4e469085c1 py/objstr: Protect against creating bytes(n) with n negative.
Prior to this patch uPy (on a 32-bit arch) would have severe issues when
calling bytes(-1): such a call would call vstr_init_len(vstr, -1) which
would then +1 on the len and call vstr_init(vstr, 0), which would then
round this up and allocate a small amount of memory for the vstr.  The
bytes constructor would then attempt to zero out all this memory, thinking
it had allocated 2^32-1 bytes.
2018-02-19 16:25:30 +11:00
Damien George
165aab12a3 py/repl: Generalise REPL autocomplete to use qstr probing.
This patch changes the way REPL autocomplete finds matches.  It now probes
the target object for all qstrs via mp_load_method_maybe to look for a
match with the given input string.  Similar to how the builtin dir()
function works, this new algorithm now find all methods and instances of
user-defined classes including attributes of their parent classes.  This
helps a lot at the REPL prompt for user-discovery and to autocomplete names
even for classes that are derived.

The downside is that this new algorithm is slower than the previous one,
and in particular will be slower the more qstrs there are in the system.
But because REPL autocomplete is primarily used in an interactive way it is
not that important to make it fast, as long as it is "fast enough" compared
to human reaction.

On a slow microcontroller (CPU running at 16MHz) the autocomplete time for
a list of 35 names in the outer namespace (pressing tab at a bare prompt)
takes about 160ms with this algorithm, compared to about 40ms for the
previous implementation (this time includes the actual printing of the
names as well).  This time of 160ms is very reasonable especially given the
new functionality of listing all the names.

This patch also decreases code size by:

   bare-arm:    +0
minimal x86:  -128
   unix x64:  -128
unix nanbox:  -224
      stm32:   -88
     cc3200:   -80
    esp8266:   -92
      esp32:   -84
2018-02-19 16:12:44 +11:00
Damien George
98647e83c7 py/modbuiltins: Simplify and generalise dir() by probing qstrs.
This patch improves the builtin dir() function by probing the target object
with all possible qstrs via mp_load_method_maybe.  This is very simple (in
terms of implementation), doesn't require recursion, and allows to list all
methods of user-defined classes (without duplicates) even if they have
multiple inheritance with a common parent.  The downside is that it can be
slow because it has to iterate through all the qstrs in the system, but
the "dir()" function is anyway mostly used for testing frameworks and user
introspection of types, so speed is not considered a priority.

In addition to providing a more complete implementation of dir(), this
patch is simpler than the previous implementation and saves some code
space:

   bare-arm:   -80
minimal x86:   -80
   unix x64:   -56
unix nanbox:   -48
      stm32:   -80
     cc3200:   -80
    esp8266:  -104
      esp32:   -64
2018-02-19 16:12:44 +11:00
Damien George
a8775aaeb0 py/qstr: Add QSTR_TOTAL() macro to get number of qstrs. 2018-02-19 16:12:44 +11:00
Damien George
2a0cbc0d38 py/gc: Update comment now that gc_drain_stack is called gc_mark_subtree. 2018-02-19 16:08:20 +11:00
Ayke van Laethem
736faef223 py/gc: Make GC stack pointer a local variable.
This saves a bit in code size, and saves some precious .bss RAM:

                 .text  .bss
minimal CROSS=1: -28    -4
unix (64-bit):   -64    -8
2018-02-19 16:05:46 +11:00
Ayke van Laethem
5c9e5618e0 py/gc: Rename gc_drain_stack to gc_mark_subtree and pass it first block.
This saves a bit in code size:

minimal CROSS=1: -44
unix:            -96
2018-02-19 16:00:59 +11:00