Commit Graph

11373 Commits

Author SHA1 Message Date
Damien George
8fff0b0acd unix/mpthreadport: Ensure enough thread stack to detect overflow.
Following up to 5e6cee07ab, some systems (eg
FreeBSD 12.0 64-bit) will crash if the stack-overflow margin is too small.
It seems the margin of 8192 bytes (or thereabouts) is always needed.  This
commit adds this much margin if the requested stack size is too small.

Fixes issue #5824.
2020-03-31 09:35:46 +11:00
David Lechner
581f9135a4 tests/run-tests: Add commands to print and clean *.exp,out files.
This adds a couple of commands to the run-tests script to print the diffs
of failed tests and also to clean up the .exp and .out files after failed
tests.  (And a spelling error is fixed while we are touching nearby code.)

Travis is also updated to use these new commands, including using it for
more builds.
2020-03-30 13:25:58 +11:00
David Lechner
6110cd3078 tests/float: Add new lexer test to test parsing of float without prefix.
Since automatically formatting tests with black, we have lost one line of
code coverage.  This adds an explicit test to ensure we are testing the
case that is no longer covered implicitly.
2020-03-30 13:23:05 +11:00
David Lechner
3dc324d3f1 tests: Format all Python code with black, except tests in basics subdir.
This adds the Python files in the tests/ directory to be formatted with
./tools/codeformat.py.  The basics/ subdirectory is excluded for now so we
aren't changing too much at once.

In a few places `# fmt: off`/`# fmt: on` was used where the code had
special formatting for readability or where the test was actually testing
the specific formatting.
2020-03-30 13:21:58 +11:00
David Lechner
488613bca6 tests/micropython/heapalloc_fail_set.py: Remove extra trailing comma.
Unlike tuples, sets do not need trailing comma when there is only one item.
2020-03-30 13:10:24 +11:00
David Lechner
2461349b27 travis: Build full unix coverage build on osx job.
This should help catch more compile errors with the clang compiler.
2020-03-30 12:05:22 +11:00
David Lechner
a2110bd3fc all: Fix implicit casts of float/double, and signed comparison.
These were found by buiding the unix coverage variant on macOS (so clang
compiler).  Mostly, these are fixing implicit cast of float/double to
mp_float_t which is one of those two and one mp_int_t to size_t fix for
good measure.
2020-03-30 12:04:21 +11:00
Lars Kellogg-Stedman
3a0f64fc7a tools/pyboard.py: Add -d as an alias for --device. 2020-03-30 11:37:32 +11:00
Lars Kellogg-Stedman
1cf994c48b tools/pyboard.py: Support setting device/baudrate from shell env vars.
Allow defaults for --device and --baudrate to be set in the environment
using PYBOARD_DEVICE and PYBOARD_BAUDRATE.
2020-03-30 11:37:32 +11:00
David Lechner
688323307a tests/basics/dict_pop.py: Remove extra comma in call and fix grammar. 2020-03-28 23:41:08 +11:00
Damien George
1a3e386c67 all: Remove spaces inside and around parenthesis.
Using new options enabled in the uncrustify configuration.
2020-03-28 23:36:44 +11:00
Damien George
b56caaf104 travis: Build NUCLEO_L073RZ instead of B_L072Z_LRWAN1 for stm32 job.
The NUCLEO_L073RZ build is slightly bigger and this MCU has only 192k flash
so this helps to catch flash overflow.
2020-03-28 13:56:16 +11:00
Damien George
83461e37d4 stm32/boards: Disable pend_throw, uheapq, utimeq on small-flash boards.
These are mainly used by the previous version of uasyncio which is now
replaced by a newer version, with built-in C module _uasyncio.  Saves about
1300 bytes of flash.
2020-03-28 13:55:59 +11:00
Damien George
09154f585f stm32/mpconfigport.h: Make most extended modules configurable by board. 2020-03-28 13:46:35 +11:00
David Lechner
6c77d5c7cd travis: Use grep with --text on qemu-arm output.
If the .out file contains non-text characters, grep won't show the output
unless we ask nicely.
2020-03-28 13:29:54 +11:00
David Lechner
0608fbff29 travis: Use custom PPA to get pre-built uncrustify.
Use PPA so that we don't have to rebuild uncrustify from source, speeding
up the job.  This also requires not running this test on arm64.
2020-03-28 11:40:47 +11:00
David Lechner
9418611c8a unix: Implement PEP 475 to retry syscalls failing with EINTR.
https://www.python.org/dev/peps/pep-0475/

This implements something similar to PEP 475 on the unix port, and for the
VfsPosix class.

There are a few differences from the CPython implementation:
- Since we call mp_handle_pending() between any ENITR's, additional
  functions could be called if MICROPY_ENABLE_SCHEDULER is enabled, not
  just signal handlers.
- CPython only handles signal on the main thread, so other threads will
  raise InterruptedError instead of retrying.  On MicroPython,
  mp_handle_pending() will currently raise exceptions on any thread.

A new macro MP_HAL_RETRY_SYSCALL is introduced to reduce duplicated code
and ensure that all instances behave the same.  This will also allow other
ports that use POSIX-like system calls (and use, eg, VfsPosix) to provide
their own implementation if needed.
2020-03-27 14:40:46 +11:00
David Lechner
5e6cee07ab unix/mpthreadport: Fix crash when thread stack size <= 8k.
The stack size adjustment for detecting stack overflow in threads was not
taking into account that the requested stack size could be <= 8k, in which
case the subtraction would overflow.  This is fixed in this commit by
ensuring that the adjustment can't be more than the available size.

This fixes the test tests/thread/thread_stacksize1.py which sometimes
crashes with a segmentation fault because of an uncaught NLR jump, which is
a "maximum recursion depth exceeded" exception.

Suggested-by: @dpgeorge
2020-03-27 13:59:18 +11:00
Damien George
dbba6b05dc stm32/mpconfigport.h: Remove unused root pointer for BTstack bindings.
This was a cut-and-paste error from the NimBLE bindings.
2020-03-27 00:30:37 +11:00
Damien George
ad004db662 esp32: Enable and freeze uasyncio. 2020-03-26 01:25:46 +11:00
Damien George
1d4d688b3b esp8266: Enable and freeze uasyncio.
Only included in GENERIC build.
2020-03-26 01:25:46 +11:00
Damien George
35e2dd0979 stm32: Enable and freeze uasyncio. 2020-03-26 01:25:45 +11:00
Damien George
3b68f36175 extmod/uasyncio: Add manifest.py for freezing uasyncio Py files. 2020-03-26 01:25:45 +11:00
Damien George
c99322f8d8 docs/library: Add initial docs for uasyncio module. 2020-03-26 01:25:45 +11:00
Damien George
91dd3948e8 unix: Enable uasyncio C helper module on coverage build. 2020-03-26 01:25:45 +11:00
Damien George
bc009fdd62 extmod/uasyncio: Add optional implementation of core uasyncio in C.
Implements Task and TaskQueue classes in C, using a pairing-heap data
structure.  Using this reduces RAM use of each Task, and improves overall
performance of the uasyncio scheduler.
2020-03-26 01:25:45 +11:00
Damien George
081d067662 tests/net_inet: Add uasyncio internet tests. 2020-03-26 01:25:45 +11:00
Damien George
38904b8937 tests/multi_net: Add uasyncio test for TCP server and client.
Includes a test where the (non uasyncio) client does a RST on the
connection, as a simple TCP server/client test where both sides are using
uasyncio, and a test for TCP stream close then write.
2020-03-26 01:25:45 +11:00
Damien George
18fa65e474 tests: Make default MICROPYPATH include extmod to find uasyncio. 2020-03-26 01:25:45 +11:00
Damien George
3667effff1 travis: Exclude some uasyncio tests on OSX. 2020-03-26 01:25:45 +11:00
Damien George
5d09a40df9 tests/run-tests: Skip uasyncio if no async, and skip one test on native. 2020-03-26 01:25:45 +11:00
Damien George
c4935f3049 tests/extmod: Add uasyncio tests.
All .exp files are included because they require CPython 3.8 which may not
always be available.
2020-03-26 01:25:45 +11:00
Damien George
63b9944382 extmod/uasyncio: Add new implementation of uasyncio module.
This commit adds a completely new implementation of the uasyncio module.
The aim of this version (compared to the original one in micropython-lib)
is to be more compatible with CPython's asyncio module, so that one can
more easily write code that runs under both MicroPython and CPython (and
reuse CPython asyncio libraries, follow CPython asyncio tutorials, etc).
Async code is not easy to write and any knowledge users already have from
CPython asyncio should transfer to uasyncio without effort, and vice versa.

The implementation here attempts to provide good compatibility with
CPython's asyncio while still being "micro" enough to run where MicroPython
runs. This follows the general philosophy of MicroPython itself, to make it
feel like Python.

The main change is to use a Task object for each coroutine.  This allows
more flexibility to queue tasks in various places, eg the main run loop,
tasks waiting on events, locks or other tasks.  It no longer requires
pre-allocating a fixed queue size for the main run loop.

A pairing heap is used to queue Tasks.

It's currently implemented in pure Python, separated into components with
lazy importing for optional components.  In the future parts of this
implementation can be moved to C to improve speed and reduce memory usage.
But the aim is to maintain a pure-Python version as a reference version.
2020-03-26 01:25:45 +11:00
Damien George
f05ae416ff stm32/softtimer: Initialise pairing-heap node before pushing to heap. 2020-03-26 01:21:04 +11:00
Damien George
f9741d18f6 unix/coverage: Init all pairheap test nodes before using them. 2020-03-26 01:21:04 +11:00
Damien George
6c7e78de72 py/pairheap: Add helper function to initialise a new node. 2020-03-26 01:21:04 +11:00
Damien George
c47a3ddf4a py/pairheap: Properly unlink node on pop and delete.
This fixes a bug in the pairing-heap implementation when nodes are deleted
with mp_pairheap_delete and then reinserted later on.
2020-03-26 01:21:04 +11:00
Damien George
98ab7643a7 travis: Print errors out for OSX job. 2020-03-26 01:21:04 +11:00
Damien George
ab00f4c44e qemu-arm: Set default board as mps2-an385 to get more flash for tests.
And use Ubuntu bionic for qemu-arm Travic CI job.
2020-03-26 01:21:04 +11:00
Damien George
f8fc78691d py/mpconfig.h: Enable MICROPY_MODULE_GETATTR by default.
To enable lazy loading of submodules (among other things), which is very
useful for MicroPython libraries that want to have optional subcomponents.

Disabled explicitly on minimal ports.
2020-03-26 01:21:04 +11:00
Zoltán Vörös
c5cc64175b ports: Add lib/libm/roundf.c to bare-metal Makefile's.
This function is not used by the core but having it as part of the build
allows it to be used by user C modules, or board extensions.  The linker
won't include it in the final firmware if it remains unused.
2020-03-25 01:22:00 +11:00
Damien George
bf4fb16250 esp32/modsocket: Handle poll of a closed socket.
This gets tests/extmod/uselect_poll_basic.py working on the esp32.
2020-03-25 01:17:12 +11:00
Andreas Motl
fbfea3b440 drivers/onewire: Fix undefined variable errors.
On CPython, and with pylint, the variables MATCH_ROM and SEARCH_ROM are
undefined.  This code works in MicroPython because these variables are
constants and the MicroPython parser/compiler optimises them out.  But it
is not valid Python because they are technically undefined within the scope
they are used.

This commit makes the code valid Python code.  The const part is removed
completely because these constants are part of the public API and so cannot
be moved to the global scope (where they could still use the MicroPython
const optimisation).
2020-03-25 01:09:14 +11:00
David Lechner
2f7d2bb3e2 py/stream.h: Include sys/types.h to get size_t and off_t for POSIX API. 2020-03-25 01:00:52 +11:00
David Lechner
58d9a4815d extmod/vfs_posix_file: Include unistd.h to get STD{IN,OUT,ERR}_FILENO. 2020-03-25 00:59:39 +11:00
David Lechner
d0edaf88a3 windows/windows_mphal: Fix missing semicolon. 2020-03-25 00:59:34 +11:00
David Lechner
3b07736b6d unix,windows: Use STDIN_FILENO, STDOUT_FILENO macros where appropriate.
This replaces 0 and 1 with STDIN_FILENO and STDOUT_FILENO to make the
intention of the code clearer.
2020-03-25 00:59:05 +11:00
David Lechner
100012bec6 windows: Remove custom definition of MP_PLAT_PRINT_STRN.
This removes the port-specific definition of MP_PLAT_PRINT_STRN on the
windows port, so that the default mp_hal_stdout_tx_strn_cooked() is always
used.  This fixes releasing the GIL during the call to write() (this was
missed in bc3499f010).

Also, mp_hal_dupterm_tx_strn() was defined but never used anywhere so it is
safe to delete it.
2020-03-25 00:56:38 +11:00
David Lechner
b1066a9f96 unix: Remove custom definition of MP_PLAT_PRINT_STRN.
This removes the port-specific definition of MP_PLAT_PRINT_STRN on the unix
port.  Since fee7e5617f this is no longer a
single function call so we are not really optimising anything over using
the default definition of MP_PLAT_PRINT_STRN which calls
mp_hal_stdout_tx_strn_cooked().
2020-03-25 00:54:18 +11:00
Damien George
9fa32169e9 esp8266/modnetwork: Add support for wlan.ifconfig('dhcp').
Fixes issue #5780.
2020-03-25 00:43:04 +11:00