Commit Graph

155 Commits

Author SHA1 Message Date
David Lechner
c118b5d0e4 extmod/extmod.mk: Separate out extmod file list from py.mk to extmod.mk.
This separates extmod source files from `py.mk`.  Previously, `py.mk`
assumed that every consumer of the py/ directory also wanted to include
extmod/.  However, this is not the case.  For example, building mpy-cross
uses py/ but doesn't need extmod/.

This commit moves all extmod-specific items from `py.mk` to `extmod.mk` and
explicitly includes `extmod.mk` in ports that use it.

Signed-off-by: David Lechner <david@pybricks.com>
2022-06-21 00:14:34 +10:00
Damien George
1f1afae622 examples/natmod/features3: Add example to test more natmod features.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-10 16:42:43 +10:00
Damien George
efe23aca71 all: Remove third argument to MP_REGISTER_MODULE.
It's no longer needed because this macro is now processed after
preprocessing the source code via cpp (in the qstr extraction stage), which
means unused MP_REGISTER_MODULE's are filtered out by the preprocessor.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-02 16:31:37 +10:00
Damien George
5956466c0e py/builtin: Clean up and simplify import_stat and builtin_open config.
The following changes are made:

- If MICROPY_VFS is enabled then mp_vfs_import_stat and mp_vfs_open are
  automatically used for mp_import_stat and mp_builtin_open respectively.

- If MICROPY_PY_IO is enabled then "open" is automatically included in the
  set of builtins, and points to mp_builtin_open_obj.

This helps to clean up and simplify the most common port configuration.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-25 13:04:45 +10:00
Damien George
b083cdba2b examples/embedding: Fix build with updated sys and os modules.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-25 11:47:30 +11:00
stijn
8bb50c6301 unix/Makefile: Remove explicit addition of -std=c++ flag.
This was added merely for building the C++ user module example, so it's a
better fit to add it in the corresponding micropython.mk.
2022-02-18 14:40:16 +11:00
Damien George
ab2923dfa1 all: Update Python formatting to latest Black version 22.1.0.
Signed-off-by: Damien George <damien@micropython.org>
2022-02-02 16:49:55 +11:00
Jim Mussared
b326edf68c all: Remove MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE.
This commit removes all parts of code associated with the existing
MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE optimisation option, including the
-mcache-lookup-bc option to mpy-cross.

This feature originally provided a significant performance boost for Unix,
but wasn't able to be enabled for MCU targets (due to frozen bytecode), and
added significant extra complexity to generating and distributing .mpy
files.

The equivalent performance gain is now provided by the combination of
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE (which has
been enabled on the unix port in the previous commit).

It's hard to provide precise performance numbers, but tests have been run
on a wide variety of architectures (x86-64, ARM Cortex, Aarch64, RISC-V,
xtensa) and they all generally agree on the qualitative improvements seen
by the combination of MICROPY_OPT_LOAD_ATTR_FAST_PATH and
MICROPY_OPT_MAP_LOOKUP_CACHE.

For example, on a "quiet" Linux x64 environment (i3-5010U @ 2.10GHz) the
change from CACHE_MAP_LOOKUP_IN_BYTECODE, to LOAD_ATTR_FAST_PATH combined
with MAP_LOOKUP_CACHE is:

diff of scores (higher is better)
N=2000 M=2000       bccache -> attrmapcache      diff      diff% (error%)
bm_chaos.py        13742.56 ->   13905.67 :   +163.11 =  +1.187% (+/-3.75%)
bm_fannkuch.py        60.13 ->      61.34 :     +1.21 =  +2.012% (+/-2.11%)
bm_fft.py         113083.20 ->  114793.68 :  +1710.48 =  +1.513% (+/-1.57%)
bm_float.py       256552.80 ->  243908.29 : -12644.51 =  -4.929% (+/-1.90%)
bm_hexiom.py         521.93 ->     625.41 :   +103.48 = +19.826% (+/-0.40%)
bm_nqueens.py     197544.25 ->  217713.12 : +20168.87 = +10.210% (+/-3.01%)
bm_pidigits.py      8072.98 ->    8198.75 :   +125.77 =  +1.558% (+/-3.22%)
misc_aes.py        17283.45 ->   16480.52 :   -802.93 =  -4.646% (+/-0.82%)
misc_mandel.py     99083.99 ->  128939.84 : +29855.85 = +30.132% (+/-5.88%)
misc_pystone.py    83860.10 ->   82592.56 :  -1267.54 =  -1.511% (+/-2.27%)
misc_raytrace.py   21490.40 ->   22227.23 :   +736.83 =  +3.429% (+/-1.88%)

This shows that the new optimisations are at least as good as the existing
inline-bytecode-caching, and are sometimes much better (because the new
ones apply caching to a wider variety of map lookups).

The new optimisations can also benefit code generated by the native
emitter, because they apply to the runtime rather than the generated code.
The improvement for the native emitter when LOAD_ATTR_FAST_PATH and
MAP_LOOKUP_CACHE are enabled is (same Linux environment as above):

diff of scores (higher is better)
N=2000 M=2000        native -> nat-attrmapcache  diff      diff% (error%)
bm_chaos.py        14130.62 ->   15464.68 :  +1334.06 =  +9.441% (+/-7.11%)
bm_fannkuch.py        74.96 ->      76.16 :     +1.20 =  +1.601% (+/-1.80%)
bm_fft.py         166682.99 ->  168221.86 :  +1538.87 =  +0.923% (+/-4.20%)
bm_float.py       233415.23 ->  265524.90 : +32109.67 = +13.756% (+/-2.57%)
bm_hexiom.py         628.59 ->     734.17 :   +105.58 = +16.796% (+/-1.39%)
bm_nqueens.py     225418.44 ->  232926.45 :  +7508.01 =  +3.331% (+/-3.10%)
bm_pidigits.py      6322.00 ->    6379.52 :    +57.52 =  +0.910% (+/-5.62%)
misc_aes.py        20670.10 ->   27223.18 :  +6553.08 = +31.703% (+/-1.56%)
misc_mandel.py    138221.11 ->  152014.01 : +13792.90 =  +9.979% (+/-2.46%)
misc_pystone.py    85032.14 ->  105681.44 : +20649.30 = +24.284% (+/-2.25%)
misc_raytrace.py   19800.01 ->   23350.73 :  +3550.72 = +17.933% (+/-2.79%)

In summary, compared to MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE, the new
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE options:
- are simpler;
- take less code size;
- are faster (generally);
- work with code generated by the native emitter;
- can be used on embedded targets with a small and constant RAM overhead;
- allow the same .mpy bytecode to run on all targets.

See #7680 for further discussion.  And see also #7653 for a discussion
about simplifying mpy-cross options.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-09-16 16:04:03 +10:00
Damien George
136369d72f all: Update to point to files in new shared/ directory.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-12 17:08:10 +10:00
Tim Radvan
f842a40df4 rp2/rp2_pio: Add fifo_join support for PIO.
The PIO state machines on the RP2040 have 4 word deep TX and RX FIFOs.  If
you only need one direction, you can "merge" them into either a single 8
word deep TX or RX FIFO.

We simply add constants to the PIO object, and set the appropriate bits in
`shiftctrl`.

Resolves #6854.

Signed-off-by: Tim Radvan <tim@tjvr.org>
2021-04-17 00:45:38 +10:00
Santeri Paavolainen
42035e5ede examples/embedding: Fix example so it compiles again.
There were a few changes that had broken this example, specifically
2cdf1d25f5 removed file.c from ports/unix.
And (at least for MacOS) mp_state_ctx must be placed in the BSS with
-fno-common so it is visible to the linker.

Signed-off-by: Santeri Paavolainen <santtu@iki.fi>
2021-04-09 15:47:54 +10:00
Damien George
d87f42b0e5 examples/usercmodules: Simplify user C module enabling.
It's a bit of a pitfall with user C modules that including them in the
build does not automatically enable them.  This commit changes the docs and
examples for user C modules to encourage writers of user C modules to
enable them unconditionally.  This makes things simpler and covers most use
cases.

See discussion in issue #6960, and also #7086.

Signed-off-by: Damien George <damien@micropython.org>
2021-04-01 16:27:38 +11:00
Phil Howard
cc497d4c6a examples/usercmodule: Add micropython.cmake to the C and CPP examples.
examples/usercmodule/micropython.cmake:

Root micropython.cmake file is responsible for including modules.

examples/usercmodule/cexample/micropython.cmake:
examples/usercmodule/cppexample/micropython.cmake:

Module micropython.cmake files define the target and link it to usermod.

Signed-off-by: Phil Howard <phil@pimoroni.com>
2021-03-31 00:27:46 +11:00
Damien George
ad4656b861 all: Rename BYTES_PER_WORD to MP_BYTES_PER_OBJ_WORD.
The "word" referred to by BYTES_PER_WORD is actually the size of mp_obj_t
which is not always the same as the size of a pointer on the target
architecture.  So rename this config value to better reflect what it
measures, and also prefix it with MP_.

For uses of BYTES_PER_WORD in setting the stack limit this has been
changed to sizeof(void *), because the stack usually grows with
machine-word sized values (eg an nlr_buf_t has many machine words in it).

Signed-off-by: Damien George <damien@micropython.org>
2021-02-04 22:46:42 +11:00
Tim Radvan
3ea05e499d examples/rp2: Add pio_uart_rx.py example.
This was adapted from the `pio/uart_rx` example from the `pico-examples`
repository:
https://github.com/raspberrypi/pico-examples/blob/master/pio/uart_rx/uart_rx.pio

It demonstrates the `jmp_pin` feature in action.

Signed-off-by: Tim Radvan <tim@tjvr.org>
2021-02-02 11:33:51 +11:00
Damien George
469345e728 rp2: Add new port to Raspberry Pi RP2 microcontroller.
This commit adds a new port "rp2" which targets the new Raspberry Pi RP2040
microcontroller.

The build system uses pure cmake (with a small Makefile wrapper for
convenience).  The USB driver is TinyUSB, and there is a machine module
with most of the standard classes implemented.  Some examples are provided
in the examples/rp2/ directory.

Work done in collaboration with Graham Sanderson.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-30 00:42:29 +11:00
Jim Mussared
f6fd46c402 examples/bluetooth: Add bonding/passkey demo.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:44:00 +11:00
Andrew Leech
1697ff335d extmod/modbluetooth: Allow setting char/desc enc/auth options.
This widens the characteristic/descriptor flags to 16-bit, to allow setting
encryption/authentication requirements.

Sets the required flags for NimBLE and btstack implementations.

The BLE.FLAG_* constants will eventually be deprecated in favour of copy
and paste Python constants (like the IRQs).

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:36:50 +11:00
stijn
25c4563f26 examples: Add example code for user C modules, both C and C++.
Add working example code to provide a starting point for users with files
that they can just copy, and include the modules in the coverage test to
verify the complete user C module build functionality.  The cexample module
uses the code originally found in cmodules.rst, which has been updated to
reflect this and partially rewritten with more complete information.
2020-10-29 15:30:42 +11:00
Jim Mussared
0fd0eb00aa examples/bluetooth: Update to use positional-only args to irq().
To match 6a6a5f9e15.
2020-09-26 21:19:18 +10:00
Damien George
06659077a8 all: Update Python code to conform to latest black formatting.
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:

- If there is a trailing comma in a list (eg [], () or function call) then
  that list is now written out with one line per element.  So remove such
  trailing commas where the list should stay on one line.

- Spaces at the start of """ doc strings are removed.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-29 15:18:01 +10:00
Jim Mussared
9d823a5d9a extmod/modbluetooth: Add event for "indicate acknowledgement".
This commit adds the IRQ_GATTS_INDICATE_DONE BLE event which will be raised
with the status of gatts_indicate (unlike notify, indications require
acknowledgement).

An example of its use is added to ble_temperature.py, and to the multitests
in ble_characteristic.py.

Implemented for btstack and nimble bindings, tested in both directions
between unix/btstack and pybd/nimble.
2020-07-20 23:26:41 +10:00
Jim Mussared
89a95b7c85 examples/bluetooth: Add simple UART demo with central and peripheral. 2020-07-18 14:34:29 +10:00
Jim Mussared
07aec4681f examples/bluetooth: In ble_advertising.py, skip appearance if not set. 2020-07-18 14:23:20 +10:00
Jim Mussared
c6fd6a0d72 examples/bluetooth: Fix event code in ble_temperature_central.py. 2020-06-10 22:39:54 +10:00
stijn
51fd6c9777 extmod/ure: Use single function for match/search/sub.
Saves about 500 bytes on unix x64 and enables CPython-conform
usage of passing a re object to these functions.
2020-06-08 09:16:09 +02:00
jxltom
834b482e67 examples/bluetooth: Fix incorrect value of BR/EDR flag in advertising.
According to Supplement to the Bluetooth Core Specification v8 Part A
1.3.1, to support BR/EDR the code should set the fifth bit (Simultaneous LE
and BR/EDR to Same Device Capable (Controller)) and fourth bit
(Simultaneous LE and BR/EDR to Same Device Capable (Host)) of the flag.
2020-06-05 14:24:09 +10:00
Jim Mussared
1cad63c0bc extmod/modbluetooth: Ensure status=0 always on success.
This commit makes sure that all discovery complete and read/write status
events set the status to zero on success.

The status value will be implementation-dependent on non-success cases.
2020-06-05 14:11:04 +10:00
Jim Mussared
c07ea3e4c2 extmod/modbluetooth: Implement read done event.
On btstack there's no status associated with the read result, it comes
through as a separate event.  This allows you to detect read failures or
timeouts.
2020-06-05 14:08:15 +10:00
Jim Mussared
6a3c89d584 extmod/modbluetooth: Add discover complete events for svc/char/desc.
Without this it's difficult to implement a state machine correctly if the
desired services are not found.
2020-06-05 14:07:52 +10:00
Jim Mussared
e6881f0829 extmod/modbluetooth: Make modbluetooth event not a bitfield.
There doesn't appear to be any use for only triggering on specific events,
so it's just easier to number them sequentially.  This makes them smaller
values so they take up only 1 byte in the ringbuf, only 1 byte for the
opcode in the bytecode, and makes room for more events.

Also add a couple of new event types that need to be implemented (to avoid
re-numbering later).

And rename _COMPLETE and _STATUS to _DONE for consistency.

In the future the "trigger" keyword argument can be reinstated by requiring
the user to compute the bitmask, eg:

    ble.irq(handler, 1 << _IRQ_SCAN_RESULT | 1 << _IRQ_SCAN_DONE)
2020-06-05 14:04:20 +10:00
Damien George
73c58150f5 extmod/modbtree: Retain reference to underlying stream so it's not GC'd.
For ports that have a system malloc which is not garbage collected (eg
unix, esp32), the stream object for the DB must be retained separately to
prevent it from being reclaimed by the MicroPython GC (because the
berkeley-db library uses malloc to allocate the DB structure which stores
the only reference to the stream).

Although in some cases the user code will explicitly retain a reference to
the underlying stream because it needs to call close() on it, this is not
always the case, eg in cases where the DB is intended to live forever.

Fixes issue #5940.
2020-05-02 16:08:04 +10:00
Damien George
5c8bf12acf all: Fix auto-enable of MICROPY_GCREGS_SETJMP to select GC behaviour.
Only enable it if MICROPY_GCREGS_SETJMP is not already defined, and no
supported architecture is defined.
2020-04-30 16:49:42 +10:00
Jim Mussared
710426024a all: Factor gchelper code to one place and use it for unix & ARM ports.
No functionality change is intended with this commit, it just consolidates
the separate implementations of GC helper code to the lib/utils/ directory
as a general set of helper functions useful for any port.  This reduces
duplication of code, and makes it easier for future ports or embedders to
get the GC implementation correct.

Ports should now link against gchelper_native.c and either gchelper_m0.s or
gchelper_m3.s (currently only Cortex-M is supported but other architectures
can follow), or use the fallback gchelper_generic.c which will work on
x86/x64/ARM.

The gc_helper_get_sp function from gchelper_m3.s is not really GC related
and was only used by cc3200, so it has been moved to that port and renamed
to cortex_m3_get_sp.
2020-04-29 23:45:19 +10:00
Jim Mussared
def76fe4d9 all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
Jim Mussared
c34e7b9d4c py/dynruntime.mk: Set MICROPY_ENABLE_DYNRUNTIME instead of per module.
So this setting could be used by other source files if needed.
2020-04-05 14:13:02 +10:00
Thomas Friebel
baf8aa286a examples/bluetooth: Replace "connectable" parameter with "adv_type".
Follow up to dd0bc26e65 which changed the
parameter list of the IRQ_SCAN_RESULT event.  Adapt
ble_temperature_central.py accordingly.
2020-03-12 10:58:08 +11:00
Damien George
9c07c973c1 examples/natmod: Add .gitignore to ignore generated example .mpy files. 2020-03-03 12:54:17 +11:00
Damien George
69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Damien George
3c58d9a1a5 examples/bluetooth/ble_temperature_central.py: Shorten comment.
So the line length is less than 100 characters.
2020-02-28 10:30:49 +11:00
Damien George
a0441fc15d examples/accellog.py: Shift long comments to their own line.
To improve interaction with black formatter.
2020-02-28 10:30:37 +11:00
Damien George
a3df152fef examples/natmod: Add very simple features0 example to compute factorial. 2019-12-19 17:06:27 +11:00
Damien George
ba12cdba85 examples/network: Add testing key/cert to SSL HTTP server example.
This example will now work on all ports with networking and ssl support,
with both axtls and mbedtls.
2019-12-18 15:04:00 +11:00
Damien George
ba84453f77 examples/natmod: Add urandom native module example. 2019-12-13 13:33:40 +11:00
Damien George
60c3c22a0d examples/natmod: Add features1 and features2 examples. 2019-12-12 20:15:28 +11:00
Damien George
42c1aed2bb examples/natmod: Add ure example. 2019-12-12 20:15:28 +11:00
Damien George
2a485e1084 examples/natmod: Add framebuf example. 2019-12-12 20:15:28 +11:00
Damien George
16e591e412 examples/natmod: Add uzlib example. 2019-12-12 20:15:28 +11:00
Damien George
83f9fb169e examples/natmod: Add uheapq example. 2019-12-12 20:15:28 +11:00
Damien George
37817ab4ba examples/natmod: Add btree example. 2019-12-12 20:15:28 +11:00