Commit Graph

331 Commits

Author SHA1 Message Date
Jim Mussared
32db4c58f7 extmod/moddeflate: Change default window size.
The primary purpose of this commit is to make decompress default to
wbits=15 when the format is gzip (or auto format with gzip detected). The
idea is that someone decompressing a gzip stream should be able to use the
default `deflate.DeflateIO(f)` and it will "just work" for any input
stream, even though it uses a lot of memory.

This is done by making uzlib report gzip files as having wbits set to 15
in their header (where it previously only set the wbits out parameter for
zlib files), and then fixing up the logic in `deflateio_init_read`.

Updates the documentation to match.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-09-01 12:23:37 +10:00
Jim Mussared
e6c290c3d1 lib/uzlib: Add a source_read_data var to pass to source_read_cb.
For better abstraction for users of this API.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:29:34 +10:00
Jim Mussared
7f16bfca9f lib/uzlib/defl_static: Optimize zlib_start/finish_block.
Collapsing the two adjacent calls to outbits saves 32 bytes.

Bringing defl_static.c into lz77.c allows better inlining, saves 24 bytes.

Merge the Outbuf/uzlib_lz77_state_t structs, a minor simplification that
doesn't change code size.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:29:34 +10:00
Jim Mussared
ef5061fefd lib/uzlib/tinflate: Implement more compact lookup tables.
Saves 68 bytes on PYBV11.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:29:34 +10:00
Jim Mussared
d75a3cd861 lib/uzlib: Combine zlib/gzip header parsing to allow auto-detect.
This supports `wbits` values between +40 to +47.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:29:34 +10:00
Jim Mussared
c2b8e6e5d6 lib/uzlib: Clean up tinf -> uzlib rename.
This library used a mix of "tinf" and "uzlib" to refer to itself.  Remove
all use of "tinf" in the public API.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:29:24 +10:00
Jim Mussared
0900976384 lib/uzlib/defl_static: Implement some code size improvements.
This commit makes the following changes:
- Replace 256-byte reverse-bits-in-byte lookup table with computation.
- Replace length and distance code lookup tables with computation.
- Remove comp_disabled check (it's unused).
- Make the dest_write_cb take the data pointer directly, rather than the
  Outbuf.

Saves 500 bytes on PYBV11.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:58:33 +10:00
Jim Mussared
82db9926ed lib/uzlib/lz77: Always use separate history buffer.
Because we only use the streaming source, this is just extra code size.

Saves 64 bytes on PYBV11.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:57:49 +10:00
Damien George
c4feb806e0 lib/uzlib: Add memory-efficient, streaming LZ77 compression support.
The compression algorithm implemented in this commit uses much less memory
compared to the standard way of implementing it using a hash table and
large look-back window.  In particular the algorithm here doesn't allocate
hash table to store indices into the history of the previously seen text.
Instead it simply does a brute-force-search of the history text to find a
match for the compressor.  This is slower (linear search vs hash table
lookup) but with a small enough history (eg 512 bytes) it's not that slow.
And a small history does not impact the compression too much.

To give some more concrete numbers comparing memory use between the
approaches:

- Standard approach: inplace compression, all text to compress must be in
  RAM (or at least memory addressable), and then an additional 16k bytes
  RAM of hash table pointers, pointing into the text

- The approach in this commit: streaming compression, only a limited amount
  of previous text must be in RAM (user selectable, defaults to 512 bytes).

To compress, say, 1k of data, the standard approach requires all that data
to be in RAM, plus an additional 16k of RAM for the hash table pointers.
With this commit, you only need the 1k of data in RAM.  Or if it's
streaming from a file (or elsewhere), you could get away with only 256
bytes of RAM for the sliding history and still get very decent compression.

In summary: because compression takes such a large amount of RAM (in the
standard algorithm) and it's not really suitable for microcontrollers, the
approach taken in this commit is to minimise RAM usage as much as possible,
and still have acceptable performance (speed and compression ratio).

Signed-off-by: Damien George <damien@micropython.org>
2023-07-21 18:54:22 +10:00
Duncan Lowther
25fb651566
lib/oofatfs: Fix speculative read in create_name.
Signed-off-by: Duncan Lowther <Duncan.Lowther@glasgow.ac.uk>
2023-06-21 09:47:44 +01:00
Damien George
bd7196e123 lib/stm32lib: Update library for H5 v1.0.0.
Changes in this new library version are:
- Add H5 HAL at v1.0.0.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-15 11:02:56 +10:00
Damien George
1de335f7ff lib/pico-sdk: Update to version 1.5.1.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-14 22:20:20 +10:00
Damien George
825a95713e lib/btstack: Update to v1.5.6.2.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-14 19:23:27 +10:00
Damien George
ca3807081a lib/cyw43-driver: Update driver to latest version v1.0.1.
Includes a fix for combined BT+WiFi when using SPI transport.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-14 19:23:27 +10:00
Takeo Takahashi
0c58e29074 lib/fsp: Update FSP for renesas-ra to the latest version v4.4.0.
Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2023-05-11 16:18:39 +10:00
Carlosgg
61b8e1b2d8 lib/mbedtls: Update to mbedtls v2.28.3.
This is the latest release in the 2.28 long-time support branch.

Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
2023-05-03 17:33:38 +10:00
Damien George
4ff148de45 lib/mbedtls_errors: Update patch and error list for new mbedtls.
Running `./do-mp.sh` now generates this mp_mbedtls_errors.c with mbedTLS
v2.28.1.

Signed-off-by: Damien George <damien@micropython.org>
2023-05-01 10:54:02 +10:00
Carlosgg
b5f4611969 lib/mbedtls_errors: Add esp32-specific mbedtls error file.
This allows updating mp_mbedtls_errors.c for the other mbedtls based ports
based on mbedTLS v2.28.1.  This esp32-specific file will not be required
after updating IDF support to >= v4.4.1.

Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
2023-05-01 10:50:01 +10:00
Carlosgg
7e0a38f7f7 lib/mbedtls: Update to mbedtls v2.28.1.
Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
2023-04-30 18:07:57 +10:00
Damien George
c2bf91219c lib/mbedtls_errors: Update error list for current version of mbedtls.
This should have been updated as part of commit
7d73b9ff99, when mbedtls was changed to the
LTS branch v2.16.

Signed-off-by: Damien George <damien@micropython.org>
2023-04-30 17:40:29 +10:00
Damien George
c27d686d80 lib/micropython-lib: Update submodule to latest.
This brings in support to publish packages to PyPI, adds missing metadata
and fixes aioble descriptor flags.

Signed-off-by: Damien George <damien@micropython.org>
2023-04-26 12:59:26 +10:00
Damien George
41a2415e08 lib/cyw43-driver: Update driver to latest version v1.0.0.
Signed-off-by: Damien George <damien@micropython.org>
2023-04-26 12:57:04 +10:00
Damien George
2e5f071020 lib/stm32lib: Update library to get H7 v1.11.0.
Signed-off-by: Damien George <damien@micropython.org>
2023-03-21 14:02:41 +11:00
Damien George
2ec101e609 lib/micropython-lib: Update submodule to latest.
This brings in the following new modules: hs3003, bmi270, bmm150, cbor2
and senml.  It also refactors lsm6dsox and lsm9ds1.

Signed-off-by: Damien George <damien@micropython.org>
2023-03-08 09:53:43 +11:00
iabdalkader
ad863fefc0 lib/cyw43-driver: Update driver to latest version v0.9.0. 2023-03-08 00:56:00 +11:00
Damien George
198579ad6a lib/pico-sdk: Update to version 1.5.0.
Signed-off-by: Damien George <damien@micropython.org>
2023-02-17 14:03:19 +11:00
Damien George
fa8b2cb26d lib/cyw43-driver: Update driver to latest version.
Changes since the previous version:
- add an API to get the BSSID
- support auto channel selection when using BSSID for join
- improve performance for the Murata 1DX module and SDIO
- return EINVAL for invalid auth type
- add support for Bluetooth over SPI
- convert 4343WA1-7.45.98.50.combined to C header files
2023-02-17 14:03:19 +11:00
Damien George
c9baf498c8 lib/micropython-lib: Update submodule to latest.
This brings in the bundle-networking package.

Signed-off-by: Damien George <damien@micropython.org>
2023-02-01 12:42:06 +11:00
Damien George
64193c7de9 lib/re1.5: Add support for named classes in class sets.
Total code size change of this and previous commit:

   bare-arm:    +0 +0.000%
minimal x86:    +0 +0.000%
   unix x64:   +32 +0.004% standard
      stm32:   +24 +0.006% PYBV10
     cc3200:   +16 +0.009%
    esp8266:   +20 +0.003% GENERIC
      esp32:   +44 +0.003% GENERIC[incl +8(data)]
     mimxrt:   +32 +0.009% TEENSY40
 renesas-ra:   +24 +0.004% RA6M2_EK
        nrf:    +0 +0.000% pca10040
        rp2:   +24 +0.005% PICO
       samd:   +32 +0.012% ADAFRUIT_ITSYBITSY_M4_EXPRESS

Addresses issue #7920.

Signed-off-by: Damien George <damien@micropython.org>
2023-01-18 09:15:32 +11:00
Damien George
bd86ce5f82 lib/re1.5: Reduce code size when checking for named class char.
Signed-off-by: Damien George <damien@micropython.org>
2023-01-18 09:15:32 +11:00
Damien George
b151422cb2 lib/cyw43-driver: Update driver to latest version.
Changes since the previous version:
- remove mDNS
- implement lwIP IGMP MAC filter callback
- implement IPv6 support
- allow building with IGMP disabled
- fix handshake meggase WAIT_G1 event value
- increase EAPOL timeout from 2500ms to 500ms
- add function to get RSSI
- fix handling of open security networks
- remove support for automatically setting auth type

Signed-off-by: Damien George <damien@micropython.org>
2022-12-20 16:06:40 +11:00
Damien George
5de8d7eb64 lib/btstack: Update to v1.5.4.
This update includes a few bug fixes for BLE, support for LE audio, updates
to classic BT support, cmake support, and other things.

Signed-off-by: Damien George <damien@micropython.org>
2022-11-11 15:14:14 +11:00
David Lechner
0eead94181 lib/libm: Use __asm__ instead of asm.
`asm` is not part of the C standard and causes a complier error when
`-std=c99` is used. `__asm__` is the recommended alternative.

https://gcc.gnu.org/onlinedocs/gcc/extensions-to-the-c-language-family/alternate-keywords.html

Signed-off-by: David Lechner <david@pybricks.com>
2022-11-10 11:36:41 +11:00
Damien George
1531a7c8ca lib/micropython-lib: Update submodule to latest.
This brings in mip-cmdline, espflash, use of machine.dht_readinto, and
other improvements to existing libraries.

Signed-off-by: Damien George <damien@micropython.org>
2022-11-09 14:58:49 +11:00
Damien George
4f946ba963 lib/btstack: Update to v1.5.3.
Signed-off-by: Damien George <damien@micropython.org>
2022-10-22 13:21:28 +11:00
Jim Mussared
ba3652f15d lib/micropython-lib: Update submodule to latest.
This brings in the `mip` tool for installing packages.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-30 00:46:26 +10:00
yn386
ae0b0e7018 lib/stm32lib: Update library to get L1 v1.10.3, and some other fixes.
Changes in this new library version are:
- Add L1 HAL at v1.10.3.
- H7_HAL/rcc_ex: Add SPI45 to HAL_RCCEx_GetPeriphCLKFreq.
- L4_HAL/gpio_ex: Add #define for GPIO_AF14_TIM2 on L4P5/L4Q5.
- F4_HAL/i2c: Fix I2C frequency calculation macros.
- L1_HAL/utils: Fix compile error when USE_HAL_DRIVER is defined.
2022-09-25 23:56:20 +10:00
Jim Mussared
aeff6911d7 lib/micropython-lib: Update submodule to latest.
This brings in the drivers and libraries that were previously in this
repo.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-08 11:27:00 +10:00
Damien George
ccd210984e lib/micropython-lib: Update to latest version with manifest changes.
Signed-off-by: Damien George <damien@micropython.org>
2022-09-05 18:24:27 +10:00
Damien George
9a826e0f24 lib/lwip: Update lwIP to v2.1.3, tag STABLE-2_1_3_RELEASE.
There don't seem to be many changes going from v2.1.2 to v2.1.3 of lwIP.
Mostly they are:
- IPv6 fixes and improvements
- changes to apps and other code that MicroPython doesn't use
- comments and tests
- minor bug fixes

In particular there doesn't look to be any change to the API of any
function used by MicroPython.

Network multi tests pass on PYBD_SF2 and PYBD_SF6.  PYBD_SF2, PYBD_SF6 and
PICO_W have unchanged iperf3 performance.  Similar results for networking
on the mimxrt port.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-29 14:41:21 +10:00
Jim Mussared
9a1b7d8448 lib/micropython-lib: Add micropython-lib as a submodule.
Several boards now depend on libraries from micropython-lib.  Rather than
expecting micropython-lib to be available as a sibling of the micropython
repo, instead make it a submodule.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-03 00:04:06 +10:00
Peter Harper
9fd8250d69 lib/cyw43-driver: Update driver to latest version.
This version of the driver adds an event hook to call during firmware
download, and the ability to query the current power mode.
2022-07-26 01:37:47 +10:00
robert-hh
d1ef3815e6 lib/tinyusb: Update to the most recent master. 2022-07-19 11:26:54 +10:00
Damien George
f4e69ab103 lib/cyw43-driver: Update cyw43-driver to fix ap_auth mode setting.
Signed-off-by: Damien George <damien@micropython.org>
2022-07-05 11:04:20 +10:00
Damien George
f525e17793 lib/pico-sdk: Update to version 1.4.0.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-30 16:03:21 +10:00
Damien George
18a010b5b1 lib/cyw43-driver: Add new submodule for CYW43xx WiFi driver.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-30 16:03:21 +10:00
Andrew Leech
cb436e357f lib/wiznet5k: Add submodule for Wiznet Ethernet drivers.
Signed-off-by: Andrew Leech <andrew@alelec.net>
2022-06-03 14:29:11 +10:00
robert-hh
dbe8b0c0ca lib/nxp_driver: Update nxp_driver to v2.10. 2022-06-03 10:48:04 +10:00
Damien George
9269835226 lib/littlefs: Remove assignment of variables to themselves.
To prevent compiler warnings (eg on clang).

Signed-off-by: Damien George <damien@micropython.org>
2022-05-24 12:51:49 +10:00
Damien George
3a544b832b lib/littlefs: Guard lfs2_file_rawopen with LFS2_NO_MALLOC.
To prevent warnings about this function being unused when malloc is
disabled.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-24 12:51:17 +10:00