Commit Graph

618 Commits

Author SHA1 Message Date
Damien George
e160fe7bc6 tools/pyboard.py: Rename ProcessPtyToTerminal member "ser" to "serial".
So that this file doesn't need to be excluded from codespell.

Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 18:03:06 +10:00
Damien George
d77c35f110 tools/ci.sh: Add functions to check code spelling using codespell.
Signed-off-by: Damien George <damien@micropython.org>

tools/ci.sh: Explicitly specify pyproject.toml.

Signed-off-by: Damien George <damien@micropython.org>

tools/ci.sh: Import tomli.

Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 16:01:14 +10:00
Damien George
07a719a315 tools/ci.sh: Build both SAMD21 and SAMD51 boards as part of samd CI.
Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 15:11:52 +10:00
Takeo Takahashi
dc7de6ed83 renesas-ra: Change MICROPY_HW_BOARD_NAME definition to product name.
Changes in this commit:
- Change MICROPY_HW_BOARD_NAME definition to match the product name.
- Rename board folder's name to match the product name style.
- Change related files like Makefile, document descriptions, test cases, CI
  and tools.

Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2023-04-27 14:12:53 +10:00
Jim Mussared
a42a8022d8 extmod/modbluetooth: Make all HCI transports trace in the same format.
- Use HCI_TRACE macro consistently.
 - Use the same colour formatting.
 - Add a tool to convert to .pcap for Wireshark.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-04-26 11:37:20 +10:00
Jonas Scharpf
b525f1c9ec tools/mpremote: Add ctrl-x as additonal mpremote disconnect shortcut.
The mpremote REPL can now be closed with either ctrl+] or ctrl+x, which
gives users a choice, useful if the ']' key is difficult to access.

Fixes issue #11197.

Signed-off-by: Jonas Scharpf <jonas@brainelectronics.de>
2023-04-15 00:07:40 +10:00
Jim Mussared
cfd3b70934 tools/manifestfile.py: Add support for publishing packages to PyPI.
This adds a new MODE_PYPROJECT, which gives basic support to allow
packaging a small subset of micropython-lib packages to PyPI.

This change allows a package in micropython-lib to:
- Add a "pypi" name to its metadata indicating that it's based on a PyPI
  package.
- Add "stdlib" to its metadata indicating that it's a micropython version
  of a stdlib package.
- Add a "pypi_publish" name to its metadata to indicate that it can be
  published to PyPI (this can be different to the package name, e.g. "foo"
  might want to be published as "micropython-foo").

When a package requires() another one, if it's in MODE_PYPROJECT then if
the package is from pypi then it will record that as a pypi dependency
instead (or no dependency at all if it's from stdlib).

Also allows require() to explicitly specify the pypi name.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-04-07 12:53:53 +10:00
Jos Verlinde
9f74ffb6eb tools/pyboard.py: Fix ESPxx boards hanging in bootloader after reset.
This is a follow up to d263438a6e, which
solved one problem (reset on disconnect) but introduced a second one (hang
in bootloader).

To solve both probles, False/False is needed for DTR/RTS for ESPxx, but
that would then block stm32 and others.  Any unconditional combination of
DTR/RTS ends up blocking normal operation on some type of board or another.

A simple overview (for windows only):

  DTR          CTS            ESP8266/ESP32          STM32/SAMD51/RP2040
  unspecified  unspecified    Reset on disconnect    OK
  True         False          Hang in bootloader     OK
  False        False          OK                     No Repl
  True         True           Reset on disconnect    No Repl
  False        True           Reset on disconnect    No Repl

  serial.manufacturer:        wch.cn/Silicon Labs    Microsoft

  serial.description:         USB-SERIAL CH340 /     USB Serial Device
                              CP210x USB to UART
                              Bridge

The updated logic will only set the DTR/RTS signals for boards that do not
use standard Microsoft drivers (based on the manufacturer).  It would also
be possible to check against a list of known driver manufactures (like
wch.cn or Silicon Labs) but this would require a list of known drivers for
all ports.

Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2023-04-04 13:30:05 +10:00
Damien George
b5ceb9d577 tools/pyboard.py: Fix joining of path in filesystem_command.
This was broken by 5327cd1021

Signed-off-by: Damien George <damien@micropython.org>
2023-03-22 15:19:09 +11:00
Christian Clauss
4376c969f6 all: Fix Python comparison to None and True, and use "not in".
These are basic PEP8 recommendations.
2023-03-10 13:32:24 +11:00
Damien George
5327cd1021 tools/pyboard.py: Use '/' exclusively when dealing with paths.
Currently, certain mpremote filesystem operations can fail on Windows due
to a mixing of '/' and '\' for path separators.  Eg if filesystem_command()
is called with a destination that ends in / then dest.endswith(os.path.sep)
will return False, which gives the wrong behaviour (it does end in a path
separator).

For similar reasons to 7e9a15966a, it's best
to use '/' everywhere in pyboard.py and mpremote, because the target device
understands only '/'.  mpremote already does this, so the remaining place
to fix it is in pyboard.y, to convert all incoming paths to use '/' instead
of '\'.

This effectively reverts 57fd66b80f which
tried to fix the problem in a different way.

See also related 1f84440538.

Signed-off-by: Damien George <damien@micropython.org>
2023-02-24 13:08:34 +11:00
Jim Mussared
ce9f7cd00a tools/mpremote: Use hatch to build mpremote package.
This allows the entire configuration to be defined in a single file,
including the logic for including pyboard.py and automatically versioning
based on the git tag.

Building the package works both via `python -m build` as well as
`hatch build`.  `python -m build ` has the advantage of automatically
fetching all dependencies, you don't need to manually install any hatch
packages.

In order to make the versioning work, and also keep things simpler for end
users, mpremote releases will now be the same as MicroPython releases and
use the same tag.  The version strings for mpremote will look like:
- X.Y.Z -- clean build at the tag
- X.Y.Z.postN+gHASH -- clean build, N revisions from the most recent tag
- X.Y.Z.postN+gHASH.dYYYYMMDD -- dirty build, N revisions from out

This commit extends on the idea from #8404.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-02-24 11:40:22 +11:00
Damien George
e0d1127246 tools/ci.sh: Add unix, stm32 and rp2 ports to code size check.
Now that the code-size-check CI action gives a nice report (as a comment)
on the code size difference, it's possible to have a few more ports
reported there.  In this commit, unix, stm32 and rp2 are added.  Unix
represents non-MCU builds, and stm32 and rp2 represent ARM-based builds,
for ports that have lots of features enabled.

Signed-off-by: Damien George <damien@micropython.org>
2023-02-20 11:26:23 +11:00
Damien George
4bd4b6a47e tools/ci.sh: Install pkg-config for macos builds.
Signed-off-by: Damien George <damien@micropython.org>
2023-02-17 13:13:57 +11:00
Martin Milata
850f09b109 tools/mpy-tool.py: Initialize line_info_top.
Without it the line number mapping doesn't work.

Signed-off-by: Martin Milata <martin@martinmilata.cz>
2023-02-01 13:17:22 +11:00
Jos Verlinde
d263438a6e tools/pyboard.py: Set DTR on Windows to avoid ESPxx hard reset.
Fixes issue #9659.

Signed-off-by: Jos Verlinde <Jos.Verlinde@Microsoft.com>
2023-01-13 16:51:31 +11:00
Jim Mussared
aa64280666 tools/pyboard.py: Add fs_{listdir,readfile,writefile,stat}.
These are for working with the filesystem when using pyboard.py as a
library, rather than at the command line.

- fs_listdir returns a list of tuples, in the same format as os.ilistdir().
- fs_readfile returns the contents of a file as a bytes object.
- fs_writefile allows writing a bytes object to a file.
- fs_stat returns an os.statresult.

All raise FileNotFoundError (or OSError(ENOENT) on Python 2) if the file is
not found (or PyboardError on other errors).

Updated fs_cp and fs_get to use fs_stat to compute file size.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-01-13 16:38:34 +11:00
Jim Mussared
6013d27dd5 tools/pyboard.py: Add parse kwarg to eval.
This is useful when using pyboard.py as a library rather than at the
command line.

    pyb.eval("1+1") --> b"2"
    pyb.eval("{'a': '\x00'}") --> b"{'a': '\\x00'}"

Now you can also do

    pyb.eval("1+1", parse=True) --> 2
    pyb.eval("{'a': '\x00'}", parse=True) --> {'a': '\x00'}

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-01-13 16:38:34 +11:00
Jim Mussared
77002a92bf tools/pyboard.py: Fix Python 2 compatibility.
In Python 2, serial.read()[0] a string, not int. Use struct.unpack to do
this instead.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-01-13 16:38:34 +11:00
robert-hh
57bb1e0474 tools/autobuild: Create .uf2 file for seeed_xiao_nrf52.
And for all other nrf boards that have or get a uf2 build tag.
2022-12-19 16:16:12 +11:00
stijn
9c7ff87643 all: Keep msvc build output in build/ directories.
This follow the change made for Makefile-based projects in b2e82402.
2022-12-13 17:18:53 +11:00
Michael Mogenson
921f397acb tools/mpremote: Only auto connect to serial device with USB VID/PID.
On MacOS and Windows there are a few default serial devices that are
returned by `serial.tools.list_ports.comports()`. For example on MacOS:

```
{'description': 'n/a',
 'device': '/dev/cu.Bluetooth-Incoming-Port',
 'hwid': 'n/a',
 'interface': None,
 'location': None,
 'manufacturer': None,
 'name': 'cu.Bluetooth-Incoming-Port',
 'pid': None,
 'product': None,
 'serial_number': None,
 'vid': None}

{'description': 'n/a',
 'device': '/dev/cu.wlan-debug',
 'hwid': 'n/a',
 'interface': None,
 'location': None,
 'manufacturer': None,
 'name': 'cu.wlan-debug',
 'pid': None,
 'product': None,
 'serial_number': None,
 'vid': None}
```

Users of mpremote most likely do not want to connect to these ports. It
would be desirable if mpremote did not select this ports when using the
auto connect behavior. These serial ports do not have USB VID or PID
values and serial ports for Micropython boards with FTDI/serial-to-USB
adapter or native USB CDC/ACM support do.

Check for the presence of a USB VID / PID int value when selecting a
serial port to auto connect to. All serial ports will still be listed by
the `list` command and can still be selected by name when connecting.

Signed-off-by: Michael Mogenson <michael.mogenson@gmail.com>
2022-11-25 17:20:14 -05:00
Tobias Thyrrestrup
2fcd93cdd0 tools/mpremote: Allow EDITOR environment variable to work on Windows. 2022-11-25 15:02:52 +11:00
Damien George
d75f49c0f0 tools/ci.sh: Don't print python2 version.
Because python2 may not be available.

Signed-off-by: Damien George <damien@micropython.org>
2022-11-23 10:32:55 +11:00
Damien George
3927ce6035 tools/ci.sh: Remove one build from zephyr CI to speed it up.
The zephyr CI takes the most time out of all CI jobs, so remove the
standard qemu_x86 build to speed it up.  The remaining builds should still
cover enough cases to catch errors.

Signed-off-by: Damien George <damien@micropython.org>
2022-11-16 10:05:17 +11:00
iabdalkader
3bae10417e tools/ci.sh: Add ARDUINO_NANO_RP2040_CONNECT to rp2 CI build. 2022-11-16 09:47:06 +11:00
Jim Mussared
1504204606 tools/gendoc.py: Remove unused script.
This used to be used to generate .rst docs from inline comments in the C
code (specifically for APIs) but is now unused.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-11-15 23:17:26 +11:00
Andrew Scheller
02ad71468f tools/gen-cpydiff.py: Use os.path.join and os.path.isdir.
Makes path handling clearer and simpler.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-11-15 23:17:26 +11:00
Angus Gratton
2e6e53057b shared/tinyusb: Further refactor static USB device implementation.
App the mp_ prefix to usbd_ symbols and files which are defined here and
not in TinyUSB.

rp2 only for now. This includes some groundwork for dynamic USB devices
(defined in Python).

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2022-11-11 16:47:36 +11:00
Damien George
b52fe52d31 tools/makemanifest.py: Fix printing of mpy_cross.CrossCompileError.
This change was missed by 5852fd7708

Signed-off-by: Damien George <damien@micropython.org>
2022-11-02 14:06:18 +11:00
Damien George
3ed017677b tools/mpremote: Make RemoteFile objects iterable.
So that filesystems mounted with "mpremote mount" can have their files
iterated over, making them consistent with other files.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-31 12:40:17 +11:00
Jim Mussared
3427e12e8f tools/mpremote: Handle FileNotFound exceptions in listdir.
The except handler for OSError didn't include the line that actually calls
os.listdir, so an invalid path wasn't handled correctly.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-31 12:38:25 +11:00
Jim Mussared
1ba0e8ff96 py/persistentcode: Only emit sub-version if generated code has native.
In order for v1.19.1 to load a .mpy, the formerly-feature-flags which are
now used for the sub-version must be zero.

The sub-version is only used to indicate a native version change, so it
should be zero when emitting bytecode-only .mpy files.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-25 14:57:04 +11:00
Jim Mussared
87011f6353 extmod/extmod.mk: Make extmod.mk handle GIT_SUBMODULES.
This applies to nimble, btstack, axtls, mbedtls, lwip.

Rather than having the ports individually manage GIT_SUBMODULES for these
components, make extmod.mk append them when the relevant feature is
enabled.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-11 23:29:09 +11:00
Jim Mussared
b8982ec5f9 tools/verifygitlog.py: Add additional help for subject line issues.
This check used to just show the regular expression that failed to match,
but the rules are pretty subtle and hard to interpret from the regular
expression alone.

Add some basic checks for the main things that go wrong:
 - Missing capitalisation.
 - Missing full-stop.
 - Missing path.
 - Single-word subject.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-07 00:26:31 +11:00
Jim Mussared
f6d06b3ce0 tools/verifygitlog.py: Ignore comment lines in commit messages.
The "signed-off" check assumes that the Signed-off-by: line is the last,
but there may me many lines of comments after this.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-07 00:19:40 +11:00
Damien George
46d02c2469 tools/mpremote: Bump version to 0.4.0.
Signed-off-by: Damien George <damien@micropython.org>
2022-10-06 01:02:39 +11:00
Jim Mussared
f13134e403 tools/mpremote: Fix argument handling for follow and add help strings.
Fixes in this commit are:
- Make --follow the default for "run" (accidentally changed in 68d094358).
- Add help strings for "repl": --capture --inject-file --inject-code
- Update help strings for "run".
- Fix encoding for --inject-code (accidentally broken in 68d094358).
- Remove ability to --no-follow for "eval". It was there previously because
  it shared the same code path with "exec" and "run", but makes no sense
  for "eval", so might as well remove.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-06 00:58:31 +11:00
Angus Gratton
0e35c4de9b tools: Add pre-commit support.
Tweak the existing codeformat.py and verifygitlog.py to allow them to be
easily called by pre-commit.

(This turned out to be easier than using any existing pre-commit hooks,
without making subtle changes in the formatting.)

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2022-10-04 14:52:48 +11:00
Angus Gratton
bdac8272d8 tools: Add note about uncrustify versions.
Uncrustify versions are not mutually compatible:

1. Version 0.73 or newer produce slightly different formatting. It may be
possible to tweak these by adding more config items, but this will cause
older versions to error out with 'Unknown option'.

2. Version 0.75 prints a range of deprecation warnings due to config file
changes, and returns a non-zero exit code. These are actually fixable
as most are the default value, and pp_indent has changed from 'true' to '1'
which is backwards compatible. However issue 1 remains, so probably better
to have it fail explicitly.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2022-10-04 14:34:54 +11:00
Jim Mussared
7705b9b9d5 tools/pyboard.py: Handle unsupported fs command.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-01 22:44:24 +10:00
Jim Mussared
12ca918eb2 tools/mpremote: Add mpremote mip install to install packages.
This supports the same package sources as the new `mip` tool.
 - micropython-lib (by name)
 - http(s) & github packages with json description
 - directly downloading a .py/.mpy file

The version is specified with an optional `@version` on the end of the
package name. The target dir, index, and mpy/no-mpy can be set through
command line args.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-01 22:44:24 +10:00
Jim Mussared
68d094358e tools/mpremote: Use argparse for command line parsing.
No functional change other than to allow slightly more flexibility in how
--foo arguments are specified.

This removes all custom handling for --foo args in all commands and
replaces it with per-command argparse configs.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-01 22:44:24 +10:00
Jim Mussared
413a69b94b tools/mpremote: Simplify dispatch of commands.
No functional change.

This makes each built-in command defined by just a handler method and
simplifies a lot of the logic around tracking the board state.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-01 22:44:24 +10:00
Jim Mussared
282401da5c tools/manifestfile.py: Replace recursive glob with os.walk.
Recursive glob isn't supported before Python 3.5.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-30 17:34:03 +10:00
Jim Mussared
924a3e03ec top: Replace upip with mip everywhere.
Updates all README.md and docs, and manifests to `require("mip")`.

Also extend and improve the documentation on freezing and packaging.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-30 17:34:03 +10:00
Jim Mussared
65ab0ec91c tools/manifestfile.py: Add author kwarg to metadata().
This allows future micropython-lib packages to specify an author.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-29 23:53:35 +10:00
Jim Mussared
920da9c5e3 unix/variants/coverage: Add test for manifest freeze_mpy().
This uses the frozentest.mpy that is also used by ports/minimal.

Also fixes two bugs that these new tests picked up:
 - File extension matching in manifestfile.py.
 - Handling of freeze_mpy results in makemanifest.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 23:51:10 +10:00
Jim Mussared
6ecdf1a240 tests/frozen: Move frozentest.mpy from ports/ to tests/.
frozentest.mpy was previously duplicated in ports/minimal and
ports/powerpc.

This needs to be re-generated on every .mpy version increase, so might as
well just have a single copy of it.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 23:51:10 +10:00
Jim Mussared
d94141e147 py/persistentcode: Introduce .mpy sub-version.
The intent is to allow us to make breaking changes to the native ABI (e.g.
changes to dynruntime.h) without needing the bytecode version to increment.

With this commit the two bits previously used for the feature flags (but
now unused as of .mpy version 6) encode a sub-version.  A bytecode-only
.mpy file can be loaded as long as MPY_VERSION matches, but a native .mpy
(i.e. one with an arch set) must also match MPY_SUB_VERSION.  This allows 3
additional updates to the native ABI per bytecode revision.

The sub-version is set to 1 because the previous commits that changed the
layout of mp_obj_type_t have changed the native ABI.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
2022-09-19 23:19:55 +10:00
Jim Mussared
c1530a0ce8 unix: Refactor mpconfigport.h and mpconfigvariant.h.
This is a no-op for coverage and minimal.

The standard and dev variants have been merged and enable the same feature
set as a typical bare-metal board.  And remove the CI for the dev build.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:39:03 +10:00
Wind-stormger
57fd66b80f tools/pyboard.py: Support Windows pathname separators.
Addresses issue #9132.
2022-09-13 13:28:39 +10:00
Andrew Leech
4e0964b59f extmod/vfs: Add finaliser to ilistdir to close directory handle.
When iterating over filesystem/folders with os.iterdir(), an open file
(directory) handle is used internally.  Currently this file handle is only
closed once the iterator is completely drained, eg. once all entries have
been looped over / converted into list etc.

If a program opens an iterdir but does not loop over it, or starts to loop
over the iterator but breaks out of the loop, then the handle never gets
closed.  In this state, when the iter object is cleaned up by the garbage
collector this open handle can cause corruption of the filesystem.

Fixes issues #6568 and #8506.
2022-09-13 13:00:42 +10:00
Jim Mussared
4903e48e34 tools/makemanifest.py: Force the repo version of the mpy_cross package.
In case the version from pypi is installed or some other version is
available in sys.path, prepend `$(TOP)/mpy-cross` to sys.path instead.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-09 18:12:55 +10:00
Jim Mussared
9070a24940 tools/autobuild: Use distinct directory for building stm32 variants.
Previous the build directory just used the board name, now make it use the
variant name too.

This shouldn't have any change because the existing directory should not
exist (all builds run by these scripts remove their build directory after
completion), but it makes debugging easier.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-08 14:54:11 +10:00
Jim Mussared
5852fd7708 tools/manifestfile.py: Allow manifests to set metadata.
The metadata can be version, description, and license.

After executing a manifest, the top-level metadata can be queried, and also
each file output from the manifest will have the metadata of the
containing manifest.

Use the version metadata to "tag" files before freezing such that they have
__version__ available.
2022-09-05 17:07:13 +10:00
Jim Mussared
bc23f207ce tools/manifestfile.py: Allow require() to specify unix packages.
By default, don't include micropython-lib/unix-ffi in the search.

If unix_ffi=True is passed to require(), then include unix-ffi and make it
take precedence over the other locations (e.g. python-stdlib).

This does two things:
 - Prevents non-unix builds from using unix-only packages.
 - Allows the unix build to optionally use a more full-featured (e.g. ffi)
   based package, even with the same name as one from e.g. stdlib.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-05 17:06:52 +10:00
Jim Mussared
e9a28ce312 tools/manifestfile.py: Allow include of directory path.
If an include path is a directory, then it implicitly grabs the manifest.py
file inside that directory. This simplifies most manifest.py files.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-05 17:06:52 +10:00
Jim Mussared
6bd0ec7a70 tools/makemanifest.py: Update to use mpy_cross module.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-05 17:06:43 +10:00
Jim Mussared
f3cdb052db tools/manifestfile.py: Add library for working with manifests.
This splits the manifest file loading logic from makemanifest.py and
updates makemanifest.py to use it.

This will allow non-freezing uses of manifests, such as defining packages
and dependencies in micropython-lib.

Also adds additional methods to the manifest "API":
 - require() - to get a package from micropython-lib.
 - module() - to define a single-file module
 - package() - to define a multi-file package

module() and package() should replace most uses of freeze() and can also
be also used in non-freezing scenarios.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-05 17:00:43 +10:00
Damien George
730e975091 esp32/boards: Merge manifest_release modules into standard manifest.
Having two separate manifests is confusing.  It's simpler to have the daily
builds use the same configuration as the stable, release builds.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-30 11:34:34 +10:00
Damien George
2e386bcf76 tools/mpremote: Print nicer errors for unsupported 'cp -r' arguments.
Also document support for 'cp :a :b'.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-26 13:30:03 +10:00
Damien George
f5fedf4676 tools/pyboard.py: Add fs_cp function for direct device-to-device copy.
Signed-off-by: Damien George <damien@micropython.org>
2022-08-26 13:30:03 +10:00
Damien George
24f1161fe2 tools/pyboard.py: Remove implicit fs_put if source starts with ./.
Signed-off-by: Damien George <damien@micropython.org>
2022-08-26 13:30:03 +10:00
Damien George
858707181d tools/mpremote: Print a nicer error when a filesystem command fails.
Signed-off-by: Damien George <damien@micropython.org>
2022-08-26 13:30:03 +10:00
Damien George
1855df6361 stm32: Remove support for CC3000 WiFi driver.
It has been about 8 years since support for this chip was added.  Reasons
to remove it are:
- It is no longer easy to obtain this part.
- There are now many other options for WiFi.
- It's not a good use of developer time to maintain it.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-26 12:51:37 +10:00
Jim Mussared
492ba5eaf2 stm32: Move board variant config to mpconfigboard.mk.
Rather than having the autobuild know about the particular variants, have
the mpconfigboard.mk describe them and make autobuild discover them
automatically.

Adds a "query-variants" target to stm32/Makefile to allow the set of
possible variants to be queried.

Removes pybv3 from the autobuild as this isn't use by the downloads page.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-26 09:38:38 +10:00
robert-hh
f6ec01d1da tools/ci.sh: Split the stm32 builds for wiznet5k and cc3k.
- Add lib/wiznet5k into the 'make submodules' step.
- Split the stm32 builds for wiznet5k and cc3k.
- Run 'make .... clean' after making the wiznet5k build.
2022-08-23 15:00:00 +10:00
Nicholas H.Tollervey
af54d2ce9f
javascript: Rename this port to 'webassembly'. 2022-08-22 12:03:39 +01:00
Jim Mussared
59e3348c10 tools/mpremote: Add "edit" command.
This allows a remote file to be edited locally by copying it over, running
the local editor, then copying it back.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-18 22:29:13 +10:00
Jim Mussared
263737ecfe tools/pyboard.py: Add "touch" filesystem command.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-18 22:24:25 +10:00
Jim Mussared
a311e9e3d4 tools/mpremote: Allow + terminator for fs commands.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-18 22:22:36 +10:00
Damien George
cbc9f944c4 tests,tools: Update path to unix micropython executable.
These were missed by 47c84286e8

Signed-off-by: Damien George <damien@micropython.org>
2022-08-18 11:47:58 +10:00
Damien George
d53c3b6ade unix/variants: Remove variant suffix from executable filename.
The executable now lives in the build directory, and since the build
directory already contains the variant name there is no need to also add
it to the executable.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-11 13:34:34 +10:00
Daniel Jour
409995ac68 tools/ci.sh: Force mpy-cross build for samd and teensy. 2022-08-11 13:33:51 +10:00
Daniel Jour
47c84286e8 all: Fix paths to mpy-cross and micropython binaries.
Binaries built using the Make build system now no longer appear in the
working directory of the build, but rather in the build directory.  Thus
some paths had to be adjusted.
2022-08-11 13:31:13 +10:00
Jim Mussared
58bed5ec14 tools/ci.sh: Initialise submodules for more ports.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-03 00:08:28 +10:00
Dan Ellis
f9cbe6bc47 py/formatfloat: Format all whole-number floats exactly.
Formerly, py/formatfloat would print whole numbers inaccurately with
nonzero digits beyond the decimal place.  This resulted from its strategy
of successive scaling of the argument by 0.1 which cannot be exactly
represented in floating point.  The change in this commit avoids scaling
until the value is smaller than 1, so all whole numbers print with zero
fractional part.

Fixes issue #4212.

Signed-off-by: Dan Ellis dan.ellis@gmail.com
2022-07-26 22:23:47 +10:00
iabdalkader
2076f2efcc tools/autobuild: Add nrf port to autobuild scripts. 2022-07-13 16:19:22 +10:00
Damien George
80bc9b3dee tools/mpremote: Don't be verbose when using cat command.
Fixes issue #8828.

Signed-off-by: Damien George <damien@micropython.org>
2022-07-08 22:28:26 +10:00
Damien George
7e5137e0ae tools/pyboard.py: Add verbose option to filesystem_command.
Signed-off-by: Damien George <damien@micropython.org>
2022-07-08 22:26:41 +10:00
David Lechner
17f0297c1a tools/ci.sh: Drop ppa requirement for code formatting CI.
The CI scripts were using a PPA to get a backported version of uncrustify
on Ubuntu 20.04.  However, this causes CI to intermittently fail due to
connection issues to launchpad.net or the key server.

Ubuntu 22.04 has a newer version of uncrustify removing the need for the
PPA.  Ubuntu 22.04 is now in beta on GitHub actions, so it can be used.

Signed-off-by: David Lechner <david@pybricks.com>
2022-06-17 16:56:41 +10:00
Maureen Helm
bada8b3cad zephyr: Upgrade to Zephyr v3.1.0.
Updates the Zephyr port build instructions and CI to use the latest Zephyr
release tag.

Tested on frdm_k64f.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
2022-06-17 16:51:34 +10:00
Damien George
dea1ba3309 tools/metrics.py: Add mimxrt and renesas-ra to list of available ports.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-17 14:57:58 +10:00
Damien George
1f1afae622 examples/natmod/features3: Add example to test more natmod features.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-10 16:42:43 +10:00
Damien George
17a0d65ee4 tools/mpy_ld.py: Support GOT entries that reference inside mp_fun_table.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-10 16:42:43 +10:00
Damien George
313f082896 tools/ci.sh: Build native .mpy examples for armv7emsp, xtensawin in CI.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-09 13:24:08 +10:00
Damien George
abb3850398 tools/mpy_ld.py: Support R_XTENSA_PDIFF32 relocation.
Newer versions of the ESP-IDF's toolchain use this relocation.

Fixes issue #8436.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-09 11:57:22 +10:00
Damien George
6a1dbaad1c tools/ci.sh: Build NUCLEO_H743ZI with -O2 optimisation level.
This tests the build when -O2 is used, which can lead to additional
compiler analysis and warnings.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-08 15:00:59 +10:00
Damien George
5290bfaefe tools/mpremote: Bump version to 0.3.0.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 23:23:14 +10:00
Damien George
646fcdadbf tools/mpremote: Add command to print the version.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 23:22:04 +10:00
Damien George
2fb413b265 tools/mpy-tool.py: Improve generated frozen identifiers.
Frozen identifiers now include their full name hierarchy, eg their class
name.  This makes it easier to understand the generated code.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 16:55:18 +10:00
Damien George
9670a156da all: Rename MICROPY_PY_WIZNET5K to MICROPY_PY_NETWORK_WIZNET5K.
To match MICROPY_PY_NETWORK_CYW43 and MICROPY_PY_NETWORK_NINAW10.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 16:55:18 +10:00
Damien George
bf92b0cbf2 tools/ci.sh: Add test for mpy-tool's merging feature.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 13:51:45 +10:00
Damien George
599a22e569 tools/mpy-tool.py: Rework .mpy merging feature.
Now that the native qstr link table is gone, merging a native .mpy file
with a bytecode .mpy file is not as simple as concatenating the .mpy data.
The qstr_table and obj_table tables from all merged .mpy files must now be
joined together, because they are global to the .mpy file (and hence global
to the merged .mpy file).  This means the bytecode needs to be be decoded,
qstr_table and obj_table indices updated to point to the correct entries in
the new tables, and then the bytecode re-encoded.

This commit makes this change to the merging feature in mpy-tool.py.  This
can now merge an arbitrary number of bytecode .mpy files, and up to one
native .mpy file.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 13:51:45 +10:00
Damien George
f506bf342a py/bc: Remove unused mp_opcode_format function.
This was made redundant by f2040bfc7e, which
also did not update this function for the change to qstr-opcode encoding,
so it does not work correctly anyway.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 13:32:38 +10:00
Damien George
b37b578214 py/persistentcode: Remove remaining native qstr linking support.
Support for architecture-specific qstr linking was removed in
d4d53e9e11, where native code was changed to
access qstr values via qstr_table.  The only remaining use for the special
qstr link table in persistentcode.c is to support native module written in
C, linked via mpy_ld.py.  But native modules can also use the standard
module-level qstr_table (and obj_table) which was introduced in the .mpy
file reworking in f2040bfc7e.

This commit removes the remaining native qstr liking support in
persistentcode.c's load_raw_code function, and adds two new relocation
options for constants.qstr_table and constants.obj_table.  mpy_ld.py is
updated to use these relocations options instead of the native qstr link
table.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 13:19:55 +10:00
Andrew Leech
b92e51efe7 tools/ci.sh: Add build of W5100S_EVB_PICO board to rp2 CI.
This exercises the inclusion of a number of libraries in the rp2 port
including mbedtls and lwip.

Signed-off-by: Andrew Leech <andrew@alelec.net>
2022-06-03 14:35:52 +10:00
Damien George
1d047617bb tools/mpy-tool.py: Remove obsolete unicode flag in .mpy header.
This was removed in c49d5207e9

Signed-off-by: Damien George <damien@micropython.org>
2022-05-26 11:43:46 +10:00
David Lechner
d42d35f56d tests/run-tests.py: Enable -X realtime option for macOS tests.
This enables the new `-X realtime` runtime option when running tests on
macOS.  This causes MicroPython to configure all threads to be high
priority so that they are allowed to use high precision timers.  This
makes tests that depend on the passage of time more likely to succeed.

CI tests that were disabled because of this are now enabled again.

Signed-off-by: David Lechner <david@pybricks.com>
2022-05-24 00:52:44 +10:00