micropython/tools/autobuild/remove_old_firmware.py
Jim Mussared 69e34b6b6b all: Switch to new preview build versioning scheme.
See https://github.com/micropython/micropython/issues/12127 for details.

Previously at the point when a release is made, we update mpconfig.h
and set a git tag. i.e. the version increments at the release.

Now the version increments immediately after the release. The workflow is:
1. Final commit in the cycle updates mpconfig.h to set (X, Y, 0, 0) (i.e.
   clear the pre-release state).
2. This commit is tagged "vX.Y.0".
3. First commit for the new cycle updates mpconfig.h to set (X, Y+1, 0, 1)
   (i.e. increment the minor version, set the pre-release state).
4. This commit is tagged "vX.Y+1.0-preview".

The idea is that a nightly build is actually a "preview" of the _next_
release. i.e. any documentation describing the current release may not
actually match the nightly build. So we use "preview" as our semver
pre-release identifier.

Changes in this commit:
 - Add MICROPY_VERSION_PRERELEASE to mpconfig.h to allow indicating that
   this is not a release version.
 - Remove unused MICROPY_VERSION integer.
 - Append "-preview" to MICROPY_VERSION_STRING when the pre-release state
   is set.
 - Update py/makeversionhdr.py to no longer generate MICROPY_GIT_HASH.
 - Remove the one place MICROPY_GIT_HASH was used (it can use
   MICROPY_GIT_TAG instead).
 - Update py/makeversionhdr.py to also understand
   MICROPY_VERSION_PRERELEASE in mpconfig.h.
 - Update py/makeversionhdr.py to convert the git-describe output into
   semver-compatible "X.Y.Z-preview.N.gHASH".
 - Update autobuild.sh to generate filenames using the new scheme.
 - Update remove_old_firmware.py to match new scheme.
 - Update mpremote's pyproject.toml to handle the "-preview" suffix in the
   tag. setuptools_scm maps to this "rc0" to match PEP440.
 - Fix docs heading where it incorrectly said "vvX.Y.Z" for release docs.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-06 12:10:14 +11:00

75 lines
2.1 KiB
Python
Executable File

#!/usr/bin/env python3
import re, subprocess, sys
DEBUG = False
DRY_RUN = False
NUM_KEEP_PER_BOARD = 4
def main():
ssh_machine = sys.argv[1]
ssh_firmware_dir = sys.argv[2]
# SSH to get list of existing files.
p = subprocess.run(
["ssh", ssh_machine, "find", ssh_firmware_dir, "-name", "\\*-preview.\\*"],
capture_output=True,
)
if p.returncode != 0:
print(p.stderr)
return
all_files = p.stdout.split(b"\n")
# Parse all files to organise into boards/date/version.
boards = {}
for file in all_files:
m = re.match(
rb"([a-z/.]+)/([A-Za-z0-9_-]+)-(20[0-9]{6})-(v[0-9.]+)-preview.([0-9]+).g[0-9a-f]+.",
file,
)
if not m:
continue
dir, board, date, version, ncommits = m.groups()
if board not in boards:
boards[board] = {}
if (date, version, ncommits) not in boards[board]:
boards[board][(date, version, ncommits)] = []
boards[board][(date, version, ncommits)].append(file)
# Collect files to remove based on date and version.
remove = []
for board in boards.values():
filelist = [
(date, version, ncommits, files) for (date, version, ncommits), files in board.items()
]
filelist.sort(reverse=True)
keep = []
for date, version, ncommits, files in filelist:
if keep and (version, ncommits) == keep[-1]:
remove.extend(files)
elif len(keep) >= NUM_KEEP_PER_BOARD:
remove.extend(files)
else:
keep.append((version, ncommits))
if DEBUG:
all_files.sort(reverse=True)
for file in all_files:
print(file, file in remove)
print(len(remove), "/", len(all_files))
# Do removal of files.
for file in remove:
file = str(file, "ascii")
print("remove:", file)
if not DRY_RUN:
p = subprocess.run(["ssh", ssh_machine, "/bin/rm", file], capture_output=True)
if p.returncode != 0:
print(p.stderr)
if __name__ == "__main__":
main()