Commit Graph

57 Commits

Author SHA1 Message Date
Damien George
7ad04d17da py/mkrules.mk: Regenerate all qstrs when config files change.
A port can define QSTR_GLOBAL_DEPENDENCIES to add extra files.
2018-06-12 13:53:43 +10:00
Damien George
ef12a4bd05 py: Refactor how native emitter code is compiled with a file per arch.
Instead of emitnative.c having configuration code for each supported
architecture, and then compiling this file multiple times with different
macros defined, this patch adds a file per architecture with the necessary
code to configure the native emitter.  These files then #include the
emitnative.c file.

This simplifies emitnative.c (which is already very large), and simplifies
the build system because emitnative.c no longer needs special handling for
compilation and qstr extraction.
2018-04-10 15:06:47 +10:00
Damien George
6af4515969 py: Use "GEN" consistently for describing files generated in the build. 2018-02-22 12:48:51 +11:00
Paul Sokolovsky
d21d029d55 py/mkrules.mk: Add "clean-frozen" target to clean frozen script/modules dir.
This target removes any stray files (i.e. something not committed to git)
from scripts/ and modules/ dirs (or whatever FROZEN_DIR and FROZEN_MPY_DIR
is set to).

The expected workflow is:

1. make clean-frozen
2. micropython -m upip -p modules <packages_to_freeze>
3. make

As it can be expected that people may drop random thing in those dirs which
they can miss later, the content is actually backed up before cleaning.
2017-12-10 01:05:29 +02:00
Christopher Arndt
1871a924c9 py/mkenv.mk: Use $(PYTHON) consistently when calling Python tools.
Rationale:

* Calling Python build tool scripts from makefiles should be done
  consistently using `python </path/to/script>`, instead of relying on the
  correct she-bang line in the script [1] and the executable bit on the
  script being set. This is more platform-independent.
* The name/path of the Python executable should always be used via the
  makefile variable `PYTHON` set in `py/mkenv.mk`. This way it can be
  easily overwritten by the user with `make PYTHON=/path/to/my/python`.
* The Python executable name should be part of the value of the makefile
  variable, which stands for the build tool command (e.g. `MAKE_FROZEN` and
  `MPY_TOOL`), not part of the command line where it is used. If a Python
  tool is substituted by another (non-python) program, no change to the
  Makefiles is necessary, except in `py/mkenv.mk`.
* This also solves #3369 and #1616.

[1] There are systems, where even the assumption that `/usr/bin/env` always
exists, doesn't hold true, for example on Android (where otherwise the unix
port compiles perfectly well).
2017-11-15 11:56:58 +11:00
Damien George
02b4b23319 Revert "py/{mkenv.mk,mkrules.mk}: Append .exe for Windows executable files."
This reverts commit 3289b9b7a7.
The commit broke building on MINGW because the filename became
micropython.exe.exe.  A proper solution to support more Windows build
environments requires more thought and testing.
2017-10-31 22:01:56 +11:00
chrismas9
3289b9b7a7 py/{mkenv.mk,mkrules.mk}: Append .exe for Windows executable files.
Building mpy-cross: this patch adds .exe to the PROG name when building
executables for host (eg mpy-cross) on Windows.  make clean now removes
mpy-cross.exe under Windows.

Building MicroPython: this patch sets MPY_CROSS to mpy-cross.exe or
mpy-cross so they can coexist and use cygwin or WSL without rebuilding
mpy-cross.  The dependency in the mpy rule now uses mpy-cross.exe for
Windows and mpy-cross for Linux.
2017-10-04 00:21:05 +11:00
Damien George
4a93801c12 all: Update Makefiles and others to build with new ports/ dir layout.
Also renames "stmhal" to "stm32" in documentation and everywhere else.
2017-09-06 14:09:13 +10:00
Damien George
b16a755a0b py/mkrules.mk: Use "find -path" when searching for frozen obj files.
This allows the command to succeed without error even if there is no
$(BUILD)/build directory, which is the case for mpy-cross.
2017-08-21 20:32:30 +10:00
Paul Sokolovsky
4dc7c5649b py/mkrules.mk: Show frozen modules sizes together with executable size.
This works for Unix and similar ports so far.
2017-08-06 15:43:21 +03:00
Alexander Steffen
55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Damien George
761e4c7ff6 all: Remove trailing spaces, per coding conventions. 2017-07-19 13:12:10 +10:00
Damien George
85f7b0b468 py/mkrules.mk: Fix auto-qstr generation when "make -B" is used.
For make v3.81, using "make -B" can set $? to empty and in this case the
auto-qstr generation needs to pass all args (ie $^) to cpp.  The previous
fix for this (which was removed in 23a693ec2d)
used if statements in the shell command, which gave very long lines that
didn't work on certain systems (eg cygwin).

The fix in this patch is to use an $if(...) expression, which will evaluate
to $? (only newer prerequisites) if it's non empty, otherwise it will use
$^ (all prerequisites).
2017-05-26 13:12:42 +10:00
Damien George
a1f254028d py/mkrules.mk: Add dependency of .mpy files upon mpy-cross.
This ensures that mpy-cross is automatically built (and is up-to-date) for
ports that use frozen bytecode.  It also makes sure that .mpy files are
re-built if mpy-cross is changed.
2017-05-11 23:40:16 +10:00
Paul Sokolovsky
29dd92c82a zephyr/Makefile: Rework to use modern, official build integration.
Build happens in 3 stages:

1. Zephyr config header and make vars are generated from prj.conf.
2. libmicropython is built using them.
3. Zephyr is built and final link happens.
2017-03-30 00:08:27 +03:00
Damien George
23a693ec2d py/mkrules.mk: Remove special check for "-B" in qstr auto generation.
When make is passed "-B" it seems that everything is considered out-of-date
and so $? expands to all prerequisites.  Thus there is no need for a
special check to see if $? is emtpy.
2017-03-15 21:50:48 +11:00
Damien George
343b4189b0 py/mkrules.mk: Add MPY_CROSS_FLAGS option to pass flags to mpy-cross.
So that ports can pass their own custom options to mpy-cross.
2017-01-05 15:51:36 +11:00
Damien George
8e5aced1fd py: Integrate Xtensa assembler into native emitter.
The config option MICROPY_EMIT_XTENSA can now be enabled to target the
Xtensa architecture with @micropython.native and @micropython.viper
decorators.
2016-12-09 16:51:49 +11:00
Damien George
659b06b250 py/*.mk: Replace uses of 'sed' with $(SED). 2016-11-15 16:09:43 +11:00
Dave Hylands
0400fa45ba py/mkrules.mk: Rework find command so it works on OSX.
The Mac version of find doesn't support -printf, so this changes
things to use sed to strip off the leading path element instead.
2016-11-15 16:07:48 +11:00
Damien George
fcda6a2a78 py: Strip leading dirs from frozen mpy files, so any path can be used.
With this patch one can now do "make FROZEN_MPY_DIR=../../frozen" to
specify a directory containing scripts to be frozen (as well as absolute
paths).

The compiled .mpy files are now stored in $(BUILD)/frozen_mpy/.
2016-11-08 14:36:06 +11:00
Damien George
bdf33bc136 py: Move frozen bytecode Makefile rules from ports to common mk files.
Now, to use frozen bytecode all a port needs to do is define
FROZEN_MPY_DIR to the directory containing the .py files to freeze, and
define MICROPY_MODULE_FROZEN_MPY and MICROPY_QSTR_EXTRA_POOL.
2016-11-08 14:28:30 +11:00
Paul Sokolovsky
f28efa1971 py: Move frozen modules rules from esp8266 port for reuse across ports.
A port now just needs to define FROZEN_DIR var and add $(BUILD)/frozen.c
to SRC_C to support frozen modules.
2016-09-17 21:00:40 +03:00
Paul Sokolovsky
1f61fe07a2 py/mkrules.mk: Allow to override name of libmicropython.a
Or alternatively, refer to an exact library file, not just phony target
"lib".
2016-08-30 00:59:17 +03:00
Paul Sokolovsky
7fb31479bf py/mkrules.mk: Allow to add more items for "clean" target using CLEAN_EXTRA. 2016-08-04 23:46:08 +03:00
Paul Sokolovsky
f469c76442 py: Rename __QSTR_EXTRACT flag to NO_QSTR.
It has more usages than just qstr extraction, for example, embedding (where
people don't care about efficient predefined qstrs).
2016-06-16 01:42:48 +03:00
Paul Sokolovsky
230d5cda05 py/mkrules.mk: Define "lib" outside conditional block.
"lib" happened to be defined inside block conditional on $(PROG).
2016-06-16 01:11:24 +03:00
Paul Sokolovsky
51cee4495e py/mkrules.mk: Typo fixes in comments. 2016-04-26 12:39:28 +03:00
Damien George
51dca54cd0 py/mkrules.mk: Remove obsolete rules for auto qstr generation. 2016-04-22 11:36:19 +01:00
Paul Sokolovsky
1b60a6dc4e py: Divide "split" and "cat" phases of qstr extraction for better efficiency.
E.g. for stmhal, accumulated preprocessed output may grow large due to
bloated vendor headers, and then reprocessing tens of megabytes on each
build make take couple of seconds on fast hardware (=> potentially dozens
of seconds on slow hardware). So instead, split once after each change,
and only cat repetitively (guaranteed to be fast, as there're thousands
of lines involved at most).
2016-04-19 14:39:08 +03:00
Paul Sokolovsky
4494b521ea py/mkrules.mk: Fix Bashism. 2016-04-19 12:32:23 +03:00
Paul Sokolovsky
21ab304c41 py/mkrules.mk: Cleanup command passed to shell. 2016-04-19 12:28:30 +03:00
Paul Sokolovsky
0dc85c9f86 py/mkrules.mk: Try to detect and emulate make -B behavior for qstr extraction.
If make -B is run, the rule is run with $? empty. Extract fron all file in
this case. But this gets fragile, really "make clean" should be used instead
with such build complexity.
2016-04-19 12:18:46 +03:00
Paul Sokolovsky
c618f91e22 py: Rework QSTR extraction to work in simple and obvious way.
When there're C files to be (re)compiled, they're all passed first to
preprocessor. QSTR references are extracted from preprocessed output and
split per original C file. Then all available qstr files (including those
generated previously) are catenated together. Only if the resulting content
has changed, the output file is written (causing almost global rebuild
to pick up potentially renumbered qstr's). Otherwise, it's not updated
to not cause spurious rebuilds. Related make rules are split to minimize
amount of commands executed in the interim case (when some C files were
updated, but no qstrs were changed).
2016-04-19 11:37:56 +03:00
Jan Čapek
53e3770b15 py/mkrules.mk: Suppress line-no output from CPP for qstr auto-gen. 2016-04-16 13:19:35 +01:00
Jan Čapek
61b560f63f py/mkrules.mk: Add mpconfig[port].h dependency to qstr generating rule. 2016-04-16 13:18:51 +01:00
Jan Čapek
d76c65f599 py: Add rules for automated extraction of qstrs from sources.
- add template rule that converts a specified source file into a qstring file

- add special rule for generating a central header that contains all
  extracted/autogenerated strings - defined by QSTR_DEFS_COLLECTED
  variable. Each platform appends a list of sources that may contain
  qstrings into a new build variable: SRC_QSTR. Any autogenerated
  prerequisities are should be appened to SRC_QSTR_AUTO_DEPS variable.

- remove most qstrings from py/qstrdefs, keep only qstrings that
  contain special characters - these cannot be easily detected in the
  sources without additional annotations

- remove most manual qstrdefs, use qstrdef autogen for: py, cc3200,
  stmhal, teensy, unix, windows, pic16bit:

   - remove all micropython generic qstrdefs except for the special strings that contain special characters (e.g. /,+,<,> etc.)
   - remove all port specific qstrdefs except for special strings
   - append sources for qstr generation in platform makefiles (SRC_QSTR)
2016-04-16 13:18:09 +01:00
Paul Sokolovsky
645045a799 py/mkrules.mk: Don't pass COPT to linker.
Oftentimes, libc, libm, etc. don't come compiled with CPU compressed code
option (Thumb, MIPS16, etc.), but we may still want to use such compressed
code for MicroPython itself.
2015-12-10 00:49:25 +02:00
Paul Sokolovsky
0dbd928cee Makefiles: Remove duplicate object files when linking.
Scenario: module1 depends on some common file from lib/, so specifies it
in its SRC_MOD, and the same situation with module2, then common file
from lib/ eventually ends up listed twice in $(OBJ), which leads to link
errors.

Make is equipped to deal with such situation easily, quoting the manual:
"The value of $^ omits duplicate prerequisites, while $+ retains them and
preserves their order." So, just use $^ consistently in all link targets.
2015-10-24 15:46:53 +03:00
Paul Sokolovsky
408b74d74c py: Allow to to build MicroPython as a static library.
The whole current port gets slurped into a static lib named
"libmicropython.a". Maybe that's not ideal, but at least something
to start with.
2015-10-12 15:32:06 +03:00
Damien George
65dc960e3b unix-cpy: Remove unix-cpy. It's no longer needed.
unix-cpy was originally written to get semantic equivalent with CPython
without writing functional tests.  When writing the initial
implementation of uPy it was a long way between lexer and functional
tests, so the half-way test was to make sure that the bytecode was
correct.  The idea was that if the uPy bytecode matched CPython 1-1 then
uPy would be proper Python if the bytecodes acted correctly.  And having
matching bytecode meant that it was less likely to miss some deep
subtlety in the Python semantics that would require an architectural
change later on.

But that is all history and it no longer makes sense to retain the
ability to output CPython bytecode, because:

1. It outputs CPython 3.3 compatible bytecode.  CPython's bytecode
changes from version to version, and seems to have changed quite a bit
in 3.5.  There's no point in changing the bytecode output to match
CPython anymore.

2. uPy and CPy do different optimisations to the bytecode which makes it
harder to match.

3. The bytecode tests are not run.  They were never part of Travis and
are not run locally anymore.

4. The EMIT_CPYTHON option needs a lot of extra source code which adds
heaps of noise, especially in compile.c.

5. Now that there is an extensive test suite (which tests functionality)
there is no need to match the bytecode.  Some very subtle behaviour is
tested with the test suite and passing these tests is a much better
way to stay Python-language compliant, rather than trying to match
CPy bytecode.
2015-08-17 12:51:26 +01:00
Paul Sokolovsky
cd87d20f46 mkrules.mk: Add comment why dependency parsing regex was tweaked.
(Windows compatibility.)
2015-05-06 15:34:33 +03:00
Ari Suutari
c097ea5dd2 Adjust sed regex that processes dependency file from compiler
so that resulting file is correct also on windows systems (ie.
with file names containing drive letter).
2015-05-06 15:32:13 +03:00
Damien George
95f53461c2 py: Replace py-version.sh with makeversionhdr.py, written in Python.
Also rename py-version.h to mpversion.h for consistency with mpconfig.h.
2015-04-28 23:52:36 +01:00
Sven Wegener
b98c162c52 py: Fix order-only dependencies in mkrules.mk and py.mk.
Currently compilation sporadically fails, because the automatic
dependency gets created *during* the compilation of objects.

OBJ is a auperset of PY_O and the dependencies apply to all objects.

Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
2014-11-06 18:48:38 +00:00
Damien George
7a37f647a5 Merge branch 'teensy-new' of github.com:dhylands/micropython into dhylands-teensy-new
Conflicts:
	stmhal/pin_named_pins.c
	stmhal/readline.c

Renamed HAL_H to MICROPY_HAL_H.  Made stmhal/mphal.h which intends to
define the generic Micro Python HAL, which in stmhal sits above the ST
HAL.
2014-07-02 13:42:37 +01:00
Paul Sokolovsky
4c4b9d15ab mkrules.mk: Pass $(COPT) to link stage.
In generalize case, optimization options should be passed to all stages of
the build process.
2014-06-20 23:49:30 +03:00
Paul Sokolovsky
0fc7efb663 makefile: Pass STRIPFLAGS_EXTRA to strip.
Expected to be set on command line, with the idea being that for different
targets, there're different smartass ABIs which strive to put unneeded
sections into executables, etc., so let people have flexible way to
strip that.

The option name is similar to previously introduced CLFAGS_EXTRA &
LDFLAGS_EXTRA.
2014-06-20 20:25:35 +03:00
Dave Hylands
4f1b7fec9f Updated teensy to build.
Refactored some stmhal files which are shared with teensy.
2014-06-15 22:48:05 -07:00
Damien George
d553be5982 build: Simplify build directory layout by putting all headers in genhdr.
Any generated headers go in $(BUILD)/genhdr/, and are #included as
'genhdr/xxx.h'.
2014-04-17 18:03:27 +01:00