From 42899a0036dcc35eb1907e032e11827b08338629 Mon Sep 17 00:00:00 2001 From: peterhinch Date: Sat, 14 Jan 2023 10:34:17 +0000 Subject: [PATCH] Add notes on import. --- README.md | 10 ++++- import/IMPORT.md | 100 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 108 insertions(+), 2 deletions(-) create mode 100644 import/IMPORT.md diff --git a/README.md b/README.md index e3a0215..78f4627 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ and modules which are documented and supported. 1.2 [Fastbuild](./README.md#12-fastbuild) Build scripts and udev rules 1.3 [A web framework](./README.md#13-a-web-framework) Microdot. 1.4 [Buildcheck](./README.md#14-buildcheck) Check firmware build date - 1.5 [Pyboard USB pitfall](./README.md#15-pyboard-usb-pitfall) Read this if you're new to Pyboards + 1.5 [A USB pitfall](./README.md#15-a-usb-pitfall) A problem with platforms which share their filesystem with the PC. 2. [Hardware information and drivers](./README.md#2-hardware-information-and-drivers) 2.1 [ESP32](./README.md#21-esp32) Pinout and notes on the reference board 2.2 [SSD1306](./README.md#22-ssd1306) Write large fonts to the SSD1306. @@ -24,6 +24,7 @@ and modules which are documented and supported. 3.1 [Resilient](./README.md#31-resilient) A guide to writing resilient WiFi code 3.2 [Serialisation](./README.md#32-serialisation) Review of MicroPython's five serialisation libraries 3.3 [Measurement of relative timing and phase of fast analog signals](./README.md#33-measurement-of-relative-timing-and-phase-of-fast-analog-signals) For Pyboard. + 3.4 [Import subtleties](./README.md#34-import-subtleties) Ways to save RAM with import statements. 4. [Code samples](./README.md#4-code-samples) Samples prefixed Pyboard are Pyboard specific 4.1 [Pyboard Mutex](./README.md#41-pyboard-mutex) Share data between threads and ISR's. 4.2 [Pyboard watchdog](./README.md#42-pyboard-watchdog) Access a Pyboard hardware WDT. @@ -80,7 +81,7 @@ The solution preferred by MicroPython maintainers is Raise an [exception](./buildcheck/buildcheck.py) if a firmware build is earlier than a given date. -## 1.5 Pyboard USB pitfall +## 1.5 A USB pitfall By default the Pyboard's `/flash/boot.py` enables MSC (mass storage) mode. This makes the Pyboard look like a USB stick, making its filesystem visible to the @@ -147,6 +148,11 @@ tutorial on a Protocol Buffer library. This describes ways of using the Pyboard to perform precision measurements of analog signals of up to around 50KHz. It is documented [here](./phase/README.md). +## 3.4 Import subtleties + +[This doc](./import/IMPORT.md) describes a way to save RAM with Damien's lazy +loader, a `reload` function, and ways to use wildcard imports. + ##### [Index](./README.md#0-index) # 4. Code samples diff --git a/import/IMPORT.md b/import/IMPORT.md new file mode 100644 index 0000000..f130a87 --- /dev/null +++ b/import/IMPORT.md @@ -0,0 +1,100 @@ +# MicroPython's import statement + +I seldom write tutorials on elementary Python coding; there are plenty online. +However there are implications specific to low RAM environments. + +# 1. The import process + +When a module comprising Python source is imported, the compiler runs on the +target and emits bytecode. The bytecode resides in RAM for later execution. +Further, the compiler requires RAM, although this is reclaimed by the garbage +collector after compilation is complete. The compilation stage may be skipped +by precompiling the module to an `.mpy` file, but the only way to save on the +RAM used by the bytecode is to use +[frozen bytecode](http://docs.micropython.org/en/latest/reference/manifest.html). + +This doc addresses the case where code is not frozen, discussing ways to ensure +that only necessary bytecode is loaded. + +# 2. Python packages and the lazy loader + +Python packages provide an excellent way to split a module into individual +files to be loaded on demand. The drawback is that the user needs to know which +file to import to access a particular item: +```python +from my_library.foo import FooClass # It's in my_library/foo.py +``` +This may be simplified using Damien's "lazy loader". This allows the user to +write +```python +import my_library +foo = my_library.FooClass() # No need to know which file holds this class +``` +The file `my_library/foo.py` is only loaded when it becomes clear that +`FooClass` is required. Further, the structure of the package is hidden from +the user and may be changed without affecting its API. + +The "lazy loader" is employed in +[uasyncio](https://github.com/micropython/micropython/tree/master/extmod/uasyncio). +making it possible to write +```python +import uasyncio as asyncio +e = asyncio.Event() # The file event.py is loaded now +``` +Files are loaded as required. + +The source code is in `__init__.py`: +[the lazy loader](https://github.com/micropython/micropython/blob/master/extmod/uasyncio/__init__.py). + +# 3. Wildcard imports + +The use of +```python +from my_module import * +``` +needs to be treated with caution for two reasons. It can populate the program's +namespace with unexpected objects causing name conflicts. Secondly all these +objects occupy RAM. In general wildcard imports should be avoided unless the +module is designed to be imported in this way. + +For example issuing +```python +from uasyncio import * +``` +would defeat the lazy loader forcing all the files to be loaded. + +## 3.1 Designing fo wildcard import + +There are cases where wildcard import makes sense. For example a module might +declare a number of constants. This module +[colors.py](https://github.com/peterhinch/micropython-nano-gui/blob/master/gui/core/colors.py) +computes a set of 13 colors for use in an interface. This is the module's only +purpose so it is intended to be imported with +```python +from gui.core.colors import * +``` +This saves having to name each color explicitly. + +In larger modules it is wise to avoid populating the caller's namespace with +cruft. This is achieved by ensuring that all private names begin with a `_` +character: +```python +_LOCAL_CONSTANT = const(42) +def _foo(): + print("foo") # A local function +``` +# 4. Reload + +Users coming from a PC background often query the lack of a `reload` command. +In practice on a microcontroller it is usually best to issue a soft reset +(`ctrl-d`) before re-importing an updated module. This is because a soft reset +clears all retained state. However a `reload` function can be defined thus: +```python +import gc +import sys +def reload(mod): + mod_name = mod.__name__ + del sys.modules[mod_name] + gc.collect() + __import__(mod_name) +```