From 45f99cb4459248116396773a72cc122edbe14a00 Mon Sep 17 00:00:00 2001 From: Damien George Date: Mon, 20 Nov 2023 19:12:02 +1100 Subject: [PATCH] docs/library: Move vfs functions and classes from os to vfs module docs. Signed-off-by: Damien George --- docs/library/esp32.rst | 2 +- docs/library/index.rst | 1 + docs/library/machine.SDCard.rst | 2 +- docs/library/os.rst | 182 ++----------------------- docs/library/pyb.Flash.rst | 2 +- docs/library/pyb.rst | 2 +- docs/library/rp2.Flash.rst | 2 +- docs/library/vfs.rst | 208 +++++++++++++++++++++++++++++ docs/library/zephyr.DiskAccess.rst | 2 +- docs/library/zephyr.FlashArea.rst | 2 +- docs/reference/filesystem.rst | 12 +- docs/zephyr/tutorial/storage.rst | 6 +- 12 files changed, 235 insertions(+), 188 deletions(-) create mode 100644 docs/library/vfs.rst diff --git a/docs/library/esp32.rst b/docs/library/esp32.rst index 422329bf1e..dc35e7905e 100644 --- a/docs/library/esp32.rst +++ b/docs/library/esp32.rst @@ -114,7 +114,7 @@ methods to enable over-the-air (OTA) updates. These methods implement the simple and :ref:`extended ` block protocol defined by - :class:`os.AbstractBlockDev`. + :class:`vfs.AbstractBlockDev`. .. method:: Partition.set_boot() diff --git a/docs/library/index.rst b/docs/library/index.rst index 4a7f78ecf3..eb29b76805 100644 --- a/docs/library/index.rst +++ b/docs/library/index.rst @@ -104,6 +104,7 @@ the following libraries. neopixel.rst network.rst uctypes.rst + vfs.rst The following libraries provide drivers for hardware components. diff --git a/docs/library/machine.SDCard.rst b/docs/library/machine.SDCard.rst index cde0bd1d14..4faa7a5557 100644 --- a/docs/library/machine.SDCard.rst +++ b/docs/library/machine.SDCard.rst @@ -27,7 +27,7 @@ vary from platform to platform. This class provides access to SD or MMC storage cards using either a dedicated SD/MMC interface hardware or through an SPI channel. - The class implements the block protocol defined by :class:`os.AbstractBlockDev`. + The class implements the block protocol defined by :class:`vfs.AbstractBlockDev`. This allows the mounting of an SD card to be as simple as:: os.mount(machine.SDCard(), "/sd") diff --git a/docs/library/os.rst b/docs/library/os.rst index 27a7d2d44a..710875e34c 100644 --- a/docs/library/os.rst +++ b/docs/library/os.rst @@ -136,192 +136,30 @@ Terminal redirection and duplication Filesystem mounting ------------------- -Some ports provide a Virtual Filesystem (VFS) and the ability to mount multiple -"real" filesystems within this VFS. Filesystem objects can be mounted at either -the root of the VFS, or at a subdirectory that lives in the root. This allows -dynamic and flexible configuration of the filesystem that is seen by Python -programs. Ports that have this functionality provide the :func:`mount` and -:func:`umount` functions, and possibly various filesystem implementations -represented by VFS classes. +The following functions and classes have been moved to the :mod:`vfs` module. +They are provided in this module only for backwards compatibility and will be +removed in version 2 of MicroPython. .. function:: mount(fsobj, mount_point, *, readonly) - Mount the filesystem object *fsobj* at the location in the VFS given by the - *mount_point* string. *fsobj* can be a a VFS object that has a ``mount()`` - method, or a block device. If it's a block device then the filesystem type - is automatically detected (an exception is raised if no filesystem was - recognised). *mount_point* may be ``'/'`` to mount *fsobj* at the root, - or ``'/'`` to mount it at a subdirectory under the root. - - If *readonly* is ``True`` then the filesystem is mounted read-only. - - During the mount process the method ``mount()`` is called on the filesystem - object. - - Will raise ``OSError(EPERM)`` if *mount_point* is already mounted. + See `vfs.mount`. .. function:: umount(mount_point) - Unmount a filesystem. *mount_point* can be a string naming the mount location, - or a previously-mounted filesystem object. During the unmount process the - method ``umount()`` is called on the filesystem object. - - Will raise ``OSError(EINVAL)`` if *mount_point* is not found. + See `vfs.umount`. .. class:: VfsFat(block_dev) - Create a filesystem object that uses the FAT filesystem format. Storage of - the FAT filesystem is provided by *block_dev*. - Objects created by this constructor can be mounted using :func:`mount`. - - .. staticmethod:: mkfs(block_dev) - - Build a FAT filesystem on *block_dev*. + See `vfs.VfsFat`. .. class:: VfsLfs1(block_dev, readsize=32, progsize=32, lookahead=32) - Create a filesystem object that uses the `littlefs v1 filesystem format`_. - Storage of the littlefs filesystem is provided by *block_dev*, which must - support the :ref:`extended interface `. - Objects created by this constructor can be mounted using :func:`mount`. - - See :ref:`filesystem` for more information. - - .. staticmethod:: mkfs(block_dev, readsize=32, progsize=32, lookahead=32) - - Build a Lfs1 filesystem on *block_dev*. - - .. note:: There are reports of littlefs v1 failing in certain situations, - for details see `littlefs issue 347`_. + See `vfs.VfsLfs1`. .. class:: VfsLfs2(block_dev, readsize=32, progsize=32, lookahead=32, mtime=True) - Create a filesystem object that uses the `littlefs v2 filesystem format`_. - Storage of the littlefs filesystem is provided by *block_dev*, which must - support the :ref:`extended interface `. - Objects created by this constructor can be mounted using :func:`mount`. + See `vfs.VfsLfs2`. - The *mtime* argument enables modification timestamps for files, stored using - littlefs attributes. This option can be disabled or enabled differently each - mount time and timestamps will only be added or updated if *mtime* is enabled, - otherwise the timestamps will remain untouched. Littlefs v2 filesystems without - timestamps will work without reformatting and timestamps will be added - transparently to existing files once they are opened for writing. When *mtime* - is enabled `os.stat` on files without timestamps will return 0 for the timestamp. +.. class:: VfsPosix(root=None) - See :ref:`filesystem` for more information. - - .. staticmethod:: mkfs(block_dev, readsize=32, progsize=32, lookahead=32) - - Build a Lfs2 filesystem on *block_dev*. - - .. note:: There are reports of littlefs v2 failing in certain situations, - for details see `littlefs issue 295`_. - -.. _littlefs v1 filesystem format: https://github.com/ARMmbed/littlefs/tree/v1 -.. _littlefs v2 filesystem format: https://github.com/ARMmbed/littlefs -.. _littlefs issue 295: https://github.com/ARMmbed/littlefs/issues/295 -.. _littlefs issue 347: https://github.com/ARMmbed/littlefs/issues/347 - -Block devices -------------- - -A block device is an object which implements the block protocol. This enables a -device to support MicroPython filesystems. The physical hardware is represented -by a user defined class. The :class:`AbstractBlockDev` class is a template for -the design of such a class: MicroPython does not actually provide that class, -but an actual block device class must implement the methods described below. - -A concrete implementation of this class will usually allow access to the -memory-like functionality of a piece of hardware (like flash memory). A block -device can be formatted to any supported filesystem and mounted using ``os`` -methods. - -See :ref:`filesystem` for example implementations of block devices using the -two variants of the block protocol described below. - -.. _block-device-interface: - -Simple and extended interface -............................. - -There are two compatible signatures for the ``readblocks`` and ``writeblocks`` -methods (see below), in order to support a variety of use cases. A given block -device may implement one form or the other, or both at the same time. The second -form (with the offset parameter) is referred to as the "extended interface". - -Some filesystems (such as littlefs) that require more control over write -operations, for example writing to sub-block regions without erasing, may require -that the block device supports the extended interface. - -.. class:: AbstractBlockDev(...) - - Construct a block device object. The parameters to the constructor are - dependent on the specific block device. - - .. method:: readblocks(block_num, buf) - readblocks(block_num, buf, offset) - - The first form reads aligned, multiples of blocks. - Starting at the block given by the index *block_num*, read blocks from - the device into *buf* (an array of bytes). - The number of blocks to read is given by the length of *buf*, - which will be a multiple of the block size. - - The second form allows reading at arbitrary locations within a block, - and arbitrary lengths. - Starting at block index *block_num*, and byte offset within that block - of *offset*, read bytes from the device into *buf* (an array of bytes). - The number of bytes to read is given by the length of *buf*. - - .. method:: writeblocks(block_num, buf) - writeblocks(block_num, buf, offset) - - The first form writes aligned, multiples of blocks, and requires that the - blocks that are written to be first erased (if necessary) by this method. - Starting at the block given by the index *block_num*, write blocks from - *buf* (an array of bytes) to the device. - The number of blocks to write is given by the length of *buf*, - which will be a multiple of the block size. - - The second form allows writing at arbitrary locations within a block, - and arbitrary lengths. Only the bytes being written should be changed, - and the caller of this method must ensure that the relevant blocks are - erased via a prior ``ioctl`` call. - Starting at block index *block_num*, and byte offset within that block - of *offset*, write bytes from *buf* (an array of bytes) to the device. - The number of bytes to write is given by the length of *buf*. - - Note that implementations must never implicitly erase blocks if the offset - argument is specified, even if it is zero. - - .. method:: ioctl(op, arg) - - Control the block device and query its parameters. The operation to - perform is given by *op* which is one of the following integers: - - - 1 -- initialise the device (*arg* is unused) - - 2 -- shutdown the device (*arg* is unused) - - 3 -- sync the device (*arg* is unused) - - 4 -- get a count of the number of blocks, should return an integer - (*arg* is unused) - - 5 -- get the number of bytes in a block, should return an integer, - or ``None`` in which case the default value of 512 is used - (*arg* is unused) - - 6 -- erase a block, *arg* is the block number to erase - - As a minimum ``ioctl(4, ...)`` must be intercepted; for littlefs - ``ioctl(6, ...)`` must also be intercepted. The need for others is - hardware dependent. - - Prior to any call to ``writeblocks(block, ...)`` littlefs issues - ``ioctl(6, block)``. This enables a device driver to erase the block - prior to a write if the hardware requires it. Alternatively a driver - might intercept ``ioctl(6, block)`` and return 0 (success). In this case - the driver assumes responsibility for detecting the need for erasure. - - Unless otherwise stated ``ioctl(op, arg)`` can return ``None``. - Consequently an implementation can ignore unused values of ``op``. Where - ``op`` is intercepted, the return value for operations 4 and 5 are as - detailed above. Other operations should return 0 on success and non-zero - for failure, with the value returned being an ``OSError`` errno code. + See `vfs.VfsPosix`. diff --git a/docs/library/pyb.Flash.rst b/docs/library/pyb.Flash.rst index 984e13f458..562bcf1e24 100644 --- a/docs/library/pyb.Flash.rst +++ b/docs/library/pyb.Flash.rst @@ -43,7 +43,7 @@ Methods These methods implement the simple and :ref:`extended ` block protocol defined by - :class:`os.AbstractBlockDev`. + :class:`vfs.AbstractBlockDev`. Hardware Note ------------- diff --git a/docs/library/pyb.rst b/docs/library/pyb.rst index c8ef2c5315..869e2f8f3b 100644 --- a/docs/library/pyb.rst +++ b/docs/library/pyb.rst @@ -217,7 +217,7 @@ Miscellaneous functions Mount a block device and make it available as part of the filesystem. ``device`` must be an object that provides the block protocol. (The - following is also deprecated. See :class:`os.AbstractBlockDev` for the + following is also deprecated. See :class:`vfs.AbstractBlockDev` for the correct way to create a block device.) - ``readblocks(self, blocknum, buf)`` diff --git a/docs/library/rp2.Flash.rst b/docs/library/rp2.Flash.rst index 1e94cf519c..70cb127a19 100644 --- a/docs/library/rp2.Flash.rst +++ b/docs/library/rp2.Flash.rst @@ -32,5 +32,5 @@ Methods These methods implement the simple and extended :ref:`block protocol ` defined by - :class:`os.AbstractBlockDev`. + :class:`vfs.AbstractBlockDev`. diff --git a/docs/library/vfs.rst b/docs/library/vfs.rst new file mode 100644 index 0000000000..fcd06eb435 --- /dev/null +++ b/docs/library/vfs.rst @@ -0,0 +1,208 @@ +:mod:`vfs` -- virtual filesystem control +======================================== + +.. module:: vfs + :synopsis: virtual filesystem control + +The ``vfs`` module contains functions for creating filesystem objects and +mounting/unmounting them in the Virtual Filesystem. + +Filesystem mounting +------------------- + +Some ports provide a Virtual Filesystem (VFS) and the ability to mount multiple +"real" filesystems within this VFS. Filesystem objects can be mounted at either +the root of the VFS, or at a subdirectory that lives in the root. This allows +dynamic and flexible configuration of the filesystem that is seen by Python +programs. Ports that have this functionality provide the :func:`mount` and +:func:`umount` functions, and possibly various filesystem implementations +represented by VFS classes. + +.. function:: mount(fsobj, mount_point, *, readonly) + + Mount the filesystem object *fsobj* at the location in the VFS given by the + *mount_point* string. *fsobj* can be a a VFS object that has a ``mount()`` + method, or a block device. If it's a block device then the filesystem type + is automatically detected (an exception is raised if no filesystem was + recognised). *mount_point* may be ``'/'`` to mount *fsobj* at the root, + or ``'/'`` to mount it at a subdirectory under the root. + + If *readonly* is ``True`` then the filesystem is mounted read-only. + + During the mount process the method ``mount()`` is called on the filesystem + object. + + Will raise ``OSError(EPERM)`` if *mount_point* is already mounted. + +.. function:: umount(mount_point) + + Unmount a filesystem. *mount_point* can be a string naming the mount location, + or a previously-mounted filesystem object. During the unmount process the + method ``umount()`` is called on the filesystem object. + + Will raise ``OSError(EINVAL)`` if *mount_point* is not found. + +.. class:: VfsFat(block_dev) + + Create a filesystem object that uses the FAT filesystem format. Storage of + the FAT filesystem is provided by *block_dev*. + Objects created by this constructor can be mounted using :func:`mount`. + + .. staticmethod:: mkfs(block_dev) + + Build a FAT filesystem on *block_dev*. + +.. class:: VfsLfs1(block_dev, readsize=32, progsize=32, lookahead=32) + + Create a filesystem object that uses the `littlefs v1 filesystem format`_. + Storage of the littlefs filesystem is provided by *block_dev*, which must + support the :ref:`extended interface `. + Objects created by this constructor can be mounted using :func:`mount`. + + See :ref:`filesystem` for more information. + + .. staticmethod:: mkfs(block_dev, readsize=32, progsize=32, lookahead=32) + + Build a Lfs1 filesystem on *block_dev*. + + .. note:: There are reports of littlefs v1 failing in certain situations, + for details see `littlefs issue 347`_. + +.. class:: VfsLfs2(block_dev, readsize=32, progsize=32, lookahead=32, mtime=True) + + Create a filesystem object that uses the `littlefs v2 filesystem format`_. + Storage of the littlefs filesystem is provided by *block_dev*, which must + support the :ref:`extended interface `. + Objects created by this constructor can be mounted using :func:`mount`. + + The *mtime* argument enables modification timestamps for files, stored using + littlefs attributes. This option can be disabled or enabled differently each + mount time and timestamps will only be added or updated if *mtime* is enabled, + otherwise the timestamps will remain untouched. Littlefs v2 filesystems without + timestamps will work without reformatting and timestamps will be added + transparently to existing files once they are opened for writing. When *mtime* + is enabled `os.stat` on files without timestamps will return 0 for the timestamp. + + See :ref:`filesystem` for more information. + + .. staticmethod:: mkfs(block_dev, readsize=32, progsize=32, lookahead=32) + + Build a Lfs2 filesystem on *block_dev*. + + .. note:: There are reports of littlefs v2 failing in certain situations, + for details see `littlefs issue 295`_. + +.. class:: VfsPosix(root=None) + + Create a filesystem object that accesses the host POSIX filesystem. + If *root* is specified then it should be a path in the host filesystem to use + as the root of the ``VfsPosix`` object. Otherwise the current directory of + the host filesystem is used. + +.. _littlefs v1 filesystem format: https://github.com/ARMmbed/littlefs/tree/v1 +.. _littlefs v2 filesystem format: https://github.com/ARMmbed/littlefs +.. _littlefs issue 295: https://github.com/ARMmbed/littlefs/issues/295 +.. _littlefs issue 347: https://github.com/ARMmbed/littlefs/issues/347 + +Block devices +------------- + +A block device is an object which implements the block protocol. This enables a +device to support MicroPython filesystems. The physical hardware is represented +by a user defined class. The :class:`AbstractBlockDev` class is a template for +the design of such a class: MicroPython does not actually provide that class, +but an actual block device class must implement the methods described below. + +A concrete implementation of this class will usually allow access to the +memory-like functionality of a piece of hardware (like flash memory). A block +device can be formatted to any supported filesystem and mounted using ``os`` +methods. + +See :ref:`filesystem` for example implementations of block devices using the +two variants of the block protocol described below. + +.. _block-device-interface: + +Simple and extended interface +............................. + +There are two compatible signatures for the ``readblocks`` and ``writeblocks`` +methods (see below), in order to support a variety of use cases. A given block +device may implement one form or the other, or both at the same time. The second +form (with the offset parameter) is referred to as the "extended interface". + +Some filesystems (such as littlefs) that require more control over write +operations, for example writing to sub-block regions without erasing, may require +that the block device supports the extended interface. + +.. class:: AbstractBlockDev(...) + + Construct a block device object. The parameters to the constructor are + dependent on the specific block device. + + .. method:: readblocks(block_num, buf) + readblocks(block_num, buf, offset) + + The first form reads aligned, multiples of blocks. + Starting at the block given by the index *block_num*, read blocks from + the device into *buf* (an array of bytes). + The number of blocks to read is given by the length of *buf*, + which will be a multiple of the block size. + + The second form allows reading at arbitrary locations within a block, + and arbitrary lengths. + Starting at block index *block_num*, and byte offset within that block + of *offset*, read bytes from the device into *buf* (an array of bytes). + The number of bytes to read is given by the length of *buf*. + + .. method:: writeblocks(block_num, buf) + writeblocks(block_num, buf, offset) + + The first form writes aligned, multiples of blocks, and requires that the + blocks that are written to be first erased (if necessary) by this method. + Starting at the block given by the index *block_num*, write blocks from + *buf* (an array of bytes) to the device. + The number of blocks to write is given by the length of *buf*, + which will be a multiple of the block size. + + The second form allows writing at arbitrary locations within a block, + and arbitrary lengths. Only the bytes being written should be changed, + and the caller of this method must ensure that the relevant blocks are + erased via a prior ``ioctl`` call. + Starting at block index *block_num*, and byte offset within that block + of *offset*, write bytes from *buf* (an array of bytes) to the device. + The number of bytes to write is given by the length of *buf*. + + Note that implementations must never implicitly erase blocks if the offset + argument is specified, even if it is zero. + + .. method:: ioctl(op, arg) + + Control the block device and query its parameters. The operation to + perform is given by *op* which is one of the following integers: + + - 1 -- initialise the device (*arg* is unused) + - 2 -- shutdown the device (*arg* is unused) + - 3 -- sync the device (*arg* is unused) + - 4 -- get a count of the number of blocks, should return an integer + (*arg* is unused) + - 5 -- get the number of bytes in a block, should return an integer, + or ``None`` in which case the default value of 512 is used + (*arg* is unused) + - 6 -- erase a block, *arg* is the block number to erase + + As a minimum ``ioctl(4, ...)`` must be intercepted; for littlefs + ``ioctl(6, ...)`` must also be intercepted. The need for others is + hardware dependent. + + Prior to any call to ``writeblocks(block, ...)`` littlefs issues + ``ioctl(6, block)``. This enables a device driver to erase the block + prior to a write if the hardware requires it. Alternatively a driver + might intercept ``ioctl(6, block)`` and return 0 (success). In this case + the driver assumes responsibility for detecting the need for erasure. + + Unless otherwise stated ``ioctl(op, arg)`` can return ``None``. + Consequently an implementation can ignore unused values of ``op``. Where + ``op`` is intercepted, the return value for operations 4 and 5 are as + detailed above. Other operations should return 0 on success and non-zero + for failure, with the value returned being an ``OSError`` errno code. diff --git a/docs/library/zephyr.DiskAccess.rst b/docs/library/zephyr.DiskAccess.rst index 3e5fa9a357..e5eac8ca49 100644 --- a/docs/library/zephyr.DiskAccess.rst +++ b/docs/library/zephyr.DiskAccess.rst @@ -34,5 +34,5 @@ Methods These methods implement the simple and extended :ref:`block protocol ` defined by - :class:`os.AbstractBlockDev`. + :class:`vfs.AbstractBlockDev`. diff --git a/docs/library/zephyr.FlashArea.rst b/docs/library/zephyr.FlashArea.rst index 9cd4dd59d6..749b90a3b4 100644 --- a/docs/library/zephyr.FlashArea.rst +++ b/docs/library/zephyr.FlashArea.rst @@ -37,4 +37,4 @@ Methods These methods implement the simple and extended :ref:`block protocol ` defined by - :class:`os.AbstractBlockDev`. + :class:`vfs.AbstractBlockDev`. diff --git a/docs/reference/filesystem.rst b/docs/reference/filesystem.rst index ca9e56344e..d8092c58dd 100644 --- a/docs/reference/filesystem.rst +++ b/docs/reference/filesystem.rst @@ -40,7 +40,7 @@ Block devices ------------- A block device is an instance of a class that implements the -:class:`os.AbstractBlockDev` protocol. +:class:`vfs.AbstractBlockDev` protocol. Built-in block devices ~~~~~~~~~~~~~~~~~~~~~~ @@ -116,8 +116,8 @@ It can be used as follows:: An example of a block device that supports both the simple and extended interface (i.e. both signatures and behaviours of the -:meth:`os.AbstractBlockDev.readblocks` and -:meth:`os.AbstractBlockDev.writeblocks` methods) is:: +:meth:`vfs.AbstractBlockDev.readblocks` and +:meth:`vfs.AbstractBlockDev.writeblocks` methods) is:: class RAMBlockDev: def __init__(self, block_size, num_blocks): @@ -148,7 +148,7 @@ interface (i.e. both signatures and behaviours of the return 0 As it supports the extended interface, it can be used with :class:`littlefs -`:: +`:: import os @@ -166,8 +166,8 @@ normally would be used from Python code, for example:: Filesystems ----------- -MicroPython ports can provide implementations of :class:`FAT `, -:class:`littlefs v1 ` and :class:`littlefs v2 `. +MicroPython ports can provide implementations of :class:`FAT `, +:class:`littlefs v1 ` and :class:`littlefs v2 `. The following table shows which filesystems are included in the firmware by default for given port/board combinations, however they can be optionally diff --git a/docs/zephyr/tutorial/storage.rst b/docs/zephyr/tutorial/storage.rst index 12bd0d0d35..e8dcbbd39a 100644 --- a/docs/zephyr/tutorial/storage.rst +++ b/docs/zephyr/tutorial/storage.rst @@ -6,14 +6,14 @@ Filesystems and Storage Storage modules support virtual filesystem with FAT and littlefs formats, backed by either Zephyr DiskAccess or FlashArea (flash map) APIs depending on which the board supports. -See `os Filesystem Mounting `_. +See `vfs Filesystem Mounting `_. Disk Access ----------- The :ref:`zephyr.DiskAccess ` class can be used to access storage devices, such as SD cards. This class uses `Zephyr Disk Access API `_ and -implements the `os.AbstractBlockDev` protocol. +implements the `vfs.AbstractBlockDev` protocol. For use with SD card controllers, SD cards must be present at boot & not removed; they will be auto detected and initialized by filesystem at boot. Use the disk driver interface and a @@ -39,7 +39,7 @@ customize filesystem configurations. To store persistent data on the device, usi API is recommended (see below). This class uses `Zephyr Flash map API `_ and -implements the `os.AbstractBlockDev` protocol. +implements the `vfs.AbstractBlockDev` protocol. Example usage with the internal flash on the reel_board or the rv32m1_vega_ri5cy board::