refactor(tools): Remove test_build_system_cmake.sh & test_build_system_spaces.py

pull/12177/head
Marek Fiala 2023-08-10 11:03:52 +02:00
rodzic e0f5d3c527
commit 7d2d8b7839
7 zmienionych plików z 0 dodań i 1559 usunięć

Wyświetl plik

@ -210,16 +210,6 @@ before_script:
before_script:
- *common-before_scripts
.before_script_macos:
before_script:
- *common-before_scripts
# On macOS, these tools need to be installed
- export IDF_TOOLS_PATH="${HOME}/.espressif_runner_${CI_RUNNER_ID}_${CI_CONCURRENT_ID}"
- $IDF_PATH/tools/idf_tools.py --non-interactive install cmake ninja
# This adds tools (compilers) and the version-specific Python environment to PATH
- *setup_tools_and_idf_python_venv
- fetch_submodules
.before_script_build_jobs:
before_script:
- *common-before_scripts

Wyświetl plik

@ -756,26 +756,6 @@ build_clang_test_apps_esp32c6:
- cd test_build_system
- ${IDF_PATH}/tools/ci/${SHELL_TEST_SCRIPT}
test_build_system_cmake:
extends: .test_build_system_template
variables:
SHELL_TEST_SCRIPT: test_build_system_cmake.sh
test_build_system_cmake_macos:
extends:
- .test_build_system_template
- .before_script_macos
- .rules:build:macos
tags:
- macos_shell
variables:
SHELL_TEST_SCRIPT: test_build_system_cmake.sh
test_build_system_spaces:
extends: .test_build_system_template
variables:
SHELL_TEST_SCRIPT: test_build_system_spaces.py
pytest_build_system:
extends: .test_build_system_template
parallel: 3

Wyświetl plik

@ -122,12 +122,9 @@
.patterns-build_macos: &patterns-build_macos
- "tools/ci/test_configure_ci_environment.sh"
- "tools/ci/test_build_system_cmake.sh"
.patterns-build_check: &patterns-build_check
- "tools/test_build_system/**/*"
- "tools/ci/test_build_system_cmake.sh"
- "tools/ci/test_build_system_spaces.py"
- "tools/ci/test_configure_ci_environment.sh"
- "tools/gen_esp_err_to_name.py"
- "tools/ci/check_soc_headers_leak.py"
@ -1570,21 +1567,6 @@
- <<: *if-dev-push
changes: *patterns-target_test-wifi
.rules:build:macos:
rules:
- <<: *if-revert-branch
when: never
- <<: *if-protected
- <<: *if-label-build
- <<: *if-label-macos
- <<: *if-label-macos_test
- <<: *if-dev-push
changes: *patterns-build_macos
- <<: *if-dev-push
changes: *patterns-build_system
- <<: *if-dev-push
changes: *patterns-downloadable-tools
.rules:build:target_test:
rules:
- <<: *if-revert-branch

Wyświetl plik

@ -76,8 +76,6 @@ tools/ci/mirror-submodule-update.sh
tools/ci/multirun_with_pyenv.sh
tools/ci/push_to_github.sh
tools/ci/test_autocomplete.py
tools/ci/test_build_system_cmake.sh
tools/ci/test_build_system_spaces.py
tools/ci/test_check_kconfigs.py
tools/ci/test_configure_ci_environment.sh
tools/ci/test_reproducible_build.sh

Wyświetl plik

@ -1,220 +0,0 @@
#!/usr/bin/env python
# SPDX-FileCopyrightText: 2021-2022 Espressif Systems (Shanghai) CO LTD
# SPDX-License-Identifier: Apache-2.0
import os
import shutil
import subprocess
import sys
import typing
import unittest
try:
IDF_PATH = os.environ['IDF_PATH'] # type: str
except KeyError:
print('IDF_PATH must be set before running this test', file=sys.stderr)
exit(1)
# Use the current directory for the builds
TEST_PATH = os.getcwd()
##################
# Helper functions
##################
def find_python(path_var: str) -> str:
"""
Find python interpreter in the paths specified in the given PATH variable.
Returns the full path to the interpreter.
"""
res = shutil.which('python', path=path_var)
if res is None:
raise ValueError('python not found')
return res
def get_idf_build_env(idf_path: typing.Optional[str]) -> typing.Dict[str, str]:
"""
Get environment variables (as set by export.sh) for the specific IDF copy
:param idf_path: if set, path of the IDF copy to use; otherwise, IDF_PATH from environment is used
:return: dictionary of environment variables and their values
"""
if idf_path is None:
idf_path = IDF_PATH
cmd = [
sys.executable,
os.path.join(idf_path, 'tools', 'idf_tools.py'),
'export',
'--format=key-value'
]
keys_values = subprocess.check_output(cmd).decode()
env_vars = {key: os.path.expandvars(value) for key, value in
[line.split('=') for line in keys_values.splitlines()]}
# not set by idf_tools.py, normally set by export.sh
env_vars['IDF_PATH'] = idf_path
return env_vars
def idf_py(*args: str, env_vars: typing.Optional[typing.Dict[str, str]] = None,
idf_path: typing.Optional[str] = None,
workdir: typing.Optional[str] = None) -> None:
"""
Run idf.py command with given arguments, raise an exception on failure
:param args: arguments to pass to idf.py
:param env_vars: environment variables to run the build with; if not set, the default environment is used
:param idf_path: path to the IDF copy to use; if not set, IDF_PATH from the environment is used
:param workdir: directory where to run the build; if not set, the current directory is used
"""
env = dict(**os.environ)
if env_vars is not None:
env.update(env_vars)
if not workdir:
workdir = os.getcwd()
# order: function argument -> value in env dictionary -> system environment
if idf_path is not None:
inferred_idf_path = idf_path
else:
inferred_idf_path = str(env.get('IDF_PATH', IDF_PATH))
python = find_python(env['PATH'])
cmd = [
python,
os.path.join(inferred_idf_path, 'tools', 'idf.py')
]
cmd += args # type: ignore
subprocess.check_call(cmd, env=env, cwd=workdir)
############
# Test cases
############
class PathsWithSpaces(unittest.TestCase):
IDF_PATH_WITH_SPACES = ''
DO_CLEANUP = True
ENV: typing.Dict[str, str] = dict()
@classmethod
def setUpClass(cls) -> None:
if ' ' in IDF_PATH:
print('IDF_PATH already contains spaces, not making a copy')
cls.IDF_PATH_WITH_SPACES = IDF_PATH
cls.DO_CLEANUP = False
else:
# Make a copy of ESP-IDF directory, with a space in its name
cls.IDF_PATH_WITH_SPACES = os.path.join(TEST_PATH, 'esp idf')
dest = cls.IDF_PATH_WITH_SPACES
print('Copying esp-idf from {} to {}'.format(IDF_PATH, dest))
shutil.copytree(IDF_PATH, dest,
# if the CWD is inside the original esp-idf directory, make sure not to go into recursion.
ignore=shutil.ignore_patterns(os.path.basename(dest)))
cls.ENV = get_idf_build_env(cls.IDF_PATH_WITH_SPACES)
@classmethod
def tearDownClass(cls) -> None:
if cls.DO_CLEANUP and os.path.exists(cls.IDF_PATH_WITH_SPACES):
shutil.rmtree(cls.IDF_PATH_WITH_SPACES, ignore_errors=True)
def test_install_export(self) -> None:
env = dict(**os.environ)
del env['IDF_PATH']
if os.name == 'nt':
install_cmd = 'install.bat esp32'
export_cmd = 'export.bat'
else:
install_cmd = './install.sh esp32'
export_cmd = '. ./export.sh'
subprocess.check_call(install_cmd, env=env, shell=True, cwd=self.IDF_PATH_WITH_SPACES)
if os.name == 'nt':
subprocess.check_call(export_cmd, env=env, shell=True, cwd=self.IDF_PATH_WITH_SPACES)
else:
# The default shell used by subprocess.Popen on POSIX platforms is '/bin/sh',
# which in esp-env Docker image is 'dash'. The export script doesn't support
# IDF_PATH detection when used in dash, so we have to override the shell here.
subprocess.check_call(export_cmd, env=env, shell=True, cwd=self.IDF_PATH_WITH_SPACES, executable='/bin/bash')
def _copy_app_to(self, app_path: str, dest_dir_name: str) -> str:
"""
Copy given app to a different directory, setting up cleanup hook.
Destination directory is first deleted if it already exists.
Returns the full path of the destination directory.
app_path: path relative to esp-idf directory
dest_dir_name: name of the destination directory, relative to the directory where the test is running
"""
dest_dir = os.path.join(TEST_PATH, dest_dir_name)
if os.path.exists(dest_dir) and os.path.isdir(dest_dir):
shutil.rmtree(dest_dir, ignore_errors=True)
src_dir = os.path.join(self.IDF_PATH_WITH_SPACES, app_path)
shutil.copytree(os.path.join(src_dir), dest_dir)
build_dir = os.path.join(dest_dir, 'build')
if os.path.exists(build_dir):
shutil.rmtree(build_dir, ignore_errors=True)
self.addCleanup(shutil.rmtree, dest_dir, ignore_errors=True)
return dest_dir
# The tests below build different ESP-IDF apps (examples or test apps) to cover different parts
# of the build system and related scripts.
# In each test, IDF_PATH and app path both contain spaces.
def test_build(self) -> None:
build_path = self._copy_app_to(os.path.join('examples', 'get-started', 'hello_world'), 'test app')
idf_py('build', env_vars=self.ENV, idf_path=self.IDF_PATH_WITH_SPACES, workdir=build_path)
def test_build_ulp_fsm(self) -> None:
build_path = self._copy_app_to(os.path.join('examples', 'system', 'ulp', 'ulp_fsm', 'ulp'), 'test app')
idf_py('build', env_vars=self.ENV, idf_path=self.IDF_PATH_WITH_SPACES, workdir=build_path)
def test_build_ulp_riscv(self) -> None:
build_path = self._copy_app_to(os.path.join('examples', 'system', 'ulp', 'ulp_riscv', 'gpio'), 'test app')
idf_py('-DIDF_TARGET=esp32s2', 'build', env_vars=self.ENV, idf_path=self.IDF_PATH_WITH_SPACES,
workdir=build_path)
def test_spiffsgen(self) -> None:
build_path = self._copy_app_to(os.path.join('examples', 'storage', 'spiffsgen'), 'test app')
idf_py('build', env_vars=self.ENV, idf_path=self.IDF_PATH_WITH_SPACES, workdir=build_path)
def test_flash_encryption(self) -> None:
build_path = self._copy_app_to(os.path.join('examples', 'security', 'flash_encryption'), 'test app')
idf_py('build', env_vars=self.ENV, idf_path=self.IDF_PATH_WITH_SPACES, workdir=build_path)
def test_secure_boot_v1(self) -> None:
build_path = self._copy_app_to(os.path.join('tools', 'test_apps', 'security', 'secure_boot'), 'test app')
idf_py('-DSDKCONFIG_DEFAULTS=sdkconfig.defaults;sdkconfig.ci.01', 'build', env_vars=self.ENV,
idf_path=self.IDF_PATH_WITH_SPACES, workdir=build_path)
def test_secure_boot_v2(self) -> None:
build_path = self._copy_app_to(os.path.join('tools', 'test_apps', 'security', 'secure_boot'), 'test app')
idf_py('-DSDKCONFIG_DEFAULTS=sdkconfig.defaults;sdkconfig.ci.00', 'build', env_vars=self.ENV,
idf_path=self.IDF_PATH_WITH_SPACES, workdir=build_path)
def test_app_signing(self) -> None:
build_path = self._copy_app_to(os.path.join('tools', 'test_apps', 'security', 'secure_boot'), 'test app')
idf_py('-DSDKCONFIG_DEFAULTS=sdkconfig.defaults;sdkconfig.ci.02', 'build', env_vars=self.ENV,
idf_path=self.IDF_PATH_WITH_SPACES, workdir=build_path)
def test_secure_boot_release_mode(self) -> None:
build_path = self._copy_app_to(os.path.join('tools', 'test_apps', 'security', 'secure_boot'), 'test app')
idf_py('-DSDKCONFIG_DEFAULTS=sdkconfig.defaults;sdkconfig.ci.04', '-DIDF_TARGET=esp32s2', 'build',
env_vars=self.ENV, idf_path=self.IDF_PATH_WITH_SPACES, workdir=build_path)
def test_x509_cert_bundle(self) -> None:
build_path = self._copy_app_to(os.path.join('examples', 'protocols', 'https_x509_bundle'), 'test app')
idf_py('build', env_vars=self.ENV, idf_path=self.IDF_PATH_WITH_SPACES, workdir=build_path)
def test_dfu(self) -> None:
build_path = self._copy_app_to(os.path.join('examples', 'get-started', 'hello_world'), 'test app')
idf_py('-DIDF_TARGET=esp32s2', 'dfu', env_vars=self.ENV, idf_path=self.IDF_PATH_WITH_SPACES,
workdir=build_path)
def test_uf2(self) -> None:
build_path = self._copy_app_to(os.path.join('examples', 'get-started', 'hello_world'), 'test app')
idf_py('uf2', env_vars=self.ENV, idf_path=self.IDF_PATH_WITH_SPACES, workdir=build_path)
if __name__ == '__main__':
unittest.main()

Wyświetl plik

@ -1,110 +0,0 @@
# Migration from test_build_system_cmake.sh to pytest
This table tracks migration of tests from [test_build_system_cmake.sh](../ci/test_build_system_cmake.sh) and [test_build_system_spaces.py](../ci/test_build_system_spaces.py) to pytest.
When all tests are migrated to pytest, remove the original tests, corresponding CI jobs, and this file.
Legacy test name | New test name | Comments
-----------------|---------------|---------
Initial clean build | test_rebuild::test_rebuild_no_changes |
Updating component source file rebuilds component | test_rebuild::test_rebuild_source_files |
Bootloader source file rebuilds bootloader | test_rebuild::test_rebuild_source_files |
Partition CSV file rebuilds partitions | test_rebuild::test_rebuild_source_files |
Partial build doesn't compile anything by default | test_rebuild::test_rebuild_no_changes |
Rebuild when app version was changed | test_rebuild.py::test_rebuild_version_change |
Change app version | test_rebuild.py::test_rebuild_version_change |
Re-building does not change app.bin | test_rebuild.py::test_rebuild_version_change |
Get the version of app from git describe. Project is not inside IDF and do not have a tag only a hash commit. | test_git.py::test_get_version_from_git_describe |
Get the version of app from Kconfig option | test_kconfig.py::test_kconfig_get_version_from_describe |
Use IDF version variables in component CMakeLists.txt file | test_components.py::test_version_in_component_cmakelist |
Project is in ESP-IDF which has a custom tag | test_git.py::test_git_custom_tag |
Moving BUILD_DIR_BASE out of tree | test_build.py::test_build_alternative_directories |
BUILD_DIR_BASE inside default build directory | test_build.py::test_build_alternative_directories |
Can still clean build if all text files are CRLFs | test_build.py::test_build_with_crlf_files |
Updating rom ld file should re-link app and bootloader | test_rebuild::test_rebuild_linker |
Updating app-only ld file should only re-link app | test_rebuild::test_rebuild_linker |
Updating ld file should only re-link app | test_rebuild::test_rebuild_linker |
Updating fragment file should only re-link app | test_rebuild::test_rebuild_linker |
sdkconfig update triggers full recompile | test_rebuild::test_rebuild_source_files |
Updating project CMakeLists.txt triggers full recompile | test_rebuild::test_rebuild_source_files |
Can build with Ninja (no idf.py) | test_build.py::test_build_cmake_ninja |
Can build with GNU Make (no idf.py) | test_build.py::test_build_cmake_makefile |
idf.py can build with Ninja | test_build.py::test_build_with_generator_ninja |
idf.py can build with Unix Makefiles | test_build.py::test_build_with_generator_makefile |
Can build with IDF_PATH set via cmake cache not environment | test_build.py::test_build_with_cmake_and_idf_path_unset |
Can build with IDF_PATH unset and inferred by build system | test_build.py::test_build_with_cmake_and_idf_path_unset |
Can build with IDF_PATH unset and inferred by cmake when Kconfig needs it to be set | test_build.py::test_build_with_cmake_and_idf_path_unset |
can build with phy_init_data | test_build.py::test_build_skdconfig_phy_init_data |
can build with ethernet component disabled | | moved to test_apps/system/build_test/sdkconfig.ci.ethernet_disabled
Compiler flags on build command line are taken into account | test_build.py::test_build_compiler_flag_in_source_file |
Compiler flags cannot be overwritten | test_build.py::test_build_compiler_flags_no_overwriting |
Can override IDF_TARGET from environment | test_non_default_target.py::test_target_from_environment_cmake |
Can set target using idf.py -D | test_non_default_target.py::test_target_using_D_parameter |
Can set target using -D as subcommand parameter for idf.py | test_non_default_target.py::test_target_using_D_parameter |
Can set target using idf.py set-target | test_non_default_target.py::test_target_using_settarget_parameter |
idf.py understands alternative target names | test_non_default_target.py::test_target_using_settarget_parameter_alternative_name |
Can guess target from sdkconfig, if CMakeCache does not exist | test_non_default_target.py::test_target_using_settarget_parameter |
Can set the default target using sdkconfig.defaults | test_non_default_target.py::test_target_using_sdkconfig |
IDF_TARGET takes precedence over the value of CONFIG_IDF_TARGET in sdkconfig.defaults | test_non_default_target.py::test_target_precedence |
idf.py fails if IDF_TARGET settings don't match in sdkconfig, CMakeCache.txt, and the environment | test_non_default_target.py::test_target_from_environment_idf_py |
Setting EXTRA_COMPONENT_DIRS works | test_components.py::test_component_extra_dirs |
Non-existent paths in EXTRA_COMPONENT_DIRS are not allowed | test_components.py::test_component_nonexistent_extra_dirs_not_allowed |
Component names may contain spaces | test_components.py::test_component_names_contain_spaces |
sdkconfig should have contents of all files: sdkconfig, sdkconfig.defaults, sdkconfig.defaults.IDF_TARGET | test_sdkconfig.py::test_sdkconfig_contains_all_files |
Test if it can build the example to run on host | test_cmake.py::test_build_example_on_host |
Test build ESP-IDF as a library to a custom CMake projects for all targets | test_cmake.py::test_build_custom_cmake_project |
Building a project with CMake library imported and PSRAM workaround, all files compile with workaround | test_cmake.py::test_build_cmake_library_psram_workaround |
Test for external libraries in custom CMake projects with ESP-IDF components linked | test_cmake.py::test_build_custom_cmake_project |
Test for external libraries in custom CMake projects with PSRAM strategy $strat | test_cmake.py::test_build_cmake_library_psram_strategies |
Cleaning Python bytecode | test_common.py::test_python_clean |
Displays partition table when executing target partition_table | test_partition.py::test_partition_table |
Make sure a full build never runs '/usr/bin/env python' or similar | test_common.py::test_python_interpreter_unix, test_common.py::test_python_interpreter_win |
Handling deprecated Kconfig options | test_kconfig.py::test_kconfig_deprecated_options |
Handling deprecated Kconfig options in sdkconfig.defaults | test_kconfig.py::test_kconfig_deprecated_options |
Can have multiple deprecated Kconfig options map to a single new option | test_kconfig.py::test_kconfig_multiple_and_target_specific_options |
Can have target specific deprecated Kconfig options | test_kconfig.py::test_kconfig_multiple_and_target_specific_options |
Confserver can be invoked by idf.py | test_common.py::test_invoke_confserver |
Check ccache is used to build | test_common.py::test_ccache_used_to_build |
Custom bootloader overrides original | test_bootloader.py::test_bootloader_custom_overrides_original |
Empty directory not treated as a component | test_components.py::test_component_can_not_be_empty_dir |
If a component directory is added to COMPONENT_DIRS, its subdirectories are not added | test_components.py::test_component_subdirs_not_added_to_component_dirs |
If a component directory is added to COMPONENT_DIRS, its sibling directories are not added | test_components.py::test_component_sibling_dirs_not_added_to_component_dirs |
toolchain prefix is set in project description file | test_common.py::test_toolchain_prefix_in_description_file |
Can set options to subcommands: print_filter for monitor | test_common.py::test_subcommands_with_options |
Fail on build time works | test_build.py::test_build_fail_on_build_time |
Component properties are set | test_components.py::test_component_properties_are_set |
should be able to specify multiple sdkconfig default files | test_sdkconfig.py::test_sdkconfig_multiple_default_files |
Supports git worktree | test_git.py::test_support_git_worktree |
idf.py fallback to build system target | test_common.py::test_fallback_to_build_system_target |
Build fails if partitions don't fit in flash | test_partition.py::test_partitions_dont_fit_in_flash |
Warning is given if smallest partition is nearly full | test_partition.py::test_partition_nearly_full_warning |
Flash size is correctly set in the bootloader image header | test_bootloader.py::test_bootloader_correctly_set_image_header |
DFU build works | test_build.py::test_build_dfu |
UF2 build works | test_build.py::test_build_uf2 |
Loadable ELF build works | test_build.py::test_build_loadable_elf |
Defaults set properly for unspecified idf_build_process args | test_cmake.py::test_defaults_for_unspecified_idf_build_process_args |
Getting component overriden dir | test_components.py::test_component_overriden_dir |
Overriding Kconfig | test_components.py::test_component_overriden_dir |
Project components prioritized over EXTRA_COMPONENT_DIRS | test_components.py::test_components_prioritizer_over_extra_components_dir |
Components in EXCLUDE_COMPONENTS not passed to idf_component_manager | test_components.py::test_exclude_components_not_passed |
Create project using idf.py and build it | test_common.py::test_create_component_and_project_plus_build |
Create component using idf.py, create project using idf.py. | test_common.py::test_create_component_and_project_plus_build |
Add the component to the created project and build the project. | test_common.py::test_create_component_and_project_plus_build |
Check that command for creating new project will fail if the target folder is not empty. | test_common.py::test_create_project |
Check that command for creating new project will fail if the target path is file. | test_common.py::test_create_project |
Check docs command | test_common.py::test_docs_command |
Deprecation warning check | test_common.py::test_deprecation_warning |
Save-defconfig checks | test_common.py::test_save_defconfig_check |
test_install_export | test_spaces.py::test_install_export | split into two tests for unix and windows
test_build | test_spaces.py::test_spaces_bundle1 |
test_build_ulp_fsm | test_spaces.py::test_spaces_bundle1 |
test_build_ulp_riscv | test_spaces.py::test_spaces_bundle1 |
test_spiffsgen | test_spaces.py::test_spaces_bundle1 |
test_flash_encryption | test_spaces.py::test_spaces_bundle2 |
test_secure_boot_v1 | test_spaces.py::test_spaces_bundle3 |
test_secure_boot_v2 | test_spaces.py::test_spaces_bundle3 |
test_app_signing | test_spaces.py::test_spaces_bundle3 |
test_secure_boot_release_mode | test_spaces.py::test_spaces_bundle3 |
test_x509_cert_bundle | test_spaces.py::test_spaces_bundle2 |
test_dfu | test_spaces.py::test_spaces_bundle2 |
test_uf2 | test_spaces.py::test_spaces_bundle2 |