From b0ce95125c407ea8266f3f60d17345f7902f32cb Mon Sep 17 00:00:00 2001 From: Vamsi-klu Date: Sat, 17 Oct 2026 11:06:58 +0000 Subject: [PATCH 1/2] feat(cli): support project paths and options from environment variables Signed-off-by: Vamsi-klu --- docs/integrations/github.md | 15 +++ docs/reference/cli.md | 61 +++++++++- docs/reference/configuration.md | 8 ++ sqlmesh/cli/main.py | 26 +++-- sqlmesh/cli/options.py | 2 + tests/cli/test_cli.py | 182 ++++++++++++++++++++++++++++++ tests/cli/test_options.py | 190 ++++++++++++++++++++++++++++++++ 7 files changed, 475 insertions(+), 9 deletions(-) create mode 100644 tests/cli/test_options.py diff --git a/docs/integrations/github.md b/docs/integrations/github.md index 328acd0d17..5581e33ee3 100644 --- a/docs/integrations/github.md +++ b/docs/integrations/github.md @@ -59,6 +59,21 @@ jobs: Next checkout the [Core Bot Behavior Configuration Guide](#core-bot-behavior-configuration-guide) to see how to configure the bot's core behavior. Then checkout [Bot Configuration](#bot-configuration) to see how to configure the bot's behavior in general. Finally, checkout [Custom Workflow Configuration](#custom-workflow-configuration) to see the full set of customizations available to the bot. +### Project paths + +The bot shares the `SQLMESH_PATHS` environment variable with the `sqlmesh` CLI. You can use it instead of `-p` in a workflow step: + +```yaml +- name: Run CI/CD Bot + env: + SQLMESH_PATHS: ${{ github.workspace }} + run: sqlmesh_cicd github --token ${{ secrets.GITHUB_TOKEN }} run-all +``` + +An explicit `-p` or `--paths` option replaces the entire environment path list. An unset or empty `SQLMESH_PATHS` uses the existing default project path. Export the variable in the workflow environment before invoking the bot; a project's `.env` file cannot locate the project that contains it. + +Spaces and Unicode characters remain part of each path. For multiple projects, separate paths with `:` on POSIX runners or `;` on Windows runners. Empty path-list segments refer to the current directory. Use repeated `-p` options for directory names that contain the platform's path separator. See the [CLI reference](../reference/cli.md#project-paths) for examples. Automatic command-option variables such as `SQLMESH_PLAN_START` apply to the main `sqlmesh` CLI; the bot shares the explicit `SQLMESH_PATHS` binding. + ## Core Bot Behavior Configuration Guide There are two fundamental ways the bot can be configured: synchronized or desynchronized production code and data. diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 460b6e520b..9dcd5c18d6 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -7,7 +7,7 @@ Usage: sqlmesh [OPTIONS] COMMAND [ARGS]... Options: --version Show the version and exit. - -p, --paths TEXT Path(s) to the SQLMesh config/project. + -p, --paths PATH Path(s) to the SQLMesh config/project. --config TEXT Name of the config object. Only applicable to configuration defined using Python script. --gateway TEXT The name of the gateway. @@ -52,6 +52,65 @@ Commands: lint Run the linter for the target model(s). ``` +## Environment variables + +CLI options accept defaults from environment variables. Use `SQLMESH_` followed by the uppercase command and option names, replacing hyphens with underscores. Include every command name for nested commands. + +| Environment variable | CLI option | +| --- | --- | +| `SQLMESH_PATHS` | `sqlmesh --paths` | +| `SQLMESH_CONFIG` | `sqlmesh --config` | +| `SQLMESH_PLAN_START` | `sqlmesh plan --start` | +| `SQLMESH_TABLE_DIFF_LIMIT` | `sqlmesh table_diff --limit` | +| `SQLMESH_STATE_EXPORT_OUTPUT_FILE` | `sqlmesh state export --output-file` | +| `SQLMESH_TEST_LOCAL` | `sqlmesh test --local` | +| `SQLMESH_LINT_LOCAL` | `sqlmesh lint --local` | + +Options with an explicit environment variable retain their names: `SQLMESH_GATEWAY`, `SQLMESH_IGNORE_WARNINGS`, and `SQLMESH_DOTENV_PATH`. Environment variables do not supply positional arguments, such as the environment in `sqlmesh plan dev`. + +An explicit CLI option takes precedence over its environment variable. For example, `sqlmesh plan --start 2025-02-01` uses that date even if `SQLMESH_PLAN_START` specifies another date. Values use the same type validation as the corresponding CLI option. + +### Project paths + +Set `SQLMESH_PATHS` to select a project when running SQLMesh from another directory: + +```bash +export SQLMESH_PATHS="/work/analytics project" +sqlmesh info +``` + +Spaces and Unicode characters remain part of each path. For multiple projects, separate paths with `:` on POSIX or `;` on Windows. For example, on POSIX: + +```bash +export SQLMESH_PATHS="/work/analytics project:/work/finance α" +sqlmesh info +``` + +In PowerShell, a single project path can be set with: + +```powershell +$env:SQLMESH_PATHS = 'C:\work\analytics project' +sqlmesh info +``` + +Windows path lists use semicolons, preserving drive-letter colons. Shell quotes delimit the assignment and are not part of the value. + +Explicit `-p` or `--paths` options replace the entire environment path list. Repeat the option to select multiple projects: + +```bash +sqlmesh -p "/work/another project" -p "/work/finance α" info +``` + +An unset or empty `SQLMESH_PATHS` uses the existing default project path. Relative paths resolve from the invocation directory. Avoid leading, trailing, or repeated path separators: empty segments are retained and refer to the current directory. If a directory name contains the platform's path separator, pass it with an explicit `-p` option. + +### Dotenv timing and precedence + +Root options, including project paths, are parsed before SQLMesh loads project `.env` files or the file selected by `--dotenv`. Export `SQLMESH_PATHS` before invoking SQLMesh. A project `.env` file cannot select the project that SQLMesh must find to load that file. Other root options, such as `SQLMESH_CONFIG` and `SQLMESH_DOTENV_PATH`, also need to be present before invocation. + +For commands that load project configuration, subcommand options are parsed after dotenv loading. Variables such as `SQLMESH_PLAN_START`, `SQLMESH_TEST_LOCAL`, and `SQLMESH_LINT_LOCAL` can therefore come from the selected project's `.env` file. SQLMesh loads dotenv values with replacement enabled, so they can override values already in the process environment. Explicit CLI options still take precedence. + +CLI option variables are separate from the `SQLMESH__...` variables that configure project settings. See the [configuration reference](./configuration.md#environment-variables). + ## audit ``` diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index ae036a7774..7e79e72bae 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -4,6 +4,14 @@ This page lists SQLMesh configuration options and their parameters. Learn more a Configuration options for model definitions are listed in the [model configuration reference page](./model_configuration.md). +## Environment variables + +Project settings use environment variables beginning with `SQLMESH__`, with double underscores separating configuration fields. For example, `SQLMESH__MODEL_DEFAULTS__DIALECT=snowflake` sets the default model dialect, and `SQLMESH__DEFAULT_GATEWAY=production` sets the project's default gateway. These variables override values in configuration files. See the [configuration guide](../guides/configuration.md#environment-variables) for details. + +CLI option defaults use a single underscore after `SQLMESH`, such as `SQLMESH_PATHS`, `SQLMESH_CONFIG`, or `SQLMESH_PLAN_START`. An explicit CLI option takes precedence over its environment variable. For example, `SQLMESH_GATEWAY` selects the gateway for a command, while `SQLMESH__DEFAULT_GATEWAY` configures the project's default gateway. + +Export root CLI option variables before invoking SQLMesh. Root options are parsed before project `.env` files or a custom `--dotenv` file are loaded, so `SQLMESH_PATHS` inside a project `.env` cannot locate that project. Subcommand option variables can come from the loaded dotenv file, whose values replace existing process environment values. See the [CLI environment-variable reference](./cli.md#environment-variables) for naming, path lists, and precedence. + ## Root configurations A SQLMesh project configuration consists of root level parameters within which other parameters are defined. diff --git a/sqlmesh/cli/main.py b/sqlmesh/cli/main.py index fc6cad8011..7f87a14372 100644 --- a/sqlmesh/cli/main.py +++ b/sqlmesh/cli/main.py @@ -66,7 +66,11 @@ def _sqlmesh_version() -> str: return "0.0.0" -@click.group(cls=_SQLMeshGroup, no_args_is_help=True) +@click.group( + cls=_SQLMeshGroup, + no_args_is_help=True, + context_settings={"auto_envvar_prefix": "SQLMESH"}, +) @click.version_option(version=_sqlmesh_version(), message="%(version)s") @opt.paths @opt.config @@ -131,13 +135,6 @@ def cli( load = True # Local-only gating must hold for any number of --paths, so it stays outside the block below. load_state = ctx.invoked_subcommand not in LOCAL_ONLY_COMMANDS - # The parent callback constructs Context before Click invokes the subcommand, so inspect its - # parsed args here. - if ( - ctx.invoked_subcommand in OPTIONAL_LOCAL_COMMANDS - and "--local" in ctx.meta["subcommand_args"] - ): - load_state = False if len(paths) == 1: path = os.path.abspath(paths[0]) @@ -152,6 +149,19 @@ def cli( load = False configs = load_configs(config, Context.CONFIG_TYPE, paths, dotenv_path=dotenv) + # The subcommand is parsed after this callback. Resolve its local flag after dotenv loading + # so Context avoids state access for the same effective value the subcommand will receive. + if ctx.invoked_subcommand and ctx.invoked_subcommand in OPTIONAL_LOCAL_COMMANDS: + if "--local" in ctx.meta["subcommand_args"]: + load_state = False + else: + command = t.cast(click.Group, ctx.command).get_command(ctx, ctx.invoked_subcommand) + assert command is not None + with click.Context(command, info_name=ctx.invoked_subcommand, parent=ctx) as child: + local = next(param for param in command.params if param.name == "local") + value = local.value_from_envvar(child) + if value is not None: + load_state = not local.type_cast_value(child, value) log_limit = list(configs.values())[0].log_limit remove_excess_logs(log_file_dir, log_limit) diff --git a/sqlmesh/cli/options.py b/sqlmesh/cli/options.py index 2e4642eb0e..751ff0dba0 100644 --- a/sqlmesh/cli/options.py +++ b/sqlmesh/cli/options.py @@ -8,8 +8,10 @@ paths = click.option( "-p", "--paths", + type=click.Path(readable=False), multiple=True, default=[os.getcwd()], + envvar="SQLMESH_PATHS", help="Path(s) to the SQLMesh config/project.", ) diff --git a/tests/cli/test_cli.py b/tests/cli/test_cli.py index f1540727b1..7509a99118 100644 --- a/tests/cli/test_cli.py +++ b/tests/cli/test_cli.py @@ -12,6 +12,7 @@ from click.testing import CliRunner from sqlglot import __version__ as SQLGLOT_VERSION from sqlmesh import RuntimeEnv +from sqlmesh.cli import main as cli_main from sqlmesh._version import __version__ as SQLMESH_VERSION from sqlmesh.cli.project_init import ProjectTemplate, init_example_project from sqlmesh.cli.main import cli @@ -2582,6 +2583,187 @@ def _setup_local_only_project(tmp_path, mocker): return _patch_state_access(mocker) +@pytest.fixture +def cli_env(monkeypatch): + for name in ( + "SQLMESH_PATHS", + "SQLMESH_CONFIG", + "SQLMESH_TEST_LOCAL", + "SQLMESH_LINT_LOCAL", + "SQLMESH_DOTENV_PATH", + "SQLMESH__MODEL_DEFAULTS__DIALECT", + ): + # Register an undo even when absent, since dotenv can add the variable directly. + monkeypatch.setenv(name, "") + monkeypatch.delenv(name) + + +@pytest.mark.parametrize("name", ["project", "analytics project 分析"]) +@pytest.mark.parametrize("relative", [False, True]) +def test_paths_env_selects_project_from_another_directory( + runner, tmp_path, monkeypatch, cli_env, name, relative +): + project = tmp_path / name + _create_local_only_project(project, "env_project") + elsewhere = tmp_path / "elsewhere" + elsewhere.mkdir() + monkeypatch.chdir(elsewhere) + monkeypatch.setenv("SQLMESH_PATHS", os.path.relpath(project) if relative else str(project)) + result = runner.invoke(cli, ["--log-file-dir", str(tmp_path), "info"]) + assert result.exit_code == 0, result.output + assert "Models: 1" in result.output + + +@pytest.mark.parametrize("command", ["test", "lint"]) +@pytest.mark.parametrize("value", ["true", "1", "false", "0", "", "invalid"]) +@pytest.mark.parametrize("explicit", [False, True]) +def test_local_env_state_gating( + runner, tmp_path, mocker, monkeypatch, cli_env, command, value, explicit +): + state = _setup_local_only_project(tmp_path, mocker) + init = mocker.spy(Context, "__init__") + monkeypatch.setenv(f"SQLMESH_{command.upper()}_LOCAL", value) + args = ["--paths", str(tmp_path), command] + if explicit: + args.append("--local") + result = runner.invoke(cli, args) + if value == "invalid" and not explicit: + assert result.exit_code == 2, result.output + assert "valid boolean" in result.output + init.assert_not_called() + state.assert_not_called() + elif explicit or value in ("true", "1"): + assert result.exit_code == 0, result.output + assert init.call_args.kwargs["load_state"] is False + state.assert_not_called() + else: + assert init.call_args.kwargs["load_state"] is True + state.assert_called() + + +@pytest.mark.parametrize("command", ["test", "lint"]) +@pytest.mark.parametrize("custom_dotenv", [False, True]) +@pytest.mark.parametrize("value", ["true", "false"]) +def test_local_env_from_dotenv( + runner, tmp_path, mocker, monkeypatch, cli_env, command, custom_dotenv, value +): + state = _setup_local_only_project(tmp_path, mocker) + init = mocker.spy(Context, "__init__") + variable = f"SQLMESH_{command.upper()}_LOCAL" + # The existing config loader lets dotenv override the process environment. + monkeypatch.setenv(variable, "false" if value == "true" else "true") + env_file = tmp_path / ("custom.env" if custom_dotenv else ".env") + env_file.write_text(f"{variable}={value}\n", encoding="utf-8") + args = ["--paths", str(tmp_path)] + if custom_dotenv: + args.extend(["--dotenv", str(env_file)]) + result = runner.invoke(cli, [*args, command]) + assert init.call_args.kwargs["load_state"] is (value == "false") + if value == "true": + assert result.exit_code == 0, result.output + state.assert_not_called() + else: + state.assert_called() + + +@pytest.mark.parametrize("command", ["test", "lint", "format"]) +def test_paths_env_multiple_projects_local(runner, tmp_path, mocker, monkeypatch, cli_env, command): + projects = (tmp_path / "first project", tmp_path / "second 分析") + for index, project in enumerate(projects): + _create_local_only_project(project, f"project_{index}") + state = _patch_state_access(mocker) + monkeypatch.setenv("SQLMESH_PATHS", os.pathsep.join(str(project) for project in projects)) + if command != "format": + monkeypatch.setenv(f"SQLMESH_{command.upper()}_LOCAL", "true") + result = runner.invoke(cli, [command]) + assert result.exit_code == 0, result.output + state.assert_not_called() + + +def test_paths_env_init_new_destination(runner, tmp_path, monkeypatch, cli_env): + project = tmp_path / "new project" + project.mkdir() + monkeypatch.setenv("SQLMESH_PATHS", str(project)) + result = runner.invoke(cli, ["init", "duckdb", "--template", "empty"]) + assert result.exit_code == 0, result.output + assert (project / "config.yaml").exists() + + +@pytest.mark.parametrize("invalid_config", [False, True]) +def test_paths_env_project_errors(runner, tmp_path, monkeypatch, cli_env, invalid_config): + project = tmp_path / "project" + if invalid_config: + project.mkdir() + (project / "config.yaml").write_text("not a config\n", encoding="utf-8") + monkeypatch.setenv("SQLMESH_PATHS", str(project)) + result = runner.invoke(cli, ["info"]) + assert result.exit_code == 1, result.output + assert ( + "Invalid YAML configuration" if invalid_config else "project config could not be found" + ) in result.output + + +def test_paths_env_in_dotenv_does_not_change_root_paths(runner, tmp_path, monkeypatch, cli_env): + project = tmp_path / "project" + alternate = tmp_path / "alternate" + _create_local_only_project(project, "selected_project") + _create_local_only_project(alternate, "alternate_project") + (alternate / "models" / "second.sql").write_text( + "MODEL(name alternate_project.second); SELECT 2 AS col", encoding="utf-8" + ) + (project / ".env").write_text(f"SQLMESH_PATHS='{alternate}'\n", encoding="utf-8") + monkeypatch.setenv("SQLMESH_PATHS", str(project)) + result = runner.invoke(cli, ["info"]) + assert result.exit_code == 0, result.output + assert "Models: 1" in result.output + + +def test_paths_env_only_in_custom_dotenv_does_not_select_project( + runner, tmp_path, monkeypatch, mocker, cli_env +): + project = tmp_path / "project" + _create_local_only_project(project, "env_project") + env_file = tmp_path / "custom.env" + env_file.write_text(f"SQLMESH_PATHS='{project}'\n", encoding="utf-8") + load_configs = mocker.spy(cli_main, "load_configs") + runner.invoke(cli, ["--dotenv", str(env_file), "info"]) + paths = next(param for param in cli.params if param.name == "paths") + assert load_configs.call_args.args[2] == tuple(paths.default) + + +@pytest.mark.parametrize("empty_segment", [False, True]) +def test_paths_env_preserves_loader_expansion( + runner, tmp_path, monkeypatch, mocker, cli_env, empty_segment +): + projects = [tmp_path / "projects" / "first", tmp_path / "projects" / "second"] + for index, project in enumerate(projects): + _create_local_only_project(project, f"env_project_{index}") + if empty_segment: + invocation = tmp_path / "invocation" + _create_local_only_project(invocation, "invocation") + monkeypatch.chdir(invocation) + env_value = os.pathsep.join([str(projects[0]), "", str(projects[1])]) + expected = {projects[0], invocation, projects[1]} + else: + env_value = str(tmp_path / "projects" / "*") + expected = set(projects) + monkeypatch.setenv("SQLMESH_PATHS", env_value) + init = mocker.spy(Context, "__init__") + result = runner.invoke(cli, ["format"]) + assert result.exit_code == 0, result.output + assert set(init.call_args.kwargs["config"]) == expected + + +def test_cli_env_separate_from_project_config(runner, tmp_path, monkeypatch, mocker, cli_env): + _create_local_only_project(tmp_path, "env_project") + monkeypatch.setenv("SQLMESH_PATHS", str(tmp_path)) + monkeypatch.setenv("SQLMESH__MODEL_DEFAULTS__DIALECT", "spark") + init = mocker.spy(Context, "__init__") + result = runner.invoke(cli, ["info"]) + assert result.exit_code == 0, result.output + assert next(iter(init.call_args.kwargs["config"].values())).model_defaults.dialect == "spark" + + def test_format_runs_without_state(runner: CliRunner, tmp_path: Path, mocker): mock = _setup_local_only_project(tmp_path, mocker) result = runner.invoke(cli, ["--paths", str(tmp_path), "format"]) diff --git a/tests/cli/test_options.py b/tests/cli/test_options.py new file mode 100644 index 0000000000..b72350983f --- /dev/null +++ b/tests/cli/test_options.py @@ -0,0 +1,190 @@ +# SPDX-License-Identifier: Apache-2.0 +from __future__ import annotations + +import os +import click +import pytest +from click.testing import CliRunner + +from sqlmesh.cicd.bot import bot +from sqlmesh.cli.main import cli + + +def _root_context(command=cli, args=()): + return command.make_context("sqlmesh", [*args, "info" if command is cli else "github"]) + + +@pytest.fixture(autouse=True) +def clear_cli_env(monkeypatch): + for name in ( + "SQLMESH_PATHS", + "SQLMESH_CONFIG", + "SQLMESH_GATEWAY", + "SQLMESH_IGNORE_WARNINGS", + "SQLMESH_DOTENV_PATH", + "SQLMESH_DOTENV", + "SQLMESH_PLAN_START", + "SQLMESH_TABLE_DIFF_LIMIT", + "SQLMESH_STATE_EXPORT_OUTPUT_FILE", + "SQLMESH_RENDER_MODEL", + ): + monkeypatch.delenv(name, raising=False) + + +@pytest.mark.parametrize("command", [cli, bot]) +@pytest.mark.parametrize( + "paths", + [("/work/project",), ("relative project",), ("/work/analytics project", "/work/分析")], +) +def test_paths_env(command, paths, monkeypatch): + monkeypatch.setenv("SQLMESH_PATHS", os.pathsep.join(paths)) + with _root_context(command) as ctx: + assert ctx.params["paths"] == paths + + +@pytest.mark.parametrize("command", [cli, bot]) +@pytest.mark.parametrize("option", ["-p", "--paths"]) +def test_paths_cli_overrides_env(command, option, monkeypatch): + monkeypatch.setenv("SQLMESH_PATHS", "/work/environment") + with _root_context(command, [option, "first project", option, "second"]) as ctx: + assert ctx.params["paths"] == ("first project", "second") + + +@pytest.mark.parametrize("value", [None, ""]) +def test_paths_env_default(value, monkeypatch): + if value is not None: + monkeypatch.setenv("SQLMESH_PATHS", value) + paths = next(param for param in cli.params if param.name == "paths") + with _root_context() as ctx: + assert ctx.params["paths"] == tuple(paths.default) + + +def test_paths_env_preserves_globs_and_empty_segments(monkeypatch): + values = ("", "projects/*", "", "last project", "") + monkeypatch.setenv("SQLMESH_PATHS", os.pathsep.join(values)) + with _root_context() as ctx: + assert ctx.params["paths"] == values + + +def test_paths_env_does_not_require_existing_destination(tmp_path, monkeypatch): + destination = tmp_path / "new project" + monkeypatch.setenv("SQLMESH_PATHS", str(destination)) + with cli.make_context("sqlmesh", ["init", "duckdb"]) as ctx: + assert ctx.params["paths"] == (str(destination),) + assert not destination.exists() + + +@pytest.mark.parametrize( + "separator, value, expected", + [ + (":", "/work/first project:/work/second", ["/work/first project", "/work/second"]), + ( + ";", + r"C:\work\first project;D:\work\second", + [r"C:\work\first project", r"D:\work\second"], + ), + ], +) +def test_paths_env_platform_separator(separator, value, expected, monkeypatch): + paths = next(param for param in cli.params if param.name == "paths") + monkeypatch.setattr(paths.type, "envvar_list_splitter", separator) + assert paths.type.split_envvar_value(value) == expected + + +def test_root_config_env_and_cli_priority(monkeypatch): + monkeypatch.setenv("SQLMESH_CONFIG", "environment_config") + with _root_context() as ctx: + assert ctx.params["config"] == "environment_config" + with _root_context(args=["--config", "explicit_config"]) as ctx: + assert ctx.params["config"] == "explicit_config" + + +@pytest.mark.parametrize( + "variable, args, name, value", + [ + ("SQLMESH_GATEWAY", ["--gateway", "explicit"], "gateway", "environment"), + ("SQLMESH_IGNORE_WARNINGS", ["--ignore-warnings"], "ignore_warnings", "false"), + ], +) +def test_existing_explicit_env_names(variable, args, name, value, monkeypatch): + monkeypatch.setenv(variable, value) + with _root_context() as ctx: + assert ctx.params[name] == (False if value == "false" else value) + with _root_context(args=args) as ctx: + assert ctx.params[name] == (True if value == "false" else "explicit") + + +def test_existing_dotenv_env_name(tmp_path, monkeypatch): + env_file = tmp_path / "custom.env" + env_file.touch() + explicit = tmp_path / "explicit.env" + explicit.touch() + monkeypatch.setenv("SQLMESH_DOTENV_PATH", str(env_file)) + monkeypatch.setenv("SQLMESH_DOTENV", str(tmp_path / "ignored.env")) + with _root_context() as ctx: + assert ctx.params["dotenv"] == env_file + with _root_context(args=["--dotenv", str(explicit)]) as ctx: + assert ctx.params["dotenv"] == explicit + + +@pytest.mark.parametrize( + "command, variable, name, value, expected, args", + [ + ("plan", "SQLMESH_PLAN_START", "start", "2020-01-01", "2020-01-01", []), + ("table_diff", "SQLMESH_TABLE_DIFF_LIMIT", "limit", "12", 12, ["source:target", "model"]), + ], +) +def test_command_env_prefix(command, variable, name, value, expected, args, monkeypatch): + monkeypatch.setenv(variable, value) + with _root_context() as parent: + child_command = cli.get_command(parent, command) + with child_command.make_context(command, list(args), parent=parent) as child: + assert child.params[name] == expected + with child_command.make_context(command, [*args, f"--{name}", "3"], parent=parent) as child: + assert child.params[name] == (3 if isinstance(expected, int) else "3") + + +def test_command_env_invalid_integer(monkeypatch): + monkeypatch.setenv("SQLMESH_TABLE_DIFF_LIMIT", "invalid") + with _root_context() as parent: + command = cli.get_command(parent, "table_diff") + with pytest.raises(click.BadParameter, match="valid integer"): + command.make_context("table_diff", ["source:target", "model"], parent=parent) + + +def test_nested_command_env_prefix(tmp_path, monkeypatch): + output_file = tmp_path / "state.json" + monkeypatch.setenv("SQLMESH_STATE_EXPORT_OUTPUT_FILE", str(output_file)) + with _root_context() as parent: + state = cli.get_command(parent, "state") + with state.make_context("state", ["export"], parent=parent) as state_ctx: + command = state.get_command(state_ctx, "export") + with command.make_context("export", [], parent=state_ctx) as child: + assert child.params["output_file"] == output_file + + +def test_positional_arguments_have_no_auto_env_binding(monkeypatch): + monkeypatch.setenv("SQLMESH_RENDER_MODEL", "some_model") + with _root_context() as parent: + command = cli.get_command(parent, "render") + with pytest.raises(click.MissingParameter): + command.make_context("render", [], parent=parent) + + +@pytest.mark.parametrize("explicit", [False, True]) +def test_bot_shared_paths_reach_command_boundary(tmp_path, monkeypatch, explicit): + monkeypatch.setenv("SQLMESH_PATHS", str(tmp_path / "environment project")) + captured = {} + + @click.command() + @click.pass_obj + def capture(obj): + captured.update(obj) + + monkeypatch.setitem(bot.commands, "capture", capture) + args = ["-p", str(tmp_path / "explicit project")] if explicit else [] + result = CliRunner().invoke(bot, [*args, "capture"]) + assert result.exit_code == 0, result.output + assert captured["paths"] == ( + str(tmp_path / ("explicit project" if explicit else "environment project")), + ) From 613b801eb94dfce87a34723c7babdaf81bed1611 Mon Sep 17 00:00:00 2001 From: Vamsi-klu Date: Sun, 18 Oct 2026 02:01:26 +0000 Subject: [PATCH 2/2] docs(cli): clarify environment variable option names Signed-off-by: Vamsi-klu --- docs/reference/cli.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 9dcd5c18d6..e2f753c8b2 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -54,13 +54,17 @@ Commands: ## Environment variables -CLI options accept defaults from environment variables. Use `SQLMESH_` followed by the uppercase command and option names, replacing hyphens with underscores. Include every command name for nested commands. +CLI options accept defaults from environment variables. Use `SQLMESH_` followed by the command names and the option's Click parameter name, all uppercase with hyphens replaced by underscores. Include every command name for nested commands. The parameter name usually matches the long option, but some differ, as shown below. Aliases share the same environment variable. | Environment variable | CLI option | | --- | --- | | `SQLMESH_PATHS` | `sqlmesh --paths` | | `SQLMESH_CONFIG` | `sqlmesh --config` | | `SQLMESH_PLAN_START` | `sqlmesh plan --start` | +| `SQLMESH_PLAN_SKIP_BACKFILL` | `sqlmesh plan --skip-backfill` or `sqlmesh plan --dry-run` | +| `SQLMESH_AUDIT_MODELS` | `sqlmesh audit --model` | +| `SQLMESH_CREATE_TEST_QUERIES` | `sqlmesh create_test --query` | +| `SQLMESH_CREATE_TEST_VARIABLES` | `sqlmesh create_test --var` | | `SQLMESH_TABLE_DIFF_LIMIT` | `sqlmesh table_diff --limit` | | `SQLMESH_STATE_EXPORT_OUTPUT_FILE` | `sqlmesh state export --output-file` | | `SQLMESH_TEST_LOCAL` | `sqlmesh test --local` |