Add support for -f/--file option to execute SQL from files

[DiegoDAF]

Summary

This PR adds support for the -f/--file option to pgcli, implementing psql-compatible behavior for executing SQL commands from files.

Features

• Single file execution: pgcli -f file.sql
• Multiple files: pgcli -f file1.sql -f file2.sql
• Long form support: pgcli --file file.sql
• Files are executed sequentially
• Pager is automatically disabled in file mode
• Proper error handling and exit codes

Implementation Details

• Added new click option -f/--file that accepts multiple file paths
• File paths are validated to ensure they exist and are readable
• Modified run_cli() to check for file mode and execute file contents before entering interactive mode
• Updated echo_via_pager() to disable pager when in file mode
• File contents are read and executed using the existing handle_watch_command() method

Testing

Comprehensive BDD tests included covering:

• Single file with -f flag
• Single file with --file flag
• Multiple statements in one file
• Multiple -f options for different files
• Files with special commands
• Error handling for invalid SQL

Compatibility

This implementation follows psql's behavior and maintains backward compatibility with existing functionality.

Made with ❤️ and 🤖 Claude Code

---

Part of the feature list in discussion #1603: this is item 7 (-c/--command and -f/--file), the -f half. The -c half is #1542.

[j-bennet]

This looks ready to merge, however, I'm wondering, what will happen if the file contains a destructive command (DROP table_name) and destructive_warning is enabled?

[DiegoDAF]

This looks ready to merge, however, I'm wondering, what will happen if the file contains a destructive command (DROP table_name) and destructive_warning is enabled?

Thanks for the review!

Short answer: -f doesn't change the destructive-command behavior at all. The file's contents go through the same execute_command path as interactive input, so destructive_warning applies exactly as it does today.

Concretely, with destructive_warning enabled and a file containing DROP table_name:

• Interactive terminal: the user gets the usual "You're about to run a destructive command. Do you want to proceed?" prompt. Answering no stops execution ("Wise choice!") and nothing from the file runs.
• Non-interactive stdin (piped, redirected, CI): confirm_destructive_query already returns None because of its existing sys.stdin.isatty() guard, so no prompt is shown and the statement runs. That is pre-existing behaviour, not something -f
  introduces: the existing \i command (execute_from_file) behaves the same way.
• If destructive_statements_require_transaction is enabled, the statement is still refused outside a transaction, with Destructive statements must be run within a transaction."

One detail worth calling out: -f executes the file as a single unit (the whole content is passed to execute_command), so the destructive check runs once over the entire file rather than per statement. If any statement in the file matches destructive_warning, the single confirmation covers the whole file.

Happy to add a behave scenario covering the destructive-file case if you'd like it
pinned down in tests.

[j-bennet]

There seems to be a persistent failure in the integration scenario: Scenario: edit sql in file with external editor. Does that pass for you locally?

[DiegoDAF]

There seems to be a persistent failure in the integration scenario: Scenario: edit sql in file with external editor. Does that pass for you locally?

Is the same flake has hit other PRs recently (#1544 and #1609 saw it on different py versions and went green on the next run), so a re-run of the failed job should clear it, I don't have permission to re-trigger it myself.

If it keeps biting, I'd be happy to send a tiny separate PR bumping that expect_exact timeout (2s -> 10s); it only affects how long the test waits, not how fast it passes.

[DiegoDAF]

Sorry for the slow reply, and thanks for your patience!

Yes, it passes locally. I dug into it, and that scenario is flaky rather than
broken: tests/features/steps/iocommands.py waits for the ex-mode banner with
expect_exact(..., timeout=2), and two seconds is not always enough on a loaded
CI runner. When it expires, behave reports it as an "error" (not a failure), and
fail-fast then cancels the sibling matrix jobs, which makes it look like a
systematic breakage.

Evidence that it is timing and not this PR:

• The unit suite and the rest of the integration suite pass in the same run
  (2718 unit tests, 27 behave scenarios, 0 failed).
• It has hit different Python versions on different runs, and other PRs too
  (Add support for -y/--yes flag to bypass confirmation prompts #1544 and Add \ne to edit a named query in the external editor #1609 saw it and went green on a later run without any code change).
• It also started hitting my own fork's CI on commits that touch nothing near
  the editor code.

I bumped those expect_exact timeouts from 2s (and the 1s/5s ones in the same
file) to 10s in my fork and the scenario has been stable since. It does not slow
the passing path down: pexpect returns as soon as the expected text shows up, so
the timeout only bounds how long a failing wait blocks.

Happy to send that as a separate one-line-ish PR against main if you want it
fixed for everyone; it is test-only and independent of this PR. Just say the
word and I will open it.

[j-bennet]

Why are you using handle_watch_command here, rather than execute_command?

[DiegoDAF]

handle_watch_command is what the interactive loop calls (main.py:1081 on main), so -f takes exactly the same path as typing the file's contents at the prompt. -c in #1542 does the same, for consistency between the two.

Concretely it adds two things over execute_command:

• \watch works inside a file, the way it does in psql -f.
• The query is appended to self.query_history, which a bare \watch (no command) relies on to pick up the previous query.

Calling execute_command directly would silently drop both. Happy to switch if you would rather -f not support \watch, but then I would make -c match.

[j-bennet]

I see. If you want to handle \watch inside a file, that's fine.

I tested the changes briefly. It looks like files with multiple lines aren't handled as expected.

Try this example:

-- save this as ~/tmp/select3.txt
select 1;
select 'hello';
select 2; \watch 2

then if you run psql -f ~/tmp/select3.txt, it will output 1, then hello, then start repeating 2.

However, pgcli -f ~/tmp/select3.txt will start repeating the whole block.

[DiegoDAF]

Good catch, thank you for testing it. Fixed.

The file now goes through sqlparse.split() and runs one statement at a time, each through the same path as the interactive prompt. So \watch repeats only its own statement, and a bare \watch picks up the statement before it through the query history. Your example now behaves exactly like psql -f: 1, hello, then only 2 repeating.

A failed statement stops the rest of the file unless on_error is RESUME, which matches what the single-block execution already did internally.

7 new tests, 5 of which fail without the fix (including one asserting that the repeated output contains only the last statement, and one checking that semicolons inside string literals are not treated as statement boundaries).

[j-bennet]

How does sqlparse.split() handle metacommands?

[DiegoDAF]

The same way the code already did, plus one rule it was missing. pgexecute.run() already splits its input with sqlparse.split() internally, so metacommands were going through this exact splitter before the fix too.

But your question made me check the edge, and there was a real pre-existing gap: sqlparse.split only cuts at semicolons, so a metacommand followed by SQL on the next line traveled as one chunk, and the metacommand swallowed the SQL. \echo hi followed by a select turned the select into echo arguments, in a file, on main as well. Interactively it never shows because the buffer submits as soon as it starts with a backslash.

Fixed in the last commit with psql's rule: a backslash command spans only its own line, and the rest of the chunk goes back through the splitter. 3 more tests, all failing without it. Verified against a live server that \d between statements, \dt; with a trailing semicolon, and consecutive metacommands behave identically to the interactive prompt.