Skip to content

Fix JDBC resource leaks in SqlToStreamOperator - #866

Open
Kaileshwar16 wants to merge 2 commits into
apache:mainfrom
Kaileshwar16:fix/853-jdbc-resource-leaks
Open

Kaileshwar16 wants to merge 2 commits into
apache:mainfrom
Kaileshwar16:fix/853-jdbc-resource-leaks

Conversation

@Kaileshwar16

Copy link
Copy Markdown
Contributor

related #853
Ensure ResultSet, Statement, and Connection resources are closed during normal execution, failures, explicit stream closure, and early termination. Adds focused regression tests for JDBC cleanup and preserves existing fetch-size and autocommit behavior.

i will do the followup with the pr, once the intentional behaviour for Fetch size configuration is confirmed

output.accept(resultSetStream);
// Terminal stream operations do not close streams. Keep a job-level fallback for
// short-circuiting consumers, without changing StreamChannel's ownership semantics.
CrossPlatformExecutor crossPlatformExecutor = executor.getCrossPlatformExecutor();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need to call the executor within an operator? this seems odd

@Kaileshwar16 Kaileshwar16 Nov 11, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the concern about coupling the operator directly to CrossPlatformExecutor, I added this so JDBC resources get cleaned up when the job ends, even if downstream processing stops early. Stream operations don’t automatically close the stream, and disposing the channel only clears its reference.

I also tried registering cleanup with JavaExecutor alone, but live channels can prevent it from being disposed after a downstream failure. Both new regression cases failed with that approach. I will add tests and have the comment more precise. is it justifed to add it? do prefer anyother way to solve it

@Kaileshwar16
Kaileshwar16 requested a review from zkaoudi November 11, 2026 13:20
@Kaileshwar16

Copy link
Copy Markdown
Contributor Author

@zkaoudi currently i have added tests and clarified the comment Would you prefer exposing a resource-registration method through the execution layer so the operator doesn’t access CrossPlatformExecutor directly?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants