Repository navigation
Fix JDBC resource leaks in SqlToStreamOperator - #866
Kaileshwar16 wants to merge 2 commits into
Conversation
| output.accept(resultSetStream); | ||
| // Terminal stream operations do not close streams. Keep a job-level fallback for | ||
| // short-circuiting consumers, without changing StreamChannel's ownership semantics. | ||
| CrossPlatformExecutor crossPlatformExecutor = executor.getCrossPlatformExecutor(); |
There was a problem hiding this comment.
why do we need to call the executor within an operator? this seems odd
There was a problem hiding this comment.
I understand the concern about coupling the operator directly to CrossPlatformExecutor, I added this so JDBC resources get cleaned up when the job ends, even if downstream processing stops early. Stream operations don’t automatically close the stream, and disposing the channel only clears its reference.
I also tried registering cleanup with JavaExecutor alone, but live channels can prevent it from being disposed after a downstream failure. Both new regression cases failed with that approach. I will add tests and have the comment more precise. is it justifed to add it? do prefer anyother way to solve it
|
@zkaoudi currently i have added tests and clarified the comment Would you prefer exposing a resource-registration method through the execution layer so the operator doesn’t access CrossPlatformExecutor directly? |
related #853
Ensure ResultSet, Statement, and Connection resources are closed during normal execution, failures, explicit stream closure, and early termination. Adds focused regression tests for JDBC cleanup and preserves existing fetch-size and autocommit behavior.
i will do the followup with the pr, once the intentional behaviour for Fetch size configuration is confirmed