Repository navigation
Conversation
…pport. Introduces the ClickHouse JDBC module following the Trino/Postgres plugin pattern so filter/projection SQL can be pushed down, as the first slice of apache#871.
Resolve the wayang-platforms/pom.xml conflict by keeping both wayang-duckdb and wayang-clickhouse modules.
…nd docs. Adds join/reduce/sort/table-sink mappings, ClickHouseOperatorsIT, README, and a wayang-applications demo so apache#871 is covered in this PR.
Preserve custom ENGINE across createCopy, clarify that the engine key is a TableSink property, document ClickHouseOperatorsIT, and scale fixtures with CROSS JOIN numbers().
Bump clickhouse-jdbc to 0.11.1, whose shaded jar relocates ANTLR, and exclude its transitive dependencies. The 0.9.0 jar shipped an unrelocated ANTLR 4.13 that shadowed Wayang's 4.9.3 runtime and broke the mathex parser. Always pass the password property (the driver requires it even when empty) and alias the numbers() table function in the fixture SQL, so the IT actually runs against a passwordless ClickHouse instead of silently skipping.
Probe the connection with SELECT 1 first and skip the class only if that fails. Fixture statements now run outside the catch, so a broken fixture fails the run instead of being reported as a skip.
ClickHouseMappingsTest applies every mapping to a logical plan and checks the replacement operator, including the cases that must not be mapped. ClickHouseOperatorsTest covers copies, channels, the sink ENGINE fallbacks and that the defaults file provides a cost model for every operator. ClickHouseOperatorsIT now reads UInt64, Date, DateTime, Nullable and LowCardinality columns into Records; the README documents the Java types.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
wayang-platforms/wayang-clickhouseas a JDBC platform module (Trino-style layout).CREATE TABLE ... ENGINE = MergeTree ORDER BY tuple() AS SELECT .... The engine can be overridden per sink by settingwayang.clickhouse.tablesink.engineon theTableSink/writeTableProperties.com.clickhouse:clickhouse-jdbc:0.11.1(allclassifier, transitive dependencies excluded). This release relocates its bundled ANTLR, so it does not clash with the ANTLR 4.9.3 runtime used by Wayang core.ClickHouseOperatorsITskips when no server is reachable.ClickHouseDemo+clickhouse.mdinwayang-applications, and docs index entry.main(keeps bothwayang-duckdbandwayang-clickhouse).Closes #871.
Test plan
./mvnw -Pskip-prerequisite-check -pl wayang-platforms/wayang-clickhouse -am verifyon JDK 17: build, RAT licence check and 41 unit tests pass.ClickHouseOperatorsITagainst a live ClickHouse 24.12.6 (CLICKHOUSE_HOST/CLICKHOUSE_PORT, passwordlessdefaultuser): 11/11 pass, covering all eight operators, twoJavaPlanBuilderplans and the ClickHouse-to-Java type mapping (UInt64,Date,DateTime,Nullable,LowCardinality).ClickHouseDemorun as documented inwayang-applications/clickhouse.md.Test results
ClickHousePlatformTestClickHousePluginTestClickHouseSqlGenerationTestClickHouseTableSinkCopyTestClickHouseMappingsTestClickHouseOperatorsTestClickHouseOperatorsIT(live ClickHouse 24.12.6)SQL executed by the integration test, as logged by
JdbcExecutor:Coverage
JaCoCo 0.8.16 over
org.apache.wayang.clickhouse.*(22 classes):ClickHouseOperatorsIT(52)The three uncovered lines are the implicit constructors of the static holder classes
ClickHouse,MappingsandChannelConversions.Notes:
*Test/*Tests, soClickHouseOperatorsITis not run bymvn verifyin CI (same asTrinoOperatorsIT/PrestoOperatorsIT); run it with-Dtest=ClickHouseOperatorsIT.JAVA_TOOL_OPTIONS, because the surefireargLinein the root pom replaces the one set byjacoco:prepare-agent.