From 3c62005ff45c8c2f907b322a7943d94c8fe93957 Mon Sep 17 00:00:00 2001 From: gongzhongqiang Date: Tue, 6 Oct 2026 18:41:19 +0800 Subject: [PATCH 1/4] feat(benchmark): add JMH benchmark module Adds a fesod-benchmark module with JMH benchmarks for Fesod, kept to pure JMH classes per review feedback: no CI integration, no report or analysis code. Closes #572. Suites (org.apache.fesod.sheet.benchmark): - FesodBenchmark: read/write x XLSX/CSV x 1K/10K/100K rows (average time, 3 forks, fixed 1g G1 heap, stable execution contract) - FesodVsPoiBenchmark: Fesod vs Apache POI on identical data (write, DOM read, batched streaming read; XLSX + XLS) - deterministic data generation (fixed seed + fixed date anchor) and unit tests for the generator and file utilities Benchmarks run manually via the shaded benchmarks.jar; see fesod-benchmark/benchmark.md. --- fesod-benchmark/benchmark.md | 103 ++++++ fesod-benchmark/pom.xml | 226 +++++++++++++ .../benchmark/BenchmarkConfiguration.java | 74 +++++ .../fesod/sheet/benchmark/FesodBenchmark.java | 152 +++++++++ .../sheet/benchmark/FesodVsPoiBenchmark.java | 302 ++++++++++++++++++ .../sheet/benchmark/StreamingBenchmark.java | 298 +++++++++++++++++ .../sheet/benchmark/data/BenchmarkData.java | 283 ++++++++++++++++ .../benchmark/util/BenchmarkFileUtil.java | 77 +++++ .../sheet/benchmark/util/DataGenerator.java | 215 +++++++++++++ .../benchmark/util/BenchmarkFileUtilTest.java | 47 +++ .../benchmark/util/DataGeneratorTest.java | 103 ++++++ pom.xml | 1 + 12 files changed, 1881 insertions(+) create mode 100644 fesod-benchmark/benchmark.md create mode 100644 fesod-benchmark/pom.xml create mode 100644 fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/BenchmarkConfiguration.java create mode 100644 fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/FesodBenchmark.java create mode 100644 fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/FesodVsPoiBenchmark.java create mode 100644 fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/StreamingBenchmark.java create mode 100644 fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/data/BenchmarkData.java create mode 100644 fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/BenchmarkFileUtil.java create mode 100644 fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/DataGenerator.java create mode 100644 fesod-benchmark/src/test/java/org/apache/fesod/sheet/benchmark/util/BenchmarkFileUtilTest.java create mode 100644 fesod-benchmark/src/test/java/org/apache/fesod/sheet/benchmark/util/DataGeneratorTest.java diff --git a/fesod-benchmark/benchmark.md b/fesod-benchmark/benchmark.md new file mode 100644 index 000000000..fc3b476fa --- /dev/null +++ b/fesod-benchmark/benchmark.md @@ -0,0 +1,103 @@ +# Fesod Benchmark Module + +JMH benchmarks for Fesod spreadsheet operations. This module is a **manual tool**: +it is not wired into any CI workflow and is not part of the Fesod public API. + +It covers three complementary suites (package `org.apache.fesod.sheet.benchmark`): + +| Suite | What it measures | +|---|---| +| `FesodBenchmark` | The core hot paths — **read** and **write**, XLSX and CSV, 1K/10K/100K rows × 20 columns, average time per op | +| `FesodVsPoiBenchmark` | Plain-API comparison with **Apache POI** on the same data: write (`ExcelWriter` vs XSSF/HSSF `Workbook`) and read (Fesod streaming reader vs POI's DOM-based `WorkbookFactory`); XLSX and XLS (XLS truncated to 65,534 data rows) | +| `StreamingBenchmark` | Streaming-API comparison for the **large-file** scenario, XLSX only: **read** — Fesod streaming reader vs POI **SAX event model** (`XSSFReader` + `XSSFSheetXMLHandler`); **write** — Fesod batched `ExcelWriter` vs POI **SXSSF** sliding-window workbook; 1K/10K/100K rows | + +Shared support code: `BenchmarkConfiguration` (dataset sizes, file formats), +`data.BenchmarkData` (the 20-column data model), `util.DataGenerator` +(fixed-seed, date-anchored — same seed always produces the same rows) and +`util.BenchmarkFileUtil` (scratch files live under `target/benchmark-testdata`, +which `mvn clean` removes). + +## Building and running + +Build the shaded benchmark jar, then use the standard JMH launcher: + +```bash +./mvnw -B -ntp -pl fesod-benchmark -am package -DskipTests + +# Whole core suite (all sizes × formats) +java -jar fesod-benchmark/target/benchmarks.jar FesodBenchmark + +# One slice, e.g. LARGE XLSX reads only +java -jar fesod-benchmark/target/benchmarks.jar 'FesodBenchmark\.read$' \ + -p datasetSize=LARGE -p fileFormat=XLSX + +# Fesod vs POI, XLSX only +java -jar fesod-benchmark/target/benchmarks.jar FesodVsPoiBenchmark -p fileFormat=XLSX + +# Streaming read for the large-file scenario: Fesod vs POI SAX event model +java -jar fesod-benchmark/target/benchmarks.jar StreamingBenchmark + +# Add allocation-per-op (bytes/op) — the most direct memory-efficiency signal +java -jar fesod-benchmark/target/benchmarks.jar FesodBenchmark -prof gc +``` + +Maven shortcuts: + +```bash +# Full run of a suite via the exec plugin +./mvnw -B -ntp -pl fesod-benchmark verify -Pbenchmark '-Dbenchmark.pattern=FesodBenchmark' + +# Quick smoke run (SMALL/CSV, 1 fork, minimal iterations) — sanity check after changes +./mvnw -B -ntp -pl fesod-benchmark verify -Pbenchmark-test +``` + +## Execution contracts + +The annotation contracts are part of each suite and should be kept stable so that +results collected at different times stay comparable: + +| Suite | Mode | Forks | Warmup | Measurement | JVM args | +|---|---|---|---|---|---| +| `FesodBenchmark` | avgt, ms/op | 3 | 3 × 1s | 5 × 2s | `-Xms1g -Xmx1g -XX:+UseG1GC` | +| `FesodVsPoiBenchmark` | avgt, ms/op | 1 | 3 × 5s | 5 × 5s | `-Xms2g -Xmx2g` | +| `StreamingBenchmark` | avgt, ms/op | 1 | 3 × 5s | 5 × 5s | `-Xms2g -Xmx2g` | + +## Notes on interpreting results + +- **Environment matters.** Time measurements fluctuate with the host (CPU, + contention, disk). Compare runs from the same machine and similar load, and + trust JMH's error bars (`score ± error`): overlapping intervals between two + runs mean "no observable difference". +- **`-prof gc` allocation per op is the most stable signal** (typically ±0.1% + run-to-run on the same machine). For Fesod's *streaming without OOM* promise, + allocation per op should stay linear in the number of rows and independent of + file size — an accidental full-file buffering shows up here as an + orders-of-magnitude jump long before timing changes. +- **LARGE (100K rows) is where streaming behavior is actually exercised.** At + 1K/10K rows even a non-streaming implementation looks fine; memory + characteristics only separate at larger sizes. +- **Read-path workload symmetry (vs-POI suites).** The write comparisons are + symmetric — both libraries convert the same objects into the same 20 columns. + The read comparisons are deliberately asymmetric, mirroring each library's + idiomatic end-to-end usage: Fesod runs its full pipeline (parse + type + conversion such as `BigDecimal`/dates + reflective row-object mapping into + `BenchmarkData`), while the POI side consumes raw cell text — POI has no + built-in row-object mapping, and hand-rolling one in the benchmark would + measure benchmark code rather than either library. The asymmetry is + conservative for Fesod: the POI side does strictly less work per row, so the + comparison cannot overstate Fesod's advantage. +- **Data is deterministic**: fixed seed and a fixed date anchor + (`2024-01-01`) mean any difference between two runs comes from the code or + the environment, not from the data. + +## Extending + +- New benchmark methods are added like any JMH benchmark (`@Benchmark` method + + `@Param`s). Keep the class-level contract annotations unchanged when adding + to an existing suite; create a new suite class if a different contract is + needed. +- If a benchmark writes files, write them through `BenchmarkFileUtil` so they + land in `target/benchmark-testdata`. Write benchmarks target a fixed file + (overwritten each invocation) and delete it from the trial `@TearDown` via + `BenchmarkFileUtil.delete` — never inside the measured method, so cleanup + I/O stays out of the reported time. diff --git a/fesod-benchmark/pom.xml b/fesod-benchmark/pom.xml new file mode 100644 index 000000000..6ddb3f66a --- /dev/null +++ b/fesod-benchmark/pom.xml @@ -0,0 +1,226 @@ + + + + + 4.0.0 + + + org.apache.fesod + fesod-parent + ${revision} + ../pom.xml + + + fesod-benchmark + fesod-benchmark + JMH benchmarks measuring Fesod read/write performance and comparing it with Apache POI + + + 1.37 + 3.1.0 + benchmarks + + + + + org.apache.fesod + fesod-sheet + ${project.version} + + + + + org.openjdk.jmh + jmh-core + ${jmh.version} + + + org.openjdk.jmh + jmh-generator-annprocess + ${jmh.version} + provided + + + + + org.apache.poi + poi + + + org.apache.poi + poi-ooxml + + + + + org.slf4j + slf4j-api + + + org.slf4j + slf4j-simple + + + + + org.junit.jupiter + junit-jupiter + test + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + + ${maven.compiler.source} + ${maven.compiler.target} + + + org.openjdk.jmh + jmh-generator-annprocess + ${jmh.version} + + + + + + + org.apache.maven.plugins + maven-shade-plugin + + + package + + shade + + + ${uberjar.name} + + + org.openjdk.jmh.Main + + + + + + + *:* + + META-INF/*.SF + META-INF/*.DSA + META-INF/*.RSA + + + + + + + + + + + + + + benchmark + + .* + + + + + org.codehaus.mojo + exec-maven-plugin + ${exec-maven-plugin.version} + + + run-benchmarks + integration-test + + java + + + org.openjdk.jmh.Main + + ${benchmark.pattern} + + + + + + + + + + + + benchmark-test + + + + org.codehaus.mojo + exec-maven-plugin + ${exec-maven-plugin.version} + + + run-benchmark-smoke-test + integration-test + + exec + + + java + + -cp + target/benchmarks.jar + org.openjdk.jmh.Main + FesodBenchmark\. + -f + 1 + -wi + 1 + -w + 1s + -i + 2 + -r + 1s + -p + datasetSize=SMALL + -p + fileFormat=CSV + + + + + + + + + + diff --git a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/BenchmarkConfiguration.java b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/BenchmarkConfiguration.java new file mode 100644 index 000000000..8c77524e1 --- /dev/null +++ b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/BenchmarkConfiguration.java @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.fesod.sheet.benchmark; + +/** + * Shared constants for the benchmark suites. + */ +public final class BenchmarkConfiguration { + + private BenchmarkConfiguration() {} + + /** + * Dataset sizes used by the benchmark suites. Rows have 20 columns each + * (see {@link org.apache.fesod.sheet.benchmark.data.BenchmarkData}). + */ + public enum DatasetSize { + SMALL(1_000, "1K"), + MEDIUM(10_000, "10K"), + LARGE(100_000, "100K"); + + private final int rowCount; + private final String label; + + DatasetSize(int rowCount, String label) { + this.rowCount = rowCount; + this.label = label; + } + + public int getRowCount() { + return rowCount; + } + + public String getLabel() { + return label; + } + } + + /** + * File formats covered by the benchmark suites. Note that XLS is limited + * to 65,536 rows per sheet. + */ + public enum FileFormat { + XLSX("xlsx"), + XLS("xls"), + CSV("csv"); + + private final String extension; + + FileFormat(String extension) { + this.extension = extension; + } + + public String getExtension() { + return extension; + } + } +} diff --git a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/FesodBenchmark.java b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/FesodBenchmark.java new file mode 100644 index 000000000..c8fd4b9da --- /dev/null +++ b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/FesodBenchmark.java @@ -0,0 +1,152 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.fesod.sheet.benchmark; + +import java.io.File; +import java.util.List; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicLong; +import org.apache.fesod.sheet.ExcelReader; +import org.apache.fesod.sheet.FesodSheet; +import org.apache.fesod.sheet.benchmark.data.BenchmarkData; +import org.apache.fesod.sheet.benchmark.util.BenchmarkFileUtil; +import org.apache.fesod.sheet.benchmark.util.DataGenerator; +import org.apache.fesod.sheet.context.AnalysisContext; +import org.apache.fesod.sheet.read.listener.ReadListener; +import org.openjdk.jmh.annotations.Benchmark; +import org.openjdk.jmh.annotations.BenchmarkMode; +import org.openjdk.jmh.annotations.Fork; +import org.openjdk.jmh.annotations.Level; +import org.openjdk.jmh.annotations.Measurement; +import org.openjdk.jmh.annotations.Mode; +import org.openjdk.jmh.annotations.OutputTimeUnit; +import org.openjdk.jmh.annotations.Param; +import org.openjdk.jmh.annotations.Scope; +import org.openjdk.jmh.annotations.Setup; +import org.openjdk.jmh.annotations.State; +import org.openjdk.jmh.annotations.TearDown; +import org.openjdk.jmh.annotations.Warmup; +import org.openjdk.jmh.infra.Blackhole; + +/** + * Core Fesod benchmark suite: read and write, the two hot paths of the library. + * + * + * + *

The execution contract (forks, warmup, measurement, fixed JVM args) is part of the + * class annotations and should be kept stable so that results collected at different + * times remain comparable. Test data is generated with a fixed seed (see + * {@link DataGenerator}) for reproducibility. Run it through the shaded + * {@code benchmarks.jar} as described in {@code fesod-benchmark/benchmark.md}. + */ +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +@Warmup(iterations = 3, time = 1) +@Measurement(iterations = 5, time = 2) +@Fork( + value = FesodBenchmark.FORKS, + jvmArgs = {"-Xms1g", "-Xmx1g", "-XX:+UseG1GC"}) +public class FesodBenchmark { + + /** Fork count used when the suite is executed directly through the JMH launcher. */ + public static final int FORKS = 3; + + @Param({"SMALL", "MEDIUM", "LARGE"}) + private String datasetSize; + + @Param({"XLSX", "CSV"}) + private String fileFormat; + + private List data; + private File readFile; + private File writeFile; + private String extension; + + @Setup(Level.Trial) + public void setupTrial() { + BenchmarkConfiguration.DatasetSize size = BenchmarkConfiguration.DatasetSize.valueOf(datasetSize); + extension = BenchmarkConfiguration.FileFormat.valueOf(fileFormat).getExtension(); + data = DataGenerator.generateTestData(size); + + readFile = BenchmarkFileUtil.createTestFile( + String.format("fesod_read_%s_%s.%s", datasetSize.toLowerCase(), extension, extension)); + FesodSheet.write(readFile, BenchmarkData.class).sheet("Sheet1").doWrite(data); + + // Fixed target for the write benchmark: every invocation overwrites the same + // file, and cleanup happens once in tearDownTrial so no I/O other than the + // write itself is part of the measured op. + writeFile = BenchmarkFileUtil.createTestFile( + String.format("fesod_write_%s_%s.%s", datasetSize.toLowerCase(), extension, extension)); + + System.out.printf("FesodBenchmark setup: %s / %s / %d rows%n", fileFormat, datasetSize, data.size()); + } + + @TearDown(Level.Trial) + public void tearDownTrial() { + BenchmarkFileUtil.delete(readFile); + BenchmarkFileUtil.delete(writeFile); + } + + /** + * Write {@code datasetSize} rows of 20 columns to the fixed output file + * (overwritten each invocation, deleted in {@link #tearDownTrial}). + */ + @Benchmark + public long write(Blackhole blackhole) { + FesodSheet.write(writeFile, BenchmarkData.class).sheet("Sheet1").doWrite(data); + blackhole.consume(writeFile.length()); + return data.size(); + } + + /** + * Read the whole pre-generated file through the streaming reader. + */ + @Benchmark + public long read(Blackhole blackhole) { + AtomicLong processedRows = new AtomicLong(0); + + try (ExcelReader excelReader = FesodSheet.read( + readFile, BenchmarkData.class, new ReadListener() { + @Override + public void invoke(BenchmarkData row, AnalysisContext context) { + processedRows.incrementAndGet(); + blackhole.consume(row); + } + + @Override + public void doAfterAllAnalysed(AnalysisContext context) { + // no-op + } + }) + .build()) { + excelReader.readAll(); + } + return processedRows.get(); + } +} diff --git a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/FesodVsPoiBenchmark.java b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/FesodVsPoiBenchmark.java new file mode 100644 index 000000000..7e396504c --- /dev/null +++ b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/FesodVsPoiBenchmark.java @@ -0,0 +1,302 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.fesod.sheet.benchmark; + +import java.io.File; +import java.io.FileInputStream; +import java.io.FileOutputStream; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicLong; +import org.apache.fesod.sheet.ExcelReader; +import org.apache.fesod.sheet.ExcelWriter; +import org.apache.fesod.sheet.FesodSheet; +import org.apache.fesod.sheet.benchmark.data.BenchmarkData; +import org.apache.fesod.sheet.benchmark.util.BenchmarkFileUtil; +import org.apache.fesod.sheet.benchmark.util.DataGenerator; +import org.apache.fesod.sheet.context.AnalysisContext; +import org.apache.fesod.sheet.read.listener.ReadListener; +import org.apache.fesod.sheet.write.metadata.WriteSheet; +import org.apache.poi.hssf.usermodel.HSSFWorkbook; +import org.apache.poi.ss.usermodel.Cell; +import org.apache.poi.ss.usermodel.Row; +import org.apache.poi.ss.usermodel.Sheet; +import org.apache.poi.ss.usermodel.Workbook; +import org.apache.poi.ss.usermodel.WorkbookFactory; +import org.apache.poi.util.IOUtils; +import org.apache.poi.xssf.usermodel.XSSFWorkbook; +import org.openjdk.jmh.annotations.Benchmark; +import org.openjdk.jmh.annotations.BenchmarkMode; +import org.openjdk.jmh.annotations.Fork; +import org.openjdk.jmh.annotations.Level; +import org.openjdk.jmh.annotations.Measurement; +import org.openjdk.jmh.annotations.Mode; +import org.openjdk.jmh.annotations.OutputTimeUnit; +import org.openjdk.jmh.annotations.Param; +import org.openjdk.jmh.annotations.Scope; +import org.openjdk.jmh.annotations.Setup; +import org.openjdk.jmh.annotations.State; +import org.openjdk.jmh.annotations.TearDown; +import org.openjdk.jmh.annotations.Warmup; +import org.openjdk.jmh.infra.Blackhole; + +/** + * Side-by-side comparison of Fesod and Apache POI on the same data, using each + * library's plain (in-memory) APIs: write via {@code ExcelWriter} vs an + * XSSF/HSSF {@code Workbook}, read via the Fesod streaming reader vs POI's + * DOM-based {@code WorkbookFactory}. The streaming-API comparison (POI SAX / + * SXSSF) lives in {@link StreamingBenchmark}. + * + *

Fair-comparison measures applied: + *

+ * + *

Workload symmetry. The write path is symmetric: both sides convert the + * same in-memory objects into the same 20 columns. The read path is deliberately + * asymmetric, mirroring each library's idiomatic end-to-end usage: Fesod runs its + * full pipeline (parse + type conversion such as {@code BigDecimal}/dates + + * reflective {@code BenchmarkData} instantiation), while the POI side consumes raw + * cell text — POI has no built-in row-object mapping, and hand-rolling one in the + * benchmark would measure benchmark code rather than either library. The asymmetry + * is conservative for Fesod: the POI side does strictly less work per row, so the + * comparison cannot overstate Fesod's advantage. + * + *

XLS is limited to 65,536 rows per sheet, so datasets are truncated to 65,534 + * data rows for that format. Run through the shaded {@code benchmarks.jar} as + * described in {@code fesod-benchmark/benchmark.md}. + */ +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +@Warmup(iterations = 3, time = 5, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 5, timeUnit = TimeUnit.SECONDS) +@Fork( + value = 1, + jvmArgs = {"-Xms2g", "-Xmx2g"}) +public class FesodVsPoiBenchmark { + + /** XLS (BIFF8) supports at most 65,536 rows per sheet; keep one row for the header. */ + private static final int XLS_MAX_DATA_ROWS = 65534; + + @Param({"SMALL", "MEDIUM", "LARGE"}) + private String datasetSize; + + @Param({"XLSX", "XLS"}) + private String fileFormat; + + private File testFile; + private File fesodWriteFile; + private File poiWriteFile; + private List testDataList; + private BenchmarkConfiguration.FileFormat format; + private String extension; + + @Setup(Level.Trial) + public void setupTrial() { + // Allow POI to load the larger files generated by this suite + IOUtils.setByteArrayMaxOverride(1024 * 1024 * 1024); + + format = BenchmarkConfiguration.FileFormat.valueOf(fileFormat); + extension = format.getExtension(); + + BenchmarkConfiguration.DatasetSize size = BenchmarkConfiguration.DatasetSize.valueOf(datasetSize); + testDataList = DataGenerator.generateTestData(size); + + if (format == BenchmarkConfiguration.FileFormat.XLS && testDataList.size() > XLS_MAX_DATA_ROWS) { + System.out.printf( + "WARN: XLS supports max %d data rows, truncating from %d rows.%n", + XLS_MAX_DATA_ROWS, testDataList.size()); + testDataList = new ArrayList<>(testDataList.subList(0, XLS_MAX_DATA_ROWS)); + } + + testFile = BenchmarkFileUtil.createTestFile( + String.format("comparison_%s_%s.%s", datasetSize.toLowerCase(), extension, extension)); + + // Pre-populate the test file used by the read benchmarks + FesodSheet.write(testFile, BenchmarkData.class).sheet("TestData").doWrite(testDataList); + + // Fixed targets for the write benchmarks: every invocation overwrites the + // same file, and cleanup happens once in tearDownTrial so no I/O other + // than the write itself is part of the measured op. + fesodWriteFile = BenchmarkFileUtil.createTestFile( + String.format("fesod_vs_poi_fesod_write_%s.%s", datasetSize.toLowerCase(), extension)); + poiWriteFile = BenchmarkFileUtil.createTestFile( + String.format("fesod_vs_poi_poi_write_%s.%s", datasetSize.toLowerCase(), extension)); + + System.out.printf("FesodVsPoiBenchmark setup: %s format, %d rows%n", fileFormat, testDataList.size()); + } + + @TearDown(Level.Trial) + public void tearDownTrial() { + BenchmarkFileUtil.delete(testFile); + BenchmarkFileUtil.delete(fesodWriteFile); + BenchmarkFileUtil.delete(poiWriteFile); + } + + // ============================================================================ + // WRITE + // ============================================================================ + + /** + * Write the whole dataset with Fesod. + */ + @Benchmark + public long fesodWrite(Blackhole blackhole) { + try (ExcelWriter excelWriter = + FesodSheet.write(fesodWriteFile, BenchmarkData.class).build()) { + WriteSheet writeSheet = FesodSheet.writerSheet("TestData").build(); + excelWriter.write(testDataList, writeSheet); + blackhole.consume(fesodWriteFile.length()); + } + return testDataList.size(); + } + + /** + * Write the same columns with Apache POI. + */ + @Benchmark + public long poiWrite(Blackhole blackhole) throws Exception { + try (FileOutputStream fos = new FileOutputStream(poiWriteFile); + Workbook workbook = createWorkbook()) { + Sheet sheet = workbook.createSheet("TestData"); + + Row headerRow = sheet.createRow(0); + String[] headers = { + "ID", "String Data", "Integer Value", "Long Value", "Double Value", + "BigDecimal Value", "Boolean Flag", "Date Value", "DateTime Value", "Category", + "Description", "Status", "Float Value", "Short Value", "Byte Value", + "Extra Data 1", "Extra Data 2", "Extra Data 3", "Extra Data 4", "Extra Data 5" + }; + for (int i = 0; i < headers.length; i++) { + headerRow.createCell(i).setCellValue(headers[i]); + } + + for (int i = 0; i < testDataList.size(); i++) { + BenchmarkData data = testDataList.get(i); + Row row = sheet.createRow(i + 1); + + row.createCell(0).setCellValue(data.getId() != null ? data.getId() : 0); + row.createCell(1).setCellValue(data.getStringData() != null ? data.getStringData() : ""); + row.createCell(2).setCellValue(data.getIntValue() != null ? data.getIntValue() : 0); + row.createCell(3).setCellValue(data.getLongValue() != null ? data.getLongValue() : 0L); + row.createCell(4).setCellValue(data.getDoubleValue() != null ? data.getDoubleValue() : 0.0); + row.createCell(5) + .setCellValue( + data.getBigDecimalValue() != null + ? data.getBigDecimalValue().doubleValue() + : 0.0); + row.createCell(6).setCellValue(data.getBooleanFlag() != null ? data.getBooleanFlag() : false); + row.createCell(7) + .setCellValue( + data.getDateValue() != null + ? data.getDateValue().toString() + : ""); + row.createCell(8) + .setCellValue( + data.getDateTimeValue() != null + ? data.getDateTimeValue().toString() + : ""); + row.createCell(9).setCellValue(data.getCategory() != null ? data.getCategory() : ""); + row.createCell(10).setCellValue(data.getDescription() != null ? data.getDescription() : ""); + row.createCell(11).setCellValue(data.getStatus() != null ? data.getStatus() : ""); + row.createCell(12).setCellValue(data.getFloatValue() != null ? data.getFloatValue() : 0.0f); + row.createCell(13).setCellValue(data.getShortValue() != null ? data.getShortValue() : 0); + row.createCell(14).setCellValue(data.getByteValue() != null ? data.getByteValue() : 0); + row.createCell(15).setCellValue(data.getExtraData1() != null ? data.getExtraData1() : ""); + row.createCell(16).setCellValue(data.getExtraData2() != null ? data.getExtraData2() : ""); + row.createCell(17).setCellValue(data.getExtraData3() != null ? data.getExtraData3() : ""); + row.createCell(18).setCellValue(data.getExtraData4() != null ? data.getExtraData4() : ""); + row.createCell(19).setCellValue(data.getExtraData5() != null ? data.getExtraData5() : ""); + } + + workbook.write(fos); + blackhole.consume(poiWriteFile.length()); + } + return testDataList.size(); + } + + // ============================================================================ + // READ + // ============================================================================ + + /** + * Stream the whole file with the Fesod reader. + */ + @Benchmark + public long fesodRead(Blackhole blackhole) { + AtomicLong processedRows = new AtomicLong(0); + + try (ExcelReader excelReader = FesodSheet.read( + testFile, BenchmarkData.class, new ReadListener() { + @Override + public void invoke(BenchmarkData data, AnalysisContext context) { + processedRows.incrementAndGet(); + blackhole.consume(data); + } + + @Override + public void doAfterAllAnalysed(AnalysisContext context) { + // no-op + } + }) + .build()) { + excelReader.readAll(); + } + return processedRows.get(); + } + + /** + * Load the whole workbook with Apache POI (the DOM-based approach most POI + * applications use). + */ + @Benchmark + public long poiRead(Blackhole blackhole) throws Exception { + long processedRows = 0; + + try (FileInputStream fis = new FileInputStream(testFile); + Workbook workbook = WorkbookFactory.create(fis)) { + Sheet sheet = workbook.getSheetAt(0); + + for (Row row : sheet) { + if (row.getRowNum() == 0) { + continue; // skip header + } + for (Cell cell : row) { + blackhole.consume(cell.toString()); + } + processedRows++; + } + } + return processedRows; + } + + // ============================================================================ + // UTILITY METHODS + // ============================================================================ + + private Workbook createWorkbook() { + return format == BenchmarkConfiguration.FileFormat.XLSX ? new XSSFWorkbook() : new HSSFWorkbook(); + } +} diff --git a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/StreamingBenchmark.java b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/StreamingBenchmark.java new file mode 100644 index 000000000..9b45975b6 --- /dev/null +++ b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/StreamingBenchmark.java @@ -0,0 +1,298 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.fesod.sheet.benchmark; + +import java.io.File; +import java.io.FileOutputStream; +import java.io.InputStream; +import java.util.List; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicLong; +import org.apache.fesod.sheet.ExcelReader; +import org.apache.fesod.sheet.ExcelWriter; +import org.apache.fesod.sheet.FesodSheet; +import org.apache.fesod.sheet.benchmark.data.BenchmarkData; +import org.apache.fesod.sheet.benchmark.util.BenchmarkFileUtil; +import org.apache.fesod.sheet.benchmark.util.DataGenerator; +import org.apache.fesod.sheet.context.AnalysisContext; +import org.apache.fesod.sheet.read.listener.ReadListener; +import org.apache.fesod.sheet.write.metadata.WriteSheet; +import org.apache.poi.openxml4j.opc.OPCPackage; +import org.apache.poi.openxml4j.opc.PackageAccess; +import org.apache.poi.ss.usermodel.DataFormatter; +import org.apache.poi.ss.usermodel.Row; +import org.apache.poi.ss.usermodel.Sheet; +import org.apache.poi.util.XMLHelper; +import org.apache.poi.xssf.eventusermodel.ReadOnlySharedStringsTable; +import org.apache.poi.xssf.eventusermodel.XSSFReader; +import org.apache.poi.xssf.eventusermodel.XSSFSheetXMLHandler; +import org.apache.poi.xssf.streaming.SXSSFWorkbook; +import org.apache.poi.xssf.usermodel.XSSFComment; +import org.openjdk.jmh.annotations.Benchmark; +import org.openjdk.jmh.annotations.BenchmarkMode; +import org.openjdk.jmh.annotations.Fork; +import org.openjdk.jmh.annotations.Level; +import org.openjdk.jmh.annotations.Measurement; +import org.openjdk.jmh.annotations.Mode; +import org.openjdk.jmh.annotations.OutputTimeUnit; +import org.openjdk.jmh.annotations.Param; +import org.openjdk.jmh.annotations.Scope; +import org.openjdk.jmh.annotations.Setup; +import org.openjdk.jmh.annotations.State; +import org.openjdk.jmh.annotations.TearDown; +import org.openjdk.jmh.annotations.Warmup; +import org.openjdk.jmh.infra.Blackhole; +import org.xml.sax.ContentHandler; +import org.xml.sax.InputSource; +import org.xml.sax.XMLReader; + +/** + * Streaming-API comparison for the large-file scenario, XLSX only: Fesod vs the + * streaming APIs POI applications use when a workbook must not be fully held in + * memory. + * + *

+ * + *

XLSX only, because the scenario is large files — XLS (BIFF8) tops out at + * 65,536 rows per sheet, and both the SAX event model and SXSSF are XSSF APIs. + * The plain-API comparison lives in {@link FesodVsPoiBenchmark}. + */ +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MILLISECONDS) +@State(Scope.Benchmark) +@Warmup(iterations = 3, time = 5, timeUnit = TimeUnit.SECONDS) +@Measurement(iterations = 5, time = 5, timeUnit = TimeUnit.SECONDS) +@Fork( + value = 1, + jvmArgs = {"-Xms2g", "-Xmx2g"}) +public class StreamingBenchmark { + + /** Row batch size for the Fesod batched write and the SXSSF window. */ + private static final int BATCH_ROWS = 1000; + + private static final String[] HEADERS = { + "ID", "String Data", "Integer Value", "Long Value", "Double Value", + "BigDecimal Value", "Boolean Flag", "Date Value", "DateTime Value", "Category", + "Description", "Status", "Float Value", "Short Value", "Byte Value", + "Extra Data 1", "Extra Data 2", "Extra Data 3", "Extra Data 4", "Extra Data 5" + }; + + @Param({"FESOD", "POI"}) + private String library; + + @Param({"SMALL", "MEDIUM", "LARGE"}) + private String datasetSize; + + private File testFile; + private File writeFile; + private List testDataList; + + @Setup(Level.Trial) + public void setupTrial() { + BenchmarkConfiguration.DatasetSize size = BenchmarkConfiguration.DatasetSize.valueOf(datasetSize); + testDataList = DataGenerator.generateTestData(size); + + testFile = BenchmarkFileUtil.createTestFile(String.format("streaming_read_%s.xlsx", datasetSize.toLowerCase())); + FesodSheet.write(testFile, BenchmarkData.class).sheet("TestData").doWrite(testDataList); + + // Fixed target for the streaming-write benchmark: overwritten each + // invocation, deleted in tearDownTrial so cleanup stays unmeasured. + writeFile = + BenchmarkFileUtil.createTestFile(String.format("streaming_write_%s.xlsx", datasetSize.toLowerCase())); + + System.out.printf("StreamingBenchmark setup: %s / %s / %d rows%n", library, datasetSize, testDataList.size()); + } + + @TearDown(Level.Trial) + public void tearDownTrial() { + BenchmarkFileUtil.delete(testFile); + BenchmarkFileUtil.delete(writeFile); + } + + @Benchmark + public long read(Blackhole blackhole) throws Exception { + if ("FESOD".equals(library)) { + return fesodRead(blackhole); + } + return poiSaxRead(blackhole); + } + + @Benchmark + public long write(Blackhole blackhole) throws Exception { + if ("FESOD".equals(library)) { + return fesodStreamingWrite(); + } + return poiSxssfWrite(blackhole); + } + + // ============================================================================ + // STREAMING READ + // ============================================================================ + + private long fesodRead(Blackhole blackhole) { + AtomicLong processedRows = new AtomicLong(0); + + try (ExcelReader excelReader = FesodSheet.read( + testFile, BenchmarkData.class, new ReadListener() { + @Override + public void invoke(BenchmarkData row, AnalysisContext context) { + processedRows.incrementAndGet(); + blackhole.consume(row); + } + + @Override + public void doAfterAllAnalysed(AnalysisContext context) { + // no-op + } + }) + .build()) { + excelReader.readAll(); + } + return processedRows.get(); + } + + private long poiSaxRead(Blackhole blackhole) throws Exception { + AtomicLong processedRows = new AtomicLong(0); + + try (OPCPackage pkg = OPCPackage.open(testFile, PackageAccess.READ)) { + ReadOnlySharedStringsTable strings = new ReadOnlySharedStringsTable(pkg); + XSSFReader xssfReader = new XSSFReader(pkg); + XSSFReader.SheetIterator sheets = (XSSFReader.SheetIterator) xssfReader.getSheetsData(); + + while (sheets.hasNext()) { + try (InputStream sheetStream = sheets.next()) { + XSSFSheetXMLHandler.SheetContentsHandler cellHandler = + new XSSFSheetXMLHandler.SheetContentsHandler() { + private int currentRow = -1; + + @Override + public void startRow(int rowNum) { + currentRow = rowNum; + } + + @Override + public void cell(String cellReference, String formattedValue, XSSFComment comment) { + if (currentRow > 0) { + blackhole.consume(formattedValue); + } + } + + @Override + public void endRow(int rowNum) { + if (rowNum > 0) { + processedRows.incrementAndGet(); + } + } + }; + + ContentHandler handler = new XSSFSheetXMLHandler( + xssfReader.getStylesTable(), strings, cellHandler, new DataFormatter(), false); + + XMLReader parser = XMLHelper.newXMLReader(); + parser.setContentHandler(handler); + parser.parse(new InputSource(sheetStream)); + } + } + } + return processedRows.get(); + } + + // ============================================================================ + // STREAMING WRITE + // ============================================================================ + + private long fesodStreamingWrite() { + try (ExcelWriter excelWriter = + FesodSheet.write(writeFile, BenchmarkData.class).build()) { + WriteSheet writeSheet = FesodSheet.writerSheet("TestData").build(); + for (int start = 0; start < testDataList.size(); start += BATCH_ROWS) { + int end = Math.min(start + BATCH_ROWS, testDataList.size()); + excelWriter.write(testDataList.subList(start, end), writeSheet); + } + } + return testDataList.size(); + } + + private long poiSxssfWrite(Blackhole blackhole) throws Exception { + try (SXSSFWorkbook workbook = new SXSSFWorkbook(BATCH_ROWS)) { + Sheet sheet = workbook.createSheet("TestData"); + + Row headerRow = sheet.createRow(0); + for (int i = 0; i < HEADERS.length; i++) { + headerRow.createCell(i).setCellValue(HEADERS[i]); + } + + for (int i = 0; i < testDataList.size(); i++) { + fillRow(sheet.createRow(i + 1), testDataList.get(i)); + } + + try (FileOutputStream fos = new FileOutputStream(writeFile)) { + workbook.write(fos); + } + } // close() also disposes the SXSSF temp files + blackhole.consume(writeFile.length()); + return testDataList.size(); + } + + private void fillRow(Row row, BenchmarkData data) { + row.createCell(0).setCellValue(data.getId() != null ? data.getId() : 0); + row.createCell(1).setCellValue(data.getStringData() != null ? data.getStringData() : ""); + row.createCell(2).setCellValue(data.getIntValue() != null ? data.getIntValue() : 0); + row.createCell(3).setCellValue(data.getLongValue() != null ? data.getLongValue() : 0L); + row.createCell(4).setCellValue(data.getDoubleValue() != null ? data.getDoubleValue() : 0.0); + row.createCell(5) + .setCellValue( + data.getBigDecimalValue() != null + ? data.getBigDecimalValue().doubleValue() + : 0.0); + row.createCell(6).setCellValue(data.getBooleanFlag() != null ? data.getBooleanFlag() : false); + row.createCell(7) + .setCellValue(data.getDateValue() != null ? data.getDateValue().toString() : ""); + row.createCell(8) + .setCellValue( + data.getDateTimeValue() != null + ? data.getDateTimeValue().toString() + : ""); + row.createCell(9).setCellValue(data.getCategory() != null ? data.getCategory() : ""); + row.createCell(10).setCellValue(data.getDescription() != null ? data.getDescription() : ""); + row.createCell(11).setCellValue(data.getStatus() != null ? data.getStatus() : ""); + row.createCell(12).setCellValue(data.getFloatValue() != null ? data.getFloatValue() : 0.0f); + row.createCell(13).setCellValue(data.getShortValue() != null ? data.getShortValue() : 0); + row.createCell(14).setCellValue(data.getByteValue() != null ? data.getByteValue() : 0); + row.createCell(15).setCellValue(data.getExtraData1() != null ? data.getExtraData1() : ""); + row.createCell(16).setCellValue(data.getExtraData2() != null ? data.getExtraData2() : ""); + row.createCell(17).setCellValue(data.getExtraData3() != null ? data.getExtraData3() : ""); + row.createCell(18).setCellValue(data.getExtraData4() != null ? data.getExtraData4() : ""); + row.createCell(19).setCellValue(data.getExtraData5() != null ? data.getExtraData5() : ""); + } +} diff --git a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/data/BenchmarkData.java b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/data/BenchmarkData.java new file mode 100644 index 000000000..174339ee4 --- /dev/null +++ b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/data/BenchmarkData.java @@ -0,0 +1,283 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.fesod.sheet.benchmark.data; + +import java.math.BigDecimal; +import java.time.LocalDate; +import java.time.LocalDateTime; +import org.apache.fesod.sheet.annotation.ExcelProperty; +import org.apache.fesod.sheet.annotation.format.DateTimeFormat; + +/** + * Standard benchmark data model with various data types for comprehensive testing + */ +public class BenchmarkData { + + @ExcelProperty(value = "ID", index = 0) + private Long id; + + @ExcelProperty(value = "String Data", index = 1) + private String stringData; + + @ExcelProperty(value = "Integer Value", index = 2) + private Integer intValue; + + @ExcelProperty(value = "Long Value", index = 3) + private Long longValue; + + @ExcelProperty(value = "Double Value", index = 4) + private Double doubleValue; + + @ExcelProperty(value = "BigDecimal Value", index = 5) + private BigDecimal bigDecimalValue; + + @ExcelProperty(value = "Boolean Flag", index = 6) + private Boolean booleanFlag; + + @ExcelProperty(value = "Date Value", index = 7) + @DateTimeFormat("yyyy-MM-dd") + private LocalDate dateValue; + + @ExcelProperty(value = "DateTime Value", index = 8) + @DateTimeFormat("yyyy-MM-dd HH:mm:ss") + private LocalDateTime dateTimeValue; + + @ExcelProperty(value = "Category", index = 9) + private String category; + + @ExcelProperty(value = "Description", index = 10) + private String description; + + @ExcelProperty(value = "Status", index = 11) + private String status; + + @ExcelProperty(value = "Float Value", index = 12) + private Float floatValue; + + @ExcelProperty(value = "Short Value", index = 13) + private Short shortValue; + + @ExcelProperty(value = "Byte Value", index = 14) + private Byte byteValue; + + @ExcelProperty(value = "Extra Data 1", index = 15) + private String extraData1; + + @ExcelProperty(value = "Extra Data 2", index = 16) + private String extraData2; + + @ExcelProperty(value = "Extra Data 3", index = 17) + private String extraData3; + + @ExcelProperty(value = "Extra Data 4", index = 18) + private String extraData4; + + @ExcelProperty(value = "Extra Data 5", index = 19) + private String extraData5; + + // Default constructor + public BenchmarkData() {} + + // Getters and setters + public Long getId() { + return id; + } + + public void setId(Long id) { + this.id = id; + } + + public String getStringData() { + return stringData; + } + + public void setStringData(String stringData) { + this.stringData = stringData; + } + + public Integer getIntValue() { + return intValue; + } + + public void setIntValue(Integer intValue) { + this.intValue = intValue; + } + + public Long getLongValue() { + return longValue; + } + + public void setLongValue(Long longValue) { + this.longValue = longValue; + } + + public Double getDoubleValue() { + return doubleValue; + } + + public void setDoubleValue(Double doubleValue) { + this.doubleValue = doubleValue; + } + + public BigDecimal getBigDecimalValue() { + return bigDecimalValue; + } + + public void setBigDecimalValue(BigDecimal bigDecimalValue) { + this.bigDecimalValue = bigDecimalValue; + } + + public Boolean getBooleanFlag() { + return booleanFlag; + } + + public void setBooleanFlag(Boolean booleanFlag) { + this.booleanFlag = booleanFlag; + } + + public LocalDate getDateValue() { + return dateValue; + } + + public void setDateValue(LocalDate dateValue) { + this.dateValue = dateValue; + } + + public LocalDateTime getDateTimeValue() { + return dateTimeValue; + } + + public void setDateTimeValue(LocalDateTime dateTimeValue) { + this.dateTimeValue = dateTimeValue; + } + + public String getCategory() { + return category; + } + + public void setCategory(String category) { + this.category = category; + } + + public String getDescription() { + return description; + } + + public void setDescription(String description) { + this.description = description; + } + + public String getStatus() { + return status; + } + + public void setStatus(String status) { + this.status = status; + } + + public Float getFloatValue() { + return floatValue; + } + + public void setFloatValue(Float floatValue) { + this.floatValue = floatValue; + } + + public Short getShortValue() { + return shortValue; + } + + public void setShortValue(Short shortValue) { + this.shortValue = shortValue; + } + + public Byte getByteValue() { + return byteValue; + } + + public void setByteValue(Byte byteValue) { + this.byteValue = byteValue; + } + + public String getExtraData1() { + return extraData1; + } + + public void setExtraData1(String extraData1) { + this.extraData1 = extraData1; + } + + public String getExtraData2() { + return extraData2; + } + + public void setExtraData2(String extraData2) { + this.extraData2 = extraData2; + } + + public String getExtraData3() { + return extraData3; + } + + public void setExtraData3(String extraData3) { + this.extraData3 = extraData3; + } + + public String getExtraData4() { + return extraData4; + } + + public void setExtraData4(String extraData4) { + this.extraData4 = extraData4; + } + + public String getExtraData5() { + return extraData5; + } + + public void setExtraData5(String extraData5) { + this.extraData5 = extraData5; + } + + @Override + public String toString() { + return "BenchmarkData{" + "id=" + + id + ", stringData='" + + stringData + '\'' + ", intValue=" + + intValue + ", longValue=" + + longValue + ", doubleValue=" + + doubleValue + ", bigDecimalValue=" + + bigDecimalValue + ", booleanFlag=" + + booleanFlag + ", dateValue=" + + dateValue + ", dateTimeValue=" + + dateTimeValue + ", category='" + + category + '\'' + ", description='" + + description + '\'' + ", status='" + + status + '\'' + ", floatValue=" + + floatValue + ", shortValue=" + + shortValue + ", byteValue=" + + byteValue + ", extraData1='" + + extraData1 + '\'' + ", extraData2='" + + extraData2 + '\'' + ", extraData3='" + + extraData3 + '\'' + ", extraData4='" + + extraData4 + '\'' + ", extraData5='" + + extraData5 + '\'' + '}'; + } +} diff --git a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/BenchmarkFileUtil.java b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/BenchmarkFileUtil.java new file mode 100644 index 000000000..cd3a09f4b --- /dev/null +++ b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/BenchmarkFileUtil.java @@ -0,0 +1,77 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.fesod.sheet.benchmark.util; + +import java.io.File; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Central place for the scratch directory benchmark files are written to + * ({@code target/benchmark-testdata}), so all suites share one location that + * Maven cleans up on {@code mvn clean}. + */ +public class BenchmarkFileUtil { + + private static final Logger logger = LoggerFactory.getLogger(BenchmarkFileUtil.class); + + private static final String TEST_DATA_DIR = "target/benchmark-testdata"; + + /** + * Create test data directory if it doesn't exist. + */ + public static void createTestDataDirectory() { + Path testDataPath = Paths.get(TEST_DATA_DIR); + if (!Files.exists(testDataPath)) { + try { + Files.createDirectories(testDataPath); + } catch (IOException e) { + throw new IllegalStateException("Failed to create test data directory: " + testDataPath, e); + } + } + } + + /** + * Create a test file handle with the specified name inside the test data directory. + */ + public static File createTestFile(String fileName) { + createTestDataDirectory(); + return new File(TEST_DATA_DIR, fileName); + } + + /** + * Delete a benchmark scratch file, logging the cause when deletion fails + * instead of failing the run. + */ + public static void delete(File file) { + if (file == null) { + return; + } + try { + Files.deleteIfExists(file.toPath()); + } catch (IOException e) { + logger.warn("Failed to delete benchmark file {}: {}", file, e.getMessage()); + } + } +} diff --git a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/DataGenerator.java b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/DataGenerator.java new file mode 100644 index 000000000..9cab6ed5e --- /dev/null +++ b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/DataGenerator.java @@ -0,0 +1,215 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.fesod.sheet.benchmark.util; + +import java.math.BigDecimal; +import java.math.RoundingMode; +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.time.temporal.ChronoUnit; +import java.util.ArrayList; +import java.util.List; +import java.util.Random; +import org.apache.fesod.sheet.benchmark.BenchmarkConfiguration; +import org.apache.fesod.sheet.benchmark.data.BenchmarkData; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Generates deterministic benchmark data: a fixed default seed and a fixed date + * anchor mean the same seed always produces the same rows, so benchmark results + * from different runs are comparable. + */ +public class DataGenerator { + + private static final Logger logger = LoggerFactory.getLogger(DataGenerator.class); + + /** Fixed anchor for generated dates; never uses the wall clock. */ + private static final LocalDate DATE_ANCHOR = LocalDate.of(2024, 1, 1); + + private static final String[] CATEGORIES = { + "Electronics", + "Books", + "Clothing", + "Home & Garden", + "Sports", + "Automotive", + "Health & Beauty", + "Toys & Games", + "Food & Beverage", + "Office Supplies" + }; + + private static final String[] STATUSES = { + "Active", "Inactive", "Pending", "Processing", "Completed", "Cancelled", "On Hold" + }; + + private static final String[] SAMPLE_WORDS = { + "Lorem", + "ipsum", + "dolor", + "sit", + "amet", + "consectetur", + "adipiscing", + "elit", + "sed", + "do", + "eiusmod", + "tempor", + "incididunt", + "ut", + "labore", + "et", + "dolore", + "magna", + "aliqua", + "enim", + "ad", + "minim", + "veniam", + "quis", + "nostrud", + "exercitation", + "ullamco", + "laboris", + "nisi", + "aliquip", + "ex", + "ea", + "commodo" + }; + + private final Random random; + + public DataGenerator() { + this(42L); + } + + public DataGenerator(long seed) { + this.random = new Random(seed); + } + + public List generateData(BenchmarkConfiguration.DatasetSize size) { + return generateData(size.getRowCount()); + } + + public List generateData(int rowCount) { + logger.info("Generating {} rows of benchmark data", rowCount); + + List data = new ArrayList<>(rowCount); + long startTime = System.currentTimeMillis(); + + for (int i = 0; i < rowCount; i++) { + data.add(generateSingleRow(i + 1)); + } + + long duration = System.currentTimeMillis() - startTime; + logger.info( + "Generated {} rows in {} ms ({} rows/sec)", + rowCount, + duration, + duration > 0 ? rowCount * 1000L / duration : "N/A"); + + return data; + } + + private BenchmarkData generateSingleRow(long id) { + BenchmarkData data = new BenchmarkData(); + + data.setId(id); + data.setStringData(generateRandomString(10, 50)); + data.setIntValue(random.nextInt(1_000_000)); + data.setLongValue(random.nextLong()); + data.setDoubleValue(random.nextDouble() * 1_000_000); + data.setBigDecimalValue( + BigDecimal.valueOf(random.nextDouble() * 1_000_000).setScale(2, RoundingMode.HALF_UP)); + data.setBooleanFlag(random.nextBoolean()); + data.setDateValue(generateRandomDate()); + data.setDateTimeValue(generateRandomDateTime()); + data.setCategory(CATEGORIES[random.nextInt(CATEGORIES.length)]); + data.setDescription(generateRandomDescription()); + data.setStatus(STATUSES[random.nextInt(STATUSES.length)]); + data.setFloatValue(random.nextFloat() * 1000); + data.setShortValue((short) random.nextInt(Short.MAX_VALUE)); + data.setByteValue((byte) random.nextInt(Byte.MAX_VALUE)); + data.setExtraData1(generateRandomString(5, 20)); + data.setExtraData2(generateRandomString(5, 20)); + data.setExtraData3(generateRandomString(5, 20)); + data.setExtraData4(generateRandomString(5, 20)); + data.setExtraData5(generateRandomString(5, 20)); + + return data; + } + + private String generateRandomString(int minLength, int maxLength) { + int length = random.nextInt(maxLength - minLength + 1) + minLength; + StringBuilder sb = new StringBuilder(length); + + for (int i = 0; i < length; i++) { + if (random.nextBoolean()) { + sb.append((char) ('a' + random.nextInt(26))); + } else { + sb.append((char) ('0' + random.nextInt(10))); + } + } + + return sb.toString(); + } + + private String generateRandomDescription() { + int wordCount = random.nextInt(8) + 3; // 3-10 words + StringBuilder sb = new StringBuilder(); + + for (int i = 0; i < wordCount; i++) { + if (i > 0) { + sb.append(" "); + } + sb.append(SAMPLE_WORDS[random.nextInt(SAMPLE_WORDS.length)]); + } + + return sb.toString(); + } + + /** Random date in the five years before {@link #DATE_ANCHOR}. */ + private LocalDate generateRandomDate() { + LocalDate fiveYearsAgo = DATE_ANCHOR.minusYears(5); + long daysBetween = ChronoUnit.DAYS.between(fiveYearsAgo, DATE_ANCHOR); + long randomDays = Math.floorMod(random.nextLong(), daysBetween); + return fiveYearsAgo.plusDays(randomDays); + } + + /** Random datetime in the year before {@link #DATE_ANCHOR}. */ + private LocalDateTime generateRandomDateTime() { + LocalDateTime oneYearAgo = DATE_ANCHOR.atStartOfDay().minusYears(1); + long secondsBetween = ChronoUnit.SECONDS.between(oneYearAgo, DATE_ANCHOR.atStartOfDay()); + long randomSeconds = Math.floorMod(random.nextLong(), secondsBetween); + return oneYearAgo.plusSeconds(randomSeconds); + } + + private static final DataGenerator defaultGenerator = new DataGenerator(42L); + + /** + * Generate test data with the default generator (fixed seed). + */ + public static List generateTestData(BenchmarkConfiguration.DatasetSize size) { + return defaultGenerator.generateData(size); + } +} diff --git a/fesod-benchmark/src/test/java/org/apache/fesod/sheet/benchmark/util/BenchmarkFileUtilTest.java b/fesod-benchmark/src/test/java/org/apache/fesod/sheet/benchmark/util/BenchmarkFileUtilTest.java new file mode 100644 index 000000000..8d35cb8c6 --- /dev/null +++ b/fesod-benchmark/src/test/java/org/apache/fesod/sheet/benchmark/util/BenchmarkFileUtilTest.java @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.fesod.sheet.benchmark.util; + +import java.io.File; +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +class BenchmarkFileUtilTest { + + @Test + void createTestFilePlacesFileInTestDataDirectory() { + File file = BenchmarkFileUtil.createTestFile("unit_test_sample.csv"); + + Assertions.assertEquals("unit_test_sample.csv", file.getName()); + Assertions.assertTrue(file.getParentFile().exists()); + Assertions.assertTrue(file.getPath() + .replace(File.separatorChar, '/') + .endsWith("target/benchmark-testdata/unit_test_sample.csv")); + } + + @Test + void createTestDataDirectoryIsIdempotent() { + BenchmarkFileUtil.createTestDataDirectory(); + BenchmarkFileUtil.createTestDataDirectory(); + + File dir = BenchmarkFileUtil.createTestFile("unit_test_dir_check.tmp").getParentFile(); + Assertions.assertTrue(dir.isDirectory()); + } +} diff --git a/fesod-benchmark/src/test/java/org/apache/fesod/sheet/benchmark/util/DataGeneratorTest.java b/fesod-benchmark/src/test/java/org/apache/fesod/sheet/benchmark/util/DataGeneratorTest.java new file mode 100644 index 000000000..e9394a09f --- /dev/null +++ b/fesod-benchmark/src/test/java/org/apache/fesod/sheet/benchmark/util/DataGeneratorTest.java @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.fesod.sheet.benchmark.util; + +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.util.List; +import java.util.stream.Collectors; +import org.apache.fesod.sheet.benchmark.BenchmarkConfiguration; +import org.apache.fesod.sheet.benchmark.data.BenchmarkData; +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; + +class DataGeneratorTest { + + private static final int ROWS = 200; + + @Test + void sameSeedProducesIdenticalData() { + List first = new DataGenerator(42L).generateData(ROWS); + List second = new DataGenerator(42L).generateData(ROWS); + + Assertions.assertEquals( + first.stream().map(BenchmarkData::toString).collect(Collectors.toList()), + second.stream().map(BenchmarkData::toString).collect(Collectors.toList())); + } + + @Test + void defaultConstructorUsesFixedSeed() { + List fromDefault = new DataGenerator().generateData(ROWS); + List fromSeed42 = new DataGenerator(42L).generateData(ROWS); + + Assertions.assertEquals( + fromSeed42.stream().map(BenchmarkData::toString).collect(Collectors.toList()), + fromDefault.stream().map(BenchmarkData::toString).collect(Collectors.toList())); + } + + @Test + void differentSeedsProduceDifferentData() { + List first = new DataGenerator(42L) + .generateData(ROWS).stream().map(BenchmarkData::toString).collect(Collectors.toList()); + List second = new DataGenerator(43L) + .generateData(ROWS).stream().map(BenchmarkData::toString).collect(Collectors.toList()); + + Assertions.assertFalse(first.equals(second)); + } + + @Test + void generatedRowsAreComplete() { + List data = new DataGenerator(7L).generateData(ROWS); + + Assertions.assertEquals(ROWS, data.size()); + for (int i = 0; i < data.size(); i++) { + BenchmarkData row = data.get(i); + Assertions.assertEquals(i + 1L, row.getId()); + Assertions.assertFalse(row.getStringData().isEmpty()); + Assertions.assertFalse(row.getCategory().isEmpty()); + Assertions.assertFalse(row.getStatus().isEmpty()); + } + } + + @Test + void datesStayWithinAnchoredWindows() { + List data = new DataGenerator(7L).generateData(ROWS); + + LocalDate dateLowerBound = LocalDate.of(2019, 1, 1); + LocalDate dateUpperBound = LocalDate.of(2024, 1, 1); + LocalDateTime dateTimeLowerBound = LocalDateTime.of(2023, 1, 1, 0, 0); + LocalDateTime dateTimeUpperBound = LocalDateTime.of(2024, 1, 1, 0, 0); + + for (BenchmarkData row : data) { + Assertions.assertTrue(row.getDateValue().isAfter(dateLowerBound.minusDays(1))); + Assertions.assertTrue(row.getDateValue().isBefore(dateUpperBound.plusDays(1))); + Assertions.assertFalse(row.getDateTimeValue().isBefore(dateTimeLowerBound)); + Assertions.assertFalse(row.getDateTimeValue().isAfter(dateTimeUpperBound)); + } + } + + @Test + void generateTestDataMatchesDatasetSize() { + for (BenchmarkConfiguration.DatasetSize size : BenchmarkConfiguration.DatasetSize.values()) { + Assertions.assertEquals( + size.getRowCount(), DataGenerator.generateTestData(size).size()); + } + } +} diff --git a/pom.xml b/pom.xml index 06c6b06d8..a91c9dd46 100644 --- a/pom.xml +++ b/pom.xml @@ -61,6 +61,7 @@ under the License. fesod-common fesod-shaded fesod-sheet + fesod-benchmark From 10845781296fe1ef2c13e0dea4627dc5a6156140 Mon Sep 17 00:00:00 2001 From: gongzhongqiang Date: Mon, 12 Oct 2026 18:20:29 +0800 Subject: [PATCH 2/4] refactor(benchmark): use Lombok for the benchmark data model Replace the hand-written accessors in BenchmarkData with @Data and @NoArgsConstructor (283 -> 92 lines), per review feedback. Lombok is injected globally by the root pom (provided scope); the module's annotationProcessorPaths overrides the root compiler config for the JMH processor, so Lombok is re-added there alongside it. End-to-end verified: full build and test suite pass, and the read/write/streaming benchmarks produce the same numbers as with the hand-written accessors (Lombok emits identical bytecode). --- fesod-benchmark/pom.xml | 5 + .../sheet/benchmark/data/BenchmarkData.java | 193 +----------------- 2 files changed, 9 insertions(+), 189 deletions(-) diff --git a/fesod-benchmark/pom.xml b/fesod-benchmark/pom.xml index 6ddb3f66a..2fdf2b876 100644 --- a/fesod-benchmark/pom.xml +++ b/fesod-benchmark/pom.xml @@ -97,6 +97,11 @@ under the License. ${maven.compiler.source} ${maven.compiler.target} + + org.projectlombok + lombok + ${lombok.version} + org.openjdk.jmh jmh-generator-annprocess diff --git a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/data/BenchmarkData.java b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/data/BenchmarkData.java index 174339ee4..a81eb5342 100644 --- a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/data/BenchmarkData.java +++ b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/data/BenchmarkData.java @@ -22,12 +22,16 @@ import java.math.BigDecimal; import java.time.LocalDate; import java.time.LocalDateTime; +import lombok.Data; +import lombok.NoArgsConstructor; import org.apache.fesod.sheet.annotation.ExcelProperty; import org.apache.fesod.sheet.annotation.format.DateTimeFormat; /** * Standard benchmark data model with various data types for comprehensive testing */ +@Data +@NoArgsConstructor public class BenchmarkData { @ExcelProperty(value = "ID", index = 0) @@ -91,193 +95,4 @@ public class BenchmarkData { @ExcelProperty(value = "Extra Data 5", index = 19) private String extraData5; - - // Default constructor - public BenchmarkData() {} - - // Getters and setters - public Long getId() { - return id; - } - - public void setId(Long id) { - this.id = id; - } - - public String getStringData() { - return stringData; - } - - public void setStringData(String stringData) { - this.stringData = stringData; - } - - public Integer getIntValue() { - return intValue; - } - - public void setIntValue(Integer intValue) { - this.intValue = intValue; - } - - public Long getLongValue() { - return longValue; - } - - public void setLongValue(Long longValue) { - this.longValue = longValue; - } - - public Double getDoubleValue() { - return doubleValue; - } - - public void setDoubleValue(Double doubleValue) { - this.doubleValue = doubleValue; - } - - public BigDecimal getBigDecimalValue() { - return bigDecimalValue; - } - - public void setBigDecimalValue(BigDecimal bigDecimalValue) { - this.bigDecimalValue = bigDecimalValue; - } - - public Boolean getBooleanFlag() { - return booleanFlag; - } - - public void setBooleanFlag(Boolean booleanFlag) { - this.booleanFlag = booleanFlag; - } - - public LocalDate getDateValue() { - return dateValue; - } - - public void setDateValue(LocalDate dateValue) { - this.dateValue = dateValue; - } - - public LocalDateTime getDateTimeValue() { - return dateTimeValue; - } - - public void setDateTimeValue(LocalDateTime dateTimeValue) { - this.dateTimeValue = dateTimeValue; - } - - public String getCategory() { - return category; - } - - public void setCategory(String category) { - this.category = category; - } - - public String getDescription() { - return description; - } - - public void setDescription(String description) { - this.description = description; - } - - public String getStatus() { - return status; - } - - public void setStatus(String status) { - this.status = status; - } - - public Float getFloatValue() { - return floatValue; - } - - public void setFloatValue(Float floatValue) { - this.floatValue = floatValue; - } - - public Short getShortValue() { - return shortValue; - } - - public void setShortValue(Short shortValue) { - this.shortValue = shortValue; - } - - public Byte getByteValue() { - return byteValue; - } - - public void setByteValue(Byte byteValue) { - this.byteValue = byteValue; - } - - public String getExtraData1() { - return extraData1; - } - - public void setExtraData1(String extraData1) { - this.extraData1 = extraData1; - } - - public String getExtraData2() { - return extraData2; - } - - public void setExtraData2(String extraData2) { - this.extraData2 = extraData2; - } - - public String getExtraData3() { - return extraData3; - } - - public void setExtraData3(String extraData3) { - this.extraData3 = extraData3; - } - - public String getExtraData4() { - return extraData4; - } - - public void setExtraData4(String extraData4) { - this.extraData4 = extraData4; - } - - public String getExtraData5() { - return extraData5; - } - - public void setExtraData5(String extraData5) { - this.extraData5 = extraData5; - } - - @Override - public String toString() { - return "BenchmarkData{" + "id=" - + id + ", stringData='" - + stringData + '\'' + ", intValue=" - + intValue + ", longValue=" - + longValue + ", doubleValue=" - + doubleValue + ", bigDecimalValue=" - + bigDecimalValue + ", booleanFlag=" - + booleanFlag + ", dateValue=" - + dateValue + ", dateTimeValue=" - + dateTimeValue + ", category='" - + category + '\'' + ", description='" - + description + '\'' + ", status='" - + status + '\'' + ", floatValue=" - + floatValue + ", shortValue=" - + shortValue + ", byteValue=" - + byteValue + ", extraData1='" - + extraData1 + '\'' + ", extraData2='" - + extraData2 + '\'' + ", extraData3='" - + extraData3 + '\'' + ", extraData4='" - + extraData4 + '\'' + ", extraData5='" - + extraData5 + '\'' + '}'; - } } From ef0bd994f8be4e3f64bbb66010ba0e15800225d4 Mon Sep 17 00:00:00 2001 From: gongzhongqiang Date: Mon, 12 Oct 2026 18:26:11 +0800 Subject: [PATCH 3/4] refactor(benchmark): adopt Lombok across the remaining support classes Module-wide pass over the benchmark sources for Lombok-replaceable boilerplate: - DataGenerator and BenchmarkFileUtil: hand-written SLF4J loggers replaced with @Slf4j (field renamed to the conventional log) - BenchmarkConfiguration: enum accessors replaced with @Getter The JMH suite classes and tests have no replaceable boilerplate (no loggers, no accessors; @Param fields are JMH-managed). --- .../sheet/benchmark/BenchmarkConfiguration.java | 16 ++++------------ .../sheet/benchmark/util/BenchmarkFileUtil.java | 8 +++----- .../sheet/benchmark/util/DataGenerator.java | 10 ++++------ 3 files changed, 11 insertions(+), 23 deletions(-) diff --git a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/BenchmarkConfiguration.java b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/BenchmarkConfiguration.java index 8c77524e1..084ac0a9d 100644 --- a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/BenchmarkConfiguration.java +++ b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/BenchmarkConfiguration.java @@ -19,6 +19,8 @@ package org.apache.fesod.sheet.benchmark; +import lombok.Getter; + /** * Shared constants for the benchmark suites. */ @@ -30,6 +32,7 @@ private BenchmarkConfiguration() {} * Dataset sizes used by the benchmark suites. Rows have 20 columns each * (see {@link org.apache.fesod.sheet.benchmark.data.BenchmarkData}). */ + @Getter public enum DatasetSize { SMALL(1_000, "1K"), MEDIUM(10_000, "10K"), @@ -42,20 +45,13 @@ public enum DatasetSize { this.rowCount = rowCount; this.label = label; } - - public int getRowCount() { - return rowCount; - } - - public String getLabel() { - return label; - } } /** * File formats covered by the benchmark suites. Note that XLS is limited * to 65,536 rows per sheet. */ + @Getter public enum FileFormat { XLSX("xlsx"), XLS("xls"), @@ -66,9 +62,5 @@ public enum FileFormat { FileFormat(String extension) { this.extension = extension; } - - public String getExtension() { - return extension; - } } } diff --git a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/BenchmarkFileUtil.java b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/BenchmarkFileUtil.java index cd3a09f4b..0d44bb702 100644 --- a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/BenchmarkFileUtil.java +++ b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/BenchmarkFileUtil.java @@ -24,18 +24,16 @@ import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.Paths; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; +import lombok.extern.slf4j.Slf4j; /** * Central place for the scratch directory benchmark files are written to * ({@code target/benchmark-testdata}), so all suites share one location that * Maven cleans up on {@code mvn clean}. */ +@Slf4j public class BenchmarkFileUtil { - private static final Logger logger = LoggerFactory.getLogger(BenchmarkFileUtil.class); - private static final String TEST_DATA_DIR = "target/benchmark-testdata"; /** @@ -71,7 +69,7 @@ public static void delete(File file) { try { Files.deleteIfExists(file.toPath()); } catch (IOException e) { - logger.warn("Failed to delete benchmark file {}: {}", file, e.getMessage()); + log.warn("Failed to delete benchmark file {}: {}", file, e.getMessage()); } } } diff --git a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/DataGenerator.java b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/DataGenerator.java index 9cab6ed5e..e5fb2daca 100644 --- a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/DataGenerator.java +++ b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/util/DataGenerator.java @@ -27,20 +27,18 @@ import java.util.ArrayList; import java.util.List; import java.util.Random; +import lombok.extern.slf4j.Slf4j; import org.apache.fesod.sheet.benchmark.BenchmarkConfiguration; import org.apache.fesod.sheet.benchmark.data.BenchmarkData; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; /** * Generates deterministic benchmark data: a fixed default seed and a fixed date * anchor mean the same seed always produces the same rows, so benchmark results * from different runs are comparable. */ +@Slf4j public class DataGenerator { - private static final Logger logger = LoggerFactory.getLogger(DataGenerator.class); - /** Fixed anchor for generated dates; never uses the wall clock. */ private static final LocalDate DATE_ANCHOR = LocalDate.of(2024, 1, 1); @@ -112,7 +110,7 @@ public List generateData(BenchmarkConfiguration.DatasetSize size) } public List generateData(int rowCount) { - logger.info("Generating {} rows of benchmark data", rowCount); + log.info("Generating {} rows of benchmark data", rowCount); List data = new ArrayList<>(rowCount); long startTime = System.currentTimeMillis(); @@ -122,7 +120,7 @@ public List generateData(int rowCount) { } long duration = System.currentTimeMillis() - startTime; - logger.info( + log.info( "Generated {} rows in {} ms ({} rows/sec)", rowCount, duration, From b1b3f5d0b0b2e18a79b1d579c7a72c2e913425a1 Mon Sep 17 00:00:00 2001 From: gongzhongqiang Date: Mon, 12 Oct 2026 19:47:56 +0800 Subject: [PATCH 4/4] refactor(benchmark): use 3 forks for the comparison suites too Align the fork count of FesodVsPoiBenchmark and StreamingBenchmark with FesodBenchmark so every suite exposes cross-fork JVM/JIT variability in its error bars and the suites are comparable with each other. The original single fork on the comparison suites was only a wall-clock convenience for manual runs, not a measurement choice. Also document the deliberate heap difference (core suite 1g/G1GC vs comparison 2g) under the contracts table. --- fesod-benchmark/benchmark.md | 10 ++++++++-- .../fesod/sheet/benchmark/FesodVsPoiBenchmark.java | 2 +- .../fesod/sheet/benchmark/StreamingBenchmark.java | 2 +- 3 files changed, 10 insertions(+), 4 deletions(-) diff --git a/fesod-benchmark/benchmark.md b/fesod-benchmark/benchmark.md index fc3b476fa..3672e8ab2 100644 --- a/fesod-benchmark/benchmark.md +++ b/fesod-benchmark/benchmark.md @@ -59,8 +59,14 @@ results collected at different times stay comparable: | Suite | Mode | Forks | Warmup | Measurement | JVM args | |---|---|---|---|---|---| | `FesodBenchmark` | avgt, ms/op | 3 | 3 × 1s | 5 × 2s | `-Xms1g -Xmx1g -XX:+UseG1GC` | -| `FesodVsPoiBenchmark` | avgt, ms/op | 1 | 3 × 5s | 5 × 5s | `-Xms2g -Xmx2g` | -| `StreamingBenchmark` | avgt, ms/op | 1 | 3 × 5s | 5 × 5s | `-Xms2g -Xmx2g` | +| `FesodVsPoiBenchmark` | avgt, ms/op | 3 | 3 × 5s | 5 × 5s | `-Xms2g -Xmx2g` | +| `StreamingBenchmark` | avgt, ms/op | 3 | 3 × 5s | 5 × 5s | `-Xms2g -Xmx2g` | + +All suites use 3 forks so cross-fork JVM/JIT variability is visible in the +error bars. The core suite deliberately keeps a smaller fixed heap (`1g`, +G1GC) — a roomy constant heap removes GC adaptivity from the timing signal — +while the comparison suites use `2g` so POI's in-memory workloads are not the +constraint being measured. ## Notes on interpreting results diff --git a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/FesodVsPoiBenchmark.java b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/FesodVsPoiBenchmark.java index 7e396504c..87aa15050 100644 --- a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/FesodVsPoiBenchmark.java +++ b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/FesodVsPoiBenchmark.java @@ -93,7 +93,7 @@ @Warmup(iterations = 3, time = 5, timeUnit = TimeUnit.SECONDS) @Measurement(iterations = 5, time = 5, timeUnit = TimeUnit.SECONDS) @Fork( - value = 1, + value = 3, jvmArgs = {"-Xms2g", "-Xmx2g"}) public class FesodVsPoiBenchmark { diff --git a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/StreamingBenchmark.java b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/StreamingBenchmark.java index 9b45975b6..c2466ddbc 100644 --- a/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/StreamingBenchmark.java +++ b/fesod-benchmark/src/main/java/org/apache/fesod/sheet/benchmark/StreamingBenchmark.java @@ -93,7 +93,7 @@ @Warmup(iterations = 3, time = 5, timeUnit = TimeUnit.SECONDS) @Measurement(iterations = 5, time = 5, timeUnit = TimeUnit.SECONDS) @Fork( - value = 1, + value = 3, jvmArgs = {"-Xms2g", "-Xmx2g"}) public class StreamingBenchmark {