Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,6 @@ trim_trailing_whitespace = true

[*.{yml, yaml}]
indent_size = 2

[phpstan-baseline.neon]
indent_style = tab
6 changes: 6 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
/.* export-ignore
/composer.lock export-ignore
/phpcs.xml.dist export-ignore
/phpstan-baseline.neon export-ignore
/phpstan.neon.dist export-ignore
/phpunit.xml.dist export-ignore
/tests export-ignore
/docs/specs export-ignore
/CLAUDE.md export-ignore
/utils/db-prepare.sh export-ignore
/utils/SchemaDiff.php export-ignore
/utils/db-schema-diff.php export-ignore
/utils/db-schema-diff-accepted.txt export-ignore
85 changes: 61 additions & 24 deletions .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,40 +11,46 @@ permissions:

jobs:
tests:
name: PHP ${{ matrix.php }} Test
name: PHP ${{ matrix.php }}, ${{ matrix.db }}
runs-on: ubuntu-latest

strategy:
fail-fast: false
matrix:
php: ['8.1', '8.2', '8.3', '8.4', '8.5']

# Every PHP version on the current MariaDB LTS; the newest PHP also on the
# oldest maintained MariaDB LTS and on both maintained MySQL LTS lines.
include:
- php: '8.1'
db: 'mariadb:12.3'
- php: '8.2'
db: 'mariadb:12.3'
- php: '8.3'
db: 'mariadb:12.3'
- php: '8.4'
db: 'mariadb:12.3'
- php: '8.5'
db: 'mariadb:12.3'
- php: '8.5'
db: 'mariadb:10.11'
- php: '8.5'
db: 'mysql:9.7'
- php: '8.5'
db: 'mysql:8.4'

# No health check: the admin binary differs per image, and db-prepare waits for the server.
services:
mariadb:
image: mariadb:10.3
db:
image: ${{ matrix.db }}
ports:
- 3306:3306
env:
MYSQL_ROOT_PASSWORD: root
options: --health-cmd="mysqladmin ping" --health-interval=5s --health-timeout=2s --health-retries=3
MARIADB_ROOT_PASSWORD: root

steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Verify MariaDB connection
run: |
while ! mysqladmin ping -h127.0.0.1 -P3306 --silent; do
sleep 1
done

- name: Test migrations
run: |
git fetch origin 0.44.1 && git checkout FETCH_HEAD -- structure.sql
mysql -h127.0.0.1 -P3306 -uroot -proot -e "create database telegrambot_migrations; use telegrambot_migrations; source structure.sql;"
git checkout HEAD -- structure.sql
mysql -h127.0.0.1 -P3306 -uroot -proot -e "create database telegrambot; use telegrambot; source structure.sql;"
for SCHEMA_UPDATE_FILE in $(ls utils/db-schema-update); do mysql -h127.0.0.1 -P3306 -uroot -proot -e "use telegrambot_migrations; source utils/db-schema-update/${SCHEMA_UPDATE_FILE};"; done;

- name: Install PHP
uses: shivammathur/setup-php@eb7c497e18156a6bbabfef1d3a82760b9eda3962 # 2.40.0
with:
Expand All @@ -66,6 +72,14 @@ jobs:
- name: Install dependencies
run: composer install --prefer-dist --no-progress

# Fresh schema, then the 0.44.1 schema plus every migration, then the diff of the two.
- name: Prepare databases and diff the schemas
env:
DB_PASSWORD: root
run: |
composer db-prepare
composer schema-diff

- name: Check PHP code
run: composer check-code

Expand All @@ -83,13 +97,36 @@ jobs:
# composer test-cov
# composer test-cov-upload

# Single status for branch protection, so the PHP matrix can change without touching repository settings.
# Runs once, on the newest PHP; the errors that existed when PHPStan was added are listed in phpstan-baseline.neon.
static-analysis:
name: Static analysis
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Install PHP
uses: shivammathur/setup-php@eb7c497e18156a6bbabfef1d3a82760b9eda3962 # 2.40.0
with:
php-version: '8.5'

- name: Install dependencies
run: composer install --prefer-dist --no-progress

- name: Run static analysis
run: composer analyse

# Single status for branch protection, so jobs and the PHP matrix can change without touching repository
# settings. Every other job must be listed in its needs.
tests-passed:
name: Tests
if: always()
needs: tests
needs: [tests, static-analysis]
runs-on: ubuntu-latest

steps:
- name: Check matrix result
run: test "${{ needs.tests.result }}" = "success"
- name: Check job results
run: |
test "${{ needs.tests.result }}" = "success"
test "${{ needs.static-analysis.result }}" = "success"
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,13 @@ Exclamation symbols (:exclamation:) note something of importance e.g. breaking c
## [Unreleased]
### Notes
- [:ledger: View file changes][Unreleased]
- A MySQL database upgraded past 0.61.0 needs DB migration scripts run again: `0.60.0-0.61.0.sql` and `0.62.0-0.63.0.sql`, then `0.82.0-0.83.0.sql` if the database is at 0.83.0 or later. Other files that applied without an error must not be run again. The migration scripts are tested on MariaDB 10.11 and 12.3 and on MySQL 8.4 and 9.7.
### Added
### Changed
### Deprecated
### Removed
### Fixed
- DB migration scripts `0.60.0-0.61.0.sql` and `0.62.0-0.63.0.sql` now run on MySQL, and both can be run more than once on any database. `0.62.0-0.63.0.sql` was meant to drop a duplicate foreign key on `message.forward_from` named `message_ibfk_6`; servers that name it differently, such as MariaDB 12, kept the duplicate, and a re-run of the file removes it. (@akalongman) (#1505)
### Security

## [0.84.0] - 2026-11-11
Expand Down
19 changes: 14 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,17 +10,22 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

```bash
composer install
composer check-code # phpcs, PSR-12 over src/, tests/, utils/ (must print nothing)
composer check-code # phpcs, PSR-12 over src/, tests/, utils/ (must report no violations)
composer analyse # PHPStan level 5 over src/, must exit 0; after fixing an error listed in phpstan-baseline.neon, regenerate that file with vendor/bin/phpstan analyse --generate-baseline
composer test # phpunit, whole suite
vendor/bin/phpunit tests/Unit/Entities/MessageTest.php
vendor/bin/phpunit --filter testGetCommand
vendor/bin/phpcbf # auto-fix coding standard violations
composer test-cov # coverage (needs Xdebug), writes clover.xml
composer db-prepare # drops and recreates telegrambot and telegrambot_migrations on the server DB_HOST and DB_PORT name (default 127.0.0.1:3306), so point it at a throwaway server; stops with exit 1 before dropping anything if either holds a row, unless DB_PREPARE_FORCE=1
composer schema-diff # compares those two databases: exit 0 clean, 1 a difference not in utils/db-schema-diff-accepted.txt or an entry there that matches no difference, 2 a usage, connection or schema read error, an unreadable accepted file, or a missing or empty database
```

Test database: only `tests/Unit/ConversationTest.php` touches MySQL. It reads the `PHPUNIT_DB_*` constants from the PHPUnit config and expects a database (default name `telegrambot`) loaded from `structure.sql`. To use other credentials, copy `phpunit.xml.dist` to `phpunit.xml` (gitignored) and edit the constants there. `TestHelpers::emptyDb()` deletes every row from the core tables, so never point it at a database holding real data.
PHPStan is pinned to an exact version in `composer.json`, because `composer.lock` is not committed and the baseline records exact error counts per file; a commit that raises that version also regenerates `phpstan-baseline.neon`.

CI runs MariaDB 10.3, which is more permissive than MySQL 8.4 (for example, MySQL 8.4 rejects a foreign key that references a non-unique column). Keep `structure.sql` loadable on both with default settings. The older migration files are MariaDB-only (`DROP FOREIGN KEY IF EXISTS`).
Test database: `tests/Unit/ConversationTest.php` and `tests/Unit/DBTest.php` touch MySQL. They read the `PHPUNIT_DB_*` constants from the PHPUnit config and expect a database (default name `telegrambot`) loaded from `structure.sql`. To use other credentials, copy `phpunit.xml.dist` to `phpunit.xml` (gitignored) and edit the constants there. `TestHelpers::emptyDb()` deletes every row from the core tables, so never point it at a database holding real data.

CI runs every job against its own fresh database server: PHP 8.1 through 8.5 on MariaDB 12.3, and PHP 8.5 on MariaDB 10.11, MySQL 9.7 and MySQL 8.4. On each of these four servers it runs the migration chain and the schema diff (see Database schema), then the test suite against the `telegrambot` database that `composer db-prepare` built. MySQL is stricter than MariaDB (for example, it rejects a foreign key that references a non-unique column), so keep `structure.sql` loadable on all four with default settings.

The PHPUnit config defines the constant `PHPUNIT_TESTSUITE`. When it is defined, `Request::send()` returns a fake `ServerResponse` built by `Request::generateGeneralFakeServerResponse()` and never calls the network, so tests need no HTTP mocking and cannot assert on real API behavior.

Expand Down Expand Up @@ -67,11 +72,15 @@ A new API version usually touches several places that must stay in step:

- New type: entity class with `@method` docblock and `subEntities()`, plus the parent entity's map.
- New method: `Request::$actions` and the `@method` docblock on `Request`.
- New update type: `Update` constant and `subEntities()` entry. If it is persisted, also a `DB::insert*Request()` method wired into `DB::insertRequest()`, the table in `DB::defineTables()`, the `telegram_update` column, `structure.sql`, and a migration.
- New update type: `Update` constant and `subEntities()` entry. If it is persisted, also a `DB::insert*Request()` method wired into `DB::insertRequest()`, the table in `DB::defineTables()`, its `TB_` constant in the `ignoreErrors` list of `phpstan.neon.dist`, the `telegram_update` column, `structure.sql`, and a migration.

### Database schema

`structure.sql` is the full current schema. Each release that changes it also adds `utils/db-schema-update/<from>-<to>.sql`; during development the file is named `<last release>-unreleased.sql` and linked from the `[Unreleased]` notes in `CHANGELOG.md`. CI loads the 0.44.1 schema, applies every migration in order, and separately loads the current `structure.sql`; both must succeed, so a schema change needs both files edited. Note that CI does not diff the two results.
`structure.sql` is the full current schema. Each release that changes it also adds `utils/db-schema-update/<from>-<to>.sql`; during development the file is named `<last release>-unreleased.sql` and linked from the `[Unreleased]` notes in `CHANGELOG.md`. On each of the four CI servers, `composer db-prepare` loads the current `structure.sql` into `telegrambot` and the 0.44.1 schema plus every migration in order into `telegrambot_migrations`, stopping at the first file that fails. `composer schema-diff` then compares the two through `information_schema`: tables with their engine and collation; columns with type, nullability, default, character set, collation and extra attributes such as `AUTO_INCREMENT`; indexes by column with whether each is the primary key, unique or non-unique; and foreign keys by column with the referenced columns and their `ON DELETE` and `ON UPDATE` rules (not comments, column order or key names). It fails on any difference not listed in `utils/db-schema-diff-accepted.txt`, and on any entry there that no longer matches a difference. Both must pass, so a schema change needs both files edited to the same result. Adding an entry to the accepted list needs the maintainer's agreement.

Every migration is plain SQL that runs unchanged on all four servers: no stored procedures and no client commands (`source`, `delimiter`). MariaDB-only syntax such as `DROP FOREIGN KEY IF EXISTS` fails on MySQL, and generated key names differ between servers, so look a key up by column in `information_schema` and apply the change through `PREPARE`, as `0.83.1-0.84.0.sql` does.

To run both scripts locally, start a throwaway server and point them at its port: `docker run -d --name tg-db -p 127.0.0.1:33123:3306 -e MARIADB_ROOT_PASSWORD=root mariadb:12.3`, then `DB_PORT=33123 DB_PASSWORD=root composer db-prepare` and `DB_PORT=33123 DB_PASSWORD=root composer schema-diff`, and remove the server afterwards with `docker rm -f tg-db`.

## Conventions

Expand Down
13 changes: 12 additions & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
"guzzlehttp/guzzle": "^7.15.2"
},
"require-dev": {
"phpstan/phpstan": "2.3.5",
"phpunit/phpunit": "^9.5",
"squizlabs/php_codesniffer": "^3.6",
"dms/phpunit-arraysubset-asserts": "^0.2",
Expand All @@ -50,13 +51,17 @@
},
"autoload-dev": {
"psr-4": {
"Longman\\TelegramBot\\Tests\\Unit\\": "tests/Unit"
"Longman\\TelegramBot\\Tests\\Unit\\": "tests/Unit",
"Longman\\TelegramBot\\Utils\\": "utils"
}
},
"scripts": {
"check-code": [
"\"vendor/bin/phpcs\""
],
"analyse": [
"\"vendor/bin/phpstan\" analyse --memory-limit=1G"
],
"test": [
"\"vendor/bin/phpunit\""
],
Expand All @@ -65,6 +70,12 @@
],
"test-cov-upload": [
"@php ocular.phar code-coverage:upload --format=php-clover clover.xml"
],
"db-prepare": [
"bash utils/db-prepare.sh"
],
"schema-diff": [
"@php utils/db-schema-diff.php telegrambot telegrambot_migrations"
]
},
"config": {
Expand Down
36 changes: 28 additions & 8 deletions docs/specs/2026-11-11-ci-hardening-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,9 +81,9 @@ file, so more may appear once these are fixed):
Rules for a repaired file:

- It runs unchanged on MariaDB and MySQL.
- It can be run again on a database where an earlier attempt stopped part-way.
This is what lets a stuck MySQL user recover: re-run the file that failed,
then continue the chain.
- It can be run again on any database, including one where an earlier attempt
stopped part-way. This is what lets a stuck MySQL user recover: re-run, in
order, each file that stopped on their database, and none that applied.
- Foreign keys are found by column in `information_schema`, never by a
generated name such as `message_ibfk_6`.

Expand All @@ -93,6 +93,16 @@ stopped. Their `message` table has no `via_bot`. After this change they re-run
`0.62.0-0.63.0.sql`: the `poll` columns are detected as present and skipped,
the primary key is rebuilt, `via_bot` is added, and every later file applies.

A second case was measured on MySQL 8.4 after the final review (2026-11-12).
A user who kept upgrading release by release (Composer updates the library
whether or not a migration applied) ran every file and went on after each
error: `0.60.0-0.61.0.sql`, `0.62.0-0.63.0.sql` and `0.82.0-0.83.0.sql`
stopped, and every other file applied. Re-running only those three, in that
order, ends without an error and with a clean schema diff. Re-running every
file from the first failure on does not: ten files stop on duplicate or
unknown columns, and `0.72.0-0.73.0.sql` adds back a column that
`0.76.1-0.77.0.sql` had renamed. The changelog gives the first recovery.

Proof that MariaDB is unaffected: before touching a file, dump the migrated
MariaDB schema; after the repair the dump must be identical.

Expand All @@ -117,10 +127,20 @@ schema diff, and runs the test suite.
A script under `utils/` compares two databases on one server through
`information_schema` and prints every difference:

- tables present on one side only;
- per column: type, nullability, default, character set and collation;
- indexes with their columns and uniqueness;
- foreign keys by column and referenced column, ignoring generated names.
- tables present on one side only, and for a table on both sides its engine
and collation;
- per column: type, nullability, default, character set, collation and extra
attributes such as `AUTO_INCREMENT` and `ON UPDATE CURRENT_TIMESTAMP`;
- indexes with their columns and uniqueness, the primary key told apart from
a unique index on the same columns;
- foreign keys by column and referenced column, with their `ON DELETE` and
`ON UPDATE` rules, ignoring generated names.

Engine, collation, extra attributes, the primary key and the foreign key rules
were added after the final review (2026-11-12): drift in any of them breaks an
upgraded install while both schemas still load. For example, an `id` that is
`AUTO_INCREMENT` in `structure.sql` and plain `bigint` in the migration fails
every upgraded install's first insert.

Column comments and column order are not compared (maintainer may overrule;
see open calls). Accepted differences live in one small file next to the
Expand All @@ -144,7 +164,7 @@ baseline stays reviewable; the builder measures and proposes it.
### Documentation

`CLAUDE.md` (commands, the database section) and `CHANGELOG.md` are updated.
The changelog tells MySQL users which file to re-run if an old upgrade stopped.
The changelog tells MySQL users which files to re-run if an old upgrade stopped.

## Calls made by the author, open to correction

Expand Down
Loading
Loading