BIPM publications bibliographic data for Relaton-BIPM

The data sources are https://github.com/metanorma/bipm-data-outcomes and https://github.com/metanorma/bipm-si-brochure. Those datasets shoudl be places in bipm-data-outcomes and bipm-si-brochure directiries respectively. crawler.rb clones them itself (along with rawdata-bipm-metrologia), skipping any directory that is already present, so reruns work against an existing local checkout; delete a directory to force a fresh clone. Do not commit a crawler with those clones disabled — CI starts from a bare checkout and fails immediately without them (spec/crawler_sources_spec.rb guards this). This repository is used by Relaton-BIPM to represent known BIPM publications.

Data is stored in Relaton YAML format.

Indexes

crawler.rb publishes two indexes (both zipped and committed by CI):

• index-v2 — the pubid-backed runtime index (_type: pubid:bipm:*) that Relaton::Bipm consumes. Built by Relaton::Bipm::DataFetcher; the curated static/ docs are appended via BipmIndexBuilder.add_static_to_index_v2.

• index-v1 — the legacy bespoke {group,type,number,year} hash index, kept for backward-compatible consumers. Built by BipmIndexBuilder.build_index_v1 over every data/ and static/ record using the retained Relaton::Bipm::Id parser.

Both builders warn-and-skip an identifier their parser can’t handle rather than aborting the crawl.

Rebuilding index-v2 without a crawl

crawler.rb stays the source of truth for data/, but it needs RELATON_CI_PAT and hours of metanorma builds, so it only runs in CI. When the records are already current and only index-v2 must be rewritten — after a Pubid::Bipm change that alters the serialized rows — rebuild it from the committed records alone:

bundle exec ruby rebuild_index.rb

It rewrites index-v2.yaml and index-v2.zip, then prints the number coverage per identifier type. index-v1 is left alone: it is built with the bespoke Relaton::Bipm::Id, so a pubid change cannot affect it.

Relaton::Index narrows a lookup by binary-searching the index on the identifier’s number, so a row without one keys to the empty string. Exactly seven rows are number-less by design — the six ordinal-less committee declarations (data/**/[year]-00.yaml) and the journal-level data/metrologia.yaml. spec/index_coverage_spec.rb holds that line.

Tests

bundle install
bundle exec rspec