Skip to content

Fix WorldBank urls and outdated data usage - #1292

Open
spjuhel wants to merge 20 commits into
developfrom
fix-worldbank-urls
Open

Fix WorldBank urls and outdated data usage#1292
spjuhel wants to merge 20 commits into
developfrom
fix-worldbank-urls

Conversation

@spjuhel

@spjuhel spjuhel commented May 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Changes proposed in this PR:

  • Remove the broken WORLD_BANK_WEALTH_ACC URL constant and FILE_WORLD_BANK_WEALTH_ACC filename constant
  • Rewrite world_bank_wealth_account to fetch data via download_world_bank_indicator instead of downloading and unzipping a CSV file; interpolation/extrapolation logic and the no_land correction are unchanged
  • Remove now-unused imports: zipfile, Path
  • Update docstring year range for world_bank_wealth_account from 1995–2014 to 1995–2020 to reflect the current dataset coverage
  • Update docstring and documentation to reflect change to 2019 real chained USD (from constant 2014)
  • Update test values to reflect changes

This PR fixes #1291 and #1293

PR Author Checklist

PR Reviewer Checklist

@spjuhel spjuhel mentioned this pull request May 15, 2026
Open
This was linked to issues May 15, 2026
world_bank_wealth_account fails with an SSL certificate error #1291
Open
WORLD_BANK_INC_GRP url is deprecated #1293
Open
spjuhel and others added 15 commits May 15, 2026 11:19
@spjuhel
Changelog
32cf61b
@spjuhel
Updates doc for real chained 2019 USD, fixes tests values
59d410e
@emanuel-schmid @spjuhel
purge .gitignore
b9d13cd
@spjuhel
makes soon to be old measures module a legacy
1d67a34
@spjuhel
Oups these shouldn't be here!
7655178
@spjuhel
Forgotten places
f648a6f
@spjuhel
Updates changelog
254a605
@spjuhel
Adds future documentation placeholders, warnings in legacy guides
dd804b7
@spjuhel
This one does not exist yet
9420254
@spjuhel
Updates rst files
3132e78
@spjuhel
Includes comment from Lukas (doc refs)
a610c8f
@spjuhel
Forgot cost-benefit tutorial
d4f1c01
@simonameiler Simona Meiler
Chahan Kropf @spjuhel
Fix TCTracks.from_FAST duplicate loading from year loop (#1269)
5a578c0 
* Fix TCTracks.from_FAST duplicate loading from year loop

* Refactor test_from_FAST: extend fixture to 2 years, remove separate regression test

The test fixture FAST_test_tracks.nc now has year=[2025,2026] (only
seeds_per_month is extended; track variables retain their n_trk dim).
The existing len(tc_track.data)==5 assertion now acts as the regression
check: the buggy year-loop code would return 5x2=10 tracks.

The separate test_from_FAST_not_multiplied_by_year_dim (with its
temporary-file scaffolding) is removed.

* Update changelo

---------

Co-authored-by: Simona Meiler <simona.meiler@usys.ethz.ch>
Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com>
@spjuhel
Changelog
bfeacb8
@spjuhel
Merge remote-tracking branch 'origin/develop' into fix-worldbank-urls
e05f8e5
@spjuhel spjuhel changed the title Fix worldbank urls Fix WorldBank urls and outdated data usage May 15, 2026
@spjuhel
spjuhel marked this pull request as ready for review May 15, 2026 12:56
@spjuhel

spjuhel commented May 15, 2026

Copy link
Copy Markdown
Collaborator Author

@emanuel-schmid or @peanutfun, this should be a quick one to review. And will fix some of the currently failing tests.

@spjuhel spjuhel mentioned this pull request May 19, 2026
Merged
13 tasks
@carmensteinmann

carmensteinmann commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

There are some administrative regions, which lead to an error when trying to create LitPop for them.
I started from an old, global LitPop object and wanted to create a new LitPop object for all region ids. The failing admin units are: ['ALA', 'ESH', 'HMD', 'PCN', 'SGS', 'UMI']
These are: Åland Islands, Western Sahara, Heard Island and McDonald Islands, Pitcairn Islands, South Georgia and the South Sandwich Islands, United States Minor Outlying Islands

Please find the code how to determine these below.

`country_names = global_litpop.gdf.region_id.unique()
iso3a_list = u_coord.country_to_iso(country_names, representation="alpha3")

info_ind = "NY.GDP.MKTP.CD"
data = []
network = []

for cntry_iso in iso3a_list:
try:
cntry_gdp = util_fin.download_world_bank_indicator(indicator=info_ind, country_code=cntry_iso, parse_dates=False)
except (ReadTimeout, RequestException):
print(f"Network issue: {cntry_iso}")
network.append(cntry_iso)
except ValueError as e:
print(f"Data/API issue for {cntry_iso}: {e}")
data.append(cntry_iso)

failing_c = []
for country in data:
print(country)
try:
exp = LitPop.from_countries(country, reference_year=2002)
except ValueError as e:
failing_c.append(country)
`

emanuel-schmid
emanuel-schmid reviewed Sep 23, 2026
View reviewed changes

@emanuel-schmid emanuel-schmid left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. In any case an improvement! 🥇
Happy to merge.

Comment thread climada/util/finance.py
Credit Suisse's Global Wealth Report 2017 (household wealth)"""

WB_URBAN_LAND_MARKUP = 1.24

@emanuel-schmid emanuel-schmid Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why the name and what does it do?

Comment thread climada/util/test/test_finance.py
14676399165833.0, # 15/05/2026 from WB website
]
) # Values updated by worldbank at different dates
ref_val_noland = ref_val / 1.24 # dito

@emanuel-schmid emanuel-schmid Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • # dito doesn't fit anymore
  • I'd replace the factor 1.24 by its name, WB_URBAN_LAND_MARKUP. (if one day they change it, we don't have to come back here an wonder.)

Comment thread climada/util/finance.py
(applies to 'NW.PCA.*' only). Default: True.
(applies to 'NW.PCA.*' only). The world bank applies a 24% markup
to include the built-up (or urban) land value, thus this divides
by a factor 1.24.

@emanuel-schmid emanuel-schmid Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this explains the WB_URBAN_LAND_MARKUP 😄 , add a reference

Comment thread climada/util/finance.py
Return requested variable for a country (cntry_iso) and a year (ref_year).
Download wealth accounting data from the World Bank API and return the requested
variable for a country (cntry_iso) and a year (ref_year).
https://datacatalog.worldbank.org/search/dataset/0042066

@emanuel-schmid emanuel-schmid Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this url correct? wasn't it meant to be replaced?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@emanuel-schmid emanuel-schmid emanuel-schmid requested changes
@chahank chahank Awaiting requested review from chahank chahank is a code owner
@peanutfun peanutfun Awaiting requested review from peanutfun peanutfun is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

WORLD_BANK_INC_GRP url is deprecated world_bank_wealth_account fails with an SSL certificate error

4 participants

@spjuhel @carmensteinmann @emanuel-schmid @simonameiler