Repository navigation

Filter metadata to avoid storing excess text in the database table - #612

Merged

meeb merged 27 commits into

meeb:mainfrom

tcely:filter-metadata-response

Jan 14, 2025

Merged

Filter metadata to avoid storing excess text in the database table#612
meeb merged 27 commits into
meeb:mainfrom
tcely:filter-metadata-response

Commits on Jan 7, 2025
- Add response filtering
  tcely
  authored
  8c22b6c
- More compact JSON
  tcely
  authored
  63fa97c
- Log the reduction of metadata length
  tcely
  authored
  8c31720
- Merge branch 'meeb:main' into filter-metadata-response
  tcely
  authored
  ca75398
- Don't reduce the actual data yet
  tcely
  authored
  25d2ff6
- Fixes from testing
  tcely
  authored
  2f34fff
- Fix formatting
  tcely
  authored
  9a4101a
- Adjusted comment
  tcely
  authored
  db25fa8
- Loop over a set of keys for each URL type
  tcely
  authored
  431de2e
- Drop keys from formats that cannot be useful
  tcely
  authored
  7b8d117
- Check that the drop_key exists
  tcely
  authored
  c7457e9
- Use a distinct try to log errors
  tcely
  authored
  2d85bcb
- Use the exception function for traceback
  tcely
  authored
  8ac5b36
- Simplify results from _url_keys
  tcely
  authored
  7793701
- Some formats are using a different URL
  tcely
  authored
  1c432cc
Commits on Jan 8, 2025
- Drop /expire/ URLs from automatic_captions too
  tcely
  authored
  d35f52f
Commits on Jan 9, 2025
- Log both compacted and reduced sizes
  tcely
  authored
  ad10bcf
- Rename compact_data to compact_json
  tcely
  authored
  100382f
- Add a filter_response test
  tcely
  authored
  682a53d
- More filter_response asserts
  tcely
  authored
  4c9fa40
- More filter_response asserts
  tcely
  authored
  3e3f80d
- Add SHRINK_NEW_MEDIA_METADATA setting
  tcely
  authored
  29c39aa
- Have filter_response return a copy, if requested
  tcely
  authored
  0f98694
- Use the new copy argument to filter_response
  tcely
  authored
  274f19f
- Use the new copy argument to filter_response
  tcely
  authored
  1ff8dfd
- Add SHRINK_OLD_MEDIA_METADATA setting
  tcely
  authored
  6292a9a
Commits on Jan 14, 2025
- Only log the extra messages with the new setting
  tcely
  authored
  45d7039

Commits

Commits on Jan 7, 2025

Commits on Jan 8, 2025

Commits on Jan 9, 2025

Commits on Jan 14, 2025