Repository navigation
Conversation
Signed-off-by: sschepens <sebastian.schepens@mercadolibre.com>
|
@sschepens Thanks for the PR. Please merge main /wait |
Signed-off-by: sschepens <sebastian.schepens@mercadolibre.com>
There was a problem hiding this comment.
Thanks for this contribution. To access the nested structure of metadata with ABI is a tricky problem and thanks for you to take it!
One comment to the ABI to start it. :)
By the way, all new ABI should have a weak stub test in test/extensions/dynamic_modules/abi_impl_test.cc
| envoy_dynamic_module_type_http_filter_envoy_ptr filter_envoy_ptr, | ||
| envoy_dynamic_module_type_metadata_source metadata_source, | ||
| envoy_dynamic_module_type_module_buffer ns, const envoy_dynamic_module_type_module_buffer* path, | ||
| size_t path_length, envoy_dynamic_module_type_envoy_buffer* result); |
There was a problem hiding this comment.
With these new ABI, we could access the nested map or a nested list. But it's impossible to access a map in the list.
I guess we could do it in this way:
Create a new Type, like the envoy_dynamic_module_type_module_meadata_path which contains a buffer and an index. The index only takes effect when the buffer is empty.
Then, we could use it to access any element in the metadata, and we basically needn't the envoy_dynamic_module_callback_http_get_metadata_nested_list_string variant.
There was a problem hiding this comment.
pushed a new implementation with a struct as you suggested, that allows for walking nested objects and lists and removed the list variants
|
@wbpcode friendly maintainer on-call ping |
|
Seems the CI is broken. Could you merge main again? |
Signed-off-by: sschepens <sebastian.schepens@mercadolibre.com>
Signed-off-by: sschepens <sebastian.schepens@mercadolibre.com>
|
@wbpcode merged main and fixed CI |
agrawroh
left a comment
There was a problem hiding this comment.
💚 Approved. The key or index encoding is worth settling before the next release cut freezes it.
- 💡 A null key pointer marks an index, and the Go SDK's string helper can encode an empty string as a null pointer, so a Go binding would turn an empty key into an index.
- 💡 Each element is one path component, and this is the only untagged struct in the header, so C code can't forward-declare it.
- 💡 A miss past the first component logs nothing, while a first-component miss logs at debug, so a wrong path is hard to find.
- 💡 A five-component path, such as a role under one client in JWT claims, spills to the heap on every call.
- 💡 Could the path component derive equality, as most SDK enums do?
- 💡 Metadata paths don't go through this helper, since
metadata_path_to_abibuilds its own buffer. - 💡 Could the note name the new callbacks and Rust methods, as most dynamic module notes do?
- 💡 This block landed between the dynamic metadata config's doc comment and its struct, so that comment now describes this config.
- 💡 This passes with a bool getter that reports every bool as false, a walk that ignores the source, or one that ignores the key length.
- 💡 The module reads only
Dynamic, never a true bool and neverKey(""), so a wrapper that drops the source, reports every bool as false or turns an empty key into an index still passes. - 💡 Could the description list the new tests under Testing, point Release Notes at the changelog fragment and mark Docs Changes N/A, as the template asks?
- 🔄 Could we track bringing these path getters to the Go and C++ SDKs, and to the access logger and the other extensions whose metadata getters split a dotted path?
- 🔄 Could we track a batched getter for a nested list's values, as
envoy_dynamic_module_callback_http_get_header_valuesdoes for headers? Reading a JWT groups claim now walks from the namespace once per element.
|
|
||
| /** | ||
| * A component of a metadata path: a literal struct key or a zero-based list index. | ||
| * When key.ptr is non-null, key selects a struct field and index is ignored. A non-null |
There was a problem hiding this comment.
💡 A null key pointer marks an index, and the Go SDK's string helper can encode an empty string as a null pointer, so a Go binding would turn an empty key into an index. Could the component carry an explicit key or index kind, as MetadataKey.PathSegment does, while it's unreleased?
| * pointer with zero length selects an empty key. When key.ptr is null, key.length must be | ||
| * zero and index selects a list element. | ||
| */ | ||
| typedef struct { |
There was a problem hiding this comment.
💡 Each element is one path component, and this is the only untagged struct in the header, so C code can't forward-declare it. Could we tag it and name it for a component, such as envoy_dynamic_module_type_module_metadata_path_component?
| return nullptr; | ||
| } | ||
| const auto* value = getMetadataValue(filter_envoy_ptr, metadata_source, ns, path[0].key); | ||
| for (size_t i = 1; i < path_length; ++i) { |
There was a problem hiding this comment.
💡 A miss past the first component logs nothing, while a first-component miss logs at debug, so a wrong path is hard to find. Could each miss in the walk log the failing component at debug, as core's metadata walk does?
|
|
||
| fn metadata_path_to_abi( | ||
| path: &[MetadataPathComponent<'_>], | ||
| ) -> smallvec::SmallVec<[abi::envoy_dynamic_module_type_module_metadata_path; 4]> { |
There was a problem hiding this comment.
💡 A five-component path, such as a role under one client in JWT claims, spills to the heap on every call. Could the inline buffer hold eight components? The heap-path test would then need nine.
| /// Converts strings to module buffers for labeled metric calls and metadata paths. | ||
| /// | ||
| /// The result is a `SmallVec` inlined for the common case of a few labels, so the typical one to | ||
| /// four labels need no heap allocation. Callers must take `as_ptr` or `as_mut_ptr` on the bound | ||
| /// The result is a `SmallVec` inlined for the common case of one to four strings, avoiding a heap | ||
| /// allocation. Callers must take `as_ptr` or `as_mut_ptr` on the bound |
There was a problem hiding this comment.
💡 Metadata paths don't go through this helper, since metadata_path_to_abi builds its own buffer. Could we keep the original comment here and put the move caveat on metadata_path_to_abi?
| /// Converts strings to module buffers for labeled metric calls and metadata paths. | |
| /// | |
| /// The result is a `SmallVec` inlined for the common case of a few labels, so the typical one to | |
| /// four labels need no heap allocation. Callers must take `as_ptr` or `as_mut_ptr` on the bound | |
| /// The result is a `SmallVec` inlined for the common case of one to four strings, avoiding a heap | |
| /// allocation. Callers must take `as_ptr` or `as_mut_ptr` on the bound | |
| /// Converts label name or value strings to module buffers for a labeled metric call. | |
| /// | |
| /// The result is a `SmallVec` inlined for the common case of a few labels, so the typical one to | |
| /// four labels need no heap allocation. Callers must take `as_ptr` or `as_mut_ptr` on the bound |
| /// A HTTP filter configuration that implements | ||
| /// [`envoy_proxy_dynamic_modules_rust_sdk::HttpFilterConfig`] to test the dynamic metadata related | ||
| /// callbacks. | ||
| struct NestedMetadataCallbacksFilterConfig {} |
There was a problem hiding this comment.
💡 This block landed between the dynamic metadata config's doc comment and its struct, so that comment now describes this config. Could we move the new block above the comment and give it its own?
|
|
||
| TestUtility::loadFromJson(R"EOF({ | ||
| "claims": { | ||
| "sub": "user-123", "exp": 123, "active": false, |
There was a problem hiding this comment.
💡 This passes with a bool getter that reports every bool as false, a walk that ignores the source, or one that ignores the key length. Could we add a true claim, a route value that differs from the dynamic one, and a key after the first that's sliced from a longer buffer?
| envoy_filter: &mut EHF, | ||
| _end_of_stream: bool, | ||
| ) -> abi::envoy_dynamic_module_type_on_http_filter_request_headers_status { | ||
| use abi::envoy_dynamic_module_type_metadata_source::Dynamic; |
There was a problem hiding this comment.
💡 The module reads only Dynamic, never a true bool and never Key(""), so a wrapper that drops the source, reports every bool as false or turns an empty key into an index still passes. Could the module and its fixture cover those three?
Co-authored-by: Rohit Agrawal <rohit.agrawal@databricks.com> Signed-off-by: Sebastian Schepens <sebastian.schepens@mercadolibre.com>
Co-authored-by: Rohit Agrawal <rohit.agrawal@databricks.com> Signed-off-by: Sebastian Schepens <sebastian.schepens@mercadolibre.com>
Commit Message: add nested metadata getters to HTTP ABI and Rust SDK
Additional Description: Fixes #48440
Risk Level: low
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]