Support exposing the 'embeds' relationship.

[mukilan]

In Servo, we use AccessKit to handle the platform API mappings and we are trying to get the core-aam tests working on Linux.

The issue is that the fixture used in core-aam tests tries to locate the tab under test by following the 'embeds' relation. Since AccessKit currently doesn't expose (AFAIU) most of the AT-SPI relationships, this approach to finding the tab doesn't work when Servo exposes the tabs as children of the root frame.

On Linux, Orca requires support for the 'embeds' relationship and both Chrome and FF follow this approach.
I'm not too familiar with Windows, but based on the IA2 fixture used by core-aam test, 'embeds' relationship seems to be prioritized over direct children.

I don't know what would be the ideal API for this, but I'm creating this issue to start the discussion. Please let me know if you need more information.

[DataTriny]

In AT-SPI the "embeds" relationship is used to indicate that the embedded element's accessibility tree is managed by a different process than the host. As far as I know this doesn't apply to Servo's architecture where each tab accessibility subtree might be built inside a separate process but is ultimately sent to the main UI process where it is merged with the tree of the shell.

Does Servo want to move away from this architecture? The AccessKit AT-SPI adapter currently doesn't support neither embedding out-of-process trees nor being embedded.

[joanmarie]

@DataTriny: Orca needs a means to quickly identify the active, top-level document in a browser window, independent of the underlying architecture of that browser.

Way back in the Sun Microsystems days, it was agreed upon that the embeds/embedded-by relation pair was a reasonable way to accomplish that. That decision predates me. But it's what Gecko does, it's what Chromium does, and it should be what WebKitGTK does (though I'd have to triple-check that last one to be sure it is still true).

Having said that, if there is a different relation that you would prefer to use to connect the window to the active, top-level document, I would be willing to modify Orca to support that other relationship. Alternatively, if AccessKit were to provide a means for toolkits and applications to easily modify the relations of a particular object, Servo could then use that API to put the embeds/embedded-by relation pair in place (right??).

I'm afraid that I do NOT plan to work around the absence of some relation between the browser window and its active document by doing an accessibility tree dive. Tree dives are super non-performant. So we need to come up with something that works for you/AccessKit and Servo that doesn't negatively impact the Orca user experience.

Thanks in advance for considering this!

[mukilan]

@DataTriny Would it make sense for AccessKit to expose grafted sub-trees as the target of 'embeds' relationship (or some other relationship we decide) on platforms that support it (AT-SPI and IA2) and fallback to regular parent-child relationship on other platforms?

Does Servo want to move away from this architecture? The AccessKit AT-SPI adapter currently doesn't support neither embedding out-of-process trees nor being embedded.

We don't have plan to move away from at that architecture right now. I am not too familiar with how other browsers build their a11y trees, but based on this document, FF also has a single a11y tree (on the bus, from the screen reader perspective), even though it builds multiple trees (UI and content) i.e it just pretends to 'embed' an independent content process tree, even though the UI process has the full tree and the client communicates only with the UI process.

So exposing 'grafted' sub-trees as 'embeds' seems to make sense to us. What you think?

One thing that @joanmarie highlighted in our discussion is that Orca also expects a single accessibility tree for the browser, so the grafted sub-trees should still not be exposed as independent trees on the bus.

cc @alice

[alice]

Or at least allow an option exclusively when grafting a sub-tree to set that relationship, or any specific relationship we can agree on with the Orca maintainers. I can imagine there are cases where you wouldn't want to set that relationship when grafting trees, but I can't imagine there are cases where you'd want to set it other than when you're setting up a graft.

[DataTriny]

As far as I understand, Orca already have a fallback mechanism to find the document so Servo would be compatible today. Under some circumstances that path might even be faster than the hack because it can hit the cache, which is never the case with Accessible.GetAttributes.

@joanmarie Have you already experimented with calling Collection.GetMatches on browser frames to query for the first object having the "showing" state and any of the document roles? I'd rather help you with that on the Orca side than implement this semantical nonsense in AccessKit.

We currently don't implement this particular interface: it is planned but it's technically tricky and require bigger changes to happen before.

[joanmarie]

@DataTriny I'm sorry to hear you feel this is "semantical nonsense."

I'm afraid I'm not understanding your point regarding Accessible.GetAttributes. Could you please clarify?

As for the Collection interface, that crossed my mind as an alternative, but based on experience in general: we cannot always count on the showing state being removed when something is not showing and added when that changes. So can you guarantee that in, say, a browser with several page tabs that the "showing" state is only exposed on the active page and updated every time it changes?

[joanmarie]

Something else that occurs to me: When asking the question "is this document active?" because an event was fired from it or one of its descendants, Orca will need to ascend the accessibility tree to get the containing window in order to use the Collection interface to re-descend the tree. (Having a relation pair makes it possible to go from one to the other in both directions.)

[DataTriny]

Sorry I meant Accessible.GetRelationSet, which always require at least one IPC roundtrip, whereas the fallback path can probably get most if not all the information from the local cache maintained by libatspi.

[joanmarie]

As for Orca's "fallback mechanism", are you referring to get_top_level_document_for_object() in active_document()?

    def active_document(self) -> Atspi.Accessible | None:
        """Returns the active document."""

        window = focus_manager.get_manager().get_active_window()
        documents = list(filter(self.is_document, AXUtilities.get_embeds(window)))
        documents = list(filter(AXUtilities.is_showing, documents))
        if len(documents) == 1:
            tokens = ["SCRIPT UTILITIES: Active document (via embeds):", documents[0]]
            debug.print_tokens(debug.LEVEL_INFO, tokens, True)
            return documents[0]

        document = self.get_top_level_document_for_object(
            focus_manager.get_manager().get_locus_of_focus(),
        )
        tokens = ["SCRIPT UTILITIES: Active document (via locus of focus):", document]
        return document

If so, it's a tree walk up. If anything triggered AT-SPI to clear the cache on the object or any ancestors, getting the parent will also require at least one IPC roundtrip.

But even if the ancestry is fully cached, this fallback will fail if the current object got destroyed (which happens a surprising amount of time in dynamic web content) and Orca hasn't yet received or processed an accessibility event via which to recover.

It will also fail if the current object is alive and well, but not a descendant of the document, for instance, in the address bar.

[DataTriny]

I have submitted web-platform-tests/wpt#62251 which would unblock Servo together with a new release here. Any help in getting it merged would be appreciated. Thanks.

[mukilan]

Thank you, @DataTriny! I tested your changes to the WPT fixture with Servo patched to use latest accesskit crates. It does seem like this would unblock us. There is one issue, however, that prevents the blockquote.py test from passing - accesskit exposes the document's URL on the Document interface as "uri" (lowercase) whereas the fixture expects "URI" (in uppercase).

That said, we had a discussion with the WPT maintainers a few days back about adding a fallback path for finding documents to AT-SPI fixture. My understanding is that such changes are not recommended as the expectation is for the test runner to match the behavior of screen readers. So I think we'll need buy-in from Orca before changing WPT.

[DataTriny]

accesskit exposes the document's URL on the Document interface as "uri" (lowercase) whereas the fixture expects "URI" (in uppercase).

Fixed in #783.

My understanding is that such changes are not recommended as the expectation is for the test runner to match the behavior of screen readers. So I think we'll need buy-in from Orca before changing WPT.

I have bigger priorities than fixing Orca at the moment. If noone else does it then it will have to wait.

[joanmarie]

As the Orca maintainer, I continue to have some doubts about the proposal to use the Collection interface for this purpose.

To repeat the question I asked the other day:

So can you guarantee that in, say, a browser with several page tabs that the "showing" state is only exposed on the active page and updated every time it changes?

Also it occurs to me that even if one removes the "showing" state from the top-level page, that might not be enough. Let's say there are three page tabs and the second one is the active one. If I use the collections interface on the root window to locate the first showing document, and the document (EDIT: from the first page tab) is in the accessibility tree with the showing state removed, won't the search descend the entire tree of that first page tab? In which case it will be necessary to also ensure the showing state has been removed from the documents in any iframes on that page.

@DataTriny: I realize that you are busy, but if you could carve out some time to answer the above, I would appreciate it.

Having said that:

@mukilan: You need to be unblocked. Don't let my doubts about this stand in the way of your getting your work done.

I suspect that there is a lot more work that will need to be done in Servo before it becomes relevant what I do or don't do/accept in Orca related to locating the active document. I think this can wait.

[joanmarie]

@DataTriny: I just discovered this: https://docs.gtk.org/atspi2/method.Collection.get_active_descendant.html:

Return value
Type: AtspiAccessible
The active descendant of the given object. Not yet implemented.

Were this actually implemented, it strikes me as a potential solution. Thoughts?

[DataTriny]

@joanmarie

So can you guarantee that in, say, a browser with several page tabs that the "showing" state is only exposed on the active page and updated every time it changes?

If by "you" you mean me as AccessKit maintainer, I can guarantee that as long as Servo properly manages the "hidden" property on the web document nodes, the AccessKit Unix adapter will set the "showing" state and send an update signal.

If I use the collections interface on the root window to locate the first showing document, and the document is in the accessibility tree with the showing state removed, won't the search descend the entire tree of that first page tab? In which case it will be necessary to also ensure the showing state has been removed from the documents in any iframes on that page.

I would make consecutive calls to Collection.GetMatchesFrom with currentObject=theFrameOrLastFoundDocument, rule=any_document_role, count=1 until I find a "showing" one. I would use this instead of GetMatches to avoid descending into document trees. But more importantly I would let Orca remember the active document for each window, if any.