Add PoC for markup-guided focus shifting lib

[knuton]

This is a proof of concept implementation with possibly poor performance and also very possibly lots of bugs. Submitted for testing and feedback on the API.

To test, just open examples/everything-bagel.html and use the arrow keys.

Current status

• linear is not really implemented, just enough to work in the current example.
• I used a JS file with JSDoc and external TypeScript checking, because I was curious about that approach and didn't want to spend time setting up a dev server. Not at all married to this setup.

[stoeffel]

Logging some stuff that I find here

Edge case moving sideways

Screen.Recording.2024-01-24.at.08.42.46.mov

I expect the element above to the right to get highlighted (it's further away but feels closer).

Edge case relation to using mouse

I think you'd expect to focus something close to where you clicked last (like in the )

Screen.Recording.2024-01-24.at.08.46.13.mov

[knuton]

Thanks!

I expect the element above to the right to get highlighted (it's further away but feels closer).

I also noticed some unintuitive jumps in such situations. I have an idea of how to address this, by dividing the actual distance to each candidate by the dot product of its direction with the ideal direction. This dot product is 1 if they point in the same direction, and 0 if they are perpendicular. This should give a penalty to elements which are close but don't lie directly in the axis of movement.

I think you'd expect to focus something close to where you clicked last (like in the )

It might not be hard to implement, but maybe we can exclude this from the lib's scope? I assume a no-mouse situation. Nothing should break with a mouse present, but it may not need to be optimized for combined use.

[stoeffel]

I also noticed some unintuitive jumps in such situations. I have an idea of how to address this, by dividing the actual distance to each candidate by the dot product of its direction with the ideal direction. This dot product is 1 if they point in the same direction, and 0 if they are perpendicular. This should give a penalty to elements which are close but don't lie directly in the axis of movement.

I was thinking the same.

It might not be hard to implement, but maybe we can exclude this from the lib's scope? I assume a no-mouse situation. Nothing should break with a mouse present, but it may not need to be optimized for combined use.

I already added very basic support. see https://github.com/knuton/focus-shift/pull/1

one more to not loose it:

Screen.Recording.2024-01-24.at.10.40.01.mov

[knuton]

This last one is tricky, I am not sure what should happen. Would you have a specific expectation?

I almost feel like it shouldn't move right at all here, but then one would have to watch out to not make navigation to certain elements impossible.

Edit: One idea might be to pre-filter candidates by "dominions". For a move to the right, at the moment we consider every element having its left edge to the right of the current element. We could instead consider only those that have their left edge within the rightwards projection of the right wedge of its bounding box. Sketch below. This way we cover the whole space around every element, so one of the four directions should always bring you towards another element.

[stoeffel]

good idea, I also played around with the algorithm to get the nextParent
Looking promising, but there are still some edges.

Screen.Recording.2024-01-24.at.11.49.58.mov

[knuton]

I tried out the dot product idea in 6d4502e, I would say it's a net improvement, but with its own side-effects. Now it jumps to "Normal Button 3" instead of "Normal Button 2". 😄

focus-shift-weighted-distance-2024-01-24_12.18.30.mp4

Are you interested in giving the dominion filtering a try?

[knuton]

Superseded by #2