As a mediaoverlay support reading system I would like mediaoverlay to support range as src of text

[Jeffxz]

Introduction

I would like mediaoverlay to have a normal format to support range so that it will not require publisher to wrap text with html element specifying id attribute to map to audio clip. Or if there is already such a way to define this in spec then I would like to know.

• who will benefit from the use case
  Content creator and reading system

• what is the use case
  So it will be easier to map each word to audio clip without adding HelloWorld

• how I will benefit from the use case
  There are a lot of benefit to have range in text. For example if content creator would like to map audio clip to each word then we don't need to wrap each word one by one. And it will be possible to use tool to generate smil automatically.

Detail

Use this smil as example it matches this text for one word. If we use span like this then it would be very difficult for reading system to highlight each word accordingly.

<span id="c01s0006">With a philosophical flourish Cato throws himself upon his sword; I quietly take to the ship.</span>

I searched around MediaOverlay but can see text src only can apply to IRI then we have a few problems.

1. IRI does not prevent us from defining range like http parameter for example http://example.org/ebook1?path=//table[1]/tr[1]/td[2]&start=412&end=414 but it is not well defined that can be used as contract between content creator and reading system. Also it can only deine very simple data type. (maybe serialize a json string to one parameter? But still need a definition)

2. The serialization format for selector. The Text Position Selector are very close to what I am looking for but it does not have dom range. While the Range Selector does not seem to have character offset which a DOM Range does support.

So I wonder if could find a way to map certain word to audio clip without adding more html element.

[mattgarrish]

This has been partially clarified in the EPUB 3.3 revision, @Jeffxz, although there's no guarantee of support beyond ID referencing.

The definition of the text element's src attribute now reads:

The value MUST be a path-relative-scheme-less-URL string, optionally followed by U+0023 (#) and a URL-fragment string.

https://w3c.github.io/epub-specs/epub33/core/#sec-smil-text-elem

And the reading system spec now says:

When presented with a Media Overlay text element [[!EPUB-33]] whose src attribute references a target element [[HTML]] or SVG Fragment Identifier [[SVG]], Reading Systems SHOULD ensure the referenced element is visible in the Viewport. Reading Systems MAY support other fragment identifier schemes.

https://w3c.github.io/epub-specs/epub33/rs/#sec-rsconf-rendering-text

The question that still needs to be solved, as you've figured out, is how ranges should be identified. Would you use CFIs or something else? That's the part that needs incubation before any stronger support statement could be incorporated.

[Jeffxz]

Thanks @mattgarrish for pointing me to epub33.

Yeah, in a wider web browser world it seems range is more preferable than CFI.

CFI is still difficult to use when a reading system try to navigate a certain dom element defined by CFI. It is not adopted by web browser (which is using range) so it will depend on different reading system to define it's own implementation which might introduce more complexity.

The DOM range would be more compatible with current web browser technology and the benefit for Reading System is we can just use standard web browser range API (which is more standard way cross web browser) to select a random character.

[Jeffxz]

This has been partially clarified in the EPUB 3.3 revision, @Jeffxz, although there's no guarantee of support beyond ID referencing.

The definition of the text element's src attribute now reads:

The value MUST be a path-relative-scheme-less-URL string, optionally followed by U+0023 (#) and a URL-fragment string.

https://w3c.github.io/epub-specs/epub33/core/#sec-smil-text-elem

Nice, it's a good start for reading system. As a reading system vendor I expect don't need to implement something that is apparently different from spec. So as long as epub33 loosed the IRI portion then I can at least implement with spec support that is part of incubation to exame spec with implementation and find "implementation fit" for the section of spec.

[iherman]

@Jeffxz I believe this problem goes beyond media overlays; exactly the same issue arises in the world of text annotation. It is not by coincidence that the references to ranges that you give (e.g., Text Position Selector) comes from a document produced by the (now defunct) Web Annotation Working Group. Unfortunately, implementations of the selection model is lacking by browsers. As EPUB relies on the Web Platform, we are also paying this price for this.

I think the ideal way forward would be to take the selector spec and provide an implementation, as an extension, in a browser (or in webview) that can then be picked up in a reading system. That would also require defining a proper API for this selection. I do not think we can expect the developers of the core browsers to do this. But the model of implementing it in an extension, and then prove its usefulness, e.g., for the publishing community, may work on long term.

[Jeffxz]

thanks @iherman. I read through Web Annotation WG document again but I think as a vender developer when I try to implement I need a mixed approach of using Range (not xpath) to locate element then use Text Position inside the specific node to locate char offset if I understand the spec correctly.

Lemme explore some quick POC in wysebee and show case in next PCG.

[Jeffxz]

@iherman I implemented this POC in wysebee to show case how to highlight a random string in web browser. The implementation difficulty I found for annotation-model is basically I needed a way to define text position under the node (with index) inside element that is specified by Range Selector. In the POC example "startSelector" and "endSelector" are url encoded but the format is <CSS Selector>:<node index>:<charOffset> which I could not find a matching one from web annotation yet. If we could still re-activate web annotation spec and change a bit then we might have a better defined way for annotation even without web browser native support.

[iherman]

@Jeffxz I cannot simply 'reactivate' the spec; it is essentially frozen at its current stage. You can (and should) submit an error message (raising an issue on the group's repository, see also the errata list) and, hopefully, at one point the community will update the spec and/or work on it further. But there is no active group at the moment... (alas!)

[Jeffxz]

ok, I can submit error message. @iherman Which spec I should submit. They both have selector session.

https://www.w3.org/TR/annotation-model/#introduction
https://www.w3.org/TR/selectors-states/#introduction

I think PCG can potentially pick up this as a TaskForce but how to do it properly on W3C track?

[iherman]

On 17 Feb 2022, at 14:14, Zheng (Jeff) Xu ***@***.***> wrote: ok, I can submit error message. @iherman <https://github.com/iherman> Which spec I should submit. They both have selector session. https://www.w3.org/TR/annotation-model/#introduction <https://www.w3.org/TR/annotation-model/#introduction>

This one.

https://www.w3.org/TR/selectors-states/#introduction <https://www.w3.org/TR/selectors-states/#introduction>That is not a normative document and the main technical aspects are taken from the previous one.

Ivan

…

I think PCG can potentially pick up this as a TaskForce but how to do it properly on W3C track? — Reply to this email directly, view it on GitHub <#27 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAD7EEZEY45ILLUFD3CXNRTU3TYEVANCNFSM5OF5MXMQ>. You are receiving this because you were mentioned.

---- Ivan Herman, W3C Home: http://www.w3.org/People/Ivan/ mobile: +33 6 52 46 00 43

[Jeffxz]

Hi @iherman

... You can (and should) submit an error message (raising an issue on the group's repository, see also the errata list) ...

looks like the link of "group repository" is for "csv on web". Is it supposed to be here ?

[iherman]

I am sorry, @Jeffxz, you are right. The pointer you found is the right one.

[Jeffxz]

ok, thanks. Filed here