Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1x1 black picture causes problems in m4b again - ability to disable it (for specific formats)? #305

Closed
sandreas opened this issue Jan 23, 2025 · 10 comments
Assignees

Comments

@sandreas
Copy link
Contributor

sandreas commented Jan 23, 2025

Recently I got an issue in m4b-tool(one of my other projects), which is using atldotnet internally to tag files:

sandreas/m4b-tool#269

Seems that tone is still suffering the 1x1 black chapter image issue somehow, although I already updated to the fixed atldotnet version 6.11 (see #296).

Would it be possible to remove the 1x1px workaround completely or at least opt out this feature via Settings here?

            int nbActualChapterImages = chapters.Count(ch => ch.Picture != null && ch.Picture.PictureData.Length > 0);
            foreach (var chapter in chapters)
            {
                if (chapter.Picture != null) w.Write(chapter.Picture.PictureData);
                else if (nbActualChapterImages > 0 && Settings.UseBlackChapterImage) w.Write(Properties.Resources._1px_black);
            }

Seems that most players have problems with this, although it might technically be correct.

@Zeugma440 Zeugma440 self-assigned this Jan 23, 2025
@Zeugma440
Copy link
Owner

Seems that tone is still suffering the 1x1 black chapter image issue somehow

If I may, you doesn't seem to be so sure about that, and the log file your user submitted says cover : embedded jpeg, 1x1, which looks super weird as :

  • Your user says he doesn't have chapter pictures
  • ATL only generates 1x1 pics for chapters, definitely not for the cover

Creating an opt-out would certainly "fix" the problem by ignoring it, but I'm not sure I want to do that unless we confirm the feature working as intended is the actual source of the problem. Is there a way to get the file described in your issue or to reproduce the problem?

@sandreas
Copy link
Contributor Author

Creating an opt-out would certainly "fix" the problem by ignoring it, but I'm not sure I want to do that unless we confirm the feature working as intended is the actual source of the problem. Is there a way to get the file described in your issue or to reproduce the problem?

You're right. I try to reproduce the problem with atldotnet and a sample file and provide some code for you. Thanks for the quick response.

@sandreas
Copy link
Contributor Author

Ok, here is the code - as always it copies the original file to prevent overwriting the original metadata.

I'm specifically interested in why the mediainfo shows the 1x1 jpg. Since there is no cover present anywhere, I don't know why the 1x1 cover is created :-) In my opinion the ONLY reason to create an 1x1px chapter image is that there is more than one cover specified, at least one chapter AND there is a GAP > 1ms (not >= 1ms) between the timestamps of the chapters.

My Version is atldotnet 6.1.3.

        var path = "";
        var sourceFile = Path.Combine(path, "sample.m4b");
        var destinationFile = Path.Combine(path, "sample_copy.m4b");
        if (File.Exists(destinationFile))
        {
            File.Delete(destinationFile);
        }
        File.Copy(sourceFile, destinationFile);

        var track = new Track(destinationFile);
        track.Album = "TestAlbum";
        track.Chapters = new List<ChapterInfo>
        {
            new(){StartTime = 0,EndTime = 2000,Title = "1"},
            new(){StartTime = 2000,EndTime = 4000,Title = "2"},
            new(){StartTime = 4000,EndTime = 60000,Title = "3"},
        };
        await track.SaveAsync();

sample.zip

mediainfo sample_copy.m4b
General
Complete name                            : sample_copy.m4b
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/mp41)
File size                                : 24.1 KiB
Duration                                 : 1 min 0 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 3 293 b/s
Frame rate                               : 0.500 FPS
Movie name                               : sample_copy
Album                                    : TestAlbum
Writing application                      : Lavf61.7.100

Video
ID                                       : 3
Format                                   : JPEG
Codec ID                                 : jpeg
Duration                                 : 1 min 0 s
Duration_LastFrame                       : 54 s 0 ms
Bit rate mode                            : Constant
Width                                    : 1 pixel
Height                                   : 1 pixel
Display aspect ratio                     : 1.000
Frame rate mode                          : Constant
Frame rate                               : 0.500 FPS
Compression mode                         : Lossy
Title                                    : Chapter pictures
Encoded date                             : 2025-01-25 13:14:32 UTC
Tagged date                              : 2025-01-25 13:14:32 UTC

Audio
ID                                       : 1
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 1 min 0 s
Source duration                          : 1 min 0 s
Source_Duration_LastFrame                : -11 ms
Bit rate mode                            : Variable
Bit rate                                 : 1 525 b/s
Maximum bit rate                         : 69.0 kb/s
Channel(s)                               : 1 channel
Channel layout                           : M
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 11.2 KiB (46%)
Source stream size                       : 11.2 KiB (46%)
Default                                  : Yes
Alternate group                          : 1
Menus                                    : 2

Menu #1
ID                                       : 2
Format                                   : Timed Text
Codec ID                                 : text
Duration                                 : 1 min 0 s
Title                                    : Chapter titles
Encoded date                             : 2025-01-25 13:14:32 UTC
Tagged date                              : 2025-01-25 13:14:32 UTC
Menu For                                 : 1
Duration_LastFrame                       : 54000
00:00:00.000                             : 1
00:00:02.000                             : 2
00:00:04.000                             : 3

Menu #2
00:00:00.000                             : 1
00:00:02.000                             : 2
00:00:04.000                             : 3

@Zeugma440
Copy link
Owner

Thanks a bunch for taking the time to send the details ❤

I'm specifically interested in why the mediainfo shows the 1x1 jpg. Since there is no cover present anywhere, I don't know why the 1x1 cover is created :-)

What happened is the following : when fixing #296, I did prevent ATL from inserting 1x1 px chapter pictures when there are no chapter pictures in the original file, but I completely forgot to turn off the creation of the video track that was aimed at containing these chapter pictures. What mediainfo sees here isn't a cover but that empty video track that has the 1x1 default dimensions.

In my opinion the ONLY reason to create an 1x1px chapter image is that there is more than one cover specified, at least one chapter AND there is a GAP > 1ms (not >= 1ms) between the timestamps of the chapters.

I eventually rolled my sleeves up and wrote a bona fide unit test to make sure everything works as planned :

// Test behaviour when creating empty chapter pics

Here are the rules I put together:

// Test behaviour when creating empty chapter pics
// Cases
// A: No chapter track should be created if there's no chapter
// B: No chapter picture (video) track should be created if none of the chapters have an attached picture
// C: No empty chapter picture should be generated if all chapters that contain a picture come one after the other
// D: An empty chapter picture should be generated for any chapter without pictures that's included between chapters that have one

Last thing that needs to be discussed is that timestamp gap thingy

AND there is a GAP > 1ms (not >= 1ms) between the timestamps of the chapters

I understand what your idea is, but the M4A/MP4 format doesn't even make it possible to have two chapters that are not contiguous, as the notion of "starting time" doesn't exist on Quicktime Chapters - everything is assumed to start at timestamp 0:00 and every chapter entry only has a duration.

Could you explain a little more the case you had in mind when you wrote that? BY any chance, are you referring to ID3v2 chapters that do have a starting time?

@Zeugma440 Zeugma440 added the bug label Jan 25, 2025
@sandreas
Copy link
Contributor Author

sandreas commented Jan 25, 2025

Wow, that was quick, thank you very much.

I completely forgot to turn off the creation of the video track that was aimed at containing these chapter pictures

Ah, that makes sense.

I eventually rolled my sleeves up and wrote a bona fide unit test to make sure everything works as planned

Very good. That also will prevent regression issues.

Here are the rules I put together:

Looks very reasonable, great work.

I understand what your idea is, but the M4A/MP4 format doesn't even make it possible to have two chapters that are not contiguous, as the notion of "starting time" doesn't exist on Quicktime Chapters - everything is assumed to start at timestamp 0:00 and every chapter entry only has a duration.

I think I just mixed up ID3 and MP4 here. Now let's see if I got it right now:

  • ID3 can have a Picture per Chapter, but also chapters with no picture and parts of the track not having any chapter information
  • MP4 prevents the "here is a part with no chapter" situation by just giving durations and assuming that chapters follow up each other without any time in between

The problem with MP4 is, that the first chapter has a picture, followed by chapter 2 with no picture, the picture of the first chapter will stay visible as long as you don't override it by a 1x1px black image in chapter 2. This problem won't occur in ID3 due to the nature of the standard - which means that my comment

AND there is a GAP > 1ms (not >= 1ms) between the timestamps of the chapters

doesn't make any sense other than you can define Chapters in atldotnet like this:

var track = new Track("sample.m4b");
track.Album = "TestAlbum";
track.Chapters = new List<ChapterInfo>
{
    new(){StartTime = 0,EndTime = 2000,Title = "1"},
    new(){StartTime = 2500,EndTime = 4000,Title = "2"}, // 500ms gap    
    new(){StartTime = 5000,EndTime = 60000,Title = "3"}, // 1s gap
};
await track.SaveAsync();

I don't know what happens in this case internally (MP4 should not allow that) and I don't know what would happen, if you add a picture to chapter 1 and chapter 3 but NO picture to chapter 2. However, you can just ignore the comment about the gap, I don't think it makes much sense.

Thank you.

@Zeugma440
Copy link
Owner

I think I just mixed up ID3 and MP4 here. Now let's see if I got it right now:

ID3 can have a Picture per Chapter, but also chapters with no picture and parts of the track not having any chapter information

Correct

MP4 prevents the "here is a part with no chapter" situation by just giving durations and assuming that chapters follow up each other without any time in between

Correct

The problem with MP4 is, that the first chapter has a picture, followed by chapter 2 with no picture, the picture of the first chapter will stay visible as long as you don't override it by a 1x1px black image in chapter 2. This problem won't occur in ID3 due to the nature of the standard

Almost correct - in the case you describe, the picture for chapter 1 will actually disappear when chapter 2 starts. However, if you add a chapter 3 with a picture, that one will immediately be displayed after chapter 1's picture (i.e. during chapter 2).

doesn't make any sense other than you can define Chapters in atldotnet like this [...] I don't know what happens in this case internally (MP4 should not allow that)

Thing is
1/ ATL should have a unified interface that covers all tagging systems
2/ Start time / end time is much more intuitive to people than bare durations

What happens under the hood is, when writing chapters to an M4A/MP4 file, ATL sets for each chapter "duration = endTime - startTime", ignoring gaps entirely.

I have a question for you, as you've worked on many audiobook files and with many audiobook users : are there actual real-life files that feature actual gaps between chapters? If so, what's the reason for those?

I don't know what would happen, if you add a picture to chapter 1 and chapter 3 but NO chapter to chapter 2.

It automatically inserts an empty 1x1 px image on chapter 2 😄

@sandreas
Copy link
Contributor Author

sandreas commented Jan 25, 2025

Thing is
1/ ATL should have a unified interface that covers all tagging systems
2/ Start time / end time is much more intuitive to people than bare durations

Absolutely, this was not meant to question the actual approach, I really worded that in a misleading way... what I really meant was, that the atldotnet generic approach does not fit perfectly to the mp4 use case and needs some lets say mapping or decision how to handle things - which is totally fine. So everything is okay with the generic approach you took. One could argument that StartTime and Duration (instead of EndTime) would result in smaller Numbers and thus possibly smaller types (and smaller footprint for API JSON Transfers) - or you could use the TimeSpan type, but I think the current approach is totally fine (it never failed me in terms of mappability and backwards compatibility is awesome ;-)).

I have a question for you, as you've worked on many audiobook files and with many audiobook users : are there actual real-life files that feature actual gaps between chapters? If so, what's the reason for those?

No, not at all. I never faced any real use case having gaps between chapters. I could think of something like commercials in recorded live streams or author comments that could interrupt the regular chapter stream but usually this is just represented by another chapter with a different name scheme.

Since metadata chapters are also available for video and subtitle content, maybe the chapter gap thing could find a use case there - in my opinion it's not required anywhere.

Some interesting things I came across in my years of audio book tooling:

  • If you set a zero to very short chapter length (mostly <= 2 seconds), some players like iPods cannot go the previous track any more (e.g. 00:00-00:50, then 00:50-00:51, then 00:51-03:56 - as soon as you are past chapter 2, you can't go back to chapter 1) - a workaround is just to rewind
  • In some audiobooks there is a concept of parts one level above chapters, e.g.
    Part 1 - India
      Chapter 1 - Arrival
      ...
    Part 2 - Africa
      Chapter 1 - Leaving India heading to Africa
      ...
    
    which also results in "mapping" problems - especially when you regard the "very short chapters problem" above. One solution I thought of was to create a chapter Part 1 - India and calculate a shift for the Chapter 1 - Arrival as 3 seconds after, but what if this chapter is also less than 3 seconds? So usually I just "merge" the chapter names as <part>: <first-part-chapter-name>, in this case Part 1 - India: Chapter 1 - Arrival. This problem is hard to solve in theory (because all chapters could be < 3s, but in practice you just jiggle around the times until it fits.

I think these kind of problems often cannot be solved very elegant - it's just a limitation of how the spec is designed, and sometimes limitations are good, because they make things less complex.

For now I think your approach to fix the 1x1 is a good fit and the gap thing can be ignored as well as the other edge cases I listed :-)

@Zeugma440
Copy link
Owner

Fix is available on today's v6.14. Please close the issue if it works on your side~

PS : Have you ever tried working with Matroska chapters? The possibilities are crazy...

@sandreas
Copy link
Contributor Author

Fix is available on today's v6.14. Please close the issue if it works on your side~

Looks good. I'm going to integrate this in the next tone release and ask for feedback from the original issue reporter. Thank you very much for fixing it that quickly.

PS : Have you ever tried working with Matroska chapters? The possibilities are crazy...

No I've never actively worked with MKV / Matroska, but I heard that it is complex. The linked "summary" of chapters sounds very over-engineered, but as in mp4 Matroska also supports Video and in the old DVD Menu days, this might have been a good way to integrate chapters supporting Menu selection. I don't know... at first I thought Menus are a good thing, but later on I mostly hated to wait 1 Minute to start watching the movie :-)

I wish I had more time to work on stuff like this, but a few days ago my NVMe drive died and I had to restore my notebook from backups - noticed that I always wanted to try NixOS and down another rabbit hole... learned a lot but switching over to arch now - we'll see.

@Zeugma440
Copy link
Owner

Thank you very much for fixing it that quickly.

You're very welcome. I should've detected it in the first place.

The linked "summary" of chapters sounds very over-engineered

You just made me smile. "Over-engineered" is the exact word that first came to mind when reading the Matroska specs. I'm relieved I'm not the onlyone to perceive that 😄

a few days ago my NVMe drive died and I had to restore my notebook from backups - noticed that I always wanted to try NixOS and down another rabbit hole

Good luck with that 😉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants