Multiple entries: Offtopic content could be ignored from "highlighted" lists

[riipah]

Applies to offtopic songs and artist entries, meaning entries that are not voice synthesizer related (such as cover unifier).

These entries should always be ignored from "highlighted" lists. That means popular song lists, and top artists/songs tagged with a specific tag.

For example, consider song Bad Apple!!. It's tagged with "digital rock". It should not appear on the page for "digital rock" tag, or it should be shown after all of the "on topic" songs. The same applies to the vocalist artist. The song can still be shown on search page, but we could consider dimming the title or show it somehow de-emphasized.

We need a consistent way of identifying offtopic songs and artists. Can this be done with special tags, or do we need a new option?

Maybe there could even be a label on the entry page, saying that it's offtopic.

Validation warning could be shown on song page if offtopic song is added without derivative voice synthesizer versions. For example, if user adds another song for ZUN, but that song does not have Vocaloid versions, a validation warning is shown. This can be an additional issue.

[Shiroizu]

With the current rule set, the off topic content should match the songs listed under the out of scope (cover unifier) -tag.

Off topic content should also be hidden from the stat graphs, as requested in #1349

[Shiroizu]

"Out-of-scope" (=OOS) -field (boolean) could be added to Artist & Song -entries:

• All custom tracks in albums could be replaced with proper song entries.
• Original versions could be added to all of the songs under the original out of scope -tag.
  • OOS-artist entries could either A) be added with minimal info (name + wikipedia page) or B) left as entryless custom artists.

This field could affect the search, stats, entry functions (rating/following/commenting), etc.

Identifying OOS entries:

OOS Songs:
    - Field not editable directly for song entries:
    - All song types:
        - OOS or in-scope dynamically based on voice synth presense.

OOS Song sources:
  - A) album tracks with no voice synths (song type = original or instrumental)
  - B) cover unifiers (and all the new entries for original versions)
  

OOS Artist: 
    - Vocalists:
        - Vocaloids etc.:
            - in-scope
        - Other voice synths:
            - Approach 1) Based on engine: 
                - OOS = other vocal synthesis engine (T/6109), etc.
                - in-scope = AISingers (T/7434), etc.
            - Approach 2) Always in-scope
        - Other vocalist: (human singers)
            - Approach 1) in-scope if featured in a same song with a voice synth.
            - Approach 2) in-scope if featured in a same song or album with a voice synth
            - Approach 3) always OOS
    - Producer:
        - Music producer: A/S
        - Cover artist: A/S
        - Animation producer: A/S
        - Illustrator: A/S
    - Group:
        - Circle:
            - Approach 1) > X in-scope members
            - Approach 2) > %X in-scope members
            - Approach 3) A/S
        - Label: A/S
        - Other group: Same as with Circle.
    - Other:
        - Lyricist: A/S
        - Instrumentalist: A/S
        - Other individual: A/S

A/S = OOS if only OOS albums/songs, in-scope if at least one in-scope album/song.

[saturclay]

When searching, could there also be an "only OOS" option alongside the "exclude OOS" option? Having that ability would be useful for auditing data (though if others are against it, I'm assuming that's something I could use the API for if I ever felt the need).

[CatgirlFrostmoon]

Ah, I see we're clarifying what counts as in-scope / out-of-scope. Fair enough; clarifications are always good. I have a few personal preferences of my own:

-I prefer for any sort of vocal synth to count as in-scope (with exceptions to jinriki, because they're a nightmare and a half to properly track and can cause serious confusion). Creating boundary lines between in-scope and out-of-scope banks can very quickly become a nightmare to keep track of. Where would we draw the lines? No talk synths? That would remove our collection of spoken word music, which is closely related enough to the general vocal synth community that it has a VSynth specific name: poemloid! We could go for only banks with humanoid icons, but that would kill a good chunk of Synthesizer-V's voicebanks, let alone so many other programs. We could draw the line at only accepting synths that speak and / or sing, but that would cut off Acme Iku (and others), who's been a huge part of the community for forever. We could draw a line between what feels like it belongs within the general Vocaloid community, but that's subjective and will likely spring arguments. In other words, I really don't think that limiting what V-Synths we accept is a wise idea.

-I feel like human singers should count as in-scope if they're at least in a song with a synth. I mean, look, they could also be producers or tuners (such as Ado), so marking them all as custom artists feels odd. The songs also contain a Vsynth, and thus shouldn't be removed from search.

-Albums with human singers are more complicated. Due to albums, we have a lot of otherwise out-of-scope songs on the database. Whether they should actually be here or instead made into custom tracks is a fair question, especially when we start getting into massive collaboration / compilation albums. The latter probably shouldn't have most non-definitely-in-scope artists attached (which is what we're currently following), but smaller albums by in-scope artists is a different question completely. Removing their general tag impact might be a good compromise, but I'm not sure.

Everything else is something I don't have an immediate opinion on, at least.

[riipah]

All custom tracks in albums could be replaced with proper song entries.

In general I think this is a good idea, but I'm slightly concerned about search performance loss if the number of song entries explodes. That can be mitigated by a search index or whatever else technical improvements have been planned.

[FinnRG]

I would also like to point that the "original out of scope" tag is currently used on ~21000 songs. @Shiroizu If I understand your proposal correctly, we would add proper original song entries for all of those songs. This doesn't seem feasible (it would take 30 days of non stop editing if you need 2 minutes for each entry).

I think a more realistic approach would be to add the new OOS field and use it for all of the songs under the "out of scope (cover unifier)" tag. This could also be done using a script.

[Shiroizu]

@CatgirlFrostmoon Thanks for the feedback. I agree that the approach 2 makes more sense for "other voice synths".

For "Other vocalist", it seems like approach 1 is the best choice. I want to reiterate that a song is in-scope if it has voice synths presents, no matter how many other human vocalists it has.

---

@saturclay The main search (https://vocadb.net/Search?filter=) would hide OOS content by default (with a toggleable search filter), but the search boxes inside entry edit pages would obviously have to include these.

---

The only remaining question in my list is the case of groups. Having a complete list is a hard pre-requisite for any feature changes in my opinion. These requirements could also of course be changed later if needed.

The other important detail is related to how the OOS-field operates in practice: For songs it makes sense to keep it automatic and read-only (if technically possible), since the presence or lack of a voice synth makes the question trivial.

If the artist scope can be automatically and efficiently determined based on the list above, maybe the field could also be automatic for artist entries as well.

If the OOS-field is manual-only for technical reasons, toggling it off by default (entry = in-scope) for new entries would make the most sense (along with relevant edit warnings).

Album entries require at least one voice synth present per our album content policy, which makes all of our album entries automatically in-scope.

---

@riipah - Are you referring specifically to the search performance or the searchability? By my rough estimate, the number of custom tracks to replace shouldn't be too large compared to the amount of existing song entries:

24000 albums * 10% albums have custom tracks * 5 custom tracks = 12 000 new OOS entries

These new entries would only include the artist entry if they are already in the database (in-scope artist), which matches our existing policy with cover unifiers.

With the current song count of ~500 000, the amount of new entries, even including all of the original-out-of-scope songs (12k + 21k), would only be a ~6 % increase.

---

@FinnRG - I don't think that a large number of entries to "fix" is a bad thing, if the overall database benefits from it. It would be a longer project that could be slowly done over the months by multiple editors.

Maybe a more productive approach is to only create the OOS song entry after it already has some other cover entry in the database (currently the requirement is 5).

[saturclay]

For TouhouDB at least, I think the OOS field would have to be manual by necessity, at least for songs, since there's not a single property that would clearly indicate whether something is OOS or not. I do already have the tags other originals and other arrangements, which could be replaced by the OOS field.

Original songs are OOS if all of the following apply:
- not by ZUN
- not Touhou-style
- not a character song for a Touhou character/fan character
- not from the soundtrack of a Touhou fangame

Arrangements are OOS if all of the following apply:
- not an arrangement of an in-scope song
- not Touhou-style
- not from the soundtrack of a Touhou fangame

Generally, for artists and albums, the A/S approach described above would work... or at least it would if the database were more filled-out. Currently, there are many artists who don't have any songs or albums, but who are definitely in-scope. It's a lot easier for me to keep track of which artists I need to add content for if they're in the database, although I know that VocaDB discourages adding artists without content. So I think that it would make the most sense for the OOS field for artists to be manual as well on TouhouDB, at least until the db is more comprehensive, which won't happen anytime soon. I do also have some groups that have never released Touhou-related songs or albums, but who I've added to the db because many or all of their members are in-scope, and those could be marked as OOS.

[andreoda]

Adding that off-topic content (like cover unifier songs) should also be ignored in the "Highlighted PVs" section.