Management: Mapping tags is broken

[andreoda]

Website
VocaDB

Describe the bug
It occurred to me that for some tags, the system automatically deletes my added mapping when I add another one.

To Reproduce
Steps to reproduce the behavior (for me at least):

1. Go to https://vocadb.net/Admin/ManageTagMappings
2. Add mapping for the tag "弱音ハク" -> "弱音ハク"
3. Click on save and add another one -> "ハクオリジナル曲"
4. Reload the page for the tag "弱音ハク" and the mapping "弱音ハク" disappeared. Same happens vice versa.

The audit log shows me this:

I did not delete any mapping. Only saved.

This problem also happened to me with another tag some time ago (I forgot which) but I just ignored it since I assumed it was some kind of temporary error. Maybe it is my fault, since it is not happening to every tag (I believe). Am I doing something wrong here?

[ycanardeau]

I cannot reproduce this issue. For your information, the log file says:

'andreoda' finished tagging 
'andreoda' updating tag mappings 
'andreoda' updated tag mappings (1 additions, 0 deletions) 
'andreoda' added [弱音ハク], deleted [] 
'andreoda' updating tag mappings 
'andreoda' updated tag mappings (1 additions, 1 deletions) 
'andreoda' added [弱音ハク], deleted [弱音ハク] 
'andreoda' updating tag mappings 
'andreoda' updated tag mappings (1 additions, 1 deletions) 
'andreoda' added [弱音ハク], deleted [弱音ハク] 

[andreoda]

Yes but as I said, I did NOT delete any mappings. The system automatically deleted it after I added another one.

[riipah]

Could be a browser issue, possibly related to caching, can you try another browser? For example, Microsoft browsers IE and Edge are more aggressive about caching.

[andreoda]

Found other entries from the audit log:

I am using Firefox

[andreoda]

Update: Just used Chrome and the same thing is happening.

Update 2: Same with Microsoft Edge

[riipah]

Also, could you test it on the beta site as well?

[riipah]

I believe I found the issue. The mappings page is only loading the first 1000 mappings. Mappings after that are lost after saving. The issue only started when the number of mappings exceeded 1000. Unfortunately there is no workaround, it has to be fixed before more mappings can be added. Good news is that it's really easy to fix. "1000 mappings should be enough for anybody" is probably what I was thinking.

I think at this point the maximum number could be removed, although some kind of server side paging should also be implemented eventually, because at some point there will be problems because of too large requests.

[andreoda]

I think it is broken again, I noticed that my previous mappings are gone.

[ycanardeau]

@andreoda @kateabr Can you please make some experiments on the beta site and paste the request payload?

Request URL: https://vocadb.net/api/tags/mappings
Request Method: PUT

[kateabr]

payload1.txt
payload2.txt

[ycanardeau]

Thank you! Can you please try to add some more tag mappings until some others are deleted (if reproducible) and paste the payload?

[kateabr]

payload3.txt
payload4.txt
payload5.txt

This time I used two different tabs to alter mappings.

Payload 3 (tab 1): deleted ~20 first mappings
Payload 4 (tab 2): added one new mapping
Payload 5 (tab 1): added one mapping from the previously deleted ones

[ycanardeau]

@kateabr Thank you!

Note that this problem could also happen to other lists like external links when two people (or two tabs) are editing the same entry at the same time. As a workaround, make sure that the tag mappings on the "Manage tag mappings" page is latest and that only one user (or one tab) is managing tag mappings.

User 1 (Tab1)	User 2 (Tab 2)
Load the "Manage tag mappings" page.
Fetch the latest tag mappings by making a GET request to /api/tags/mappings.
	Load the "Manage tag mappings" page.
	Fetch the latest tag mappings by making a GET request to /api/tags/mappings.
Add tag mappings, let's say, mapping 1, mapping 2 and mapping 3.
Click [Save changes] and make a PUT request to /api/tags/mappings.
The API controller compares the current lists in the database and the new ones, and determines which tag mappings have been added/removed. In this case, added [mapping 1, mapping 2, mapping 3], deleted [].
	Add tag mappings, let's say, mapping 4, mapping 5 and mapping 6.
	Click [Save changes] and make a PUT request to /api/tags/mappings.
	The API controller compares the current lists in the database and the new ones, and determines which tag mappings have been added/removed. In this case, added [mapping 4, mapping 5, mapping 6], deleted [mapping 1, mapping 2, mapping 3].

[riipah]

The same concurrent editing issue applies to almost all editing pages, but on normal editing pages you should see the "user X is already editing this entry" warning. Potentially the same warning could be added to other pages that can be managed by multiple users, such as this one.

[andreoda]

Additionally, can we have a way to see in the audit log which tag mappings were added and deleted? For now, I have no clue which accidentally got removed by the system.

[riipah]

I think more details are already mentioned in syslog.

[andreoda]

How can I view the syslog?

[riipah]

You should ask yca the details, I can't remember myself what permissions it requires. There's nothing stopping adding the same information to audit log too, except that audit log is intended for short messages only, and the list of modified tag mappings can be too verbose. Some sort of version history would be optimal (same as entry pages), but probably a lot of work to implement.

[ycanardeau]

How can I view the syslog?

You can't. It's stored in a log file on the server.

vocadb/VocaDbModel/Database/Repositories/IAuditLogger.cs

Lines 10 to 15 in 585136f

	/// <summary>
	/// Logs an action in syslog.
	/// Syslog is saved through NLog to a file.
	/// This override uses the currently logged in user, if any.
	/// </summary>
	/// <param name="doingWhat">What the user was doing.</param>

[riipah]

How can I view the syslog?

You can't. It's stored in a log file on the server.

vocadb/VocaDbModel/Database/Repositories/IAuditLogger.cs

Lines 10 to 15 in 585136f

	/// <summary>
	/// Logs an action in syslog.
	/// Syslog is saved through NLog to a file.
	/// This override uses the currently logged in user, if any.
	/// </summary>
	/// <param name="doingWhat">What the user was doing.</param>

Um, that's not true. There is a page for viewing that. I never told you?

[ycanardeau]

Oh, I didn't know that. Then, @andreoda should be able to view that.

vocadb/VocaDbWeb/Controllers/AdminController.cs

Line 316 in 41af697

PermissionContext.VerifyPermission(PermissionToken.ViewAuditLog);

[ycanardeau]

The same concurrent editing issue applies to almost all editing pages, but on normal editing pages you should see the "user X is already editing this entry" warning. Potentially the same warning could be added to other pages that can be managed by multiple users, such as this one.

It would be nice if we could implement something like ConcurrencyStamp, which is used in ASP.NET Core Identity. It's not only so simple, but also powerful enough.