Remove invalid user EIDs and UIDs from bid request #3891

Pubmatic-Supriya-Patil · 2024-08-28T06:08:01Z

This PR solves issue #3859

Pubmatic-Supriya-Patil · 2024-09-04T03:56:48Z

Hi @bsardo @bretg
Please review this PR and let me know if any changes are required.

SyntaxNode · 2024-11-01T03:26:26Z

errortypes/code.go

@@ -35,6 +35,8 @@ const (
 	InvalidBidResponseDSAWarningCode
 	SecCookieDeprecationLenWarningCode
 	SecBrowsingTopicsWarningCode
+	InvalidUserEIDsWarningCode
+	InvalidUserUIDsWarningCode


Do these new warning codes match with PBS-Java? We try to keep in sync.

Can you please help me to get those error codes. Do we have any document or PR to check new warning codes match with PBS-Java.

As per prebid-server-java/pull/3465, I understand that the same 999 warning code is being used when removing eids arrays. In the Prebid Server Go repository, 999 corresponds to UnknownErrorCode in the error codes reference. Are we intending to use UnknownErrorCode here as well?

In this PR, I introduced InvalidUserEIDsWarningCode (10013) and InvalidUserUIDsWarningCode (10014). If we are okay with a generic error code, we could use UnknownErrorCode (999). However, please confirm whether we want separate error codes for InvalidUserEIDs and InvalidUserUIDs, or if using 999 would suffice.

Kindly provide your input on this.

SyntaxNode · 2024-11-01T03:27:01Z

endpoints/openrtb2/sample-requests/invalid-whole/user-ext-eids-uids-id-empty.json

@@ -1,42 +0,0 @@
-{


Instead of removing these tests cases, please modify them to prove the new behavior.

SyntaxNode · 2024-11-01T03:28:31Z

endpoints/openrtb2/auction.go

@@ -1296,8 +1296,14 @@ func (deps *endpointDeps) validateUser(req *openrtb_ext.RequestWrapper, aliases
 	// Check Universal User ID
 	eids := userExt.GetEid()
 	if eids != nil {
-		eidsValue := *eids
-		for eidIndex, eid := range eidsValue {
+


Nitpick; Please remove the leading empty line.

SyntaxNode · 2024-11-01T03:30:17Z

endpoints/openrtb2/auction.go

+
+		if len(eidErrors) > 0 {
+			errL = append(errL, eidErrors...)
+		}


Likely fine to keep this logic here for now. We want to separate validation from fixing in the future. We can refactor this new logic along with the rest later. Thoughts @bsardo?

SyntaxNode · 2024-11-01T03:32:18Z

endpoints/openrtb2/auction.go

+			validEIDs = append(validEIDs, eid)
+		} else {
+			errorsList = append(errorsList, &errortypes.Warning{
+				Message:     fmt.Sprintf("Removed EID with empty UIDs (source: %s)", eid.Source),


Please keep with the established error message format. Perhaps something like:

request.user.ext.eids[0] (source: %s) contains only empty uids and is removed from the request

SyntaxNode · 2024-11-01T03:37:56Z

endpoints/openrtb2/auction_test.go

+		expectedErrorMessages []string
+	}{
+		{
+			name: "Valid EID with non-empty UID",


Go test names cannot contain spaces so they are replaced with a dash character. This makes it hard to locate the failing test based on its name. Recommend using simpler test names like:

one-eid-one-uid-valid

one-eid-one-uid-empty

many-mixed

one-eid-many-uid-empty

The parent test name is within scope so it's easy to locate in source. Please apply this guidance to other tests added in this PR.

SyntaxNode · 2024-11-01T03:41:27Z

endpoints/openrtb2/auction_test.go

+		t.Run(tc.name, func(t *testing.T) {
+			validEIDs, errorsList := validateEIDs(tc.input)
+
+			if len(validEIDs) != len(tc.expected) {


Please use testify assert statements where possible. This can be written as follows which will compare the contents instead of just the array lengths, yielding more concrete tests.

assert.ElementsMatch(e, tc.expected, validEIDs)

SyntaxNode · 2024-11-01T03:42:36Z

endpoints/openrtb2/auction_test.go

+			expectedValidUIDs: nil,
+			expectedErrors:    nil,
+		},
+	}


Please add a test case for a nil input.

bsardo · 2024-11-04T16:40:16Z

Hi @Pubmatic-Supriya-Patil, can you please merge with master (no rebase) and resolve conflicts? Thanks!

SyntaxNode · 2024-11-05T16:30:23Z

Thank you for contributing this feature Please address the comments and we will re-review.

…ver into Fix_3859

Pubmatic-Supriya-Patil · 2024-12-02T09:22:23Z

Hi @Pubmatic-Supriya-Patil, can you please merge with master (no rebase) and resolve conflicts? Thanks!

HI @bsardo I have updated branch and addressed review comments.

VeronikaSolovei9 · 2024-12-10T04:07:20Z

endpoints/openrtb2/auction.go

+			uidErrors = append(uidErrors, &errortypes.Warning{
+				Message:     fmt.Sprintf("request.user.eids.uids[%d] contains empty ids and is removed from the request", uidIndex),
+				WarningCode: errortypes.InvalidUserUIDsWarningCode,


I used the example of user.EIDs from the issue description:

"eids": [ { "source": "uidapi.com", "uids": [ {"id": ""}, {"id": "AgAACYa0cG2jcVcM42xG"} ] }, { "source": "liveramp.com", "uids": [ {"id": "AgAACYa0cG2jcVcM42xG"} ] }, { "source": "yahoo.com", "uids": [ {"id": ""}, {"id": ""} ] } ]

As a result I received a list of errors that looked like this:

request.user.eids.uids[0] contains empty ids and is removed from the request request.user.eids.uids[0] contains empty ids and is removed from the request request.user.eids.uids[1] contains empty ids and is removed from the request request.user.eids[2] (source: yahoo.com) contains only empty uids and is removed from the request

In order to avoid confusion and duplicates may you please pass eidIndex to validateUIDs:
Lines 1311 and 1312:

for eidIndex, eid := range eids { validUIDs, uidErrors := validateUIDs(eid.UIDs, eidIndex)

and then add eidIndex to error message like this:
Line 1338:

Message: fmt.Sprintf("request.user.eids[%d].uids[%d] contains empty ids and is removed from the request", eidIndex, uidIndex),

This will give error list with no duplicate error messages:

request.user.eids[0].uids[0] contains empty ids and is removed from the request request.user.eids[2].uids[0] contains empty ids and is removed from the request request.user.eids[2].uids[1] contains empty ids and is removed from the request request.user.eids[2] (source: yahoo.com) contains only empty uids and is removed from the request

VeronikaSolovei9 · 2024-12-10T04:32:25Z

endpoints/openrtb2/auction.go

+			for uidIndex, uid := range eid.UIDs {
+				if uid.ID == "" {
+					return append(errL, fmt.Errorf("request.user.eids[%d].uids[%d] missing required field: \"id\"", eidIndex, uidIndex))
+				}


At this time we will not have empty eid.ID, correct? there is a check for this already in line 1334.

Also do we need to set "clean" EIDs back to req.User.EIDs?
Something like req.User.EIDs = validEids after the for loop?

Please correct me if I'm wrong. We don't want to fail the entire request if there is at least one invalid EID, correct? Or do we want to drop invalid ones and continue the execution?
If we want to continue the execution - please add this fix and json tests where we validate "clean" user.EIDs and error messages.

Can we remove if len(eid.UIDs) == 0 { validation also from validateUser function. This is also not possible because validateEids wil return error for emtpy uids. Please provide your suggestions @VeronikaSolovei9

…ver into Fix_3859

ShriprasadM · 2025-01-13T15:24:31Z

@Pubmatic-Supriya-Patil : Please respond to this comments.

Remove invalid user EIDs and UIDs from bid request

0f955e9

Pubmatic-Supriya-Patil mentioned this pull request Aug 28, 2024

Drop the User EID object if the all the ID values are empty #3859

Open

bsardo self-assigned this Oct 21, 2024

SyntaxNode reviewed Nov 1, 2024

View reviewed changes

Merge branch 'master' of github.com:Pubmatic-Supriya-Patil/prebid-ser…

9352604

…ver into Fix_3859

bsardo assigned VeronikaSolovei9 Dec 4, 2024

VeronikaSolovei9 reviewed Dec 10, 2024

View reviewed changes

Merge branch 'master' of github.com:Pubmatic-Supriya-Patil/prebid-ser…

541cbed

…ver into Fix_3859

Removed redundant check for uid.id validation

22e6b21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove invalid user EIDs and UIDs from bid request #3891

Remove invalid user EIDs and UIDs from bid request #3891

Pubmatic-Supriya-Patil commented Aug 28, 2024

Pubmatic-Supriya-Patil commented Sep 4, 2024

SyntaxNode Nov 1, 2024

Pubmatic-Supriya-Patil Dec 2, 2024

Pubmatic-Supriya-Patil Dec 2, 2024

SyntaxNode Nov 1, 2024

SyntaxNode Nov 1, 2024

SyntaxNode Nov 1, 2024

SyntaxNode Nov 1, 2024

SyntaxNode Nov 1, 2024

SyntaxNode Nov 1, 2024

SyntaxNode Nov 1, 2024

bsardo commented Nov 4, 2024

SyntaxNode commented Nov 5, 2024

Pubmatic-Supriya-Patil commented Dec 2, 2024

VeronikaSolovei9 Dec 10, 2024

VeronikaSolovei9 Dec 10, 2024 •

edited

Loading

VeronikaSolovei9 Dec 10, 2024 •

edited

Loading

Pubmatic-Supriya-Patil Jan 14, 2025

ShriprasadM commented Jan 13, 2025

Remove invalid user EIDs and UIDs from bid request #3891

Are you sure you want to change the base?

Remove invalid user EIDs and UIDs from bid request #3891

Conversation

Pubmatic-Supriya-Patil commented Aug 28, 2024

Pubmatic-Supriya-Patil commented Sep 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bsardo commented Nov 4, 2024

SyntaxNode commented Nov 5, 2024

Pubmatic-Supriya-Patil commented Dec 2, 2024

Choose a reason for hiding this comment

VeronikaSolovei9 Dec 10, 2024 • edited Loading

Choose a reason for hiding this comment

VeronikaSolovei9 Dec 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ShriprasadM commented Jan 13, 2025

VeronikaSolovei9 Dec 10, 2024 •

edited

Loading

VeronikaSolovei9 Dec 10, 2024 •

edited

Loading