96247 add organization address validation to trexler file process job

[opticbob]

Note: Delete the description statements, complete each step. None are optional, but can be justified as to why they cannot be completed as written. Provide known gaps to testing that may raise the risk of merging to production.

Summary

• This work is behind a feature toggle (flipper): NO The flipper controlling access to this was enabled to a small number of users this morning.
• (Summarize the changes that have been made to the platform): A new job has been added that updates the addresses and locations of veteran service organizations based on a spreadsheet downloaded from the va.gov-team-sensitive repo.
• (If bug, how to reproduce)
• (What is the solution, why is this the solution?): We need users to be able to search for orgs and they need locations in the db to do that.
• (Which team do you work for, does your team own the maintenance of this component?): FAR/ARM, we will own the maintenance.
• (If introducing a flipper, what is the success criteria being targeted?)

Related issue(s)

• Add Organization address validation to Trexler File process job va.gov-team#96247
• Link to previous change of the code/bug (if applicable)
• Link to epic if not included in ticket

Testing done

• New code is covered by unit tests
• Describe what the old behavior was prior to the change: We didn't process the org page in the downloaded spreadsheet at all.
• Describe the steps required to verify your changes are working as expected. Exclusively stating 'Specs run' is NOT acceptable as appropriate testing: Populate your db with orgs in the spreadsheet, then enqueue the Organizations::QueueUpdates job. It will reach out, get the spreadsheet, and process the data.
• If this work is behind a flipper:
  • Tests need to be written for both the flipper on and flipper off scenarios. Docs.
  • What is the testing plan for rolling out the feature?

Screenshots

Note: Optional

What areas of the site does it impact?

(Describe what parts of the site are impacted andifcode touched other areas): Just the representation_management module.

Acceptance criteria

• I added unit tests and integration tests for each feature (if applicable).
• No error nor warning in the console.
• Events are being sent to the appropriate logging solution
• Documentation has been updated (link to documentation): Ruby docs in PR.
• No sensitive information (i.e. PII/credentials/internal URLs/etc.) is captured in logging, hardcoded, or specs
• Feature/bug has a monitor built into Datadog (if applicable)
• If app impacted requires authentication, did you login to a local build and verify all authenticated routes work as expected
• I added a screenshot of the developed feature

Requested Feedback

(OPTIONAL)What should the reviewers know in addition to the above. Is there anything specific you wish the reviewer to assist with. Do you have any concerns with this PR, why?

[github-actions]

1 Error

🚫

This PR changes 1298 LoC (not counting whitespace/newlines). In order to ensure each PR receives the proper attention it deserves, those exceeding 500 will not be reviewed, nor will they be allowed to merge. Please break this PR up into smaller ones. If you have reason to believe that this PR should be granted an exception, please see the Submitting pull requests for approval - FAQ. File Summary Files lib/periodic_jobs.rb (+1/-0) modules/veteran/app/models/veteran/service/organization.rb (+12/-0) modules/veteran/app/sidekiq/organizations/queue_updates.rb (+53/-0) modules/veteran/app/sidekiq/organizations/update.rb (+192/-0) modules/veteran/app/sidekiq/organizations/xlsx_file_processor.rb (+156/-0) modules/veteran/spec/sidekiq/organizations/queue_updates_spec.rb (+53/-0) modules/veteran/spec/sidekiq/organizations/update_spec.rb (+756/-0) modules/veteran/spec/sidekiq/organizations/xlsx_file_processor_spec.rb (+75/-0) Note: We exclude files matching the following when considering PR size: *.csv, *.json, *.tsv, *.txt, *.md, Gemfile.lock, app/swagger, modules/mobile/docs, spec/fixtures/, spec/support/vcr_cassettes/, modules/mobile/spec/support/vcr_cassettes/, db/seeds, modules/vaos/app/docs, modules/meb_api/app/docs, modules/appeals_api/app/swagger/, *.bru, *.pdf Big PRs are difficult to review, often become stale, and cause delays.

Generated by 🚫 Danger

[jvcAdHoc]

There are some spots where this is expected to be labeled as 'id'. I think the specs pass because the test json object has the id key instead of poa.

[opticbob]

I've converted all of them back to id.

[jvcAdHoc]

This looks good to me, but I'd like @holdenhinkle to get eyes on it too since he's more familiar with the file processing

[holdenhinkle]

LGTM!

[LindseySaari]

Since these are all true, is it better to just have an array of State abbreviations?

[opticbob]

Yep, that works for me!

[LindseySaari]

Would it be better to log this to datadog instead? Then you could put a monitor in place to look for this log

[opticbob]

@LindseySaari Do you have a place in vets-api where you could point me to a good example of logging to datadog? I see some code referencing statsd and some for Datadog::Tracing, but I'm not sure what you're looking for.

[opticbob]

@LindseySaari would it work to investigate datadog integration in a future PR? We are waiting for this to merge before we expand the percentage of users exposed to our product launch.

[LindseySaari]

Same here

[holdenhinkle]

Let's do DD after we go to 50% on Monday.

@LindseySaari can you please approve this if we promise to circle back and update the logging? :-)