Release 2.3.8 #395
Merged
Release 2.3.8 #395
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Refactoring the forgotten password code to split it up into more modular, smaller views. * black refactoring * black refactoring * Splitting up long line * black reformatting * Fixing up local.env.example * [ci skip] AUTOMATED - update fitness functions * Fixing up local.env.example * [ci skip] AUTOMATED - update fitness functions * Added line break to local.env.example * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]>
* build(deps): bump minimist from 1.2.5 to 1.2.6 Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> * [ci skip] AUTOMATED - update fitness functions Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]>
* Refactoring the forgotten password code to split it up into more modular, smaller views. * black refactoring * black refactoring * Splitting up long line * black reformatting * Prelminary work on inviting existing third parties to cases * [ci skip] AUTOMATED - update fitness functions * Updated requirements.txt to use new client branch * [ci skip] AUTOMATED - update fitness functions * Updated requirements.txt to use new client branch * made trade_remedies_public a module, getting module not found errors on Jenkins * [ci skip] AUTOMATED - update fitness functions * Monkey patching the BaseRegisterView as build still failing * [ci skip] AUTOMATED - update fitness functions * Checking is user is logged in * New modified invite flow for third parties * [ci skip] AUTOMATED - update fitness functions * Changed the wording to better reflect when the emails get sent * [ci skip] AUTOMATED - update fitness functions * black and flake8 * [ci skip] AUTOMATED - update fitness functions * Moved BaseRegisterView * Black * [ci skip] AUTOMATED - update fitness functions * Removed the special client branch from requirements * [ci skip] AUTOMATED - update fitness functions * Re-removed the redirect invite code * [ci skip] AUTOMATED - update fitness functions * remove redundant noqa occurences * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]> Co-authored-by: nboyse <[email protected]>
* Changed Procfile to stop compiling CSS * [ci skip] AUTOMATED - update fitness functions * Changed Procfile to compile CSS before collectstatic * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]>
* feature: add codecov to repo * [ci skip] AUTOMATED - update fitness functions * Remove redundant variable and extra arguments * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]>
* Refactored circleci config.yml * Added python flake8 to requirements * Updated circleci ssh key fingerprint * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]>
* Refactored circleci config.yml * Added python flake8 to requirements * Updated circleci ssh key fingerprint * [ci skip] AUTOMATED - update fitness functions * Updated pflake8 exception ignore to match original * [ci skip] AUTOMATED - update fitness functions * Added pre-commit hooks to circleci * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]>
* Fix - update python runtime Updating python version in runtime.txt as 3.9.10 is no longer supported in Python Buildpack 1.7.53 * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]>
* Refactoring the forgotten password code to split it up into more modular, smaller views. * black refactoring * black refactoring * Splitting up long line * black reformatting * Prelminary work on inviting existing third parties to cases * [ci skip] AUTOMATED - update fitness functions * Updated requirements.txt to use new client branch * [ci skip] AUTOMATED - update fitness functions * Updated requirements.txt to use new client branch * made trade_remedies_public a module, getting module not found errors on Jenkins * [ci skip] AUTOMATED - update fitness functions * Monkey patching the BaseRegisterView as build still failing * [ci skip] AUTOMATED - update fitness functions * Checking is user is logged in * New modified invite flow for third parties * [ci skip] AUTOMATED - update fitness functions * Changed the wording to better reflect when the emails get sent * [ci skip] AUTOMATED - update fitness functions * black and flake8 * [ci skip] AUTOMATED - update fitness functions * Moved BaseRegisterView * Black * [ci skip] AUTOMATED - update fitness functions * Removed the special client branch from requirements * [ci skip] AUTOMATED - update fitness functions * Re-removed the redirect invite code * [ci skip] AUTOMATED - update fitness functions * remove redundant noqa occurences * [ci skip] AUTOMATED - update fitness functions * Hopefully resolved some of the issues regarding the incorrect name being displayed to public users; 'you are x representing y' * [ci skip] AUTOMATED - update fitness functions * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]> Co-authored-by: nboyse <[email protected]>
…ew type radio button branch. (#127) * Renamed summary to is_notice * [ci skip] AUTOMATED - update fitness functions * Black reformatting * [ci skip] AUTOMATED - update fitness functions * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]>
* Updated DJango to 3.2.13 * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]>
* Updated async to >=2.6.4 * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]>
* Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Release 1 5 15 (#104) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 😐 * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Release 1 5 16 (#107) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 😐 * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * TRLST-550 - Merge release back into develop (#105) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> * Public client-side changes to allow for TRLST-536 - review type radio… (#106) * Public client-side changes to allow for TRLST-536 - review type radio buttons appear when applying for a new investigation and selecting a Notice from the dropdown. * black code reformat * HTML reformatting * Updated version Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> Co-authored-by: Mark Higham <[email protected]> * Release 1.5.17 (#113) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 😐 * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * TRLST-550 - Merge release back into develop (#105) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> * Public client-side changes to allow for TRLST-536 - review type radio… (#106) * Public client-side changes to allow for TRLST-536 - review type radio buttons appear when applying for a new investigation and selecting a Notice from the dropdown. * black code reformat * HTML reformatting * Release 1 5 16 (#108) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Release 1 5 15 (#104) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 😐 * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Updated version Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Upgrading django to 2.2.27 to resolve CVE-2022-23833 and CVE-2022-23833 (#109) * Upgrading django to 2.2.27 to resolve CVE-2022-23833 and CVE-2022-23833 * Generated .txt. requirement files Updated pip-tools to >6.5.0 * prod.txt generated * Update to 3.9.10 * Bump version Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> Co-authored-by: Mark Higham <[email protected]> * Bumped version to 1.5.18 * [ci skip] AUTOMATED - update fitness functions * [ci skip] AUTOMATED - update fitness functions * Deleted pytest.ini and updated config.yml * [ci skip] AUTOMATED - update fitness functions Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]>
* Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Release 1 5 15 (#104) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Release 1 5 16 (#107) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * TRLST-550 - Merge release back into develop (#105) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> * Public client-side changes to allow for TRLST-536 - review type radio… (#106) * Public client-side changes to allow for TRLST-536 - review type radio buttons appear when applying for a new investigation and selecting a Notice from the dropdown. * black code reformat * HTML reformatting * Updated version Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> Co-authored-by: Mark Higham <[email protected]> * Release 1.5.17 (#113) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * TRLST-550 - Merge release back into develop (#105) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> * Public client-side changes to allow for TRLST-536 - review type radio… (#106) * Public client-side changes to allow for TRLST-536 - review type radio buttons appear when applying for a new investigation and selecting a Notice from the dropdown. * black code reformat * HTML reformatting * Release 1 5 16 (#108) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Release 1 5 15 (#104) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Updated version Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Upgrading django to 2.2.27 to resolve CVE-2022-23833 and CVE-2022-23833 (#109) * Upgrading django to 2.2.27 to resolve CVE-2022-23833 and CVE-2022-23833 * Generated .txt. requirement files Updated pip-tools to >6.5.0 * prod.txt generated * Update to 3.9.10 * Bump version Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> Co-authored-by: Mark Higham <[email protected]> * Release 1.5.18 (#131) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * TRLST-550 - Merge release back into develop (#105) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> * Public client-side changes to allow for TRLST-536 - review type radio… (#106) * Public client-side changes to allow for TRLST-536 - review type radio buttons appear when applying for a new investigation and selecting a Notice from the dropdown. * black code reformat * HTML reformatting * Release 1 5 16 (#108) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Release 1 5 15 (#104) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in…
…used (#135) * Deleted references to node and npm as no longer being used * [ci skip] AUTOMATED - update fitness functions * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]>
* Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Release 1 5 15 (#104) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Release 1 5 16 (#107) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * TRLST-550 - Merge release back into develop (#105) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> * Public client-side changes to allow for TRLST-536 - review type radio… (#106) * Public client-side changes to allow for TRLST-536 - review type radio buttons appear when applying for a new investigation and selecting a Notice from the dropdown. * black code reformat * HTML reformatting * Updated version Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> Co-authored-by: Mark Higham <[email protected]> * Release 1.5.17 (#113) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * TRLST-550 - Merge release back into develop (#105) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> * Public client-side changes to allow for TRLST-536 - review type radio… (#106) * Public client-side changes to allow for TRLST-536 - review type radio buttons appear when applying for a new investigation and selecting a Notice from the dropdown. * black code reformat * HTML reformatting * Release 1 5 16 (#108) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Release 1 5 15 (#104) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Updated version Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Upgrading django to 2.2.27 to resolve CVE-2022-23833 and CVE-2022-23833 (#109) * Upgrading django to 2.2.27 to resolve CVE-2022-23833 and CVE-2022-23833 * Generated .txt. requirement files Updated pip-tools to >6.5.0 * prod.txt generated * Update to 3.9.10 * Bump version Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> Co-authored-by: Mark Higham <[email protected]> * Release 1.5.18 (#131) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * TRLST-550 - Merge release back into develop (#105) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> * Public client-side changes to allow for TRLST-536 - review type radio… (#106) * Public client-side changes to allow for TRLST-536 - review type radio buttons appear when applying for a new investigation and selecting a Notice from the dropdown. * black code reformat * HTML reformatting * Release 1 5 16 (#108) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Release 1 5 15 (#104) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not…
* get selenium tests running * black * attempt to get chrome onto circle ci * use browsers image * do not specify chromedriver path * install browsers via orb * try without orb * try again without orb * Revert "try again without orb" This reverts commit f9c1628. * Revert "try without orb" This reverts commit 5d0d66a. * remove unused commands * ps ignore * added first functional test * lands on right page * added styling * added back button and redesigned password request * fixed back button, defined sign in flow in middleware * sends reset email * use base with form * update password reset email sent template * added redesigned reset password template * back button middleware handles url kwargs * clean up finding non back urls * added password show/hide * added password criteria checks * use jquery and better special character criteria * added password reset success * updated non back urls * added page title * black * remove unused import * use deployed regex for special characters * Revert "added first functional test" This reverts commit c29ffaf * Revert "get selenium tests running" This reverts commit 0cf3745 * Revert "install browsers via orb" This reverts commit 61e606e. # Conflicts: # .circleci/config.yml * skip false positives * make pre-commit hooks happy * pre-commit ignores font files * stop using browsers image * [ci skip] AUTOMATED - update fitness functions * Update trade_remedies_public/password/views.py Co-authored-by: Christopher Pettinga <[email protected]> * [ci skip] AUTOMATED - update fitness functions * use try-except for reverse match * [ci skip] AUTOMATED - update fitness functions * remove background image * use base.html instead * use base.html instead * remove unused static * [ci skip] AUTOMATED - update fitness functions * import js files * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]> Co-authored-by: Christopher Pettinga <[email protected]>
* Making headway on the V2 login journey, abstracting HTML templates * Implementing the V2 login journey * Clearing up CSS Refactoring two-factor view * [ci skip] AUTOMATED - update fitness functions * get selenium tests running * black * attempt to get chrome onto circle ci * use browsers image * do not specify chromedriver path * install browsers via orb * try without orb * try again without orb * Revert "try again without orb" This reverts commit f9c1628. * Revert "try without orb" This reverts commit 5d0d66a. * remove unused commands * ps ignore * [ci skip] AUTOMATED - update fitness functions * MVP for the V2 login journey. * Made the V1 navbar more aesthetically similar to the new V2 one, not perfect yet. Fixed password show-hide button not centering when errors appear No longer show password show-hide button when JS is disabled Modified V2 decorator to reraise non validation-exceptions * Selenium tests should run headless * Ran black and added browsertools to circleci config.yml to run selenium front-end tests * flake8 fixes * Temporaily removing frontend teting from circleci * black formatting * [ci skip] AUTOMATED - update fitness functions * added first functional test * lands on right page * Moved the v2_error_handling decorator around as it causes issues with importing from trade_remedies_public module? * [ci skip] AUTOMATED - update fitness functions * Added static files to the right source folder in the templates directory, so that collectstatic moves them to public/static. * [ci skip] AUTOMATED - update fitness functions * added styling * added back button and redesigned password request * fixed back button, defined sign in flow in middleware * sends reset email * use base with form * update password reset email sent template * Changed client github link to point to correct branch * [ci skip] AUTOMATED - update fitness functions * added redesigned reset password template * back button middleware handles url kwargs * clean up finding non back urls * Updated requirements.txt to pull correct client branch * [ci skip] AUTOMATED - update fitness functions * added password show/hide * added password criteria checks * use jquery and better special character criteria * added password reset success * updated non back urls * added page title * black * remove unused import * use deployed regex for special characters * Revert "added first functional test" This reverts commit c29ffaf * Revert "get selenium tests running" This reverts commit 0cf3745 * Revert "install browsers via orb" This reverts commit 61e606e. # Conflicts: # .circleci/config.yml * skip false positives * make pre-commit hooks happy * pre-commit ignores font files * stop using browsers image * [ci skip] AUTOMATED - update fitness functions * Update trade_remedies_public/password/views.py Co-authored-by: Christopher Pettinga <[email protected]> * [ci skip] AUTOMATED - update fitness functions * use try-except for reverse match * [ci skip] AUTOMATED - update fitness functions * remove background image * use base.html instead * use base.html instead * remove unused static * [ci skip] AUTOMATED - update fitness functions * import js files * Impementing copy changes * [ci skip] AUTOMATED - update fitness functions * added email validation * added password validation * [ci skip] AUTOMATED - update fitness functions * fix error message styling * [ci skip] AUTOMATED - update fitness functions * flake8 and black * Mary requested changes from the 7th May: 1. <fieldset> shouldn’t be used to encapsulate entire form - they’ve been removed from the proto. All headers should therefore be a standard <h1> 2. Form error summary should have links to each error. See GDS: Error summary or see bottom example on proto: Sign in - GOV.UK Prototype Kit 3. All "functional" links .e.g. "Create account" or "Back" should not show visited state use class: "govuk-link--no-visited-state". Llet's keep visited state for links to outside articles etc 4. Start page. Missing a link for the Active Cases. Use this: Trade remedies 5. Start page. I've also changed the H1 of the start page and made it blue since you started it. Sign in - GOV.UK Prototype Kit * [ci skip] AUTOMATED - update fitness functions * Black formatting * [ci skip] AUTOMATED - update fitness functions * Flake8 formatting * [ci skip] AUTOMATED - update fitness functions * PII exclusion * [ci skip] AUTOMATED - update fitness functions * Mary's comments - https://uktrade.atlassian.net/browse/TRSV2-174?focusedCommentId=94102 * [ci skip] AUTOMATED - update fitness functions * PII exclusions. * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]> Co-authored-by: Johnny <[email protected]> Co-authored-by: Jonathan Li <[email protected]>
* get selenium tests running * black * attempt to get chrome onto circle ci * use browsers image * do not specify chromedriver path * install browsers via orb * try without orb * try again without orb * Revert "try again without orb" This reverts commit f9c1628. * Revert "try without orb" This reverts commit 5d0d66a. * remove unused commands * ps ignore * added first functional test * lands on right page * added styling * added back button and redesigned password request * fixed back button, defined sign in flow in middleware * sends reset email * use base with form * update password reset email sent template * added redesigned reset password template * back button middleware handles url kwargs * clean up finding non back urls * added password show/hide * added password criteria checks * use jquery and better special character criteria * added password reset success * updated non back urls * added page title * black * remove unused import * use deployed regex for special characters * Revert "added first functional test" This reverts commit c29ffaf * Revert "get selenium tests running" This reverts commit 0cf3745 * Revert "install browsers via orb" This reverts commit 61e606e. # Conflicts: # .circleci/config.yml * skip false positives * make pre-commit hooks happy * pre-commit ignores font files * stop using browsers image * [ci skip] AUTOMATED - update fitness functions * Update trade_remedies_public/password/views.py Co-authored-by: Christopher Pettinga <[email protected]> * [ci skip] AUTOMATED - update fitness functions * use try-except for reverse match * [ci skip] AUTOMATED - update fitness functions * remove background image * use base.html instead * use base.html instead * remove unused static * [ci skip] AUTOMATED - update fitness functions * import js files * added email validation * added password validation * [ci skip] AUTOMATED - update fitness functions * fix error message styling * added expired reset link page * use try-except * enable password reset request via user primary key * black * flake8 * ps ignore * handle expected TypeError * updated to current prototype * simplify * use show_password.js * use url reverse * use request id uuid instead of user pk uuid * remove unused imports * stop using fieldset * functional links do not show visited state * summary error messages link to input * no visited state for forgot your password * ps ignore * unfocus from input when submitting * put it back on the same line * remove commented out lines * update to use new base templates and error handling from api * remove unused imports * [ci skip] AUTOMATED - update fitness functions Co-authored-by: FITNESS-FUNCTIONS MACHINE USER <[email protected]> Co-authored-by: Christopher Pettinga <[email protected]>
* Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Release 1 5 15 (#104) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Release 1 5 16 (#107) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * TRLST-550 - Merge release back into develop (#105) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> * Public client-side changes to allow for TRLST-536 - review type radio… (#106) * Public client-side changes to allow for TRLST-536 - review type radio buttons appear when applying for a new investigation and selecting a Notice from the dropdown. * black code reformat * HTML reformatting * Updated version Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> Co-authored-by: Mark Higham <[email protected]> * Release 1.5.17 (#113) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * TRLST-550 - Merge release back into develop (#105) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> * Public client-side changes to allow for TRLST-536 - review type radio… (#106) * Public client-side changes to allow for TRLST-536 - review type radio buttons appear when applying for a new investigation and selecting a Notice from the dropdown. * black code reformat * HTML reformatting * Release 1 5 16 (#108) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Release 1 5 15 (#104) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Updated version Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Upgrading django to 2.2.27 to resolve CVE-2022-23833 and CVE-2022-23833 (#109) * Upgrading django to 2.2.27 to resolve CVE-2022-23833 and CVE-2022-23833 * Generated .txt. requirement files Updated pip-tools to >6.5.0 * prod.txt generated * Update to 3.9.10 * Bump version Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> Co-authored-by: Mark Higham <[email protected]> * Release 1.5.18 (#131) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * TRLST-550 - Merge release back into develop (#105) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> * Public client-side changes to allow for TRLST-536 - review type radio… (#106) * Public client-side changes to allow for TRLST-536 - review type radio buttons appear when applying for a new investigation and selecting a Notice from the dropdown. * black code reformat * HTML reformatting * Release 1 5 16 (#108) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Release 1 5 15 (#104) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn …
* Moving requirements around and adding dependabot.yml to point towards the new good stuff * changed directory * changed directory * added requirements.txt * Feature - TSS-1112 - Aligning vulnerability resolution (#374) * Moving requirements around and adding dependabot.yml to point towards the new good stuff * changed directory * changed directory * added requirements.txt * migrated to poetry and updated MakeFile * fixing circleci and Dockerfile * black * Feature - TSS-1112 - Aligning vulnerability resolution (#375) * Moving requirements around and adding dependabot.yml to point towards the new good stuff * changed directory * changed directory * added requirements.txt * migrated to poetry and updated MakeFile * fixing circleci and Dockerfile * black * inducing vulnerability * Copying over the good changes * updated poetry.lock * fixing circleci * adding gunicorn * black * adding gevent
* Bump django from 3.2.17 to 4.2.7 Bumps [django](https://github.com/django/django) from 3.2.17 to 4.2.7. - [Commits](django/django@3.2.17...4.2.7) --- updated-dependencies: - dependency-name: django dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> * Regenerating requirements.txt file --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Christopher Pettinga <[email protected]>
* Bump boto3 from 1.17.89 to 1.29.6 Bumps [boto3](https://github.com/boto/boto3) from 1.17.89 to 1.29.6. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](boto/boto3@1.17.89...1.29.6) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Regenerating requirements.txt file --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Christopher Pettinga <[email protected]>
* Bump django-environ from 0.4.5 to 0.11.2 Bumps [django-environ](https://github.com/joke2k/django-environ) from 0.4.5 to 0.11.2. - [Release notes](https://github.com/joke2k/django-environ/releases) - [Changelog](https://github.com/joke2k/django-environ/blob/main/CHANGELOG.rst) - [Commits](joke2k/django-environ@v0.4.5...v0.11.2) --- updated-dependencies: - dependency-name: django-environ dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Regenerating requirements.txt file --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Christopher Pettinga <[email protected]>
* Bump werkzeug from 2.3.8 to 3.0.1 Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.3.8 to 3.0.1. - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@2.3.8...3.0.1) --- updated-dependencies: - dependency-name: werkzeug dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> * Regenerating requirements.txt file --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Christopher Pettinga <[email protected]>
* Bump whitenoise from 5.3.0 to 6.6.0 Bumps [whitenoise](https://github.com/evansd/whitenoise) from 5.3.0 to 6.6.0. - [Changelog](https://github.com/evansd/whitenoise/blob/main/docs/changelog.rst) - [Commits](evansd/whitenoise@v5.3.0...6.6.0) --- updated-dependencies: - dependency-name: whitenoise dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> * Regenerating requirements.txt file --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Christopher Pettinga <[email protected]>
…oduct DB if it exists
Also fully removed feedback references and changed never_cache to classmthoddecorator for Django 4.x
* Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Release 1 5 15 (#104) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Release 1 5 16 (#107) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * TRLST-550 - Merge release back into develop (#105) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> * Public client-side changes to allow for TRLST-536 - review type radio… (#106) * Public client-side changes to allow for TRLST-536 - review type radio buttons appear when applying for a new investigation and selecting a Notice from the dropdown. * black code reformat * HTML reformatting * Updated version Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> Co-authored-by: Mark Higham <[email protected]> * Release 1.5.17 (#113) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * TRLST-550 - Merge release back into develop (#105) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> * Public client-side changes to allow for TRLST-536 - review type radio… (#106) * Public client-side changes to allow for TRLST-536 - review type radio buttons appear when applying for a new investigation and selecting a Notice from the dropdown. * black code reformat * HTML reformatting * Release 1 5 16 (#108) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Release 1 5 15 (#104) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Updated version Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> * Upgrading django to 2.2.27 to resolve CVE-2022-23833 and CVE-2022-23833 (#109) * Upgrading django to 2.2.27 to resolve CVE-2022-23833 and CVE-2022-23833 * Generated .txt. requirement files Updated pip-tools to >6.5.0 * prod.txt generated * Update to 3.9.10 * Bump version Co-authored-by: Dave Charles <[email protected]> Co-authored-by: Luisella Strona <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: nboyse <[email protected]> Co-authored-by: Tash Boyse <[email protected]> Co-authored-by: Mark Higham <[email protected]> * Release 1.5.18 (#131) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunicorn was sinking them. Added --capture-output config to gunicorn. * feat: remove help mailto. The [email protected] email is being retired and as such needs to be removed from the public portal. TRLST-295 raised replace this with a case specific email. * fix: trlst-299 django vulnerability - Bump django version to 2.2.20 * fix: TRLST-281 save and continue. This change fixes an issue where a third party invitee's organisation details were not displayed if the user edited the contact in the submission. This was mainly solved in the API, this change extracts the right attr name for organisation_address as well as tidy up the rendering. * feat: trlst 296 merge release into develop Merge release branch back into develop. Updated mismatched deps after conflict resolution required when merging release branch with develop. Also includes updated version. * feat: TRLST-306 add pip-tools to dev deps TRLST 304 introduced an improved mechanism in TR public service to use pip-compile on the container rather than in the developer's local env. However, pip-tools (required for pip-compile) is not in the API/Public/Caseworker deps and therefore does not get included in the container build. This change adds pip-tools as a dev dep, and also performs on-container build of requirements. * merge: trlst 305 merge hotfix into develop This change merges branch 'master' into develop, incorporating the hotfix made under TRLST 304. It also reintroduces appnope, seems this is required as it is a dep of iPython. * feat: TRLST-301 add audit log middleware. Added audit log middleware to service in production. See https://github.com/uktrade/django-audit-log-middleware. * feat: TRLST-315 bump runtime. (#54) PaaS buildpack python support requires bump of python runtime. * fix: bump hosted-git-info to resolve vulnerability. (#55) * feat: add IHTC compliance settings Added IHTC compliance settings. This included some wrangling of npm packages, mainly solved by dropping CI node docker image version to `lts`. * feat: trlst 327 hardening 2 - Restructured so we can apply settings in lower envs if required. - included in staging. - Also tidies logging def inconsistencies. * fix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 * LSGH-5 sec: bump django to resolve vulnerability * fix: TRLST-339 malicious file upload. - Updated deps to use new chunk uploader - Bumped boto and django-storages to match - Removed and ignored sqlite3 db that is built on deployment - Update view to use new handler and detect malware response - Added necessary settings to django config - Clean-up, removed cruft routes * hotfix: TRLST-343 Prevent external redirection in login flow * hotfix: TRLST-342 escape text_element template tag value to prevent s-xss. * hotfix: TRLST-344 cycle session key just like django.contrib.auth.login. * hotfix: TRLST 345 vulnerable third-party libraries - Bumped JQuery to 3.5.1 - Updated gov template to use JQuery 3.5.1 - Used same JQUI version as caseworker for consistency * hotfix: TRLST 349 information leakage Remediation for pen test observations: - redacted gunicorn server signature. * fix: trlst-308 third party invited org * render third party organisation and role correctly * tidy formatting and use correct extends ref * fix up invites. - Improved validations - Handle country properly - Fix issues with edits * set ci node img to lts * make invite email verify message clearer. * repair user creation and editing. * don't show edit link when a third party invite. * don't shadow built in name. * add validators for 3p contact form. * sanitise add 3p user route. * only one 3p can be added. * 3P contact processing and validation. * validate address. * intercept 3rd party draft submission navigation. * manage draft form data. * display inviting org banner. * show draft and completed correctly. * feat: trlst 295 case specific email When on a case related page this change updates the help box to include a case specific email address. * feat: Fix countdown and improve page layout. Fix bug that caused 2FA lock countdown to not be displayed, and improve 2FA challenge layout. This is a quick and dirty fix as the API 2FA views are a mess. If/when the API layer is cleaned up then Public and Caseworker portal's 2FA usage can be refactored to be less abysmal. - ensure 2fa locked countdown is displayed. - improve page layout. * merge: release 1.5.10 back into develop * fix: reset errors and redirect correctly. - Reset the session correctly when validations pass - After a user has been onboarded make sure we navigate them to the email verify page. * fix: Update config folder name Updated config folder name to be consistent with LST practice. * merge: master hotfix into develop Merge 1.5.10.1 hotfix into develop. * fix: updated example config. - Updated example config for use with new chunk uploader - Added SQLite DB updates containing chunk uploader migration. These aren't actually used, just catering for way chunk uploader app is installed. - Fixed some hitherto undetected CI fails (probs not detected because of creds refresh requirement) * merge: release 1.5.11 into develop * fix: upgrade pip packages with pip-compile * fix: change received to submitted * fix: amending breadcrumb wording for archive and active cases * Amend archive field * fix: remove change details link on public account details page * Removing all references to ACCOUNT_INFO_READ_ONLY * Flake8 :neutral_face: * fix: LSGH-41 - package-lock.json update * merge: release 1.5.12 into develop - Version bump - Runtime bump * feature: pre-commit hooks * fix: amend pii-ignore comment to prevent public site breaking * fix: Allow notices to be registered in public site under 'Apply for a new investigation' (#93) * Fix: TRLST-475 refactor and remove av scan logic As part of the effort to remove all the API related anti-virus scan logic this is a set of related tidy up changes: - There is no concept of an "unsafe" document anymore as malware is rejected at point of upload, updated download links and tests accordingly. - `django_chunk_upload_handlers` package expects an `AWS_REGION` setting (although this was not a functional issue), added to stop package logging complaint. - There we some tests that were not being executed properly due to invalid `pytest` config. The tests needed fixing up too. * merge: TRLST-499 release into develop - version bump * fix: amend regex check to accept integers * merge: TRLST-528 release into develop * TRLST-551 - upgrading iPython version due to CVE-2022-21699 (#102) * Upgrade Django to 2.2.26 (#103) * TRLST-550 - Merge release back into develop (#105) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Updated version to 1.5.15 * Forced package.json to use "node-fetch": ">=2.6.7" to avoid CVE-2022-0235 Co-authored-by: Mark Higham <[email protected]> Co-authored-by: Ross Miller <[email protected]> Co-authored-by: davecharles <[email protected]> * Public client-side changes to allow for TRLST-536 - review type radio… (#106) * Public client-side changes to allow for TRLST-536 - review type radio buttons appear when applying for a new investigation and selecting a Notice from the dropdown. * black code reformat * HTML reformatting * Release 1 5 16 (#108) * Release 1.5.6 * Update traitlets * Black and flake8 checks * Feature/trlst 171 func tests (#17) * Display date when document was submitted to TRA (TRLST 160) * Branding changes * Bump version * Bump version * bump version to 1.5.7 * merge: release 1.5.8 branch into master Release TR Public 1.5.8 * Hotfix/trlst 282 high sev vulnerabilities hotfix: release for Django vulnerability - Bumped django to 2.2.18 * merge: release 1.5.9 branch into master Release TR Public 1.5.9 * TRLST-304 Dependabot alert * Switch requirement generation to container * Update requirements and add relevant folders to compose config * hotfix: bump python runtime Required to enable deployment of TRS rebranding: - bump python runtime - set circle CI node img to lts * hotfix: bump deps to resolve vulnerabilities. - Bumped django - CVE-2021-31542 - Bumped boto3, requests, urllib3 - CVE-2021-33503 - Bump service version * LSGH-5 sec: bump django version to resolve vulnerability * merge: release 1.5.10 branch into master * hotfix: replace gulp SASS processing with django-sass-processor * Restructured to allow SASS compilation to be comprehended more easily * Removed gulp sass * Bumped version for hotfix. * Added compile SASS cmd to Procfile * merge: release 1.5.11 branch into master * Release 1.5.12 * fix: bump python runtime to 3.9.7. * Release 1.5.13 * Release 1.5.14 * Release 1 5 15 (#104) * Merge release 1.5.7 into develop (#30) This merge into develop from the 1.5.7 release branch incorporates release branch fixes and master branch hot fixes: * Update traitlets * Rationalise Procfile triggered processes * Branding changes * Dependabot alert fixes * Patch front end vulnerability * bump version to 1.5.7 * fix: http error page templates (trlst-252) Fixed 404/500 pages and add 400/403 * fix: TRLST_246 dependabot issues. * feat: trlst-131 add ecs logging Added logging per environment utilising ecs logger for json formatted log messages. * feat: trlst-262 bump python to 3.9.2 - Bumped python to 3.9.2 - Bumped Dockerfile python layer to 3.9.2 - Bumped PaaS runtime directive accordingly - Due to 3.9 api changes bumped gevent to latest - Bumped gunicorn accordingly - pip-compile latest grnerates more readable txt output hence significant txt diffs. - Bumped circleci python to 3.9.2 - Appease flake8 (method redefinition) * feat: trlst-216 Allow to remove non-confidential files without a confidential version. (#36) Normally a public user can only upload a confidential file, followed by the corresponding non-confidential version. A user managed to upload an extra non-confidential file (possibly by uploading from two different windows). The extra file is stops him from completing the submission, but there is no provision to remove it. Added an extra category when showing the uploaded file. The category lists the single non-confidential version, and ask the user to remove it. The interface allows to download the file before deleting it. * fix: trlst 259 issue post tasks Investigating an issue under TRLST 259 found some non critical issues in caseworker. This PR is a tidy up of the base settings file and the local.env.example file: - Employed consistent use of django-environ. - Removed environ.Env.read_env(). We use docker-compose for local development and feed the container settings from a local.env file. In PaaS we pick up the app's environment and do not deploy a '.env' file. - Simplified REDIS URL definition and removed hard coding of database number. - Removed 'TRUSTED_USER_TOKEN' setting, this is not used in the code base. - Removed cruft comments. - Pepified. * feat: bump develop to 1.5.9~a0. * fix: Django vulnerability - Bumped django to 2.2.18 * fix: vulnerabilities - Bumped ipython and pygments to resolve vulnerabilities. Note that iPython bump alone didn't bump pygments but updated anyway - Ran npm audit fix to resolve yargs-parser vulnerability. Note that large diff is due to v2 of lockfile format. - Had to perform some gulp wrangling to get build npm ci to succeed. I bumped my local npm to latest and looks like longer hashes are now generated in the package lock file. * fix: logging config. Django logs were not being emitted in PaaS because gunico…
# Conflicts: # .circleci/config.yml # .gitignore # Dockerfile # Makefile # fitness/fitness_metrics.db # pii-ner-exclude.txt # pii-secret-exclude.txt # pyproject.toml # requirements.in/base.in # requirements.in/dev.in # requirements.in/prod.in # requirements.txt # requirements/base.txt # requirements/dev.txt # requirements/prod.txt # trade_remedies_public/cases/views.py # trade_remedies_public/config/settings/base.py # trade_remedies_public/config/utils.py # trade_remedies_public/core/utils.py # trade_remedies_public/core/views.py # trade_remedies_public/password/tests.py # trade_remedies_public/registration/views.py # trade_remedies_public/templates/partials/widgets/help_box.html # trade_remedies_public/templates/v2/active_investigations/single_case_view.html
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.