v2.7.0 SQLAlchemy 2.0/Flask 3.* Update
- Updating from old deprecated werkzeug - general version update #474
- Update code from deprecated/legacy SQLAlchemy 1.4 calls
- pyflakes-directed linting of code to remove unnecessary parts, like unused f-strings
- Adding a changelog generated from the release notes #473 This created a CHANGELOG.md file based on the Github Release note information.
The main reason for the update was to fix a semicolon in a dockerfile, but this lead to discovering that a number of the libraries are deprecated, and needed to be updated. Due to the legacy code based on older SQLAlchemy patterns
Full Changelog: https://github.com/thegetty/lod-gateway/compare/v2.6.0b...v2.7.0
v2.6.0b RDF Prefix fix v2.6.0b - Created: 2024-09-23T17:25:48Z
- RDF prefixes used in 'id' fields should not be treated as relative URIs by @benosteen in #472
Full Changelog: https://github.com/thegetty/lod-gateway/compare/v2.6.0...v2.6.0b
v2.6.0 Dashboard, skip-to-datetime activitystream, and updates. v2.6.0 - Created: 2024-09-03T17:22:26Z
- bug of single record fixed by @ViktorGetty in #443
- Migrate to 'main' branch as default by @benosteen in #445
- Dev 16217 landing page gives an error if database is empty by @ViktorGetty in #446
- Handling language tags in JSON-LD by @benosteen in #447
- Fix Entity Title Text Overflow Bug by @ramonemunoz in #448
- Update Orb to Version 3 – DEV-16254 by @bluebinary in #449
- oldest and newest AS links by @ViktorGetty in #450
- Dev=16219 implement rest delete by @ViktorGetty in #452
- Shifting to explicitly use /dev/shm for tmp space by @benosteen in #453
- Fix Memento Typos in Dashboard by @garciagregg in #454
- Fix memento typo in Python code by @garciagregg in #455
- Update Docker Compose settings for Fuseki by @workergnome in #442
- CircleCI deploy to JPC (stage) by @benosteen in #456
- CircleCI update + homepage JSON version by @benosteen in #457
- Bump gunicorn from 20.1.0 to 22.0.0 by @dependabot in #458
- Removing caching possibilities from CircleCI by @benosteen in #459
- Reformatting to black 24 by @benosteen in #460
- Fix Activity Stream Responses for Records that Never Existed – DEV-17363 by @elenamujal in #463
- Bump flask-cors from 3.0.10 to 4.0.1 by @dependabot in #461
- Bump requests from 2.31.0 to 2.32.0 by @dependabot in #462
- Adding the skip-to-datetime API call for the main activity-stream by @benosteen in #464
- Homepage fixes when db empty by @benosteen in #465
- Adding lod-gateway-media-jpc-l2 to JPCA deploy by @benosteen in #466
- Not prefixing 'id' values that contain ':' if it has a valid scheme by @benosteen in #467
- Bugfix on subaddressing on deleted records by @benosteen in #468
- Small auth cleanup by @benosteen in #469
- Marking datetime when startup script starts + skip flask db upgrade on demand by @benosteen in #470
- Bump flask-cors from 4.0.1 to 5.0.0 by @dependabot in #471
- @elenamujal made their first contribution in #463
Full Changelog: https://github.com/thegetty/lod-gateway/compare/v2.5.3...v2.6.0
v2.5.3 HTML Frontpage with fixes to Memento & Prefix search v2.5.3 - Created: 2023-10-02T16:02:00Z
- Hotfix - prefix search by @benosteen in #415
- Simple hosting without an application root by @benosteen in #416
- Keepalive option by @benosteen in #417
- Keepalive by @benosteen in #418
- Dev 15801 create home page for lod gateway by @ViktorGetty in #419
- changed style.css location by @ViktorGetty in #420
- styles moved in template file by @ViktorGetty in #421
- dashboard route added by @ViktorGetty in #422
- Added Material Design and Custom Styles by @ramonemunoz in #423
- Dev 15842 use env to fill links by @ViktorGetty in #424
- Readme modified for Link Bank by @ViktorGetty in #425
- Dev 15801 lod gateway version by @ViktorGetty in #426
- Removed LOD Gateway from Page Title by @ramonemunoz in #428
- Added New Disabled State for Badge Indicators by @ramonemunoz in #429
- General Styling and Word Capitalizing by @ramonemunoz in #430
- Programmatically Set the Word From Change to Changes Depending on a Value by @ramonemunoz in #431
- Added Margin Top to Align Items by @ramonemunoz in #432
- write_version location changed by @ViktorGetty in #427
- revert version to local file by @ViktorGetty in #433
- copy version.txt to image by @ViktorGetty in #434
- copy version.txt by @ViktorGetty in #435
- change version capitalization by @ViktorGetty in #436
- Sort key setting by @benosteen in #440
- Bump gevent from 22.10.2 to 23.9.1 by @dependabot in #439
- Memento TimeMap - make application/link-format the default by @benosteen in #441
- Revise README File for Open Sourcing – DEV-16094 by @bluebinary in #438
- @ramonemunoz made their first contribution in #423
Full Changelog: https://github.com/thegetty/lod-gateway/compare/v2.5.2...v2.5.3
v2.5.2 Hotfix - nquads regex v2.5.2 - Created: 2023-08-01T16:14:51Z
Combinations of BNode subject/objects with bnode named graphs were being missed by the previous regex, causing a resource to fail being expanded (as the nquads was not being successfully converted to ntriples in this specific case)
- Updating the quads regex to handle bnodes in the RDFLib format by @benosteen in #413
Full Changelog: https://github.com/thegetty/lod-gateway/compare/v2.5.1...v2.5.2
v2.5.1 PyLD dethroning v2.5.1 - Created: 2023-07-19T23:14:17Z
The huge parse and reserialization times of large resources using PyLD is a huge problem. RDFLib v6+ has a json-ld parser and reserializer that seems to be functionally identical to the basic uses of PyLD. It does not handle framing, but this is not something used in the LOD Gateway application. In tests, a large (14MB) JSON-LD document took between 207-235 seconds to parse using PyLD. The same document took ~3s usig RDFLib. The ntriples output from both parsed forms were identical.
This release adds the ability to switch to use RDFLib entirely for the parsing and reserialization functionality in the gateway. The environment variable USE_PYLD_REFORMAT controls this - it currently defaults to true and will use PyLD. Set it explicitly to "false" to make the application only use RDFLib for handling the JSON-LD both on ingest and on reserialization.
- Yet another pyld workaround by @benosteen in #406
- Adding a default large timeout parameter of 600 by @benosteen in #407
- RDFlib parsing as an option by @benosteen in #408
- Versions of JSON-LD should also accept content negotiation by @benosteen in #409
- Version accept-datetime redirect fix by @benosteen in #410
Full Changelog: https://github.com/thegetty/lod-gateway/compare/v2.5...v2.5.1
v2.5 Gunicorn and JSON Logging v2.5 - Created: 2023-07-17T18:09:10Z
Switches from the uWSGI deployment to use Gunicorn. Also includes options for JSON formatted log messages for status and access logs, as well as a small fix for the basegraph handling.
- Test mode basegraph by @benosteen in #392
- Tweaks to the uwsgi to see if it helps by @benosteen in #393
- Adding timing log information for upstream SPARQL service calls by @benosteen in #394
- Logging sparql calls by @benosteen in #395
- Adding some better than defaults to the keep alive by @benosteen in #396
- Not sure why it's missing these packages by @benosteen in #397
- Switching to 75s keep alive to match nginx by @benosteen in #398
- Fixing a bug with the base graph by @benosteen in #399
- Changed local thesaurus option by @garciagregg in #401
- Gunicorn migration by @benosteen in #400
- Changing default worker/threads by @benosteen in #402
- Switching some small defaults to new values by @benosteen in #403
- Access Logs as JSON via Env variable control by @benosteen in #404
- Gunicorn JSON log tweaks by @benosteen in #405
Full Changelog: https://github.com/thegetty/lod-gateway/compare/v2.4.1...v2.5
Cache Control v2.4.1 - Created: 2023-06-23T22:28:56Z
Hotfix to include 'Cache-Control: no-cache' with every request, to try to get anything downstream from trying to cache resources without checking.
- Adding 'Cache-control: no-cache' to all responses by @benosteen in #390
- Turning off strict mode to debug in staging by @benosteen in #391
Full Changelog: https://github.com/thegetty/lod-gateway/compare/v2.4...v2.4.1
RDF improvements v2.4 v2.4 - Created: 2023-06-14T00:25:03Z
- Base graph change log warning in wrong place by @benosteen in #382
- Dev 14133 by @ViktorGetty in #383
- Fix for when the nquads export includes ntriples mixed in by @benosteen in #384
- Force all graph replacements to be triples by @benosteen in #385
- Bump requests from 2.27.1 to 2.31.0 by @dependabot in #387
- Bump flask from 2.0.3 to 2.3.2 by @dependabot in #386
- Shorter fix for the datetime comparison for TimeGate Memento API by @benosteen in #388
- Adding the custom doc loader to the RDF handling. by @benosteen in #389
Full Changelog: https://github.com/thegetty/lod-gateway/compare/v2.3...v2.4
RDF Enhancements; Momento Bug Fixes; uWSGI Performance Enhancements v2.3 - Created: 2022-12-16T00:13:11Z
- Updating versions, and black linter by @benosteen in #366
- Added TMS L1 to staging deploy by @garciagregg in #368
- Add TMS L1 to prod deploy by @garciagregg in #369
- Sparql -> openjdk:20 by @benosteen in #370
- Added staging L1 and L2 instances for JPC data to the deployment by @garciagregg in #371
- Adding Consonance L1 to staging and prod deployment by @garciagregg in #372
- Refresh/revert Graph API call by @benosteen in #373
- Adding a more refined uwsgi.ini file, with worker scaling by @benosteen in #374
- Optional Prev Link Header by @benosteen in #375
- RDF Transforms by @benosteen in #376
- Base Graph Filter by @benosteen in #377
- Suppress basegraph from the activitystream by @benosteen in #378
- Trying to switch uwsgi to lazy mode by @benosteen in #379
- Fixing the first/last in the timemap list by @benosteen in #380
- Relative id flag by @benosteen in #381
Full Changelog: https://github.com/thegetty/lod-gateway/compare/v2.2...v2.3
SQLAlchemy Optimizations v2.2 v2.2 - Created: 2022-06-30T21:18:29Z
- More use of default and explicit deferred queries for ORM objects, as well as
load_onlyfor Memento-based requests - New index to speed up common queries
- Removal of unused indexes to increase write performance
- DEV-12717 Version SQLAlchemy optimizing by @benosteen in #365
Full Changelog: https://github.com/thegetty/lod-gateway/compare/v2.1...v2.2
Rollback activity-stream to fixed width offset v2.1 - Created: 2022-04-29T18:22:05Z
The query was changed to try to accommodate activity-stream event deletions without leaving gaps in the activity-stream. This was not performant enough at the larger scale without removing events, so rolling back to the original fixed-width query.
Deletions of old duplicated or stale events will leave gaps in the record and there will also not be a reduction in the number of pages a client will have to retrieve to get the whole history.
- Revert to fixed offset queries by @benosteen in #364
Full Changelog: https://github.com/thegetty/lod-gateway/compare/v2.0...v2.1
Full Versioning, Subaddressing and Activity-stream updates v2.0 - Created: 2022-04-26T22:06:27Z
Main new features:
- Full versioning, with Memento support
- Subaddressing (relative URIs within documents stored in the LOD Gateway will resolve if turned on)
- Activity-stream truncation API (per entity)
- Shifted to python 3.9, updated all main dependencies by @benosteen in #353
- Should have been math.ceil not int by @benosteen in #355
- Full versioning by @benosteen in #354
- Hotfix - fixing a memento date by @benosteen in #358
- Per-entity activitystreams, and truncation by @benosteen in #357
- Tweaking the query to be more perfomant at the end of an activitystream by @benosteen in #359
- Optimize activitystream query by @benosteen in #360
- Adding subaddressing functionality by @benosteen in #356
- Changing orb version to latest v2 by @benosteen in #362
Full Changelog: https://github.com/thegetty/lod-gateway/compare/v1.1.0...v1.2
Sparql Update Refactor v1.1.0 - Created: 2022-03-05T17:59:01Z
- Refactoring of Sparql update functionality in ingest
- Enhanced error reporting in ingest with Retry-After added to response header when graph store is overloaded
- Local thesaurus modeling switched to Linked.Art from SKOS =======================================
Sparql POST Return Headers v1.0.8 - Created: 2022-02-09T16:15:03Z
Sparql POST Modifications v1.0.7 - Created: 2022-02-09T01:10:58Z
Sparql Request Debugging v1.0.6 - Created: 2022-02-08T20:20:23Z
Glob listing update v1.0.5 - Created: 2021-12-21T00:33:27Z
Listing the contents of the LOD Gateway by using a wildcard suffix (*) now only includes current versions of resources.
Add Local Thesaurus to Production Deployment v1.0.4 - Created: 2021-10-15T00:09:36Z
Local Thesaurus Functionality v1.0.3 - Created: 2021-10-14T23:00:29Z
Added functionality for creating a local thesaurus with an in-memory SQLlite database. Local thesaurus data is loaded at container startup from a CSV file located at URL set in the LOCAL_THESAURUS_URL envar.
Update SPARQL HTTP Error Handling v1.0.2 - Created: 2021-09-13T19:16:08Z
Activity Stream Update v1.0.1 - Created: 2021-09-03T00:42:43Z
- added "endTime" property to activity stream items to eventually replace "created" in a later version
- improved error reporting for invalid JSON ingest payloads =======================================
LOD Gateway Version 1.0.0 v1.0.0 - Created: 2021-08-10T19:05:21Z