Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collection paging #208

Merged
merged 27 commits into from
Feb 4, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
b84bd82
feat: paging strategy
tpluscode Jan 13, 2025
3370532
refactor: class plugins
tpluscode Jan 14, 2025
f659123
chore: update lock
tpluscode Jan 14, 2025
83420d9
test: explicit variable representation
tpluscode Jan 14, 2025
0cdb2ba
test: applying template to resource URL
tpluscode Jan 14, 2025
935e1fa
refactor: finding plugin by name
tpluscode Jan 15, 2025
3e71812
chore: update deploy plugin to class
tpluscode Jan 15, 2025
71893f1
chore: update express plugin middlewares
tpluscode Jan 15, 2025
e210c2a
ci: add smoke tests
tpluscode Jan 15, 2025
856cf5e
fix: replacing params
tpluscode Jan 15, 2025
de47193
ci: request wait timeout
tpluscode Jan 15, 2025
ea141e2
ci: add logs to artifacts
tpluscode Jan 15, 2025
337987a
ci: more logs to smoke test api
tpluscode Jan 15, 2025
5557d47
ci: smoke test prod command
tpluscode Jan 15, 2025
7fbf54b
ci: smoke test with lando
tpluscode Jan 15, 2025
7ed0278
ci: ignore lando certificate
tpluscode Jan 15, 2025
769153f
ci: log on smoke test failure
tpluscode Jan 15, 2025
db71d73
ci: wait longer
tpluscode Jan 15, 2025
833d0b1
Merge remote-tracking branch 'origin/master' into collection-paged
tpluscode Feb 3, 2025
219ef54
fix: prev/next links only when they make sense
tpluscode Feb 3, 2025
87e7dbb
example: iterate collection
tpluscode Feb 3, 2025
c834ab0
ci: install swipl
tpluscode Feb 3, 2025
175c538
feat: inline config
tpluscode Feb 4, 2025
ac238a7
ci: tests
tpluscode Feb 4, 2025
0062522
ci: run compose in example dir
tpluscode Feb 4, 2025
41e7da1
ci: npm install for smoke tests
tpluscode Feb 4, 2025
e782988
ci: wait for resource which exists
tpluscode Feb 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/four-baboons-protect.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@kopflos-cms/vite": patch
---

Add inline `config` to plugin config
5 changes: 5 additions & 0 deletions .changeset/short-lemons-agree.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@kopflos-cms/core": minor
---

Changed plugin setup to require classes
5 changes: 5 additions & 0 deletions .changeset/shy-bobcats-reply.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@kopflos-cms/core": patch
---

Added helper to easily access plugin instance
5 changes: 5 additions & 0 deletions .changeset/tall-maps-thank.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@kopflos-cms/hydra": minor
---

Created extensible method for collection paging strategies
7 changes: 7 additions & 0 deletions .changeset/tidy-mice-tell.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"@kopflos-cms/express": patch
"@kopflos-cms/vite": patch
"kopflos": patch
---

Plugins are now implemented as classes
19 changes: 19 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ jobs:
path: packages/sparql-path-parser
steps:
- uses: actions/checkout@v4
- uses: fabasoad/setup-prolog-action@v1
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
Expand All @@ -48,6 +49,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: fabasoad/setup-prolog-action@v1
- uses: actions/setup-node@v4
with:
node-version: lts/*
Expand All @@ -58,8 +60,25 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: fabasoad/setup-prolog-action@v1
- uses: actions/setup-node@v4
with:
node-version: lts/*
- run: npm ci
- run: npm run -ws --if-present build

smoke-test-example:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: fabasoad/setup-prolog-action@v1
- uses: actions/setup-node@v4
with:
node-version: lts/*
- run: npm ci
- run: docker compose up -d
working-directory: example
- run: npm run -w example start &
- run: npx wait-on http://localhost:1429/plaque/newton-s-apple-tree-monash-university -t 30000

- run: npx api-tuner example/tests/*.n3 --base-iri http://localhost:1429/
34 changes: 34 additions & 0 deletions .lando.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: read-the-plaque
services:
oxigraph:
api: 3
type: lando
entrypoint: /usr/local/bin/oxigraph
app_mount: false
scanner: false
ssl: true
services:
image: ghcr.io/oxigraph/oxigraph:0.4.6
user: root
command: serve --location /data --bind 0.0.0.0:7878
ports:
- 7878
volumes:
- ./example/oxigraph:/data
app:
type: node:20
ssl: true
scanner: false
build:
- npm i
command: cd example; kopflos serve --mode development --trust-proxy
overrides:
environment:
API_BASE: https://read-the-plaque.lndo.site
DB_URI: http://db.read-the-plaque.lndo.site

proxy:
oxigraph:
- db.read-the-plaque.lndo.site:7878
app:
- read-the-plaque.lndo.site:1429
2 changes: 1 addition & 1 deletion example/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
services:
oxigraph:
image: ghcr.io/oxigraph/oxigraph:0.4.4
image: ghcr.io/oxigraph/oxigraph:0.4.6
user: root
command: serve --location /data --bind 0.0.0.0:7878
ports:
Expand Down
20 changes: 14 additions & 6 deletions example/kopflos.config.ts
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
import * as url from 'node:url'
import type { KopflosConfig } from '@kopflos-cms/core'

const baseIri = process.env.API_BASE || 'http://localhost:1429'
const dbUri = process.env.DB_URI || 'http://localhost:7878'

export default <KopflosConfig> {
baseIri: 'http://localhost:1429',
apiGraphs: ['http://localhost:1429/api'],
baseIri,
apiGraphs: [baseIri + '/api'],
sparql: {
default: {
endpointUrl: 'http://localhost:7878/query?union-default-graph',
updateUrl: 'http://localhost:7878/update',
storeUrl: 'http://localhost:7878/store',
endpointUrl: dbUri + '/query?union-default-graph',
updateUrl: dbUri + '/update',
storeUrl: dbUri + '/store',
},
lindas: 'https://lindas.admin.ch/query',
},
Expand All @@ -27,9 +30,14 @@ export default <KopflosConfig> {
'@kopflos-cms/vite': {
root: 'ui',
entrypoints: ['ui/*.html'],
config: {
server: {
allowedHosts: ['read-the-plaque.lndo.site'],
},
},
},
'@kopflos-cms/hydra': {
apis: ['http://localhost:1429/api'],
apis: [baseIri + '/api'],
},
},
}
13 changes: 7 additions & 6 deletions example/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,22 +12,23 @@
},
"dependencies": {
"@kopflos-cms/hydra": "0.0.0",
"@kopflos-cms/vite": "0.1.1",
"@kopflos-cms/serve-file": "0.1.0",
"@kopflos-labs/html-template": "0.1.0",
"@kopflos-cms/vite": "0.1.1",
"@kopflos-labs/handlebars": "0.1.0",
"@kopflos-labs/html-template": "0.1.0",
"@kopflos-labs/lit": "0.1.0",
"@openlayers-elements/core": "^0.3.0",
"@openlayers-elements/maps": "^0.3.0",
"@shoelace-style/shoelace": "^2.17.1",
"@webcomponents/template-shadowroot": "^0.2.1",
"compression": "^1.7.4",
"cors": "^2.8.5",
"express": "^5.0.1",
"kopflos": "0.1.3",
"lit-element": "^4.1.1",
"cors": "^2.8.5",
"compression": "^1.7.4"
"lit-element": "^4.1.1"
},
"devDependencies": {
"@types/express": "^5"
"@types/express": "^5",
"api-tuner": "github:zazuko/api-tuner"
}
}
22 changes: 22 additions & 0 deletions example/resources/countries.ttl
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,26 @@ prefix kl-hydra: <https://kopflos.described.at/hydra#>
sh:property [ sh:path schema:name ; sh:languageIn ("en" "de") ] ;
sh:property [ sh:path schema:identifier ] ;
] ;
hydra:limit 10 ;
sh:orderBy
[
sh:nodes [ sh:path schema:name ; ] ;
sh:filterShape
[
sh:languageIn ( "en" ) ;
] ;
] ;
hydra:search
[
hydra:template "{?limit,offset}" ;
hydra:mapping
[
hydra:variable "limit" ;
hydra:property hydra:limit ;
],
[
hydra:variable "offset" ;
hydra:property hydra:offset ;
] ;
] ;
.
57 changes: 57 additions & 0 deletions example/tests/collection-iterate.n3
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
PREFIX lg: <https://purl.org/lg/>
base <http://localhost:1429/>
PREFIX log: <http://www.w3.org/2000/10/swap/log#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX earl: <http://www.w3.org/ns/earl#>
prefix : <https://always-read-the-plaque.tests/plaques/>
prefix tuner: <https://api-tuner.described.at/>
prefix hydra:<http://www.w3.org/ns/hydra/core#>
prefix schema: <http://schema.org/>

:fetchEntireCollection
a earl:TestCase ;
rdfs:label "Check chaining requests to fetch entire collection" ;
:fetch <countries?limit=50> ;
.

{
:fetchEntireCollection :fetch ?page .
} => {
?page a tuner:Request ;
tuner:method "GET" ;
tuner:url ?page ;
.
} .

{
?req tuner:response ?res .

?res tuner:http_code 200 .
?res tuner:body ?body .
?body log:includes {
<countries> hydra:member ?member .
?member schema:name ?name .
} .

(
{
?body log:includes {
<countries>!hydra:view hydra:next ?nextPage .
} .
}
{
?body log:includes {
<countries>!hydra:view hydra:next ?nextPage .
} .
true log:becomes {
:fetchEntireCollection :fetch ?nextPage
}
}
{
true log:becomes { :fetchEntireCollection earl:outcome earl:passed }
}
) log:ifThenElseIn [] .
} => {
<countries> hydra:member ?member .
?member schema:name ?name .
} .
26 changes: 26 additions & 0 deletions example/tests/html-served.n3
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
base <http://localhost:1429/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX earl: <http://www.w3.org/ns/earl#>
PREFIX tuner: <https://api-tuner.described.at/>
prefix : <https://always-read-the-plaque.tests/html/>
prefix string: <http://www.w3.org/2000/10/swap/string#>

:fetchHTML
a earl:TestCase ;
rdfs:label "Check HTML is served" ;
.

:getPage
a tuner:Request ;
tuner:method "GET" ;
tuner:url <plaque/newton-s-apple-tree-monash-university.html> ;
.

{
:getPage tuner:response ?res .

?res tuner:http_code 200 .
?res tuner:header ( "content-type" "text/html.*" string:matches ) .
} => {
:fetchHTML earl:outcome earl:passed .
} .
Loading
Loading