Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update database variable #189

Merged
merged 6 commits into from
Jan 17, 2024
Merged

update database variable #189

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
aa2dabc
update database variable
peterstadler Dec 7, 2023
437fee7
fix XQuery declaration
peterstadler Jan 4, 2024
4618afd
update documentation
peterstadler Jan 4, 2024
0672ad4
add workaround for relative graphic URLs
peterstadler Jan 4, 2024
167618c
fix and improve query
peterstadler Jan 4, 2024
3d677be
apply improvement provided by @peterprovaznik
peterstadler Jan 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions manual/index_tools.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ <h2>Tools for generating indexes</h2>
<p>All of the tools described below are located in the 'tools' folder of your eXist
database. In order to generate any output they need a query parameter named 'c'
added to the URL to specify which file collection to generate an index for (for
instance: http://my_server_name.org/storage/tools/title_index?c=CNW ).</p>
instance: <code>http://localhost:8080/modules/tools/title_index.xq?c=Demo</code>).</p>
<p>The lists generated are meant as a starting point for an index. Most likely you will
need to make manual corrections.</p>
<p>If you want to change the default formatting such as the use of italics, you may want
Expand All @@ -40,7 +40,7 @@ <h2>Tools for generating indexes</h2>
<p>&#160;</p>
<p><b>Title index</b></p>
<p>To generate a simple list of titles, alphabetically ordered, use the <a
href="../modules/tools/title_index.xq">title_index.xq</a> script located in the
href="../modules/tools/title_index.xq?c=Demo">title_index.xq</a> script located in the
'tools' folder of your eXist database.</p>
<p>The script generates a list in two sections: First a list of main titles with
subtitles and (in parentheses) alternative titles. Below that, it generates a list
Expand All @@ -49,7 +49,7 @@ <h2>Tools for generating indexes</h2>
<p><img alt="title index" src="../resources/images/manual/title_index.png" class="illustration" onload="this.width/=1.5;" /></p>
<p>&#160;</p>
<p><b>Index of names</b></p>
<p>The script <a href="../modules/tools/name_index.xq">name_index.xq</a> produces a list
<p>The script <a href="../modules/tools/name_index.xq?c=Demo">name_index.xq</a> produces a list
of all names encoded as &lt;persName&gt; elements within the &lt;sourceDesc&gt; and
&lt;workList&gt; elements. This restriction ensures that persons such as the encoder
or project manager (usually encoded in &lt;fileDesc&gt;) are not included.</p>
Expand All @@ -59,41 +59,41 @@ <h2>Tools for generating indexes</h2>
entire database several times. If you experience a server timeout when running
it, you may want to try a two-step approach instead. However, this solution
involves editing one of the scripts on the server. First, use the script <a
href="../modules/tools/name_index_step1.xq">name_index_step1.xq</a> to
href="../modules/tools/name_index_step1.xq?c=Demo">name_index_step1.xq</a> to
generate a list of all the persons found. This list is pasted into the source
code of the second script (the code indicates where), <a
href="../modules/tools/name_index_step2.xq">name_index_step2.xq</a>, which in
href="../modules/tools/name_index_step2.xq?c=Demo">name_index_step2.xq</a>, which in
turn searches for alle occurrences of each name on the list.</li>
</ul>
<p>&#160;</p>
<p><b>Index of text authors</b></p>
<p>The index of text authors, <a href="../modules/tools/text_author_index.xq"
<p>The index of text authors, <a href="../modules/tools/text_author_index.xq?c=Demo"
>text_author_index.xq</a>, is a just subset of the general index of names, only
including the &lt;persName&gt; elements having @role='author'. This index is
intended to list the authors of vocal text (lyrics) only; it does not include
authors of documentary material or literature such as letters, articles and books,
which MerMEId defines using the &lt;author&gt; element.</p>
<p>&#160;</p>
<p><b>Index of opus numbers</b></p> The script <a href="../modules/tools/opus_index.xq"
<p><b>Index of opus numbers</b></p> The script <a href="../modules/tools/opus_index.xq?c=Demo"
>opus_index.xq</a> lists the titles of works having an identifier named 'Opus' at
the 'Work' tab:<br />
<img alt="opus number" src="../resources/images/manual/opus_number.png" class="illustration" /><br />
<img alt="opus index" src="../resources/images/manual/opus_index.png" class="illustration" onload="this.width/=1.5;" />
<p>&#160;</p>
<p><b>Index of performance locations</b></p>
<p>A list of performance places (cities) is generated by the <a href="../modules/tools/performance_city_index.xq">performance_city_index.xq</a> script.<br />
<p>A list of performance places (cities) is generated by the <a href="../modules/tools/performance_city_index.xq?c=Demo">performance_city_index.xq</a> script.<br />
<img alt="opus index" src="../resources/images/manual/performance_place_index.png" class="illustration" onload="this.width/=1.5;" />
</p>
<p>&#160;</p>
<p><b>Index of item (source) locations</b></p>
<p>The script <a href="../modules/tools/location_index.xq">location_index.xq</a> generates
<p>The script <a href="../modules/tools/location_index.xq?c=Demo">location_index.xq</a> generates
a list of the locations of items (that is, manuscripts or specific copies of printed
sources).<br /> The script tries to resolve RISM sigla; locations not recognizable
as RISM sigla are reproduced as written.</p>
<p>&#160;</p>
<p><b>List of incipit graphics</b></p>
<p>For proofreading incipits added as graphics (images), a tool <a
href="../modules/tools/list_incipit_graphics.xq">list_incipit_graphics.xq</a> is
href="../modules/tools/list_incipit_graphics.xq?c=Demo">list_incipit_graphics.xq</a> is
available to output all graphics including the movement titles and file URLs. The
script does not (yet) include incipits to be rendered at runtime. <br/>
<img alt="incipit list" src="../resources/images/manual/list_incipit_graphics.png" class="illustration" onload="this.width/=1.5;" />
Expand Down
10 changes: 6 additions & 4 deletions modules/tools/batch_transform.xq
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
xquery version "1.0" encoding "UTF-8";

import module namespace login="http://kb.dk/this/login" at "../login.xqm";
import module namespace config="https://github.com/edirom/mermeid/config" at "../config.xqm";

declare namespace request="http://exist-db.org/xquery/request";
declare namespace response="http://exist-db.org/xquery/response";
declare namespace transform="http://exist-db.org/xquery/transform";
Expand All @@ -12,11 +9,16 @@ declare namespace util="http://exist-db.org/xquery/util";
declare namespace ft="http://exist-db.org/xquery/lucene";
declare namespace ht="http://exist-db.org/xquery/httpclient";
declare namespace xdb="http://exist-db.org/xquery/xmldb";
declare namespace output="http://www.w3.org/2010/xslt-xquery-serialization";

declare namespace local="http://kb.dk/this/app";
declare namespace m="http://www.music-encoding.org/ns/mei";

declare option exist:serialize "method=xml media-type=text/html";
import module namespace login="http://kb.dk/this/login" at "../login.xqm";
import module namespace config="https://github.com/edirom/mermeid/config" at "../config.xqm";

declare option output:method "xhtml5";
declare option output:media-type "text/html";

declare variable $coll := request:get-parameter("coll", "") cast as xs:string;
declare variable $query := request:get-parameter("query","") cast as xs:string;
Expand Down
8 changes: 5 additions & 3 deletions modules/tools/batch_transform_list.xq
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
xquery version "1.0" encoding "UTF-8";

import module namespace config="https://github.com/edirom/mermeid/config" at "../config.xqm";

declare namespace xl="http://www.w3.org/1999/xlink";
declare namespace request="http://exist-db.org/xquery/request";
declare namespace response="http://exist-db.org/xquery/response";
Expand All @@ -10,11 +8,15 @@ declare namespace file="http://exist-db.org/xquery/file";
declare namespace util="http://exist-db.org/xquery/util";
declare namespace ft="http://exist-db.org/xquery/lucene";
declare namespace ht="http://exist-db.org/xquery/httpclient";
declare namespace output="http://www.w3.org/2010/xslt-xquery-serialization";

declare namespace local="http://kb.dk/this/app";
declare namespace m="http://www.music-encoding.org/ns/mei";

declare option exist:serialize "method=xml media-type=text/html";
import module namespace config="https://github.com/edirom/mermeid/config" at "../config.xqm";

declare option output:method "xhtml5";
declare option output:media-type "text/html";

declare variable $coll := request:get-parameter("coll","") cast as xs:string;
declare variable $query := request:get-parameter("query","") cast as xs:string;
Expand Down
4 changes: 3 additions & 1 deletion modules/tools/batch_transform_settings.xq
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
xquery version "1.0" encoding "UTF-8";

declare namespace request="http://exist-db.org/xquery/request";
declare namespace output="http://www.w3.org/2010/xslt-xquery-serialization";

import module namespace config="https://github.com/edirom/mermeid/config" at "../config.xqm";

declare option exist:serialize "method=xml media-type=text/html;charset=UTF-8";
declare option output:method "xhtml5";
declare option output:media-type "text/html";

declare variable $coll := request:get-parameter("coll", "") cast as xs:string;
declare variable $query := request:get-parameter("query","") cast as xs:string;
Expand Down
20 changes: 15 additions & 5 deletions modules/tools/list_incipit_graphics.xq
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
xquery version "1.0" encoding "UTF-8";
xquery version "3.1" encoding "UTF-8";

(: Generates a list of incipits graphic files ordered by work (catalogue) number :)

Expand All @@ -9,13 +9,17 @@ declare namespace file="http://exist-db.org/xquery/file";
declare namespace util="http://exist-db.org/xquery/util";
declare namespace ft="http://exist-db.org/xquery/lucene";
declare namespace ht="http://exist-db.org/xquery/httpclient";
declare namespace output="http://www.w3.org/2010/xslt-xquery-serialization";

declare namespace local="http://kb.dk/this/app";
declare namespace m="http://www.music-encoding.org/ns/mei";

declare option exist:serialize "method=xml media-type=text/html";
import module namespace config="https://github.com/edirom/mermeid/config" at "../config.xqm";

declare variable $database := "/db/dcm";
declare option output:method "xhtml5";
declare option output:media-type "text/html";

declare variable $database := $config:data-root;
declare variable $collection := request:get-parameter("c","");
(: desired resolution; MerMEId supports values "lowres", "hires", and "print" :)
declare variable $resolution := request:get-parameter("res","lowres");
Expand Down Expand Up @@ -57,15 +61,21 @@ declare function local:movement($expression) as node()
{local:movement-title($expression)}<br/>
{
for $img at $pos in $expression/m:incip/m:graphic[@targettype=$resolution and @target!='']
let $img.graphic.target := $expression/m:incip/m:graphic[@targettype=$resolution][$pos]/@target
let $img.src :=
(: special treatment for relative links :)
if(starts-with($img.graphic.target, 'http') or starts-with($img.graphic.target, '//'))
then $img.graphic.target => string()
else $config:exist-endpoint || '/data/' || $img.graphic.target
return
<div style="margin-bottom:1em;">
{
element img {
attribute src {$expression/m:incip/m:graphic[@targettype=$resolution][$pos]/@target}
attribute src {$img.src}
}
}
<br/>
{string($expression/m:incip/m:graphic[@targettype=$resolution][$pos]/@target)}
{$img.graphic.target => string()}
</div>
}
{
Expand Down
8 changes: 6 additions & 2 deletions modules/tools/list_source_provenance.xq
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,17 @@ declare namespace file="http://exist-db.org/xquery/file";
declare namespace util="http://exist-db.org/xquery/util";
declare namespace ft="http://exist-db.org/xquery/lucene";
declare namespace ht="http://exist-db.org/xquery/httpclient";
declare namespace output="http://www.w3.org/2010/xslt-xquery-serialization";

declare namespace local="http://kb.dk/this/app";
declare namespace m="http://www.music-encoding.org/ns/mei";

declare option exist:serialize "method=xml media-type=text/html";
import module namespace config="https://github.com/edirom/mermeid/config" at "../config.xqm";

declare variable $database := "/db/dcm";
declare option output:method "xhtml5";
declare option output:media-type "text/html";

declare variable $database := $config:data-root;
declare variable $collection := request:get-parameter("c","");


Expand Down
8 changes: 6 additions & 2 deletions modules/tools/list_text_incipits.xq
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,17 @@ declare namespace file="http://exist-db.org/xquery/file";
declare namespace util="http://exist-db.org/xquery/util";
declare namespace ft="http://exist-db.org/xquery/lucene";
declare namespace ht="http://exist-db.org/xquery/httpclient";
declare namespace output="http://www.w3.org/2010/xslt-xquery-serialization";

declare namespace local="http://kb.dk/this/app";
declare namespace m="http://www.music-encoding.org/ns/mei";

declare option exist:serialize "method=xml media-type=text/html";
import module namespace config="https://github.com/edirom/mermeid/config" at "../config.xqm";

declare variable $database := "/db/dcm";
declare option output:method "xhtml5";
declare option output:media-type "text/html";

declare variable $database := $config:data-root;
declare variable $collection := request:get-parameter("c","");
(: desired resolution; MerMEId supports values "lowres", "hires", and "print" :)
declare variable $resolution := "lowres";
Expand Down
44 changes: 23 additions & 21 deletions modules/tools/location_index.xq
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,27 @@
xquery version "1.0" encoding "UTF-8";
xquery version "3.1" encoding "UTF-8";

declare namespace loop="http://kb.dk/this/getlist";

declare namespace request="http://exist-db.org/xquery/request";
declare namespace response="http://exist-db.org/xquery/response";
declare namespace fn="http://www.w3.org/2005/xpath-functions";
declare namespace file="http://exist-db.org/xquery/file";
declare namespace util="http://exist-db.org/xquery/util";
declare namespace ft="http://exist-db.org/xquery/lucene";
declare namespace ht="http://exist-db.org/xquery/httpclient";
(:declare namespace file="http://exist-db.org/xquery/file";:)
(:declare namespace util="http://exist-db.org/xquery/util";:)
(:declare namespace ft="http://exist-db.org/xquery/lucene";:)
(:declare namespace ht="http://exist-db.org/xquery/httpclient";:)
declare namespace marc="http://www.loc.gov/MARC21/slim";
declare namespace output="http://www.w3.org/2010/xslt-xquery-serialization";

declare namespace local="http://kb.dk/this/app";
(:declare namespace local="http://kb.dk/this/app";:)
declare namespace m="http://www.music-encoding.org/ns/mei";

declare option exist:serialize "method=xml media-type=text/html";
import module namespace config="https://github.com/edirom/mermeid/config" at "../config.xqm";

declare variable $database := "/db/dcm";
declare variable $rism_dir := "/db/rism_sigla";
declare option output:method "xhtml5";
declare option output:media-type "text/html";

declare variable $database := $config:data-root;
declare variable $rism_dir := $config:app-root || "/rism_sigla";
declare variable $country_codes := doc(concat($rism_dir,'/RISM_country_codes.xml'));
declare variable $collection := request:get-parameter("c","");

Expand All @@ -40,7 +44,6 @@ declare function loop:sort-key ($identifier as xs:string) as xs:string
declare function loop:lookup ($location as xs:string) as xs:string
{
let $country_code:=substring-before($location,'-')
let $filename:=concat($country_code,'.xml')
let $txt:=
if($country_codes/m:list/m:li/m:geogName[@codedval=$country_code])
then
Expand Down Expand Up @@ -82,24 +85,23 @@ declare function loop:lookup-archive ($location as xs:string, $country_code as x
<p>Please choose a file collection/catalogue by adding &apos;?c=[your collection name]&apos;
(for instance, ?c=CNW) to the URL</p>
else
for $c in distinct-values(
for $c in
collection($database)/m:mei/m:meiHead[m:fileDesc/m:seriesStmt/m:identifier[@type="file_collection"] = $collection]/
m:manifestationList/m:manifestation//m:physLoc/m:repository/
(m:identifier[@authority='RISM' and normalize-space(.)] | m:corpName[normalize-space(.)]) )
order by normalize-space(string($c))
(m:identifier[@auth='RISM' and normalize-space(.)] | m:corpName[normalize-space(.)])
group by $ident := normalize-space($c)
order by loop:lookup($ident)
return
<div>
{concat(loop:lookup($c),' &#160; ',$collection,' ')}
{concat(loop:lookup($ident),' &#160; ',$collection,' ')}
{let $numbers :=
for $n in collection($database)/m:mei/m:meiHead[m:fileDesc/m:seriesStmt/m:identifier[@type="file_collection"] = $collection]
where $n/m:manifestationList/m:manifestation//m:physLoc/m:repository/
(m:identifier[@authority='RISM' and normalize-space(.)] | m:corpName[normalize-space(.)]) = $c
order by loop:sort-key($n/m:workList/m:work/m:identifier[@label=$collection]/string())
return $n/m:workList/m:work/m:identifier[@label=$collection]/string()
return string-join($numbers,', ')
for $n in $c
let $number := $n/ancestor::m:meiHead/m:workList/m:work/m:identifier[@label=$collection] => normalize-space()
order by loop:sort-key($number)
return $number
return $numbers => distinct-values() => string-join(', ')
}
</div>

}
</div>

Expand Down
8 changes: 6 additions & 2 deletions modules/tools/name_index.xq
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,17 @@ declare namespace file="http://exist-db.org/xquery/file";
declare namespace util="http://exist-db.org/xquery/util";
declare namespace ft="http://exist-db.org/xquery/lucene";
declare namespace ht="http://exist-db.org/xquery/httpclient";
declare namespace output="http://www.w3.org/2010/xslt-xquery-serialization";

declare namespace local="http://kb.dk/this/app";
declare namespace m="http://www.music-encoding.org/ns/mei";

declare option exist:serialize "method=xml media-type=text/html";
import module namespace config="https://github.com/edirom/mermeid/config" at "../config.xqm";

declare variable $database := "/db/dcm";
declare option output:method "xhtml5";
declare option output:media-type "text/html";

declare variable $database := $config:data-root;
declare variable $collection := request:get-parameter("c","");


Expand Down
8 changes: 6 additions & 2 deletions modules/tools/name_index_step1.xq
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,17 @@ declare namespace file="http://exist-db.org/xquery/file";
declare namespace util="http://exist-db.org/xquery/util";
declare namespace ft="http://exist-db.org/xquery/lucene";
declare namespace ht="http://exist-db.org/xquery/httpclient";
declare namespace output="http://www.w3.org/2010/xslt-xquery-serialization";

declare namespace local="http://kb.dk/this/app";
declare namespace m="http://www.music-encoding.org/ns/mei";

declare option exist:serialize "method=xml media-type=text/html";
import module namespace config="https://github.com/edirom/mermeid/config" at "../config.xqm";

declare variable $database := "/db/dcm";
declare option output:method "xhtml5";
declare option output:media-type "text/html";

declare variable $database := $config:data-root;
declare variable $collection := request:get-parameter("c","");


Expand Down
Loading