This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 857201. The European Commission assumes no responsibility for any content of this repository.

Prefetcher

Requests content from a caching server in order to make the caching server cache the requested content.

What is this?

The Prefetcher service is part of the platform for media caching on trains. The Prefetcher requests from the Online Cache content which is reported to be missing in the Online Cache. As the Online Cache operates in reverse proxy mode, it will cache the requested content. Subsequent request to the content will be responded to with the cached content. Missing state is reported by the Cache Monitor to the State API from where the Prefetcher retrieves a list of the missing videos.

How does it work?

The below diagram illustrates the main software modules of the Prefetcher. The core logic module controls the service processes. atVia the Message Streamer Client it sets listeners for messages that inform of new cache states as well as of new configurations of the Aggregator.

When the core logic module receives the message that a new cache state has been initialised, it loads information about the missing media items from the State API via the HTTP Client. From the media items' resource locations, the core logic module creates a Request List (implemented in request-map.ts). The Request List module sends requests via the HTTP Client module. For received responses it evaluates the HTTP status codes. It counts how often a certain status code has occurred for a certain request. If a limit value is exceeded, the request is deleted from the list. Limit values can be configured in a rule set. For example, it is useful to configure that 404 should only occur once, because it is unlikely that the resource will suddenly be available at the corresponding address on the next request. For a 500 status code, however, it can be useful to allow multiple attempts. Requests with a response status that indicates success are also deleted from the Request List. If the Request List is empty, the Core Logic module again queries the missing media items from the State API. This is necessary because even if all requests of the prefetcher were answered with success indicating status codes like 200, corresponding media items still might be missing in the cache, for example because the download failed or because the media item was overwritten by a more recent entry in the cache. With the received media items, the core logic module instantiates a new Request List. The below figure shows this process as a sequence diagram.

If the core logic module receives a message that a new aggregator configuration is present, the fetch cycle described above is interrupted. Only when the notification about the newly initiated cache state is sent, a new fetch cycle starts.

Install, build, run

Note: Typically you would use the up.sh script from the Platform project to install, build and run this service as part of a composite of docker services. Read on if you intend to run the service directly on your host system.

Prerequestits: Following software needs to be installed on your host machine in order to execute the subsequent steps.

• Node.js
• NPM

First, git clone this project and change into its root directory. Than run the following command to install its dependencies:

$ npm install

You can than run the service in three different modes.

# development
$ npm run start

# watch mode
$ npm run start:dev

# production mode
$ npm run start:prod

With following command you can build a docker image for this service. But again, typically you use the startup script up.sh of the Platform project to do the job.

$ docker build -t 5gv-prefetcher .

Modify the sequence diagram

The sequence diagram in this Doc has been build with mermaid.js and its description language for sequence diagrams. To modify the diagram you need to install the npm depencies and render the diagram with the mermaid cli tool mmdc, which will be installed as a local npm dev dependency and can be run with npx:

$ npm install
$ npx mmdc -i doc/diagrams/fetch-cycle.mmd -o doc/images/fetch-cycle.svg