can we connect same indexer to different opengrok web instances

[jkjha]

I have a prod application running with huge indexer size on URL example-prod.com. Now I wish to setup a staging instance using same data. Any tips on how I can connect to same indexer from different web URL example-uat.com?

[vladak]

I assume this is not a one time thing, i.e. basically you'd like to have a mirror of the production instance. I've been asked to set this up for the internal production instance I maintain, it's just there was no time to pursue this, however in general it should work.

There are multiple ways how to approach this. I assume the machines have separate storage.

The first way would be to perform initial transfer of the production data (source + data root and fresh configuration retrieved from the webapp) to the staging machine and let it run mirroring+indexing periodically. In this setup, there would be two independent machines, so any change done to the production would have to be also applied to the staging. The downside is that each problem with indexing/mirroring would have to be solved twice.

The second way is to perform the synchronization periodically (can use file system snapshots and incremental transfers, if your file system supports that) and then set the configuration to the web app via API call. This should trigger index reader refresh and suggester rebuild. In this setup the staging will not run mirroring+indexer at all.

[jkjha]

Thanks @vladak for the reply. As I understand, I need to set up a mirror indexer and connect my stage/uat instance to that. However, is this possible to connect to same indexer data from 2 different instances (For ex. connecting to same Mysql DB from different clients). May be my question sounds stupid but thats something I am trying to figure out (near goal) after recently getting the ownership to maintain.

[vladak]

The indexer data is all file based so there is no notion of connecting to a index instance like there is with classical databases. Either it is necessary to synchronize the files with the usual means (rsync, backup/restore, ZFS send/recv, block replication etc.) or (this would be third option, complementing the 2 above) you would need to serve the data from the production instance via some sort of network file system (NFS, AFS, etc.) to make them available to the staging instance. The indexing process would then have to be adapted to poke both web apps to refresh the index readers, reload configuration, rebuild suggesters etc. The main question though is if and how would this scheme work for the Lucene indexes as these are usually mapped into memory (via mmap() on Unix-like systems) by the web application.