fixes #409: Prevent loading all archived messages in memory when inde…

…xing

When iterating over all rows of a potentially large table, ensure that the database result set is configured to be 'linear' (forward-only and read-only).

This gives a better chance of the database driver to release all rows that have been iterated over, which prevents the fetch buffer to eventually include all rows (and potentially cause out-of-memory issues).

changelog.html

@@ -48,6 +48,7 @@ <h1>
 <ul>
     <li>Requires Openfire 5.0.0</li>
     <li>Note: Chat rooms can, in certain circumstances, be destroyed, and then be recreated using the same name. For data recorded by previous versions of this plugin, the plugin will associate all room history with the latest 'reincarnation' of a room. Starting with this version, new data will be associated to the correct 'incarnation' of the room.</li>
+    <li>[<a href='https://github.com/igniterealtime/openfire-monitoring-plugin/issues/409'>Issue #409</a>] - Building Lucene index should not load all messages in memory</li>
     <li>[<a href='https://github.com/igniterealtime/openfire-monitoring-plugin/issues/401'>Issue #401</a>] - Fixes: Update Jersey from 2.35 to 2.45</li>
     <li>[<a href='https://github.com/igniterealtime/openfire-monitoring-plugin/issues/398'>Issue #398</a>] - Fixes: Missing translation for system property</li>
     <li>[<a href='https://github.com/igniterealtime/openfire-monitoring-plugin/issues/392'>Issue #392</a>] - Fixes: Compatibility issue with Openfire 5.0.0</li>

src/java/com/reucon/openfire/plugin/archive/impl/MucIndexer.java

@@ -134,16 +134,21 @@ private Instant indexMUCMessages( IndexWriter writer, Instant since )
             // The entire process took under 8 seconds.
             // Preventing the driver to collect all results at once depends on auto-commit from being disabled, at
             // least for postgres. Getting a 'transaction' connection will ensure this (if supported).
+            // MSSQL differentiates between client-cursored and server-cursored result sets. For server-cursored result
+            // sets, the fetch buffer and scroll window are the same size (as opposed to fetch buffer containing all
+            // the rows). To hint that a server-cursored result set is desired, it should be configured to be 'forward
+            // only' as well as 'read only'.
             con = DbConnectionManager.getTransactionConnection();
 
             if ( since.equals( Instant.EPOCH ) ) {
-                pstmt = con.prepareStatement(ALL_MUC_MESSAGES);
+                pstmt = con.prepareStatement(ALL_MUC_MESSAGES, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
             } else {
-                pstmt = con.prepareStatement(NEW_MUC_MESSAGES);
+                pstmt = con.prepareStatement(NEW_MUC_MESSAGES, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
                 pstmt.setString(1, StringUtils.dateToMillis(Date.from(since))); // This mimics org.jivesoftware.openfire.muc.spi.MUCPersistenceManager.saveConversationLogBatch
             }
 
             pstmt.setFetchSize(250);
+            pstmt.setFetchDirection(ResultSet.FETCH_FORWARD);
             rs = pstmt.executeQuery();
 
             long progress = 0;