[Enhancement] Lake compaction scheduler optimize in fe restart scenarios

[wxl24life]

Why I'm doing:

What I'm doing:

Fixes #54883

What type of PR is this:

• BugFix
• Feature
• Enhancement
• Refactor
• UT
• Doc
• Tool

Does this PR entail a change in behavior?

• Yes, this PR will result in a change in behavior.
• No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

• Interface/UI changes: syntax, type conversion, expression evaluation, display information
• Parameter changes: default values, similar parameters but with different default values
• Policy changes: use new policy to replace old one, functionality automatically enabled
• Feature removed
• Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

• I have added test cases for my bug fix or my new feature
• This pr needs user documentation (for new or modified features or behaviors)
  • I have added documentation for my new feature or new function
• This is a backport pr

Bugfix cherry-pick branch check:

• I have checked the version labels which the pr will be auto-backported to the target branch
  • 3.4
  • 3.3
  • 3.2
  • 3.1
  • 3.0

[starrocks-cr]

The most risky bug in this code is:
Potential deadlock due to holding a read lock while performing long-running operations (e.g., sorting/stream operations) which might inadvertently acquire further locks.

You can modify the code like this:

public List<TransactionState> getLakeCompactionCommittedTxnList() {
    List<TransactionState> committedTransactionStates;
    readLock();
    try {
        // Collect transactions while holding the read lock
        committedTransactionStates = idToRunningTransactionState.values().stream()
                .filter(transactionState -> (transactionState.getTransactionStatus() ==
                        TransactionStatus.COMMITTED))
                .filter(transactionState -> transactionState.getSourceType() ==
                        TransactionState.LoadJobSourceType.LAKE_COMPACTION)
                .collect(Collectors.toList());
    } finally {
        readUnlock();
    }
    
    // Perform sorting outside of the lock to reduce risk of deadlocks
    return committedTransactionStates.stream()
            .sorted(Comparator.comparing(TransactionState::getCommitTime))
            .collect(Collectors.toList());
}

[starrocks-cr]

The most risky bug in this code is:
Potential deadlock due to improper lock acquisition and release pattern.

You can modify the code like this:

protected Map<PartitionIdentifier, CompactionJob> rebuildRunningCompactionsOnRestart() {
    Map<PartitionIdentifier, CompactionJob> runningCompactions = new ConcurrentHashMap<>();
    List<Long> activeTxnIds =
            GlobalStateMgr.getCurrentState().getGlobalTransactionMgr().getLakeCompactionActiveTxnIds();
    for (PartitionStatistics statistics : partitionStatisticsHashMap.values()) {
        PartitionIdentifier partitionIdentifier = statistics.getPartition();
        long compactionTxnId = statistics.getLastCompactionTxnId();
        if (!activeTxnIds.contains(compactionTxnId)) {
            continue;
        }
        Database db = GlobalStateMgr.getCurrentState().getDb(partitionIdentifier.getDbId());
        if (db == null) {
            continue;
        }
        
        // Ensure that locker is only created if necessary
        Locker locker = new Locker();

        try {
            locker.lockDatabase(db, LockType.READ);
            
            OlapTable table = (OlapTable) db.getTable(partitionIdentifier.getTableId());
            if (table == null) {
                continue;
            }
            PhysicalPartition partition = table.getPhysicalPartition(partitionIdentifier.getPartitionId());
            if (partition == null) {
                continue;
            }
            CompactionJob job = new CompactionJob(db, table, partition, compactionTxnId,
                    Config.lake_compaction_allow_partial_success);
            runningCompactions.put(partitionIdentifier, job);
        } catch (Exception e) {
            LOG.error("Rebuild running compactions on fe restart failed: {}", e.getMessage(), e);
        } finally {
            // Ensure that unlock is called safely without overwriting with a new locker
            try {
                locker.unLockDatabase(db, LockType.READ);
            } catch (Exception e) {
                LOG.error("Failed to unlock database: {}", e.getMessage(), e);
            }
        }
    }
    return runningCompactions;
}

In the suggested fix, we ensure that any failures during unlocking are also logged, minimizing the risk of issues being swallowed silently. Additionally, ensuring that the locker is consistently used aids in maintaining robust locking semantics.

[sonarqubecloud]

Quality Gate passed

Issues
6 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[github-actions]

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

[github-actions]

[BE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)