Draining Steps

1. Tell WorkQueueManager to not accept anymore work blocks:

open config file at:
    /data/srv/wmagent/current/config/wmagent/config.py

Look for the workqueue params
    config.WorkQueueManager.queueParams = {'ParentQueueCouchUrl': 'https://cmsweb.cern.ch/couchdb/workqueue'}

Add the 'DrainMode': True to the params.
    config.WorkQueueManager.queueParams = {'ParentQueueCouchUrl': 'https://cmsweb.cern.ch/couchdb/workqueue', 'DrainMode': True}

save the changes

restart WorkQueueManager, AnalyticsDataCollector and AgentStatusWatcher. (We can change the code to automatically pick up the config change #7994)
    $manage execute-agent wmcoreD --restart --components=WorkQueueManager,AnalyticsDataCollector,AgentStatusWatcher

Undo these changes if you want to set the agent out of drain.

2. check all the stuck job condition (site is not available, etc) and apply proper procedure. (Alan, could you add what need to be done.

3. check all the workflow is the finished state.(#7493) Ref

   1. all the workflow is completed in the agent (no subscription left)
   2. all the blocks are closed.
   3. all the files are injected to PhEDEx
   4. all the files are uploaded to DBS

4. report to WMStats (#7493)