[Enhancement]: Optimize task scheduling logic to enable concurrent execution of tasks

[xiaocai2333]

Is there an existing issue for this?

• I have searched the existing issues

What would you like to be added?

Optimize task scheduling logic to enable concurrent execution of tasks.

Why is this needed?

Stats tasks are primarily io operations, which can be executed concurrently.
Additionally, when segments are relatively small, index tasks can also be executed concurrently.

Anything else?

No response

[xiaocai2333]

Configure slots for tasks and IndexNode. By default, the slot count for IndexNode is 16, and the slot count for index tasks is also 16. This means that under normal circumstances, an IndexNode can execute only one index task at a time. The slot count for stats tasks is 4, meaning an IndexNode can execute up to 4 stats tasks concurrently.

[xiaofan-luan]

I think indexNode slot number should be related to cpu cores and memory?
by default 2core 8GB memory is a slot.

A index node has 16core 64GB is 8 slot, 8 core 64GB is 4 slot.

For Segment > 500MB, Each index task takes 8 slots
For Segment between 100MB - 500MB, each index task takes 4 slots.
For Segment between 10MB - 100MB, each index task takes 2 lots.
For Segment smaller than 10MB, each index task takes 1 slots.

On the other side, For Segment > 500MB, each index takes 2 slots.
For segment < 500MB, each index takes 1 slots.

If a task request is larger than slot number, then any index task can take this task and can take only one task at then same time.

please comment @xiaocai2333 @czs007