Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: Add resource management topic and split docs #20010

Merged
merged 27 commits into from
Feb 7, 2025
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
7374391
*: Add resource management topic and split docs
nolouch Jan 15, 2025
ef39b08
update
nolouch Jan 15, 2025
fa06931
update
nolouch Jan 15, 2025
8ef4b25
Update tidb-resource-control-ru-groups.md
hfxsd Feb 7, 2025
07e0f33
Update TOC.md
nolouch Feb 7, 2025
2615167
Update faq/sql-faq.md
nolouch Feb 7, 2025
359f834
Update functions-and-operators/tidb-functions.md
nolouch Feb 7, 2025
b15c9ee
Update information-schema/information-schema-resource-groups.md
nolouch Feb 7, 2025
8dc8eee
Update tidb-resource-control-background-tasks.md
nolouch Feb 7, 2025
1d4095a
Update tidb-resource-control-background-tasks.md
nolouch Feb 7, 2025
5595fcd
Update tidb-resource-control-ru-groups.md
nolouch Feb 7, 2025
cbbf7f1
Update tidb-resource-control-ru-groups.md
nolouch Feb 7, 2025
3e43c49
Update tidb-resource-control-ru-groups.md
nolouch Feb 7, 2025
1c3d33f
Update tidb-resource-control-runaway-queries.md
nolouch Feb 7, 2025
0afdc6a
Update tidb-resource-control-runaway-queries.md
nolouch Feb 7, 2025
9e9f9bc
Update tiflash/tiflash-pipeline-model.md
nolouch Feb 7, 2025
cee61e3
Update tidb-resource-control-runaway-queries.md
nolouch Feb 7, 2025
45f9886
Update tikv-configuration-file.md
nolouch Feb 7, 2025
35ed4ae
Update user-account-management.md
nolouch Feb 7, 2025
b434893
Update functions-and-operators/tidb-functions.md
nolouch Feb 7, 2025
e3bfb1d
Merge remote-tracking branch 'upstream/master' into pr/20010
hfxsd Feb 7, 2025
49272ee
Merge branch 'split-docs' of https://github.com/nolouch/docs into pr/…
hfxsd Feb 7, 2025
34f09a8
Update system-variable-reference.md
hfxsd Feb 7, 2025
dbcbd7a
Update system-variable-reference.md
hfxsd Feb 7, 2025
1a7a01a
Update TOC-tidb-cloud.md
hfxsd Feb 7, 2025
8b6a84b
Update sql-statement-calibrate-resource.md
hfxsd Feb 7, 2025
58e8050
Apply suggestions from code review
hfxsd Feb 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion TOC-tidb-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -736,7 +736,7 @@
- [update](/tidb-cloud/ticloud-update.md)
- [help](/tidb-cloud/ticloud-help.md)
- [Table Filter](/table-filter.md)
- [Resource Control](/tidb-resource-control.md)
- [Resource Control](/tidb-resource-control-ru-groups.md)
hfxsd marked this conversation as resolved.
Show resolved Hide resolved
- [URI Formats of External Storage Services](/external-storage-uri.md)
- [DDL Execution Principles and Best Practices](/ddl-introduction.md)
- [Batch Processing](/batch-processing.md)
Expand Down
5 changes: 4 additions & 1 deletion TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,7 +262,10 @@
- [Primary-Secondary DR](/dr-secondary-cluster.md)
- [Multi-Replica Cluster DR](/dr-multi-replica.md)
- [BR-based DR](/dr-backup-restore.md)
- [Resource Control](/tidb-resource-control.md)
- Resource Manager
- [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md)
- [Manage Runaway Queries](/tidb-resource-control-runaway-queries.md)
- [Manage Background Tasks](/tidb-resource-control-background-tasks.md)
- [Configure Time Zone](/configure-time-zone.md)
- [Daily Checklist](/daily-check.md)
- [Maintain TiFlash](/tiflash/maintain-tiflash.md)
Expand Down
6 changes: 3 additions & 3 deletions basic-features.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,9 +258,9 @@ You can try out TiDB features on [TiDB Playground](https://play.tidbcloud.com/?u
| [Global memory control](/configure-memory-usage.md#configure-the-memory-usage-threshold-of-a-tidb-server-instance) | Y | Y | Y | Y | Y | N | N | N | N | N |
| [Cross-cluster RawKV replication](/tikv-configuration-file.md#api-version-new-in-v610) | E | E | E | E | E | N | N | N | N | N |
| [Green GC](/system-variables.md#tidb_gc_scan_lock_mode-new-in-v50) | E | E | E | E | E | E | E | E | E | E |
| [Resource control](/tidb-resource-control.md) | Y | Y | Y | Y | N | N | N | N | N | N |
| [Runaway Queries management](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries) | Y | Y | E | N | N | N | N | N | N | N |
| [Background tasks management](/tidb-resource-control.md#manage-background-tasks) | E | E | E | N | N | N | N | N | N | N |
| [Resource control](/tidb-resource-control-ru-groups.md) | Y | Y | Y | Y | N | N | N | N | N | N |
| [Runaway Queries management](/tidb-resource-control-runaway-queries.md) | Y | Y | E | N | N | N | N | N | N | N |
| [Background tasks management](/tidb-resource-control-background-tasks.md) | E | E | E | N | N | N | N | N | N | N |
| [TiFlash Disaggregated Storage and Compute Architecture and S3 Support](/tiflash/tiflash-disaggregated-and-s3.md) | Y | Y | Y | E | N | N | N | N | N | N |
| [Selecting TiDB nodes for the Distributed eXecution Framework (DXF) tasks](/system-variables.md#tidb_service_scope-new-in-v740) | Y | Y | N | N | N | N | N | N | N |
| PD Follower Proxy (controlled by [`tidb_enable_tso_follower_proxy`](/system-variables.md#tidb_enable_tso_follower_proxy-new-in-v530)) | Y | Y | Y | Y | Y | Y | Y | Y | N | N |
Expand Down
2 changes: 1 addition & 1 deletion dashboard/dashboard-intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ See [Search Logs Page](/dashboard/dashboard-log-search.md) for details.

## Estimate cluster capacity for resource control

To implement resource isolation using the [Resource Control](/tidb-resource-control.md) feature, cluster administrators can create resource groups and set quotas for each group.
To implement resource isolation using the [Resource Control](/tidb-resource-control-ru-groups.md) feature, cluster administrators can create resource groups and set quotas for each group.

Before resource planning, you need to know the overall capacity of the cluster. For more details, see [Resource Manager page](/dashboard/dashboard-resource-manager.md).

Expand Down
4 changes: 2 additions & 2 deletions dashboard/dashboard-resource-manager.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ summary: TiDB Dashboard Resource Manager Page helps cluster administrators imple

# TiDB Dashboard Resource Manager Page

To implement resource isolation using the [Resource Control](/tidb-resource-control.md) feature, cluster administrators can create resource groups and set quotas for each group. Before resource planning, you need to know the overall capacity of the cluster. This document helps you view the information about resource control, so you can estimate the cluster capacity before resource planning and allocate resources more effectively.
To implement resource isolation using the [Resource Control](/tidb-resource-control-ru-groups.md) feature, cluster administrators can create resource groups and set quotas for each group. Before resource planning, you need to know the overall capacity of the cluster. This document helps you view the information about resource control, so you can estimate the cluster capacity before resource planning and allocate resources more effectively.

## Access the page

Expand Down Expand Up @@ -34,7 +34,7 @@ The Resource Manager page contains the following three sections:

## Estimate Capacity

Before resource planning, you need to know the overall capacity of the cluster. TiDB provides two methods to estimate the capacity of [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru#what-is-request-unit-ru) in the current cluster:
Before resource planning, you need to know the overall capacity of the cluster. TiDB provides two methods to estimate the capacity of [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru#what-is-request-unit-ru) in the current cluster:

- [Estimate capacity based on hardware deployment](/sql-statements/sql-statement-calibrate-resource.md#estimate-capacity-based-on-hardware-deployment)

Expand Down
6 changes: 3 additions & 3 deletions error-codes.md
Original file line number Diff line number Diff line change
Expand Up @@ -484,7 +484,7 @@ TiDB is compatible with the error codes in MySQL, and in most cases returns the

* Error Number: 8249

The resource group does not exist. This error is returned when you modify or bind a resource group that does not exist. See [Create a resource group](/tidb-resource-control.md#create-a-resource-group).
The resource group does not exist. This error is returned when you modify or bind a resource group that does not exist. See [Create a resource group](/tidb-resource-control-ru-groups.md#create-a-resource-group).

* Error Number: 8250

Expand All @@ -508,11 +508,11 @@ TiDB is compatible with the error codes in MySQL, and in most cases returns the

* Error Number: 8253

The query stops because it meets the condition of a runaway query. See [Runaway Queries](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries).
The query stops because it meets the condition of a runaway query. See [Runaway Queries](/tidb-resource-control-runaway-queries.md).

* Error Number: 8254

The query stops because it meets the quarantined watch condition of a runaway query. See [Runaway Queries](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries).
The query stops because it meets the quarantined watch condition of a runaway query. See [Runaway Queries](/tidb-resource-control-runaway-queries.md).

* Error Number: 8260

Expand Down
4 changes: 2 additions & 2 deletions faq/sql-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ In addition, you can also use the [SQL binding](/sql-plan-management.md#sql-bind

## How to prevent the execution of a particular SQL statement?

For TiDB v7.5.0 or later versions, you can use the [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md) statement to terminate specific SQL statements. For more details, see [Manage queries that consume more resources than expected (Runaway Queries)](/tidb-resource-control.md#query-watch-parameters).
For TiDB v7.5.0 or later versions, you can use the [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md) statement to terminate specific SQL statements. For more details, see [Manage queries that consume more resources than expected (Runaway Queries)](/tidb-resource-control-runaway-queries.md#query-watch-parameters).

For versions earlier than TiDB v7.5.0, you can create [SQL bindings](/sql-plan-management.md#sql-binding) with the [`MAX_EXECUTION_TIME`](/optimizer-hints.md#max_execution_timen) hint to limit the execution time of a particular statement to a small value (for example, 1ms). In this way, the statement is terminated automatically by the threshold.

Expand Down Expand Up @@ -211,7 +211,7 @@ TiDB supports changing the priority on a [global](/system-variables.md#tidb_forc

> **Note:**
>
> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control.md) to manage resource usage for different SQL statements.
> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control-ru-groups.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority will no longer take effect. It is recommended that you use Resource Control to manage resource usage for different SQL statements.

You can combine the above two parameters with the DML of TiDB to use them. For example:

Expand Down
6 changes: 3 additions & 3 deletions functions-and-operators/tidb-functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The following functions are TiDB extensions, and are not present in MySQL:

| Function name | Function description |
| :-------------- | :------------------------------------- |
| [`CURRENT_RESOURCE_GROUP()`](#current_resource_group) | Returns the name of the resource group that the current session is bound to. See [using resource control to achieve resource isolation](/tidb-resource-control.md). |
| [`CURRENT_RESOURCE_GROUP()`](#current_resource_group) | Returns the name of the resource group that the current session is bound to. See [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md). |
| [`TIDB_BOUNDED_STALENESS()`](#tidb_bounded_staleness) | Instructs TiDB to read the most recent data within a specified time range. See [reading historical data using the `AS OF TIMESTAMP` clause](/as-of-timestamp.md). |
| [`TIDB_CURRENT_TSO()`](#tidb_current_tso) | Returns the current [TimeStamp Oracle (TSO) in TiDB](/tso.md). |
| [`TIDB_DECODE_BINARY_PLAN()`](#tidb_decode_binary_plan) | Decodes binary plans. |
Expand All @@ -36,7 +36,7 @@ The following functions are TiDB extensions, and are not present in MySQL:

| Function name | Function description |
| :-------------- | :------------------------------------- |
| [`CURRENT_RESOURCE_GROUP()`](#current_resource_group) | Returns the resource group name that the current session is bound to. See [using resource control to achieve resource isolation](/tidb-resource-control.md). |
| [`CURRENT_RESOURCE_GROUP()`](#current_resource_group) | Returns the resource group name that the current session is bound to. See [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md). |
| [`TIDB_BOUNDED_STALENESS()`](#tidb_bounded_staleness) | Instructs TiDB to read most recent data within a specified time range. See [reading historical data using the `AS OF TIMESTAMP` clause](/as-of-timestamp.md). |
| [`TIDB_CURRENT_TSO()`](#tidb_current_tso) | Returns the current [TimeStamp Oracle (TSO) in TiDB](/tso.md). |
| [`TIDB_DECODE_BINARY_PLAN()`](#tidb_decode_binary_plan) | Decodes binary plans. |
Expand All @@ -59,7 +59,7 @@ The following functions are TiDB extensions, and are not present in MySQL:

## CURRENT_RESOURCE_GROUP

The `CURRENT_RESOURCE_GROUP()` function is used to show the resource group name that the current session is bound to. When the [Resource control](/tidb-resource-control.md) feature is enabled, the available resources that can be used by SQL statements are restricted by the resource quota of the bound resource group.
The `CURRENT_RESOURCE_GROUP()` function is used to show the resource group name that the current session is bound to. When the [Resource control](/tidb-resource-control-ru-groups.md) feature is enabled, the available resources that can be used by SQL statements are restricted by the resource quota of the bound resource group.

When a session is established, TiDB binds the session to the resource group that the login user is bound to by default. If the user is not bound to any resource groups, the session is bound to the `default` resource group. Once the session is established, the bound resource group will not change by default, even if the bound resource group of the user is changed via [modifying the resource group bound to the user](/sql-statements/sql-statement-alter-user.md#modify-basic-user-information). To change the bound resource group of the current session, you can use [`SET RESOURCE GROUP`](/sql-statements/sql-statement-set-resource-group.md).

Expand Down
2 changes: 1 addition & 1 deletion glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ Remote Procedure Call (RPC) is a communication way between software components.

### Request Unit (RU)

Request Unit (RU) is a unified abstraction unit for the resource usage in TiDB. It is used with [Resource Control](/tidb-resource-control.md) to manage resource usage.
Request Unit (RU) is a unified abstraction unit for the resource usage in TiDB. It is used with [Resource Control](/tidb-resource-control-ru-groups.md) to manage resource usage.

### Restore

Expand Down
4 changes: 2 additions & 2 deletions grafana-resource-control-dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,15 @@ If you use TiUP to deploy the TiDB cluster, the monitoring system (Prometheus &

The Grafana dashboard is divided into a series of sub dashboards which include Overview, PD, TiDB, TiKV, Node\_exporter, Disk Performance, and Performance\_overview.

If your cluster has used the [Resource Control](/tidb-resource-control.md) feature, you can get an overview of the resource consumption status from the Resource Control dashboard.
If your cluster has used the [Resource Control](/tidb-resource-control-ru-groups.md) feature, you can get an overview of the resource consumption status from the Resource Control dashboard.

TiDB uses the [token bucket algorithm](https://en.wikipedia.org/wiki/Token_bucket) for flow control. As described in the [RFC: Global Resource Control in TiDB](https://github.com/pingcap/tidb/blob/master/docs/design/2022-11-25-global-resource-control.md#distributed-token-buckets), a TiDB node might have multiple Resource Groups, which are flow controlled by GAC (Global Admission Control) on the PD side. The Local Token Buckets in each TiDB node periodically (5 seconds by default) communicate with the GAC on the PD side to reconfigure the local tokens. In TiDB, the Local Token Buckets are implemented as Resource Controller Clients.

This document describes some key monitoring metrics displayed on the Resource Control dashboard.

## Metrics about Request Unit

- RU: the [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) consumption information of each resource group, calculated in real time. `total` is the sum of the Request Units consumed by all Resource Groups. The Request Unit consumption of each resource group should be equal to the sum of its read consumption (Read Request Unit) and write consumption (Write Request Unit).
- RU: the [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) consumption information of each resource group, calculated in real time. `total` is the sum of the Request Units consumed by all Resource Groups. The Request Unit consumption of each resource group should be equal to the sum of its read consumption (Read Request Unit) and write consumption (Write Request Unit).
- RU Per Query: the average number of Request Units consumed by each SQL statement per second. It is obtained by dividing the above RU metric by the number of SQL statements executed per second.
- RRU: the Read Request Unit consumption information of each resource group, calculated in real time. `total` is the sum of the Read Request Units consumed by all Resource Groups.
- RRU Per Query: the average number of Read Request Units consumed by each SQL statement per second. It is obtained by dividing the above RRU metric by the number of SQL statements executed per second.
Expand Down
4 changes: 2 additions & 2 deletions information-schema/information-schema-resource-groups.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ summary: Learn the `RESOURCE_GROUPS` information_schema table.

# RESOURCE_GROUPS

The `RESOURCE_GROUPS` table shows the information about all resource groups. For more information, see [Use Resource Control to Achieve Resource Isolation](/tidb-resource-control.md).
The `RESOURCE_GROUPS` table shows the information about all resource groups. For more information, see [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md).

> **Note:**
>
Expand Down Expand Up @@ -79,7 +79,7 @@ SELECT * FROM information_schema.resource_groups WHERE NAME = 'rg1'; -- View the
The descriptions of the columns in the `RESOURCE_GROUPS` table are as follows:

* `NAME`: the name of the resource group.
* `RU_PER_SEC`: the backfilling speed of the resource group. The unit is RU/second, in which RU means [Request Unit](/tidb-resource-control.md#what-is-request-unit-ru).
* `RU_PER_SEC`: the backfilling speed of the resource group. The unit is RU/second, in which RU means [Request Unit](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru).
* `PRIORITY`: the absolute priority of tasks to be processed on TiKV. Different resources are scheduled according to the `PRIORITY` setting. Tasks with high `PRIORITY` are scheduled first. For resource groups with the same `PRIORITY`, tasks will be scheduled proportionally according to the `RU_PER_SEC` configuration. If `PRIORITY` is not specified, the default priority is `MEDIUM`.
* `BURSTABLE`: whether to allow the resource group to overuse the available system resources.

Expand Down
2 changes: 1 addition & 1 deletion information-schema/information-schema-runaway-watches.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ summary: Learn the `RUNAWAY_WATCHES` INFORMATION_SCHEMA table.

# RUNAWAY_WATCHES

The `RUNAWAY_WATCHES` table shows the watch list of runaway queries that consume more resources than expected. For more information, see [Runaway Queries](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries).
The `RUNAWAY_WATCHES` table shows the watch list of runaway queries that consume more resources than expected. For more information, see [Runaway Queries](/tidb-resource-control-runaway-queries.md).

> **Note:**
>
Expand Down
2 changes: 1 addition & 1 deletion optimizer-hints.md
Original file line number Diff line number Diff line change
Expand Up @@ -851,7 +851,7 @@ SELECT /*+ NTH_PLAN(3) */ count(*) from t where a > 5;

### RESOURCE_GROUP(resource_group_name)

`RESOURCE_GROUP(resource_group_name)` is used for [Resource Control](/tidb-resource-control.md) to isolate resources. This hint temporarily executes the current statement using the specified resource group. If the specified resource group does not exist, this hint will be ignored.
`RESOURCE_GROUP(resource_group_name)` is used for [Resource Control](/tidb-resource-control-ru-groups.md) to isolate resources. This hint temporarily executes the current statement using the specified resource group. If the specified resource group does not exist, this hint will be ignored.

Example:

Expand Down
4 changes: 2 additions & 2 deletions pd-configuration-file.md
Original file line number Diff line number Diff line change
Expand Up @@ -503,7 +503,7 @@ Configuration items related to the replication mode of all Regions. See [Enable

## controller

This section describes the configuration items that are built into PD for [Resource Control](/tidb-resource-control.md).
This section describes the configuration items that are built into PD for [Resource Control](/tidb-resource-control-ru-groups.md).

### `degraded-mode-wait-duration`

Expand All @@ -513,7 +513,7 @@ This section describes the configuration items that are built into PD for [Resou

### `request-unit`

The following are the configuration items about the [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru).
The following are the configuration items about the [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru).

#### `read-base-cost`

Expand Down
Loading
Loading