Update the log redaction documentation #17835

JmPotato · 2024-07-02T03:45:32Z

What is changed, added or deleted? (Required)

Update the log redaction documentation.

Which TiDB version(s) do your changes apply to? (Required)

master (the latest development version)
v8.3 (TiDB 8.3 versions)
v8.2 (TiDB 8.2 versions)

What is the related PR or file link(s)?

Other reference link(s): Enhance log desensitization tikv/pd#8305

Do your changes match any of the following descriptions?

Change aliases

JmPotato · 2024-07-02T03:48:38Z

log-redaction.md

+[2024/07/02 11:35:01.426 +08:00] [INFO] [conn.go:1146] ["command dispatched failed"] [conn=1482686470] [session_alias=] [connInfo="id:1482686470, addr:127.0.0.1:52258 status:10, collation:utf8mb4_0900_ai_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="insert into `t` values ( ‹1› ) , ( ‹1› )"] [txn_mode=PESSIMISTIC] [timestamp=450859185309483010] [err="[kv:1062]Duplicate entry '‹1›' for key 't.a'"]
+```
+
+从以上报错日志可以看到，当把 `tidb_redact_log` 的值设为 `MARKER` 后，TiDB 日志中会用单角形引号 `‹›` 标记出敏感信息，用户可以根据自己的需求自定义脱敏规则来处理日志中的敏感信息。


The term "单角形引号" is sourced from https://zh.wikipedia.org/wiki/Guillemet, though I am uncertain about its accurate translation in Chinese.

tiflash/tiflash-configuration.md

JaySon-Huang · 2024-07-17T09:33:28Z

log-redaction.md


 ## PD 组件日志脱敏

-PD 侧的日志脱敏需要将 [`security.redact-info-log`](/pd-configuration-file.md#redact-info-log-从-v50-版本开始引入) 的值设为 `true`。该配置项值默认为 `false`，即关闭脱敏。
+PD 侧的日志脱敏需要将 [`security.redact-info-log`](/pd-configuration-file.md#redact-info-log-从-v50-版本开始引入) 的值设为 `true` 或 `"MARKER"`。该配置项值默认为 `false`，即关闭脱敏。


PD 支持设置为小写的 "marker" 么？因为配置文件里面的其他配置项 value 应该都是小写的，配置文件里面出现大写会有些不和谐。
(tiflash 代码上是支持这个配置项的 value 大小写忽略的，但文档上只写小写的 "marker")

PD 也是这样，大小写均可，我统一一下只写小写 marker 吧。

LykxSassinator · 2024-07-17T09:40:51Z

/cc @qiancai

tikv-configuration-file.md

log-redaction.md

pd-configuration-file.md

tiflash/tiflash-configuration.md

Signed-off-by: JmPotato <[email protected]>

Signed-off-by: JaySon-Huang <[email protected]>

Signed-off-by: JmPotato <[email protected]>

log-redaction.md

Co-authored-by: xixirangrang <[email protected]>

log-redaction.md

JaySon-Huang

LGTM

ti-chi-bot · 2024-08-01T02:39:28Z

@JaySon-Huang: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

LGTM

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ti-chi-bot · 2024-08-01T06:38:46Z

@LykxSassinator: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ti-chi-bot · 2024-08-02T07:09:07Z

@ljun0712: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

log-redaction.md

pd-configuration-file.md

tikv-configuration-file.md

log-redaction.md

lilin90

/lgtm

ti-chi-bot · 2024-08-14T08:22:58Z

[LGTM Timeline notifier]

Timeline:

2024-08-12 06:44:28.805108046 +0000 UTC m=+163953.508577689: ☑️ agreed by lilin90.
2024-08-14 08:22:57.187394947 +0000 UTC m=+342661.890864590: ☑️ agreed by hfxsd.

hfxsd · 2024-08-14T08:23:18Z

/approve

ti-chi-bot · 2024-08-14T08:23:25Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hfxsd

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [hfxsd]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot bot added missing-translation-status This PR does not have translation status info. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jul 2, 2024

JmPotato force-pushed the update_log_redaction branch from ed4783e to fdda754 Compare July 2, 2024 03:46

JmPotato commented Jul 2, 2024

View reviewed changes

JmPotato mentioned this pull request Jul 2, 2024

Enhance log desensitization tikv/pd#8305

Closed

5 tasks

JaySon-Huang reviewed Jul 17, 2024

View reviewed changes

tiflash/tiflash-configuration.md Outdated Show resolved Hide resolved

JaySon-Huang reviewed Jul 17, 2024

View reviewed changes

JaySon-Huang mentioned this pull request Jul 17, 2024

tikv: support security.redact-info-log == marker mode to redact logs. #17848

Closed

18 tasks

LykxSassinator mentioned this pull request Jul 17, 2024

Security: logs support marker-based sensitive data desensitization capabilities. tikv/tikv#17206

Closed

ti-chi-bot bot requested a review from qiancai July 17, 2024 09:40

hfxsd self-assigned this Jul 23, 2024

hfxsd added v8.3 translation/doing This PR’s assignee is translating this PR. and removed missing-translation-status This PR does not have translation status info. labels Jul 23, 2024

hfxsd self-requested a review July 23, 2024 03:04

hfxsd reviewed Jul 23, 2024

View reviewed changes

tikv-configuration-file.md Outdated Show resolved Hide resolved

log-redaction.md Outdated Show resolved Hide resolved

log-redaction.md Outdated Show resolved Hide resolved

pd-configuration-file.md Outdated Show resolved Hide resolved

tiflash/tiflash-configuration.md Show resolved Hide resolved

qiancai removed their request for review July 24, 2024 02:44

hfxsd added the type/compatibility-or-feature-change This PR involves compatibility changes or feature behavior changes. label Jul 24, 2024

JaySon-Huang mentioned this pull request Jul 25, 2024

log densensitation in tiflash pingcap/tiflash#8977

Closed

JmPotato and others added 4 commits July 25, 2024 11:21

Update the log redaction documentation

50b5518

Signed-off-by: JmPotato <[email protected]>

Update docs about tiflash log-redaction

a238e5f

Signed-off-by: JaySon-Huang <[email protected]>

Update docs about tikv log-redaction and align

6e95da3

Signed-off-by: JaySon-Huang <[email protected]>

Address the comments

76b401a

Signed-off-by: JmPotato <[email protected]>

JmPotato force-pushed the update_log_redaction branch from c975ef1 to 76b401a Compare July 25, 2024 03:21

JmPotato commented Jul 25, 2024

View reviewed changes

log-redaction.md Outdated Show resolved Hide resolved

JmPotato requested review from hfxsd and JaySon-Huang July 25, 2024 03:23

pd config use "marker"

4df202f

Co-authored-by: xixirangrang <[email protected]>

hfxsd reviewed Jul 31, 2024

View reviewed changes

log-redaction.md Outdated Show resolved Hide resolved

JaySon-Huang approved these changes Aug 1, 2024

View reviewed changes

Update log-redaction.md

c899b07

LykxSassinator approved these changes Aug 1, 2024

View reviewed changes

ljun0712 approved these changes Aug 2, 2024

View reviewed changes

hfxsd mentioned this pull request Aug 6, 2024

Update the log redaction documentation pingcap/docs#18487

Merged

17 tasks

hfxsd added translation/done This PR has been translated from English into Chinese and updated to pingcap/docs-cn in a PR. and removed translation/doing This PR’s assignee is translating this PR. labels Aug 6, 2024

hfxsd requested a review from lilin90 August 6, 2024 07:26

lilin90 reviewed Aug 7, 2024

View reviewed changes

lilin90 reviewed Aug 8, 2024

View reviewed changes

log-redaction.md Outdated Show resolved Hide resolved

lilin90 added 2 commits August 8, 2024 14:57

Update wording and format

3386514

Update format

0a67512

lilin90 approved these changes Aug 12, 2024

View reviewed changes

ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Aug 12, 2024

hfxsd approved these changes Aug 14, 2024

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Aug 14, 2024

ti-chi-bot bot added the approved label Aug 14, 2024

ti-chi-bot bot merged commit a39d6d9 into pingcap:master Aug 14, 2024
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update the log redaction documentation #17835

Update the log redaction documentation #17835

JmPotato commented Jul 2, 2024 •

edited by hfxsd

Loading

JmPotato Jul 2, 2024

JaySon-Huang Jul 17, 2024 •

edited

Loading

JmPotato Jul 22, 2024

LykxSassinator commented Jul 17, 2024

JaySon-Huang left a comment

ti-chi-bot bot commented Aug 1, 2024

ti-chi-bot bot commented Aug 1, 2024

ti-chi-bot bot commented Aug 2, 2024

lilin90 left a comment

ti-chi-bot bot commented Aug 14, 2024

hfxsd commented Aug 14, 2024

ti-chi-bot bot commented Aug 14, 2024

Update the log redaction documentation #17835

Update the log redaction documentation #17835

Conversation

JmPotato commented Jul 2, 2024 • edited by hfxsd Loading

What is changed, added or deleted? (Required)

Which TiDB version(s) do your changes apply to? (Required)

What is the related PR or file link(s)?

Do your changes match any of the following descriptions?

JmPotato Jul 2, 2024

Choose a reason for hiding this comment

JaySon-Huang Jul 17, 2024 • edited Loading

Choose a reason for hiding this comment

JmPotato Jul 22, 2024

Choose a reason for hiding this comment

LykxSassinator commented Jul 17, 2024

JaySon-Huang left a comment

Choose a reason for hiding this comment

ti-chi-bot bot commented Aug 1, 2024

ti-chi-bot bot commented Aug 1, 2024

ti-chi-bot bot commented Aug 2, 2024

lilin90 left a comment

Choose a reason for hiding this comment

ti-chi-bot bot commented Aug 14, 2024

[LGTM Timeline notifier]

hfxsd commented Aug 14, 2024

ti-chi-bot bot commented Aug 14, 2024

JmPotato commented Jul 2, 2024 •

edited by hfxsd

Loading

JaySon-Huang Jul 17, 2024 •

edited

Loading