From 61fa4e35f62fa6473ade53d14ec9709afbf96b62 Mon Sep 17 00:00:00 2001 From: Teiva Harsanyi Date: Fri, 8 Mar 2024 00:40:56 +0100 Subject: [PATCH] Abbreviation, structure --- docs/5-interface-pollution.md | 1 + docs/92-false-sharing.md | 1 + docs/index.md | 21 +- includes/abbreviations.md | 4 + mkdocs.yml | 21 +- site/20-slice/index.html | 357 ++++---- site/28-maps-memory-leaks/index.html | 351 ++++---- site/404.html | 302 ++++--- site/5-interface-pollution/index.html | 419 ++++++---- site/56-concurrency-faster/index.html | 415 ++++++---- site/89-benchmarks/index.html | 409 ++++++---- site/9-generics/index.html | 409 ++++++---- site/92-false-sharing/index.html | 351 ++++---- .../98-profiling-execution-tracing/index.html | 427 ++++++---- site/book/index.html | 452 ++++++----- site/chapter-1/index.html | 590 ++++++++------ site/external/index.html | 760 ++++++++++-------- site/index.html | 465 ++++++----- site/ja/index.html | 465 ++++++----- site/search/search_index.json | 2 +- site/sitemap.xml.gz | Bin 343 -> 343 bytes site/zh/index.html | 451 ++++++----- 22 files changed, 3923 insertions(+), 2750 deletions(-) create mode 100644 includes/abbreviations.md diff --git a/docs/5-interface-pollution.md b/docs/5-interface-pollution.md index 9e185b9d..8df3a4ee 100644 --- a/docs/5-interface-pollution.md +++ b/docs/5-interface-pollution.md @@ -3,6 +3,7 @@ title: Interface pollution (#5) comments: true hide: - toc +status: new --- # Interface pollution diff --git a/docs/92-false-sharing.md b/docs/92-false-sharing.md index 3c54639d..181b097d 100644 --- a/docs/92-false-sharing.md +++ b/docs/92-false-sharing.md @@ -3,6 +3,7 @@ title: Writing concurrent code that leads to false sharing (#92) comments: true hide: - toc +status: new --- # Writing concurrent code that leads to false sharing diff --git a/docs/index.md b/docs/index.md index 97267f66..99515a11 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,5 +1,6 @@ --- comments: true +status: new --- # Common Go Mistakes @@ -18,7 +19,7 @@ This page is a summary of the mistakes in the [100 Go Mistakes and How to Avoid You're viewing a beta version enriched with significantly more content. However, this version is not yet complete, and I'm looking for volunteers to help me summarize the remaining mistakes ([GitHub issue #43](https://github.com/teivah/100-go-mistakes/issues/43)). Progress: - + ## Code and Project Organization @@ -187,8 +188,8 @@ type Bar struct { In the `Foo` struct, the `Bar` type is declared without an associated name; hence, it’s an embedded field. -We use embedding to promote the fields and methods of an embedded type. Because Bar contains a Baz field, this field is -promoted to `Foo`. Therefore, Baz becomes available from Foo. +We use embedding to promote the fields and methods of an embedded type. Because `Bar` contains a `Baz` field, this field is +promoted to `Foo`. Therefore, `Baz` becomes available from `Foo`. What can we say about type embedding? First, let’s note that it’s rarely a necessity, and it means that whatever the use case, we can probably solve it as well without type embedding. Type embedding is mainly used for convenience: in most cases, to promote behaviors. @@ -592,7 +593,7 @@ This code updates the last index to 10. However, if we run this code, it does no [:simple-github: Source code](https://github.com/teivah/100-go-mistakes/tree/master/src/04-control-structures/31-range-loop-arg-evaluation/) -### Ignoring the impacts of using pointer elements in `range` loops (#32) +### :warning: Ignoring the impacts of using pointer elements in `range` loops (#32) ???+ warning @@ -1333,7 +1334,7 @@ In summary, when we work in concurrent applications, it’s essential to underst ???+ info "TL;DR" - When creating a certain number of goroutines, consider the workload type. Creating CPU-bound goroutines means bounding this number close to the `GOMAXPROCS` variable (based by default on the number of CPU cores on the host). Creating I/O-bound goroutines depends on other factors, such as the external system. + When creating a certain number of goroutines, consider the workload type. Creating CPU-bound goroutines means bounding this number close to the GOMAXPROCS variable (based by default on the number of CPU cores on the host). Creating I/O-bound goroutines depends on other factors, such as the external system. In programming, the execution time of a workload is limited by one of the following: @@ -1511,7 +1512,7 @@ In summary, let’s be mindful that a goroutine is a resource like any other tha [:simple-github: Source code](https://github.com/teivah/100-go-mistakes/tree/master/src/09-concurrency-practice/62-starting-goroutine/) -### Not being careful with goroutines and loop variables (#63) +### :warning: Not being careful with goroutines and loop variables (#63) ???+ warning @@ -2241,7 +2242,7 @@ Let’s make sure we are comfortable reading these messages. Go always logs the * Where accesses occur in the code: in this case, lines 9 and 10. * When these goroutines were created: goroutine 7 was created in main(). -In addition, if a specific file contains tests that lead to data races, we can (temporarily! :wink:) exclude it from race detection using the `!race` build tag: +In addition, if a specific file contains tests that lead to data races, we can exclude it :material-information-outline:{ title="temporarily! 😉" } from race detection using the `!race` build tag: ```go //go:build !race @@ -2389,7 +2390,7 @@ Read the full section [here](92-false-sharing.md). ???+ info "TL;DR" - Use instruction-level parallelism (ILP) to optimize specific parts of your code to allow a CPU to execute as many parallel instructions as possible. Identifying data hazards is one of the main steps. + Use ILP to optimize specific parts of your code to allow a CPU to execute as many parallel instructions as possible. Identifying data hazards is one of the main steps. [:simple-github: Source code](https://github.com/teivah/100-go-mistakes/tree/master/src/12-optimizations/93-instruction-level-parallelism/) @@ -2443,9 +2444,9 @@ Read the full section [here](98-profiling-execution-tracing.md). To help avoid CPU throttling when deployed in Docker and Kubernetes, keep in mind that Go isn’t CFS-aware. -The `GOMAXPROCS` variable defines the limit of OS threads in charge of executing user-level code simultaneously. By default, it's set to the number of OS-apparent logical CPU cores. +By default, GOMAXPROCS is set to the number of OS-apparent logical CPU cores. -When running some Go code inside Docker and Kubernetes, we must know that Go isn't CFS-aware ([github.com/golang/go/issues/33803](https://github.com/golang/go/issues/33803)). Therefore, `GOMAXPROCS` isn't automatically set to the value of `spec.containers.resources.limits.cpu` (see [Kubernetes Resource Management for Pods and Containers](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/)); instead, it's set to the number of logical cores on the host machine. The main implication is that it can lead to an increased tail latency in some specific situations. +When running some Go code inside Docker and Kubernetes, we must know that Go isn't CFS-aware ([github.com/golang/go/issues/33803](https://github.com/golang/go/issues/33803)). Therefore, GOMAXPROCS isn't automatically set to the value of `spec.containers.resources.limits.cpu` (see [Kubernetes Resource Management for Pods and Containers](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/)); instead, it's set to the number of logical cores on the host machine. The main implication is that it can lead to an increased tail latency in some specific situations. One solution is to rely on [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) that automatically set `GOMAXPROCS` to match the Linux container CPU quota. diff --git a/includes/abbreviations.md b/includes/abbreviations.md new file mode 100644 index 00000000..b3c80a67 --- /dev/null +++ b/includes/abbreviations.md @@ -0,0 +1,4 @@ +*[GC]: Garbage Collector +*[ILP]: Instruction-Level Parallelism +*[CFS]: Completely Fair Scheduler +*[GOMAXPROCS]: The variable defines the limit of OS threads in charge of executing user-level code simultaneously diff --git a/mkdocs.yml b/mkdocs.yml index 8b9dedfe..d7f94aeb 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -23,6 +23,7 @@ theme: - navigation.tracking - toc.follow - content.code.annotate + - content.tooltips palette: # Palette toggle for light mode - scheme: default @@ -62,17 +63,13 @@ extra: social: - icon: fontawesome/brands/twitter link: https://twitter.com/teivah + status: + new: New content extra_css: - stylesheets/extra.css nav: - - Book: - - book.md - - chapter-1.md - - external.md - Go Mistakes: - index.md - - '简体中文': zh.md - - '日本語': ja.md - Full Sections: - 5-interface-pollution.md - 9-generics.md @@ -82,7 +79,15 @@ nav: - 89-benchmarks.md - 92-false-sharing.md - 98-profiling-execution-tracing.md + - Translations: + - '简体中文': zh.md + - '日本語': ja.md + - Book Details: + - book.md + - chapter-1.md + - external.md markdown_extensions: + - abbr - admonition - pymdownx.details - pymdownx.highlight: @@ -91,7 +96,9 @@ markdown_extensions: pygments_lang_class: true - pymdownx.highlight - pymdownx.inlinehilite - - pymdownx.snippets + - pymdownx.snippets: + auto_append: + - includes/abbreviations.md - pymdownx.superfences - tables - attr_list diff --git a/site/20-slice/index.html b/site/20-slice/index.html index 18b43149..aea221a4 100644 --- a/site/20-slice/index.html +++ b/site/20-slice/index.html @@ -300,14 +300,16 @@ + + -
  • - +
  • + - Book + Go Mistakes
    • @@ -318,16 +320,14 @@ - - -
  • - +
  • + - Go Mistakes + Book Details
    • @@ -391,41 +391,44 @@ + + + + + -
  • +
  • - - - + -
    • + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + + + + - + + + + + + +
  • + + + + + External Resources + + + +
    • @@ -893,7 +970,7 @@

    Maps and memory leaks

    1. Allocate an empty map.
    2. Add 1 million elements.
      3. -
    3. Remove all the elements, and run a Garbage Collection (GC).
      4. +
    4. Remove all the elements, and run a Garbage Collection (GC).

    After each step, we want to print the size of the heap (using a printAlloc utility function). This shows us how this example behaves memory-wise:

    func main() {
@@ -921,12 +998,12 @@ 
    Maps and memory leaks
    
         fmt.Printf("%d MB\n", m.Alloc/(1024*1024))
 }
    -

    We allocate an empty map, add 1 million elements, remove 1 million elements, and then run a GC. We also make sure to keep a reference to the map using runtime.KeepAlive so that the map isn’t collected as well. Let’s run this example:

    +

    We allocate an empty map, add 1 million elements, remove 1 million elements, and then run a GC. We also make sure to keep a reference to the map using runtime.KeepAlive so that the map isn’t collected as well. Let’s run this example:

    0 MB   <-- After m is allocated
 461 MB <-- After we add 1 million elements
 293 MB <-- After we remove 1 million elements
    -

    What can we observe? At first, the heap size is minimal. Then it grows significantly after having added 1 million elements to the map. But if we expected the heap size to decrease after removing all the elements, this isn’t how maps work in Go. In the end, even though the GC has collected all the elements, the heap size is still 293 MB. So the memory shrunk, but not as we might have expected. What’s the rationale? We need to delve into how a map works in Go.

    +

    What can we observe? At first, the heap size is minimal. Then it grows significantly after having added 1 million elements to the map. But if we expected the heap size to decrease after removing all the elements, this isn’t how maps work in Go. In the end, even though the GC has collected all the elements, the heap size is still 293 MB. So the memory shrunk, but not as we might have expected. What’s the rationale? We need to delve into how a map works in Go.

    A map provides an unordered collection of key-value pairs in which all the keys are distinct. In Go, a map is based on the hash table data structure: an array where each element is a pointer to a bucket of key-value pairs, as shown in figure 1.

    @@ -948,7 +1025,7 @@

    Maps and memory leaks

    After adding 1 million elements, the value of B equals 18, which means 2¹⁸ = 262,144 buckets. When we remove 1 million elements, what’s the value of B? Still 18. Hence, the map still contains the same number of buckets.

    The reason is that the number of buckets in a map cannot shrink. Therefore, removing elements from a map doesn’t impact the number of existing buckets; it just zeroes the slots in the buckets. A map can only grow and have more buckets; it never shrinks.

    -

    In the previous example, we went from 461 MB to 293 MB because the elements were collected, but running the GC didn’t impact the map itself. Even the number of extra buckets (the buckets created because of overflows) remains the same.

    +

    In the previous example, we went from 461 MB to 293 MB because the elements were collected, but running the GC didn’t impact the map itself. Even the number of extra buckets (the buckets created because of overflows) remains the same.

    Let’s take a step back and discuss when the fact that a map cannot shrink can be a problem. Imagine building a cache using a map[int][128]byte. This map holds per customer ID (the int), a sequence of 128 bytes. Now, suppose we want to save the last 1,000 customers. The map size will remain constant, so we shouldn’t worry about the fact that a map cannot shrink.

    However, let’s say we want to store one hour of data. Meanwhile, our company has decided to have a big promotion for Black Friday: in one hour, we may have millions of customers connected to our system. But a few days after Black Friday, our map will contain the same number of buckets as during the peak time. This explains why we can experience high memory consumption that doesn’t significantly decrease in such a scenario.

    What are the solutions if we don’t want to manually restart our service to clean the amount of memory consumed by the map? One solution could be to re-create a copy of the current map at a regular pace. For example, every hour, we can build a new map, copy all the elements, and release the previous one. The main drawback of this option is that following the copy and until the next garbage collection, we may consume twice the current memory for a short period.

    @@ -974,7 +1051,7 @@

    Maps and memory leaks

    182 MB -Remove all the elements and run a GC +Remove all the elements and run a GC 293 MB 38 MB @@ -1110,7 +1187,7 @@

    Comments

    - + diff --git a/site/404.html b/site/404.html index c2345d0a..80337739 100644 --- a/site/404.html +++ b/site/404.html @@ -258,10 +258,10 @@
  • - + - Book + Go Mistakes
    • @@ -276,10 +276,10 @@
  • - + - Go Mistakes + Book Details
    • @@ -365,7 +365,7 @@ - Book + Go Mistakes @@ -375,7 +375,7 @@ - - - - - - - - - - - @@ -470,24 +429,24 @@ - + - + + - - - - @@ -1299,7 +1373,7 @@

    Summary

    - + diff --git a/site/external/index.html b/site/external/index.html index 5a6ef0d3..a3b50502 100644 --- a/site/external/index.html +++ b/site/external/index.html @@ -14,8 +14,6 @@ - - @@ -300,16 +298,14 @@ - - -
  • - +
  • + - Book + Go Mistakes
    • @@ -320,14 +316,16 @@ + + -
  • - +
  • + - Go Mistakes + Book Details
    • @@ -389,44 +387,41 @@ - - - - - -
  • +
  • - + + + -
    • + + + + + + + + + + + + + + + + + + +
  • + + + + + + + + + + +