Performance issues in long lists when using italics.

[Mihara]

Linux, Emacs 26.2, current markdown-mode MELPA release.

When editing a list of 30+ items, each of them a few lines long with at least one word in italics, performance degrades spectacularly in linear proportion to the number of items in the list, eventually resulting in latencies approaching 0.5s between a keypress and it showing up on screen.

Let us have a file like that:

+ So I have a markdown file, and inside I have a long-ish list of items, each a paragraph long, or long enough to wrap around, using some *italics*.

Repeat this line 30-50 times. The slowdown increases proportionally to the number of items in the list, and it persists with emacs -Q, though obviously, it takes a few more lines to achieve the same kind of slowdown. (76 is sufficient in my case.)

profiler-report reports something like this:

- flyspell-post-command-hook                                     2347  52%
 - if                                                            2347  52%
  - progn                                                        2347  52%
   - condition-case                                              2336  52%
    - let                                                        2336  52%
     - let                                                       2336  52%
      - if                                                       2336  52%
       - flyspell-check-word-p                                   2332  52%
        - let                                                    2332  52%
         - cond                                                  2332  52%
          - cond                                                 2332  52%
           - and                                                 2332  52%
            - sit-for                                            2332  52%
             - redisplay                                         1893  42%
              - redisplay_internal (C function)                  1873  42%
               - jit-lock-function                               1857  41%
                - jit-lock-fontify-now                           1857  41%
                 - jit-lock--run-functions                       1857  41%
                  - run-hook-wrapped                             1857  41%
                   - #<compiled 0x284ae81>                       1857  41%
                    - font-lock-fontify-region                   1853  41%
                     - font-lock-default-fontify-region               1853  41%
                      - font-lock-fontify-keywords-region               1837  41%
                       - markdown-match-italic                   1701  38%
                        - markdown-inline-code-at-pos-p               1669  37%
                         - markdown-inline-code-at-pos               1665  37%
                          - markdown-match-code                  1426  32%
                           - markdown-search-until-condition               1426  32%
                              apply                              1426  32%
                            markdown-beginning-of-text-block                199   4%
                            markdown-end-of-text-block                 36   0%
                        + markdown-match-inline-generic                 32   0%
                       + markdown-fontify-plain-uris                 28   0%

The situation improves a lot if lines are hard-wrapped, but does not completely disappear, it just takes a longer file to bring emacs to a halt, and hard-wrapping is in itself a pain.

Could something be done about that?

[Ruin0x11]

Same problem. It would be nice to have a toggle for markdown-mode's more expensive font locking functions if the markdown gets too complex. Right now I can't edit my own GitHub wiki in Emacs because it's way too slow.

[quanticle]

I'm getting the same problem as well, but strangely enough, it's manifesting in a file that has a lot of inline code snippets, whereas other files (which do have a lot of italics) don't display any perceptible input lag. Here's the profiler report when emacs is slow (i.e. when running on a file with lots of code snippets):

- redisplay_internal (C function)                                3786  60%
 - jit-lock-function                                             3784  60%
  - jit-lock-fontify-now                                         3784  60%
   - jit-lock--run-functions                                     3769  59%
    - run-hook-wrapped                                           3769  59%
     - #<compiled 0x1ff64e3db5f1>                                3769  59%
      - font-lock-fontify-region                                 3769  59%
       - font-lock-default-fontify-region                        3769  59%
        - font-lock-fontify-keywords-region                      3717  59%
         - markdown-match-italic                                 2971  47%
          - markdown-match-italic                                2822  44%
           - markdown-match-italic                               2740  43%
            - markdown-match-italic                              2605  41%
             - markdown-match-italic                             2439  38%
              - markdown-match-italic                            2343  37%
               - markdown-match-italic                           1938  30%
                - markdown-match-italic                          1811  28%
                 - markdown-match-italic                         1502  23%
                  - markdown-match-italic                        1301  20%
                   - markdown-match-italic                       1132  17%
                    - markdown-inline-code-at-pos-p                931  14%
                     + markdown-inline-code-at-pos                929  14%
                    + markdown-match-italic                       201   3%
                   + markdown-inline-code-at-pos-p                167   2%
                     markdown-match-inline-generic                  1   0%
                  + markdown-inline-code-at-pos-p                 197   3%
                  + markdown-match-inline-generic                   3   0%
                 + markdown-inline-code-at-pos-p                  306   4%
                 + markdown-match-inline-generic                    3   0%
                + markdown-inline-code-at-pos-p                   124   1%
                + markdown-match-inline-generic                     3   0%
               + markdown-inline-code-at-pos-p                    402   6%
               + markdown-match-inline-generic                      3   0%
              + markdown-inline-code-at-pos-p                      93   1%
              + markdown-match-inline-generic                       3   0%
             + markdown-inline-code-at-pos-p                      153   2%
             + markdown-match-inline-generic                       13   0%
            + markdown-inline-code-at-pos-p                       131   2%
            + markdown-match-inline-generic                         3   0%
           + markdown-inline-code-at-pos-p                         76   1%
           + markdown-match-inline-generic                          6   0%
          + markdown-inline-code-at-pos-p                         139   2%
          + markdown-match-inline-generic                           9   0%
         + markdown-match-code                                    254   4%
         + markdown-fontify-plain-uris                             75   1%
         + markdown-fontify-sub-superscripts                       68   1%
         + markdown-match-bold                                     61   0%
         + markdown-fontify-inline-links                           25   0%
         + markdown-match-inline-attributes                        19   0%
         + markdown-fontify-gfm-code-blocks                        15   0%
         + #<compiled 0x42de642f>                                  14   0%
         + markdown-fontify-list-items                              6   0%
           markdown-fontify-tables                                  5   0%
         + markdown-match-yaml-metadata-key                         5   0%
         + markdown-fontify-blockquotes                             3   0%
         + markdown-fontify-angle-uris                              2   0%
         + markdown-fontify-reference-links                         2   0%
         + markdown-match-html-tag                                  2   0%
         + markdown-fontify-fenced-code-blocks                      2   0%
         + markdown-match-leanpub-sections                          2   0%
         + markdown-match-includes                                  2   0%
         + markdown-match-gfm-open-code-blocks                      1   0%
         + markdown-match-fenced-end-code-block                     1   0%
         + markdown-match-pre-blocks                                1   0%
         + markdown-match-yaml-metadata-begin                       1   0%
         + markdown-match-yaml-metadata-end                         1   0%
         + markdown-fontify-headings                                1   0%
         + markdown-match-gfm-close-code-blocks                     1   0%
          font-lock-fontify-syntactically-region                   22   0%
        + font-lock-unfontify-region                                8   0%
   + run-with-timer                                                 6   0%
 + eval                                                             1   0%
   file-remote-p                                                    1   0%
+ ...                                                            2275  36%
+ command-execute                                                 176   2%
+ timer-event-handler                                              28   0%
+ jit-lock--antiblink-post-command                                 25   0%

It seems like markdown-match-italic is being called a bunch recursively, which is the main cause of the slowdown.

EDIT: I just took another look at the file, and I realized that it doesn't have any italics at all in the body text. It does, however, contain several dozen snippets of Python code, which have numerous underscores. Is there a way to exclude code blocks from markdown-match-italic?

[tejasvi]

Can confirm the issue occurs with deeply nested lists using the * as delimeters. Since markdown uses * delimeters for italics as well, it causes huge penality during parsing. I personally use _ delimeters for italics so I remove * from the italics regex and I am seeing major improvements in input latency.

(defconst markdown-regex-italic "\\(?:^\\|[^\\]\\)\\(?1:\\(?2:[_]\\)\\(?3:[^ \n\t\\]\\|[^ \n\t]\\(?:.\\|\n[^\n]\\)[^\\ ]\\)\\(?4:\\2\\)\\)")
;; or
(defconst markdown-regex-gfm-italic
  "\\(?:^\\|[^\\]\\)\\(?1:\\(?2:[_]\\)\\(?3:[^ \\]\\2\\|[^ ]\\(?:.\\|\n[^\n]\\)\\)\\(?4:\\2\\)\\)"

[dsedivec]

FYI still seeing this using latest Emacs master. I, too, have a long numbered list with a bunch of inline code blocks that have underlines (I'm writing a bunch of documentation for Python code, so naturally all the symbols I'm putting in inline code markup have underscores). I used straight.el to start emacs -Q and loaded latest markdown-mode from the master branch. The Emacs profiler shows 72% of CPU samples in font-lock-fontify-keywords-region with 61% of samples in markdown-match-italic. It looks very much like in quanticle's comment.

Emacs is basically unusable for editing this file as long as font-lock-mode is on. Turning off font-lock-mode, of course, makes it instantly better.

[ptitan]

Same here: the longer the list the slower it is. And if the list is too long, emacs becomes unusable. The only "solution" I found is to disable font-lock, which is not ideal...

Does anyone have an idea?

[hmelman]

I see the slowdown in a gfm list of 224 items without any italics and using - as the list indicator (all the items have checkboxes). The only use of * in the file is several cases of bold. Indeed, removing the * from markdown-regex-gfm-italic significantly improves the performance. I haven't looked any further into it than that, but just looking at the stack trace above, I'd investigate making markdown-match-italic not be recursive.

[ackerleytng]

I found this workaround to be effective: #799 (comment)

[hmelman]

I found this workaround to be effective: #799 (comment)

I also find it to be effective. It should make debugging the issue easier to get to the root cause of the problem.

[ackerleytng]

It seems like jit-lock-debug-mode isn't actually a workaround. It effectively turns off highlighting, which improves performance at the cost of functionality.