Hangs while listing a large directory

[yyshk]

Cyberduck hangs while listing a large directory (with 30,000+ files) on Google Cloud Storage.

I noticed that the file listing limit prompt had been disabled, so I tried re-enabling it. Now the dialog box is displayed,
but when I answer to it, the application hangs. It doesn't matter whether I choose CONTINUE or CANCEL.

Version 8.4.5 - 8.5.3 (Version8.4.4 works fine)
Windows 10

[dkocher]

I cannot reproduce with a bucket of <10'000 files.

[yyshk]

Thank you for your response.
I found that the issue is reproducible only when the directory contains a large number of folders.

(1) A directory with 10,000+ text files

• File listing limit prompt is displayed.
• Click on "Continue".
• The directory is listed successfully (i.e. the issue is not reproduced).

(2) A directory with 10,000 folders (each folder contains one text file)

• File listing limit prompt is displayed.
• Click on "Continue".
• The application hangs.

Cyberduck 8.5.6
Windows10 Pro 22H2

[AliveDevil]

Diff from 8.4.4 to 8.4.5 is #13774, which updated the tree view you are seeing from an ancient to a newer, still ancient, version.
Maybe something changed there, that the rendering just can't keep up, thus locking up.

Checking what we can do to mitigate this.

[AliveDevil]

Can't reproduce it directly … it takes a while, until all 10000 items are displayed, taking a bit of a CPU load there, eventually gets displayed. Until all 10000 items are shown, Cyberduck is unresponsive.

[AliveDevil]

So, Cyberduck retrieves the items in sub-second speed, then forwards these to the ObjectListView which takes forever to rebuild the entire tree - synchronously. May need to check source of OLV 2.5 and 2.9 what changed here.

cyberduck/windows/src/main/csharp/ch/cyberduck/ui/winforms/BrowserForm.cs

Line 664 in 0b1b04a

browser.RefreshObjects(list);

[AliveDevil]

Grabbed latest sources, rebuilt them to include source information for easier instrumentation.
Result:

[AliveDevil]

There's something interesting.
If I just expand a bucket with 10000 items, then that runs immediately - while navigating into that bucket takes forever to load.

[AliveDevil]

Options here:
Get rid of ObjectListView, redoing the browser, history, bookmarks and bonjour views with more performant WPF-components
Try to fix ObjectListView

[dkocher]

Will be part of #13036.

[gianluca-poli]

Hi, what about dir listing with more than one million objects?
CD take extraordinarily long time to display the list and sometimes hangs.
thanks