You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
json2csv uses a lot of CPU, but only one core. json.Unmarshal seems to be taking up most of the time so I made a forked version that launches a whole bunch of them in parallel
A 207Mb json file with 241804 lines takes 15s to convert to csv with the single-threaded version, using all cores (2 real + 2 hyperthreading) it takes 7s. However using just one core on the multi-threaded version takes 17s It might be worth to keep the single-threaded version in the code and use that when running on one core.
json2csv uses a lot of CPU, but only one core. json.Unmarshal seems to be taking up most of the time so I made a forked version that launches a whole bunch of them in parallel
A 207Mb json file with 241804 lines takes 15s to convert to csv with the single-threaded version, using all cores (2 real + 2 hyperthreading) it takes 7s. However using just one core on the multi-threaded version takes 17s It might be worth to keep the single-threaded version in the code and use that when running on one core.
parallel version
The text was updated successfully, but these errors were encountered: