-
Notifications
You must be signed in to change notification settings - Fork 37
Handling huge document sets
Overview is designed to support document sets that contain tens of thousands of documents -- or more. If you'd like to try to get the most out of your own installation of Overview, here are some tips.
Before reading these tips, try handling a huge document set. Maybe it will work! If it doesn't, read on....
If you see an OutOfMemoryError, your document set is too large for the worker process. Try increasing the -Xmx parameter of your worker instance.
There is a specific case of this worth mentioning: 32-bit Java. Unfortunately, 32-bit versions of Java don't allow you to increase -Xmx beyond around 1.5 gigabytes. So make sure you're running a 64-bit version of Java: type java -version on the command-line and scan for the key words, "64-bit". If it doesn't say "64-bit" it isn't 64-bit; uninstall Java and install a 64-bit version from the JDK download page.
(Note: 64-bit Java won't run on 32-bit Windows machines. Also, Java's automatic installers tend to default to a 32-bit version on many machines, which is why we linked to the JDK download page directly. See (the reasons)[http://java.com/en/download/faq/java_win64bit.xml] if you're curious.)
CSV upload is the quickest to upload and parse, so try it if you're frustrated by other methods.
At Overview, we love working with huge document sets. Ask the (overview-users)[https://groups.google.com/forum/#!forum/overview-users] group for help and you'll probably get some good advice.