Performance Becomes ~1,000 Times Slower When Adding New Objects to Large Models

[chriswmackey]

Enhancement Request

I am opening this issue as a feature request primarily because it's not preventing simulations from running or making simulations incorrect. However, it seems a little bit buggy since it does not feel intentional and it has become a bit of a blocker for a number of users who are trying to work with large OpenStudio Models that have ~1,000 spaces or more (very typical of hospitals these days as my users have shown me).

Namely, the performance of OpenStudio SDK becomes roughly 1-2 thousand times slower for adding new objects to the model when the model is large (around 1,000 spaces with HVAC equipment and zones) compared to when the model is small (less than 100 spaces). So the time it takes to add new objects to the model has a steep exponential relationship with the total number of model objects that cannot be easily explained by typical checks.

The problem is not specific to a single type of object in the model (eg. adding a new piece of zone equipment seems to take as long as adding a new building surface). This problem only affects the addition of new objects to the model and editing existing objects in the model is as fast for large models as it is for small models. Based on this and a number of tests that I have run (described below), I have been able to rule out several possible explanations. The slowness seems to be the result of something very fundamental (perhaps in the ModelObject class).

Detailed Description

The clearest way to illustrate the issue is with screenshots of some tests that I ran using the OpenStudio SDK to build a large model from data that one of our users supplied from a CAD environment. The first few Spaces get added to the model in 1-3 milliseconds, which is very fast in my opinion and, had all 5,000 spaces been added to the model as fast as this, the process would have finished in under 2 minutes.

However, the addition of each new room gets slower and slower such that, by the time we are up to 5,000 spaces, the addition of each new space takes anywhere from 2-8 seconds, which is more than 1,000 times slower:

This ultimately causes the translation process to be drawn out for ~5 hours.

Here is a graph showing the relationship between the number of objects in the model and the amount of time that it takes to add a new space. In this case, each space is a simple box with no sub-faces (so the slow down is not as extreme as the example above):

How to Recreate

The chart above was created with the following simple code (note that the two lines can be uncommented to run a test without unicity checks).

import time
import openstudio

model_faces = [
    ((0, 0, 0), (0, 10, 0), (10, 10, 0), (10, 0, 0)),
    ((0, 0, 0), (10, 0, 0), (10, 0, 3), (0, 0, 3)),
    ((10, 0, 0), (10, 10, 0), (10, 10, 3), (10, 0, 3)),
    ((10, 10, 0), (0, 10, 0), (0, 10, 3), (10, 10, 3)),
    ((0, 10, 0), (0, 0, 0), (0, 0, 3), (0, 10, 3)),
    ((0, 0, 3), (10, 0, 3), (10, 10, 3), (0, 10, 3))
]

model = openstudio.model.Model()
# model.setStrictnessLevel(openstudio.StrictnessLevel("None"))
# model.setFastNaming(True)

for i in range(5000):
    r_start = time.time()
    # create the space and zone
    os_space = openstudio.model.Space(model)
    os_zone = openstudio.model.ThermalZone(model)
    os_space.setThermalZone(os_zone)
    # create the building faces
    for face in model_faces:
        os_vertices = openstudio.Point3dVector()
        for pt in face:
            os_vertices.append(openstudio.Point3d(pt[0], pt[1], pt[2]))
        os_face = openstudio.model.Surface(os_vertices, model)
        os_face.setSpace(os_space)
    r_end = time.time()
    print(r_end - r_start)

If you are instead looking for a unit test that doesn't take as long to run, I have provided a sample of a large model below that is comparable to what our users typically send us:

https://drive.google.com/file/d/1E0wXRr59BV0zp7xSHX2bFWLHKlnqSn6N/view?usp=sharing

It has 1,000 Spaces and Zones along with detailed HVAC equipment. You can do a comparison of how fast it is to add a Space to a brand new model vs. this existing large model using the following code (just uncomment the lines and replace the path to the OSM with your machine and you will see the difference):

import time
import openstudio

model_faces = [
    ((0, 0, 0), (0, 10, 0), (10, 10, 0), (10, 0, 0)),
    ((0, 0, 0), (10, 0, 0), (10, 0, 3), (0, 0, 3)),
    ((10, 0, 0), (10, 10, 0), (10, 10, 3), (10, 0, 3)),
    ((10, 10, 0), (0, 10, 0), (0, 10, 3), (10, 10, 3)),
    ((0, 10, 0), (0, 0, 0), (0, 0, 3), (0, 10, 3)),
    ((0, 0, 3), (10, 0, 3), (10, 10, 3), (0, 10, 3))
]

model = openstudio.model.Model()
# osm_file = 'C:/Users/Chris/Desktop/run/in.osm'
# exist_os_model = openstudio.model.Model.load(osm_file)
# if exist_os_model.is_initialized():
#     model = exist_os_model.get()

r_start = time.time()
# create the space and zone
os_space = openstudio.model.Space(model)
os_zone = openstudio.model.ThermalZone(model)
os_space.setThermalZone(os_zone)
# create the building faces
for face in model_faces:
    os_vertices = openstudio.Point3dVector()
    for pt in face:
        os_vertices.append(openstudio.Point3d(pt[0], pt[1], pt[2]))
    os_face = openstudio.model.Surface(os_vertices, model)
    os_face.setSpace(os_space)
r_end = time.time()
print(r_end - r_start)

Explanations that can be Ruled Out

Unicity Checks - The checks for unique EnergyPlus names feels like it would be the most intuitive explanation given that the time it takes to perform these checks should have an exponential relationship with the number of objects in a model. However, I am fairly confident that I can rule this out based on the test that I ran setting the following before trying to add objects to the model:

model.setStrictnessLevel(openstudio.StrictnessLevel("None"))
model.setFastNaming(True)

In theory, that should turn off the unicity checks because OpenStudio SDK will use unique GUIDs instead of checking against other objects in the model to determine the name. However, in my tests, this only reduced the time it takes to add new objects to large models by ~15% (see the chart above). So it's clearly not the root of the issue.

Checks That Are Specific to a Type of Model Object - I know that most of my tests above are with geometry but I have found that the addition of any object type to the model experiences this slowness. Adding a new piece of zone equipment takes just as long as adding a new building surface. So whatever process is slowing things down is likely in the base initializer for all model objects or some other base-level class.

Checks That Are Specific to a Certain Bindings - My samples above are using OpenStudio SDK Python bindings but I have found that the Ruby bindings exhibit the same behavior.

Checks That Run in Serialization of a Model from an OSM File - Surprisingly, the serialization of large OpenStudio models from OSM files does not suffer from the exponential slowness I noted above. The large sample model that I provided above took over an hour to create with the OpenStudio SDK but it can be serialized from the OSM file within a couple of minutes. So, whatever is slowing things down does not seem to be implemented in the serializer from OSM files.

Conclusion

This leaves me to conclude that there is some process that happens when any new object is added to an OpenStudio Model, which is somehow related to the total number of objects in the model and ultimately makes it very challenging to work with large models using the OpenStudio SDK. Whatever process is slowing things down when adding new objects to the model does not run when serializing large OpenStudio Models from OSM files.

Possible Implementation

With my limited knowledge of the best way to solve the problem, I sense that we should be able to achieve the desired speed with large models if there were options to turn off checks and other processes such that we can basically run the OpenStudio Model Object initializers with the same speed that they do when serializing the model from OSM files.

[arif-hanif]

we have had similar issues at AEI for healthcare projects with 1000+ zones