Current sparsity pattern computation strategy can require "over-ghosting"

[lindsayad]

My use case is mortar. With how we've implemented mortar in MOOSE, the secondary face drives the construction of the residuals and Jacobians. More specifically we loop over mortar segments with local processor ids (which match the secondary lower-dimensional processor ids from which they were initially spawned (and then possibly split by projection of primary nodes)) to build the residuals and Jacobians. Conceptually, in order to accomplish this, we only really need to ghost elements for a given process if it has secondary lower-dimensional elements, e.g. the following ghosting strategy conceptually should work

I have a secondary lower element -> ghost primary interior parent and secondary interior parent
I have a primary interior parent -> I don't actually need to ghost anything
I have a secondary interior parent -> I don't actually need to ghost anything

However, because the primary/secondary interior parent degrees of freedom (is) are coupled to the secondary lower element degrees of freedom (js), with the way we build our sparsity pattern, I have to ensure that I do this:

I have a secondary interior parent -> I need to ghost the secondary lower element
I have a primary interior parent -> I need to ghost the secondary lower element

I don't think this is that big of an issue, but it's been something that's been tickling my brain today.

[lindsayad]

I think it's more of an issue if I imagine a distributed displaced mesh problem. For the sake of argument we can imagine that meshes slide all the way across each other over the course of the simulation. In such a scenario every secondary element (lower and higher) will couple with every primary element (higher) over the course of the simulation. With the current sparsity pattern construction strategy you would have to ghost the entire secondary face if you own a primary face element on top of ghosting the entire primary face if you own a secondary element (unless you mitigated this with some communication strategy). If you let the “driver” element fill in the “dependent” elements sparsity rows relevant columns, then you could get away with only ghosting the primary face if you own a secondary element, and you could avoid ghosting the entire secondary face if you own a primary element.

[lindsayad]

Mortar was the initial example given here, but you could put forward a similar proposal for DG. In MOOSE the element with a lower id is the element that drives forward the assembly of the face element and Jacobian. So the process that owns the element with the lower ID must ghost the neighbor. If the driver could add the sparsity pattern coupling of dneighbor/delem (and in this discretization case also dneighbor/dneighbor), then there would be no need for doing any ghosting for the process that owns the neighbor (with the higher element ID)