I would like to inject a speculative idea. There is a form of matter that would only interact with visible matter gravitationally, i.e. only through the exchange of gravitons. This would be matter that existed IN ANOTHER UNIVERSE. The idea that matter in our universe could be interacting with matter in another universe is an idea taken seriously by string theorists. For example, the idea is discussed in the book “The Hidden Reality, Parallel Universes and the Deep Laws of the Cosmos” by Brian Greene There is also a tie in between what I suggested and the so called Ekpyrotic Universe, which is based on the idea that our hot big bang universe was created from the collision of two three-dimensional worlds moving along a hidden, extra dimension. The tie in is that the Ekpyrotic model posits as I do the gravitational interaction between two “branes“ that are situated close to each other. See http://wwwphy.princeton.edu/~steinh/npr/

Here is an animation by Neil Turok showing the interaction between two “brane” universes, which illustrate his Ekpyrotic Universe model. His animation could also serve to illustrate the gravitational interaction between two “branes” corresponding to different universes. From the perspective of our universe, the matter in the other universe would be “dark matter” whose presence is felt only via gravitational attraction  i. e. the exchange of gravitons.