Enabling Graph Neural Networks at Exascale

Graph neural networks (GNNs), which extend the successful ideas of deep learning to irregularly structured data, are a recent addition to the field of artificial intelligence. While traditional deep learning has focused on regular inputs such as images composed of pixels in two-dimensional space, graph neural networks can analyze and learn from unstructured connections between objects. This gives GNNs the ability to tackle completely new classes of problems, such as analyzing social networks and power grids or uncovering molecule structures in computational chemistry. Some experts in the field also believe that graph networks, due to their capacity for combinatorial generalization, represent an important next step towards the development of general artificial intelligence. However, such tasks require vast amounts of computation, which can only be provided by parallel processing.

It is well known that parallel computation for irregular problems is much more challenging than for regular ones, and GNNs are no exception. While traditional deep learning has been scaled up to run on entire supercomputers efficiently, GNNs currently do not scale to multiple processors. This proposal aims to overcome this limitation by drawing upon decades of experience in scalable graph algorithms and sparse linear algebra and adapting techniques that are proven to be effective for distributing graph computations over large parallel systems to GNNs.

We aim to create a new computational framework that allows users to specify a GNN while the framework handles the task of distributing graphs over parallel machines, as well as selecting and running the algorithms that are best suited for the computation automatically. Recently, frameworks such as TensorFlow have made traditional deep neural networks accessible for a large number of users. In the same way, our goal is to create a proof-of-concept framework that will be a crucial factor to the successful GNNs real-world appliance.

Partners

• The University of Bergen