Lots of matrices don't have an inverse.

GraphBLAST - High-Performance Linear Algebra-based Graph Primitives on GPUs.

Graph BLAS - Graph algorithms in the language of linear algebra. (HN)

Cotila - Compile-time linear algebra system for C++.

BLIS - Portable software framework for instantiating high-performance BLAS-like dense linear algebra libraries.