Computing the inverse is fine for most ‘normal cases’ (i.e. the matrix is square, not super-large and actually invertible), but when you start working with matrices that are either big or have a high condition number (read: they are almost singular), you will have to use other methods. If you want to compute the preimage of a lot of vectors, the best approach is still to just compute the inverse (I think John Cook mentioned this as well in his article).

If you want to compute A^{-1}x for a huge matrix A, you might want to use conjugate gradient (for symmetric positive definite matrices), BiCGSTAB, and/or GMRES. All work great for sparse matrices.

If you want to compute A^{-1}x for a nonsquare matrix, you can solve the normal equations A^T A x = A^T x.

If you want to compute A^{-1}x for a singular or almost-singular matrix A, you can compute the generalized or Moore-Penrose inverse (using, for example, the singular value decomposition).

Computing the inverse is fine for most ‘normal cases’ (i.e. the matrix is square, not super-large and actually invertible), but when you start working with matrices that are either big or have a high condition number (read: they are almost singular), you will have to use other methods. If you want to compute the preimage of a lot of vectors, the best approach is still to just compute the inverse (I think John Cook mentioned this as well in his article).

If you want to compute A^{-1}x for a huge matrix A, you might want to use conjugate gradient (for symmetric positive definite matrices), BiCGSTAB, and/or GMRES. All work great for sparse matrices.

If you want to compute A^{-1}x for a nonsquare matrix, you can solve the normal equations A^T A x = A^T x.

If you want to compute A^{-1}x for a singular or almost-singular matrix A, you can compute the generalized or Moore-Penrose inverse (using, for example, the singular value decomposition).

I happen to have had a situation recently where inversion was the correct call for two strange reasons:

That said, this practice of avoiding inversion is a good gateway to responsible matrix-manipulation techniques, and I like the writeup.