The method of fundamental solutions (MFS) is a numerical method for solving boundary value problems involving linear partial differential equations. It is well known that it can be very effective assuming regularity of the domain and boundary conditions. The main drawback of the MFS is that the matrices involved are typically ill-conditioned and this may prevent the method from achieving high accuracy. In this talk, we will present some new approaches involving tools from numerical linear algebra such as the singular value decomposition and Arnoldi iteration for reducing the ill conditioning in the MFS. Several numerical examples show that these approaches are much superior to the classical MFS in terms of conditioning and accuracy.