In this thesis, a new characterization of solutions to overdetermined systems of linear equations is described based on a simple quadratic penalty function, which is used to change the problem into an unconstrained one. Piecewiselinear non-interior pathways to the set of optimal solutions are generated from the minimization of the unconstrained function. It is shown that the entire set of solutions is obtained from the paths for sufficiently small values of a scalar parameter. As a consequence, a new finite penalty algorithm is given for fx, problems. The algorithm is implemented and exhaustively tested using random and function approximation problems. .A comparison with the Barrodale-Phillips algorithm is also done. The results indicate that the new algorithm shows promising performance on random (non-function approximation) problems.