Thomas Aquinas made this argument in the 13th century, writing, "If a thing can be done adequately by means of one, it is superfluous to do it by means of several; for we observe that nature does not employ two instruments [if] one suffices." Occam's razor has gained strong empirical support in helping to converge on better theories (see "Applications" section below for some examples).
In the related concept of overfitting, excessively complex models are affected by statistical noise (a problem also known as the bias-variance trade-off), whereas simpler models may capture the underlying structure better and may thus have better predictive performance.
His principle states that among competing hypotheses, the one with the fewest assumptions should be selected or when you have two competing theories that make exactly the same predictions, the simpler one is the better.
In science, Occam's razor is used as a heuristic guide in the development of theoretical models, rather than as a rigorous arbiter between candidate models.
Libert Froidmont, in his On Christian Philosophy of the Soul, takes credit for the phrase, speaking of "novacula occami".