James P. Waldorf, 1844-1920