The differences that really count are the seat and handlebars, which are relatively inexpensive to swap. Prior to '05, the performance difference was all in the gearing, with the Speedmaster geared to accelerate a bit better through the gears and the America having an edge at the top end of the speedometer. In '05, the Speedmaster got more displacement and milder cams, giving it noticably more low end torque, but the peak power is listed as less and at a lower engine speed. The end result is that the performance differences are similar, but more so.
So, ride both bikes. If you find one more comfortable, make yourself a note that you want to end up with that seat and/or handlebars. If you like the look of one, but the performance range of the other, you will have to decide which is most important to you. You can put chrome on the Speedmaster or powdercoat on the America. You can get a lot more torque from the America or more topend from the Speedmaster, but you are going to spend some time and money to get there.
