This is what flusser had to say on the subject in Dressing The Man.
"As to the ideal number of buttons for the blazer's front and sleeves, personal taste tends to defer to tradition. ...the number of sleeve buttons is related to the coast front's button arrangement. With the most popular jacket model being the two-button single breasted, four sleeve buttons are the norm, although two are equally proper. Three sleeve buttons on a two button coast seem slightly out of balance, whears with the three-button model, three or four sleeve buttons harmonize handsomely."
"..... a classic double breasted blazer with flap pockets is traditionally trimmed with 6 front buttons, two that button, two that don't and two upper display...buttons demands four sleeve buttons."
Its more about balance than anything else.
I don't think it matters personally. Like most things, style is a matter of taste and what you are comfortable with.
I have a two button blazer with 4 buttons on each sleeve. but i got bored with this, and chopped the other 2 off.