I vote for the brown. Combining blue, a more city, formal color, with Harris tweed, the most hairy and rustic of tweeds, strikes me as very dissonant. Donegals look better in blue--but then I must admit to being biased, as I own such a jacket.
Also, the brown appears to have a lot more of the mottled, multi-colored depth that makes Harris tweed so special. The blue is a lot more flat looking.
The blue is herringbone in technique while the brown is not, right? It seems to me, though, that the blue is a little cleaner with not as much of a "halo" of fibers above the material which thus makes it a little cleaner looking.