I've never had faith in a creative team like I have in this one. Each season has been better than the last...that's not just according to me, but to Metacritic. Unlike Mad Men, I've never felt BB has wavered in its story quality.
That being said, they now have a clean slate, story-wise. They can go in any direction they want. Since they have carte blanche, I have no doubt they've very carefully weighed all options and gone with something that won't just be "more of the same but bigger."
Every season -- EVERY one -- a number of people have predicted that the show would falter, not for any reason known to them, but they just couldn't conceive of the producers' sustained creativity. They've been wrong every time. Therefore, I see no reason to declare the patient dead until the heart actually falters.
There is exactly one time the show ever let me down, and that was the finale of season 2. Especially since the penultimate episode was so great. Other than that, though, I agree that it's been a consistent upward trend. I suppose that I could see people complain that the show became increasingly detached from reality, but that's part of what I like about it. Season 4 was like a fever dream.