I had a couple of Androids and enjoyed how I could modify my user environment, the open source nature of apps, etc. I find the Android experience not to be as amalgamated (for lack of a better word) as the iExperience as Android is spread across multiple hardware platforms which all get cludged and tweaked by the maker. Having a fractured market vs. the huge mono-market of iPhone creates a more handy tool, IMO.
That said I can see a return to Android. It is fun to play with, way more open source apps and customization, etc. It was about size and camera for me to go from the 5 to the 6.
Android has come a long way since I remember you making the switch. Unfortunately, the caveat to that is the fractured market you describe.
Core android is awesome on a phone (although, I think it gets a little awkward on a tablet). Unfortunately, depending on what phone you buy, you might not get anything close to the core android experience. I think Samsung has gotten better with the s6, but I have an S5 sitting in front of me (company loaner for international travel) and there is so much stupid shit on it. I'm just going to pull the SIM card and put it in my motorola phone when I leave tomorrow, because I don't want to deal with eliminating all of this shit.
When I use a loaner iphone, I just reset the device, log in to my apple account, and have a tolerable experience.
Its not like I do a crazy lot of customization either...When I reset my moto x, I basically just have to rearrange the icons how I like, and switch the stock SMS app out for Textra. All of my apps install when I sign into my google account, and I am good to go.
I keep thinking about an iphone upgrade...but I always run into something that makes me pause. The hidden filesystem and lack of "Open With..." functions was what got me most recently. Trying to send a video from my phone to my GFs without having to use itunes...took about 3 different apps before we found one that would let her actually save the video into her own photo stream (needed the dropbox app, couldn't do it from the website). And all of that was because the app she was trying to load the video into didn't know how to access anything but her photo stream. On android, when an app asks me to select a photo/video, I can grab it from anywhere...google drive app shows up there, as does dropbox, as does my downloads folder and anywhere else you might conceivably find a video (and app developers don't have to do anything special to enable this).
Or I had a coworker I used to send code to for review...I'd have to re-save as a .txt file if he needed to look at it on his iphone...because on a normal unjailbroken iphone, I guess there was no easy way to just tell the phone "nah, this file is fine, just open it with the text editor" (or even a way to save it somewhere, and go change the file extension to .txt).
TL;DR, I love the "just works" nature of the iphone...except when it doesn't work. The non-bloatware android phones just work 95% of the time but are still easy to manage that other 5%.