Yes, I definitely have to agree with Jerry and kennyt JohnJ. Headphone listening I`m into myself. And since you are a headphone listener already, based on what you said, then why not have some real fun.
Leave the components alone, and invest in a nice pair of high end headphones with a Headroom amplifier. The folks at Headroom are very knowledgeable, and have a solution and headphone system for just about any situation. If you are not familiar with them, go to www.headphone.com/
Trust me, you`ll be glad you did!!