I would use the phone as it is and dip your toe in the water with a Fiio E07, for the PC
Basically this replaces your sound card and
has a decent headphone amp built in. It will give you access to hi-Rez music and could be used in the future with a phone or player that has line or digital out. I don't know your phone but I assume it has neither of these.