I just started playing with your code and when I try to add a value like $wp_session['colors'] = 'daa' or try to unset a value like that. I'm not sure if it should be supported like that.
When reading the array and use a variable and manipulate that variable there isn't an issue.
The error is something like: Indirect modification of overloaded property. I found this as a read: http://mwop.net/blog/131-Overloading-arrays-in-PHP-5.2.0.html