This is a rough version of the summary of the complete answer I posted on math.stackexchange, where more details are discussed in a long digression, in particular mathematical motivations for your points 1., 3. and 4. (Any reader interested in more explanations and a longer updated list of references should check out that other answer in Math.SE). I eagerly recommend you read carefully the following wonderful references:
Systems are completely described by the observable properties they have (or the degrees of freedom available for measurement or chosen to be described), properties which may not have simultaneously-defined values. This is because observing some aspects of a system may destroy/"undefine" some of the previous properties, precisely those which were incompatible with the new ones. For example, composing Stern-Gerlach apparatus in succession in different axis, allows you to measure the spin components $S_z$ and $S_x$, but after the second filter, a new measurement of the first will be random again, i.e. the observable property "having defined spin in direction __" is not simultaneously defined for independent directions: measuring one makes the system lose its definite value in the other. Another example is the position and linear momentum, which were the commuting degrees of freedom of classical phase space. Quantum mechanical experiments, very related to the originally called "wave-particle duality", determined that position and momentum wave functions (measuring the probability amplitude distributions of possible measurements) were related by a Fourier transformation of each other, thus implying that their commutator was $[\hat x,\hat p_x]=i\hbar$, so they became noncommuting variables better studied under operator algebras.
Since classically all observables are functions on phase space, this forces the algebra of general observables to be noncommutative: the order of which observables are measured in succession may not commute. Since any physical measurement is numerical, in general real-valued, the noncommutative algebra fits nicely within self-adjoint operators on Hilbert spaces, because these are the only operators unitarily equivalent to multiplicative diagonal operators of real spectrum. Since we can regard an observable property as just the set of its possible values, working with operator algebras allows for different such properties/observables to be encoded in the mutual algebra they define. Thus, we determine experimentally which sets of observables commute, so we can talk of complete sets of compatible operators, which define the state of the system completely by a data string of eigenvalues (the actual values of the parameters/properties measured). Such possible strings of eigenvalues for each operator in a compatible set define a "state of the system at a certain moment", but also define basis vectors of a Hilbert space upon which the operators act; hence different such sets define different basis, so that if our system is given by a common eigenvector of one basis, it will be generally a linear superposition of the eigenvectors of another basis, producing interfering cross terms in the transition probabilities from one state to another. This approach is found experimentally to be the right one since, besides complementarity, the other major quantum feature is linear superposition of states. Since the eigenvalues are the observable properties of the system, the superposition in other basis has no uniquely defined eigenvalues and thus those other properties of the system in that state are not well-defined at that time. Since common eigenstates can be given by projection operators which project onto the successive eigensubspaces, a pure state of the system can be given by either a ray of the projective Hilbert space or by a suitable projection operator: every operator of a complete commuting set is expanded by the spectral theorem as a sum of projection operators weighed by its respective eigenvalues, where the projectors project onto each common eigenvector, and that is the way in which operators encode both the list of possible measurable values of a property and their mutual algebra. Then, one can formalize quantum mechanics only with operator algebras (where such things as C*-algebras and Gel'fand-Naimark enter) and characterize which operators are observables and which are pure states, or mixed states if one includes uncertainties in the preparation initial states. In this sense one recovers Heisenberg's picture/matrix mechanics, where the state was just the list of observed values for a chosen set of compatible measuring devices, and the observables evolved with time between measurements so one obtained probabilistically a new set of values for maybe a different set of compatible measurements. Complementarity and uncertainty was a direct mathematical consequence of the noncommutativity, and interference/wave-particle duality a direct byproduct of the linear character of unitary evolution in between observations.
Therefore, observables are given by algebras of self-adjoint operators because these are those having real spectra corresponding to the possible real empirical values. Maximal commuting subsets of operators are simultaneously diagonalizable, i.e. they represent compatible observables that can be measured at the same time, so the state of the system at every observation is characterized by such a list of eigenvalues. This defines a Hilbert space by linear superposition upon which the operators act: an observable has a definite value if and only if the state is in one of its eigenvectors. Given an eigenstate in a chosen basis, unitary evolution in Schrödinger's picture moves the vector in the Hilbert space so its projected component to every other possible basis vector changes, where the squared modulus of those complex components of the present vector state over any other vector give the probability of observing the second set of eigenvalues in our next measurements given the first set in our initial measurement. Since these components are given in terms of scalar products $(|\chi\rangle,\, |\psi\rangle)$, by Riesz theorem one can use linear functionals $\langle\chi|$ to act on vectors and give the desired transition amplitudes; since these functionals form also a (dual) linear space, one can think of them as possible final states in our computations. Physical states however are just empirical list of values for every observed degree of freedom of the system, very careful metaphysical considerations must be taken into account before giving meaning to intermediate, not observed superposition states: they statistically correlate observed states at different times but the system cannot be said to have a superposition of its properties. As Dirac claimed: Heisenberg's picture is the [empirical] right one (no one ever observes superpositions!).
As Asher Peres said: "Experiments occur in a laboratory, not in a Hilbert space" and "unperformed experiments have no results". The ontological and epistemological interpretational problems of the formalism enter when one attempts to think of it beyond an empiricist stance, attributing reality to parts of the formalism (e.g. superposition of cats) which cannot be observed experimentally (yet!?) beyond the structural/correlational level. (If you are interested in these issues I would recommend reading articles by Carlo Rovelli on relational quantum mechanics, and anything on the consisten/coherent histories formalism, besides the book by Isham listed at the top, in order to contrast standard orthodox references with warier understandings).