The story of “hidden variables” dates back to 1935 and grew out of Einstein’s worries about the completeness of quantum theory.
Consider, for example, a single qubit.
Recalling our previous discussion on compatible operators (Section 4.6), we know that no quantum state of a qubit can be a simultaneous eigenstate of two non-commuting operators, such as
In this view, the indeterminacy found in quantum theory is merely due to our ignorance of these “hidden variables” that are present in nature but not accounted for in the theory. Einstein came up with a number of pretty good arguments for the existence of “hidden variables”, perhaps the most compelling of which was described in his 1935 paper (known as “the EPR paper”), co-authored with his younger colleagues, Boris Podolsky and Nathan Rosen. It stood for almost three decades as the most significant challenge to the completeness of quantum theory. Then, in 1964, John Bell showed that the (local) hidden variable hypothesis can be tested and refuted.
Any theory can make predictions, but just because the predictions turn out to be correct, this does not make the theory true — there may be other, maybe equivalent, explanations. The key to the scientific method is falsifiability: make one prediction incorrectly, and you have proven your theory is not true.