From the transcript here: http://www.aip.org/history/ohilist/4947.html
"(...) time in relativity seems to be treated in a manner which is quite different from the quantum theoretical manner. I think Heisenberg was quite aware, very early, of the fact that in quantum theory time is not to be compared with any space coordinate, but is, so to speak, far more basic. You can have all sorts of quantum mechanical systems without speaking of space and position, but you cannot have a quantum mechanical system without speaking of time. In relativity time enters in a manner which, at least apparently, is mathematically symmetrical to the manner in which space enters, and I think that this was an important point for Heisenberg to have thought that here was perhaps something quite different.
The other thing was that, in connection with the finite velocity of light, relativity brought in the necessity of considering actions with the finite velocity of propagation. That meant that the problems of the continuum would enter the game in a manner which was avoided in quantum theory. If you look back into the history of physics, especially of quantum theory, but wherever you go, I think you will always find that the continuum was the point where theories usually broke down. As soon as you treat a continuum as a dynamical entity and not just as a set of coordinates, that is as soon as you say that there is a body with an infinite number of degrees of freedom, in general theories break down. Planck's quantum hypothesis just started by eliminating certain elements of continuity. It may very well be that Heisenberg at that time already felt, as he certainly felt later, that perhaps to unite quantum theory and relativity would mean to do another step against continuity. I remember at that time, or perhaps a bit later, hearing the argument from him that evidently there are three basic constants of nature, because there are three fundamental units in Newtonian mechanics, and two of the constants, h and c, had been found; the third one, perhaps the length, was still to be found and understood."