We benchmark the decoherence of superconducting transmon qubits to examine the temporal stability of energy relaxation, dephasing, and qubit transition frequency. By collecting statistics during measurements spanning multiple days, we find the mean parameters T1 = 49 μs and T2 = 95 μs; however, both of these quantities fluctuate, explaining the need for frequent re-calibration in qubit setups. Our main finding is that fluctuations in qubit relaxation are local to the qubit and are caused by instabilities of near-resonant two-level-systems (TLS). Through statistical analysis, we determine sub-millihertz switching rates of these TLS and observe the coherent coupling between an individual TLS and a transmon qubit. Finally, we find evidence that the qubit’s frequency stability produces a 0.8 ms limit on the pure dephasing which we also observe. These findings raise the need for performing qubit metrology to examine the reproducibility of qubit parameters, where these fluctuations could affect qubit gate fidelity