Weak values of quantum observables are a powerful tool for investigating a broad spectrum of quantum phenomena. For this reason, several methods to measure them in the laboratory have been proposed. Some of these methods require weak interactions and postselection, while others are deterministic, but require statistics over a number of experiments growing exponentially with the number of measured particles. Here we propose a deterministic dimension-independent scheme for estimating weak values of arbitrary observables. The scheme, based on coherently controlled SWAP operations, does not require prior knowledge of the initial and final states, nor of the measured observables, and therefore can work with uncharacterized preparation and measurement devices. As a byproduct, our scheme provides an alternative expression for two-time states, that is, states describing quantum systems subject to pre and post-selections. Using this expression, we show that the controlled-SWAP scheme can be used to estimate weak values for a class of two-time states associated to bipartite quantum states with positive partial transpose.