Neural network models have a long history in fuel cell and battery modeling. With the recent advent of deep learning, there is potential for further improvements in these models. Conversely, deep learning is primarily designed for image detection and classification using large data sets and its performance on typical regression tasks in fuel cell and battery modeling remains largely unexplored. In this article, we present a new method for applying deep learning to general vector outputs from battery and fuel cell models and investigate the use of different deep learning architectures. We compare these methods to equivalent Gaussian process (GP) models on a range of regression tasks. We further provide the first rigorous error and asymptotic analysis of the multivariate GP model. For scalar outputs, deep networks are found to be less accurate on small data sets, but for large data sets, convolutional and recurrent networks are able to marginally exceed the accuracy of GP models.