Long short-term memory (LSTM) is a widely used machine-learning model in hydrology, but it is difficult to extract knowledge from it. We propose HydroLSTM, which represents processes like a hydrological reservoir. Models based on HydroLSTM perform similarly to LSTM while requiring fewer cell states. The learned parameters are informative about the dominant hydrology of a catchment. Our results show how parsimony and hydrological knowledge extraction can be achieved by using the new structure.