This essay outlines the applicability of Recurrent Neural Networks for supervised binary classification tasks in Natural Language Processing. It gives a general overview of the relevant theory and different recurrent layers. The theory is than applied to a binary classification task of offensive German language in twitter posts. Different model structures are compared to discover dependencies on batch size and number of iterations for generalization.