Seq2Seq (Encoder-Decoder) wiht Attention Mechanism for Grammar Correction in Keras.
At first, we should create our parallel dataset for training our model. In preprocess folder, lang8 and nucle modules convert each dataset into proper format. Lang8 dataset is very noisy, so I decided to do small preprocessing on that. I remove non-ascii characters, reduce length of character with 3 or more with 1 (e.g token like
!!!!!!! convert to
!), and remove unnecessary punctuation (all puntuation except
At the final preprocessing step, I do some data augmentation. In each pair (source, target), in addition to existing error, I inject some typo/grmmatical error into the source samples. Things I do in this step include:
Accourding to this paper, above cases will cover most of the errors in English learner writings.
In training step, I used famous seq2seq Attention model here. The best hyper-parameters for seq2seq explored by the team at google in "Massive Exploration of Neural Machine Translation Architectures" paper. I used one layer encoder/decoder to keep things as simple as posible. It can be easily extend to 4 layer encoder/decoder famework (considering regularization and dropout).
Computer vision and Deep learning in SRU
A Deep-Learning-Based Persian Speech Recognition System
Multi-class confusion matrix library in Python
دوره 12 ساعته یادگیری عمیق با چارچوب Keras
My exercises in the deep learning course
Face Mask Detection using TensorFlow and Keras
Model Generalization on CIFAR-10 Dataset, accuracy higher than 91% on test data
LSA : Layer Sustainability Analysis framework for the analysis of layer vulnerability in a given neural network. LSA can be a helpful toolkit to assess deep neural networks and to extend the adversarial training approaches towards improving the...