Extracting structure from text documents based on machine learning

K.A. Kudim, G.Yu. Proskudina

Abstract


This study is devoted to a method that facilitates the task of extracting structure from the text documents using an artificial neural network. The method consists of  data preparation,  building and training the model and results evaluation. Data preparation includes collecting corpora of documents, converting a variety of file formats into plain text, and manual labeling each document structure. Then documents are split into tokens and into paragraphs. The text paragraphs are represented as feature vectors to provide input to the neural network. The model is trained and validated on the selected data subsets. Trained model results evaluation is presented.  The final performance is calculated per label using precision, recall, and F1 measures, and overall average. The trained model can be used to extract sections of documents bearing similar structure.

Prombles in programming 2022; 3-4: 154-160


Keywords


natural language processing; information extraction; machine learning; neural network

Full Text:

PDF

References


Kudim K.A., Proskudina G.Yu., 2019. Methods and tools for extracting personal data from theses abstracts. Problems in programming, no. 2, pp. 38-46. (in Russian). Available from: http://pp.isofts.kiev.ua/ojs1/article/view/359 [Accessed 04/08/2022].

CrossRef

Kudim K.A., Proskudina G.Yu., 2020. A method for extracting data from semistructured documents. Problems in programming. no. 1, pp. 25-32. (in Russian). Available from: http://pp.isofts.kiev.ua/ojs1/article/view/388 [Accessed 04/08/2022].

CrossRef

Yi He., 2017. Extracting Document Structure of a Text with Visual and Textual Cues. University of Twente. Elsevier. 78 р. (in English). Available from: https://essay.utwente.nl/72979/1/Yi He - master thesis - final version.pdf [Accessed 05/08/2022]

Steffen Nissen, 2005. Neural Networks Made Simple. Software 2.0. no. 2, pp. 14-19. Available from: http://fann.sourceforge.net/fann_en.pdf [Accessed 05/08/2022].

Martin Riedmiller, Heinrich Braun., 1993. A Direct Adaptive Method for Faster Backpropagation Learning: The RPROP Algorithm - Neural Networks. IEEE International Conference. pp. 586-591. Available from: https://paginas.fe.up.pt/~ee02162/dissertacao/RPROP paper.pdf [Accessed 05/08/2022].




DOI: https://doi.org/10.15407/pp2022.03-04.154

Refbacks

  • There are currently no refbacks.