Navigation – Plan du site

AccueilTroisième série22Egyptian press archive of CEDEJ. ...

Egyptian press archive of CEDEJ. A challenging case study of Arabic OCR

Les archives de presse du CEDEJ. Une étude de cas ambitieuse sur l'OCR en arabe
أرشيف الصحافة المصرية في «مركز الدراسات والوثائق الاقتصادية والقانونية والاجتماعية». دراسة حالة تمثل تحدياً في مجال التعرُّف الضوئي على الحروف العربية
Rami Khalil Rouchdi, Mohamed Ahmed Ellotf et Hala Bayoumi
p. 41-55

Résumés

Cet article évalue trois systèmes commerciaux de reconnaissance optique de caractères (OCR) pour la numérisation des archives de presse : Sakhr Automatic Reader (AR) version 11.2 gold; Abbyy FineReader (FR) version 12 ; et NovoVerus (NV) version 4.2.0 dans un contexte de dégradation de la qualité du texte. Contrairement à d'autres tentatives similaires, nous avons développé notre propre ensemble de données pour étudier les meilleures spécifications et outils afin de réaliser la plus grande précision dans le projet d'archives de presse égyptiennes du Centre d'Études et de Documentation Économiques, Juridiques et Sociales (CEDEJ). Nous décrivons l'approche du développement de l'ensemble de données, ainsi que l'effet des différentes spécifications des images, sur la précision de l'OCR. Notre jeu de données se compose de 30 extraits de presse qui représentent différentes qualités, en termes de fond d'image, de taille de texte et d'autres effets dus à l'âge et au stockage des documents papier. Chaque échantillon a été scanné dans différentes résolutions et modes de couleur, pour produire un ensemble de 180 échantillons (six versions de chaque extrait), puis transmis aux logiciels d’OCR pour évaluer leur précision de reconnaissance. Le résultat de l'expérience a ensuite été appliqué à plus d'un million d’extraits, le corpus du projet CEDEJ, et a donné des résultats cohérents. Dans cet article, nous présentons principalement une approche de la numérisation et de l'OCR de documents ayant un contenu textuel arabe de faible qualité, ce qui garantit une précision élevée et constante. Notre approche est basée sur l'évaluation des performances des logiciels OCR par rapport à différentes spécifications de capture et de manipulation d'images.

Haut de page

Texte intégral

Introduction1

  • 1 We thank our colleagues in the Digitization Operations section at Bibliotheca Alexandrina; Rasha F (...)

1The CEDEJ was born in 1968 from a desire to preserve the library of the French Law School in Cairo. The political scientist Jean-Claude Vatin initiated this press archive in 1970. In 2007, after the closure of the CEDEJ library, the collection of press clippings was stored in the French Consulate in Alexandria. Two years later, a framework agreement for scientific cooperation was signed between the National Centre for Scientific Research (CNRS) and the Bibliotheca Alexandrina (BA) to set up an online system for documentary research from the CEDEJ archives. A tripartite partnership agreement was signed in 2012 (renewed in 2016 and 2019) between the CNRS, the French Ministry of Europe and Foreign Affairs (MEAE) and the BA. The latter agreement worked indeed towards the creation of two web-based research platforms common to the CEDEJ and the Bibliotheca Alexandrina to promote the corpus of press clippings of the CEDEJ (2015) and a corpus of 12,000 press cartoons (2018).

2The CEDEJ archive has 1.7 million press clippings from Egyptian daily and weekly newspapers and a corpus of 12,000 press cartoons (mostly in Arabic) that cover the same period and was archived in the same way as the former collection. The two collections’ content (clippings and caricatures) cover 40 years of Egyptian political, cultural and economic life. It is the fruit of a collaboration between the BA and the CEDEJ teams in order to provide to the community two open access, web-based documentary research tools: cedej.bibalex.org (inaugurated in 2015) and cedejcaricatures.bibalex.org (in 2018). Each press-clip has three translations of the metadata (subject, theme, author, keyword...) to allow a trilingual search, in addition to the Optical Character Recognition (OCR) of its textual contents, which allows researchers to explore the body of the text and to find a word and/or its derivatives. Over several decades (1970-2010), the archive content was classified manually by CEDEJ archivists according to 20 thematic areas: economics; history; education, etc. The content as well as the add-on value of the archive represent a major scientific interest for research in the human and social sciences. Statistics (google analytics) showed 60 million hits for the year 2019 on the cedej.bibalex.org portal.

The project workflow

3The project workflow consisted of the following stages:

  • Pest control: According to international standards of restoration and conservation, the contents of the boxes were placed for three months in a plastic tent, filled with nitrogen to eliminate insects and bacteria and to prevent the paper from drying out.
  • Indexing: To facilitate the tracking, cataloguers labeled each file and recorded the classification, so that each of the clips and cartoons were indexed according to the theme of its box.
  • Digitization: The digitization stage involved three main phases: the scanning phase in which a digital image of the article is captured, the image manipulation phase to enhance the digital image quality and the OCR phase to automate the extraction of textual contents of each press-clip. In addition to that, several quality control phases occurred to verify the artifact compliance to the quality standards, throughout its progress from phase to phase.
  • Cataloging: Finally, after the digitization and text extraction and before the publishing, the digitized press-clippings and cartoons were communicated by the Bibliotheca Alexandrina to the CEDEJ team in Cairo. The later team carried out the documentary analysis and cataloguing. After diagonal reading of the digitized press, the cataloguer registers its metadata in French, English and Arabic. For cartoons, analysis and reflection on the drawing is done to fill in the metadata.

Management and tracking

4Throughout the workflow of the project, several challenges were faced. Firstly, the challenge of management, tracking and coordination between the teams located in different geographic places as well as integrating their work of different nature. In this context, the CEDEJ team (centralized in Cairo) and the BA team (centralized in Alexandria) cooperatively integrated their effort in successive phases under a well optimized workflow. Accordingly, the BA developed the Digital Asset Management (DAM) and the Digital Assets Factory (DAF), that are BA in-house developed software suites used to centralize the management and tracking of the hardcopies and their corresponding digital materials. In addition, the same software suites can verify data integrity and provide tracking reports of the different work units at the different stages.

Optical Character Recognition (OCR) as a challenge

5The second and major challenge in this project was to provide an automated mass-production and high accuracy OCR considering the low-tech printing in addition to the age and storage conditions of the archive materials.

6To decide about digitization specifications, we produced several versions of each sample articles with different digital image specifications and evaluated the accuracy of extracted texts using three commercial Arabic Optical Character Recognition (OCR) systems: (a) Sakhr Automatic Reader (AR) version 11.2 gold; one of the oldest Arabic OCR suites in the market. It produced high recognition accuracy all over our Arabic content projects. (b) Abbyy FineReader (FR) version 12, an OCR suite that proved solid existence in the market due to its high efficiency in document structure analysis and non-Arabic scripts OCR, as well as its positive ranking after the addition of Arabic OCR to its features. (c) NovoVerus (NV) version 4.2.0, a product of NovoDynamics that showed high accuracy results in case of degraded text quality. In contrast to other similar evaluation trials, we developed our own dataset to study the best specifications and tools in order to realize highest accuracy in the Egyptian press archive project of CEDEJ (Egyptian Press Archive of CEDEJ). We describe the approach of developing the dataset, as well as the effect of different image specifications on the OCR accuracy. Our dataset consists of 30 press-clips that represent different qualities, in terms of image background, text size and other effects due to age and storage. Each sample was scanned in different resolutions and color modes, to produce a set of 180 samples (6 versions of each press-clip), then fed to Automatic Reader, Abbyy FineReader and NovoVerus, to evaluate its recognition accuracy. The experiment was then applied on more than 1 million press-clips, the corpus of the CEDEJ project, and showed consistent results. In this paper, we mainly introduce an approach to digitize and OCR documents having low quality Arabic textual content, which guarantees high and consistent accuracy. Our introduced approach is based on evaluating OCR suites performance against different image capturing and manipulation specifications.

7With reference to its experience in the digitization, the BA led and took the first steps into the digitization projects in the Middle East, focusing mainly on the techniques related to the nature of Arabic scripts, including the best image capturing specifications, image manipulation techniques and the OCR approaches to be used (Eldakar, El Gazzar, Adly et al. 2005).

8BA focused on digitizing Arabic books. The analysis of the experiments’ results led to the technique adopted by the BA: to scan books in black and white color mode, at a resolution of 300 dpi, and save the captured images in lossless format, Tagged Image File Format (TIFF) and CCITT G4 compression. Following the scanning stage, comes the image manipulation, which is mainly aimed at dealing with the defects resulting from the image-capturing phase. These defects are due to the limitations of image capturing technology and the document’s physical conditions, as well as the human factor. The image manipulation methods were chosen carefully to prevent any deformation to the captured letter, as they affect the OCR’s efficiency. The text page defects handled in the image manipulation phase include: skew fixing, unwanted pixels removal and finally delivering the page to the final user in a polished form (such as unifying all pages’ dimensions and centering text to the page). Finally, the OCR phase starts by extracting text from the manipulated images. The BA adopts a supervised machine learning approach that produces a character level recognition accuracy of 99.5% versus ground-truth text, using Sakhr Automatic Reader (AR) to OCR books with Arabic textual contents (Eldakar, El Gazzar, Adly et al. 2005). Additionally, the BA uses a useful feature of AR to develop pre-trained 16 basic fonts based on font shape, quality and size. To OCR a new book, two pages are chosen (as a training dataset) and one of the basic pre-trained fonts that matches the book’s font (shape, size and quality), and the initial recognition accuracy is calculated for a test page (as a test dataset). After a further training in the basic font adapted to that specific book, the recognition accuracy for the same test page is recalculated and recorded for evaluation.

9In the following sections: We describe the challenges to OCR Arabic script, we describe the Egyptian Press archive of CEDEJ project, we list the related work regarding Arabic newspapers press archives, where OCR stands as an obstacle to provide full search features, we explore other attempts to evaluate Arabic OCR suites, we introduce the characteristics of the CEDEJ dataset, as well as our attempt to evaluate the OCR suites, its results and the accuracy realized over the project, by applying the decisions made after the analysis of our experiment results.

Arabic script characteristics

10Although the Arabic language is one of the official six language of the UN and an official language for over 421 million speakers all over the world, the OCR of Arabic script is still immature and forms an obstacle for several projects. The reason is that the nature of Arabic script needs a different approach to OCR (Alkhateeb, Abu Doush, Albousoul 2017), as the Arabic-script consists of 28 characters, each of which has one to four glyphs (isolated, initial, middle and final) depending on its position in the word. While representing the same character, glyphs of each character are completely different in shape, which adds more challenge to recognize more than 100 classes (Al-A‘ali, Ahmed 2007). Moreover, Arabic is a cursively written script, which makes segmentation more challenging than non-cursive scripts, as Arabic script characters are not of identical width, neither are the different glyphs of the same character. In addition, an overlapping of characters may vary upon font shape, as well as ligatures, which are overlapping characters that share the same space. Furthermore, dots form a main component of Arabic letters, yet they are not attached to the character’s body. They may lead to ambiguity if they are misrecognized due to small font size, degraded document quality, bad image capturing setting, as well as bad image manipulation parameters that may lead to merging or removing dots. Connected blocks of letters in Arabic script may lead to a sub-word or a complete word. Closed loop letters in case of small font sizes may disappear from the features of the captured image, leading to misrecognition of these characters. All these factors make OCR of Arabic scripts very different and challenging.

Egyptian press archive of CEDEJ project

11When the BA started the Egyptian Press Archive of CEDEJ project, the previously described procedure of Arabic books’ OCR was not suitable for newspaper press-clips, because the corpus consists of press-clips from different sources dating back to 1970 and spanning 40 years of Egyptian history. As a result, the collection has different printing technologies, different font shapes and sizes, depending on the publishing houses as well as different factors of storage and time, which affected some of the press-clips.

12Contrary to the book scenario (described earlier), and while we still aim to produce the highest recognition accuracy, the learning approach is neither applicable nor productive as it will be per single press-clip. The BA decided to try other Arabic OCR suites in the market, such as ABBY FineReader and NovoVerus, to compare it with Sakhr Automatic Reader and to evaluate their performance on degraded text as is the case of press-clips. With consideration that the characteristics of the dataset affects the recognition accuracy, we chose a dataset that covers a large variety of good quality, degraded quality, and different text size in the same page, some having text written on shaded background frames while others on clear backgrounds, in addition to some stained pages.

13Our aim was to find a productive, yet accurate, OCR approach that may handle such a widely diverse collection to produce a fully searchable collection of texts available to the public and researchers.

Online Arabic Press Archive Attempts

14We’ll describe now two online Arabic digital press archives, to demonstrate that the OCR of Arabic scripts is an obstacle preventing such projects to be fully featured. But knowing that the Egyptian Press Archive of CEDEJ is considered an exception to that case, as the presence of the full text extraction and the robust search engine allows autocomplete and morphological text search (Samir et al. 2015).

15Historic Arabic Newspaper Digitization Project: The Moise A. Khayrallah Center for Lebanese Diaspora Studies – founded in 2010 – is a public history and research center in North Carolina State University’s College of Humanities and Social Sciences. The center is concerned with preserving the memories and history of the Lebanese American community, with a collection dating back to 1892. A dedicated team is working on a challenging project to build their collection from microfilmed versions, private collectors, antique shops, and many more. The center has a long-term goal to OCR their collection and they described the difficulties to OCR it, which included the technical issues of the Arabic scripts, in addition to the bad quality of printed and microfilmed copies, as well as the bad storage effect on the hardcopies collected.

16On the other hand, the CEDEJ has developed its collection on a daily basis since 1970. It has a higher printing quality than the Khayrallah collection, as the printing technology is considered relatively modern during this period. Also, due to the previous expertise of the Bibliotheca Alexandrina in digitization of Arabic text, the specifications of the digitization process were established to produce the highest possible accuracy of Arabic OCR.

17Khayrallah collection is searchable only through keywords and metadata tags (Moise A. Khayrallah Center for Lebaneese Diaspora Studies/searching items), in contrast to the features of the Egyptian Press Archive of CEDEJ project, which enables searching both the full text and the metadata.

18Jrayed: is an attempt to digitize and mad make accessible an online collection of 156,287 pages, from Arabic Periodicals between 1908 and 1948. Regarding the technical aspects of the project, they digitized the microfilmed periodicals in either binary or grey tone. While for colored paper materials, they were captured in color to keep the loyalty to the originals, with a varying of resolutions between 300 dpi to 400 dpi depending on the level of details in the original material.

19As their collection is very old, Jrayed decided not to use OCR technology, which would result in low OCR accuracy of about 20% to 30% success rate, making it inadequate for performing text searches. However, they chose to incorporate manual typing of article headlines and author names, providing partial search features, although it is not optimal in comparison to full OCR.

20The searching tools provided on the website rely on precise typing of the original headlines as printed in the source, for example (ال) has to be written to match the searching text, if it exists in the source. Also, due to the evolution of spelling over decades (e.g. the word “انجليزية” was written as “انكليزية” at that time), users must try different spellings to match the exact word.

Evaluation of Arabic OCR Systems Attempts

21As the OCR technology keeps evolving and as more case studies are emerging, there have been several attempts to evaluate Arabic OCR systems. We explore here two of them, then we discuss our evaluation method and describe the parameters to select and prepare our dataset for the newspapers case study.

22Mansour Alghamdi and William Teahan (Alghamdi, Teahan 2017) used the public KAFD dataset, developed by Luqman, Mahmoud, Awaida et al. in 2014. It is an Arabic printed text page image with the corresponding ground-truth text files. For the evaluation, they chose samples of 10 fonts of different shapes varying from simplicity to complexity, also font sizes: 8, 12, 18 and 24 were used. Within the previous criteria, 80 random TIFF images of resolution 100, 200, 300 dpi were chosen. They collected 240 samples dataset to evaluate four Arabic OCR systems: Sakhr Automatic Reader, Abby FineReader, RDI and Teseract. They introduced also, 7 performance metrics to evaluate them: “Overall character accuracy”, described in equation (1).

23Here m is the number of characters in the ground-truth and e is the number of erroneous characters. In addition, we may find: “Character accuracy based on character position”, “Dot character accuracy and no-dot character accuracy”, “Dot character accuracy based on baseline”, “Zigzag-shaped character accuracy”, “Loop-shaped character accuracy”, “Diacritics accuracy”, “Digit accuracy” and “Punctuation accuracy”. Their study showed that low resolution reduces recognition accuracy. Abby FineReader and Sakhr were ranked as the best two OCR suites at resolution 300 dpi, respectively, while all Arabic OCR systems’ performance accuracy rates were below 75% for all the matrices. The strength of this study is that it shows the impact of each of the Arabic script specifications on the recognition accuracy. This may be useful to list the strengthes and weaknesses for each of the evaluated OCR suites.

24Alkhateeb et al. (Alkhateeb, Abu Doush, Albousoul 2017) evaluated six Arabic OCR systems: Abby FineReader, Leadtools, Readiris, Sakhr Automatic Reader, Teseract and NovoVerus. They used 250 images from the public Arabic printed Text Image APTI dataset, where each text image contains only one single Arabic word, in addition to 8 images from an Arabic book (45,313,600 word images), to evaluate the six OCR suites. They conducted a test using the character error rate:

25where CER is the sum of Inserted characters (), Deleted characters () and the substituted characters (S ) divided by the total number of characters in the ground-truth (τ). They also proposed a new measure for Arabic language, which defines a new metric to detect a letter recognition accuracy according to its position: isolated, one way connected and two way connected characters.

26Based on the Character Error Rate (CER – equation 2), Abby FineReader, Readiris, Verus and Sakhr were ranked as the best four Arabic OCR suites respectively.

The CEDEJ dataset

27The CEDEJ dataset spans a large variety of good, degraded, different text sizes and shapes in the same page. Some samples had text written on a shaded background, others were printed on clear background and most of the samples were printed in low-tech economic printing. Some of them had condensed lines (and tend to have no line spacing, as shown in figure1-a), which makes line segmentation very difficult, while other samples had hazy printing (figure 1-b), leading to unrecognizable character dots, cut and/or hazy characters. Other samples had stained pages due to storage and age.

Figure 1. Samples of bad quality newspaper press-clips.

Figure 1. Samples of bad quality newspaper press-clips.

28To evaluate the performance of the digitization process, we consider mainly if the accuracy of the extracted text fits to the target of the project. This paper is a guide on how to deal with the immaturity of Arabic OCR studies in the case of a degraded text collection. Our aim is to find a productive, yet accurate, OCR approach that may handle such widely varying collection to produce a fully searchable text collection available to the public researchers. Considering the immaturity of Arabic OCR studies, we decided to adopt the Character Error Rate (CER) rather than the Word Error Rate (WER), where WER is usually twice the CER (Alkhateeb, Abu Doush, Albousoul 2017).

29To decide about image capturing specifications, we adopted the U.S. National Archives and Records Administration guide. The guide refers to the Quality Index formula (QI) for scanning text (Moving Theory into Practice: Digital Imaging Tutorial), designed by Cornell University and relates quality (QI) to character size (h) in inch and resolution (dpi), where good quality index starts from 5.0 and excellent quality index starts from 8.0. Equations (3) and (4) show the relation between the quality index formula for black and white images (QIbw) and for gray images (QIgr).

30QIbw = (dpi × ℎ) ⁄ 3 Équation (3)

31QIgr = (dpi × ℎ) ⁄ 2 Équation (4)

32As the smallest character height in the samples was 0.1 inch, we decided to do our trials with 300 dpi and 600 dpi resolutions, which led to the QI values shown in table 1 with reference to equations (3, 4).

Table 1. QI values of dataset images

300 dpi 600 dpi
Black and white QIbw = 10 QIbw = 10
Gray Scale QIgr= 15 QIgr= 15

33The above values show that the chosen resolutions (300 & 600 dpi) are enough to provide excellent quality images.

34Regarding Image Manipulation, with respect to National Archives and Records Administration (NARA) image processing guidelines (Puglia, Reed, Rhodes 2004), only image cropping, adjusting color levels (optionally) and unsharpen filter are applied manually for individual gray scale images, while post-scanning black and white image manipulation consisted of automated basic image processing options, using ScanFix, that may be applied in batch. This includes deskew, noise removal, sand and fill, to fix eroded or smudged characters. For Arabic script nature described earlier, noise removal and sand/fill options had to be applied carefully to keep dots, diacritics and fine details of character structure such as teeth and closed ring characters. As image manipulation of Grayscale is almost manual for productivity purpose, a bitonal version of captured images was produced, using moderate threshold to keep Arabic script details, so that normal bitonal image processing steps described earlier could be applied. This spared the time of manually adjusting levels of grayscale images.

35Optical Character Recognition (OCR): This is the core phase of our evaluation experiment. The previous phase produced six versions for each press-clip image: Gray, Black and White and Gray converted to Black and White (as described earlier). 300 dpi and 600 dpi versions were produced for each of the three color-mode versions. The six versions were fed to the three OCR suites subject to evaluation. Table 2 shows the 18 versions of each press-clip sample, where the first two characters of each sample define the OCR suite: Automatic Reader (AR), FineReader (FR) and NovoVerus (VR). The second two characters of the name stands for the color mode: grayscale (gr), black and white (bw) and gray converted to black and white (gb). The last letter of the name corresponds to the resolution: 3 for 300 dpi and 6 for 600 dpi.

Table 2. Sample naming convention

300 dpi 600 dpi
AR ARgr3 ARbw3 ARgb3 ARgr6 ARbw6 ARgb6
VR VRgr3 VRbw3 VRgb3 VRgr6 VRbw6 VRgb6
FR FRgr3 FRbw3 FRgb3 FRgr6 FRbw6 FRgb6

36In our study, we noticed that stacked lines segmentation failure occurred more frequently when FineReader was used, as shown in figure 2. Figure 3 shows how these four words stacked lines were recognized in Automatic Reader with no errors, while NovoVerus produced two errors (in other character areas). On the other hand, FineReader recognized only one word but misrecognized the words in stacked lines.

Figure 2. Line segmentation error in FineReader

Figure 2. Line segmentation error in FineReader

Figure 3. Recognition resulting from stacked line segmentation

Figure 3. Recognition resulting from stacked line segmentation

37To evaluate recognition accuracy, we used an equation (1) for each sample, where accuracy was recorded for each OCR suite on every sample version to sum up six groups (table 2) for the 180 samples tested on the three OCR suites. Figure 4 shows a box plot of the accuracy results of the three OCR suites for each of the six versions of the 30 samples. Table 3 shows the color mode and resolution that produced the highest stable accuracy for each OCR suite.

Table 3. Median accuracy and standard deviation

Color Mode Resolution Median Accuracy Standard deviation
Novo Verus Gray to bitonal 300 dpi 99.19 % 0.0089 %
Automatic Reader Gray 300 dpi 98.46 % 0.00241 %
FineReader Gray 600 dpi 97.14 % 0.00412 %

38In contrast to the previous evaluation attempts, which reported that FineReader realized thebest accuracy on their datasets, our evaluation in the case of degraded quality press clips dataset, shows that NovoVerus ranked first compared to Automatic Reader and FineReader, in terms of median accuracy and standard deviation on all samples. Based on the previous results, all CEDEJ press clips were captured and manipulated with reference to the following settings: gray scale 300, converted to black and white, cleaned with ScanFix then fed to NovoVerus. Figure 5 shows that the normal distribution of the realized accuracies in the CEDEJ project press-clips have a median of 98.25% with standard deviation 2.3%.

Figure 4. Comparing accuracies of samples

Figure 4. Comparing accuracies of samples

Figure 5. Normal distribution of CEDEJ press-clips accuracy

Figure 5. Normal distribution of CEDEJ press-clips accuracy

Interoperability as a challenge

  • 2 Compatibility means the ability of two systems to communicate unambiguously while interoperability (...)

39Interoperability and/or compatibility were additional challenges that we faced with the CEDEJ press archive project. Interoperability is more robust than compatibility2. In order to guarantee interoperability, we have to use open file formats, i.e. file formats whose specifications are known and accessible to all.

40The French law of 21 June 2004 for confidence in the digital economy gives a precise definition of an open format: Open format means any communication, interconnection or exchange protocol and any interoperable data format whose technical specifications are public and without restriction of access or implementation.

41Interoperability: various procedures allowing the different tools to communicate together. It is based on one open format (such as W3C-compliant HTML) or several such formats, independent of the software used to create and use it; these open formats easily provide tools to switch to other open formats (PDF to HTML for example), and/or enhancements (different versions of HTML to XHTML).

42The interoperability of digital content (images and text) is the main issue in the case of the press clippings digital archive. The digitized and preserved newspaper clippings and the version published on the web are in a standard file format (open format). The same applies to the metadata, which are based on the METS and MODS standard to create an XML value for data exchange, which is the only data format that allows interoperability as it allows linking between the different existing systems.

43However, this interoperability confronts us to other challenges:

  • First, a legal challenge, which is the willingness of the actors to share their data and to design their databases accordingly.
  • The second challenge is the implementation of the “exchange standards”: The Semantic Web provides tools that might at first sight seem complex, but we could quite easily integrate these technologies and derive substantial benefits for our data access portals.

44Difficulties do exist, however: difficulties in standardization and in adapting to actual use, among others. The work is heavy and the Return on Investment (ROI) is not always there. In addition, there is sometimes the need for manual reprocessing, but Artificial Intelligence (AI) can come into play to qualify or requalify the data and provide adequate metadata.

45Within a year, the data of the corpus of press clippings will be harvested.

Conclusion

46Digitizing a collection like CEDEJ Egyptian press archive was very challenging in terms of management and tracking, OCR and interoperability.

47All Arabic OCR suites evaluation attempts agreed that this domain of science is immature. However, considering the image capturing settings and pre-OCR image manipulation procedures would help enhance OCR accuracy.

48Due to Arabic OCR immaturity, we cannot evaluate Arabic OCR suites in general, but it would be better to evaluate them with reference to each collection’s characteristics.

49In case of collections having a wide range of characteristics, the standard deviation of realized accuracy must be considered to evaluate the specifications and the processing procedure to be adopted.

50By applying these concepts on the Egyptian press Archive of CEDEJ collection, we produced six versions of each press-clip (30 samples). After evaluation of the three Arabic OCR suites (Automatic Reader, Fine Reader and NovoVerus), we concluded that NovoVerus realized the highest and most stable accuracy with reference to widely varying characteristics of the collection. All press-clips were scanned as gray scale, 300 dpi, then converted to black and white and manipulated by deskew, noise cleaning. This ended up producing an image on text digital version of the collecting of a median accuracy 98.25% with standard deviation 2.33 %.

51On the scientific side, the BA plans to build similar dataset of newspapers in the form of images with different specifications, paired with its ground truth text. This would help researchers working on the Arabic OCR domain to evaluate their work.

52On the implementation side, Artificial Intelligence (AI) can come into play to qualify or requalify the data and provide more adequate metadata.

Haut de page

Bibliographie

Alkhateeb F., Abu Doush I., Albousoul A. 2017, “Arabic optical character recognition software: A review”, Pattern Recognition and Image Analysis, vol. 27, no. 4, p. 763-776.

“Egyptian Press Archive of CEDEJ”, URL: http://cedej.bibalex.org/, consulted on 27/07/2019.

Jrayed/about the initiative, URL: https://web.nli.org.il/sites/NLIS/en/jrayed/Pages/about.aspx, consulted on 27/07/2019.

Al-A‘ali M., Ahmed J., 2007, “Optical Character Recognition System for Arabic Text Using Cursive Multi-Directional Approach”, Journal of Computer Science, vol. 3, no. 7, p. 549-555, URL: https://thescipub.com/abstract/jcssp.2007.549.555, consulted on 11/12/2020.

Alghamdi M., Teahan W. 2017, “Experimental evaluation of Arabic OCR systems”, PSU Research Review, vol. 1, no. 3, p. 229-241, URL: https://www.emerald.com/insight/content/doi/10.1108/PRR-05-2017-0026/full/html, consulted on 11/12/2020.

Eldakar Y., El-Gazzar K., Adly N. et al., 2005, “The Million Book Project at Bibliotheca Alexandrina”, Journal of Zhejiang University-SCIENCE, vol. 6, p. 1327-1340.

Luqman H., Mahmoud S.A., Awaida S., 2014, “KAFD Arabic font database”. Pattern Recognition, vol. 47, no. 6, p. 2231-2240 URL: https://doi.org/10.1016/j.patcog.2013.12.012.

Moise A. Khayrallah Center for Lebanese Diaspora Studies, URL: https://lebanesestudies.news.chass.ncsu.edu/2018/01/10/archive-spotlight-historic-arabic-newspaper-digitization-project/, consulted on 27/07/2019.

Moise A. Khayrallah Center for Lebaneese Diaspora Studies/searching items, URL: https://lebanesestudies.omeka.chass.ncsu.edu/items/search, consulted on 27/07/2019.

Moving Theory into Practice: Digital Imaging Tutorial, URL: http://preservationtutorial.library.cornell.edu/conversion/conversion-04.html, consulted on 04/08/2019

Puglia S., Reed J., Rhodes E., 2004, “Technical Guidelines for Digitization Archival Meterials for Electronic Access: Creation of Production Master Files – Raster Images”, U.S. National Archives and Records Administration (NARA), p. 38-41, URL: https://www.archives.gov/files/preservation/technical/guidelines.pdf, consulted on 11/12/2020.

Samir et al. 2015, “Digital Preservation: Handling Large Collections Case Study: Digitizing Egyptian Press Archive at Centre for Economic, Judicial, and Social Study and Documentation (CEDEJ)”, Conference: ICADLA 4: International Conference on African Digital At: Alexandria Egypt, published January 2015, URL: https://www.researchgate.net/publication/344227529_Digital_Preservation_Handling_Large_Collections_Case_Study_Digitizing_Egyptian_Press_Archive_at_Centre_for_Economic_Judicial_and_Social_Study_and_Documentation_CEDEJ, consulted on 27/07/2019.

Haut de page

Notes

1 We thank our colleagues in the Digitization Operations section at Bibliotheca Alexandrina; Rasha F. Alzarka, Ayat E. Osman, Pansee E. Elkastawy, Sally M. Soliman, Fatma Elzahraa E. Elgammal, Tamer K. Elhamamy and Haiman M. Hanna, for their efforts and contribution. They were the main source of information used in this evaluation, by doing the real work of preparing the dataset, image capturing and image manipulation, as well as OCRing the dataset and calculating the accuracy of all samples on each OCR suite.

2 Compatibility means the ability of two systems to communicate unambiguously while interoperability means the ability to make two systems compatible.

Haut de page

Table des illustrations

Titre Équation (1)
URL http://journals.openedition.org/ema/docannexe/image/13156/img-1.jpg
Fichier image/jpeg, 15k
Titre Équation (2)
URL http://journals.openedition.org/ema/docannexe/image/13156/img-2.jpg
Fichier image/jpeg, 17k
Titre Figure 1. Samples of bad quality newspaper press-clips.
URL http://journals.openedition.org/ema/docannexe/image/13156/img-3.jpg
Fichier image/jpeg, 235k
Titre Figure 2. Line segmentation error in FineReader
URL http://journals.openedition.org/ema/docannexe/image/13156/img-4.jpg
Fichier image/jpeg, 67k
Titre Figure 3. Recognition resulting from stacked line segmentation
URL http://journals.openedition.org/ema/docannexe/image/13156/img-5.jpg
Fichier image/jpeg, 218k
Titre Figure 4. Comparing accuracies of samples
URL http://journals.openedition.org/ema/docannexe/image/13156/img-6.jpg
Fichier image/jpeg, 299k
Titre Figure 5. Normal distribution of CEDEJ press-clips accuracy
URL http://journals.openedition.org/ema/docannexe/image/13156/img-7.jpg
Fichier image/jpeg, 237k
Haut de page

Pour citer cet article

Référence papier

Rami Khalil Rouchdi, Mohamed Ahmed Ellotf et Hala Bayoumi, « Egyptian press archive of CEDEJ. A challenging case study of Arabic OCR »Égypte/Monde arabe, 22 | 2020, 41-55.

Référence électronique

Rami Khalil Rouchdi, Mohamed Ahmed Ellotf et Hala Bayoumi, « Egyptian press archive of CEDEJ. A challenging case study of Arabic OCR »Égypte/Monde arabe [En ligne], 22 | 2020, mis en ligne le 03 janvier 2022, consulté le 01 décembre 2023. URL : http://journals.openedition.org/ema/13156 ; DOI : https://doi.org/10.4000/ema.13156

Haut de page

Auteurs

Rami Khalil Rouchdi

B.Sc. of Computer Engineering, 1997 from Arab Academy for Science and Technology. He has been working for the Bibliotheca Alexandrina (BA) since 2002 in the Information and Communications Technology (ICT) department. Since 2005, he was involved in the field of digitization with special focus on Arabic text. He oversaw the Digitization Laboratory Unit since 2005 till 2011, then was assigned to lead the digitization operations Section to date. He follows and investigates about the state of art techniques to digitize different types of Arabic textual documents. He cooperated in many large-scale national and international digitization projects at the BA. To share the BA expertise, he is responsible of organizing trainings about the digitization of Arabic contents for organizations, among which Yale University, AUB, BRILL publishing house and Kuwait Institute for Scientific Research (KISR). Recently he developed a mass production workflow that integrates the digitization technologies of Arabic text to produce electronic Arabic books in ePub format suitable for smart phones.

Mohamed Ahmed Ellotf

He has got a B.Sc. of Computer Engineering, 2003 from Alexandria University and a MBA, 2016 from Arab Academy for Science, Technology (AAST). He started his tenure at the Bibliotheca Alexandrina (BA) in 2003 as a system administrator then served in several managerial positions within the ICT sector until he occupied his current position as Director of the ICT Infrastructure & Operations Department in October 2016. He is responsible for leading, developing and maintaining the library ICT infrastructure interoperability and operations management thus contributing to the fulfillment of main objectives of the BA as a Library in the digital age. His responsibilities encompass directing system acquisition, installations, server management, system support and maintenance, managing network and communication systems, voice, data and office automation. He also participated in several endeavors pursued by the BA such as the technical infrastructure implementation of the Encyclopedia of Life project and the Kuwait Institute for Scientific Research digitization system.

Hala Bayoumi

Dr Hala Bayoumi is a research engineer at the CNRS France (National Centre for Scientific Research) and head of the Digital Humanities cluster at the CEDEJ. She holds a Ph.D. in Computer Science from EPHE (Paris, France) and a habilitation in Geography from University Jean Moulin (Lyon, France). She is the winner of the CNRS 2017 crystal medal, and ambassador of the program “La Science taille Xxelle”. She published different books and websites and co-edited recently with Karine Bennafla Atlas de l’Égypte Contemporaine, 2020. She coordinated since 2010 a Master’s level diploma in Data Analysis and Spatial Analysis Techniques (GIS 4D) for Egyptian officials from CAPMAS (inter-university diploma EPHE-university of Lyon).

Articles du même auteur

Haut de page

Droits d’auteur

CC-BY-NC-SA-4.0

Le texte seul est utilisable sous licence CC BY-NC-SA 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.

Haut de page
Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search