Automating Invoice Processing with OCR and Deep Learning

Client Requirements

The client is a USA-based specialist group dedicated to providing the best-personalized care for infectious diseases. They used to maintain the records from the invoices of all the inventories, such as medicines, equipment, etc., for further analysis of costs and sellers of the inventories so that they can filter the sellers and eliminate unnecessary costs from the inventories that can increase their profits. For collecting those records from the invoices, they used the manual data entry process to extract key information from the invoices for further analysis. That process was time-consuming, needed more human resources, and was not cost-effective.
Therefore the requirement was to develop an application using Artificial intelligence that can extract key information like Seller, Date, Invoice Number, Amount, and tax from the invoices of any format such as .pdf, .jpg, .jpeg, .png

Thinkitive Solution

Thinkitive's Data Scientists and domain experts team started with the requirement analysis (Discovery) phase. The Thinkitive team took multiple calls with the client and created a detailed SRS requirement document and the system's architecture to be developed. Thinkitive established a team of professionals with expertise in Artificial Intelligence Development. The Thinkitive team leveraged microservice-based architecture and developed separate microservices for each module, like data preprocessing, model training, flask API, and many more. The diagram below explains the workflow of the application.

Solution Highlights

• Optical Character Recognition(OCR):

  The application uses optical character recognition to recognize the text from the invoices and gives the text output, which gets preprocessed and then fed to the model for information extraction.

• Entities Annotations:

  In the model which extracts the key information from the invoices that need to In the model which extracts the key information from the invoices that need to be trained on our training data, we have used the NER annotator to annotate the training data as per the labels, namely, Name, Date, Invoice Number, Amount, and Tax.

• Models:

  In this application, we are leveraging two different A.I. models to ensure the key entities extracted from the invoices are accurate. Two different models are used in such a way that if one model cannot extract the key information, the other model can extract that key information. It will combine the output of both models and give the output with the best possible accuracy.

• Transfer Learning:

  Here we have used the transfer learning approach, transfer learning is an approach in artificial intelligence where we can leverage the pre-trained models by training them on our datasets, so here we have used a pre-trained model which was trained on a specific dataset previously; further, we have trained that model on our dataset to use that for our use case.

• Integration with Frontend Application:

  This entire solution has been integrated with a frontend application that was already used to extract the key information based on regex and invoice templates. That platform was valid for only certain types of invoice templates and if the input invoice template was not in their template, then it failed to extract the information.

• Application Input:

  This application is independent of the invoice templates, if the invoice templates will be different it still will be able to extract the information and also the application allows stakeholders to give the input of the invoices in any formats such as .pdf, .jpg, .jpeg, .png

• Application Output:

  The application allows stakeholders to extract the key information from the invoices in .json format.

  Output Example: {"Date": "2022-01-02T00:00:00", " Name": "xyz.", "Amount": "356.42", "Invoice Number": "525603"}

Value Delivered

• Enable stakeholders to extract the information from the invoices.

• It has increased the efficiency of the stakeholder in terms of data processing.

• Time consumption for the data processing was reduced.

• The solution also reduced the cost by reducing human intervention in the process.

• The solution caused a surge in profit for the client as the client was able to take the data-dependent decisions quickly.