Tuesday, March 5, 2019

Reading Handwritten User choice with OCR (Optical Character Recognition):

Reading Handwritten User choice with OCR (Optical Character Recognition):


Guyz,

In a day to day programming activities, you might have come across a requirement, where you must read the hand-written user choice in a form as shown in Fig 1,2,3 and 4.

The user selects his choice by marking or blurring his options with a pen, pencil, sketch or marker, and you cannot control thousands of users to have a unique way of input or unique marking tool.


Fig: Reading with OCR


Fig 1: Enhancement to ABC Project WI#999482:


Fig 2: TAG Inspection for SITE XYZ 003AC Project:



Fig 3: FIRE & SAFETY CHECKLIST BUILDING#002 STREET 22:

         


Fig 4: SAMPLE APPROVAL FORM FOR ABC ACTIVITY:



How to achieve this in C#?
               





The above flow diagram explains the complete steps to implement this requirement. Firstly, the scanned document containing user-entered choices is uploaded to the application, which is converted to image format and read using Microsoft office document imaging library.

You can find this DLL in Add Reference รจ COM components. If you can’t see it, you can download here.

After getting the OCR output, our internal logic is used to predict the exact user’s choice.

Let’s consider the scanned form in Fig 4, we will read it with OCR.

To Start, including the special libraries as below.


Step 1:

using iTextSharp.text.pdf;
using iTextSharp.text.pdf.parser; // Used to convert the scanned PDF to JPEG Image
using MODI; // Used to Read the Image with OCR

Step 2:

The below method is to extract the image from a scanned document.


/// <summary>Extracts all images (of types that iTextSharp knows how to decode) from a PDF file.</summary>
public static Dictionary<string, System.Drawing.Image> ExtractImages(string filename)
{
    try
    {
        var images = new Dictionary<string, System.Drawing.Image>();

        using (var reader = new PdfReader(filename))
        {
            var parser = new PdfReaderContentParser(reader);
            ImageRenderListener listener = null;

            for (var i = 1; i <= reader.NumberOfPages; i++)
            {
                parser.ProcessContent(i, (listener = new ImageRenderListener()));
                var index = 1;

                if (listener.Images.Count > 0)
                {
                    Console.WriteLine("Found {0} images on page {1}.",
                        listener.Images.Count, i);

                    foreach (var pair in listener.Images)
                    {
                        images.Add(string.Format("{0}_Page_{1}_Image_{2}{3}",
                            Path.GetFileNameWithoutExtension(filename),
                            i.ToString("D4"), index.ToString("D4"), pair.Value), pair.Key);
                        index++;
                    }
                }
            }
            return images;
        }
    }
    catch {
        Console.WriteLine("Cant read file :"+ filename);
        return "";
    }
}

** Note the scanned PDF will always have an image result, the above method is to extract images from the scanned PDF document.


If you already have an Image file, then skip the above Step.

Step 3:

After getting the Image file, let's read it with our Microsoft Document Imaging library as shown:

private static string ExtractTextFromImage(string imagePath)
        {
            Document modiDocument = new Document();
            modiDocument.Create(imagePath);
            modiDocument.OCR(MiLANGUAGES.miLANG_ENGLISH);
            MODI.Image modiImage = (modiDocument.Images[0] as MODI.Image);
            string extractedText = modiImage.Layout.Text;
            modiDocument.Close();
            return extractedText;
        }


For Fig 4 it returned the result as:








Step 4:

I have written a small logic to extract a portion of text.  You can use it by defining start text and end text in the start_token , end_token variables, which gives you the entire text in-between. However, If you want to process the entire text, then keep these variables empty.

/*Start Custom Logic to read between start and end token*/
string start_token = "";
string end_token = "";
string result = "";
if (!string.IsNullOrEmpty(start_token) && (!string.IsNullOrEmpty(end_token))
{
  String line;
  String text = extractText;
  StringReader reader = new StringReader(text);
  while (!(line = reader.ReadLine()).Equals(start_token))
  {
   //ignore
  }
  while (!(line = reader.ReadLine()).StartsWith(end_token))
  {
      result += line;
  }
}
else
{
    result = extractText;
}


Step 5:

Now, let’s write the logic to detect what is the user choice. For reading SUBCONTRACTOR choice, in the below code we are getting the options between SUBCONTRACTOR & CONTRACTOR text, and processing it with CheckForTickMark method for APPROVE/REJECT.

string subconSign = Between(result, "SUBCONTRACTOR", "CONTRACTOR");
string subconApprove = Before(subconSign, "APPROVE", 1);
Console.WriteLine("Subcontractor : Approve:"
+ CheckForTickMark(subconApprove));

string subconReject = Before(subconSign, "REJECT", 2);
Console.WriteLine("Subcontractor : Reject:"
+ CheckForTickMark(subconReject));

/// Repeat the same for other options
/// Method containing Internal Logic to Detect User Choice
public static string CheckForTickMark(string a)
{
if (string.IsNullOrEmpty(a))
    return "Ticked";

if (a.Equals("na"))
    return "Ticked";

a = a.Replace("\r","").Replace("\n", "").Replace(" ", "");
if (string.IsNullOrEmpty(a))
    return "Ticked";
else if (a.Equals("D") || a.ToUpper().Equals("E") ||
        a.ToUpper().Equals("L") || a.ToUpper().Equals("EI")
        || a.ToUpper().Equals("EL") || a.ToUpper().Equals("FL")
        || a.ToUpper().Equals("FI") || a.ToUpper().Equals("LI")
        || a.ToUpper().Equals("D!") || a.Equals("LI")  ||
        a.ToUpper().Equals("DI") || a.ToUpper().Equals("D|")
        || a.ToUpper().Equals("E|")
        || a.ToUpper().Equals("L|") || a.ToUpper().Equals("VE")
        || a.ToUpper().Equals("''") || a.Equals("'")
        || a.Equals("0") || a.ToUpper().Equals("O"))
{ return "NotTicked"; }
                  
return "Ticked";
}


Result:



My logic for detecting the user choice is I will compare the checkbox interpreted text with few standard OCR outputs for a checkbox which are nothing but characters [LI, EI, E, DI..etc], if my OCR library gives checkbox interpretation other than the defined standard characters then it means that something happened here i.e it is Ticked.

To increase the accuracy, I have collected different format user inputs through pen, pencil, sketch, and marker to tune up exact perdition in CheckForTickMark method. The application is tested with different inputs giving 99% perfect results.

However, to increase more accuracy and to prevent any distortion, I would suggest you to impose a few limitations as a NOTE:

  •          Selection of user choice (checkbox) should be a tick mark with Pen or Pencil.
  •          Form columns/row lines should not be disturbed while marking.

The code is checked for different user scanned inputs. You can download the sample app here.

Hope you enjoyed.  Drop your query’s/comments in the box to answer.


Wednesday, May 9, 2018

Angular Performance Improvement & Reducing CPU Usage Techniques

Angular Performance Improvement & Reducing CPU Usage Techniques

Folks ! after going in deep to improve my angular 4 application performance and to reduce its CPU consumption , I landed with the below points, which is nothing but the summary in nutshell.

   

   1)  Need to remove unnecessary components/DOM elements from the page
(Include child components/DOM elements only when its required)

Can see in the screen I am using the variable  addExtracomponents to include the components only when its required

   2) Releasing memory for subscribers, events, intervals.
Use: ng2-rx-collector, which auto release the memory after usage.

   3)  Functions declared in *ngIf are called on every change detection cycle, angular checks the expression in the *ngIf for changes by calling the declared function multiple time.
One way to avoid is by putting a class variable in your function, which override the function manipulation every time , which is of course a performance boost.
*ngIf="isTrue" //(variable approach)
*ngIf="fun()"   //(function approach)

   4)  Using ngFor with trackBy , Since ngFor will continuously generates the DOM, the presence of trackBy would reduce the DOM manipulations, will be help to ngFor, it modifies a behaviors so that it compares new data to old based on the return value of the supplied trackBy function.

  Example:
     In .html (view):
            <ul>
                    <li *ngFor="let emp of empList; trackBy: trackByName" >
                   <span>Employee Name: {{ emp.name }}</span>
                    </li>
                  </ul>

              In .ts(component):
          // Treat the emp name as the unique identifier for the object
         trackByName(index, emp) {
            return emp.name;
          }

   5) Use the OnPush change detection strategy to tell Angular 2 there have been no changes. This lets you skip the entire change detection step on most of your application most of the time.
dashboard)

   6) Clearing the declared arrays/objects in ngOnDestroy (Other than Point#2)
         
  Example:
        public displayAttachments: RFIAttachments[]; //declared variable
            //declared interval
        this.f1();
        this.id = setInterval(() => {
              this.f1();
         }, 5000);

         ngOnDestroy() { //clearing all
              this.displayAttachments = null;
              if (this.id) {
                clearInterval(this.id);
              }
        }
   7) Need to analyze run time performance with my favorite chrome tool and apply a fix   where you find the cause.



   8) If your using Angular KendoGrids which takes too more CPU usage when dataStateChange event is used, to avoid you can use Kendo Built in directive [kendoGridBinding] which take care of all the operations (filtering, sorting, paging) by itself with less CPU consumption, or else you can go with separate events for each operation
Example:
 <kendo-grid
     [kendoGridBinding]="gridData"
     [pageSize]="10"
     [pageable]="true"
     [sortable]="true"
     [filterable]="true"
            [groupable]="true"
            [height]="510">
            <kendo-grid-column field="CompanyName" [width]="140"></kendo-grid-column>
            <kendo-grid-column field="ContactName" [width]="120"></kendo-grid-column>
            <kendo-grid-column field="City" [width]="100"></kendo-grid-column>
            <kendo-grid-column field="ContactTitle" [width]="130"></kendo-grid-column>
        </kendo-grid>

   9)  Give deployment build with ng build --prod --build-optimizer
It will do the bundling, pre-compilations, removes the dead code and fast rendering with AoT (ahead of time).

Can read more info at: https://angular.io/guide/deployment

Its always better idea to take care of above points from the start of development , It may be difficult to apply all the above points to already developed application.

Thanks.