HOW TO: Poor man's ID scanner with automated data insertion into HD (it works!...sort of)

[ddam]

Hi all,

not sure wether I'm the only one, but I hate spending time registering client ID data for legal reports and have been thinking of a way to speed things up (NB: the script is still in debugging phase).

My set-up (how data goes from IDs into HD):
ID scanning app --> email (as "interface" with the scanning app) --> bash script --> HD

I'm using BlinkID from Microblink on Android. This app does a fairly good job scanning the Machine Readable Zone (MRZ) available on all modern IDs. I estimate accuracy to be well above 95%, as long as scanning conditions are taken care of:
- light (get enough of it)
- reflection (get rid of it)
- stabilisation (both the phone and the ID should stand still)
The best way to achieve this is to build some sort of stand, ideally with an integrated light source at the right angle (using the camera LED will produce reflection).

The result:
- scanning each ID takes about 3 seconds (provided scanning conditions are alright)
- at the end of the day, running the script will insert all data into HD
- from inside HD, I can then generate a document containing all the info for the authorities
- as a bonus, my client profiles are accurate and contain additionnal information that might prove usefull (e.g. gender, birthdate)

Installation
1. if you use Windows, you need to install WSL to use bash:
https://docs.microsoft.com/en-gb/windows/wsl/install-win10
2. create a modified lista_nazioni.php file, replacing two-letter with three letter codes, or use this one:
https://pastebin.com/v3Y1LeF8
This file is used to match country ICAO codes found on IDs with actual country names, for registration inside HD.
3. install curl, getmail and pcregrep
4. download the script from here: https://pastebin.com/Br67DYAJ and make it executable (i.e. chmod +x yourscriptname)
5. set-up getmail config file and needed directories (additionnal info inside the script; sample .getmailrc: https://pastebin.com/7hwr4UpZ)
6. install NetGuard firewall app on your android tab/phone (optional but recommended if you don't want to send your guests' ID data to Microblink)
7. install BlinkId app on your android tab/phone and set up NetGuard to forbid internet access to BlinkId
8. install and set-up k-9 mail on your android tab/phone (optionnal, but handy as it allows to specify a default Bcc address, saving the hastle of entering a recepient after every scan)

Usage:
Open BlinkId app and scan an ID
Share to k-9 mail, send and go back for next scan
Once all IDs have been scanned, run the script to insert the data inside HD

Final thoughts
This is a obviously a WIP. As stated inside the script, there are things left to be done. Furthermore, I'm not a bash expert, so there surely is room for improvements: if anyone has some corrections to submit, please do!
But Microblink's technology does work surpisingly well for OCR, and their SDK would allow for a nice app integrating into HD

EDIT 1 (01/03/18): new script version with some minor errors corrected. Also added pcregrep to installation.

[Niko18]

Hello Ddam


That's Great! Long time ago I was looking for something similar, an id scanner integration that would work with hotel druid, in such a way to automate the insertion of data from our customers on the platform, looking for the result that you propose improve the speed and reduce the margin of mistake.

Here I leave a link where I talked about something related

http://www.hoteldruid.com/forums/viewtopic.php?f=5&t=13470

Thanks

Niko

[ddam]

Long time ago I was looking for something similar

Hello Niko,

The professionnal ID scanners are not cheap (had found a 3M model for $500), that's why I have been looking for an alternative.
I have updated my OP with detailed info. Hope it helps!

Cheers,
ddam

[Niko18]

thx for the script , when I have sometime I will give it a try.

I have noticed that the microblink app doesn't show you the second surname and the address.


thanks

Niko

[ddam]

thx for the script , when I have sometime I will give it a try.

you're welcome I realise this setup is rather cumbersome. I'm thinking of an android only solution, getting rid of the whole email "interface" (using BlinkID + Tasker + IntentTask, for exemple).

I have noticed that the microblink app doesn't show you the second surname and the address.

the app will show whatever is inside the MRZ (the same way any other passport scanner would). As far as I know, the address is never included into the MRZ of any ID. Regarding second surnames, I guess it depends on legal requirements of the issuing country.

[Niko18]

It looks like you are already working with the app and HD, could make a video with your phone and upload it to youtube, i'm really curious about the way your solutions works.

[ddam]

i'm really curious about the way your solutions works.

It really is as simple as described in the Usage and Set-up sections above.
But I'm still testing, so cannot give any detailed feedback at this stage

[mdb91]

Glad to see someone else has that issue as well. I used to have a professional scanner of ID's. This seems like a good solution to provide faster check ins. I am no expert on coding/programming, but just wonder if it would be possible to join this script to a normal scanner, where the light and accuracy never changes thus should Always produce the best quality?

Thanks already so much for having shared this, I hope to integrate it soon.

[ddam]

just wonder if it would be possible to join this script to a normal scanner, where the light and accuracy never changes thus should Always produce the best quality?

The "heart" of an ID scanner is its OCR engine: as far as I know, there aren't any free ones available that would have as good a recognition rate as microblink's. But OCR is an ever evolving field, so maybe that will change soon.
As mentionned before, if you build some sort of stand where you can control the scanning conditions, the quality should be fine with BlinkId.

[ddam]

Reviving this old thread to give some feedback: I have given up on the bash script approach here-above, and have gone for a UIpath approach instead. It turns out to be easier to set-up, debug, maintain and share.

I also wanted to know if anyone else would be curious about a passport / ID scanning web-app, that would also allow for online check-in by guests themselves. If you are interested, please let the community know in the dedicated thread:
https://www.hoteldruid.com/forums/viewtopic.php?f=5&t=15449&sid=af2ee97c968c7e43aead5cdfe0d3a2d2