Business Card CRM a showcase of modern browser technology: In this blog post I will give a short overview about my business card crm. It is a proof of concept for using almost serverless technology to solve an old problem.
What is the application good for?
This application can be used to make a photo of a business card. The image will be analyzed by the Google Vision API and afterwards the text will be feed into the Google Named Entity Recognition Engine. Based on some look up tables and regular expressions typically attributes like first name, last name, phone number and email address are extracted. The data will be saved in the local storage of the device. If multiple clients like a smartphone and a desktop PC want to share business card data they can join together through a WebRTC session and data can be send through this connection.
Video tour
Architecture
Source: Manuel Blechschmidt, created with Signavio Process Editor
Example image
The image above shows a typical business card like you can collect on a lot of different occasions. Typically these are entered manually into a CRM system. More advanced systems do employ ocr recognition technology to automate parts of this error prone process.
Google offers an API for analyzing such images and get the text. This API can directly be consumed with JavaScript after creating an access key.
The code looks in OpenUI5 like this:
var _gApiLoaded = new Promise(function(resolve) { jQuery.sap.includeScript("https://apis.google.com/js/api.js", "gapi", function() { gapi.load("client", function() { gapi.client.setApiKey(""); resolve(); }); }); }); // https://www.googleapis.com/auth/cloud-platform _gApiLoaded.then(function() { // https://developers.google.com/discovery/v1/reference/apis/list Just hit execute to get a list gapi.client.load("https://vision.googleapis.com/$discovery/rest", "v1", function() { var reader = new FileReader(); reader.readAsDataURL(oBlob); reader.onloadend = function() { var base64data = reader.result; gapi.client.vision.images.annotate({ "requests": [{ "features": [{ "type": "TEXT_DETECTION" }], "image": { "content": base64data.replace(/^data:image/jpeg;base64,/, "") } }] }).then(function(oResponse) { console.log({ "ImageBase64": base64data, "OCRResult": oResponse.result }); }); }; }); });
For the image above the result looks like this:
{ "responses": [ { "textAnnotations": [ { "locale": "en", "description": "Incentergy GmbH Manuel Blechschmidt CEO www.incentergy.de Phone: +49 173 632 26 21 Mail: [email protected] ", "boundingPoly": { "vertices": [ { "x": 42, "y": 151 }, { "x": 901, "y": 151 }, { "x": 901, "y": 661 }, { "x": 42, "y": 661 } ] } }, { "description": "Incentergy", ...
Afterwards the whole text is send to google NER engine.
{ "entities": [ { "name": "CEO", "type": "PERSON", "metadata": {}, "salience": 0.3584892, "mentions": [ { "text": { "content": "CEO", "beginOffset": 36 }, "type": "COMMON" } ] }, { "name": "Phone", "type": "CONSUMER_GOOD", "metadata": {}, "salience": 0.2520285, "mentions": [ { "text": { "content": "Phone", "beginOffset": 58 }, "type": "COMMON" } ] }, { "name": "Incentergy GmbH Manuel Blechschmidt", "type": "PERSON", "metadata": {}, "salience": 0.22339708, "mentions": [ { "text": { "content": "Incentergy GmbH Manuel Blechschmidt", "beginOffset": 0 }, "type": "PROPER" } ] }, ...
After letting Google do the hard lifting of OCR and NER the application find with some basic rules the correct strings.
Screenshots
SAP Web IDE
The application was completely develop in SAP Web IDE and published to github.
WebRTC
For synchronizing data between different devices the App can create a RTC session. For the details please consult the source code and the video shown above.
Summary
This applications shows what is possible with the newest HTML5 technology and machine learning including OpenUI5. It uses WebRTC, OpenUI5, LocalStorage, Google Vision API.
Business Card CRM a showcase of modern browser technology
