Past Client

AWS - Amazon Augmented Reality (A2I) Product Uses and Features

Amazon Augmented AI is a machine learning service which makes it easy to build the workflows required for human review. Amazon A2I brings human review to all developers, removing the undifferentiated heavy lifting associated with building human review systems or managing large numbers of human reviewers whether it runs on AWS or not.‍

Today’s autonomous machines are yet at the level where they can run on their own. There are certain technicalities and nuances in the way humans think and perform that are not fully comprehensible to AI.

Think about how many frustrating interactions you’ve had with a chatbot, only to get the same repetitive response: “I didn’t quite get that, try again.” Or the grammar corrector script that keeps adding an unneeded preposition to a phrase.

These are just a few examples of how bots don’t always get it right.

Humans can work in tandem with machines to increase the accuracy of certain information and results, especially as it pertains to sensitive information (ie. identity confirmation, financial statements, real estate contracts..etc). Humans can also input new information for the purpose of improving and retraining models with better predictions.

One problem is that these services require building human review systems and complex workflows, which end up being expensive to manage. Amazon Augmented Reality (A2I) solves this problem by removing the manual human review process from common ML applications, and automatically sending samples that don’t meet the minimum threshold to a human review team. A2I is able to sift high confidence results from low confidence ones, thus, reducing time and increasing accuracy for important documents.

How are thresholds calculated?

One of the outputs from A2I is a confidence score. This is used to determine whether a sample gets sent to the human review team or to the client application for model retraining. 

The threshold is established by a backend team and the metrics to analyze performance are set by a data scientist.

Predictions with below threshold scores are sent for manual review. Once the prediction is manually reviewed, this data point is sent back to the client application for retraining. Retraining means integrating this newfound information to improve future predictions with more accuracy.

Human Review Activation Conditions

Use A2I activation conditions to specify when a document is sent to humans for review and what content on the form needs to be analyzed.

Here are some examples of triggers you can set it to:

  • Trigger for specific form keys based on form key confidence score. Human reviewers are asked to review form keys and associated values.

  • Trigger human review when there are missing form keys. Human reviewers are asked to identify missing form keys and associated values.

  • Trigger review when form keys identified fall within a certain confidence range.

  • Send a random sample of forms to be reviewed. Human reviewers are asked to review all forms and key values detected by Amazon Textract.

Two Types of Threshold Conditions

  1. Identification confidence: confidence score for key-valued pairs detected within a form

  1. Qualification confidence: confidence score for text contained within a key-value pair in a form

Functionality Features

The Amazon A2I dashboard makes integration easy— and the platform contains step-by-step instructions on tasks for reviewers. 

For Reviewers

Amazon A2I enables you to work with reviewers both inside and outside of your company. There’s a private option and a public option. For the private option, you can use your own team of private reviewers. For the public option, you can utilize Amazon Mechanical Turk, especially if the data is not private and you need a large number of reviewers. Amazon Mechanical Turk provides a 24/7 workforce of over 500,000 independent contractors. You can also manually select how many reviewers you want.

Use cases for Amazon Augmented Reality

A2I is compatible with many of Amazon’s AWS products. In particular ML applications that have images, recordings, or transcripts that require review.

Amazon Textract: A2I can be used to review and sample important key-value pairs in single-page documents.

Amazon Rekognition: Unsafe images and explicit adult or violent content can be sent to A2I for review. If Amazon Rekognition returns a low confidence score it will be sent to humans for manual review.

Real-time ML inferences: Review low confidence inferences made by the model deployed to a SageMaker-hosted endpoint.

Amazon Comprehend: Have humans review Amazon Comprehend inferences about text data ie. sentimental analysis, text syntax, and entity detection.

Amazon Transcribe: Use the results of human-reviewed transcription scripts to create custom vocabulary and improve further transcriptions of similar video and audio content.

 

Amazon Translate: Review low-confidence translations.

Tabular data: Integrate a human review loop into an ML application that uses tabular data

Prices

There’s a standardized price per each item reviewed by a human (which can be an image, audio recording, a section of a text..etc).

There’s an extra fee per human evaluated object if you utilize AWS Marketplace Vendor or Amazon Mechanical Turk.

There’s also a different pricing scheme with custom models.

However, there’s no additional fee per reviewed object when you use your own workers to conduct reviews.

Conclusion

A2I is a cost-effective solution that reduces the burden of managing workflows where review processes are required.

Sources:

https://aws.amazon.com/augmented-ai/?did=ap_card&trk=ap_card

https://docs.aws.amazon.com/textract/latest/dg/a2i-textract-core-components.html

https://insaid.medium.com/introduction-to-amazon-augmented-ai-a2i-a-bird-view-800f445aa89d

https://insaid.medium.com/amazon-augmented-ai-in-depth-39c0009842e4