Went out for bread - bought a house: augmented reality as the future of banking

The idea for the post was born thanks to research by Infosys, in which it systematized the use of AR in the banking and insurance industries. The authoritative research company IDC predicts that in the next four years, global VR / AR market profits will grow at least twice a year. There is an even more accurate forecast - from $ 11.4 billion this year to almost $ 215 billion in 2021. And after 2018, augmented reality will take a large share in the VR / AR duet. We decided to present and evaluate the scenarios of its application in the financial sector, which is closest to us.

A bit about technology

Since the mid-2000s, there has been a steady tendency to transfer online banking services. Here AR is the next stage of technological development, bringing banking applications to a new level of comfort and visibility, and AR also opens up new scenarios and benefits for users.

We’ll evaluate from a technical perspective how a typical AR script is implemented using the Apple ARKit framework. In augmented reality, the main tool is a smartphone. Through it, using the ARWorldTrackingConfiguration base class, the augmented reality application detects flat surfaces in the image captured by the camera, and also tracks the movement of the device in three-dimensional space with six degrees of freedom (6DOF). The relative position and orientation of the planes are then analyzed using ARPlaneAnchor. It is worth highlighting the ARFaceTrackingConfiguration class, created specifically for working with faces using the TrueDepth cameras of the new iPhone X.

After analyzing and processing information about the real world, you can proceed to work with virtual objects. For integration into a real image of three-dimensional objects created using SceneKit, the ARSCNView class is used. Binding SceneKit objects to specific real-world objects is also possible - using the ARSCNViewDelegate protocol. To integrate two-dimensional objects created on SpriteKit, the ARSKView class is used.

The ARFrame class, which is responsible for collecting information about a real picture in the camera lens, can work not only with spatial parameters, but also collect data on illumination. They are taken into account when rendering 3D objects, which makes them more realistic.

Based on this general scheme, specific scenarios are created in the banking industry. Some of them are already familiar, but some look original and promising.


The most simple and therefore well-developed scenario today. The algorithm of actions in this case is quite simple:

  • The user points the phone in the right direction.

  • The image shows the nearest ATMs / bank branches.

  • The user selects an ATM and gets to the place using virtual signs.

Many banks are experimenting with similar features. The Dutch bank ING and the Spanish Bankinter have already implemented them in their applications.

Planning a visit to the bank

This is a complicated version of the previous scenario. Suppose you need to meet with a bank representative in the nearest branch:

  • You turn the phone in the right direction.

  • Choose the desired compartment from those that are displayed there.

  • Get in the virtual queue, take the number, get the approximate waiting time ...

  • ... and do not roll your eyes in the stuffy room of the bank!

Details of offers

This option is already actively used not only in the banking industry. For example, Delaval, an agricultural equipment company, builds 3D models of farms using augmented reality. Here the principle of action is quite simple:

  • You direct the phone to the booklet / billboard of interest or other advertising medium.

  • The system recognizes the photo proposal and sends you detailed information in response.

Personal information can also be obtained in the same way. For example, this is the interface of the Westpac New Zealand Bank application, where you can clearly see the balance of your cards, the payment schedule, the services used and the accumulated bonuses.

Buying a property

British Halifax and Australian St. George Bank with the help of AR allows potential home buyers to view data on objects directly on the street:

  • You point the phone in the right direction.

  • You see on the screen data on the value of real estate, the number of offers.

  • Customize the offer using the connected mortgage calculator.

  • Discuss with the manager all the formalities and send a request to the bank.

  • Using biometric authentication tools (voice, iris), the bank confirms your identity and processes the request without losing time

Car purchase

A scenario that is close to buying a property, but more technological. Indeed, unlike the purchase of real estate, the system will be much more difficult to navigate by the location of the object:

  • You take pictures of the car (or read its unique identifier - a chip or a QR code), which you are going to purchase from a dealer, and send a request to the server.

  • You get all the information available on the car: the history of repairs and inspections, mileage, previous owners.

  • If you decide to buy a car, you send a request to the insurance.

  • The server calculates the cost of insurance and sends it back.

  • You contact the insurance company if the offer suits you.

  • You will immediately receive all the necessary documents.

Customer interaction

Augmented reality technology helps not only customers, but also bank employees. With the help of augmented reality, bank managers can receive information about the client directly during communication, plan work and make decisions based on this information:

  • The employee receives a video call from the client.

  • At the request of the client, the manager can plot a route for him.

  • The status of the client and information about it are displayed on the received image.

The employee can even track the location of customers for more convenient communication. Of course, from the point of view of the law, this is forbidden, but if you, as a client, without looking, have allowed the banking application to access information about your location, then do not be surprised.

Augmented reality has great potential in the financial industry. But to reveal it, it will require the development of other related technologies - machine learning and natural language. It’s hardly possible to change the world alone, but what do you think about this? Let's chat in the comments.