Has screen scraping seen its day?

In the banking world, screen scraping has been around for over 25 years – it’s how many fintechs got their start. Let’s look at how screen-scraping works, why it can be problematic and even risky, and how open banking is the secure solution needed to move beyond screen scraping.

What is screen scraping?

Screen scraping (also known as web scraping) is what a developer might do to get access to information that’s usually only shared via a webpage. It works by copying information displayed on a screen (such as text or images) and using it for another purpose.

To put it simply, web scraping is driven by bots/web crawlers. They work in the same manner as search engines do – Fetch and Copy. But web scraping focuses on retrieving particular data from a website, whereas search engines frequently scrape the majority of webpages on the internet.

In its early days, companies used web scraping to take advantage of their competition. If you wanted to keep tabs on a competitor’s product pricing, steal their leads, hijack their marketing campaigns, divert APIs, or just blatantly rip off their content and data… web scraping was the way to go.

​​Once you have access to the raw data, you can reuse it to create your own page (or pages) with your own colors, brand, structure, and so on. The possibilities are endless.

How screen scraping works

The technique is especially useful for data aggregation. For example, a company that wants to consolidate points and status for an individual across all the person’s airline mileage accounts could offer a point tracking portal.

When used ethically, customers agree to share their credentials in this manner with a third party – let’s say, a fintech.

You’re likely familiar with Mint, the personal financial management website and mobile app: it has its roots in the technique, and now uses a third-party solution that employs screen scraping and other means to aggregate bank accounts, credit cards, investments, and bills all in one place.

The third party now can access the customer’s data, and the customer now has access to new financial applications. The third party is happy as they have a new customer, and the bank is blissfully unaware. All good, right?

The problem with screen scraping

Even when it’s used ethically and with users’ consent, though, there are some cracks in screen scraping, and it has downsides for all parties concerned: the customer, the third party, and the bank.

For customers, the user experience may not be consistent. The third party accessing the customer’s data is at the mercy of the bank’s HTML: what if they make changes to their website?

It may also pose security risks: there are no set standards, and the passwords customers share are stored in plain text, making them more vulnerable to hacker attacks.

Screen scraping also causes a major inconvenience for the third party every time a bank changes their login page or interface. It’s estimated that these types of transactions fail at least 30% of the time, resulting in unhappy customers and headaches for the third party.

Finally, the bank is impacted as they now need to handle the rise in calls to their backend servers. Systems designed for a single human user must now handle a steep increase in non-human based calls to their systems.

Moving to open banking

50% of participants polled during a recent American Banker webinar sponsored by Axway – half of respondents – identified screen scraping as a potential issue for their organization.

And yet, 23% of them said they had given their credentials to a third party as a consumer.

There’s a delicate balance between eliminating screen scraping and maintaining the convenience of sharing your data with a third party. This is where open banking comes to the rescue.

Open banking offers a common, open API standard to communicate and exchange data, with all the built-in security protocols such as FAPI and OIDC, allowing for efficient and secure data exchange. One organization that is taking the lead on the creation and use of a common standard is the Financial Data Exchange (FDX), a consortium of all the players in the financial ecosystem.

There’s a growing call to move away from screen scraping for all of these reasons and more. And in Canada, the government has taken notice of the pitfalls associated with screen scraping.

The Government of Canada’s Advisory Committee on Open Banking announced last year that they would launch the initial phase of an open banking framework by January 2023 to help eliminate screen scraping.

As Eyal Sivan, Axway’s Head of Open Banking, puts it, this is a big step and will change how data is shared in the financial ecosystem, paving the way for more innovation and opening opportunities.

Beyond just providing a more secure and reliable alternative to screen scraping, open banking is shaking up the world of finance with more ways to save money, move money, and improve financial well-being.

Open banking is an API-fueled financial ecosystem for the future.