Has screen scraping seen its day?

In the banking world, screen scraping has been around for over 25 years – it’s how many fintechs got their start. But a continued push towards open banking in Europe and recent regulatory moves in North America aim to do away with the practice.

Let’s look at how screen-scraping works, why it can be problematic and even risky, and how open banking is the secure solution needed to move beyond screen scraping.

What is screen scraping?

Screen scraping (also known as web scraping) is what a developer might do to get access to information that’s usually only shared via a webpage. It works by copying information displayed on a screen (such as text or images) and using it for another purpose.

To put it simply, web scraping is driven by bots/web crawlers. They work in the same manner as search engines do – Fetch and Copy. But web scraping focuses on retrieving particular data from a website, whereas search engines frequently scrape the majority of webpages on the internet.

In its early days, companies used web scraping to take advantage of their competition. If you wanted to keep tabs on a competitor’s product pricing, steal their leads, hijack their marketing campaigns, divert APIs, or just blatantly rip off their content and data… web scraping was the way to go.

​​Once you have access to the raw data, you can reuse it to create your own page (or pages) with your own colors, brand, structure, and so on. The possibilities are endless.

How screen scraping works

The technique is especially useful for data aggregation. For example, a company that wants to consolidate points and status for an individual across all the person’s airline mileage accounts could offer a point tracking portal.

When used ethically, customers agree to share their credentials in this manner with a third party – let’s say, a fintech.

You’re likely familiar with Mint, the personal financial management website and mobile app: it has its roots in the technique, and now uses a third-party solution that employs screen scraping and other means to aggregate bank accounts, credit cards, investments, and bills all in one place.

The third party now can access the customer’s data, and the customer now has access to new financial applications. The third party is happy as they have a new customer, and the bank is blissfully unaware. All good, right?

The problem with screen scraping

Even when it’s used ethically and with users’ consent, though, there are some cracks in screen scraping, and it has downsides for all parties concerned: the customer, the third party, and the bank.

For customers, the user experience may not be consistent. The third party accessing the customer’s data is at the mercy of the bank’s HTML: what if they make changes to their website?

It may also pose security risks: there are no set standards, and the passwords customers share are stored in plain text, making them more vulnerable to hacker attacks.

Screen scraping also causes a major inconvenience for the third party every time a bank changes their login page or interface. It’s estimated that these types of transactions fail at least 30% of the time, resulting in unhappy customers and headaches for the third party.

Finally, the bank is impacted as they now need to handle the rise in calls to their backend servers. Systems designed for a single human user must now handle a steep increase in non-human based calls to their systems.

Moving to open banking

50% of participants polled during a recent American Banker webinar sponsored by Axway – half of respondents – identified screen scraping as a potential issue for their organization.

And yet, 23% of them said they had given their credentials to a third party as a consumer.

There’s a delicate balance between eliminating screen scraping and maintaining the convenience of sharing your data with a third party. This is where open banking comes to the rescue.

Open banking offers a common, open API standard to communicate and exchange data, with all the built-in security protocols such as FAPI and OIDC, allowing for efficient and secure data exchange.

One organization that is taking the lead on the creation and use of a common standard is the Financial Data Exchange (FDX), a consortium of all the players in the financial ecosystem.

There’s a growing call to move away from screen scraping for all of these reasons and more. In Canada, the government has taken notice of the pitfalls associated with screen scraping.

The Government of Canada’s Advisory Committee on Open Banking announced last year that they would launch the initial phase of an open banking framework by January 2023 to help eliminate screen scraping.

This is a big step and will change how data is shared in the financial ecosystem, paving the way for more innovation and opening opportunities.

Meanwhile, in the United States, the Consumer Financial Protection Bureau (CFPB) is gathering feedback ahead of an upcoming proposed U.S. open banking rule.

By enabling fast, secure access to quality data through APIs, the rule would aim to curb screen scraping and misuse of financial information while also reducing bias and reliance on credit scores. 

The Office of the Comptroller of the Currency (OCC) just announced the establishment of the Office of Financial Technology designed to supervise the bank-fintech partnerships. The CFPB is expected to issue an open banking proposal later this year, with finalization and implementation planned for 2024. 

What this means for banks and financial institutions 

Regardless of where the CFPB or Canada’s open banking committee land on screen-scraping, the market is resolutely marching toward adoption of open financial APIs, with FDX emerging as a front-runner standard in North America.  

Beyond the security gains of moving away from screen scraping, it’s time for banks to embrace open banking because of the inherent opportunity. 

The core benefit of open banking is being able to reach and capture new customers that [banks] wouldn’t have access to otherwise. By changing their business model to embrace open digital ecosystems, banks can uncover opportunities that previously would never have been available to them. In this new landscape, closed data silos are unable to respond to the modern speed of business.

Moreover, a decentralized system built upon a common, open standard prevents the emergence of a single dominant player, one of the fundamental goals stated by the CFPB.

To fully participate in this expanding financial ecosystem, an open banking solution with a universal API marketplace component is an essential tool to help support a bank’s digital business strategy, governance, and security for APIs and API products. Amplify Platform lets companies govern and control the APIs they need to stay securely connected to customers and partners. 

Beyond just providing a more secure and reliable alternative to screen scraping, open banking is shaking up the world of finance with more ways to save money, move money, and improve financial well-being.

Open banking is an API-fueled financial ecosystem for the future.