Proxy Market Research 2023

Proxies, and the data extraction industry at large, have been on a growth spurt for years now. There’s no need to look far for proof – observing the major proxy providers is enough. In December 2021, Bright Data cleared the 100 million revenue mark, with a team that grew thrice in three years. Oxylabs has over 300 employees (30% yearly growth), and Smartproxy employs over a hundred. The providers that claim to serve enterprises are becoming enterprises themselves. 

That’s not to mention the newer entrants which have successfully found their place under the sun. SOAX and IPRoyal, both scrappy newcomers just a few years ago, today have a well-defined business presence, their own proxy networks, and conferences to attend around the world. 

In our survey, IProyal reported tenfold growth in data usage compared to 2021. Webshare, which runs on self-service and inbound marketing channels, doubled its revenue in 2022 and introduced new proxy types. NetNut, whose money was hemorrhaged by legal disputes and infrastructure investments, has managed to become profitable again. Rayobyte’s residential proxy product took off, and GeoSurf hired additional people to build customized solutions for its clients. It was a good year across the board. 

When asked about competition, most participants believed that it had increased. However, they didn’t necessarily treat it as a bad thing. According to Webshare, many are resellers who will have a hard time scaling down the road. GeoSurf noticed that the number of providers returned to its starting position after growing early in the year. The majority found it hardest to compete in the enterprise segment.

Rayobyte raised a point that we also share: the industry is reaching a consolidation stage, and we should soon start seeing mergers and acquisitions. Several have already taken place: NetNut owns ChiProxies, and Oxylabs acquired Webshare in what we’re sure was an eye-watering deal. We expect to see at least one more high-caliber acquisition in 2023. 

At the same time, the technical threshold for entering the market remains low, maybe even lower than it’s been for years. We have to thank the availability of 4G/5G proxy software and hardware for that. The real challenge for now lies in finding and retaining clients by occupying the niches that major providers ignore. 

The broader economic outlook for 2023 is one of uncertainty. So far, however, it has yet to affect any participants; some think this will continue to be the case, while others are more cautious. Even then, they believe that data infrastructure is one of the last resources to be cut, and they’re ready to weather possible storms. Defying all broader predictions, Webshare expects to double its growth for the second year in a row. Ambitious indeed.

Growing fast in a booming industry, proxy providers naturally want to capture as much of it as possible. In addition to increasing market penetration with existing products, there are two ways to accelerate growth: horizontal and vertical expansion. 

Horizontal expansion allows providers to boost their addressable audience through new proxy products. We saw multiple companies bolster their line-ups in 2022: 

New proxy product releases 2022-2023

That being said, proxy servers are a resource, and they occupy only a small portion of the data extraction lifecycle. Having accumulated more manpower, expertise, and financial leeway, providers have increasingly started moving up the data value chain. 

The data extraction lifecycle according to Oxylabs.

So far, the focus is on addressing the scraping (and, to some extent, parsing) stages with APIs and smart proxy networks like Zyte’s Smart Proxy Manager. This makes sense from multiple perspectives: the provider can save resources by better optimizing the scraping logic, while developers have an easier time accessing websites behind toughening protection systems. 

The expansion to APIs has been going on for some time now, but it especially accelerated during the last year. Infatica, Oxylabs, Shifter, Smartproxy, and SOAX all launched new products within that time frame. Today, only four out of the 12 participants offer no web scraping tools, with one of them (NetNut) planning to introduce something in the near future. 

Available web scraping products (* – released in 2022-2023)

As is often the case, Bright Data has gone the furthest in the process. In 2022, it started offering pre-collected data sets. And after the purchase of Market Beyond, the company entered into data analytics by launching Bright Insights. With these two moves, Bright Data effectively covered the full data extraction value chain. 

Bright Data’s illustration from March 2022. Since then, the circle has expanded with another layer for data insights.

Providers have high expectations for the new tools. All survey participants expected their usage share to grow in relation to proxies, forecasting that fewer companies will run in-house scrapers in the future. Bright Data’s scraper, Web Unlocker, was already the driving force in 2022. The provider also places high bets on data sets and on-request scrapers, as companies look to cut operational costs in the face of economic uncertainty. (According to Bright Data, 30% of IT budgets go to unstructured data management.) 

Despite this trend, proxies are going nowhere. Every respondent identified their role as core, foundational, or at least significant, with no plans to remove them from product portfolios. At the same time, most have plans to further build upon proxy networks, which may impact the ways how clients can use raw proxy servers (for example, by reducing the number of available targets). 

When we interviewed Bright Data a year ago, we found that datacenter and residential proxies took 95% of the provider’s proxy use. In other words, the other two types, ISP and mobile proxies, were used only 5% of the time. This remains true a year later, with datacenter proxies being favored by budget-conscious customers and residential proxies seeing the most popularity with enterprises. 

What about the other survey participants? In all but two cases, residential proxies proved to be the most popular proxy type. One exception was Webshare, which sold only datacenter IPs until very recently; its shared proxies generated three quarters of the revenue. The second was Rayobyte, which specializes in datacenter proxies and experienced steady growth. 

These findings cause little surprise, and they confirm the survey we ran with Proxyway’s visitors back in October. At the same time, they raise questions whether mobile and ISP proxies really are such niche products or if their demand is better met by smaller and specialized proxy services. 

The situation with ISP proxies is interesting. One of their primary use cases, managing multiple accounts, is successfully covered by mobile addresses. The other, item scalping, faces tough competition in the sneaker niche, where specialized proxy sellers consistently manage to cook up excellent IPs from major internet service providers. Still, this proxy type has potential, and Webshare with Rayobyte both pointed to its growth. 

Some of the ISP proxies specialized providers manage to procure.

The number of businesses operating mobile proxy farms remains high, and some have experienced significant success. Yet, most major providers either refuse to adopt them or, at best, intermingle dongle-hosted IPs with their peer-to-peer mobile addresses. IPRoyal is the only participant that sells such proxies in their most popular unlimited traffic configuration, while Rayobyte runs a fast-rotation network for web scraping.

Proxy sourcing challenges might play a role here. With IPv4 prices so high, buying datacenter proxies is much easier than maintaining them in-house; the acquisition process of residential addresses is both demanding and legally challenging. Currently, the most acceptable sourcing method is via bandwidth-sharing apps, but they require too much effort for minor market participants and in-house data infrastructure. 

On the other hand, the barrier to entry for mobile proxy farms is low, so it’s relatively simple to build a device fleet for internal use or set up sufficient infrastructure for reselling. It won’t be easy to scale, but the myriad of providers that serve individuals or small businesses don’t need thousands of devices to turn a profit. We believe this is one area that the major players currently underserve. 

Proxy servers have various uses, many of which we listed last year. However, their distribution isn’t equal, meaning that some use cases, targets, and, by extension, products that work best with them, naturally dominate the provider’s marketing and technical resources.

As we discovered from our own experience, surveys, and third-party data from sources like Apify (which published an excellent report on the state of web scraping earlier this year), e-commerce data collection is currently the dominant use case for proxy servers. To give you some concrete data, it constitutes 40% of Rayobyte’s and 50% of Infatica’s business, whereas the number of SOAX’s e-commerce users spiked by 32% in 2022. 

We asked the survey participants to list between three and five of their top industries. The other oft-mentioned use cases were digital marketing (including SEO), cybersecurity, ad verification, account management, and generic web data extraction. In brief, all the usual suspects.

The survey also included a question about changes that took place in 2022. Half of the respondents reported no major changes, other than the overall increase in demand. NetNut mentioned a drop in sneaker scalping, which we’ll get to in a second. Bright Data managed to unlock the financial services industry and expects it to continue growing fast.

Looking into the future, one use case that has inevitably surfaced is data for AI models. SOAX named this emerging business model Decisions-as-a-Service, where technologies like ChatGPT help people get quick and simple answers to their questions. Multiple surveyed providers emphasized the need for accurate and large-scale data to train these models. To rephrase Oxylabs, the potential in this space for data collection companies is enormous. 

Last year, we wrote about the hard times that had befallen the sneaker botting industry. Pressured by tightening website defenses and fewer releases, sneakerheads were leaving for more profitable avenues like GPUs, consoles, and NFTs. What has happened since then, and where does the niche stand in 2023? In brief, it’s trudging along, but changes had to be made by all members of the ecosystem. 

First, sneakerheads no longer target the same websites. The former bread baskets, Yeezy Supply and Footsites, have both fallen off. Yeezy Supply is gone due to a series of unprofitable drops and recent Kanye scandals, whereas Footsites have largely transitioned to raffle releases and no longer offer the generous stocks they used to. At this point, Nike and around a dozen Shopify stores remain the targets worth the effort. 

The prices of most sneaker bots have tanked. Past faves like Cybersole now cost hundreds instead of tens of thousands of dollars; even Wrath, which consistently brought results, goes for less than $3,000 for lifetime access. It’s not the best time to be a shoe bot seller. 

Sneaker proxy vendors have also been affected. Some, namely Leaf Proxies and Oculus Proxies, have merged together. Others, such as Bart Proxies and Ping Proxies, have expanded their services to general-purpose use cases in places like Black Hat World. Quite a few smaller resellers likely closed without fanfare. 

Formerly specialized sneaker proxy providers on Blackhatworld.

The general sentiment ranges from negative to cautiously realistic. Is it possible to earn money reselling shoes? Yes, but no longer easy. Can you still make a living doing so? Nowadays, the answer is often no, else it comes with multiple asterisks (such as sizable prior experience or ability to survive the slow months). For many, shoe botting has once again taken the role of a side hustle in addition to their regular day job. 

Still, the interest in sneaker botting hasn’t dried out. We are privileged to occupy some of the top Google positions for sneaker-related queries, and sneaker bot listings remain among our largest pages. 

Our sneaker-bot related pages still bring thousands of visitors per month.

Are these simply the latecomers, or does the industry still have life left? That’s tough to say, but for now, shoe cooks continue looking for their grails. 

An irresponsibly managed proxy network is a menace. It burrows into the devices of unwitting people, using their resources to commit ad fraud, bring down websites, and empty stolen credit cards. This was the dominant image of proxies for a long while, and to this day it occupies a sizable portion of the discourse. 

The reasons aren’t hard to understand. It’s still very possible to acquire proxy servers by creating large botnets or running SIM card farms, and malicious businesses remain as profitable as ever. According to the FBI, the number of credential stuffing attacks through residential proxies is increasing, and even state actors have been using makeshift proxy networks to commit disinformation campaigns against their adversaries. 

In 2022, two big crackdowns shook the industry. Mid-June, visitors of RSocks, a then-popular proxy service, were met with a big banner informing that the FBI had seized the website. More than a month later 911 S5, another high-profile proxy provider, announced its permanent shutdown. Both businesses were well known in hacking communities, and some cybersecurity experts even called them critical cybercrime infrastructure. 

Source: rsocks.net

On the bright side, commercial proxy services have started treating ethical business practices as an increasing priority. Initially a concern of Bright Data and Oxylabs, ethics has now made its way into the websites of SOAX, IPRoyal, and other emergent companies, at times even taking center stage. More providers now feel the need to explain their IP sourcing practices, and their KYC processes have become more stringent, often with the help of external specialists. 

Last year also saw some interesting initiatives. Five major proxy providers banded together to start The Ethical Web Data Collection Initiative, and SOAX launched a podcast focusing on ethics in the web data collection industry. We too have conducted interviews with Bright Data, Oxylabs, SOAX, Rayobyte, and the newly formed i2Coalition about the ethicality of sourcing and using proxies. 

The majority of our surveyed participants approached the topic with optimism. They believed that the outlook toward proxies has shifted from perceiving them as hacking tools to treating proxy servers as a foundational part of the internet. Clients (especially enterprises) are increasingly asking about IP sourcing practices, and they’re less likely to treat the business relationship as a necessary but shameful secret. 

Fully understanding that most of the replies would be PR-friendly, we decided to throw a curveball: how do you distinguish a real commitment to ethics from virtue signaling? The answer is two-fold: research and ask. 

You should research how the provider vets customers, how it gets proxies, and whether the sources express clear consent. Ideally, this information should be publicly available; otherwise, you can always ask. Third-party audits help, and a little open source intelligence (like Googling the company’s name with certain keywords) goes a long way toward giving you a more balanced view of the service. 

The practice of web scraping is rarely straightforward from the legal standpoint. It may involve extracting information that is copyrighted, personal, or private. What’s more, the websites worth scraping often have incentives against giving away their data. For example, Ryanair’s business model relies on additional services you can get when booking a flight directly on its website. Third-party aggregators threaten this model. 

Despite prohibiting automated access, major public data sources in e-commerce and search (read: Google and Amazon) have been reluctant to pursue legal action against web scrapers. Social media networks, however, are a different story. LinkedIn and Meta vigilantly fight data collection, both in and out of court. 

Out of the many cases (such as Meta v. Social Data Trading, Meta v. Octoparse, or the recent Meta v. Voyager Labs), HiQ v. LinkedIn has been the most impactful in clarifying where web scraping stands when faced with websites that don’t want to be scraped. The developments in 2022 gave us two important messages:

1. One, the CFAA (an anti-hacking law from 1986) doesn’t apply to publicly accessible data. The community lauded this as a great win, as companies relied on the CFAA to bully web scrapers – and because it incurs criminal liability! So, when the Ninth Circuit affirmed its position in April 2022, many sighed in relief.
2. Two, breach of contract will become a new basis for lawsuits, and so far courts have been much friendlier to corporations here. HiQ lost on this count in October, even if the litigation process continues. This causes concern: this January, Meta sued Voyager Labs for allegedly running multiple bot accounts; however, the recent lawsuit against Bright Data barely even mentions bots – it’s enough that the company had business profiles on the platform (you can read the complaint here or listen to Bright Data CEO’s take here).

While we inch toward a resolution, the industry suffers from the effects of legal ambiguity. Multiple companies we’ve spoken to refuse to advertise social media scraping in fear of repercussions. As this Hacker News thread shows, small one-man businesses aren’t exempt from receiving cease & desist letters either. And even though HiQ fights on, the litigation destroyed its business years ago. 

Many web scraping companies face similar challenges behind closed doors.

All things considered, scrapers have won some important battles. But with websites increasingly gating up, we still have a long way to go toward an open web. We predict that the industry will stay on the low while its unwilling representatives wrestle with the social media giants. Meta v. Bright Data (breach of contract) and 3taps v. LinkedIn (clarifying the HiQ v. LinkedIn decisions) are some of the cases worth following. 

Residential and Mobile Proxy Infrastructures Have Become Impressively Robust

Webshare believes that the majority of proxy services are still slow and unreliable. While we tend to agree, the performance of peer-to-peer proxy networks has improved greatly. Residential proxies today barely lag behind datacenter addresses (without special optimizations), which was unthinkable several years ago.

For example, here’s the success rate of US-filtered proxies when connecting to the nearest server of a global CDN:

And here’s the response time – with our benchmark webpage (several kilobytes) and Amazon’s product pages (around 1 MB response size):

SOAX is a great case in point for how much optimization can impact performance. And even providers like Oxylabs, which already led the pack, still manage to make significant speed gains every year despite facing diminishing returns:

Sometimes, it’s possible to find creative solutions. Bright Data’s scale allows it to experiment with request caching, which returns pages other customers recently visited without making another request. In theory, this can speed up response times by 20 times, reduce website load, and decrease costs.

Mobile IPs Cost x3 More than Residential, x30 More Than Rotating Datacenter Proxies

It’s no secret that mobile and residential proxies are expensive compared to datacenter proxies. But by how much, exactly? The graph below illustrates the price differences between these three products. Simply put, a rotating datacenter proxy network will let you open 30 times as many pages as a similarly-priced mobile plan.

Of course, this comes at a cost: fewer locations and a significantly worse success rate with strict websites. Even in our limited-scale test (~2,600 connections), every third request with rotating datacenter proxies failed to reach Amazon. Running them with social media or Google wouldn’t have been a pretty sight:

ISP Proxies Face Issues with IP Databases, Inflate US Residential Proxy Pools

In most aspects, ISP proxies are very similar to datacenter proxy servers. But they do have one important difference: such IPs must be related to internet service providers that offer services to consumers.

This raises several challenges: 1) the provider may contract obscure regional ISPs  – their IPs may not appear as residential in databases; 2) It takes time until an ISP announces new IPs on its network and the updates appear in databases.

We saw the effect of these challenges first hand. Each participant had the opportunity to hand-pick 100 ISP proxies for our tests. Even then, the latest (March) version of IP2Location still identified a good part of them as datacenter:

The same problem affected residential proxy networks, as well. In particular, providers tended to inflate their US pools with what we assume were ISP proxies. As a result, the same IP2Location identified up to a third of their IPs as non-residential:

While this may not necessarily affect you in real-world use, some websites care about IP reputation and could block such proxies more.

Self-Service Has Become a Necessity – Even for Premium Providers

Gone are the times when you had to fill out lengthy forms and wait for salesmen to get in touch. Today, most providers have on-boarding flows that require no interaction with people. Mobile proxy networks remain an outlier, but that’s because some participants treat them as beta or situational products. We expect things to change soon.

One necessary obstacle in most self-service flows affecting residential or mobile proxies is the KYC procedure. Even here providers have found ways to reduce friction. Bright Data offers immediate access to 240+ pre-approved websites (though you have to install a certificate), while Rayobyte has a 50MB no-KYC trial.

Finally, it’s interesting to see how elaborate some self-service tools have become. Webshare’s success shows the power of customization. The provider took the rigid pricing model of datacenter proxies and gave it with way more flexibility: customers can tinker with traffic allowance, available threads, IP quality, and network priority. They can even create rotating endpoints for proxy lists, which others would sell as a separate product.

New Proxy Filtering Options Have Entered the Scene

After several relatively stagnant years, residential and mobile proxy networks are starting to see new features. In particular, it’s now possible to get proxies with ZIP code and OS filtering.

The features aren’t exactly new. 911 S5, the formerly popular proxy provider, had ZIP-level targeting for years. But legitimate businesses were reluctant to implement it. For now, the two providers that have it, Bright Data and Oxylabs, don’t advertise the option much – you can mostly find mentions in the documentation.

OS-level targeting came with Smartproxy’s mobile proxies in 2023. It’s a useful feature if you need to emulate a particular mobile system, and changing request headers isn’t enough.

For many, these features will be nice to have. But the need for geo-targeting has risen in general. SOAX reported that the share of requests with precise targeting has increased by 2.5 times – in some countries, by four to seven times.

Proxy Pool

Infrastructure Performance

Performance with Popular Targets

Features

Pricing

Proxy Pool

Infrastructure Performance

Performance with Popular Targets

Features

Pricing

Proxy Pool

Infrastructure Performance

Performance with Popular Targets

Features

Pricing

Infrastructure Performance

Performance with Popular Targets

Features

Pricing

Infrastructure Performance

Performance with Popular Targets

Features

Pricing