Bug bounty platforms: global market study

Contents

• Summary
• Introduction
• Bug bounty programs and platforms
  • Challenges of implementing bug bounty programs
  • Bug bounty platforms
  • Services and prices
  • How to score vulnerabilities and what influences the reward size
  • Bug bounty platform statistics
• Prospects and problems of the bug bounty market
• Conclusions
• Research methodology

Summary

• A bug bounty program engages multiple freelance cybersecurity researchers to find vulnerabilities in software, web applications, and IT infrastructure.
• Bug bounty programs are result-oriented, meaning that rewards are paid only for vulnerabilities found and not for the time spent on looking for them.
• The ingredients of a successful bug bounty program are: appropriate infrastructure, competent people, relevant expertise, company's public image, the ability to pick the right IT assets to be tested, and a clearly defined program scope.
• Special bug bounty platforms help to conduct bug bounty programs. They organize the process of vulnerability hunting, provide the required infrastructure, and give support and expert advice. But most importantly, such platforms engage the community of cybersecurity researchers to perform a security assessment of the system in question.
• Public programs offered by bug bounty platforms are accessible to all community members regardless of their competence level, while private programs are intended only for researchers invited by the customer. Most platforms (88%) run both public and private programs, while 8% offer only private programs; the remaining 4% offer only public programs.
• The following sectors show the highest demand for bug bounty platforms: IT (16%), online services (14%), services (13%), trade (11%), finance (9%), and blockchain projects (9%).
• The reward size depends on the severity level of vulnerabilities discovered. On average, customers pay $7,200 for a critical vulnerability, $3,000 for a high-severity vulnerability, $1,100 for a medium-severity vulnerability, and $254 for a low-severity vulnerability.
• The highest average bug bounty rewards for critical and high-severity vulnerabilities are paid by blockchain projects ($13,000 and $5,300, respectively) and IT companies ($6,600 and $2,200, respectively). In third place are medical institutions, offering $5,500 for a critical vulnerability and $1,600 for a high-severity vulnerability.
• The total price for platform services includes several components. First, platforms charge an initial placement fee, which is calculated individually for each customer. Second, they charge a subscription fee, which covers utilization of the platform infrastructure, interaction with researchers, and vulnerability verification. An annual subscription to bug bounty platform services is $16,000 on average. Third, platforms charge a commission fee, which depends on the subscription plan or vulnerability severity. On average, it is calculated as 20% of each payment to a researcher and is not deducted from the researcher's reward.

Introduction

Vulnerabilities are flaws and bugs in software design, development, or administration that allow attackers to trigger events which are not tolerable for an organization. The organization's cybersecurity team must find and fix all vulnerabilities as quickly as possible to improve application security and stay ahead of cybercriminals. However, sometimes these efforts are not enough, and attackers can cause great damage by exploiting just one critical vulnerability.

Vulnerabilities can be found in the most unexpected or most obvious places. To err is human, and developers make mistakes when designing and deploying applications. According to our study, critical vulnerabilities were found in 62% of web applications in 2020–2021, whereby the average number of vulnerabilities per web application fell by more than a third over the period of 2019–2021. This decrease can be attributed to development and adoption of vulnerability detection tools (for example, code analyzers) and secure software development practices. Automated vulnerability detection solutions are still relevant but not always effective, especially when it comes to finding critical access control vulnerabilities or business logic flaws. The rise in cyberattacks and discovery of previously unknown vulnerabilities make it more difficult to protect IT assets: the first half of 2022 saw 16% more cyberattacks than the same period of the previous year. With applications being constantly updated, manual application security assessment becomes too time-consuming and expensive. The idea of being able to find the most dangerous vulnerabilities in the shortest timeframe and at low cost seems utopian.

But what if automated vulnerability scanning and manual security assessment were combined and carried out not by a couple of infosec experts during their limited working hours, but by the vast global community of security researchers working around the clock and seven days a week? We have analyzed 24 platforms offering a variety of bug bounty programs. Now we share our findings in this report and explain how these platforms can be useful, what challenges they help to address, how much their services cost for organizations, and what rewards security researchers can expect. The research methodology is described in detail at the end of the report.

The crowdsourcing approach to cybersecurity is one of the best solutions to the business challenges mentioned above. It allows organizations to continuously test their software, websites, and infrastructure and detect vulnerabilities by engaging an unlimited number of security researchers. The best way to put this approach into practice is to launch a bug bounty program.

The key to a successful crowdsourcing cybersecurity program is to attract as many qualified researchers as possible.

Bug bounty programs and platforms

A bug bounty program is a way for businesses to engage freelance cybersecurity researchers, security analysts, and penetration testers to probe corporate software, web applications, and infrastructure, with rewards paid for vulnerabilities detected.

Bug bounty programs give companies the opportunity to test their IT assets from different angles: any researcher can participate, using diverse approaches and tools to find vulnerabilities. Companies are in full charge of defining the program scope, controlling the budget, verifying vulnerability reports, and determining the reward size for each vulnerability.

Bug bounty is a result-oriented approach. Under the traditional approach to security analysis, organizations have to pay for the time spent on looking for vulnerabilities, regardless of the results. With bug bounty, organizations pay rewards to researchers for discovered and confirmed vulnerabilities, depending on their severity level. On top of that, competition among community members and result-based rewards motivate researchers to think outside the box and find the most business-relevant vulnerabilities with the highest damage potential.

Bug bounty programs reward researchers for vulnerabilities found, and not for the time spent on looking for them.

By engaging external experts, organizations can identify security flaws more efficiently and reduce the burden on their in-house IT teams, so they can focus on strengthening weak spots and developing their products and services. By paying only for detected vulnerabilities, companies can manage their budgets more wisely.

Challenges of implementing bug bounty programs

Despite the benefits of bug bounty programs, not all organizations can afford them for several reasons:

1. All reports submitted by researchers have to be screened to filter out duplicates and incorrect reports, determine the severity levels of vulnerabilities, and discard vulnerability reports with extremely low severity levels. In addition, effective collaboration with researchers, continuous feedback, and verification of detected vulnerabilities must be established. All these actions require extra resources.
2. To ensure transparency and assess efficiency of a bug bounty program, it is vital to define key performance indicators, monitor them, and create detailed reports. Organizations that lack experience in conducting such programs might face difficulties if they attempt to manage such tasks on their own.
3. To get the most out of a bug bounty program, it is essential to define the program scope and the relevant IT assets very clearly and well in advance. This might prove to be a nontrivial task for some organizations.
4. The organization's public image plays an important role. Some bug bounty programs do not resonate with the researcher community, especially those announced by little-known organizations, because researchers have to consider the risks involved. First of all, they want to know whether it is worth their time to search for vulnerabilities in the organization's IT systems, whether the rewards will be paid fairly and on time, and whether the bug bounty program is managed in a professional way.

Bug bounty platforms

To help businesses cope with these and other challenges, bug bounty platforms emerged. They are like marketplaces that aggregate programs from multiple organizations and allow security researchers to find projects they are interested in. Such platforms provide organizations with the infrastructure required to run bug bounty programs efficiently, arrange collaboration with researchers, and offer support by experts during vulnerability verification.

Bug bounty platforms have the advantage of combining two elements:

• Community of cybersecurity researchers. Community is the most valuable resource of any platform: the more community members with advanced skillsets, the higher the effectiveness of bug bounty programs. Platforms work hard on developing and supporting their communities, as it can take years to build them. An average IT security assessment firm can assign five to fifteen employees to test a customer's application. Meanwhile, that application could be tested by hundreds or even thousands of experts and researchers from all over the world who are registered on a bug bounty platform. They can actively test your applications for months or even years, while the duration of a traditional security assessment is limited to about one month. This increases the chances of detecting vulnerabilities and ensures continuous application security analysis.
• A highly skilled cybersecurity team that reviews reports from researchers and verifies vulnerabilities. The platform team is also involved in customer support and interacts with researchers.

All researchers who start searching for vulnerabilities commit to the principles of responsible disclosure. According to them, only the platform and the researcher will know about the vulnerabilities discovered, while the organization will get sufficient information about each vulnerability and enough time to fix it.

The world's largest bug bounty platforms are HackerOne, BugCrowd, Intigrity, Synack, and YesWeHack.

Services and prices

First, the organization and the platform define the program scope, the non-tolerable events (in the context of Standoff 365), the pricing policy, and the vulnerability report template. Then the platform publishes the program and promotes it among community members, encouraging them to participate. Researchers find vulnerabilities and report them to the platform. The platform team verifies the existence of each vulnerability, its uniqueness, and compliance with the program scope.

Figure 1. Hosting a bug bounty program on a platform

If all three criteria (existence, uniqueness, compliance) are met, the report is accepted. The researcher gets a reward and rating points, and the organization receives a detailed vulnerability report.

The price for bug bounty platform services includes several components:

• Initial placement fee. For this fee, the platform helps the organization to define the program scope, the non-tolerable events, the pricing policy, and the vulnerability report template. The initial placement fee is calculated individually depending on multiple factors, including the organization's line of business, its size, and capitalization.
• Subscription to platform services. The subscription fee includes community activation, use of the platform infrastructure to get reports from researchers and generate analytical reports on the ongoing program, reviews of reports submitted by researchers, vulnerability verification (triage), and interaction with researchers.
• Platform's commission on payouts. The size of the commission depends on the subscription plan (the more expensive the subscription, the lower the commission) or on the vulnerability severity level (the higher the severity, the higher the commission).

The average annual subscription to bug bounty platform services is $16,000. Platforms also charge a 20% commission on each payout (on average).

When it comes to rewarding the efforts of researchers, there are two payout models:

• Direct payouts. Customers can pay a one-time reward for each vulnerability to researchers through the platform. This model is better suited for short-term programs. However, customers themselves have to handle financial matters with researchers, which can be challenging.
• Payouts from a dedicated program fund reserved by the customer in advance. This model is better suited for long-term bug bounty programs. It allows organizations to avoid direct interaction with researchers on financial matters and gives platforms more autonomy.

How to score vulnerabilities and what influences the reward size

The pricing policy may depend on how dangerous a particular vulnerability is to the business. Such vulnerability severity scoring can be based, for example, on the CVSS 3.1 framework.

 

Table 1. Vulnerability severity scoring based on CVSS 3.1

On average, organizations are willing to pay over $7,000 for a critical vulnerability. For example, a researcher can get $12,000 for discovering an SQL Injection vulnerability that allows unauthorized access to data on Twitter. Various platforms offer an average of $3,000 for vulnerabilities related to authorization and authentication flaws. Zerocopter reports that payouts for the most common Cross-Site Scripting (XSS) vulnerability, which was discovered in 13% of applications, can range from $250 to $700, depending on the impact.

Figure 2. Average rewards by vulnerability severity level

Researchers may receive additional rewards if they assist organizations in fixing identified vulnerabilities as soon as possible. This approach is used on Huntr.

Figure 3. Average rewards per vulnerability

Rewards for medium- and low-severity vulnerabilities are quite small, as opposed to critical and high-severity vulnerabilities that pose the most serious threats, such as confidential information leaks, unauthorized access to applications, and attacks on local resources. Businesses understand the implications of such vulnerabilities and are willing to pay significantly more for high-severity and critical vulnerabilities. For example, the increasing number of attacks on blockchain projects in recent months has forced developers to be more vigilant in detecting vulnerabilities in their products and announce enticing rewards reaching $13,100 for a critical vulnerability and $5,300 for a high-severity vulnerability. Some especially dangerous vulnerabilities can be priced at $100,000 and higher. Solutions marketed by tech companies must also be immune to attacks. For example, Sony and Intel offer $50,000 and $100,000, respectively, for critical vulnerabilities.

In order to determine the severity rating more accurately, every organization needs to know how potential exploitation of vulnerabilities could affect its operations and whether it could lead to business-critical consequences. Standoff 365, a bug bounty platform, suggests a new approach: in addition to hunting for vulnerabilities and submitting reports, researchers are encouraged to demonstrate how the security flaws they discover can be used to trigger non-tolerable events. If a researcher submits a report with a clear and comprehensive description of the complete attack vector and the vulnerabilities exploited, and this report is verified by the platform team, the researcher is eligible for a reward that is several times higher than payouts for ordinary vulnerabilities.

A non-tolerable event is an event that occurs as a result of cybercriminal activity, making it impossible to achieve operational and strategic goals or leading to long-term disruption of core operations.

This approach benefits all parties involved. The customer receives a detailed report on exploitation of a range of vulnerabilities that led to a real attack and triggered a non-tolerable event. As a result, the customer can quickly fix the vulnerabilities and form a realistic understanding of the attack scenarios and their consequences. The researcher gets a significantly bigger reward and a higher ranking, while the platform can check a coherent chain of vulnerabilities instead of disparate reports.

Bug bounty platform statistics

Bug bounty platforms are represented unevenly in the global market, and not every country has large and trustworthy platforms. The highest number of large bug bounty platforms is concentrated in Asia, which is home to 38% of the platforms covered by this study. Europe ranks second with one-third of the platforms, including some of the largest, such as Intigriti, YesWeHack, Zerocopter, and Standoff 365. North America and the Middle East account for 21% and 8%, respectively.

Figure 4. Bug bounty platforms by region

The large number of participants is one of the main advantages of the crowdsourcing approach to cybersecurity. However, not all researchers are sufficiently qualified or specialized in the required area (for example, web applications or blockchain), and some organizations are not ready to handle a large number of vulnerability reports at once. That is why platforms offer two types of programs: public and private.

 

1. Public programs are open to any researcher: anyone can get access and start looking for vulnerabilities.
2. Private programs are aimed at a certain group of researchers who get access by invitation.

Public programs allow for more coverage, broader categories, and more vulnerabilities being discovered due to the diverse skills and expertise of researchers. But one should keep in mind that the qualification levels of participants can vary greatly, which is not always conducive to finding critical and high-severity vulnerabilities. Private programs allow organizations to handpick professionals who meet their requirements, or invite the most advanced researchers to increase the chances of discovering serious vulnerabilities. Platforms like Synack and Cobalt run only private programs with thoroughly vetted community members.

Figure 5. Bug bounty platforms by program type

Businesses are embracing these opportunities to enhance their cybersecurity and integrating bug bounty programs into their processes. The most frequent customers (16%) of such platforms are IT companies that are constantly working on improving their applications. One in ten applications on bug bounty platforms comes from customers in finance or trade. According to a HackerOne report, in 2021 the number of customers from these sectors grew by 62% and 51%, respectively. The recent hacks ([1] and [2]) of cryptocurrency platforms (9%) highlighted the need for bug bounty programs to find vulnerabilities in crypto protocols and smart contracts.

Figure 6. Bug bounty platform customers by industry

Prospects and problems of the bug bounty market

Bug bounty programs and platforms are becoming increasingly popular among organizations seeking to ensure cybersecurity of their assets. HackerOne analysts found that in 2021 the number of bug bounty programs increased by 34% compared to 2020, with security researchers discovering 21% more vulnerabilities. According to an AllTheResearch report, the global bug bounty market is expected to reach $5.4 billion in revenue by 2027. This growth is driven by the following factors:

• Global Internet penetration
• Growing awareness of the need to ensure cybersecurity
• Attractive rewards for vulnerabilities discovered by researchers
• Widespread use of online services in organizations due to the fact that employees are increasingly using mobile devices and other Internet-connected devices at work
• High demand for cybersecurity tools among organizations
• Technological advances and trends, such as IoT, IIoT, cloud computing, artificial intelligence, machine learning, and Industry 4.0

However, the following factors may restrict the projected growth:

• Lack of market expansion in less developed countries
• Difficulties in detecting vulnerabilities without buying additional specialized software
• Fierce competition in the industry
• Cybersecurity measures in web application development making it more challenging to find vulnerabilities

Conclusions

The crowdsourcing approach to cybersecurity is very promising, and its implementation in the form of bug bounty programs gives many advantages over traditional solutions. Among such advantages are result orientation, continuous testing, program flexibility and scalability, and a transparent reward system.

However, not all organizations are capable of implementing self-managed bug bounty programs: some of them struggle with scope definition, some lack experience or resources to launch their own programs, and others are not trusted by security researchers.

To overcome these challenges, bug bounty platforms were created, aggregating bug bounty programs from multiple organizations. Such platforms give security researchers the freedom to find projects they like. Platforms help customers define program scope, target applications, and testing systems. They also take care of communication with researchers and verification of their reports. This frees up resources of customers' IT teams, so they can focus on improving their systems and applications based on vulnerability reports.

For a reasonable fee, organizations get all the necessary infrastructure, support, the ability to pay only for results, and, most importantly, researchers with diverse skillsets who are willing to hunt for vulnerabilities 24/7. Standoff 365 offers organizations a unique chance to test how well their products are protected from non-tolerable events. This approach allows customers to understand the real-life consequences of attacks on their applications and get detailed reports that accelerate fixing the security issues.

Security researchers, in turn, can benefit from speedy feedback provided by platform representatives, fair rewards that are significantly higher if a detected vulnerability could trigger a non-tolerable event, and rating systems that pave the road to more profitable private programs. All these measures foster competition and drive improvement among community members.

Research methodology

This report presents the results of our study covering 24 of the largest and most active (at the time of the study) bug bounty platforms from across the globe. We considered the following parameters: geographic location, types of programs offered by platforms, customers' industry, average rewards depending on vulnerability severity and industry, average cost of joining a platform, and average commission charged per payout to the researcher. Table 2 shows the bug bounty platforms included in this study and their geographic location.

All information was obtained from official platform websites and does not contain any confidential data. The average rewards by industry are calculated based on the average maximum and average minimum values for each industry and platform. The average rewards by severity are calculated based on the average maximum and average minimum payouts for each severity level and platform. All amounts are in U.S. dollars.

The severity of vulnerabilities was assessed according to the Common Vulnerability Scoring System (CVSS) version 3.1. The resulting scores were used to determine the qualitative severity values: critical, high, medium, and low.

Table 2. Bug bounty platforms covered by this studyy