An Overview of Open Source Intelligence: Benefits, Tools, Sources, and Use Cases
August 26, 2021
Information surrounds us. Whether at home, in social settings, and even in the workplace. Likewise, when considering the vast amounts of information available from a financial perspective, many would be shocked to realize the full-scale of information produced in every business transaction. Likewise, in order to understand, track, and evaluate the vast amounts of information produced in business, many companies utilze open source intelligence.
The main idea behind open source intelligence (OSINT) involves collecting, analyzing, and generating actionable insights from publicly available sources. Although OSINT has been around for as long as records have been collected and analyzed by government-based transactions, companies have only gradually unlocked its true potential in recent decades. Let’s look at what it is and what we can achieve with it in business nowadays.
What is open source intelligence?
The most common definition of open source intelligence is the practice of utilizing publicly available sources of information for various intelligence goals. These sources include everything that any member of the public can access. Some such information is directly available online or as part of the media, for example, on TV or the radio. Other pieces of information are accessed either by request, like census data and other statistics kept by governments, or by subscribing to particular publications or newsletters.
It is sometimes believed that advanced technical skills or tools are necessary to access open source information. However, this statement needs to be qualified as, broadly speaking, a typical-consumer laptop and any search engine are able to access and process open source data. Ultimately, you don’t need any unique tools, for example tools only used by specialists and corporate-level softwares to access this information.
However, not all usage of information counts as intelligence. Thus, it must be added to the definition of OSINT that the collection and analysis of such data have to be carried out promptly and purposefully address an appropriate audience regarding specific strategic objectives.
The history of OSINT
Ironically, the beginning of open source intelligence is associated with secret intelligence agencies. Although people have used openly available information before, it has only truly shaped into the full-fleshed methodology and practice with the advancements made in spying in the 20th century.
During the Second World War, the predecessor of CIA – Office of Strategic Services - had a Research and Analysis Branch, tasked with gathering and analyzing pieces of media, such as newspapers, broadcast reports, and photos. The goal was to gather information and find any clues that would give away something important about the enemy states. Thus, OSINT appeared to be a crucial part of secret services during the most critical times. Not only was new intelligence produced, but the information from deeper and secret sources could be better understood within the OSINT framework.
As computer technology developed and local networks turned into the worldwide web, open source intelligence emerged as an important part of IT security. This takes us to the dark side of OSINT, where malicious actors may try to utilize sensitive information which could be gathered and pieced together from public sources. This makes it a task of IT specialists to find weak points in particular networks, prevent potential leakage, and check if something that was never meant to be OSINT data is now publicly available.
Finally, we are gradually moving from spycraft to tradecraft as open source information is being used for strategic business purposes. Business OSINT shares the same characteristics of intelligence as it gathers information from open sources to achieve particular objectives. However, in the private sector OSINT is used together with similar techniques and practices aiming at improved decision-making, thus it may take on different roles and features in specific organizations.
Types of OSINT data collection
OSINT data collection methods can be divided into two major categories, one of which is passive collection. This type of OSINT research is carried out with the researcher staying largely anonymous and refraining from taking active steps to put them in the spotlight. Threat intelligence platforms and other automatic security scanners that warn about potential security risks are commonly used examples of passive OSINT.
The other category of data collection is active OSINT investigations and research. Active OSINT requires researchers to expose their collection methods and web identities more. For example, active collection includes joining online and offline communities, attending particular events, and subscribing to publications. Additionally, active participation may be necessary when some AI tool signals intelligence threats or other issues that cannot be solved without the decision-making of human intelligence.
OSINT tools and sources
As mentioned above, open source intelligence comes from all kinds of published and otherwise accessible content, whether it is written down, broadcasted, or recorded in some other way. But there is more to harvesting data for OSINT than just collecting information from the surface content of these sources. And that especially goes for digitally accessed information.
It's not only the front-end of the web pages or the actual message of a blog post that interest OSINT researchers, be they good or malicious actors. They are also keen to dive deeper and gather data points that are harder to find. Sources of intelligence available or discoverable online include the following examples.
- IP addresses
- DNS records
- E-mail addresses
- Social media platforms, forums, blogs, and message boards
- Usernames and profile information
- Phone numbers
- Various documents, including images and videos
- Metadata for these documents
Such information is accessible through various online tools, the most common of which are, of course, search engines. However, open source intelligence analysts go beyond Google or popular websites and often use more specialized search engines. For example, in order to find the data on the dark web, which is out of reach for the usual internet search tools, one has to use the Tor browser and its dark web search engines.
Choosing the right tools for OSINT
As there are many tools for uncovering open source intelligence, one must wonder how to choose the right one. The first thing to remember is that open source intelligence is meant to help deal with information overload, which is especially felt online. This means that it is crucial to define the clear goals of research before gathering information. Then the tools can be suited according to the specific intelligence requirement.
For example, device search engines like Shodan can be used to discover a hardware-based weak point in the network. And such tools as Spiderfoot assist with the automation of OSINT procedures. There are many such tools that are either open-source themselves or have both free and commercial versions.
Business risk intelligence
Turning now to look more closely at the usage of OSINT in business, a good way to start is by looking at business threat intelligence. The four major types of assessing risk through open source intelligence can be categorized as follows.
Cybersecurity and data breaches
First, we have cybersecurity concerns that come directly from the history of OSINT usage in IT. Companies today must utilize broad networks and record a lot of data, thus making themselves potential targets for malicious actors. Luckily, OSINT also offers ways to combat digital industrial espionage and other threats to cybersecurity. It is done by utilizing tools that warn of potential data breaches or an insufficiently secured entry point into the network.
Open source intelligence is also used to inform about potential threats to the physical security of staff and company assets in particular locations. News about dangerous events such as natural disasters or riot outbreaks travels fastest through the open sources of information like social media posts. Monitoring such data sources will allow security teams to assess risks of physical harm and give time to determine and take appropriate measures regarding the security of, first of all, people and then property.
OSINT enables businesses to assess fraud risks by researching openly available information about particular persons and firms. Additionally, open source intelligence tools can link people with companies and organizations and help to determine their reliability. Cross-checking the facts available for the public and running social media analysis will reveal inconsistencies that may indicate fraud, thus preventing damaging transactions and giving a chance to notify law enforcement.
Operational and reputational risks
Finally, companies constantly face risks related to their brand reputation and various undertakings. Starting at a new territory or forming a new partnership may harm both the business stance and the company’s image if done carelessly. Open sources of information are pretty much indispensable for comprehensive intelligence and deep understanding in such cases. For example, a thorough analysis of the market and local conditions, public sentiment, and business culture is necessary when a firm wishes to open a new location in a foreign market.
Importance and benefits
Open source intelligence market size is predicted to keep gaining at over 25% CAGR between 2021 and 2027. Thus, its value and importance are quite clear for the market experts. Here are some of the key benefits of giving a sense of how business is boosted by utilizing information available to us all.
The first benefit is clear from what has been said before – open sources are the fastest routes for relevant information to travel freely. News regarding business risks, opportunities, or market events can often reach the firms much sooner than through private or specialized channels. And whether it is social engineering attacks, market-critical transactions, or an angry mob in the location, management and security teams certainly need to know about it immediately.
A great volume of information
Most of the relevant information that can be turned into actionable intelligence is in fact open source. The sheer volume of data created every day is a two-edged sword, as it can cause information overload, making it hard to keep track of what matters. However, on the other hand, open source intelligence techniques are meant precisely to efficiently navigate through all the available data and find the answers one is looking for. Thus, as long as analysts know what they are doing, the fact that OSINT deals with an incredible volume of information testifies its versatility and potential of advancing different business goals.
Of course, the clearest advantage of open source intelligence is precisely what’s in the name – openness. Since OSINT deals with information that is publicly accessible, it provides everyone with an equal opportunity of turning discovered information into valuable business insights. While other types of information can only be accessed by taking extra steps if at all, open source is right there waiting for those willing to work with it.
Raw historical records
Only specific types of historical information are classified and only for a limited period of time before being released to the public. That means that most historical records from statistics to public consumption data are open to the public. Such raw information can help financial firms train algorithms thus advancing their machine learning strategies. Additionally, various business modeling and strategizing can be informed by looking at historical information.
Business and financial use cases
To end this overview of OSINT, let’s look at a few major intelligence purposes for which it is used by firms today.
The general purpose of all business intelligence is to provide insights that would boost decision-making. And OSINT does precisely that by collecting, disseminating, and analyzing openly available data to provide concrete answers regarding pressing business matters. With the variety of knowledge available through OSINT, decision-making in every department from HR to marketing can be significantly enhanced.
Identifying investment opportunities
Investors use OSINT due to its above-mentioned benefits of speed and size. A single tweet or a just published image can be an early signal of a major market event that would completely change the winning strategy regarding particular stocks. Furthermore, after the role played by Reddit users in raising GME stock price, it’s no longer possible to overlook the power of retail investors when they are united by online communities. Keeping track of such developments will be more and more important for professional investors, and that’s exactly where OSINT can help.
Internal network testing
Companies must rely on large-scale internal networks to efficiently do business in our times. For instance, a good senior security consultant will aim to utilize OSINT to inform decisions regarding how these networks should be developed. Likewise, constantly test such networks to identify weak spots and prevent data leakage or hacker attacks.
After big data revolutionized, business companies usually store and refresh various datasets that hold commercially important information. The data for these sets can be acquired in numerous ways, from internal procedures to using the services of data brokers. Open source intelligence should not be forgotten as a great way to enrich and supplement datasets with constantly updated and readily available information. Such supplements increase the quantity of data and improve its quality by correcting or removing defective and outdated data points.
Free data sample
- See the sample structure of our resume and company JSON records
- Explore the main resume and firmographic data points
- Find out the definition of each data point
When one gets used to it, it’s easy to forget just how incredibly vast the information that we can all access today is. The successful practices of open source intelligence remind us why it’s worthwhile to remember it when doing business. And as we keep living in a society that exponentially produces data, the ability of OSINT tools and techniques to turn this data into actionable intelligence will certainly play a role in determining what’s next for big data in business.
August 05, 2021
What is Data Intelligence?
Data intelligence refers to all of the tools and informational assets that companies leverage to help derive insights with the...
November 29, 2021
Data as a Service: Benefits, Challenges, and Solutions for Your Business
Data as a Service, also known as DaaS, allows companies to outsource data storing and maintaining services. Businesses now resort...
June 03, 2021
What are Data Brokers and What Value do They Bring?
Data brokers, also known as data providers, are either private persons or, more often than not, firms specializing in gathering...