Platform Rate Limits: Effects on OSINT Missions

What Is a Rate Limit?  

A rate limit is a restriction that websites use to limit the number of requests on a web server that a person can make over a specific time period. These websites will implement session- and IP-based limits to control the number of requests made to a particular API in a certain time frame and prevent system overload. The limits also ensure the fair usage of websites and protect systems from denial of service (DoS) attacks, where cyber threat actors overload systems with extremely large amounts of requests in a short period of time. When a rate limit has been reached, the user is normally blocked from accessing the website until a designated amount of time has passed.   

There are three main levels of rate limits: user, server, and geographical. A user level limit is when a specific user can be identified, and the number of API requests that the user makes is both tracked and restricted. Similarly, a server level limit will block too many requests originating from the same server. A geographically based limit will block too many requests originating from the same geographical location.   

Popular Platform Rate Limits   

Social media platforms such as Facebook, Twitter, Instagram, and LinkedIn are the most common websites that use and enforce rate limits. For instance, Twitter rate limits restrict you to approximately 1000 direct messages, 2,400 tweets, 400 follows, or four account-level or email address changes per day. This includes all actions performed from every accessible device and third-party application. You are not exempt from rate limits by not logging in to the Twitter platform. On the contrary, Twitter tracks all authenticated and unauthenticated access. Unauthenticated access means the user’s location or IP address will be tracked, rather than usage statistics, or how many tweets the user posted.  

  

Challenges to the OSINT Researcher  

When conducting OSINT missions requiring extensive web searches, rate limits can become a hindrance. A lot of time can be lost if you are blocked and unable to complete your research. It may seem that rate limits, such as those enforced by Twitter, are too steep to reach, making them unlikely to interfere with your online activities. That is not the case for many large-scale investigations. If multiple investigators are operating out of the same location and conducting similar searches, they can reach these limits much faster. It is important to keep in mind during your investigation that these limits do exist and to take the necessary precautions to prevent from reaching the rate limit quota.