How to prevent spam attacks from rotating proxies
My website has hundreds of thousands of html pages that are open to public. Each time a html page is requested, a call will be made to my database to get the correct data. Therefore the cost of each html request is not cheap (a call to databse is needed).
I know there are rotating proxy services out there that let users send each request with a different IP. They have a pool of hundreds or thousands of IP addresses so any rate limit applied to IP addresses will not work.
I wonder what is the best practice to defend my website if an attacker uses a proxy service to spam requests to my website? Appreciate any help
I thought about caching the database'data but since the list of my website's hundreds of thousands of html pages are available in the sitemap, the attacker could easily loop through all the html pages to request. I cannot cache the entire database.
