Categories
Generally Sysadmin

How to block site grabber bots even if they are using the tor network

If you are managing websites displaying data from non free third party APIs like stock exchange, geo location or travel agency data, sooner or later you will notice site grabbers trying to load that data from your website. Even more so if the data changes on daily bases. Blocking those can lead to a never-ending task.