This all started with the API changes. Before they were charging a fair price for the API and companies happily paid it for the convenience. Then Elon got greedy he started charging a ridiculous amount for the API so those companies decided it would be worth it to just deal with the annoyance and switch to scraping the website instead. But when you hit the website it’s way less efficient than hitting the API because it’s doing a bunch of mixing and ranking which is a lot more complicated and costly than serving static content, which costs a small fraction of real time algorithmic ranking. So now instead of making money off companies that want their data they’re losing money on the cloud costs to serve the scrapers.
More recently, they haven’t been paying their hosting bills to Google and their service was set to expire at the end of last month. What just happened at the start of this month? They added the limitations. That can’t be a coincidence. Now the Google services weren’t hosting the site itself, otherwise they’d probably be fully down right now, but it was hosting their trust and safety services for things like fighting spam. That could have also been hosting anti scraper services as well. Since they’re so under staffed they probably couldn’t swap the services in time, so instead of having anti scraping services like every other big company on the internet, they decided to cut service instead. It’s just built up incompetence coming to a head.
It goes deeper than that.
This all started with the API changes. Before they were charging a fair price for the API and companies happily paid it for the convenience. Then Elon got greedy he started charging a ridiculous amount for the API so those companies decided it would be worth it to just deal with the annoyance and switch to scraping the website instead. But when you hit the website it’s way less efficient than hitting the API because it’s doing a bunch of mixing and ranking which is a lot more complicated and costly than serving static content, which costs a small fraction of real time algorithmic ranking. So now instead of making money off companies that want their data they’re losing money on the cloud costs to serve the scrapers.
More recently, they haven’t been paying their hosting bills to Google and their service was set to expire at the end of last month. What just happened at the start of this month? They added the limitations. That can’t be a coincidence. Now the Google services weren’t hosting the site itself, otherwise they’d probably be fully down right now, but it was hosting their trust and safety services for things like fighting spam. That could have also been hosting anti scraper services as well. Since they’re so under staffed they probably couldn’t swap the services in time, so instead of having anti scraping services like every other big company on the internet, they decided to cut service instead. It’s just built up incompetence coming to a head.