Example Reference
Good bots
Baidu: https://help.baidu.com/question?prod_id=99&class=0&id=3001 https://udger.com/resources/ua-list/bot-detail?bot=Baiduspider
Google: https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers https://developers.google.com/static/search/apis/ipranges/googlebot.json
Meta/Faceboot: https://developers.facebook.com/docs/sharing/webmasters/crawler/