试了一下这个网站,按照浏览器方式构造请求能拿到json数据,python也照这样填写请求header就行,也许还能精剪,你可以慢慢试。
命令行用的msys2方式,用powershell或者cmd.exe试可能需要改一下格式。
curl 'https://www.cmegroup.com/services/sofr-strip-rates/' \
-H 'authority: www.cmegroup.com' \
-H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7' \
-H 'accept-language: en-US;q=0.9,en;q=0.9' \
-H 'cache-control: max-age=0' \
-H 'sec-ch-ua: "Chromium";v="118", "Google Chrome";v="118", "Not=A?Brand";v="99"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "Windows"' \
-H 'sec-fetch-dest: document' \
-H 'sec-fetch-mode: navigate' \
-H 'sec-fetch-site: cross-site' \
-H 'sec-fetch-user: ?1' \
-H 'upgrade-insecure-requests: 1' \
-H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36' \
--compressed
【 在 t430 的大作中提到: 】
: 页面
:
https://www.cmegroup.com/services/sofr-strip-rates/: (这个页面需要爬到外面才能访问, 我是在外面AWS用开发工具做测试)
: ...................
--
FROM 171.221.52.*