request with sessions
Short answer
Use a requests.Session to get connection pooling and keep-alive automatically.
Tune pool size and retries by mounting a custom HTTPAdapter.
Add caching with a library: requests-cache (easy, flexible) or CacheControl (RFC-compliant HTTP caching).
Details and examples
Connection pooling and keep-alive with requests
requests uses urllib3 under the hood. A Session keeps a pool of persistent HTTP connections per host (scheme://host:port) and will reuse them for subsequent requests.
Keep-alive is on by default for HTTP/1.1; you do not need to set Connection: keep-alive manually.
Always reuse a Session instead of calling requests.get(...) repeatedly.
Close your Session when done (or use it as a context manager).
request with sessions
Short answer
Use a requests.Session to get connection pooling and keep-alive automatically.
Tune pool size and retries by mounting a custom HTTPAdapter.
Add caching with a library: requests-cache (easy, flexible) or CacheControl (RFC-compliant HTTP caching).
Details and examples
Connection pooling and keep-alive with requests
requests uses urllib3 under the hood. A Session keeps a pool of persistent HTTP connections per host (scheme://host:port) and will reuse them for subsequent requests.
Keep-alive is on by default for HTTP/1.1; you do not need to set Connection: keep-alive manually.
Always reuse a Session instead of calling requests.get(...) repeatedly.
Close your Session when done (or use it as a context manager).