# Python Requests

The simplest way to add helodata to a Python scraper.

## Install

```bash
pip install requests
# For SOCKS5 support:
pip install requests[socks]
```

## Residential / Mobile (gateway)

```python
import requests

USER = "helo_s1a2b3c4d5e-type-res-region-us"      # change type-res to type-mob for Mobile
PASS = "PASSWORD"
proxy = f"http://{USER}:{PASS}@gate.helodata.io:7777"

r = requests.get(
    "https://ipv4.icanhazip.com",
    proxies={"http": proxy, "https": proxy},
    timeout=30,
)
print(r.text.strip())
```

## ISP

```python
PROXY = "http://helo_s1a2b3c4d5e:PASSWORD@198.51.100.42:8000"

r = requests.get(
    "https://ipv4.icanhazip.com",
    proxies={"http": PROXY, "https": PROXY},
    timeout=30,
)
```

## SOCKS5 with remote DNS

Crucial: use `socks5h://` (with the `h`) so the proxy resolves DNS — otherwise the local resolver leaks DNS and geo-targeting breaks.

```python
proxy = f"socks5h://{USER}:{PASS}@gate.helodata.io:7777"
```

## Session (HTTP keep-alive)

Reusing connections across requests is faster, especially for ISP (stable IPs):

```python
s = requests.Session()
s.proxies = {"http": proxy, "https": proxy}

for url in urls:
    r = s.get(url, timeout=30)
```

## Rotating residential sessions

Mint a fresh `session-XXX` for each batch of N requests:

```python
import random, string

def make_user(country="us"):
    sid = "".join(random.choices(string.ascii_lowercase + string.digits, k=8))
    return f"helo_s1a2b3c4d5e-type-res-region-{country}-session-{sid}-sesstime-10"
```

## ISP round-robin

```python
import itertools
with open("isp.txt") as f:
    pool = itertools.cycle([l.strip() for l in f if l.strip()])

def fetch(url):
    ip, port, user, password = next(pool).split(":", 3)
    proxy = f"http://{user}:{password}@{ip}:{port}"
    return requests.get(url, proxies={"http": proxy, "https": proxy}, timeout=30)
```

See [Client-side rotation](/products/overview-2/rotation.md) for production-quality patterns.

## Retries

Use `urllib3.Retry` for transient errors:

```python
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

retry = Retry(total=5, backoff_factor=0.5, status_forcelist=[429, 502, 522, 524])
s.mount("https://", HTTPAdapter(max_retries=retry))
```

## Verify

```python
r = s.get("https://ipv4.icanhazip.com")
print(r.text.strip(), r.headers.get("X-Helodata-Exit-IP"))
```

## Common pitfalls

* **Forgetting `https` key in proxies dict** — Python `requests` requires both `http` and `https` entries even when only HTTPS targets are used.
* **`socks5://` instead of `socks5h://`** — DNS leak. Always use `socks5h://` for SOCKS5.
* **No timeout** — requests can hang forever on a flaky upstream. Always pass `timeout`.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.helodata.com/integrations/scraping-tools/python-requests.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.