Skip to content

Commit d571a9d

Browse files
committed
Add requests compatibility documentation to README
1 parent 7360372 commit d571a9d

1 file changed

Lines changed: 110 additions & 0 deletions

File tree

README.md

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ Official Python wrapper for [Scrappey.com](https://scrappey.com) - Web scraping
1616
- **Proxy Support** - Built-in proxy rotation with country selection
1717
- **Async Support** - Both sync and async clients included
1818
- **Type Hints** - Full type annotations for IDE support and AI assistants
19+
- **Drop-in Replacement** - Use as a replacement for the `requests` library
1920

2021
## Installation
2122

@@ -68,6 +69,115 @@ async def main():
6869
asyncio.run(main())
6970
```
7071

72+
## Drop-in Replacement for `requests`
73+
74+
Scrappey provides a **drop-in replacement** for the popular `requests` library. Simply change your import and your existing code will work with Scrappey's Cloudflare bypass and antibot capabilities!
75+
76+
### Migration
77+
78+
```python
79+
# Before (using requests)
80+
import requests
81+
82+
response = requests.get("https://example.com")
83+
print(response.text)
84+
85+
# After (using Scrappey) - just change the import!
86+
from scrappey import requests
87+
88+
response = requests.get("https://example.com")
89+
print(response.text)
90+
```
91+
92+
That's it! Your code now uses Scrappey for automatic Cloudflare bypass.
93+
94+
> **Note**: Set the `SCRAPPEY_API_KEY` environment variable with your API key.
95+
96+
### Response Object
97+
98+
The Response object works exactly like `requests.Response`:
99+
100+
```python
101+
from scrappey import requests
102+
103+
response = requests.get("https://httpbin.org/get")
104+
105+
# All standard attributes
106+
print(response.status_code) # 200
107+
print(response.ok) # True
108+
print(response.text) # Response body as text
109+
print(response.content) # Response body as bytes
110+
print(response.headers) # Response headers
111+
print(response.cookies) # Response cookies
112+
print(response.url) # Final URL
113+
print(response.elapsed) # Time elapsed
114+
115+
# Methods
116+
data = response.json() # Parse JSON
117+
response.raise_for_status() # Raise on 4xx/5xx
118+
```
119+
120+
### Sessions
121+
122+
Sessions maintain cookies and headers across requests:
123+
124+
```python
125+
from scrappey import requests
126+
127+
session = requests.Session()
128+
129+
try:
130+
# Login
131+
session.post("https://example.com/login", data={"user": "test"})
132+
133+
# Subsequent requests include cookies from login
134+
response = session.get("https://example.com/dashboard")
135+
136+
# Session-level headers
137+
session.headers.update({"Authorization": "Bearer token"})
138+
finally:
139+
session.close() # Clean up Scrappey session
140+
```
141+
142+
Or use as a context manager:
143+
144+
```python
145+
from scrappey import requests
146+
147+
with requests.Session() as session:
148+
session.get("https://example.com")
149+
# Session automatically closed when exiting
150+
```
151+
152+
### Supported Parameters
153+
154+
| Parameter | Supported | Notes |
155+
|-----------|-----------|-------|
156+
| `params` | Yes | Query parameters |
157+
| `data` | Yes | Form data |
158+
| `json` | Yes | JSON data |
159+
| `headers` | Yes | Custom headers |
160+
| `cookies` | Yes | Request cookies |
161+
| `timeout` | Yes | Request timeout |
162+
| `proxies` | Yes | Proxy configuration |
163+
| `allow_redirects` | Warn | Handled by browser |
164+
| `verify` | Warn | SSL handled by service |
165+
| `stream` | Warn | Not supported |
166+
| `files` | Warn | Not supported |
167+
| `auth` | Warn | Use headers instead |
168+
169+
### Why Use This?
170+
171+
Sites protected by Cloudflare, Datadome, PerimeterX, and other antibots will **just work**:
172+
173+
```python
174+
from scrappey import requests
175+
176+
# This would fail with regular requests, but works with Scrappey!
177+
response = requests.get("https://nowsecure.nl/") # CF-protected
178+
print(response.status_code) # 200
179+
```
180+
71181
## Examples
72182

73183
### Cloudflare Bypass

0 commit comments

Comments
 (0)