Skip to content

Commit b797332

Browse files
AndreiDrangAI Agent
AndreiDrang
and
AI Agent
committed
docs: add ARCHITECTURE.md with system architecture documentation
Document layered architecture (Service → Base → Instrument → Support), dual sync/async design, msgspec serialization, and tenacity retry patterns. Co-Authored-By: AI Agent <ai-agent@localhost>
1 parent 12c5416 commit b797332

1 file changed

Lines changed: 308 additions & 0 deletions

File tree

‎ARCHITECTURE.md‎

Lines changed: 308 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,308 @@
1+
# Architecture
2+
3+
## 1. High-Level Overview
4+
5+
`python3-capsolver` is a Python 3.8+ client library for the Capsolver captcha-solving service API. It provides a unified, type-safe interface for solving various captcha types (ReCaptcha, Cloudflare Turnstile, GeeTest, DataDome, AWS WAF, etc.) through both synchronous and asynchronous execution models.
6+
7+
**Observed**: The library follows a layered architecture where core infrastructure (`core/`) provides base classes, serialization, and HTTP instruments, while service-specific modules (e.g., `recaptcha.py`, `cloudflare.py`) implement concrete captcha types. Dual sync/async support is achieved through separate instrument classes (`SIOCaptchaInstrument`, `AIOCaptchaInstrument`) that share a common interface.
8+
9+
**Evidence Anchors**:
10+
- `pyproject.toml`: Dependencies (`requests`, `aiohttp`, `msgspec`, `tenacity`), build system (setuptools), Python >=3.8
11+
- `src/python3_capsolver/core/base.py`: `CaptchaParams` base class with `captcha_handler()` and `aio_captcha_handler()`
12+
- `src/python3_capsolver/core/enum.py`: Type-safe enums for captcha types, endpoints, response statuses
13+
- `src/python3_capsolver/*.py`: Service-specific implementations (10+ captcha types)
14+
- `README.md`: Usage examples, feature list, supported captcha types
15+
16+
**Inferred**: The library prioritizes performance (hence `msgspec` over `json`) and resilience (retry logic via `tenacity` and `requests.Retry`). The separation of concerns between core infrastructure and service implementations suggests an intentional design for extensibility.
17+
18+
**Unknown**: Whether there are any plans to support additional captcha types beyond those currently implemented.
19+
20+
## 2. System Architecture (Logical)
21+
22+
### Logical Components
23+
24+
The library consists of four logical layers:
25+
26+
1. **Service Layer** (`src/python3_capsolver/*.py`): High-level classes for each captcha type (e.g., `ReCaptcha`, `Cloudflare`, `Control`). Each class encapsulates captcha-specific parameters and provides sync/async handlers.
27+
28+
2. **Base Layer** (`src/python3_capsolver/core/base.py`): The `CaptchaParams` class serves as the common base for all service classes. It handles payload serialization, URL configuration, and delegates to appropriate instruments.
29+
30+
3. **Instrument Layer** (`src/python3_capsolver/core/*instrument.py`): HTTP client abstractions that manage API communication:
31+
- `CaptchaInstrumentBase`: Abstract base with retry logic and result polling
32+
- `SIOCaptchaInstrument`: Synchronous implementation using `requests`
33+
- `AIOCaptchaInstrument`: Asynchronous implementation using `aiohttp`
34+
- `FileInstrument`: File/URL/base64 processing utilities
35+
36+
4. **Support Layer** (`src/python3_capsolver/core/`): Utilities including:
37+
- `serializer.py`: `msgspec.Struct` classes for request/response serialization
38+
- `enum.py`: Type-safe enumerations
39+
- `const.py`: Configuration constants (API URLs, retry settings)
40+
- `utils.py`: Utility functions
41+
- `context_instr.py`: Context manager mixins for session cleanup
42+
43+
### Dependency Direction
44+
45+
```
46+
Service Layer (recaptcha.py, cloudflare.py, ...)
47+
↓
48+
Base Layer (base.py: CaptchaParams)
49+
↓
50+
Instrument Layer (*instrument.py: HTTP clients)
51+
↓
52+
Support Layer (serializer, enum, const, utils)
53+
↓
54+
External Dependencies (requests, aiohttp, msgspec, tenacity)
55+
```
56+
57+
**Allowed Dependencies**:
58+
- Service classes → `CaptchaParams` (inheritance) + enums
59+
- `CaptchaParams` → Instruments + serializers
60+
- Instruments → Serializers + constants + external HTTP libraries
61+
- Support layer → External libraries only (no internal dependencies)
62+
63+
**Forbidden Dependencies** (Inferred from structure):
64+
- Support layer → Service/Base layers (would create circular dependencies)
65+
- Instruments → Service-specific logic (violates separation of concerns)
66+
- Service classes → Direct HTTP calls (must go through instruments)
67+
68+
### SSR/Hybrid Boundaries
69+
70+
Not applicable. This is a pure Python library with no frontend/web rendering components.
71+
72+
### Monorepo Status
73+
74+
Not applicable. Single-package repository with standard `src/` layout.
75+
76+
## 3. Code Map (Physical)
77+
78+
```
79+
python3-capsolver/
80+
├── src/python3_capsolver/ # Main library package
81+
│ ├── core/ # Core infrastructure (stable, rarely changes)
82+
│ │ ├── base.py # CaptchaParams base class
83+
│ │ ├── captcha_instrument.py # Base + File instrument
84+
│ │ ├── sio_captcha_instrument.py # Sync HTTP client
85+
│ │ ├── aio_captcha_instrument.py # Async HTTP client
86+
│ │ ├── serializer.py # msgspec serialization
87+
│ │ ├── enum.py # Type-safe enums
88+
│ │ ├── const.py # Constants
89+
│ │ └── utils.py # Utilities
90+
│ │
91+
│ ├── control.py # Direct API methods (balance, task management)
92+
│ ├── recaptcha.py # ReCaptcha V2/V3/Enterprise
93+
│ ├── cloudflare.py # Cloudflare Turnstile/Challenge
94+
│ ├── gee_test.py # GeeTest V3/V4
95+
│ ├── datadome_slider.py # DataDome slider captcha
96+
│ ├── mt_captcha.py # MtCaptcha
97+
│ ├── aws_waf.py # AWS WAF bypass
98+
│ ├── friendly_captcha.py # FriendlyCaptcha
99+
│ ├── yandex.py # Yandex SmartCaptcha
100+
│ ├── image_to_text.py # OCR text extraction
101+
│ ├── vision_engine.py # AI-based image recognition
102+
│ ├── __init__.py # Package entry (exports version)
103+
│ └── __version__.py # Version string
104+
│
105+
├── tests/ # Pytest test suite (mirrors src/ structure)
106+
│ ├── conftest.py # Pytest fixtures and configuration
107+
│ ├── test_control.py # Control class tests
108+
│ ├── test_core.py # Core module tests
109+
│ ├── test_recaptcha.py # ReCaptcha tests
110+
│ ├── test_cloudflare.py # Cloudflare tests
111+
│ ├── test_instrument.py # Instrument tests
112+
│ └── ... # One test file per service
113+
│
114+
├── docs/ # Sphinx documentation
115+
│ ├── source/ # Documentation source files
116+
│ └── AGENTS.md # Documentation module context
117+
│
118+
├── .github/workflows/ # CI/CD pipelines
119+
│ ├── build.yml # Package build
120+
│ ├── test.yml # Test execution
121+
│ ├── lint.yml # Code quality checks
122+
│ ├── sphinx.yml # Documentation build
123+
│ └── install.yml # Installation verification
124+
│
125+
├── pyproject.toml # Build, dependency, tool configuration
126+
├── Makefile # Developer convenience commands
127+
├── README.md # Usage guide and quick start
128+
├── AGENTS.md # Project knowledge base (for LLMs)
129+
└── uv.lock # Dependency lock file (uv package manager)
130+
```
131+
132+
**Where to Look**:
133+
- **Adding a new captcha type**: Create new file in `src/python3_capsolver/`, inherit from `CaptchaParams`, follow pattern in `recaptcha.py`
134+
- **Changing API communication**: Modify instrument classes in `src/python3_capsolver/core/`
135+
- **Updating serialization**: Edit `src/python3_capsolver/core/serializer.py`
136+
- **Adding captcha types**: Update `CaptchaTypeEnm` in `src/python3_capsolver/core/enum.py`
137+
- **Test patterns**: See `tests/test_recaptcha.py` for service tests, `tests/test_core.py` for core tests
138+
139+
## 4. Life of a Request / Primary Data Flow
140+
141+
### Synchronous Flow (e.g., ReCaptcha solving)
142+
143+
1. **Initialization** (`recaptcha.py:ReCaptcha.__init__()`):
144+
- User instantiates `ReCaptcha(api_key="...", captcha_type=CaptchaTypeEnm.ReCaptchaV2Task)`
145+
- Calls `CaptchaParams.__init__()` which:
146+
- Serializes API key into `create_task_payload` (via `RequestCreateTaskSer`)
147+
- Initializes `task_params` with captcha type (via `TaskSer`)
148+
- Creates `get_result_params` (via `RequestGetTaskResultSer`)
149+
150+
2. **Handler Invocation** (`recaptcha.py:ReCaptcha.captcha_handler()` via inheritance):
151+
- User calls `.captcha_handler(task_payload={"websiteURL": "...", "websiteKey": "..."})`
152+
- Updates `self.task_params` with user-provided payload
153+
- Instantiates `SIOCaptchaInstrument(captcha_params=self)`
154+
155+
3. **Task Creation** (`sio_captcha_instrument.py:SIOCaptchaInstrument.processing_captcha()`):
156+
- Sends POST to `{request_url}/createTask` with serialized payload
157+
- Receives `taskId` in response
158+
159+
4. **Result Polling** (`sio_captcha_instrument.py`):
160+
- Polls `{request_url}/getTaskResult` with `taskId` at intervals (`sleep_time`, default 5s)
161+
- Retries on transient failures (via `requests.Retry` adapter, 5 attempts)
162+
- Continues until status is `ready`, `failed`, or retry limit exceeded
163+
164+
5. **Response**:
165+
- Returns dict with `errorId`, `taskId`, `status`, `solution` fields
166+
- User extracts `solution` object containing captcha solution
167+
168+
### Asynchronous Flow
169+
170+
Identical to sync flow, except:
171+
- Uses `AIOCaptchaInstrument` with `aiohttp.ClientSession`
172+
- Retries via `tenacity` (async retries decorator)
173+
- Handler is `await solver.aio_captcha_handler(task_payload)`
174+
175+
### Context Manager Flow
176+
177+
Both sync and async classes support context managers for automatic session cleanup:
178+
179+
```python
180+
# Sync
181+
with ReCaptcha(api_key="...") as solver:
182+
result = solver.captcha_handler(task_payload)
183+
184+
# Async
185+
async with ReCaptcha(api_key="...") as solver:
186+
result = await solver.aio_captcha_handler(task_payload)
187+
```
188+
189+
**Evidence Anchors**:
190+
- `src/python3_capsolver/core/base.py`: Lines 25-41 (initialization), 43-61 (sync handler), 63-80 (async handler)
191+
- `src/python3_capsolver/core/sio_captcha_instrument.py`: `processing_captcha()` method
192+
- `src/python3_capsolver/core/aio_captcha_instrument.py`: Async `processing_captcha()` method
193+
- `src/python3_capsolver/core/context_instr.py`: `SIOContextManager`, `AIOContextManager` mixins
194+
195+
## 5. Architectural Invariants & Constraints
196+
197+
### Invariant 1: Dual Sync/Async Support
198+
199+
- **Rule**: Every captcha-solving operation must provide both synchronous and asynchronous implementations.
200+
- **Rationale**: Users may operate in sync or async codebases; the library must support both without forcing a choice.
201+
- **Enforcement / Signals**:
202+
- `CaptchaParams` base class defines both `captcha_handler()` and `aio_captcha_handler()` methods
203+
- Separate instrument classes (`SIOCaptchaInstrument`, `AIOCaptchaInstrument`) implement each mode
204+
- Tests include both sync and async variants (e.g., `test_recaptcha.py`)
205+
206+
### Invariant 2: Service Classes Are Thin Wrappers
207+
208+
- **Rule**: Service-specific classes (e.g., `ReCaptcha`, `Cloudflare`) must not contain HTTP logic or API communication code. They only specify captcha type and inherit behavior from `CaptchaParams`.
209+
- **Rationale**: Separation of concerns—API communication is infrastructure, captcha parameters are domain logic. This enables easy addition of new captcha types.
210+
- **Enforcement / Signals**:
211+
- All service classes inherit from `CaptchaParams` and typically only define `__init__()` (see `recaptcha.py:79-81`, 3 lines total)
212+
- Code review / lint checks would flag HTTP calls in service files
213+
214+
### Invariant 3: Serialization via msgspec
215+
216+
- **Rule**: All request/response serialization must use `msgspec.Struct` classes, not raw dicts or `json` module.
217+
- **Rationale**: Performance—`msgspec` is significantly faster than `json` for serialization/deserialization.
218+
- **Enforcement / Signals**:
219+
- `src/python3_capsolver/core/serializer.py` defines `RequestCreateTaskSer`, `CaptchaResponseSer`, etc.
220+
- Dependencies list `msgspec` but not alternative serializers
221+
- `pyproject.toml` explicitly notes "msgspec preferred over json for performance" (AGENTS.md)
222+
223+
### Invariant 4: Retry Logic Is Mandatory
224+
225+
- **Rule**: All HTTP requests must include retry logic with exponential backoff.
226+
- **Rationale**: Captcha-solving is time-sensitive and external API calls may fail transiently; automatic retries improve reliability.
227+
- **Enforcement / Signals**:
228+
- `SIOCaptchaInstrument` uses `requests.adapters.HTTPAdapter(max_retries=RETRIES)` (see `const.py` for retry config)
229+
- `AIOCaptchaInstrument` uses `tenacity` async retries (`ASYNC_RETRIES` in `const.py`)
230+
- Instruments are the only classes performing HTTP operations; base layer enforces their use
231+
232+
### Invariant 5: Type Safety via Enums
233+
234+
- **Rule**: Captcha types, endpoint names, and response statuses must use enumerations (`CaptchaTypeEnm`, `EndpointPostfixEnm`, `ResponseStatusEnm`), not raw strings.
235+
- **Rationale**: Prevents typos, enables IDE autocomplete, documents valid values explicitly.
236+
- **Enforcement / Signals**:
237+
- `CaptchaParams.__init__()` accepts `captcha_type: CaptchaTypeEnm` (typed parameter)
238+
- All service examples in docstrings use enum values (e.g., `CaptchaTypeEnm.ReCaptchaV2Task`)
239+
- `enum.py` defines all valid values in one location
240+
241+
### Invariant 6: No Direct HTTP Calls in Service Layer
242+
243+
- **Rule**: Service classes (`recaptcha.py`, `cloudflare.py`, etc.) must not import or use `requests`/`aiohttp` directly. All HTTP operations go through instruments.
244+
- **Rationale**: Maintains separation of concerns, enables consistent retry/error-handling logic.
245+
- **Enforcement / Signals**:
246+
- Service classes import only from `.core.base` and `.core.enum`
247+
- Instruments encapsulate all HTTP session management
248+
- Static analysis / linting would catch violations
249+
250+
### Invariant 7: Context Manager Support
251+
252+
- **Rule**: All captcha-solving classes must support Python context managers (`with` / `async with`) for automatic resource cleanup.
253+
- **Rationale**: Ensures HTTP sessions are properly closed, prevents resource leaks.
254+
- **Enforcement / Signals**:
255+
- `CaptchaParams` inherits from `SIOContextManager` and `AIOContextManager` (see `base.py:14`)
256+
- Context managers defined in `src/python3_capsolver/core/context_instr.py`
257+
- Usage examples in docstrings show context manager patterns
258+
259+
## 6. Documentation Strategy
260+
261+
### Documentation Hierarchy
262+
263+
**ARCHITECTURE.md** (this document):
264+
- High-level system map and component relationships
265+
- Architectural invariants and constraints
266+
- Primary data flow and execution paths
267+
- Physical code map ("where is X?")
268+
- Stable information unlikely to change frequently
269+
270+
**Module-Level AGENTS.md/README.md** files:
271+
- `AGENTS.md` (root): Project overview, structure, conventions, commands
272+
- `src/python3_capsolver/AGENTS.md`: Package-level context, service list, usage patterns
273+
- `src/python3_capsolver/core/AGENTS.md`: Core module details, class responsibilities, conventions
274+
- Module docstrings: Class/function-level documentation with examples
275+
276+
**External Documentation**:
277+
- Sphinx documentation in `docs/` directory (generated from docstrings)
278+
- README.md: Quick start, installation, usage examples
279+
- CHANGELOG.md: Version history and breaking changes
280+
281+
### What Belongs Where
282+
283+
| Information Type | Location |
284+
|------------------|----------|
285+
| System architecture, boundaries | ARCHITECTURE.md |
286+
| Dependency rules, invariants | ARCHITECTURE.md |
287+
| "Where is X?" questions | ARCHITECTURE.md (Code Map) |
288+
| Module purpose, structure | `<module>/AGENTS.md` |
289+
| Class/function usage examples | Docstrings (inline) |
290+
| Installation, quick start | README.md |
291+
| API parameter details | Sphinx docs (auto-generated) |
292+
| Version changes | CHANGELOG.md |
293+
| Development workflows | AGENTS.md (COMMANDS section) |
294+
295+
### Documentation Conventions
296+
297+
- **AGENTS.md files**: Generated knowledge base for LLMs and developers, following consistent template (OVERVIEW, STRUCTURE, WHERE TO LOOK, CONVENTIONS, COMMANDS)
298+
- **Docstrings**: Google-style with Args, Returns, Examples, Notes sections
299+
- **Type hints**: Full type annotations on all public APIs
300+
- **Code examples**: Both sync and async variants shown in docstrings
301+
302+
### Evidence Anchors for Documentation
303+
304+
- Root `AGENTS.md`: Lines 1-41 (project structure, conventions, commands)
305+
- `src/python3_capsolver/AGENTS.md`: Service pattern, dual handlers, retry logic
306+
- `src/python3_capsolver/core/AGENTS.md`: Instrument patterns, serialization, retries
307+
- `pyproject.toml`: Tool configuration (black, isort, pytest)
308+
- `docs/`: Sphinx documentation source (auto-generated from docstrings)

0 commit comments

Comments
 (0)