Skip to content

Commit 6cb78dc

Browse files
proxymeshaiproxymeshai
committed
Initial release of ruby-proxy-headers gem
Adds extensions for Ruby HTTP libraries to support sending custom headers during HTTPS CONNECT tunneling and receiving proxy response headers. Supported libraries: - Net::HTTP (stdlib) - Faraday (middleware) - HTTParty - HTTP.rb - Typhoeus - Excon - RestClient Includes comprehensive documentation for ReadTheDocs and marketing plan. Made-with: Cursor
0 parents  commit 6cb78dc

37 files changed

Lines changed: 4134 additions & 0 deletions

‎.github/FUNDING.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
github: proxymeshai
2+
custom: ["https://proxymesh.com"]

‎.readthedocs.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
version: 2
2+
3+
build:
4+
os: ubuntu-22.04
5+
tools:
6+
python: "3.11"
7+
8+
mkdocs:
9+
configuration: mkdocs.yml
10+
11+
python:
12+
install:
13+
- requirements: docs/requirements.txt

‎CHANGELOG.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Changelog
2+
3+
All notable changes to this project will be documented in this file.
4+
5+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7+
8+
## [Unreleased]
9+
10+
## [0.1.0] - 2024-02-28
11+
12+
### Added
13+
14+
- Initial release
15+
- Core `Connection` class for proxy CONNECT handling with custom headers
16+
- `NetHTTP` module for Net::HTTP integration
17+
- `Faraday` middleware for Faraday integration
18+
- `HTTParty` module for HTTParty integration
19+
- `HTTPGem` module for HTTP.rb integration
20+
- `Typhoeus` module for Typhoeus integration
21+
- `Excon` module for Excon integration
22+
- `RestClient` module for RestClient integration
23+
- Comprehensive documentation
24+
- Test suite with RSpec
25+
26+
[Unreleased]: https://github.com/proxymeshai/ruby-proxy-headers/compare/v0.1.0...HEAD
27+
[0.1.0]: https://github.com/proxymeshai/ruby-proxy-headers/releases/tag/v0.1.0

‎Gemfile‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# frozen_string_literal: true
2+
3+
source 'https://rubygems.org'
4+
5+
gemspec
6+
7+
group :development, :test do
8+
gem 'pry', '~> 0.14'
9+
gem 'yard', '~> 0.9'
10+
end

‎IMPLEMENTATION_PRIORITY.md‎

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
# Implementation Priority for ruby-proxy-headers
2+
3+
This document outlines the implementation plan for the ruby-proxy-headers gem.
4+
5+
## Architecture Overview
6+
7+
```
8+
┌─────────────────────────────────────────────────────────────────┐
9+
│ ruby-proxy-headers gem │
10+
├─────────────────────────────────────────────────────────────────┤
11+
│ ┌─────────────────────────────────────────────────────────┐ │
12+
│ │ Core: ProxyHeadersConnection │ │
13+
│ │ - Direct socket handling for CONNECT with headers │ │
14+
│ │ - TLS upgrade after CONNECT │ │
15+
│ │ - Response header capture │ │
16+
│ └─────────────────────────────────────────────────────────┘ │
17+
│ ▲ │
18+
│ │ │
19+
│ ┌─────────────────────────────────────────────────────────┐ │
20+
│ │ Net::HTTP Extension │ │
21+
│ │ - Prepend module to override #connect │ │
22+
│ │ - Store proxy headers on Net::HTTPResponse │ │
23+
│ └─────────────────────────────────────────────────────────┘ │
24+
│ ▲ │
25+
│ ┌───────────────┼───────────────┐ │
26+
│ │ │ │ │
27+
│ ┌───────────┴───┐ ┌───────┴───────┐ ┌───┴───────────┐ │
28+
│ │ Faraday │ │ HTTParty │ │ RestClient │ │
29+
│ │ Middleware │ │ Extension │ │ Extension │ │
30+
│ └───────────────┘ └───────────────┘ └───────────────┘ │
31+
│ │
32+
│ ┌─────────────────────────────────────────────────────────┐ │
33+
│ │ Typhoeus Extension (via Ethon FFI) │ │
34+
│ │ - Direct libcurl CURLOPT_PROXYHEADER support │ │
35+
│ └─────────────────────────────────────────────────────────┘ │
36+
└─────────────────────────────────────────────────────────────────┘
37+
```
38+
39+
## Phase 1: Core Implementation
40+
41+
### 1.1 ProxyHeadersConnection (lib/ruby_proxy_headers/connection.rb)
42+
- Pure Ruby socket handling
43+
- Build CONNECT request with custom headers
44+
- Parse CONNECT response and capture headers
45+
- TLS upgrade handling
46+
- Thread-safe header storage
47+
48+
### 1.2 Net::HTTP Extension (lib/ruby_proxy_headers/net_http.rb)
49+
- Prepend module to `Net::HTTP`
50+
- Override `connect` method to use ProxyHeadersConnection
51+
- Add `proxy_response_headers` accessor to responses
52+
- Configuration via class-level or instance-level options
53+
54+
## Phase 2: Popular Library Integration
55+
56+
### 2.1 Faraday Middleware (lib/ruby_proxy_headers/faraday.rb)
57+
- Register as Faraday middleware
58+
- Wrap connection with proxy header support
59+
- Store proxy headers in `env[:proxy_response_headers]`
60+
- Access via `response.env[:proxy_response_headers]`
61+
62+
### 2.2 HTTParty Extension (lib/ruby_proxy_headers/httparty.rb)
63+
- Module that can be included in HTTParty classes
64+
- Add `proxy_headers` option to requests
65+
- Store proxy response headers on response object
66+
- Works with existing HTTParty patterns
67+
68+
### 2.3 HTTP.rb Extension (lib/ruby_proxy_headers/http_gem.rb)
69+
- Extend HTTP::Client or use feature injection
70+
- Add `.with_proxy_headers(headers)` chainable method
71+
- Access via `response.proxy_headers`
72+
73+
## Phase 3: Additional Libraries
74+
75+
### 3.1 Typhoeus/Ethon Extension (lib/ruby_proxy_headers/typhoeus.rb)
76+
- Extend Ethon::Easy to expose `CURLOPT_PROXYHEADER`
77+
- Use `CURLOPT_HEADERFUNCTION` for response capture
78+
- Leverage native libcurl support for best performance
79+
80+
### 3.2 Excon Extension (lib/ruby_proxy_headers/excon.rb)
81+
- Middleware/interceptor for Excon
82+
- Wrap connection handling
83+
- Store headers in connection data
84+
85+
### 3.3 RestClient Extension (lib/ruby_proxy_headers/rest_client.rb)
86+
- Leverages Net::HTTP extension
87+
- Add accessor for proxy headers on response
88+
89+
## File Structure
90+
91+
```
92+
ruby-proxy-headers/
93+
├── lib/
94+
│ ├── ruby_proxy_headers.rb # Main entry point
95+
│ ├── ruby_proxy_headers/
96+
│ │ ├── version.rb # Gem version
97+
│ │ ├── connection.rb # Core proxy connection handler
98+
│ │ ├── net_http.rb # Net::HTTP extension
99+
│ │ ├── faraday.rb # Faraday middleware
100+
│ │ ├── httparty.rb # HTTParty extension
101+
│ │ ├── http_gem.rb # HTTP.rb extension
102+
│ │ ├── typhoeus.rb # Typhoeus/Ethon extension
103+
│ │ ├── excon.rb # Excon extension
104+
│ │ └── rest_client.rb # RestClient extension
105+
├── spec/
106+
│ ├── spec_helper.rb
107+
│ ├── connection_spec.rb
108+
│ ├── net_http_spec.rb
109+
│ ├── faraday_spec.rb
110+
│ ├── httparty_spec.rb
111+
│ └── ...
112+
├── examples/
113+
│ ├── net_http_example.rb
114+
│ ├── faraday_example.rb
115+
│ └── ...
116+
├── ruby_proxy_headers.gemspec
117+
├── Gemfile
118+
├── Rakefile
119+
├── README.md
120+
├── LICENSE
121+
├── CHANGELOG.md
122+
└── docs/ # ReadTheDocs documentation
123+
├── index.md
124+
├── getting-started.md
125+
├── faraday.md
126+
└── ...
127+
```
128+
129+
## Technical Challenges
130+
131+
### 1. Socket Interception
132+
Ruby's Net::HTTP creates sockets internally. We need to:
133+
- Intercept before TLS upgrade
134+
- Inject CONNECT request with custom headers
135+
- Parse CONNECT response
136+
- Continue with normal TLS handshake
137+
138+
### 2. Thread Safety
139+
Multiple concurrent requests may share connection pools:
140+
- Use thread-local storage for proxy response headers
141+
- Or attach headers to response object directly
142+
143+
### 3. Connection Pooling
144+
Libraries like Faraday may reuse connections:
145+
- Ensure headers are reset per-request
146+
- Don't interfere with keep-alive
147+
148+
### 4. libcurl Integration (Typhoeus)
149+
- Ethon uses FFI, may need to define additional functions
150+
- `CURLOPT_PROXYHEADER` requires curl >= 7.37.0
151+
- Need to handle version detection
152+
153+
## API Design
154+
155+
### Basic Usage
156+
157+
```ruby
158+
require 'ruby_proxy_headers'
159+
160+
# Net::HTTP
161+
response = RubyProxyHeaders::NetHTTP.get(
162+
'https://example.com',
163+
proxy: 'http://user:pass@proxy:8080',
164+
proxy_headers: { 'X-ProxyMesh-Country' => 'US' }
165+
)
166+
puts response.proxy_response_headers['X-ProxyMesh-IP']
167+
168+
# Faraday
169+
conn = Faraday.new(url: 'https://example.com') do |f|
170+
f.use RubyProxyHeaders::Faraday::Middleware,
171+
proxy_headers: { 'X-ProxyMesh-Country' => 'US' }
172+
f.proxy = 'http://user:pass@proxy:8080'
173+
end
174+
response = conn.get('/')
175+
puts response.env[:proxy_response_headers]['X-ProxyMesh-IP']
176+
177+
# HTTParty
178+
class ProxyClient
179+
include HTTParty
180+
include RubyProxyHeaders::HTTParty
181+
182+
http_proxy 'proxy', 8080, 'user', 'pass'
183+
proxy_headers 'X-ProxyMesh-Country' => 'US'
184+
end
185+
response = ProxyClient.get('https://example.com')
186+
puts response.proxy_response_headers['X-ProxyMesh-IP']
187+
```
188+
189+
## Dependencies
190+
191+
### Required
192+
- Ruby >= 2.7 (for pattern matching, numbered block params)
193+
- No external gems for core functionality
194+
195+
### Optional (for specific integrations)
196+
- faraday (>= 1.0) for Faraday middleware
197+
- httparty (>= 0.18) for HTTParty extension
198+
- http (>= 5.0) for HTTP.rb extension
199+
- typhoeus (>= 1.4) for Typhoeus extension
200+
- excon (>= 0.80) for Excon extension
201+
202+
## Testing Strategy
203+
204+
### Unit Tests
205+
- Mock socket connections
206+
- Test CONNECT request building
207+
- Test response parsing
208+
209+
### Integration Tests
210+
- Use actual proxy (local or test service)
211+
- Verify headers sent and received
212+
- Test with each supported library
213+
214+
### Compatibility Tests
215+
- Test with multiple Ruby versions (2.7, 3.0, 3.1, 3.2, 3.3)
216+
- Test with multiple versions of each library

0 commit comments

Comments
 (0)