Skip to content

Commit 30f50bb

Browse files
committed
chore(doc/internal): standardize docs
1 parent a33f690 commit 30f50bb

5 files changed

Lines changed: 81 additions & 81 deletions

File tree

doc/internal/architecture.md

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
1-
# Why We Had to Reimplement Node.js's SQLite Code
1+
# Why we had to reimplement Node.js's SQLite code
22

33
This document explains why we couldn't directly use Node.js's C++ SQLite implementation and instead had to create a compatibility layer.
44

5-
## The Core Problem: Node.js Internals
5+
## The core problem: Node.js internals
66

77
Node.js's SQLite implementation (`node_sqlite.cc`) is deeply integrated with Node.js internals that are not accessible to addon developers. These internals include:
88

9-
### 1. BaseObject Class Hierarchy
9+
### 1. BaseObject class hierarchy
1010

1111
```cpp
1212
// Node.js internal code
@@ -19,7 +19,7 @@ class DatabaseSync : public BaseObject {
1919
- Provides automatic memory management and handle wrapping
2020
- Not exposed in Node.js public headers
2121
22-
### 2. Environment Class
22+
### 2. Environment class
2323
2424
```cpp
2525
// Extensive use of Environment throughout
@@ -34,7 +34,7 @@ env->sqlite_*_string() // Cached strings
3434
- Permission system integration
3535
- Not available to addons
3636

37-
### 3. Internal Headers
37+
### 3. Internal headers
3838

3939
The implementation requires headers that aren't installed with Node.js:
4040

@@ -44,7 +44,7 @@ The implementation requires headers that aren't installed with Node.js:
4444
- `node_mem-inl.h`
4545
- `util-inl.h`
4646

47-
### 4. Error Handling System
47+
### 4. Error handling system
4848

4949
```cpp
5050
// Internal error macros
@@ -56,7 +56,7 @@ CHECK_ERROR_OR_THROW(env, r, SQLITE_OK, void());
5656
- Integrated with Node.js error codes
5757
- Not available in public API
5858
59-
### 5. Memory Tracking
59+
### 5. Memory tracking
6060
6161
```cpp
6262
void MemoryInfo(MemoryTracker* tracker) const override {
@@ -68,7 +68,7 @@ void MemoryInfo(MemoryTracker* tracker) const override {
6868
- Required for all BaseObject derivatives
6969
- Not relevant for addons
7070

71-
### 6. ThreadPoolWork Integration
71+
### 6. ThreadPoolWork integration
7272

7373
```cpp
7474
class NodeSqliteWork : public ThreadPoolWork {
@@ -79,7 +79,7 @@ class NodeSqliteWork : public ThreadPoolWork {
7979
- Direct integration with libuv thread pool
8080
- Not exposed to addon developers
8181
82-
## Our Solution: Shim Layer
82+
## Our solution: shim layer
8383
8484
Instead of rewriting from scratch, we created a shim layer (`src/shims/`) that provides minimal implementations of these Node.js internals:
8585
@@ -93,7 +93,7 @@ class BaseObject : public Napi::ObjectWrap<T> {
9393
};
9494
```
9595

96-
### 2. Environment → Minimal Implementation
96+
### 2. Environment → minimal implementation
9797

9898
```cpp
9999
// Our shim provides just what SQLite needs
@@ -104,15 +104,15 @@ class Environment {
104104
};
105105
```
106106
107-
### 3. Error Handling → N-API
107+
### 3. Error handling → N-API
108108
109109
```cpp
110110
// Convert internal macros to N-API
111111
#define THROW_ERR_INVALID_ARG_TYPE(env, msg) \
112112
Napi::TypeError::New(env, msg).ThrowAsJavaScriptException()
113113
```
114114

115-
### 4. Memory TrackingStub
115+
### 4. Memory trackingstub
116116

117117
```cpp
118118
// Stub implementation since we don't need heap snapshots
@@ -130,22 +130,22 @@ void QueueWork(std::function<void()> work) {
130130
}
131131
```
132132

133-
## Key Adaptations Made
133+
## Key adaptations made
134134

135135
1. **V8 API → N-API**: Converted all direct V8 API usage to N-API equivalents
136136
2. **Internal Classes → N-API Patterns**: Replaced BaseObject with Napi::ObjectWrap
137137
3. **Error System → N-API Exceptions**: Adapted error handling to use N-API
138138
4. **String Caching → Local Implementation**: Created simple string cache for performance
139139
5. **Memory Tracking → Removed**: Not needed for addon use case
140140

141-
## Benefits of This Approach
141+
## Benefits of this approach
142142

143143
1. **Maintains Compatibility**: Same API surface as Node.js built-in
144144
2. **Preserves Logic**: SQLite usage patterns remain identical
145145
3. **Easier Updates**: Can sync with upstream changes
146146
4. **Cross-Version Support**: Works with any Node.js version that has N-API
147147

148-
## Alternative Approaches Considered
148+
## Alternative approaches considered
149149

150150
1. **Complete Rewrite**: Would lose compatibility and require extensive testing
151151
2. **Fork Node.js**: Would require users to use custom Node.js build

doc/internal/async-design.md

Lines changed: 28 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
1-
# Async API Design Analysis for @photostructure/sqlite
1+
# Async API design analysis for @photostructure/sqlite
22

33
## Summary
44

55
This document analyzes options for adding asynchronous API support to @photostructure/sqlite (currently sync-only, matching Node.js's built-in sqlite module). We recommend a separate package for the async API rather than integrating it into the existing library.
66

7-
## Current State
7+
## Current state
88

9-
### What We Have
9+
### What we have
1010

1111
The @photostructure/sqlite library provides:
1212

@@ -16,15 +16,15 @@ The @photostructure/sqlite library provides:
1616
- **Full SQLite functionality** including user-defined functions, aggregates, and more
1717
- **Cross-platform support** with prebuilds for major platforms
1818

19-
### Technical Foundation
19+
### Technical foundation
2020

2121
The library is built on:
2222

2323
- **SQLite amalgamation** (sqlite3.c) compiled directly into the addon
2424
- **Node-addon-api** for C++ to JavaScript bindings
2525
- **Synchronous execution model** where all operations block the JavaScript thread
2626

27-
## The Challenge
27+
## The challenge
2828

2929
SQLite's C API is fundamentally synchronous. Operations like `sqlite3_step()`, `sqlite3_exec()`, and `sqlite3_prepare()` block until completion. To provide an async API, we need to:
3030

@@ -33,9 +33,9 @@ SQLite's C API is fundamentally synchronous. Operations like `sqlite3_step()`, `
3333
3. Handle concurrent access safely
3434
4. Maintain proper connection lifecycle
3535

36-
## Design Options Analysis
36+
## Design options analysis
3737

38-
### Option 1: Integrated Async API in Existing Library
38+
### Option 1: integrated async API in existing library
3939

4040
Add async classes alongside sync classes in the same package:
4141

@@ -60,7 +60,7 @@ export { Database, Statement } from "./async";
6060
- Larger package size for all users
6161
- Testing complexity increases significantly
6262

63-
### Option 2: Separate Async Package (Recommended)
63+
### Option 2: separate async package (recommended)
6464

6565
Create a new package `@photostructure/sqlite-async`:
6666

@@ -89,7 +89,7 @@ export class Statement { ... }
8989
- Some code duplication
9090
- Need to coordinate SQLite version updates
9191

92-
### Option 3: Modular Architecture
92+
### Option 3: modular architecture
9393

9494
Create three packages with shared core:
9595

@@ -111,7 +111,7 @@ Create three packages with shared core:
111111
- Three packages to maintain
112112
- Dependency versioning complexity
113113

114-
## Recommended Approach: Separate Async Package
114+
## Recommended approach: separate async package
115115

116116
We recommend **Option 2** for these reasons:
117117

@@ -120,9 +120,9 @@ We recommend **Option 2** for these reasons:
120120
3. **Risk Mitigation**: No chance of breaking existing sync users
121121
4. **Clean Implementation**: Can use AsyncWorker pattern from the start
122122

123-
## Async API Design Principles
123+
## Async API design principles
124124

125-
### 1. Promise-Based API
125+
### 1. Promise-based API
126126

127127
All operations return promises:
128128

@@ -146,7 +146,7 @@ class Statement {
146146
}
147147
```
148148

149-
### 2. Connection Pooling
149+
### 2. Connection pooling
150150

151151
Since operations run on worker threads, we can support concurrent operations:
152152

@@ -158,7 +158,7 @@ interface PoolOptions {
158158
}
159159
```
160160

161-
### 3. AsyncWorker Implementation
161+
### 3. AsyncWorker implementation
162162

163163
Use node-addon-api's AsyncWorker for all operations:
164164

@@ -181,14 +181,14 @@ class OpenWorker : public Napi::AsyncWorker {
181181
};
182182
```
183183
184-
### 4. Thread Safety
184+
### 4. Thread safety
185185
186186
- Each Database instance owns its sqlite3\* connection
187187
- Operations are serialized per connection
188188
- Multiple Database instances can work in parallel
189189
- Use SQLITE_OPEN_FULLMUTEX for thread safety
190190
191-
### 5. Streaming Support
191+
### 5. Streaming support
192192
193193
For large result sets:
194194
@@ -202,45 +202,45 @@ for await (const row of statement.iterate()) {
202202
statement.stream().pipe(transform).pipe(output);
203203
```
204204

205-
## Implementation Roadmap
205+
## Implementation roadmap
206206

207-
### Phase 1: Core Architecture
207+
### Phase 1: core architecture
208208

209209
1. Create new repository/package structure
210210
2. Set up AsyncWorker base classes
211211
3. Implement Database.open() and Database.close()
212212
4. Add basic error handling
213213

214-
### Phase 2: Statement Operations
214+
### Phase 2: statement operations
215215

216216
1. Implement prepare() with AsyncWorker
217217
2. Add run(), get(), all() methods
218218
3. Implement parameter binding
219219
4. Add finalize() support
220220

221-
### Phase 3: Advanced Features
221+
### Phase 3: advanced features
222222

223223
1. Connection pooling
224224
2. Async iterators
225225
3. Stream support
226226
4. Transaction helpers
227227

228-
### Phase 4: Feature Parity
228+
### Phase 4: feature parity
229229

230230
1. User-defined functions (async callbacks)
231231
2. Backup API (progress callbacks)
232232
3. Busy handlers
233233
4. All remaining features
234234

235-
## Technical Considerations
235+
## Technical considerations
236236

237-
### 1. Memory Management
237+
### 1. Memory management
238238

239239
- AsyncWorker automatically handles worker thread lifecycle
240240
- Need careful management of sqlite3\* pointers
241241
- Statement objects must track their parent Database
242242

243-
### 2. Error Handling
243+
### 2. Error handling
244244

245245
- Errors in Execute() are automatically converted to promise rejections
246246
- SQLite error messages must be copied (not referenced)
@@ -258,15 +258,15 @@ statement.stream().pipe(transform).pipe(output);
258258
- Match Node.js sqlite error behaviors
259259
- Provide migration guide from sync API
260260

261-
## Testing Strategy
261+
## Testing strategy
262262

263263
1. **Port existing tests** - Adapt sync tests to async
264264
2. **Concurrency tests** - Verify thread safety
265265
3. **Performance benchmarks** - Compare with sync API
266266
4. **Stress tests** - Connection pool limits
267267
5. **Integration tests** - Real-world usage patterns
268268

269-
## Migration Guide (Future)
269+
## Migration guide (future)
270270

271271
For users moving from sync to async:
272272

@@ -282,7 +282,7 @@ const stmt = await db.prepare("SELECT * FROM users WHERE id = ?");
282282
const user = await stmt.get(userId);
283283
```
284284

285-
## Open Questions
285+
## Open questions
286286

287287
1. **Package naming**: `@photostructure/sqlite-async` or `@photostructure/async-sqlite`?
288288
2. **API style**: Mirror better-sqlite3's async API or create our own?
@@ -293,7 +293,7 @@ const user = await stmt.get(userId);
293293

294294
A separate async package is the recommended approach. It follows Node.js's own design philosophy and avoids any risk to existing sync users. The AsyncWorker pattern from node-addon-api is the right building block.
295295

296-
## Next Steps
296+
## Next steps
297297

298298
1. **Decision**: Confirm separate package approach
299299
2. **Repository**: Create new repo or subdirectory

0 commit comments

Comments
 (0)