Skip to content

Commit bfc5d38

Browse files
lukepistrollukepistrol
committed
add docs
1 parent bb97af5 commit bfc5d38

4 files changed

Lines changed: 371 additions & 0 deletions

File tree

‎Sources/CodeEditLanguages/Documentation.docc/Add-Languages.md‎

Lines changed: 214 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,214 @@
1+
# Add Languages
2+
3+
This article is a writedown on how to add support for more languages to ``CodeLanguage``.
4+
5+
## Overview
6+
7+
First of all have a look at the corresponding [GitHub Issue](https://github.com/CodeEditApp/CodeEditTextView/issues/15) to see which languages still need implementation.
8+
9+
## Add SPM support
10+
11+
If you find one you want to add, fork and clone the linked repo and create a new branch `feature/spm`.
12+
13+
> In the following code samples replace `{LANG}` or `{lang}` with the language you add (e.g.: `Swift` or `CPP` and `swift` or `cpp` respectively)
14+
15+
### .gitignore
16+
17+
Edit the `.gitignore` file to exclude the `.build/` directory from git.
18+
19+
### Package.swift
20+
21+
Create a new file `Package.swift` in the `root` directory of the repository and add the following configuration.
22+
23+
> Make sure to remove the comment in 'sources'.
24+
25+
```swift
26+
// swift-tools-version:5.3
27+
import PackageDescription
28+
29+
let package = Package(
30+
name: "TreeSitter{LANG}",
31+
platforms: [.macOS(.v10_13), .iOS(.v11)],
32+
products: [
33+
.library(name: "TreeSitter{LANG}", targets: ["TreeSitter{LANG}"]),
34+
],
35+
dependencies: [],
36+
targets: [
37+
.target(name: "TreeSitter{LANG}",
38+
path: ".",
39+
exclude: [
40+
"binding.gyp",
41+
"bindings",
42+
"Cargo.toml",
43+
"corpus",
44+
"examples",
45+
"grammar.js",
46+
"LICENSE",
47+
"Makefile",
48+
"package.json",
49+
"README.md",
50+
"src/grammar.json",
51+
"src/node-types.json",
52+
// any additional files to exclude
53+
],
54+
sources: [
55+
"src/parser.c",
56+
"src/scanner.cc", // this might be `scanner.c` or not present at all
57+
],
58+
resources: [
59+
.copy("queries")
60+
],
61+
publicHeadersPath: "bindings/swift",
62+
cSettings: [.headerSearchPath("src")])
63+
]
64+
)
65+
```
66+
67+
### Swift Bindings
68+
69+
Now you need to create the Swift bindings which are a `header` file exposing the `tree_sitter_{lang}()` function.
70+
71+
First of all create the following directories inside the `bindings/` directory:
72+
73+
`./bindings/swift/TreeSitter{LANG}/`
74+
75+
Inside that folder create a new header file called `{lang}.h`.
76+
77+
```cpp
78+
#ifndef TREE_SITTER_{LANG}_H_
79+
#define TREE_SITTER_{LANG}_H_
80+
81+
typedef struct TSLanguage TSLanguage;
82+
83+
#ifdef __cplusplus
84+
extern "C" {
85+
#endif
86+
87+
extern TSLanguage *tree_sitter_{lang}();
88+
89+
#ifdef __cplusplus
90+
}
91+
#endif
92+
93+
#endif // TREE_SITTER_{LANG}_H_
94+
```
95+
96+
## Add it to CodeEditLanguages
97+
98+
In order to add a language to ``CodeEditLanguages`` you need to open the `.xcodeproj` file located inside `CodeLanguage-Container`.
99+
100+
Add the `tree-sitter` package you created earlier as a dependency like you would in a regular Xcode project.
101+
102+
Then make sure the framework target loads the package module.
103+
104+
Now open the `CodeLanguages_Container.h` header file and add:
105+
106+
```cpp
107+
extern TSLanguage *tree_sitter_{lang}();
108+
```
109+
110+
> Please keep an alphabetical order
111+
112+
Now create the `xcframework` by running the `build_framework.sh` script from the Package's root directory.
113+
114+
After that also run the `copy_resources.sh` script to copy the `*.scm` files into the Package's Resource folder.
115+
116+
You are now done in the Xcode Project and may close it now. Open the Package and continue.
117+
118+
## Add it to CodeLanguage
119+
120+
Now move over to the `Sources/CodeEditLanguages` folder where 3 files need to be updated.
121+
122+
### TreeSitterLanguage.swift
123+
124+
Add a case for your language to ``TreeSitterLanguage``:
125+
126+
```swift
127+
public enum TreeSitterLanguage: String {
128+
// other cases
129+
case {lang}
130+
}
131+
```
132+
133+
### CodeLanguage.swift
134+
135+
Find the `tsLanguage` computed property and add a `case` to it:
136+
137+
```swift
138+
private var tsLanguage: UnsafeMutablePointer<TSLanguage>? {
139+
switch id {
140+
// other cases
141+
case .{lang}:
142+
return tree_sitter_{lang}()
143+
}
144+
// other cases
145+
}
146+
```
147+
148+
On the bottom of the file add a new `static` constant:
149+
150+
```swift
151+
static let {lang}: CodeLanguage = .init(id: .{lang}, tsName: {lang}, extensions: [...])
152+
```
153+
154+
> in 'extensions' add the proper file extensions your language uses.
155+
156+
Now find the static constant ``CodeLanguage/allLanguages`` and add your language to it:
157+
158+
```swift
159+
static let allLanguages: [CodeLanguage] = [
160+
// other languages
161+
.{lang},
162+
// other languages
163+
]
164+
```
165+
166+
### TreeSitterModel.swift
167+
168+
Create a new query like so:
169+
170+
```swift
171+
public private(set) lazy var {lang}Query: Query? = {
172+
return queryFor(.{lang})
173+
}()
174+
```
175+
176+
Find the ``TreeSitterModel/query(for:)`` method and add a `case` for your language:
177+
178+
```swift
179+
public func query(for language: TreeSitterLanguage) -> Query? {
180+
switch language {
181+
// other cases
182+
case .{lang}:
183+
return {lang}Query
184+
// other cases
185+
}
186+
}
187+
```
188+
189+
## Test it!
190+
191+
In order to test whether is working or not, add ``CodeEditTextView`` as a local dependency to `CodeEdit`.
192+
193+
In order to do that close ``CodeEditTextView`` in Xcode and open `CodeEdit`. Then inside `CodeEditModules` replace the `CodeEditTextView` dependency with:
194+
195+
```swift
196+
.package(name: "CodeEditTextView", path: "/PATH/TO/CodeEditTextView")
197+
```
198+
199+
After that, you may need to reset packages caches but then it should compile and run.
200+
201+
When everything is working correctly push your `tree-sitter-{lang}` changes to `origin` and also create a Pull Request to the official repository.
202+
203+
> Take [this PR description](https://github.com/tree-sitter/tree-sitter-javascript/pull/223) as a template and cross-reference it with your Pull Request.
204+
205+
Now you can remove the local dependencies and replace it with the actual package URLs and submit a Pull Request for ``CodeEditTextView``.
206+
207+
## Documentation
208+
209+
Please make sure to add the newly created properties to the documentation `*.md` files.
210+
211+
## Unit Tests
212+
213+
Also make sure to add test cases for your new language in `Tests/CodeEditLanguagesTests`.
214+

‎Sources/CodeEditLanguages/Documentation.docc/CodeLanguage.md‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# ``CodeEditLanguages/CodeLanguage``
2+
3+
## Overview
4+
5+
If the language required is known they can be accessed by using the type properties below.
6+
7+
```swift
8+
let language = CodeLanguage.swift
9+
```
10+
11+
If the language needs to be discovered by the file extension this can be done by calling ``detectLanguageFrom(url:)``.
12+
13+
```swift
14+
let fileURL = URL(fileURLWithPath: "/path/to/file.swift")
15+
let language = CodeLanguage.detectLanguageFrom(url: fileURL)
16+
```
17+
18+
> In case the language is not supported yet, the resulting ``CodeLanguage`` will be ``default`` (plain text).
19+
20+
### Supported Languages
21+
22+
- Bash
23+
- C
24+
- C++
25+
- C#
26+
- CSS
27+
- Elixir
28+
- Go
29+
- Go Mod
30+
- Haskell
31+
- HTML
32+
- Java
33+
- JavaScript
34+
- JSON
35+
- JSX
36+
- PHP
37+
- Python
38+
- Ruby
39+
- Rust
40+
- Swift
41+
- YAML
42+
- Zig
43+
44+
## Topics
45+
46+
### Guides
47+
48+
- <doc:Add-Languages>
49+
50+
### Instance Properties
51+
52+
- ``id``
53+
- ``tsName``
54+
- ``extensions``
55+
- ``parentQueryURL``
56+
- ``tsName``
57+
- ``queryURL``
58+
- ``language``
59+
- ``additionalHighlights``
60+
61+
### Type Properties
62+
63+
- ``allLanguages``
64+
- ``default``
65+
- ``bash``
66+
- ``c``
67+
- ``cpp``
68+
- ``cSharp``
69+
- ``css``
70+
- ``elixir``
71+
- ``go``
72+
- ``goMod``
73+
- ``haskell``
74+
- ``html``
75+
- ``java``
76+
- ``javascript``
77+
- ``json``
78+
- ``jsx``
79+
- ``php``
80+
- ``python``
81+
- ``ruby``
82+
- ``rust``
83+
- ``swift``
84+
- ``yaml``
85+
- ``zig``
86+
87+
### Type Methods
88+
89+
- ``detectLanguageFrom(url:)``

‎Sources/CodeEditLanguages/Documentation.docc/Documentation.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# ``CodeEditLanguages``
2+
3+
A collection of `tree-sitter` languages for syntax highlighting.
4+
5+
## Overview
6+
7+
This package includes a binary framework `CodeLanguagesContainer.xcframework` which bundles all languages as a binary.
8+
9+
The languages are then served as a ``CodeLanguage``.
10+
11+
## Topics
12+
13+

‎Sources/CodeEditLanguages/Documentation.docc/TreeSitterModel.md‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# ``CodeEditLanguages/TreeSitterModel``
2+
3+
## Overview
4+
5+
Since fetching queries *can* be expensive the queries are fetched lazily and kept in memory for the entire session.
6+
7+
> Be aware that running the application in `Debug` configuration will lead to worse performance. Make sure to run it in `Release` configuration.
8+
9+
## Usage
10+
11+
```swift
12+
let language = CodeLanguage.swift
13+
14+
// this call might be expensive
15+
let query = TreeSitterModel.shared.query(for: language.id)
16+
```
17+
Or access it directly
18+
```swift
19+
// this call might be expensive
20+
let query = TreeSitterModel.shared.swiftQuery
21+
```
22+
23+
## Topics
24+
25+
### Type Properties
26+
27+
- ``shared``
28+
29+
### Instance Methods
30+
31+
- ``query(for:)``
32+
33+
### Instance Properties
34+
35+
- ``bashQuery``
36+
- ``cQuery``
37+
- ``cppQuery``
38+
- ``cSharpQuery``
39+
- ``cssQuery``
40+
- ``elixirQuery``
41+
- ``goQuery``
42+
- ``goModQuery``
43+
- ``haskellQuery``
44+
- ``htmlQuery``
45+
- ``javaQuery``
46+
- ``javascriptQuery``
47+
- ``jsonQuery``
48+
- ``jsxQuery``
49+
- ``phpQuery``
50+
- ``pythonQuery``
51+
- ``rubyQuery``
52+
- ``rustQuery``
53+
- ``swiftQuery``
54+
- ``yamlQuery``
55+
- ``zigQuery``

0 commit comments

Comments
 (0)