Skip to content

Commit 9e34788

Browse files
authored
Add a dashboard to visualise the Job queue and explore logs (#761)
1 parent 84b700f commit 9e34788

23 files changed

Lines changed: 3597 additions & 135 deletions

.github/workflows/dashboard.yml

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
name: Deploy Dashboard
2+
3+
on:
4+
push:
5+
branches: [master]
6+
paths: ['dashboard/**']
7+
pull_request:
8+
types: [opened, reopened, synchronize, closed]
9+
paths: ['dashboard/**']
10+
11+
permissions:
12+
contents: write
13+
pull-requests: write
14+
15+
concurrency: pages-${{ github.ref }}
16+
17+
jobs:
18+
deploy:
19+
runs-on: ubuntu-latest
20+
steps:
21+
- uses: actions/checkout@v4
22+
23+
- name: Setup Node
24+
if: github.event.action != 'closed'
25+
uses: actions/setup-node@v4
26+
with:
27+
node-version: 22
28+
29+
- name: Install tools
30+
if: github.event.action != 'closed'
31+
run: npm install --no-save purescript@0.15.16 spago@1.0.4 esbuild
32+
33+
- name: Build dashboard
34+
if: github.event.action != 'closed'
35+
run: npm run dashboard:build
36+
37+
- name: Verify bundle
38+
if: github.event.action != 'closed'
39+
run: test -f dashboard/app.js
40+
41+
- name: Prepare deploy directory
42+
if: github.event.action != 'closed'
43+
run: |
44+
mkdir -p _site
45+
cp dashboard/index.html _site/
46+
cp dashboard/app.js _site/
47+
cp -r dashboard/static _site/
48+
49+
- name: Deploy to Pages
50+
if: github.ref == 'refs/heads/master'
51+
uses: JamesIves/github-pages-deploy-action@v4
52+
with:
53+
folder: _site
54+
clean-exclude: pr-preview
55+
56+
# On 'closed' events this removes the preview directory from gh-pages;
57+
# on all other PR events it deploys the build to pr-preview/pr-<number>/.
58+
- name: Deploy PR preview
59+
if: github.event_name == 'pull_request'
60+
uses: rossjrw/pr-preview-action@v1
61+
with:
62+
source-dir: ./_site/

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
/scratch
55
/.vscode
66
/scripts/analysis
7+
/generated-docs
78

89
result*
910

@@ -21,5 +22,8 @@ TODO.md
2122
.spec-results
2223
/generated-docs
2324

25+
# Generated bundle
26+
dashboard/app.js
27+
2428
# Keep it secret, keep it safe.
2529
.env

CONTRIBUTING.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,10 @@ There are two additional PureScript directories focused on testing only:
3636
- `app-e2e` contains tests that exercise the server API, which requires that a server and associated wiremock services are running
3737
- `test-utils` contains utility code intended only for tests
3838

39+
There is one additional directory for the web dashboard:
40+
41+
- `dashboard` contains the static HTML/CSS/JS dashboard for monitoring registry jobs. It is deployed independently to GitHub Pages and requires no build step.
42+
3943
There are three more directories containing code for the registry.
4044

4145
- `db` contains schemas and migrations for the sqlite3 database used by the server.
@@ -253,6 +257,36 @@ services.wiremock-github-api = {
253257

254258
It is also possible to include specific files that should be returned to requests via the `files` key. Here's another short example of setting up an S3 mock, in which we copy files from the fixtures into the wiremock service's working directory given a particular file name, and then write request/response mappings that respond to requests by reading the file at path given by `bodyFileName`.
255259

260+
## Dashboard Development
261+
262+
The `dashboard/` directory contains a Halogen (PureScript) application for monitoring registry jobs. It is deployed to GitHub Pages and calls the registry API cross-origin.
263+
264+
### Building
265+
266+
To produce a browser JS bundle:
267+
268+
```sh
269+
npm run dashboard:build
270+
```
271+
272+
This outputs `dashboard/app.js`, which `dashboard/index.html` loads via `<script src="./app.js">`.
273+
274+
### Local Development
275+
276+
For iterative development with live rebuild and a local dev server:
277+
278+
```sh
279+
npm run dashboard:dev
280+
```
281+
282+
This starts esbuild in watch mode: it serves the dashboard at `http://localhost:8000` and automatically rebuilds `app.js` when PureScript source files change (after running `spago build`).
283+
284+
The API base URL is hardcoded in `dashboard/src/Dashboard/API.purs` (`defaultConfig`) to the production registry URL. The server includes CORS headers with `Access-Control-Allow-Origin: *`, so cross-origin requests from the dashboard work without any additional configuration.
285+
286+
### Deployment
287+
288+
The dashboard is deployed automatically by the `.github/workflows/dashboard.yml` workflow. Pushing changes to `dashboard/**` on `master` triggers a GitHub Pages deployment.
289+
256290
## Deployment
257291

258292
The registry is continuously deployed. The [deploy.yml](./.github/workflows/deploy.yml) file defines a GitHub Actions workflow to auto-deploy the server when a new commit is pushed to `master` and test workflows have passed.

app/spago.yaml

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ package:
2121
- dotenv
2222
- effect
2323
- either
24+
- enums
2425
- exceptions
2526
- exists
2627
- fetch
@@ -33,7 +34,6 @@ package:
3334
- httpurple
3435
- identity
3536
- integers
36-
- js-date
3737
- js-fetch
3838
- js-promise-aff
3939
- js-uri
@@ -51,7 +51,6 @@ package:
5151
- nullable
5252
- numbers
5353
- ordered-collections
54-
- orders
5554
- parallel
5655
- parsing
5756
- partial
@@ -64,14 +63,12 @@ package:
6463
- run
6564
- safe-coerce
6665
- strings
67-
- these
6866
- transformers
6967
- tuples
7068
- typelevel-prelude
7169
- unicode
7270
- unsafe-coerce
7371
- uuidv4
74-
- variant
7572
test:
7673
main: Test.Registry.Main
7774
dependencies:

dashboard/index.html

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
<!DOCTYPE html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="utf-8">
5+
<meta name="viewport" content="width=device-width, initial-scale=1">
6+
<title>PureScript Registry Dashboard</title>
7+
<link href="https://fonts.googleapis.com/css?family=Roboto+Mono|Roboto:300,400,400i,700,700i" type="text/css" rel="stylesheet">
8+
<link rel="stylesheet" href="static/style.css">
9+
<link rel="icon" href="data:,">
10+
</head>
11+
<body>
12+
<noscript>This dashboard requires JavaScript to run.</noscript>
13+
<script src="./app.js"></script>
14+
</body>
15+
</html>

dashboard/spago.yaml

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
package:
2+
name: registry-dashboard
3+
publish:
4+
license: BSD-3-Clause
5+
version: 0.0.1
6+
dependencies:
7+
- aff
8+
- arrays
9+
- codec-json
10+
- const
11+
- control
12+
- datetime
13+
- effect
14+
- either
15+
- exceptions
16+
- fetch
17+
- foldable-traversable
18+
- formatters
19+
- halogen
20+
- halogen-subscriptions
21+
- integers
22+
- json
23+
- lists
24+
- maybe
25+
- newtype
26+
- now
27+
- parallel
28+
- prelude
29+
- registry-lib
30+
- routing-duplex
31+
- strings
32+
- tailrec
33+
- web-events
34+
- web-html
35+
- web-uievents

dashboard/src/Dashboard/API.purs

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
-- | HTTP client for making requests to the registry server from the dashboard.
2+
-- | Provides typed helpers for fetching job data from the Registry API.
3+
module Dashboard.API
4+
( ApiConfig
5+
, ApiError(..)
6+
, defaultConfig
7+
, fetchJobs
8+
, fetchJob
9+
, printApiError
10+
) where
11+
12+
import Prelude
13+
14+
import Codec.JSON.DecodeError as CJ.DecodeError
15+
import Control.Alt ((<|>))
16+
import Control.Parallel (parallel, sequential)
17+
import Data.Codec.JSON as CJ
18+
import Data.DateTime (DateTime)
19+
import Data.Either (Either(..))
20+
import Data.Maybe (Maybe(..))
21+
import Data.String as String
22+
import Effect.Aff (Aff, Milliseconds(..))
23+
import Effect.Aff as Aff
24+
import Effect.Exception as Exception
25+
import Fetch (Method(..))
26+
import Fetch as Fetch
27+
import JSON as JSON
28+
import Registry.API.V1 (Job, JobId, LogLevel, Route(..), SortOrder)
29+
import Registry.API.V1 as V1
30+
import Routing.Duplex as Routing
31+
32+
-- | Configuration for the API client.
33+
type ApiConfig =
34+
{ baseUrl :: String
35+
}
36+
37+
-- | Default API configuration pointing to the production registry server.
38+
defaultConfig :: ApiConfig
39+
defaultConfig =
40+
{ baseUrl: "https://registry.purescript.org"
41+
}
42+
43+
-- | Errors that can occur when making API requests.
44+
data ApiError
45+
= HttpError { status :: Int, body :: String }
46+
| ParseError { message :: String, raw :: String }
47+
48+
-- | Render an ApiError as a human-readable string.
49+
printApiError :: ApiError -> String
50+
printApiError = case _ of
51+
HttpError { status, body } ->
52+
"HTTP " <> show status <> ": " <> body
53+
ParseError { message, raw } ->
54+
"Parse error: " <> message <> "\nResponse: " <> String.take 500 raw
55+
56+
-- | Print a V1 Route to its URL path string.
57+
printRoute :: Route -> String
58+
printRoute = Routing.print V1.routes
59+
60+
-- | Parse a JSON string using a codec, returning Either ApiError.
61+
parseJson :: forall a. CJ.Codec a -> String -> Either ApiError a
62+
parseJson codec str = case JSON.parse str of
63+
Left jsonErr ->
64+
Left $ ParseError { message: "JSON: " <> jsonErr, raw: str }
65+
Right json -> case CJ.decode codec json of
66+
Left decodeErr ->
67+
Left $ ParseError { message: CJ.DecodeError.print decodeErr, raw: str }
68+
Right a ->
69+
Right a
70+
71+
-- | Request timeout in milliseconds.
72+
requestTimeout :: Milliseconds
73+
requestTimeout = Milliseconds 10000.0
74+
75+
-- | Run an Aff action with a timeout. Returns Nothing if the action does not
76+
-- | complete within the given duration, or Just the result if it does.
77+
timeout :: forall a. Milliseconds -> Aff a -> Aff (Maybe a)
78+
timeout ms action = sequential do
79+
parallel (Just <$> action) <|> parallel (Nothing <$ Aff.delay ms)
80+
81+
-- | Make a GET request to the given URL path and decode the response body.
82+
get :: forall a. CJ.Codec a -> ApiConfig -> String -> Aff (Either ApiError a)
83+
get codec config path = do
84+
result <- Aff.try $ timeout requestTimeout do
85+
response <- Fetch.fetch (config.baseUrl <> path) { method: GET }
86+
body <- response.text
87+
pure { status: response.status, body }
88+
case result of
89+
Left err ->
90+
pure $ Left $ HttpError { status: 0, body: Exception.message err }
91+
Right Nothing ->
92+
pure $ Left $ HttpError { status: 0, body: "Request timed out" }
93+
Right (Just { status, body })
94+
| status >= 200 && status < 300 ->
95+
pure $ parseJson codec body
96+
| otherwise ->
97+
pure $ Left $ HttpError { status, body }
98+
99+
-- | Fetch the list of jobs from the registry server.
100+
-- |
101+
-- | Parameters:
102+
-- | - `since`: Only return jobs created after this time
103+
-- | - `until`: Only return jobs created before this time
104+
-- | - `order`: Sort order for results (ASC or DESC)
105+
-- | - `includeCompleted`: When true, include finished jobs in the results
106+
fetchJobs
107+
:: ApiConfig
108+
-> { since :: Maybe DateTime, until :: Maybe DateTime, order :: Maybe SortOrder, includeCompleted :: Maybe Boolean }
109+
-> Aff (Either ApiError (Array Job))
110+
fetchJobs config params = do
111+
let route = Jobs { since: params.since, until: params.until, order: params.order, include_completed: params.includeCompleted }
112+
get (CJ.array V1.jobCodec) config (printRoute route)
113+
114+
-- | Fetch a single job by its ID.
115+
-- |
116+
-- | Parameters:
117+
-- | - `level`: Minimum log level to include in the response
118+
-- | - `since`: Only return log lines after this time
119+
-- | - `until`: Only return log lines before this time
120+
-- | - `order`: Sort order for log lines (ASC or DESC)
121+
fetchJob
122+
:: ApiConfig
123+
-> JobId
124+
-> { level :: Maybe LogLevel, since :: Maybe DateTime, until :: Maybe DateTime, order :: Maybe SortOrder }
125+
-> Aff (Either ApiError Job)
126+
fetchJob config jobId params = do
127+
let route = Job jobId { level: params.level, since: params.since, until: params.until, order: params.order }
128+
get V1.jobCodec config (printRoute route)

0 commit comments

Comments
 (0)