Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ FROM golang:1.23-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY main.go main.go
COPY *.go ./
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o /cozy-telemetry-server

FROM scratch
Expand Down
50 changes: 50 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,53 @@ Client source code:

Read more about how telemetry is collected from docs:
- https://cozystack.io/docs/telemetry/

## Public Statistics API

The server exposes a `GET /api/overview?year=YYYY&month=MM` endpoint that returns aggregated usage statistics in JSON format. This data is used by the [Cozystack website](https://cozystack.io/) to display public telemetry on the [Telemetry page](https://cozystack.io/oss-health/telemetry/).

### How it works

1. The endpoint requires `year` and `month` query parameters (e.g. `/api/overview?year=2026&month=03`). Requests without them return 400.
2. On first request for a given month, the server queries VictoriaMetrics at the end of that month, writes a snapshot to `--snapshot-dir`, and caches it in memory. Subsequent requests for the same month are served from cache.
3. Concurrent requests for the same uncached month are coalesced into a single VictoriaMetrics query (per-month singleflight).
4. The app list is fetched from [cozystack/cozystack packages/apps](https://github.com/cozystack/cozystack/tree/main/packages/apps) so newly added applications are picked up automatically; a built-in fallback list is used if GitHub is unreachable.
5. The response aggregates snapshots into three time periods relative to the requested month: **that month**, **last quarter** (3 months), and **last 12 months**.

The snapshot directory is backed by an `emptyDir` volume — cache is per-pod and is rebuilt on restart from VictoriaMetrics on demand.

### Response format

```json
{
"generated_at": "2026-04-01T07:01:00Z",
"periods": {
"month": {
"label": "March 2026",
"start": "2026-03-01",
"end": "2026-03-31",
"clusters": 42,
"total_nodes": 210,
"avg_nodes_per_cluster": 5.0,
"total_tenants": 84,
"avg_tenants_per_cluster": 2.0,
"apps": {
"postgres": 120,
"redis": 85,
"kubernetes": 30
}
},
"quarter": { "..." : "averaged over 3 months" },
"year": { "..." : "averaged over 12 months" }
}
}
```

### Configuration flags

| Flag | Default | Description |
|------|---------|-------------|
| `--forward-url` | `http://vminsert-cozy-telemetry:8480/insert/0/prometheus/api/v1/import/prometheus` | URL to forward ingested metrics to |
| `--listen-addr` | `:8081` | Address to listen on |
| `--vmselect-url` | `http://vmselect-cozy-telemetry:8481` | VictoriaMetrics vmselect base URL for queries |
| `--snapshot-dir` | `/data/snapshots` | Directory to store monthly snapshot JSON files |
8 changes: 8 additions & 0 deletions charts/cozy-telemetry/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,15 @@ spec:
args:
- "--forward-url={{ .Values.config.forwardURL }}"
- "--listen-addr={{ .Values.config.listenAddr }}"
- "--vmselect-url={{ .Values.config.vmSelectURL }}"
- "--snapshot-dir={{ .Values.config.snapshotDir }}"
ports:
- containerPort: {{ .Values.service.port }}
resources:
{{- toYaml .Values.resources | nindent 10 }}
volumeMounts:
- name: snapshots
mountPath: {{ .Values.config.snapshotDir }}
readinessProbe:
tcpSocket:
port: 8081
Expand All @@ -35,3 +40,6 @@ spec:
port: 8081
initialDelaySeconds: 15
periodSeconds: 10
volumes:
- name: snapshots
emptyDir: {}
4 changes: 3 additions & 1 deletion charts/cozy-telemetry/values.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
replicaCount: 2
replicaCount: 1

image:
repository: "ghcr.io/cozystack/cozystack-telemetry-server"
Expand All @@ -13,6 +13,8 @@ service:
config:
forwardURL: "http://vminsert-cozy-telemetry:8480/insert/0/prometheus/api/v1/import/prometheus"
listenAddr: ":8081"
vmSelectURL: "http://vmselect-cozy-telemetry:8481"
snapshotDir: "/data/snapshots"

resources:
requests:
Expand Down
14 changes: 14 additions & 0 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -149,20 +149,34 @@ func main() {
// Define flags
forwardURL := flag.String("forward-url", "http://vminsert-cozy-telemetry:8480/insert/0/prometheus/api/v1/import/prometheus", "URL to forward the metrics to")
listenAddr := flag.String("listen-addr", ":8081", "Address to listen on for incoming metrics")
vmSelectURL := flag.String("vmselect-url", "http://vmselect-cozy-telemetry:8481", "VictoriaMetrics vmselect base URL for queries")
snapshotDir := flag.String("snapshot-dir", "/data/snapshots", "Directory to store monthly snapshots")
flag.Parse()

// Initialize overview manager
overview := NewOverviewManager(*vmSelectURL, *snapshotDir)

server := &http.Server{
Addr: *listenAddr,
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
}

// The overview handler may query VictoriaMetrics on cache miss (up to 30s),
// so it gets its own 55s timeout instead of inheriting the global 10s WriteTimeout.
http.Handle("/api/overview", http.TimeoutHandler(
http.HandlerFunc(overview.HandleOverview),
55*time.Second,
`{"error":"request timeout"}`,
))
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
handleTelemetry(w, r, *forwardURL)
})

log.Printf("Starting server on %s", *listenAddr)
log.Printf("Forwarding metrics to %s", *forwardURL)
log.Printf("VictoriaMetrics select URL: %s", *vmSelectURL)
log.Printf("Snapshot directory: %s", *snapshotDir)

if err := server.ListenAndServe(); err != nil {
log.Fatalf("Server error: %v", err)
Expand Down
Loading