Skip to content
This repository was archived by the owner on Aug 8, 2025. It is now read-only.

Commit ea5d5a7

Browse files
authored
Update README.md
1 parent d91b29e commit ea5d5a7

1 file changed

Lines changed: 1 addition & 134 deletions

File tree

README.md

Lines changed: 1 addition & 134 deletions
Original file line numberDiff line numberDiff line change
@@ -1,137 +1,4 @@
1-
# data-caterer-example
1+
# Data Caterer Examples moved to main repository found [here](https://github.com/data-catering/data-caterer/tree/main/example)
22

33
![Data Catering](misc/logo/logo_landscape_banner.svg)
44

5-
Data Caterer is a metadata driven test data management tool that aids in creating production like data across batch and
6-
event data systems. Run data validations to ensure your systems have ingested it as expected. Use the Java, Scala API,
7-
UI, or YAML files to help with setup or customisation that are all run via Docker.
8-
9-
This repo contains example Java and Scala API usage for Data Caterer.
10-
11-
![Basic data flow of Data Caterer](misc/diagram/high_level_flow-run-config-basic-flow-basic-flow.svg)
12-
13-
## How
14-
15-
Check out any of [the Scala examples](src/main/scala/io/github/datacatering/plan) or
16-
[Java examples](src/main/java/io/github/datacatering/plan). If you are looking for more information, you can follow
17-
[**detailed documentation found here**](https://data.catering/latest/docs/).
18-
19-
Not comfortable with Java or Scala? No worries.
20-
[You can use the UI via these steps](https://data.catering/latest/get-started/quick-start/).
21-
22-
Want some YAML instead? Also, no worries. Check the example [plan](docker/data/custom/plan) and
23-
[task](docker/data/custom/task) YAML files here.
24-
25-
### Java
26-
27-
1. Create new Java class similar
28-
to [DocumentationJavaPlanRun.java](src/main/java/io/github/datacatering/plan/DocumentationJavaPlanRun.java)
29-
1. Needs to extend `io.github.datacatering.datacaterer.javaapi.api.PlanRun`
30-
31-
### Scala
32-
33-
1. Create new Scala class similar
34-
to [DocumentationPlanRun.scala](src/main/scala/io/github/datacatering/plan/DocumentationPlanRun.scala)
35-
1. Needs to extend `io.github.datacatering.datacaterer.api.PlanRun`
36-
37-
### YAML
38-
39-
1. Copy existing plan file (such as [foreign-key.yaml](docker/data/custom/plan/foreign-key.yaml)) in directory
40-
[docker/data/custom/plan](docker/data/custom/plan)
41-
2. Copy existing task file (such as [json-account-task.yaml](docker/data/custom/task/file/json/json-account-task.yaml))
42-
in directory [docker/data/custom/task](docker/data/custom/task)
43-
1. If you want to run data validations, copy the file [simple-validation.yaml](docker/data/custom/validation/simple-validation.yaml)
44-
and add validation to plan via:
45-
```yaml
46-
validations:
47-
- "<name of validation (i.e. account_checks)>"
48-
```
49-
3. [Use JSON schema to help creating metadata for plan, tasks or validations](schema/data-caterer-latest.json).
50-
You can import this schema into your IDE for validation of your YAML files. Links below show how you can import the schema:
51-
- [IntelliJ](https://www.jetbrains.com/help/idea/json.html#ws_json_schema_add_custom)
52-
- [VS Code](https://code.visualstudio.com/docs/languages/json#_json-schemas-and-settings)
53-
54-
55-
## Run
56-
57-
Requires:
58-
59-
- Docker
60-
61-
```shell
62-
./run.sh
63-
#check results under docker/sample/report/index.html folder
64-
```
65-
66-
## Docker
67-
68-
Create your own Docker image via:
69-
70-
```shell
71-
./gradlew clean build
72-
docker build -t <my_image_name>:<my_image_tag> .
73-
docker run -e PLAN_CLASS=io.github.datacatering.plan.DocumentationPlanRun -v ${PWD}/docs/run:/opt/app/data <my_image_name>:<my_image_tag>
74-
#check results under docs/run folder
75-
```
76-
77-
## Docker Compose
78-
79-
Run with own class from either Java or Scala API:
80-
81-
```shell
82-
./gradlew clean build
83-
cd docker
84-
PLAN_CLASS=io.github.datacatering.plan.DocumentationPlanRun DATA_SOURCE=postgres docker-compose up -d datacaterer
85-
```
86-
87-
[Details from docs](https://data.catering/latest/get-started/quick-start/).
88-
Docker compose sample found under `docker` folder.
89-
90-
```shell
91-
cd docker
92-
docker-compose up -d datacaterer
93-
```
94-
95-
Check result under [here](docker/data/custom).
96-
97-
Change to another data source via:
98-
99-
- postgres
100-
- mysql
101-
- cassandra
102-
- solace
103-
- kafka
104-
- http
105-
106-
```shell
107-
DATA_SOURCE=cassandra docker-compose up -d datacaterer
108-
```
109-
110-
### Run with YAML files
111-
112-
Example YAML files can be found here:
113-
- [Plan](docker/data/custom/plan): Define tasks, data sources, foreign keys, etc. to run
114-
- [Task](docker/data/custom/task): Define data generation details such as schema and number of records
115-
- [Validation](docker/data/custom/validation): Define data validation details to run on data sources
116-
117-
If you want to use a [different YAML plan](docker/data/custom/plan) for the data source, you can run:
118-
119-
```shell
120-
PLAN=plan/postgres-multiple-tables DATA_SOURCE=postgres docker-compose up -d datacaterer
121-
```
122-
123-
## Helm
124-
125-
```shell
126-
helm install data-caterer ./data-caterer-example/helm/data-caterer
127-
```
128-
129-
## Benchmarks
130-
131-
Base benchmark tests can be run via:
132-
133-
```shell
134-
bash benchmark/run_benchmark.sh
135-
```
136-
137-
Results can be found under [benchmark/results](benchmark/results).

0 commit comments

Comments
 (0)