Documentation · Report Bug · Request Feature

About the Project

XFlow is a lightweight modular machine-learning framework.

Originally created for physics research, it's now evolving toward generic scientific applications ML workflows: Data → Processing → Modeling

---

Core Data Processing Pipeline (Computational Map example)

flow is a step-based computation map for data processing.

Inputs (possibly different data types) move through discrete steps. At each step, a sample either passes through unchanged (identity) or is transformed by a node. Nodes can be multi-input and multi-output, so the map can split and merge data streams. Optional meta nodes (debug, checks, routing) can log, validate, stop, or redirect (no loops, deterministic) the pipeline without changing the core step structure.

%%{init: {"themeVariables": {"fontSize": "15px"}, "flowchart": {"htmlLabels": true}}}%%
flowchart TD
  classDef src fill:#0b1220,stroke:#334155,stroke-width:1px,color:#e2e8f0;
  classDef op fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#e2e8f0;
  classDef io fill:#111827,stroke:#94a3b8,stroke-width:1px,color:#e5e7eb;
  classDef gate fill:#1f2937,stroke:#f59e0b,stroke-width:2px,color:#fde68a;
  classDef stop fill:#2a0f12,stroke:#fb7185,stroke-width:2px,color:#fecdd3;

  subgraph Inputs["Inputs"]
    DIR["dir: str<br/>/data/run_042"]:::src
    CFG["config: str<br/>YAML or JSON"]:::src
    A1["sensor A:<br/>array&lt;float&gt;"]:::src
    A2["sensor B:<br/>int"]:::src
  end

  READ["<b>ReadImages</b><br/>(dir -> images)"]:::op
  PARSE["<b>ParseConfig</b><br/>(str -> dict)"]:::op

  DIR --> READ
  CFG --> PARSE

  IMGS["images:<br/>tensor[H,W,C,N]"]:::io
  CONF["config:<br/>dict"]:::io

  READ --> IMGS
  PARSE --> CONF

  LOG["<b>LogConfig</b><br/>(print or save)"]:::op
  CONF --> LOG

  JOIN["<b>AlignAndEnrich</b><br/>(images -> 2 outputs)"]:::op
  IMGS --> JOIN

  subgraph JOIN_OUT[" "]
    direction LR
    ALN["aligned_images:<br/>tensor[...]"]:::io
    REP["report:<br/>md or json"]:::io
  end
  style JOIN_OUT fill:transparent,stroke:transparent

  JOIN --> ALN
  JOIN --> REP

  FUSE["<b>FuseSensors</b><br/>(2 signals -> 1 feature vector)"]:::op
  A1 --> FUSE
  A2 --> FUSE

  FEAT["features:<br/>vector&lt;float&gt;"]:::io
  FUSE --> FEAT

  GATE{"<b>QualityGate</b><br/>(meets requirements?)"}:::gate
  ALN --> GATE

  FIX["<b>Remediate</b><br/>(cleanup, re-run, notify)"]:::op
  STOP["STOP<br/>(fail fast)"]:::stop

  GATE -->|fail| FIX
  FIX --> STOP

  subgraph Outputs["Outputs"]
    OUT["artifacts:<br/>aligned_images + features + report"]:::io
  end

  ALN --> OUT
  FEAT --> OUT
  REP --> OUT

  GATE -->|pass| OUT

Loading

Getting Started

Installation

Install from PyPI:

pip install xflow-py

Clone the repository and install in editable mode:

git clone https://github.com/Andrew-XQY/XFlow.git
cd XFlow
pip install -e .

---

Built With

• Python 3.12
• TensorFlow 2.x
• Keras 3.x
• PyTorch 2.5.x

---

License

This project is licensed under the MIT License. See the LICENSE file for details.