Provide shaders via artifact repo

[timon-schelling]

Build and upload graphene raster nodes shaders in a separate artifacts repo.
Needed for build environments were we can't use the rust gpu pinned rust nightly toolchain in. I'm most concerned about nixpkgs. Can also be used to significantly speed up builds (desktop on all platforms).

Currently running on every push we can think about doing this only on master, but runtime is <3m so should be fine for now.

Currently using my own repo, should preoby live in a GraphiteInfra org.
https://github.com/timon-schelling/graphite-artifacts
I have set the following for the repo.

vars:
ARTIFACTS_REPO_OWNER = timon-schelling
ARTIFACTS_REPO_NAME = graphite-artifacts
ARTIFACTS_REPO_BRANCH = main

secrets:
ARTIFACTS_REPO_TOKEN = github_pat_*** (access token limited to write on that one repo)

Also includes: desktop enable shader nodes

[Firestar99]

With @TrueDoctor adding the automatic switches between the GPU and CPU variants, end users shouldn't need to use GPU nodes directly. I assume if GPU nodes aren't build, we just default to the CPU variants. So for a typical end-user who doesn't build shaders, everything should still work as expected, at most they get a failing CI later cause the shaders don't compile. Now you'd have to make sure a regular contributor changing a shader node uses the CPU implementation and not the GPU implementation from cached master, without their changes.
(In general, I'm not a fan of caching things externally from your project, due to cache invalidation concerns.)

Or is this only for building desktop on some platforms? Couldn't we have a github job / nix thing to build the shaders and pass the artifacts to the next step, entering whatever build env doesn't support rust-gpu's separate toolchain?

[timon-schelling]

Or is this only for building desktop on some platforms?

Yes and also only a PoC.

Couldn't we have a github job / nix thing to build the shaders and pass the artifacts to the next step, entering whatever build env doesn't support rust-gpu's separate toolchain?

This essentially dose that, but nixpkgs is not part of our ci thus we need to host the artifacts needed for that build statically somewhere (probably a bucket in the end).
The source for the build and shader are build from the exact same commit, this way we ensure that they are never out of sync.

[TrueDoctor]

What's the state on this? I'd prefer if we can start using this for a bit to have enough time to iron out any bugs we might encounter

[timon-schelling]

I used this for all nixpkg PR updates so far, seems to work fine, just needs some cleanup, like renaming some stuff, limiting to master etc.

[timon-schelling]

@TrueDoctor I guess a review from you would be helpfull.