Flowmetal

A shining mercurial metal, laden with sensors and almost infinitely reconfigurable. The stuff of which robots and servitors are made.

Flowmetal is the glue you always wanted for connecting between services and developing “workflow” automation across your compute grid.

Key Features

  • Simple, Familiar Syntax Write flows in Starlark—a Python subset that’s easy to learn, tooling-friendly, and production-ready.

  • Always be reusing Flows are just functions which can always be invoked and reused without ceremony. No need to refactor existing components to allow reuse; reuse is the default.

  • Scripting ease Just flowmetal run ./script.flow, no need to wait for a scheduled deployment or slow builds.

  • The grid to hand Want to go parallel? Just run a function using a threading.Thread-like API and go as wide as your grid quota allows.

  • Be restart oblivious Flows’ state is managed for you. Engine state management renders flows oblivious to worker failures ensures the safety of restarting a flow from the database and guarantees execution.

  • Scalable & Adaptable Failure oblivious flows allow the grid to scale up, down and undergo maintenance effortlessly without interruption.

  • Reliability by construction The restricted Starlark environment ensures that flows cannot accidentally become restart unsafe.

  • Multitenancy by construction The restrictions of Starlark ensure that flows cannot destabilize the grid for other users.

How It Works

Flowmetal provides users with a Starlark programming environment for writing what we call “flows”. Starlark has no native I/O capabilities, ensuring that programs are stateless and idempotent. Flowmetal provides a structured interface in Starlark for performing limited kinds of I/O while ensuring idempotency. This allows flows to “just do side-effects” as users are accustomed while ensuring reliability both for the flow and for the Flowmetal grid.

We are indebted to Meiklejohn et. all for AMBROSIA, which describes this approach in detail.

Let me see the code!

We’ve prepared an introduction length example, but for a quick flavor here’s a hello world.

# Let's declare a quick workflow
hello = flow(
    implementation = _hello_impl,
)

# Read the user's name from the flow's context
def _hello_impl(fctx: FlowContext):
    print("Hello from Flowmetal, {}!".format(fctx.quota.username))

# And python-like specification of script behavior
if __name__ == "__flow___":
    hello()

Can I try it?

While these ideas have had years of refinement, Flowmetal is in early stages of development. We hope to have an interest mailing list and more open development forum soon.

Inquiries may be directed to reid AT domain DOT tld.