Wishbone

A Python framework to build composable event pipeline servers with minimal effort.

https://github.com/smetj/wishbone

Bootstrap a server

Wishbone servers are started with bootstrap file:

$ wishbone start --config eventprocessor.yaml

Bootstrap files define the builtin and external modules to initialize and how these should be connected to each other:

_images/intro.png
lookups:
  randomword:
    module: wishbone.lookup.randomword

modules:
  input:
    module: wishbone.input.testevent
    description: I generate a random word.
    arguments:
      message: ~~randomword()

  mixing:
    module: wishbone.flow.roundrobin
    description: I roundrobin incoming messages

  output1:
    module: wishbone.output.stdout
    description: I write incoming messages to stdout.
    arguments:
      prefix: "I am output #1: "

  output2:
    module: wishbone.output.stdout
    description: I write incoming messages to stdout.
    arguments:
      prefix: "I am output #2: "

routingtable:
  - input.outbox  -> mixing.inbox
  - mixing.one    -> output1.inbox
  - mixing.two    -> output2.inbox
...

Running a server:

[smetj@dev-container ~]$ wishbone debug --config simple.yaml
Instance started in foreground with pid 5434
2016-02-17T20:42:59 wishbone[5434]: debug output2: Connected queue output2.logs to _logs.output2
2016-02-17T20:42:59 wishbone[5434]: debug output2: Connected queue output2.metrics to _metrics.output2
2016-02-17T20:42:59 wishbone[5434]: debug output2: preHook() found, executing
2016-02-17T20:42:59 wishbone[5434]: debug output2: Initialized.
2016-02-17T20:42:59 wishbone[5434]: debug output2: Started with max queue size of 100 events and metrics interval of 1 seconds.
2016-02-17T20:42:59 wishbone[5434]: debug output1: Connected queue output1.logs to _logs.output1
2016-02-17T20:42:59 wishbone[5434]: debug output1: Connected queue output1.metrics to _metrics.output1
2016-02-17T20:42:59 wishbone[5434]: debug output1: preHook() found, executing
2016-02-17T20:42:59 wishbone[5434]: debug output1: Initialized.
2016-02-17T20:42:59 wishbone[5434]: debug output1: Started with max queue size of 100 events and metrics interval of 1 seconds.
2016-02-17T20:42:59 wishbone[5434]: debug input: Connected queue input.logs to _logs.input
2016-02-17T20:42:59 wishbone[5434]: debug input: Connected queue input.metrics to _metrics.input
2016-02-17T20:42:59 wishbone[5434]: debug input: Connected queue input.outbox to mixing.inbox
2016-02-17T20:42:59 wishbone[5434]: debug input: preHook() found, executing
2016-02-17T20:42:59 wishbone[5434]: debug input: Started with max queue size of 100 events and metrics interval of 1 seconds.
2016-02-17T20:42:59 wishbone[5434]: debug mixing: Connected queue mixing.logs to _logs.mixing
2016-02-17T20:42:59 wishbone[5434]: debug mixing: Connected queue mixing.metrics to _metrics.mixing
2016-02-17T20:42:59 wishbone[5434]: debug mixing: Connected queue mixing.one to output1.inbox
2016-02-17T20:42:59 wishbone[5434]: debug mixing: Connected queue mixing.two to output2.inbox
2016-02-17T20:42:59 wishbone[5434]: debug mixing: preHook() found, executing
2016-02-17T20:42:59 wishbone[5434]: debug mixing: Started with max queue size of 100 events and metrics interval of 1 seconds.
I am output #2: seawater's
I am output #1: hinders
I am output #2: stigmatism
I am output #1: damnedest
I am output #2: ejects
I am output #1: legates
I am output #2: lobos
I am output #1: punctures
I am output #2: port
I am output #1: condominium's
I am output #2: banqueted
I am output #1: bucker
I am output #2: efficiencies
... snip ...