Process

Note

Process modules process and therefor modify events in one way or another.

Process modules usually aren’t very cooperative in the Gevent sense of doing things since they aren’t supposed to do any IO.

The builtin Wishbone Output modules:

Name Description
wishbone.module.process.modify Modify and manipulate datastructures.
wishbone.module.process.pack Packs multiple events into a bulk event.
wishbone.module.process.template Renders Jinja2 templates.
wishbone.module.process.unpack Unpacks bulk events into single events.

Process modules must base wishbone.module.ProcessModule:

class wishbone.module.ProcessModule(config)[source]

Bases: wishbone.actor.Actor

generateEvent(data={}, destination=None)

Generates a new event.

This function can get overridden by wishbone.module.InputModule._generateNativeEvent.

The provided data will be traversed in search of valid templates which then will be rendered.

Parameters:
  • data – The payload to add to the event.
  • destination – The destination key to write the data to
Returns:

An event containing data as a payload.
Return type:

wishbone.event.Event
loop()

The global lock for this module.

Returns:True when module is in running mode. False if not.
Return type:bool
postHook()

Is executed when module exits.

preHook()

Is executed when module starts. Can be overriden by the user.

registerConsumer(function, queue)

Registers <function> to process all events in <queue>

Don’t not trap errors here. When <function> fails then the event will be submitted to the “failed” queue, If <function> succeeds to the success queue.

Registering function to consume queue will also apply all the registered module functions against the events consumed from it.

Parameters:
  • function – The function which processes events
  • queue – The name of the queue from which function will process the events.
Returns:

None
renderEventKwargs(event, queue=None)

Renders kwargs using the content of event and stores the result under event.kwargs.

Parameters:
  • event – An Event instance
  • queue – The queue name so RenderKwargs can store the results in the correct queue context.
Returns:

The provided event instance.
Return type:

wishbone.event.Event
renderKwargs()

Renders kwargs without making use of event content. This is typically used when initiliazing a module and render the defined kwargs which do not need a event data for rendering.

Returns:None
sendToBackground(function, *args, **kwargs)

Executes a function and sends it to the background. Such a function should never exit until self.loop returns False. This method wraps function again in a loop as long self.loop returns False so that function is restarted and an error is logged.

Parameters:
  • function – The function which has to be executed.
  • *args – Variable length argument list.
  • **kwargs – Arbitrary keyword arguments.
start()

Starts the module.

Returns:None
stop()

Makes self.loop return False and handles shutdown of of the registered background jobs.

submit(event, queue)

Submits <event> to the queue with name <queue>.

Parameters:
  • event – An event instance.
  • queue – The name of the queue
Returns:

None