Output

Note

Output modules submit data to external services.

Output module properties:

• They have a protocol encoder method mapped to wishbone.module.OutputModule.encode() in order to convert the desired wishbone.event.Event payload into the desired format prior to submitting it to the external service.
• Should always provide a selection, payload, native_events and parallel_streams module parameter. If payload is not None, then it takes precendence over selection. Selection defines the event key to submit whilst template comes up with a string to submit.``payload`` usually makes no sense with bulk events.
• Should use wishbone.module.OutputModule.getDataToSubmit() to retrieve the actual data to submit to the external service.This automatically takes care of bulk events.
• Through inheriting wishbone.module.OutputModule Output modules override wishbone.actor.Actor._consumer() with their own version which executes the registered function in parallel greenthreads by using a threadpool. The module’s parallel_streams parameter defines the size of the pool and therefor the number of parallel greenthreads submitting the event data externally.It depends on the nature of your output protocol whether this makes sense.Normally you shouldn’t really bother with this as long a Gevent’s monkey patching works on the code you’re using to speak to the remote service.
• If you’re setting a default encoder function make sure you use wishbone.module.OutputModule.setEncoder() as this method will prevent overwrite any user defined encoder set via wishbone.actorconfig.ActorConfig.

Warning

Be aware that if parallel_streams is larger than 1, the equal amount of events will be processed concurrently by the function registered with wishbone.actor.Actor.registerConsumer() to consume the queue. Within that function do NOT change shared (module) variables but only use local (to the function) ones.

The builtin Wishbone Output modules:

Name	Description
wishbone.module.output.null	Purges events.
wishbone.module.output.stdout	Prints event data to STDOUT.
wishbone.module.output.syslog	Submits event data to syslog.

---

Output modules must base wishbone.module.OutputModule:

class wishbone.module.OutputModule(config)

Bases: wishbone.actor.Actor

generateEvent(data={}, destination=None)

Generates a new event.

This function can get overridden by wishbone.module.InputModule._generateNativeEvent.

The provided data will be traversed in search of valid templates which then will be rendered.

Parameters:

• data – The payload to add to the event.
• destination – The destination key to write the data to

Returns:

An event containing data as a payload.

Return type:

wishbone.event.Event

getDataToSubmit(event)

Derives the data to submit from event taking into account native_events, payload and selection module parameters.

Parameters:event – The event to extract data from. Returns:The data to submit. Return type:dict/str/…

loop()

The global lock for this module.

Returns:True when module is in running mode. False if not. Return type:bool

postHook()

Is executed when module exits.

preHook()

Is executed when module starts. Can be overriden by the user.

registerConsumer(function, queue)

Registers <function> to process all events in <queue>

Don’t not trap errors here. When <function> fails then the event will be submitted to the “failed” queue, If <function> succeeds to the success queue.

Registering function to consume queue will also apply all the registered module functions against the events consumed from it.

Parameters:

• function – The function which processes events
• queue – The name of the queue from which function will process the events.

Returns:

None

renderEventKwargs(event, queue=None)

Renders kwargs using the content of event and stores the result under event.kwargs.

Parameters:

• event – An Event instance
• queue – The queue name so RenderKwargs can store the results in the correct queue context.

Returns:

The provided event instance.

Return type:

wishbone.event.Event

renderKwargs()

Renders kwargs without making use of event content. This is typically used when initiliazing a module and render the defined kwargs which do not need a event data for rendering.

Returns:None

sendToBackground(function, *args, **kwargs)

Executes a function and sends it to the background. Such a function should never exit until self.loop returns False. This method wraps function again in a loop as long self.loop returns False so that function is restarted and an error is logged.

Parameters:

• function – The function which has to be executed.
• *args – Variable length argument list.
• **kwargs – Arbitrary keyword arguments.

setEncoder(name, *args, **kwargs)

Sets the encoder with name <name> unless there’s already an encoder defined via actorconfig.ActorConfig.

Parameters:

• name – The name of the encoder to initialize
• *args – Variable length argument list.
• **kwargs – Arbitrary keyword arguments.

Returns:

True if the encoder is set, False when an encoder was already

set via actorconfig.ActorConfig

Return type:

Bool

start()

Starts the module.

Returns:None

stop()

Makes self.loop return False and handles shutdown of of the registered background jobs.

submit(event, queue)

Submits <event> to the queue with name <queue>.

Parameters:

• event – An event instance.
• queue – The name of the queue

Returns:

None