Oban Pro v0.11.1

Changelog for Oban Pro v0.11

pro-testing-module

 Pro Testing Module

In league with Oban v2.12, Pro v0.11 focused extensively on testing improvements. The centerpeice of those improvements is the Oban.Pro.Testing module, a drop-in replacement for Oban.Testing with considerations for unit, integration, and acceptance testing Pro workers.

The helpers provided by Oban.Pro.Testing are dogfooded—that is, they are what Pro uses for testing internally!

Here's a taste of how one of the new testing helpers, run_workflow/2, makes it simple to run a workflow optimally, inline, without any extra configuration:

workflow =
  MyWorkflow.new_workflow()
  |> MyWorkflow.add(:com, MyWorkflow.new(%{text: text, mode: :complexity}))
  |> MyWorkflow.add(:sen, MyWorkflow.new(%{text: text, mode: :sentiment}))
  |> MyWorkflow.add(:syn, MyWorkflow.new(%{text: text, mode: :syntax}))
  |> MyWorkflow.add(:exp, MyWorkflow.new(%{}), deps: [:com, :sen, :syn])

# Using with_summary: false gives us a list of executed jobs, including the
# job's recorded result.
assert [_com, _sen, _syn, exp_job] = run_workflow(workflow, with_summary: false)

assert {:ok, 0.8} = MyWorkflow.fetch_result(exp_job)

Furthermore, there are new guides to introduce Pro testing, and walk you through testing Pro workers.

fewer-plugins-same-functionality

 Fewer Plugins, Same Functionality

In an effort to simplify testing scenarios we've pulled some functionality out of plugins and made that functionality available out of the box, without any extra configuration.

The BatchManager and Relay plugins are deprecated and it's no longer necessary to run them at all! Each deprecated plugin will emit a warning when your app starts, so you'll probably want to remove them:

config :my_app, Oban, plugins: [
- Oban.Pro.Plugins.BatchManager,
- Oban.Pro.Plugins.Relay
]

enhanced-plugin-validation

 Enhanced Plugin Validation

Every Pro plugin implements the Oban.Plugin behaviour, which exposes a validate/1 function that's used for configuration validation at runtime. Pro plugins offer richer functionality (with more options) than OSS plugins, and therefore benefit the most from validation.

v0-11-1-2022-05-24

 v0.11.1 — 2022-05-24

bug-fixes

 Bug Fixes

  • [SmartEngine] Stop immediately deleting producer records with same node

    In some deployments, notably with Kubernetes, the node name is the same between all instances. That caused a new queue to delete the old queue's producer record immediately on boot, subsequently crashing the old producer, restarting it, and then the old producer would delete the new one's record and the cycle would repeat.

    Now only nodes that have exceeded 2x the refresh interval (30s) are deleted.

v0-11-0-2022-04-19

 v0.11.0 — 2022-04-19

enhancements

 Enhancements

  • [Testing] A drop-in replacement for Oban.Testing with helpers for unit and integration testing Batch, Chunk, and Workflow workers.

  • [Workflow] Add all_workflow_jobs/2 for fetching other jobs in a workflow without the need for streaming. It operates in three different modes: fetch all jobs in the workflow, only the current job's dependencies, or only specific dependencies by name. The same options are used by stream_workflow_jobs/2 as well.

  • [Batch] Add all_batch_jobs/2 helper to fetch other non-callback batch jobs without the need for streaming.

  • [SmartEngine] Handle old producer record cleanup while refreshing, rather than relying on the DynamicLifeline plugin. This eliminates race conditions where the lifeline would delete a producer record for an active producer before it could refresh.

  • [Worker] Add safe_decode option to recorded workers to optionally enable binary_to_term decoding safety. Previously the [:safe] option was always used, which could cause decoding errors when decoded atoms hadn't been loaded before. Now safety is disabled by default and it must be turned on explicitly:

    use Oban.Pro.Worker, recorded: [safe_decode: true]

deprecations

 Deprecations

  • [Relay] Is no longer a plugin and it can be removed from your plugins list. Instead, it is considered an extension and was renamed Oban.Pro.Relay.

  • [BatchManager] Is no longer needed and can be removed from your plugins list. Batch management is enabled automatically and runs from within the job's process.

bug-fixes-1

 Bug Fixes

  • [Workflow] Fix pattern matching on partial workflow meta to make unit testing workflow jobs easier.

For changes prior to v0.11 see the v0.10 docs.