红联Linux门户
Linux帮助

Onyx 0.9.0发布,分布式计算系统

发布时间:2016-03-24 09:36:51来源:红联作者:baihuo
Onyx 0.9.0 发布了。

Onyx 是什么?

Onyx 是一个无中心、支持云、容错的分布式计算系统

使用 Clojure 编写

支持批处理和流处理混合

提供信息模型用于描述和构建分布式工作流

竞争对手: Storm, Cascading, Map/Reduce, Dryad, Apache Sqoop, Twitter Crane 等等

用途?

实时的事件流处理

连续计算

ETL

在 MapReduce 进行数据转换

数据清理

数据摄入和存储媒介传输

此版本更新如下:

API breaking change: onyx/id in peer-config and env-config was renamed to :onyx/tenancy-id

API breaking change: :aggregation/fn was renamed to :aggregation/create-state-update.

API breaking change: changed the signatures of: :aggregation/init, :aggregation/create-state-update, :aggregation/apply-state-update, and :aggregation/super-aggregation-fn. All now take the window as the first argument.

API breaking change: internal messaging functions no longer take the event map as an argument. Note, this will break any plugins that manually manage the ack count, such as plugins using async callbacks.

API breaking change: triggers and refinements functions are now resolved to vars via namespace lookup e.g. Trigger example: :trigger/on :segment -> :trigger/on :onyx.triggers/segment, Refinement example: :trigger/refinement :accumulating -> :trigger/refinement :onyx.refinements/accumulating,

Breaking change in onyx-metrics: :metrics/workflow-name was deprecated in favor of metadata map. See onyx-metrics changes

Bug fix: fixed exceptions losing their main .getCause message when thrown inside a lifecycle

New feature: Jobs now support metadata by including a metadata map with job submission data e.g. {:workflow ... :catalog ... :job-metadata {:name "myjobname"}}. This makes it easier to correlate information about jobs, query the cluster for jobs, etc.

Design change: Implemented trigger refinements in terms of state updates. In order to implement a refinement, one must now implement a refinement calls map, analagous to the aggregation calls map. This takes the form {:refinement/create-state-update (fn [trigger state state-event]) :refinement/apply-state-update (fn [trigger state entry])}

Enhancement: Lifecycles can now catch flow condition exceptions through :lifeycycle/handle-exception from the :lifecycle/execute-flow-conditions and :lifecycle/apply-fn phases.

Enhancement: loosened the peer configuration schema needed for log subscription

Dependency change: Upgraded org.clojure/clojure to 1.8.0

下载地址:https://github.com/onyx-platform/onyx/archive/0.9.0.tar.gz

来自:开源中国社区
文章评论

共有 0 条评论