红联linux Linux通用技术 - Linux发行版技术 - 企业应用 - 实验室  Linux论坛

您的位置: 红联linux >> Linux通用技术 >>

Linux软件下载

>> 查看内容 -学Linux必读Linux学习教程推荐下载发新话题 (游客可发贴)

JStorm 2.2.1发布,分布式计算系统

uineqa发布于 2017-1-10 | 次阅读   字号:     (网友评论 0 条) 我要评论


JStorm 2.2.1 发布,更新内容如下:

新功能

Performance is improved by 200%~300%, compared to Release 2.1.1 and 0.9.8.1 in several testing scenarios, while 120%~200% compared to Flink and 300%~400% compared to Storm.

Restructure the batch solution

Improve serialization and deserialization to reduce the cost of cpu and network

Improve the cost of cpu on critical path and metrics

Improve the strategy of netty client and netty server

Support consume and publish of disruptor queue under batch mode

Introduce snapshot exactly once framework

Compared to Trident solution, the performance of new framework is increased by several times. Besides it.

The new framework also support "at least once" mode. Compared to the acker mechanism,it will reduce the cost of relative calculation in acker, and the cost of network, which will improve the performance singificantly.

Support JStorm on yarn

Currently, jstorm cluster is capable of fast deployments,and fast scale-in/scale-out. It will improve the utility of resource.

Re-design the solution of backpressure. Currently, the flow control is stage by stage。

The solution is simple and effective now. The response is much more faster when the exchange of switch on/off of backpressure.

The performance and stability is improved significantly, compared to the original solution.

Introduce Window API

Support tumbling window,sliding window

window support two collection mode, count and duration.

Support watermark mechanism

Introduce the support of Flux

Flux is a programing framework or component which is aim to help create and deploy the topology of jstorm quickly.

Isolate the dependencies of jstorm and user topology by maven shade plugin to fix the conflict problem.

Improve Shuffle grouping solution

Integrate shuffle, localOrShuffle and localFirst. The grouping solution will be auto adapted according to the assignment of topology.

Introduce load aware in shuffle to ensure the load balance of downstreams.

Support to configure blacklist in Nimbus to exclude some problematic nodes

Support batch mode in trident

Supervisors will synchronize cluster configuration from nimbus master automatically

Add buildTs to supervisor info and heartbeats

Add ext module for nimbus and supervisor to support external plugins

Add jstorm-elasticsearch support, thanks to @elloray for your contribution

改进

Restructure nimbus metrics implementation. Currently, the topology metrics runnable is event-driven.

Restructure topology master. Currently, the processor in TM is event-drive.

Add some examples to cover more scenarios

Disable stream metrics to reduce the cost of sending metrics to Nimbus

Support metrics in local mode

Improve the implementation of gauge by changing the instantaneous value of each minute,to the average value of some sample values in each minute.

Introduce an approximate histogram calculation to reduce memory usage of histogram metrics

Add Full GC and supervisor network related metrics

修复

Fix message disorder bug

Fix the bug that some connections to zookeeper are not closed by expected when encountering exception in supervisor.

The deactivate might be called by mistake when task init

The rootId might be duplicated occasionally. It will cause the unexpected message failure.

Fix the bug when local mode

Fix logwriter's bug

Some task metrics(RecvTps ProcessLatency) might not be aggregated correctly.

Fix the racing condition of AsmCounter during flushing

软件详情:https://github.com/alibaba/jstorm/

下载地址:https://github.com/alibaba/jstor ... .1/jstorm-2.2.1.zip

来自:开源中国社区

 我要评论
发表评论,与各位同人交流。回复请点击下方的我要评论按钮(游客可回复),要发表贴子请点击 发新话题 (游客可发贴)
 热点主题
·YiiCms 1.1发布,企业站管
·Android Studio 2.3 Beta2
·talent-aio 1.0.0正式版,
·Kodi v17 "Krypton" rc2发
·即时通讯框架talent-aio 0
·JEECMS v8.1发布,首个微信
·WordPress 4.7.1发布,解决
·Quirky 8.1.6,用Woof创建
·自动化测试框架AutoTest发
·Debian 9 “Stretch”安装
 最新主题
·YiiCms 1.1发布,企业站管
·Java组播框架JGroups-4.0.
·Mybatis-Plus 2.0.1拥抱Po
·SmartWiki 0.3.3发布,新增
·NutzWk 3.3.4发布,Java企
·OpenVMS移植到x86硬件的更新
·Apache Johnzon 1.0.0,用
·koahubjs 1.0.6发布,基于
·基于TP5的lyadmin 2.0发布
·FastQuery 1.0.20发布,Ja
 关注热点

Linux教程下载?
Linux学习教程推荐下载”(请点击),Linux教程免费下载。

求助Linux问题?
论坛有39版块,覆盖所有Linux技术层面。前往“红联Linux论坛

 

Linux系统教程 | Linux入门 | Linux管理 | Linux技巧 | Linux命令 | Ubuntu系统 | Linux服务器 | Linux数据库 | Linux开发

合作 | 关于我们 | 联系我们 | 导航 | 会员注册

©2017 红联 Powered by SupSite