Apache MRUnit 1.1 发布了,MRUnit是由Couldera公司开发的专门针对 Hadoop中 编写MapReduce单元测试的框架,基本原理是JUnit4和 EasyMock。MR就是Map和Reduce的缩写。MRUnit框架非常精简,其核心的单元测试依赖于JUnit。而且MRUnit实现了一套 Mock对象来控制OutputCollector的操作,从而可以拦截OutputCollector的输出,和我们的期望结果进行比较,达到自动断言 的目的。
有了MRUnit,对MR程序做重构的时候,只要明确输入和输出,就可以写出单元测试,并且在放到群集校验前进行试验,从而节省时间和资源,也 能更快的定位到问题。而进行重构的话,只要写得足够详细的单元测试都是绿色的话,那么基本就可以保证在群集运行的结果也是正常的。
改进记录包括:
Bug
[MRUNIT-129] - Key object re-use in Reducer is inconsistent with MapReduce behaviour (new API)
[MRUNIT-160] - DistributedCache unit tests fail on Mac OS X 10.8.2
[MRUNIT-185] - IllegalFormatException for '%' in Output
[MRUNIT-186] - method context.getNumReduceTasks() does not work
[MRUNIT-188] - Throw assertion error if drivers are reused
[MRUNIT-189] - MapDriverBase.withInput comments state same as setInput but uses addInput
[MRUNIT-190] - Use mvn enforcer plugin to ensure that a non-vulnerable version of java is used for javadoc generation
[MRUNIT-192] - MRUNIT-180 requires values be comparable
[MRUNIT-193] - Serialization.copy throws NPE instead of ISE (missing serilization impl) for Hadoop 2.x
[MRUNIT-198] - Serialization is missing in MockMultipleOutputs
[MRUNIT-199] - Mapper.Context and Reducer.Context are not fully mocked
[MRUNIT-203] - tarballs should extract to dirs with the same name as tarball
[MRUNIT-204] - Jira and Wiki tutorial are outdated
Dependency upgrade
[MRUNIT-209] - Can't use the maven dependency plugin with maven 3 on mrunit
Improvement
[MRUNIT-164] - Remove duplicated InputPathStoringMapper classes
[MRUNIT-180] - Being able to use a comparator on runTest
[MRUNIT-182] - TemporaryPath should support test data in subdirectories
[MRUNIT-194] - Update versions of Hadoop we compile against
[MRUNIT-208] - mrunit unnecessarily depends on mockito-all
New Feature
[MRUNIT-13] - Add support for MultipleOutputs
[MRUNIT-162] - MapReduce testing utilties
[MRUNIT-205] - Add support for MultipleInputs
Task
[MRUNIT-201] - Update list of committers everywhere (pom/site)
[MRUNIT-207] - Test failure on trunk-hadoop1 branch
项目主页:http://mrunit.apache.org/
下载地址:http://mrunit.apache.org/general/downloads.html
来自:开源中国社区