Intel¿ªÔ´ÁË»ùÓÚApache SparkµÄ·Ö²¼Ê½Éî¶Èѧϰ¿ò¼ÜBigDL£¬BigDL½èÖúÏÖÓеÄSpark¼¯ÈºÀ´ÔËÐÐÉî¶Èѧϰ¼ÆË㣬²¢¼ò»¯´æ´¢ÔÚHadoopÖеĴóÊý¾Ý¼¯µÄÊý¾Ý¼ÓÔØ¡£
ÔÚXeon·þÎñÆ÷ÉÏÔËÐеIJâÊÔ½á¹û±íÃ÷£¬BigDL±ÈÆäËû¿ªÔ´¿ò¼ÜCaffe¡¢TorchºÍTensorFlowÓÐÏÔÖøµÄÐÔÄÜÌáÉý¡£BigDLËٶȿÉÓëÖ÷Á÷GPUÆ¥µÐ£¬¶øÇÒÄܹ»À©Õ¹µ½ÊýÊ®¸öXeon·þÎñÆ÷£¬BigDL¿âÖ§³ÖSpark 1.5¡¢1.6ºÍ2.0°æ±¾£¬²¢ÈÝÐí½«Éî¶ÈѧϰǶÈëÔÚÏÖÓеÄSpark³ÌÐòÖУ¬BigDL¿âÖÐÓаÑSpark RDDsת»»ÎªBigDL DataSetµÄ·½·¨£¬²¢ÇÒ¿ÉÒÔÖ±½ÓÓëSpark ML PipelinesÒ»ÆðʹÓá£
¶ÔÓÚÄ£ÐÍѵÁ·£¬BigDLʹÓÃÁËÔÚ¶à¸öÖ´ÐÐÆ÷ÖÐÖ´Ðе¥¸öSparkÈÎÎñµÄͬ²½Ð¡ÅúÁ¿SGD£¨Stochastic Gradient Descent£©£¬Ã¿¸öÖ´ÐÐÆ÷ÔËÐÐÒ»¸ö¶àÏß³ÌÒýÇæ²¢´¦ÀíÒ»²¿·Ö΢Åú´ÎÊý¾Ý£¬ÔÚµ±Ç°°æ±¾ÖУ¬ËùÓеÄѵÁ·ºÍÑéÖ¤Êý¾Ý¶¼´æ´¢µ½´æ´¢Æ÷ÖС£
BigDLʹÓÃScala¿ª·¢£¬²¢²Î¿¼ÁËTorchµÄÄ£ÐÍ£»ÏñTorchÒ»Ñù£¬ËüÓÐÒ»¸öʹÓÃIntel MKL¿â½øÐмÆËãµÄTensorÀ࣬Intel MKL£¨Math Kernel Library£©ÊÇÓÉһϵÁÐΪ¼ÆËãÓÅ»¯¹ýµÄС³ÌÐòËù×é³ÉµÄ¿â£¬ÕâЩС³ÌÐò´ÓFFT£¨¿ìËÙ¸µÁ¢Ò¶±ä»»£©µ½¾ØÕó³Ë·¨¾ùÓÐÉæ¼°£¬³£ÓÃÓÚÉî¶ÈѧϰģÐÍѵÁ·£¬ModuleÊÇÁíÒ»¸ö´ÓTorch½è¼ø¶øÀ´µÄ¸ÅÄËüµÄÁé¸ÐÀ´×ÔTorchµÄnn package£¬Module´ú±íµ¥¶ÀµÄÉñ¾ÍøÂç²ã¡¢TableºÍCriterion¡£
BigDLÌṩÁËÒ»¸öAWS EC2¾µÏñºÍһЩʾÀý£¬±ÈÈçʹÓþí»ýÉñ¾ÍøÂç½øÐÐÎı¾·ÖÀ࣬»¹ÓÐͼÏñ·ÖÀàÒÔ¼°ÈçºÎ½«ÔÚTorch»òCaffeÖÐԤѵÁ·¹ýµÄÄ£ÐͼÓÔص½SparkÖнøÐÐÔ¤²â¼ÆË㣬À´×ÔÉçÇøµÄÇëÇóÖ÷Òª°üÀ¨Ìṩ¶ÔPythonµÄÖ§³ÖºÍMKL-DNN£¨MKLµÄÉî¶ÈѧϰÀ©Õ¹£©¡£
±¾ÎĵØÖ·£ºhttp://www.linuxprobe.com/base-spark-bigdl.html
Ronny ÓÚ 2017-01-22 22:26:02·¢±í:
´óÊý¾ÝºÜÍþÎä