ºìÁªLinuxÃÅ»§
Linux°ïÖú

¡¶SRE£ºGoogleÔËά½âÃÜ¡·

·¢²¼Ê±¼ä:2018-01-19 21:05:38À´Ô´:ºìÁª×÷Õß:Ronny
ÇëÊäÈëͼƬÃèÊö
Ç°ÑÔ
ÎÊÊÀ½üÒ»ÄêÒÔÀ´£¬¡¶SRE: Google ÔËά½âÃÜ¡·Ò»ÊéÏúÁ¿ÀÛ¼ÆÒÑÁ½ÍòÓà²á¡£ÎÒÏëÊ×ÏȸÐл¸÷λ¶ÁÕ߶Ա¾ÊéµÄÖ§³Ö£¬ÕæµÄÊÇÒÂʳ¸¸Ä¸Ñ½£¡Èç¹û»¹Ã»ÓÐϵ¥¹ºÂò£¬ÊDz»ÊÇ¿´¹ý±¾ÎÄÖ®ºó¿ÉÒÔ¿¼ÂǾ¡¿ìÄØ£¿
Ëæ×ÅSRE¸ÅÄîÔÚÔÚ¹úÄÚÍâµÄ»ð±¬´«²¥£¬ÏàÐźܶàÅóÓÑÒ²¶Ô´ËÊéÓÐÁËÒ»¶¨³Ì¶ÈµÄÁ˽⡣¸Ðл GitChat ƽ̨£¬ÎÒÕâ´ÎÓлú»áºÍ´ó¼ÒÔÙ·ÖÏíÒ»ÏÂÊéÄÚÊéÍâµÄ¹ÊÊ¡£Ï£Íû²»¹ÜÄúÊÇ·ñ¿´¹ý±¾Ê飬±¾ÎĵÄÐÅÏ¢¶¼ÄܶÔÄúÓÐÓã¡
³ÉÊ黨Ðõ
Ê×ÏÈÒªÃ÷È·µÄÊÇ£¬Õâ±¾ÊéÊÇÒ»±¾Îļ¯¡£GoogleÄÚ²¿ÔÚ2014Äê½øÐÐÁËÒ»´ÎÌâÄ¿+×÷Æ·Õ÷¼¯¹¤×÷£¬ÓÉ·âÃæÉϵıàÕßÍŽ«ÊÕ¼¯À´µÄͶ¸åɸѡ£¬¾Ý³Æ¿³µô2/3ÄÚÈݲŠ£¨Ô­Ê¼×ÊÁϾݳƳ¬¹ý1500Ò³£© ×îºóµÃÒÔÎÊÊÀ¡£ ¼¼ÊõÀàÊé¼®³ÉÊéÄÑ£¬ºÍ´óÊý¾ÝÏîÄ¿µÄ¹Ø¼üÄѵãºÜÏñ¡£ÄǾÍÊÇ----ûÓÐÊý¾Ý¡£¾ÝÎҺ͹úÄÚÔËά¸Úλ½»Á÷µÄ¸Ð¾õ£¬±ð˵дÎĵµÁË£¬¾ÍÁ¬·þÎñÆ÷ÃÜÂ붼ÊÇÖ»¼ÇÔÚÄÔ×ÓÀïµÄ£¬¸ù±¾Ã»Ê±¼äдÏÂÀ´ºÃÂï¡£ÍæЦ¹éÍæЦ£¬µ«ÊǵÀÀí²»±ä£¬ Google SRE ¸øÎÒÁôϵÄÒ»¸ö²»¿ÉÄ¥ÃðµÄÓ¡¼Ç¾ÍÊÇ----ÎĵµÖعýÒ»ÇС£Ã»Ð´Îĵµ£¬¾È»ðÔðÈÎÓÀÔ¶ÔÚÄã×Ô¼ºÉíÉϱ³×Å£¬³öÎÊÌâ°ëÒ¹Ò²ÒªÅÀÆðÀ´½â¾ö°¡¡£Ð´ÍêÎĵµ£¬¾È»ðÔðÈξͱä³É´ó¼Ò¹²ÏíµÄÁË£¬¾Í¿ÉÒÔ¿ªÐĵÄ˯¾õÁË¡£Ã¿¸öÍŶӳÉÔ±»áÖ÷¶¯Î¬»¤±¾×éÎĵµ£¬Ê±¿Ì±£³ÖÆäΪ×îС£ÖªÊ¶¹²ÏíµÄµÚÒ»²½£¬µ±È»¾ÍÊÇÒªÏÈ°Ñ֪ʶ¼Ç¼ÏÂÀ´°¡¡£
2016Äê³õÎÒ¾ÍÌý˵Õâ±¾ÊéºÜ¿ì»á·¢ÐУ¬4Ô³õÓ¢ÎÄ°æÉϼÜÖ®ºó£¬ÁªÏµ³ö°æÉçµÈһϵÁÐÊÂÇé×öÍêÖ®ºó£¬ÒѾ­Êǽü5Ôµ×ÁË¡£6ÔºÍ7ÔÂÕæµÄÊÇÈÕÒÔ¼ÌÒ¹£¬Ã¿Ìì·­Òë²î²»¶à 6- 8Сʱ £¬8ÔÂ×öÍêÉóУµÈ¹¤×÷£¬9ÔÂÕýʽÍê³ÉÊ×·¢ºÍ´ó¼Ò¼ûÃæ ¡£ÎÒ×ö¹ýͳ¼Æ£¬Ã¿´ÎÎÒÄܼá³ÖÈ«Éñ¹á×¢·­ÒëÁ½¸öСʱ£¬×î¶àÖ»ÄÜ·­Òë10Ò³Ô­ÎÄ¡£¿ÉÊǸÃÊéÓ¢ÎÄ°æ¾Í550Ò³°¡£¬¿ÞËÀ¡£ÔÚÎÞÊý¸öÌôµÆÒ¹Õ½µÄÒ¹Íí¹ýºó£¬ÎÒ×ÔÎÊ»¹ËãÊǽ»ÉÏÁËÒ»·Ý²îÇ¿ÈËÒâµÄ´ð¾í£¬Ï£ÍûûÓй¼¸º´ó¼ÒµÄÆÚ´ýºÍÐÅÈΡ£
±¾Êé·âÃ棺¾Þòá
¾Ý±àÕß͸¶£¬ËûÃÇÑ¡ÔñÁ˺ܶදÎïµÄ·½°¸£¬¿Éϧ¶¼Ôø±» O¡¯Reilly ÆäËûµÄÊéÓùýÁË£¬×îÖÕ¾ö¶¨Õâ¸ö·½°¸µÄÔ­ÒòÊÇ£¬¾ÞòáµÄÓ¢ÎÄÃû£¨(Monitor Lizard)£¬Monitor Óë SRE³£ÓÃ´Ê ¡±¼à¿Ø¡° һģһÑù£¬ÓÚÊǾÍÕâôÓä¿ìµÄ¾ö¶¨ÁË¡££¨nerdµÄÈÕ³££¬Ð¦£©
Ê×·¢ÏÖ³¡Çëµ½Á˱àÕßÖ®Ò» Chris Jones, ʱÈÎ App Engine SRE£¬ºÜ¶à³ÉÊéµÄÄÚÄ»ÏûÏ¢¾ÍÊÇËû¸æËßÎҵġ£¾Ý³ÆÃÀ¹úÊ×·¢ÒÇʽδ×öÐû´«£¬Ò»¹²Ö»·¢³ö²»µ½10±¾¡£¶øÔÚÖйú£¬ÔÚ GOPS ´ó»áÖ÷ϯÏôÌï¹úÏô°ïÖ÷µÄ´óÁ¦Ö§³ÖºÍÐû´«Ï£¬Ê×·¢Ò»¸öСʱ֮ÄÚ×¼±¸µÄÁ½°Ù±¾Ç©ÃûÊéÈ«²¿ÊÛóÀ¡£
ÏÂÃæÔÛÃÇÑÔ¹éÕý´«£¬ÏêϸÖðһ˵һ˵ÊéÖеÄÄÚÈÝ¡£
µÚÒ»Õ£º SRE ¸ÅÊö
»ù±¾ÉÏÔÚÿ´Î½»Á÷µÄ¹ý³ÌÖУ¬´ó¼Ò×Ü»á̸µ½ SRE ÀíÂÛÌåϵµÄ½¨Á¢¹ý³Ì£¬ÒÔ¼°Óë¹úÄÚ³£¼ûµÄÔËά¸Ú붨ÒåÖ®¼äµÄÇø±ðÓëÏà֮ͬ´¦¡£ÕâÀïÎÒÌá³ö¼¸¸ö¿´·¨£º
ÂíºóÅÚ
ÎÒÔø¾­ÔÚÊý´ÎÑݽ²ÖÐÓùý¡°ÂíºóÅÚ¡±Õâ¸ö´ÊÀ´ÃèÊöSREµÄÀíÂÛÌåϵ¡£ÖÐÎÄ¡±ÂíºóÅÚ¡°Ò»´ÊµÄÈ·ÂÔÓÐ ±áÒ壬µ«ÊǸоõÓÃÔÚÕâÀïÈ´Ç¡ÈçÆä·ÝµÄÃèÊöÁËSRE½¨Á¢¹ý³ÌÖÐÃþ×Åʯͷ¹ýºÓµÄ¹ý³Ì¡£ÊÀ½çÉϲ»´æÔÚÍòÄÜÒ©£¬SRE ÌåϵҲ¾ø·ÇÒ»ÌìÖ®ÄÚÉè¼Æ½¨³ÉµÄÍêÃÀÂÞÂí³Ç¡£ÕâÀïÓÐÎҵĸöÈËÀí½â£ºSRE ´ú±íµÄÊÇÒ»ÖÖÐµĹ¤×÷·½Ê½¡£Í¨¹ýÖ÷¶¯ÊÕ¼¯ÒÔÇ°É¢Âä¸ø¸÷¸ö²¿ÃŵÄһϵÁоßÓÐÏà¹ØÐÔµÄÖ°Ôð(Accountability)£¬¹ÜÀí²ãͬʱÅä·¢ÁËÒ»¶¨µÄ¾ö²ßȨ(Authority )£¬ÔÙÀûÓÃÒ»¶¨³Ì¶ÈµÄ×ÔÖÎȨ(Autonomy) ´ïµ½¿É³ÖÐø·¢Õ¹µÄÄ¿µÄ£¬ÕâÈýÖÖÒòËØÏ໥½áºÏ£¬Ï໥×÷ÓÃ×îÖÕÔì¾ÍÁË Google ÄÚ²¿¾Ù×ãÇáÖصÄǧÓàÈË SRE ÍŶӡ£
Óë¹úÄÚÔËάµÄ±¾ÖÊÇø±ðÔÚÓÚÖ°ÔðÓëȨÁ¦µÄͳһ
ÊéÖб¾ÕÂËùÁаËÌõ·½·¨ÂÛ£¬¿ÉÒÔºÍÉÏÊöAAA·Ö±ð¶ÔÓ¦¡£ÎÒÔÚ½»Á÷Öо­³£¿ªÍæЦµÄ˵£¬SREÒ²ÊDZ³¹øÏÀ£¬²»Ò»ÑùµÄÊÇ SRE ÊÇ×ÔºÀµÄ±³¹øÏÀ¡£²»½öÓйø±³¹ø£¬»¹¾­³£ÕÒµ½ÔÝʱûÓÐÔì³ÉÒµÎñÓ°ÏìµÄ¹øÀ´Ö÷¶¯±³£¬×ÔºÀµÄ±³¡£Èç¹ûÒªÎÊÕâÊÇΪʲô£¿ÄÇÎÒ¿ÉÒÔ·´ÎÊ£¬ÕâÄѵÀ²»¾ÍÊÇÖ°ÔðËùÔÚÂ𣿼ÈÈ» SRE ¸ºÔð·þÎñµÄÎȶ¨ÐÔ£¬ÄǾÍÓ¦¸Ã³Ðµ£·þÎñ²»Îȶ¨µÄºó¹û¡£¶ÔÖ°ÔðµÄ¿¹¾Üͨ³£ÊÇÓÉÓÚ¾ö²ßȨ²»Í³Ò»£¬ÔËάÍŶÓÎÞ·¨Ïû³ýÎÊÌâÒþ»¼£¬Ò²ÎÞ·¨¸Ä½ø´úÂ룬ÔÙ¼ÓÉϳ£³£¹éÔðÓÚÈ˵ÄÖƶȶøÔì³ÉµÄ¡£¾ÍÊÂÂÛÊ£¬ËµÆðÀ´ÈÝÒ××öÆðÀ´ÄÑѽ¡£µ«ÊÇÎÒ¾õµÃÒ»¶¨ÒªÒâʶµ½£¬¸öÈËÓ¢ÐÛÖ÷ÒåÓÀÔ¶ÊDz»Ì«Îȶ¨µÄÒÀ¿¿¡£¸ÖÌúÏÀÒ²ÓдóÆð´óÂ䣬¶Ô°É¡£Ïà±ÈÖйúÎÄ»¯Öг£³£¹àÊäµÄÍì¿ñÀ½Óڼȵ¹ÕâÑùµÄ´ó³¡Ã棬SRE ¸üϲ»¶Æ½Ê±ÅųýÒþ»¼£¬µ½µã¾Í»Ø¼Ò˯¾õ¡£ÕâЩÎÊÌâÔÚÊéÖкóÃæµÄÕ½ڲ»ÊÇ»áÓÐÌÖÂÛ£¬Ï£Íû´ó¼Ò¿´µÃʱºò¿ÉÒÔ¶àÁôÒâһЩ¡£
¶ÔSRE·ç³±µÄ¸öÈËÀí½â
ºÜ¶àÆóÒµµÄ²úÆ·Á÷³ÌÈÔȻͣÁôÔÚËùν¼Æ»®¾­¼Ã+Á´ÌõÐÍÌåÖÆÏÂÔËת¡£ ÒµÎñ½»ÐèÇó¸øÑз¢£¬Ñз¢½»´úÂë¸øÔËάÉÏÏߣ¬Ðí¶àÐí¶àÎÊÌⶼÊÇÔÚÕâÖÖ¡°ÈÓ¹ýǽ¡±µÄ¹ý³ÌÖвúÉúµÄ¡£
Ä¿Ç°Öð½¥·çÐÐµÄ DevOps ÀíÄîÇ¿µ÷µÄÊǽ«ITÄÜÁ¦ÏòÁ´ÌõÇ°²¿Íƽø£¬Èç¹û½«´ËÓ¦ÓÃÔÚ Dev Óë Ops Ö®¼ä£¬ÄÇô¾ÍÊÇͨ¹ý CI/CD µÈÊֶΣ¬´ïµ½´Ù½ø Dev ºÍ Ops Ö®¼äµÄºÏ×÷¹ØϵµÄÄ¿µÄ¡£ ¶ø SRE£¬ »òÕß˵ Google£¬ Ôò¸ü½øÒ»²½£¬½«Á´ÌõʽÌåϵÉý¼¶»»´úΪÈý¸ùÖ§ÖùÌåϵ£¨ÎÒ¹ÚÃûµÄ£©¡£ ÒµÎñ¡¢Ñз¢¡¢ SRE Èý¸ùÖ§Öù»¥ÏàÖ§³Å£¬»¥ÏàЭ×÷£¬´ïµ½ÁËÒ»¸ö¸ü¸ßЧµÄЭ×÷¸ß¶È¡£Õâ±¾ÊéÖÐÔ̺¬µÄ·½·½ÃæÃ棬¶¼»ù±¾¿ÉÒÔ¹é½áΪ SRE ÓëÒµÎñÍŶӺÍÑз¢ÍŶÓÖ®¼ä½»»¥ËùÉæ¼°µ½µÄ·½·½ÃæÃæ¡£ ËùÒÔ£¬SRE±ÈÎÒÃÇÖйúÓï¾³ÖеÄÔËά¸ÚλҪ¹ã·ººÜ¶à£¬¿ÉÒÔ³ÆΪһ¸öÖ°ÒµÁË¡£
µÚ¶þÕ£º Google µÄÉú²ú»·¾³
ÔÚ±¾ÕÂÖÐ Google ÃèÊöÁËÆäÊÀ½çÎÅÃûµÄÊý¾ÝÖÐÐÄÉè¼ÆºÍÄÚ²¿Ó¦ÓÃÈí¼þ¼Ü¹¹¡£ÓÃÒ»¾ä¼òµ¥µÄ»°½²£¬ GoogleÄÚ²¿ÊµÏÖµÄÊÇ×ÊÔ´½»¸¶µÄƽ̨»¯¡£Ò»¸öÆÕͨԱ¹¤£¬ÉõÖÁ·Ç³ÌÐòÔ±ËùÄܵ÷¶¯µÄ×ÊÔ´¹æÄ£¶¼ÊǼ¸ºõÁîÈËÄÑÒÔÖÃÐŵġ£¾Ù¸öÀý×Ó£¬Ã¿¸ö Google ³ÌÐòÔ±ÈëÖ°µÚÒ»Öܶ¼»áÔËÐÐÒ»¸öÔËÐÐÔÚÊý°Ų̀»úÆ÷Éϵķֲ¼Ê½ÅÅÐò³ÌÐò£¬²»ÐèÒªÉêÇ룬Ҳ²»ÐèÒªÅŶӣ¬ËæʱÐèÒªËæʱÔËÐС£Google ×î½ü¼ÆËã³öµÄ SHA1 ³åͻֵ£¬ºÄ·ÑÁË 6500 ¸öCPU Ä꣬ÒÔ¼°110 ¸ö GPU Äê¡£ÆäʵËãËãÒ²²»¹ýÒ»Íò¶ą̀8ºË·þÎñÆ÷ÅÜÒ»¸öÔÂŶ£¬»¹ºÃ£¬»¹ºÃ¡£
ÎÒÔÚÕâÀォÕâÒ»ÕÂÊéÖн²µÄºÜ¶àСµã¹é½áΪÈýÀà»ù±¾×ÊÔ´·Ö±ð½øÐÐÃèÊö¡£
¼ÆËã×ÊÔ´
¼ÆËã×ÊÔ´µÄ·ÖÅäºÍʹÓã¬ÊÇÏÖÔÚ³´µÃ×îÈÈÃŵĻ°ÌâÁË¡£²»¹ÜÊÇ֮ǰµÄOpenStack, vmware, xen, Ganeti ( ºÜ¶àÈ˲»ÖªµÀÕâ¸öÓÅÐãµÄGoogle Xen ¹ÜÀíÌ×¼þ¿ªÔ´ÏîÄ¿£©, »¹ÊÇ×î½ü»ðÈȵÄÈÝÆ÷ÔÆ, Docker, Kubernetes, Mesos ±¾ÖÊÉ϶¼ÊÇÔÚ×öͬһ¼þÊ ·ÖÅäºÍʹÓÃÎïÀí×ÊÔ´¡£ ÔÚ±¾ÕÂÖУ¬GoogleÃèÊöÁËBorgϵͳµÄ¼Ü¹¹¡£
ÓÉÓÚ Google ÎïÀí×ÊÔ´¹æģʮ·Ö¾Þ´ó£¬È«Çò°ÙÓà¸ö¼¯Èº£¬°ÙÍò+Á¿¼¶µÄ·þÎñÆ÷ÊýÁ¿£¬ºÜ¶à´«Í³µÄ¹ÜÀí·½Ê½ÒѾ­²»ÊÊÓÃÁË¡£ Ê×ÏÈ£¬ËäÈ»µ¥¸öÎïÀí×ÊÔ´Ëð»µÂʲ»¸ß£¬µ«ÊÇÔÚÕâôÅÓ´óµÄ»ùÊýÏ£¬»ù±¾Ã¿·ÖÖÓ¶¼ÓзþÎñÆ÷å´»ú£¬ÖØÆô£¬»òÕß³öÏÖÓ²¼þ¹ÊÕÏ¡£Èç¹ûÒÔ´«Í³ Ops µÄ·½Ê½À´´¦ÀíÕâЩ¹ÊÕÏ£¬ÓÉijOpsÔ¶³Ì´¦Àí»òÕßÇ××Էɵ½¶ÔÓ¦µØµã£¬µ±³¡µ÷ÊÔ½â¾öËùÓÐÎÊÌâÔٻؼң¬ÄÇÕжàÉÙÈËÒ²²»¹»ÓÃѽ¡£ËùÒÔÔÚGoogleÕâ¸öÌåÁ¿¹æģϣ¬È«Çò²¿ÊðµÄÏÖ×´Ï£¬ÕâÑù×ö·¨Ã÷ÏÔÊDz»Ðеġ£
ÓÐλÃûÈËÔøÖ¸³ö £¨Õæ˵¹ý£¬²»¹ý°ë¿ªÍæЦ£©£ºAll computer problems can be solved with one more layer of indirection. ΪÁ˽â¾ö¼«´ó¹æÄ£¼¯ÈºÊ¹ÊÔì³ÉµÄά»¤À§ÄÑÎÊÌ⣬SREÍÆÐÐÁËÎïÀí×ÊÔ´ÓëÂß¼­·ÖÀëµÄ×ö·¨¡£ÀûÓà Borg ϵͳ£¬´´ÔìÁËÒ»ÖÖеÄ×ÊÔ´µ¥Î»----ÈÝÆ÷¡£ ÎÒµ±Ê±Ëù´¦µÄBorg SRE ÍŶÓÆðµÄÕýÊÇÕâÑùÒ»¸ö³ÐÉÏÆôϵÄ×÷Óá£×ÜÓÐÈËÎÊÎÒ Google ÄÚ²¿ÊDz»ÊÇÒ²ÓÃÈÝÆ÷ÔÆ£¬´Ë´¦ÎÒÖ»ÄÜ˵£¬Google ÓõÄʱºò£¬ÕâÍæÒ⻹ľÓÐÃû×ÖÄØ¡£
ÏòÏ£¬Borg SRE ͨ¹ýÊéÖÐÃèÊöµÄ¡°ÏµÍ³¹ÜÀíÈí¼þ¡± £¨×Ô¶¯»¯ÏµÍ³£©×Ô¶¯»¯µÄ´¦ÀíËùÓÐÎïÀí×ÊÔ´µÄÒ쳣״̬£¬¸ù¾ÝÐèÒªÀûÓÃ×Ô¶¯»¯¹¤µ¥ÏµÍ³ÓëÊý¾ÝÖÐÐÄפ³¡·þÎñÈËÔ±¹µÍ¨½â¾öÎÊÌ⡣פ³¡ÈËÔ±¸ºÔðÖ´ÐÐÎïÀí²Ù×÷£¬SRE Ôò¸ºÔðÖƶ¨¼Æ»®¡£·Ö¹¤Ã÷È·ÓÖ¿ÉÒÔ¹²Í¬ÌáÉý¡£
ÏòÉÏ£¬Borg ÔòÌṩÁËÒ»Ì×ÐéÄâ×ÊÔ´µÄ³éÏó²ã£¬°üÀ¨ÏàÓ¦µÄ½»»¥Ä£ÐÍ£¬ÈÃÓû§¿ÉÒÔÖ±½ÓÒÔTask, Job µÈÔ­Óï¹¹½¨×Ô¼ºµÄÓ¦Óã¬ÎÞÐèÔÙ¹ØÐÄÎïÀí×ÊÔ´µÄ·ÖÅäÓëʹÓÃÎÊÌâ¡£
Borg SREÍŶӸºÔðÔËÐÐά»¤×Ô¶¯»¯ÏµÍ³£¬±£ÕÏÐéÄâ×ÊÔ´µÄ½»¸¶ÄÜÁ¦ (Scheduabity) ºÍ½»¸¶ÖÊÁ¿ (Scheduling Dealy)¡£Borg SRE ͬʱ¸ºÔðά»¤»ù´¡µÄÉú²úϵͳ»·¾³±ä¸ü£¬ÀýÈçÿÁ½ÖܸüÐÂÒ»´ÎÈ«²¿Éú²ú·þÎñÆ÷µÄKernel°æ±¾¡£ÔÚÕâ¸öÌåÁ¿Ï£¬ÈκÎÐèÒªÈ˹¤¸ÉÔ¤µÄ¹ý³Ì¶¼ÊDz»¿ÉÏëÏóµÄ¡£
´æ´¢×ÊÔ´
ÎÒ¾­³£Ëµ·Ö²¼Ê½ÏµÍ³ÖÐ×î´óµÄÎÊÌâ¾ÍÊÇ´æ´¢¡£Èç¹û½«½ø³ÌµÄ״̬´æÔÚÓÚij¸ö·Ö²¼Ê½´æ´¢ÏµÍ³ÖУ¬½ø³Ì±¾Éí¾Í¿ÉÒÔ×öµ½ÎÞ״̬£¬¿ÉÉìËõ£¬¿ÉǨÒÆ¡£
GoogleµÄ´æ´¢ÏµÍ³Ã÷ÏÔÊÇ·Ö²ãÐÎʽ¹¹½¨µÄ£¬´óÖ·ÖΪ »úÆ÷±¾µØSSD / £¨HD»ù±¾Öð½¥Òѱ»ÌÔÌ­£©£¬¼¯Èº±¾µØµÄ Chubby, Bigtable, Colossus µÈ£¬ÒÔ¼°¿ç¼¯Èº×Ô¶¯Í¬²½µÄ Megastore, Spanner µÈµÈ¡£ ÕâÑù£¬Ñз¢¿ÉÒÔ¸ù¾ÝÐèҪѡÔñ×î·ûºÏµ±Ç°Ê¹ÓÃÐèÇóµÄ³¡¾°µÄ´æ´¢·þÎñ¡£
ÕâÀïÖµµÃÒ»ÌáµÄÊÇ£¬Google´ó²¿·Öϵͳ¶¼ÊÇ»ùÓÚ¡°¼¯Èº±¾µØ¡± (Cluster local) µÄ¸ÅÄî¹¹½¨µÄ ¡£Õû¸ö¼¯ÈºÄÚ²¿Í¨³£¿ÉÒÔ±»ÊÓΪ¡°¾ÖÓòÍø¡±£¬ÆäÖÐËùÓлúÆ÷¹²ÏíÒ»¸ö´óµÄÍøÂçÔÖÄÑÓò£¬ÒÔ¼°Êý¸öСµÄ¹©µçÔÖÄÑÓò¡£Google¼¯ÈºËä¶à£¬µ«ÊÇÿ¸ö¼¯ÈºµÄ½á¹¹¶¼¸ß¶ÈÒ»Ö¡£¿ÉÒÔÕæÕý×öµ½ÎÞ²î±ðǨÒÆ¡£SRE¾­³£Î¨Ò»ÐèÒª¼ÇסµÄ¾ÍÊǼ¯ÈºµÄÃüÃû¹æÔò£¨ÎïÀíλÖã©¡£ÔÚ¼¯Èº¼äǨÒƱäµÃÈç´Ë¼òµ¥£¬ÉõÖÁÓеÄÍŶÓÿ¸ö¼¾¶È¶¼È«Çò²¼¾ÖÒ»´Î¡£Èç¹ûµ×²ãϵͳ²»½«ÕâÀàÐÅÏ¢¹«²¼¸øʹÓÃÕߣ¬¶øÊÇ×ÔÏÂÏòÉÏÌæÓû§¾ö²ß£¬½«Ôì³É¼«´óµÄÀË·Ñ¡£ ËùÒÔÓÐÈËÎÊÎÒ£¬GoogleÔÆƽ̨ÊÇ·ñ³¬Âô£¬´ð°¸ÊÇ·ñ¡£Èç¹ûÐéÄâ×ÊÔ´³¬Âô£¬Ö±½Óµ¼ÖÂÓû§ÐÔÄÜûÓб£ÕÏ£¬Ò²¾ÍʹµÃÓû§±ØÐëÂò¸ü¶àµÄ×ÊÔ´£¬ÕâÖмäµÄÀË·Ñ£¬Ë­À´Âòµ¥ÄØ£¿Ôø¾­ÓÐÈË˵¹ý£¬Èç¹û¹«Ë¾ÄÚ²¿¼¤Àø»úÖƳöÁËÎÊÌ⣬ÊýÄ¿¾ªÈ˵ÄÀË·Ñ»¹Õæ²»ÈçÖ±½ÓÉÕÇ®£¬Ã¿Ì켸°ÙÍò£¬»¹ÄÜ¿¾¿¾»ðÄØ£¡
ÍøÂç×ÊÔ´
Google ÍøÂçÉè¼ÆÖзÖÊý¾ÝÖÐÐÄÍøÂ磬B4 (Backend backbone), B2 (Backbone) ÈýÀà¡£ÆäÖм¯ÈºÍøÂç´ó¼Ò¿ÉÒÔÔĶÁ Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google¡¯s Datacenter Network Õâƪ paper. Êý¾ÝÖÐÐÄÄÚ²¿È«Èý²ãÍøÂ磬˫»úÖ®¼ä5΢ÃëÑÓ³Ù£¬Ã¿Ì¨»úÆ÷40GÉÏÏÂÐдø¿í£¬ÕæµÄºÜÄÑÏëÏó¡£
2008ÄêµÄ½ð¶îΣ»úÆڼ䣬ÃÀ¹ú¡¢Å·Ö޺ܶàÖÐСÐÍISPÊÜËðµ¹±Õ¡£Google×÷Ϊµ±Ê±µÄÏÖ½ð´ó»§ÊÕ¹ºÁ˳¬¼¶¶àµÄDark fiber£¬ËæºóÕâЩ¾ÍGoogle¹Ç¸ÉÍøÂçµÄºËÐÄ¡£ÎÒÔÚ Youtube Æڼ䣬ÓÖ²ÎÓë¹¹½¨ÁËGoogleµÄCDNÍøÂ磬ÔÚÕâÀïÎҾͲ»¶àдÁË£¬ÓÐÐËȤ¿ÉÒÔµ¥¶À½»Á÷¡£
ÀíÂÛƪ
ÈçͬǰÎÄËù½²£¬ ÎÒÈÏΪSREµÄ³É¹¦»ù´¡ÔÚÓÚÖ°Ôð¡¢¾ö²ßȨºÍ×ÔÖÎȨµÄͳһ¡£ÕâһƪµÄÄÚÈݾͳä·ÖÌåÏÖÁËÕâÈý·½ÃæµÄÄÚÈÝ¡£
Ê×ÏÈ£¬SREÍŶÓÔË×÷¹ý³ÌµÄ¹Ø¼ü¾ÍÔÚÓÚÖƶ¨ºÍά»¤Ò»Ì׿ÉÒԸ߶ÈÁ¿»¯µÄÄ¿±ê£¬¼´·þÎñÖÊÁ¿Ö¸±ê£¨SLO£©¡£ÓÐÁËÕâÑùÒ»¸öÄ¿±ê£¬Í¨¹ý²»¶ÏµØÖƶ¨ºÍÍêÉÆÖ´Ðз½°¸£¬¶ÔÖ´Ðйý³ÌÖÐÓöµ½µÄÎÊÌⲻͣµØ×·¸ùËÝÔ´£¬ÐγÉÁËSRE¶ÀÌصĶø¸ßЧµÄ¹¤×÷·½Ê½¡£ ÊéÖеÚ3Õ£¬·çÏÕµÄÁ¿»¯¶¨Ò壬ÊÇSREµÄÁ¢ÉíÖ®±¾¡£Îȶ¨ÐԺͿɿ¿ÐÔÓ¦¸Ã×÷Ϊ²úÆ·µÄÒ»¸öÖØÒªÊôÐÔ½øÐж¨Òå¡£Ìṩ¶¨ÒåÒÀ¾Ý£¬Öƶ¨ºâÁ¿±ê×¼£¬±£Ö¤´ï±êÔòÊÇSREÖ°ÔðËùÔÚ¡£
SRE Ó¦¸ÃÖ±½ÓÏò×îÖÕÓû§¸ºÔð£¬SRE ¹¤×÷ÖÊÁ¿µÄºâÁ¿±ê×¼¾ÍÊÇ·þÎñµÄÖÊÁ¿¸ßµÍ¡£ÎÒÔÚ¹úÄÚij´Î½»Á÷ÖУ¬Ôø¾­ÓëÈËÁĹýGoogle´óÁ¿²ÉÓÃ×ÔÏÂÏòÉ쵀 OKR ¹ÜÀí·½Ê½¡£µ«ÊÇÕâÍùÍùÖ»ÊÇÒ»¸ö²àÃæÓ¡Ïó£¬Æäʵ Google ÄÚ²¿µ±È»Ò²ÓÐ ×ÔÉÏÖÁϵĹÜÀí·½Ê½£¬SLO ¾Í¿ÉÒÔËã×÷ Google SRE µÄ KPI¡£ ÕâÀïµÄÏêÇé¿ÉÔĶÁÊéÖеÚ4Õ£¬·þÎñÖÊÁ¿Ö¸±ê£¬¾Í²»ÔÙ׸ÊöÁË¡£
È·Á¢ÁËÄ¿±êÖ®ºó£¬ÓйØÈçºÎÌá¸ß»òÕß±£³Ö¸ÃÄ¿±ê¡£¿ÉÒÔÔĶÁÊéÖеÚ6Õ£¬·Ö²¼Ê½ÏµÍ³µÄ¼à¿ØÀ´ÏêϸÁ˽⡣±¾ÕÂÀïÃæÌáµ½µÄËÄ´ó»Æ½ðÖ¸±êÊÇ»ù´¡ÖеĻù´¡¡£µ«ÊǸüÈÃÈËÄÔ¶´´ó¿ªµÄÒ»µã»¹ÊǼ¯ÖÐÓÚ¡°³¤Î²¡±ÎÊÌâµÄÌÖÂÛÉÏ¡£¡±³¤Î²¡±ÄÑÌ⼸ºõÊÇ´óÐÍ·Ö²¼Ê½ÏµÍ³ÔÚ½â¾öÍêÉú´æÎÊÌâÖ®ºóÓöµ½µÄ±êÅäÎÊÌâ¡£ÓÉÓÚ³¤Î²ÎÊÌ⸴ÏÖÀ§ÄÑ£¬¶ÔÊý¾Ý·ÖÎöÄÜÁ¦£¬ÏµÍ³µ÷ÊÔÓë²âÁ¿ÄÜÁ¦ÒªÇóºÜ¸ß£¬ÄÑÒÔ¸ùÖΡ£
³¤Î²ÎÊÌ⻹ÓзŴóЧӦ£¬µ×²ã·þÎñµÄ³¤Î²ÎÊÌâ»á¸øµ÷ÓÃƵ·±µÄÉϲãÓ¦ÓôøÀ´Êý±¶ÒÔÉϵÄÐÔÄÜÓ°Ïì¡£Ôø¾­ÎÒÓÐÒ»¸ö Google ͬÊ£¬ÈÎÖ°ÆÚ¼äËù×öµÄΨһ¹¤×÷¾ÍÊǸú×Ùµ÷²éij´æ´¢ÏµÍ³´ÅÅÌIOµÄ³¤Î²ÄÑÌ⣬´ÓÓ¦ÓÃһ·²éµ½Äںˡ£×îºóÔÚÄÚºËÖз¢ÏÖ²¢½â¾öÁËÒ»¸öËø¾ºÕùÎÊÌâ¡£ÓÉÓÚËûµÄÕ⼸ÐиĶ¯£¬µ×²ã´æ´¢·þÎñµÄ³¤Î²ÑÓ³Ù½µµÍÁË£¬Éϲã´óÁ¿ÒÀÀµ²Ù×÷´æ´¢µÄÓ¦Ó÷þÎñ±ã¿ÉÒÔÊ¡È¥ºÜ¶à³¬¶î²¿ÊðµÄÈÝÁ¿£¬Õ⼸ÐдúÂëΪ Google ½ÚÊ¡ÁËÿÄ꼸ǧÍòÃÀ½ðµÄ×ÊÔ´³É±¾¡£ËûЦ³Æ£¬×Ô¼ºÓ¦¸Ã¿ÉÒÔÃâ·ÑÄÃÊ®Ä깤×ÊÁË¡££¨µ±È»ÁË£¬¹À¼Æû·¢£¬ËùÒÔËû×ßÁË¡££©
±¾´óÕÂÖеÄÆäÓàÕ½ڣ¬·Ö±ðÃèÊöÁËÔÚSRE¹¤×÷¹ý³ÌÖÐËùÉæ¼°µ½µÄ·½·½ÃæÃæ¡£ÆäÖжÔËöÊ (Toil) µÄ´¦ÀíºÜÓÐÒâ˼¡£Google SRE ÿ¸ö¼¾¶È¶¼Ôø¾­·¢ËÍÄÚ²¿µ÷²éÎʾíÊÕ¼¯¹ØÓÚËöÊÂËùÕ¼±ÈÀýµÄÐÅÏ¢¡£µ«ÊǹÀ¼Æijλ Manager ´ßÊÖÏÂÌîÎʾí´ßµÄ¼±ÁËһЩ£¬Õâλ´óÏÀһŭ֮ÏÂÏòÈ«Ìå SRE ·¢ËÍÁË¡±¹ØÓÚÿ¼¾¶ÈËöʵ÷²éÎÊ¾í·±Ëö³Ì¶ÈµÄµ÷²éÎÊ¾í¡°£¬Ð¦ÅçÁË¡£
µÚ9Õ£¬¼òµ¥»¯£¬ÕâÒ»ÕÂÖеÄÄÚÈÝÒ²ºÜÓÐÒâ˼¡£ÔÚ Google ÄÚ²¿ÎÒÔø¾­²ÎÓë×éÖ¯¹ýºÜ¶à´Î ¡± ɾ´úÂ뾺Èü¡° »î¶¯£¬Ò»É¾¼¸ÍòÐС£ºóÀ´ÔÚ Coding Ò²Ôø¾­¾Ù°ì£¬Ð§¹ûºÜºÃ¡£ ³ÌÐòÔ±Ò»¶¨Òª¼ÇµÃ£¬´úÂë¼´Õ®Îñ¡£Èç¹ûÎÞ·¨°²È«µÄɾµôÀ¬»ø´úÂ룬²»½ö½öÊDZàÒëËٶȵÄÎÊÌ⣬ÕâÇ¡Ç¡·´Ó³ÁËÉî²ã´ÎµÄÑз¢¼Ü¹¹ÎÊÌâ¡£Google ¹¤³Ìʦ¶¼ÒÔÉÙд´úÂëΪÈÙ£¬ÎÒÔø¾­ÓÐÁ¬Ðø¼¸¸öÔ¶¼ÊÇ´úÂëÐÐÊý¾»Êä³öΪ¸º£¬Èç¹û°´ÎåÐÐÒ»ÔªÆð·¢¹¤×Ê£¬ÄÇÆñ²»ÊÇÒª¿ÞËÀÁË¡£
Óйؼò»¯ÈÏÖª¸´ÔӶȵÄÎÊÌ⣬ÊÇÒ»¸öÖµµÃÉî˼µÄ»°Ìâ¡£Ò»¸öÈ˵ÄÈÏÖªÄÜÁ¦×ÜÊÇÓÐÏ޵ģ¬ÔÚ Google ÕâÑùµÄÌåÁ¿Ï£¬±ØÐëÒª½øÐÐͳһ»¯£¬¼òµ¥»¯£¬·ñÔòË­Ò²²»¿ÉÄܼÇס¶Ëµ½¶ËµÄËùÓÐÐÅÏ¢¡£ÕýÈç Borg ͨ¹ý³éÏó×ÊÔ´½µµÍÁË´ó²¿·Ö Google ³ÌÐòÔ±¶ÔÎïÀí×ÊԴʹÓõÄÈÏÖª¸´ÔÓ¶ÈÒ»Ñù£¬´ó²¿·Ö Google ×Ô¶¯»¯ÏµÍ³¶¼ÔÚÓÉ Ö¸ÁîÐÍϵͳÏòÒâÔ¸ÐÍϵͳת±ä£¬ÕâÒ»µãÓÐÐËȤµÄ»°£¬¿ÉÒÔÏßϽ»Á÷¡£
ʵ¼ùƪ
ÕâÒ»²¿·ÖÄÚÈݽ϶࣬ÎÒ½«ÊéÖÐÕ½ڰ´ÕÕ×Ô¼ºµÄÀí½âÖØÐÂÅÅÐòÁË, ·½±ã´ó¼ÒÔĶÁºÍÀí½â¡£
Ç°ÎÄ˵¹ý Google SRE ¶¼ÊÇ×ÔºÀµÄ±³¹øÏÀ£¬ÄÇôÈçºÎרҵµÄ¶ø¸ºÔðµÄ±³¹øÄØ£¿ÕâÀï¾ÍÓ¦¸Ã¿´ÒÔÏÂÕ½ÚÁË£ºµÚ11Õ Oncall £¬ µÚ12Õ ¹ÊÕÏÅŲ鷽ʽ£¬ µÚ13Õ ½ô¼±Ê¼þÏìÓ¦ µÚ14ՠʹÊÁ÷³Ì¹ÜÀí £¬µÚ15ՠʺó×ܽᣬ ÒÔ¼°µÚ16Õ ¹ÊÕϸú×Ù¹¤¾ß¡£ ÎÒÔø¾­¾Ù¹ýºÃ¶àÀý×Ó£¬SRE µÄ Oncall ÊÇÒ»¸öÈ˸ºÔðÕû¸öÍŶӵĽô¼±ÊÂÎñ´¦Àí£¬¾ø·Ç¸÷¹Ü¸÷µÄ¡£ ÕâÑù£¬·ÇOncallµÄÈË¿ÉÒÔÕæÕýͶÈëµ½¿ª·¢ÏîÄ¿ÖÐÈ¥¡£ÎÒ³£³£¿ªÍæЦµÄ˵£¬Ê²Ã´Ê±ºòÔËάÍŶӳÉÔ±¿ÉÒÔ×öµ½Äܹ»Ëæʱ°´ÐèÇë¼ÙÁË£¬Ï°à¹ØÊÖ»ú£¬ÄǾÍ˵Ã÷ÍŶÓ×ßÉÏÕý¹ìÁË¡£Ò»¸öµ¥¶ÀµÄ SRE ÍŶÓÊÇÒ»¸ö¸ß¶ÈЭ×÷µÄÍŶӣ¬Ã¿¸öÈËÓÐ×Ô¼ºµÄÊÂҵרעµã£¬Ò²Óй²Í¬Î¬»¤µÄһĶÈý·ÖµØ¡£Æ½Ê±µÄ¼à¿Øƽ̨ά»¤£¬ÎĵµÐ޸ģ¬±¨¾¯¹æÔòµÈµÈ£¬¶¼»á¼°Ê±Óë´ó¼Òͬ²½¡£
ÁíÍâÒ»¸öÖµµÃ¹Ø×¢µÄÖصãÊǸºÔؾùºâÎÊÌ⣬Ö÷Òª²Î¿´ÒÔϼ¸Õ£º µÚ19Õ ÃèÊöÁËÊý¾ÝÖÐÐÄÖ®¼äµÄ¸ºÔؾùºâϵͳµÄÉè¼ÆʵÏÖ£¬µÚ20Õ ÃèÊöÁËÊý¾ÝÖÐÐÄÄÚ²¿µÄ¸ºÔؾùºâϵͳÉè¼ÆʵÏÖ¡£µÚ21ÕÂÌÖÂÛ¹ýÔØÎÊÌ⣬µÚ22ÕÂÌÖÂÛÁËÑ©±ÀЧӦºÍÁ¬Ëø¹ÊÕÏÎÊÌâ¡£ÎÒÔø¾­ºÍºÜ¶àÈËÌÖÂÛ¹ý£¬Google µÄÊý¾ÝÖÐÐÄÉè¼ÆSLOΪ99%¡£ ¡±ÈßÓà¡° »ù±¾ÉÏÊǸ÷ÖÖ·þÎñÉÏÏßµÚÒ»Ìì¾ÍËù±Ø±¸µÄÒªÇó¡£ÈßÓàÓÖ·Ö¡±µ¥»î¡°£¬¡±¶à»î¡°µÈµÈ£¬ÔÚ´æ´¢²ãÃæµÄ¡±¶à»î¡°¹¦ÄÜÖ§³ÖÏ£¬Google ´ó²¿·Öϵͳ¶¼¿ÉÒÔ×öµ½ËæÒâǨÒÆ£¬¿çÇøÓò¶à»î£¬ÕâÖмäÖ÷Òª¾Í¿¿¸ºÔؾùºâÌåϵµÄÍêÉÆ¡£Ò»¸öÓû§ÇëÇó´Ó·¢Æðµ½µÖ´ï Google Éú²ú·þÎñÆ÷£¬ÖмäÒª¾­¹ýÖÁÉÙÈý²ã¸ºÔؾùºâϵͳ£¬´¦Àí¹ý³ÌÖиüÊÇ´¥·¢N¸öRPC ÔÚÐí¶à»úÆ÷Éϲ¢·¢Ö´ÐС£ÕâÖмäÓÖÀë²»¿ªÊý¾ÝÖÐÐÄÄÚ²¿¸ºÔؾùºâϵͳµÄ°ïÖú¡£ÓÐÈËÎÊ N + M ÈßÓàÈçºÎ×öµ½£¬ÎÒ×ÜÊÇЦ×ÅÏÈÎÊËû N ÊǶàÉÙ¡£ ²»Ñ¹Á¦²âÊÔ£¬¸ù±¾Ã»°ì·¨Ì¸Ê²Ã´ÈßÓà¡£ºÜ¶àʱºò£¬ÎªÁ˸üºÃµÄ£¬¸ü¾«È·µÄÖƶ¨N£¬ÎÒÃÇ»¹Òª²ð·Ö·þÎñ£¬ÖØ×é¼Ü¹¹£¬ÕâЩ¶¼ÊÇ SRE µÄ¹Ø×¢·¶Î§¡£
¹ýÔØÓëÑ©±ÀЧӦÕâÁ½ÕÂдµÄ·Ç³£Ö®ºÃ£¬ÕâºÍÏÂÎÄ»áÌáµ½µÄ·Ö²¼Ê½¹²Ê¶ÏµÍ³Ò»Í¬³É±¾ÊéÎÒ×îϲ»¶µÄÇ°ÈýÕ¡£Ãæ¶Ô×ÅÆÌÌì¸ÇµØµÄÓû§Á÷Á¿£¬¸ºÔؾùºâϵͳ£¬·þÎñÆ÷×é¼þ£¬ÉõÖÁÖØÊÔÂß¼­µÈµÈÉè¼ÆÉÔÓв»É÷¾Í»á´¥·¢±ÈÈ«Ìì·åÖµ»¹Òª¸ß¼¸±¶µÄÁ÷Á¿¹¥»÷×Ô¼º£¬ÊµÔÚÊDz»ÄÜ´óÒâ°¡¡£Ò»µ©³öÏÖÑ©±ÀЧӦ£¬ºÜÄѻָ´Õý³££¬³£³£ÐèҪȫ·þÎñÍ£»úÒ»¶Îʱ¼ä²ÅÐС£App Engine ¾ÍÔø¾­³öÏÖ¹ýÕâÑùµÄ´óÎÊÌ⡣ijÖÖÒòËص¼ÖµÄÊý¾ÝÖÐÐÄÏÂÏߣ¬µ¼ÖÂÈ«ÇòÁ÷Á¿¾ÍÏñºéˮһ°ãÒ»¸öÒ»¸ö½«²ÐÁôµÄÊý¾ÝÖÐÐÄÑÍûÁË¡£
µÚ23Õ ·Ö²¼Ê½¹²Ê¶ÏµÍ³£¬µÚ24Õ ·Ö²¼Ê½ÖÜÆÚÐÔÈÎÎñϵͳ µÚ25Õ Êý¾Ý´¦ÀíÁ÷Ë®Ïß ¸üÊǾø¼ÑÖ®×÷¡£ÔÚÎÒ²ÎÓëÉè¼ÆʵÏÖYoutubeÖ±²¥ÏµÍ³Ö®Ê±£¬Ôø¾­»¨·Ñ´óÁ¿¾«Á¦²ÎÓëÉè¼ÆÁËÒ»Ì×»ùÓÚ BASE ´æ´¢ÏµÍ³µÄÒ»ÖÂÐÔ½â¾ö·½°¸¡£ÔÚ½áÊøÁ½ÖÜÌÖÂÛÖ®ºó£¬»Ø³Ì·É»úµÄ·ÉÏ£¬ÎÒ·­¿´ÄÚ²¿Îĵµ¿´µ½Á˶ÔPaxosϵͳµÄÌÖÂÛ£¬µ±Ê±ÕæÈçõ®õ­¹à¶¥£¬Ä¿µÉ¿Ú´ô¡£Ç°ÎÄÌÖÂÛ¹ý£¬·Ö²¼Ê½ÏµÍ³×îÄѵÄÉè¼ÆÄѵã¾ÍÔÚÓÚ״̬´æÈ¡Ò»ÖÂÐÔÎÊÌ⣬¶øÓÐÁËPaxosÖ®ºó£¬Ò»Çж¼²»ÔÙÊÇÎÊÌâ¡£»ØÏëÆðÀ´£¬¿´Ñù»¹ÊÇÓ¦¸ÃºÃºÃÉÏѧ£¬80Äê´ú¾Í·¢±íÁ赀 Paxos ϵͳÉè¼Æ¾ÓȻֱµ½2010Äê²Åѧµ½£¬¶ÁÊéÉÙ×ܱ»Æ­°¡£¡
ÎÒÇ¿ÁÒ½¨Òé´ó¼ÒÕæµÄ×Ðϸ¶ÁÕâÒ»Õ£¬Ä¿Ç°¿ªÔ´ÊµÏÖÖÐZooKeeper, Etcd, ConsulµÈ»ùÓÚµÄÒ²¶¼ÊÇPaxosЭÒéµÄ±äÖÖ£¬ÏàÐŶÁ¹ýÕâƪ»á¶Ô´ó¼ÒδÀ´µÄ·Ö²¼Ê½Ó¦ÓÃÉè¼ÆÓÐÖ±½ÓµÄÆô·¢×÷Óá£ÕâÖ®ºóµÄ·Ö²¼Ê½ÖÜÆÚϵͳ£¬Êý¾Ý´¦ÀíÁ÷Ë®Ïßϵͳ£¬»ù±¾µÄˮƽÀ©Õ¹·½Ê½¶¼ÒÀÀµ Paxos £¬ÀíÂÛÖ¸µ¼¼Ó¾ßÌåʵ¼ù£¬¸Ð¾õÈ«Êé¾Í¿´ÕâÈýÕ£¬×ãÒÓ¡£
ÆäËûÕ½ÚÖÐÀýÈçµÚ26Õ Êý¾ÝÍêÕûÐÔ£¬ÊDZȽÏÄѿеÄÒ»Õ¡£Google ´æ´¢µÄº£Á¿Êý¾Ý¹ÜÀíËùÓöµ½µÄÀ§ÄÑÊýÁ¿Ò²¾ø¶ÔÊǺ£Á¿µÄ¡£Gmail µ±Äê³öÊÂÖ®ºó£¬¿¿´Å´ø±¸·Ýϵͳ³É¹¦»Ö¸´ÁËÓû§Êý¾Ý£¬ÓÐÐËȤµÄÈË¿ÉÒÔËÑË÷Ïà¹ØµÄ Youtube ÊÓƵÓÐÏêϸ½éÉÜ¡£ÊÓƵÖÐÔø˵¹ýÒ»¾ä»°£¬±¸·Ýϵͳ²»ÖØÒª£¬ÖØÒªµÄÊǻָ´ÏµÍ³¡£Èç¹ûÄãÖ»ÊÇ´îÁËÒ»Ì×±¸·Ýϵͳ£¬Ã»ÓÐƽʱ¾­³£ÑÝÏ°Êý¾Ý»Ö¸´£¬ÄÇô¹Ø¼üʱ¿ÌËüÒ»¶¨»áʧЧµÄ¡£Î¨Ò»ÄÜÈ·±£°²È«µÄ·½·¨£¬¾ÍÊǽ¨Á¢×Ô¶¯»¯µÄ£¬Ëæ»úµÄÊý¾Ý»Ö¸´»úÖÆ¡£
±¾²¿·ÖÆäËûÕ½ÚÔÚÕâÀï¾Í²»ÔÙ׸ÊöÁË£¬²âÊÔÕâÒ»ÕÂÒ²±È½ÏÄѿУ¬ÈçÓÐÐËȤ´ó¼Ò¿ÉÒÔÖ±½ÓÕÒÎÒ˽Ͻ»Á÷¡£
SRE¹ÜÀí
Ç°ÎÄÌáµ½£¬SRE µÄ×ÔÖÎÖ÷ÒªÌåÏÖÔÚ 50% µÄÑз¢¹¤×÷ºìÏßÉÏ¡£ ±¾²¿·ÖÖÐ×îÓÐÆô·¢ÐÔµÄÌÖÂÛ£¬Äª¹ýÓÚ µÚ29Õ£¬¶ÔÖжÏÐÔÈÎÎñµÄ´¦Àí¡£ ÎÒÃÇÏÖÔÚ¶¼Éú»îÔÚÒ»¸öÐÅÏ¢±¬·¢µÄÄê´ú£¬Ã¿Ìì¸÷ÖÖ·½Ê½µÄ֪ͨÏûÏ¢²»Í£µØ´ò¶ÏÎÒÃǵÄ˼·£¬ÉõÖÁºÜ¶àÈ˸ù±¾ÒѾ­ÎÞ·¨³¤Ê±¼ä¼¯ÖÐ×¢ÒâÁ¦¡£×Ô´ÓѧϰÁËÓÐ¹Ø Á÷״̬ £¨Flow state) µÄÏà¹Ø֪ʶ֮ºó£¬ÎÒ³£³£×ÔÊ¡£¬½ñÌìÓÖÔÚ»Ø΢ÐÅ£¬²éÓʼþ£¬ÒÔ¼°Ë¢ÅóÓÑȦÖжȹýÁË£¬ÏîĿľÓнøÕ¹°¡£¬³¤´ËÒÔÍùÇéºÎÒÔ¿°¡£
ÕâµÄÈ·ÊÇÖµµÃÎÒÃÇÉî˼µÄÎÊÌ⣬ÔËάÍŶÓÓÉÓÚ¸ºµ£ÈÕ³£ÔËά¹¤×÷£¬ÖжÏÐÍÈÎÎñ²»¿É±ÜÃ⣬Èç¹û³¤Ê±¼äÈÃ×Ô¼º´¦ÓÚÕâ¸ö״̬ÖлáÓÐÐí¶à²»Á¼ÏÖÏó¡£ËùÒÔ£¬ÎªÁË×Ô¼ºµÄÉíÐĽ¡¿µ£¬Ò²Ó¦¸Ã¶à¶à¹Ø×¢±¾Õ¡£
±¾²¿·ÖÆäËûÕ½ÚÏêϸÃèÊöÁËSREÓëÑз¢ÍŶӣ¬²úÆ·ÍŶӵĺÏ×÷¹Øϵ¡£ÖÐÃÀ×éÖ¯½á¹¹£¬ÆóÒµÎÄ»¯£¬ÍŶӹØϵ»¹ÊÇÂÔÓÐÇø±ð£¬¼Æ»®¾­¼ÃÓëÊг¡¾­¼ÃµÄ¶Ô±È£¬´ÓÕ⼸ÕÂÖпÉÒÔ¿ú¼ûÒ»°ß¡£
ÓëÆäËûÐÐÒµµÄ¶Ô±È
Õⲿ·ÖËäÈ»°²ÅÅÔÚ×îºó£¬µ«ÊÇÎÒÈÏΪÊǷdz£ÖµµÃÒ»¶ÁµÄ¡£°éËæ¶øÀ´µÄÎÊÌâÒ²ÈÃÈËÄÔ¶´´ó¿ª£¬±ÈÈçSRE/ÔËάÍŶÓÓëÃñº½´ó·É»ú¼ÝʻԱµÄ¹²Í¬µãÓëÇø±ðÔÚÄÄ£¿ÓëÏû·À¶ÓÔ±£¬¾ÈÉúÔ±Ïà±ÈÄØ£¿ÕâЩÐÐÒµ¶¼Ô¶±ÈÈí¼þÐÐÒµ³ÉÊ죬ËûÃÇÖ°ÒµµÄ·¢Õ¹Â·ÏßÊDz»ÊǶÔSREµÄÖ°Òµ·¢Õ¹·½ÏòÓвο¼¼ÛÖµÄØ£¿ÏàÐÅ¿´Íê±¾Õ£¬Äã»áÕÒµ½×Ô¼ºµÄ´ð°¸µÄ¡£
½áÊøÓï
¸ÐлÄÍÐĶÁÕß¿´µ½×îºó£¬Ï£Íû±¾ÎÄÄܸøÄú´øÀ´Ò»Ð©ÓÐÓõÄÐÅÏ¢£¬¡¶SRE£ºGoogle ÔËά½âÃÜ¡·Ä¿Ç°¸÷´óÍøµê¾ùÓÐÏúÊÛ£¬ÆÚ´ýÓë´ó¼ÒµÄ½»Á÷»î¶¯£¡
Ô­ÎÄÀ´×Ô£ºhttp://www.linuxprobe.com/google-sre.html
​Ãâ·ÑÌṩ×îÐÂLinux¼¼Êõ½Ì³ÌÊé¼®£¬Îª¿ªÔ´¼¼Êõ°®ºÃÕßŬÁ¦×öµÃ¸ü¶à¸üºÃ£¬¿ªÔ´Õ¾µã£ºhttp://www.linuxprobe.com/
ÎÄÕÂÆÀÂÛ

¹²ÓÐ 1 ÌõÆÀÂÛ

  1. Ronny ÓÚ 2018-01-19 21:06:07·¢±í:

    ¡¶SRE£ºGoogleÔËά½âÃÜ¡·