•  瑾茬▼鐩

  鏈绋嬮鍏堜粙绱逛簡Spark鐢熸厠,Spark鍩虹锛涙繁鍏ヨ瑳浜嗗垎甯冨紡闆嗙兢鐨勬惌寤猴紝Spark绶ㄧ▼瀵︽埌銆傛渶鍚嶴park鏁告摎鍒嗘瀽鍜屾暩鎿氭寲鎺楳Lib,缍滃悎闋呯洰瀵︽埌銆�

•  甯硣鍦橀殜

• 鑿竻鍓靛浼佹キ鍏ц〒璎涘斧锛屽潎鏄締鑷悇鍊嬮牁鍩熺殑璩囨繁灏堝锛屽潎鎿佹湁6骞翠互涓婂ぇ鍨嬮爡鐩稉椹椼€�

•  鍩归灏嶈薄

  瀛稿摗瀛哥繏鏈绋嬫噳鍏峰倷涓嬪垪鍩虹鐭ヨ瓨锛�
  鈼� 鍏峰倷Python瑾炶█鐨勫熀绀庯紱
  鈼� 灏嶅ぇ鏁告摎Spark闁嬬櫦鎰熻垐瓒ｇ殑瀛稿摗锛�

•  鍩硅〒鏂瑰紡

绗竴绋細鑿竻鍓靛璎涘斧闈㈡巿
瑾叉檪锛氬叡3澶╋紝姣忓ぉ6瀛告檪锛岀附瑷�18瀛告檪
鈼嗚不鐢紙鍚暀鏉愯不锛夛細3600鍏�
鈼嗗鍦板鍝★細浠ｇ悊瀹夋帓椋熷锛堥渶鎻愬墠闋愬畾锛�

绗簩绋細绶氫笂鐩存挱鎺堣
鐩存挱瑾叉檪锛氬叡6澶╋紝姣忓ぉ3瀛告檪锛岀附瑷�18瀛告檪锛�
杓斿皫锛氭巿瑾叉湡闁擄紝杓斿皫鑰佸斧姣忓ぉ鏈�1灏忔檪鐨勮紨灏庣洿鎾�
鈼嗚不鐢紙鍚暀鏉愯不锛夛細3600鍏�

绗笁绋細浼佹キ瑷傚埗鍩硅〒
瑾叉檪锛氭牴鎿氬畾鍒剁殑澶х侗纰哄畾瑾叉檪
璨荤敤锛氭牴鎿氳绋嬮洠搴︼紝姣忚鏅�1500~3000鍏�

• •  璩噺淇濊瓑

    1銆佸煿瑷撻亷绋嬩腑锛屽鏈夐儴鍒嗗収瀹圭悊瑙ｄ笉閫忔垨娑堝寲涓嶅ソ锛屽彲鍏嶈不鍦ㄤ笅鏈熷煿瑷撶彮涓噸鑱斤紱

    2銆佸煿瑷撶祼鏉熷悗鍏嶈不鎻愪緵涓€鍊嬫湀鐨勬妧琛撴敮鎸侊紝鍏呭垎淇濊瓑鍩硅〒鍚庡嚭鏁堟灉锛�

    3銆佸煿瑷撳悎鏍煎鍝″彲浜彈鍏嶈不鎺ㄨ枽灏辨キ姗熸渻銆�

  •  瑾茬▼澶х侗

    
    

    Spark鍏ラ杸鍙婄敓鎱嬮珨绯� 姒傝堪

    Spark鐢熸厠

    Spark锛堝収瀛樿▓绠楁鏋讹級

    SparkSteaming锛堟祦寮忚▓绠楁鏋讹級

    Spark SQL锛�ad-hoc锛�

    Mllib锛�Machine Learning锛�

    GraphX锛�bagel灏囪鍙栦唬锛�

    褰堟€у垎甯冨紡鏁告摎闆嗭紙RDD锛�

    
    

    Python Spark鍩虹浠嬬垂 Spark 绶ㄧ▼妯″瀷

    RDD绶╁瓨绛栫暐

    Spark Python绶ㄧ▼鍏ラ杸

    PySpark

    鎯版€ц▓绠楋紙Lazy Evaluation锛�

    娴佹按绶氾紙Pipelines锛�

    
    

    鍒嗗竷寮忛泦缇ゆ惌寤� Spark銆�Hadoop銆�VMware Ubuntu鍒嗗竷寮忛泦缇ゆ惌寤哄叏閬庣▼

    Ubuntu鍩烘湰鐠板閰嶇疆

    闆嗙兢瀹夎婧栧倷

    瀹夎閰嶇疆Hadoop

    瀹夎閰嶇疆Spark

    
    

    鍩轰簬Python鐨�Spark绶ㄧ▼瀵︽埌 姒傝堪

    閫ｆ帴Spark

    鍒濆鍖�Spark

    浣跨敤鍛戒护琛�

    褰堟€у垎甯冨紡鏁告摎闆嗭紙RDD锛�

    RDD鎿嶄綔

    RDD鎸佷箙鍖�

    鍦ㄩ泦缇や笂閮ㄧ讲

    鐢�Python绶ㄥ鐨勪竴鍊嬬啊鍠�Spark鎳夌敤

    
    

    Spark MLlib鐨勪娇鐢� 姗熷櫒瀛哥繏姒傚康

    Spark MLlib浠嬬垂

    Spark MLlib鏋舵瑙ｆ瀽

    MLlib鐨勭畻娉曞韩鍒嗘瀽

    鐢�Spark Python妲嬪缓鍒嗛妯″瀷

    浣跨敤 Spark MLlib 鍋� K-means 鑱氶鍒嗘瀽

    
    

    Spark妗堜緥瀵︽埌 "1. 浜掕伅缍查噾铻嶅弽娆鸿⿶妗堜緥鍒嗘瀽

    2. Spark RDD绶ㄧ▼鎶€宸�

    3. 鐢ㄦ埗鏁告摎鐗瑰緛杞夋彌瑙ｆ瀽鍙�Spark瀵︾従

    4锛屽垎椤炴寲鎺樼畻娉曞鐝惧強鎳夌敤

      1锛� 浣跨敤Spark Decision Tree瀵︾従閲戣瀺鍙嶆瑭愭ā鍨�

      2锛� 浣跨敤Spark Navie Bayes瀵︾従閲戣瀺鍙嶆瑭愭ā鍨�

      3锛� Spark鍒嗛绠楁硶甯哥敤鍫存櫙鍙婇枊鐧兼祦绋�"