{"id":1441,"date":"2024-05-08T11:04:54","date_gmt":"2024-05-08T03:04:54","guid":{"rendered":"http:\/\/oneai.eu.org\/?p=1441"},"modified":"2024-05-08T11:04:54","modified_gmt":"2024-05-08T03:04:54","slug":"hdfs-%e5%b8%b8%e7%94%a8%e5%91%bd%e4%bb%a4","status":"publish","type":"post","link":"https:\/\/oneai.eu.org\/?p=1441","title":{"rendered":"HDFS \u5e38\u7528\u547d\u4ee4"},"content":{"rendered":"<pre><code class=\"language-shell\">Hadoop\u4e2dHDFS\u5e38\u7528\u547d\u4ee4 \n\n    hadoop fs -mkdir \/tmp\/input              \u5728HDFS\u4e0a\u65b0\u5efa\u6587\u4ef6\u5939\n    hadoop fs -put input1.txt \/tmp\/input     \u628a\u672c\u5730\u6587\u4ef6input1.txt\u4f20\u5230HDFS\u7684\/tmp\/input\u76ee\u5f55\u4e0b\n    hadoop fs -put input1.txt                   \u628ainput1.txt\u653e\u5728\u9ed8\u8ba4\u7684\u76ee\u5f55\u4e0b\u3002 \/user\/\u7528\u6237\u540d\/\n    hadoop fs -get   \/tmp\/input\/input1.txt  input1.txt  \u628aHDFS\u6587\u4ef6\u62c9\u5230\u672c\u5730\n    hadoop fs -ls \/tmp\/output                  \u5217\u51faHDFS\u7684\u67d0\u76ee\u5f55\u53ca\u6587\u4ef6\n    hadoop fs -ls hdfs:\/\/\u4e3b\u673a\u540d:9000\/           \u5217\u51fa\u4e0d\u540c\u670d\u52a1\u4e0a\u7684\u6587\u4ef6\u6216\u76ee\u5f55\n    hadoop fs -lsr hdfs:\/\/192.168.134.128:9000\/ \u5217\u51fa192.168.134.128\u670d\u52a1\u4e0a\u7684\u6587\u4ef6\u6216\u76ee\u5f55\n    hadoop fs -lsr \/tmp\/output                  \u9012\u5f52\u5217\u51fa\n    hadoop fs -cat \/tmp\/ouput\/output1.txt         \u67e5\u770bHDFS\u4e0a\u7684\u6587\u4ef6\n    hadoop fs -rmr \/home\/less\/hadoop\/tmp\/output          \u5220\u9664HDFS\u4e0a\u7684\u76ee\u5f55\u53ca\u6587\u4ef6\n    hadoop fs -rm \/home\/less\/hadoop\/tmp\/output          \u5220\u9664HDFS\u4e0a\u7684\u6587\u4ef6\n\n    hadoop dfsadmin -report \u67e5\u770bHDFS\u72b6\u6001\uff0c\u6bd4\u5982\u6709\u54ea\u4e9bdatanode\uff0c\u6bcf\u4e2adatanode\u7684\u60c5\u51b5\n    hadoop dfsadmin -safemode leave  \u79bb\u5f00\u5b89\u5168\u6a21\u5f0f\n    hadoop dfsadmin -safemode enter  \u8fdb\u5165\u5b89\u5168\u6a21\u5f0f\n\nhadoop fs                                              \u67e5\u770bHadoop HDFS\u652f\u6301\u7684\u6240\u6709\u547d\u4ee4 \nhadoop fs \u2013chgrp [-R] \/user\/sunlightcs       \u4fee\u6539HDFS\u7cfb\u7edf\u4e2d\/user\/sunlightcs\u76ee\u5f55\u6240\u5c5e\u7fa4\u7ec4\uff0c\u9009\u9879-R\u9012\u5f52\u6267\u884c\uff0c\u8ddflinux\u547d\u4ee4\u4e00\u6837 \n\nhadoop fs \u2013chown [-R] \/user\/sunlightcs       \u4fee\u6539HDFS\u7cfb\u7edf\u4e2d\/user\/sunlightcs\u76ee\u5f55\u62e5\u6709\u8005\uff0c\u9009\u9879-R\u9012\u5f52\u6267\u884c \nhadoop fs \u2013chmod [-R] MODE \/user\/sunlightcs  \u4fee\u6539HDFS\u7cfb\u7edf\u4e2d\/user\/sunlightcs\u76ee\u5f55\u6743\u9650\uff0cMODE\u53ef\u4ee5\u4e3a\u76f8\u5e94\u6743\u9650\u76843\u4f4d\u6570\u6216+\/-{rwx}\uff0c\u9009\u9879-R\u9012\u5f52\u6267\u884c \n\nhadoop fs \u2013count [-q] PATH \n\u67e5\u770bPATH\u76ee\u5f55\u4e0b\uff0c\u5b50\u76ee\u5f55\u6570\u3001\u6587\u4ef6\u6570\u3001\u6587\u4ef6\u5927\u5c0f\u3001\u6587\u4ef6\u540d\/\u76ee\u5f55\u540d\n\nhadoop fs \u2013cp SRC [SRC \u2026] DST      \n\u5c06\u6587\u4ef6\u4eceSRC\u590d\u5236\u5230DST\uff0c\u5982\u679c\u6307\u5b9a\u4e86\u591a\u4e2aSRC\uff0c\u5219DST\u5fc5\u987b\u4e3a\u4e00\u4e2a\u76ee\u5f55 \n\nhadoop fs \u2013du PATH \n\u663e\u793a\u8be5\u76ee\u5f55\u4e2d\u6bcf\u4e2a\u6587\u4ef6\u6216\u76ee\u5f55\u7684\u5927\u5c0f \n\nhadoop fs \u2013dus PATH \n\u7c7b\u4f3c\u4e8edu\uff0cPATH\u4e3a\u76ee\u5f55\u65f6\uff0c\u4f1a\u663e\u793a\u8be5\u76ee\u5f55\u7684\u603b\u5927\u5c0f\nhadoop fs \u2013expunge \n\u6e05\u7a7a\u56de\u6536\u7ad9\uff0c\u6587\u4ef6\u88ab\u5220\u9664\u65f6\uff0c\u5b83\u9996\u5148\u4f1a\u79fb\u5230\u4e34\u65f6\u76ee\u5f55.Trash\/\u4e2d\uff0c\u5f53\u8d85\u8fc7\u5ef6\u8fdf\u65f6\u95f4\u4e4b\u540e\uff0c\u6587\u4ef6\u624d\u4f1a\u88ab\u6c38\u4e45\u5220\u9664 \n\nhadoop fs \u2013getmerge SRC [SRC \u2026] LOCALDST [addnl]     \n\u83b7\u53d6\u7531SRC\u6307\u5b9a\u7684\u6240\u6709\u6587\u4ef6\uff0c\u5c06\u5b83\u4eec\u5408\u5e76\u4e3a\u5355\u4e2a\u6587\u4ef6\uff0c\u5e76\u5199\u5165\u672c\u5730\u6587\u4ef6\u7cfb\u7edf\u4e2d\u7684LOCALDST\uff0c\u9009\u9879addnl\u5c06\u5728\u6bcf\u4e2a\u6587\u4ef6\u7684\u672b\u5c3e\u5904\u52a0\u4e0a\u4e00\u4e2a\u6362\u884c\u7b26 \n\nhadoop fs \u2013touchz PATH  \n\u521b\u5efa\u957f\u5ea6\u4e3a0\u7684\u7a7a\u6587\u4ef6 \n\nhadoop fs \u2013test \u2013[ezd] PATH    \n\u5bf9PATH\u8fdb\u884c\u5982\u4e0b\u7c7b\u578b\u7684\u68c0\u67e5\uff1a \n-e PATH\u662f\u5426\u5b58\u5728\uff0c\u5982\u679cPATH\u5b58\u5728\uff0c\u8fd4\u56de0\uff0c\u5426\u5219\u8fd4\u56de1 \n-z \u6587\u4ef6\u662f\u5426\u4e3a\u7a7a\uff0c\u5982\u679c\u957f\u5ea6\u4e3a0\uff0c\u8fd4\u56de0\uff0c\u5426\u5219\u8fd4\u56de1 \n-d \u662f\u5426\u4e3a\u76ee\u5f55\uff0c\u5982\u679cPATH\u4e3a\u76ee\u5f55\uff0c\u8fd4\u56de0\uff0c\u5426\u5219\u8fd4\u56de1 \nhadoop fs \u2013text PATH \n\u663e\u793a\u6587\u4ef6\u7684\u5185\u5bb9\uff0c\u5f53\u6587\u4ef6\u4e3a\u6587\u672c\u6587\u4ef6\u65f6\uff0c\u7b49\u540c\u4e8ecat\uff0c\u6587\u4ef6\u4e3a\u538b\u7f29\u683c\u5f0f\uff08gzip\u4ee5\u53cahadoop\u7684\u4e8c\u8fdb\u5236\u5e8f\u5217\u6587\u4ef6\u683c\u5f0f\uff09\u65f6\uff0c\u4f1a\u5148\u89e3\u538b\u7f29 \n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Hadoop\u4e2dHDFS\u5e38\u7528\u547d\u4ee4 hadoop fs -mkdir \/tmp\/input \u5728HDFS\u4e0a\u65b0\u5efa\u6587\u4ef6\u5939 hadoop fs -put input1.txt \/tmp\/input \u628a\u672c\u5730\u6587\u4ef6input1.txt\u4f20\u5230HDFS\u7684\/tmp\/input\u76ee\u5f55\u4e0b hadoop fs -put input1.txt \u628ainput1.txt\u653e\u5728\u9ed8\u8ba4\u7684\u76ee\u5f55\u4e0b\u3002 \/user\/\u7528\u6237\u540d\/ hadoop fs -get \/tmp\/input\/input1.txt input1.txt \u628aHDFS\u6587\u4ef6\u62c9\u5230\u672c\u5730 hadoop fs -ls \/tmp\/output \u5217\u51faHDFS\u7684\u67d0\u76ee\u5f55\u53ca\u6587\u4ef6 hadoop fs -ls hdfs:\/\/\u4e3b\u673a\u540d:9000\/ \u5217\u51fa\u4e0d\u540c\u670d\u52a1\u4e0a\u7684\u6587\u4ef6\u6216\u76ee\u5f55 hadoop fs -lsr hdfs:\/\/192.168.134.128:9000\/ \u5217\u51fa192.168.134.128\u670d\u52a1\u4e0a\u7684\u6587\u4ef6\u6216\u76ee\u5f55 hadoop fs -lsr \/tmp\/output \u9012\u5f52\u5217\u51fa hadoop fs -cat \/tmp\/ouput\/output1.txt \u67e5\u770bHDFS\u4e0a\u7684\u6587\u4ef6 hadoop fs -rmr \/home\/less\/hadoop\/tmp\/output \u5220\u9664HDFS\u4e0a\u7684\u76ee\u5f55\u53ca\u6587\u4ef6 hadoop fs -rm \/home\/less\/hadoop\/tmp\/output \u5220\u9664HDFS\u4e0a\u7684\u6587\u4ef6 hadoop dfsadmin -report \u67e5\u770bHDFS\u72b6\u6001\uff0c\u6bd4\u5982\u6709\u54ea\u4e9bdatanode\uff0c\u6bcf\u4e2adatanode\u7684\u60c5\u51b5 hadoop dfsadmin -safemode leave \u79bb\u5f00\u5b89\u5168\u6a21\u5f0f hadoop dfsadmin -safemode enter \u8fdb\u5165\u5b89\u5168\u6a21\u5f0f hadoop fs \u67e5\u770bHadoop HDFS\u652f\u6301\u7684\u6240\u6709\u547d\u4ee4 hadoop fs \u2013chgrp [-R] \/user\/sunlightcs \u4fee\u6539HDFS\u7cfb\u7edf\u4e2d\/user\/sunlightcs\u76ee\u5f55\u6240\u5c5e\u7fa4\u7ec4\uff0c\u9009\u9879-R\u9012\u5f52\u6267\u884c\uff0c\u8ddflinux\u547d\u4ee4\u4e00\u6837 hadoop fs \u2013chown [-R] \/user\/sunlightcs \u4fee\u6539HDFS\u7cfb\u7edf\u4e2d\/user\/sunlightcs\u76ee\u5f55\u62e5\u6709\u8005\uff0c\u9009\u9879-R\u9012\u5f52\u6267\u884c hadoop fs \u2013chmod [-R] MODE \/user\/sunlightcs \u4fee\u6539HDFS\u7cfb\u7edf\u4e2d\/user\/sunlightcs\u76ee\u5f55\u6743\u9650\uff0cMODE\u53ef\u4ee5\u4e3a\u76f8\u5e94\u6743\u9650\u76843\u4f4d\u6570\u6216+\/-{rwx}\uff0c\u9009\u9879-R\u9012\u5f52\u6267\u884c hadoop fs \u2013count [-q] PATH \u67e5\u770bPATH\u76ee\u5f55\u4e0b\uff0c\u5b50\u76ee\u5f55\u6570\u3001\u6587\u4ef6\u6570\u3001\u6587\u4ef6\u5927\u5c0f\u3001\u6587\u4ef6\u540d\/\u76ee\u5f55\u540d hadoop fs \u2013cp SRC [SRC \u2026] DST \u5c06\u6587\u4ef6\u4eceSRC\u590d\u5236\u5230DST\uff0c\u5982\u679c\u6307\u5b9a\u4e86\u591a\u4e2aSRC\uff0c\u5219DST\u5fc5\u987b\u4e3a\u4e00\u4e2a\u76ee\u5f55 hadoop fs \u2013du PATH \u663e\u793a\u8be5\u76ee\u5f55\u4e2d\u6bcf\u4e2a\u6587\u4ef6\u6216\u76ee\u5f55\u7684\u5927\u5c0f hadoop fs \u2013dus PATH \u7c7b\u4f3c\u4e8edu\uff0cPATH\u4e3a\u76ee\u5f55\u65f6\uff0c\u4f1a\u663e\u793a\u8be5\u76ee\u5f55\u7684\u603b\u5927\u5c0f hadoop fs \u2013expunge \u6e05\u7a7a\u56de\u6536\u7ad9\uff0c\u6587\u4ef6\u88ab\u5220\u9664\u65f6\uff0c\u5b83\u9996\u5148\u4f1a\u79fb\u5230\u4e34\u65f6\u76ee\u5f55.Trash\/\u4e2d\uff0c\u5f53\u8d85\u8fc7\u5ef6\u8fdf\u65f6\u95f4\u4e4b\u540e\uff0c\u6587\u4ef6\u624d\u4f1a\u88ab\u6c38\u4e45\u5220\u9664 hadoop fs \u2013getmerge SRC [SRC \u2026] LOCALDST [addnl] \u83b7\u53d6\u7531SRC\u6307\u5b9a\u7684\u6240\u6709\u6587\u4ef6\uff0c\u5c06\u5b83\u4eec\u5408\u5e76\u4e3a\u5355\u4e2a\u6587\u4ef6\uff0c\u5e76\u5199\u5165\u672c\u5730\u6587\u4ef6\u7cfb\u7edf\u4e2d\u7684LOCALDST\uff0c\u9009\u9879addnl\u5c06\u5728\u6bcf\u4e2a\u6587\u4ef6\u7684\u672b\u5c3e\u5904\u52a0\u4e0a\u4e00\u4e2a\u6362\u884c\u7b26 hadoop fs \u2013touchz PATH \u521b\u5efa\u957f\u5ea6\u4e3a0\u7684\u7a7a\u6587\u4ef6 hadoop fs \u2013test \u2013[ezd] PATH \u5bf9PATH\u8fdb\u884c\u5982\u4e0b\u7c7b\u578b\u7684\u68c0\u67e5\uff1a -e PATH\u662f\u5426\u5b58\u5728\uff0c\u5982\u679cPATH\u5b58\u5728\uff0c\u8fd4\u56de0\uff0c\u5426\u5219\u8fd4\u56de1 -z \u6587\u4ef6\u662f\u5426\u4e3a\u7a7a\uff0c\u5982\u679c\u957f\u5ea6\u4e3a0\uff0c\u8fd4\u56de0\uff0c\u5426\u5219\u8fd4\u56de1 -d \u662f\u5426\u4e3a\u76ee\u5f55\uff0c\u5982\u679cPATH\u4e3a\u76ee\u5f55\uff0c\u8fd4\u56de0\uff0c\u5426\u5219\u8fd4\u56de1 hadoop fs \u2013text PATH \u663e\u793a\u6587\u4ef6\u7684\u5185\u5bb9\uff0c\u5f53\u6587\u4ef6\u4e3a\u6587\u672c\u6587\u4ef6\u65f6\uff0c\u7b49\u540c\u4e8ecat\uff0c\u6587\u4ef6\u4e3a\u538b\u7f29\u683c\u5f0f\uff08gzip\u4ee5\u53cahadoop\u7684\u4e8c\u8fdb\u5236\u5e8f\u5217\u6587\u4ef6\u683c\u5f0f\uff09\u65f6\uff0c\u4f1a\u5148\u89e3\u538b\u7f29<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_eb_attr":"","footnotes":""},"categories":[85],"tags":[],"class_list":["post-1441","post","type-post","status-publish","format-standard","hentry","category-hadoop"],"_links":{"self":[{"href":"https:\/\/oneai.eu.org\/index.php?rest_route=\/wp\/v2\/posts\/1441","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/oneai.eu.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/oneai.eu.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/oneai.eu.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/oneai.eu.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1441"}],"version-history":[{"count":1,"href":"https:\/\/oneai.eu.org\/index.php?rest_route=\/wp\/v2\/posts\/1441\/revisions"}],"predecessor-version":[{"id":1442,"href":"https:\/\/oneai.eu.org\/index.php?rest_route=\/wp\/v2\/posts\/1441\/revisions\/1442"}],"wp:attachment":[{"href":"https:\/\/oneai.eu.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1441"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/oneai.eu.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1441"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/oneai.eu.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1441"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}