-大数据入门-2-Hadoop-mapred&yarn常用命令

maprde常用命令

[hadoop@hadoop001 bin]$ ./mapred --help
Usage: mapred [--config confdir] COMMAND
       where COMMAND is one of:
  pipes                run a Pipes job
  job                  manipulate MapReduce jobs    作业

Most commands print help when invoked w/o parameters.
[hadoop@hadoop001 bin]$ 

[hadoop@hadoop001 bin]$ ./mapred job
Usage: CLI <command> <args>
	[-submit <job-file>]
	[-status <job-id>]
	[-counter <job-id> <group-name> <counter-name>]
	[-kill <job-id>]    杀掉
	[-set-priority <job-id> <priority>]. Valid values for priorities are: VERY_HIGH HIGH NORMAL LOW VERY_LOW
	[-events <job-id> <from-event-#> <#-of-events>]
	[-history [all] <jobHistoryFile|jobId> [-outfile <file>] [-format <human|json>]]
	[-list [all]]        查看
	[-list-active-trackers]
	[-list-blacklisted-trackers]
	[-list-attempt-ids <job-id> <task-type> <task-state>]. Valid values for <task-type> are MAP REDUCE. Valid values for <task-state> are running, completed
	[-kill-task <task-attempt-id>]
	[-fail-task <task-attempt-id>]
	[-logs <job-id> <task-attempt-id>]
Generic options supported are
-conf <configuration file>     specify an application configuration file
-D <property=value>            use value for given property
-fs <local|namenode:port>      specify a namenode
-jt <local|resourcemanager:port>    specify a ResourceManager
-files <comma separated list of files>    specify comma separated files to be copied to the map reduce cluster
-libjars <comma separated list of jars>    specify comma separated jar files to include in the classpath.
-archives <comma separated list of archives>    specify comma separated archives to be unarchived on the compute machines.
The general command line syntax is
bin/hadoop command [genericOptions] [commandOptions]
[hadoop@hadoop001 bin]$ 


[hadoop@hadoop001 bin]$ ./mapred job -list     查看作业
19/07/19 17:55:22 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/07/19 17:55:22 INFO client.RMProxy: Connecting to ResourceManager at hadoop001/10.9.6.136:8032
Total jobs:2
                  JobId	     State	     StartTime	    UserName	       Queue	  Priority	 UsedContainers	 RsvdContainers	 UsedMem	 RsvdMem NeededMem	   AM info
 job_1563043900125_0002	      PREP	 1563529638402	      hadoop	 root.hadoop	    NORMAL	              1	              0	   2048M	      0M     2048M	http://hadoop001:8081/proxy/application_1563043900125_0002/
 job_1563043900125_0003	      PREP	 1563530062534	      hadoop	 root.hadoop	    NORMAL	              1	              0	   2048M	      0M     2048M	http://hadoop001:8081/proxy/application_1563043900125_0003/
[hadoop@hadoop001 bin]$ 


[hadoop@hadoop001 bin]$ ./mapred job -kill job_1563043900125_0003      杀掉某夯住的作业
19/07/20 12:46:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/07/20 12:46:28 INFO client.RMProxy: Connecting to ResourceManager at hadoop001/10.9.6.136:8032
19/07/20 12:46:30 INFO impl.YarnClientImpl: Killed application application_1563043900125_0003
Killed job job_1563043900125_0003
[hadoop@hadoop001 bin]$ ./mapred job -kill job_1563043900125_0004
19/07/20 12:46:36 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/07/20 12:46:36 INFO client.RMProxy: Connecting to ResourceManager at hadoop001/10.9.6.136:8032
19/07/20 12:46:38 INFO impl.YarnClientImpl: Killed application application_1563043900125_0004
Killed job job_1563043900125_0004

yarn

yarn jar 
yarn application –list
yarn application –kill -applicationId <application ID>
yarn logs -applicationId <application ID> 

实操

[hadoop@hadoop001 bin]$ yarn --help
Usage: yarn [--config confdir] COMMAND
where COMMAND is one of:
  resourcemanager -format-state-store   deletes the RMStateStore
  resourcemanager                       run the ResourceManager
  nodemanager                           run a nodemanager on each slave
  timelineserver                        run the timeline server
  rmadmin                               admin tools
  version                               print the version
  jar <jar>                             run a jar file          重点
  application                           prints application(s)
                                        report/kill application
  applicationattempt                    prints applicationattempt(s)
                                        report
  container                             prints container(s) report
  node                                  prints node report(s)
  queue                                 prints queue information
  logs                                  dump container logs
  classpath                             prints the class path needed to
                                        get the Hadoop jar and the
                                        required libraries
  daemonlog                             get/set the log level for each
                                        daemon
  top                                   run cluster usage tool
 or
  CLASSNAME                             run the class named CLASSNAME

Most commands print help when invoked w/o parameters.


[hadoop@hadoop001 bin]$ 
[hadoop@hadoop001 bin]$ yarn application
19/07/19 18:03:32 INFO client.RMProxy: Connecting to ResourceManager at hadoop001/10.9.6.136:8032
19/07/19 18:03:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Invalid Command Usage : 
usage: application
 -appStates <States>             Works with -list to filter applications
                                 based on input comma-separated list of
                                 application states. The valid application
                                 state can be one of the following:
                                 ALL,NEW,NEW_SAVING,SUBMITTED,ACCEPTED,RUN
                                 NING,FINISHED,FAILED,KILLED
 -appTypes <Types>               Works with -list to filter applications
                                 based on input comma-separated list of
                                 application types.
 -help                           Displays help for all commands.
 -kill <Application ID>          Kills the application.
 -list                           List applications. Supports optional use
                                 of -appTypes to filter applications based
                                 on application type, and -appStates to
                                 filter applications based on application
                                 state.
 -movetoqueue <Application ID>   Moves the application to a different
                                 queue.
 -queue <Queue Name>             Works with the movetoqueue command to
                                 specify which queue to move an
                                 application to.
 -status <Application ID>        Prints the status of the application.



[hadoop@hadoop001 bin]$ yarn application -list
19/07/19 18:03:47 INFO client.RMProxy: Connecting to ResourceManager at hadoop001/10.9.6.136:8032
19/07/19 18:03:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Total number of applications (application-types: [] and states: [SUBMITTED, ACCEPTED, RUNNING]):3
                Application-Id	    Application-Name	    Application-Type         User	     Queue	             State	       Final-State            Progress	                       Tracking-URL
application_1563043900125_0002	          word count	           MAPREDUCE       hadoop	root.hadoop	          ACCEPTED	         UNDEFINED                  0%	                                N/A
application_1563043900125_0003	          word count	           MAPREDUCE       hadoop	root.hadoop	          ACCEPTED	         UNDEFINED                  0%	                                N/A
application_1563043900125_0004	          word count	           MAPREDUCE       hadoop	root.hadoop	          ACCEPTED	         UNDEFINED                  0%	                                N/A
[hadoop@hadoop001 bin]$