Seonglae ChoYet Another Resource Negotiator
Hadoop 2.0 YARN splits up job scheduling and resource management
The two major functions of Hadoop Job Tracker into two separate daemons
- Global resource manager
- Per-application application master for job scheduling and monitoring
Hadoop 1.0 limitations
- Single Point of Failure – JobTracker and NameNode are the single point of failure. If it fails, the entire Job will fail.
- Limitation in running applications – Hadoop 1.0 is limited to run only a MapReduce application and supports only the batch mode of processing.
- Imbalance in Resource Utilization – Each TaskTracker is allocated predefined numbers of map and reduce slots, and hence resources may not be utilized completely when the map slots are performing tasks and might be full while the reduce slots are available to perform tasks and vice versa.
YARN NodeManager & Resource Manager
Commands
Scheduler
Apache Hadoop YARN
The fundamental idea of YARN is to split up the functionalities of resource management and job scheduling/monitoring into separate daemons. The idea is to have a global ResourceManager ( RM) and per-application ApplicationMaster ( AM). An application is either a single job or a DAG of jobs.
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html
