Home > Mechanisms > Resource Manager

Resource Manager

Data that is held in storage can be processed in a variety of ways by a given Big Data solution, and all data processing requests require the allocation of processing resources. Users of Big Data solutions can make numerous data processing requests, each of which can have different processing workload requirements.

A resource manager acts as a scheduler that schedules and prioritizes processing requests according to individual processing workload requirements. The resource manager essentially acts as a resource arbitrator that manages and allocates available resources, as shown in the example in Figure 1.

Resource Manager: Figure 1 - A realtime processing job (1) and a batch processing job (2) are submitted for execution. The resource manager allocates resources according to the job workload requirements and then schedules the jobs on a realtime processing engine (3) and a batch processing engine (4) respectively.

Figure 1 - A realtime processing job (1) and a batch processing job (2) are submitted for execution. The resource manager allocates resources according to the job workload requirements and then schedules the jobs on a realtime processing engine (3) and a batch processing engine (4) respectively.