Queue management
One of the most powerful features of Rapidminer Server is that it can execute RapidMiner processes either immediately or as scheduled processes. Processes can be scheduled to be executed once at a specific point in time or periodically at specific intervals.
Rapidminer Server utilizes a queueing infrastructure where processes are submitted when they are ready for execution. The Job Agent picks up processes once they are made available in the queue and executes them.
Note: A queue can have one or several Job Agents connected to it. Each of these connected Job Agents can execute an arbitrary number of RapidMiner processes concurrently. However, the default value for parallel execution is set to three.
Queues
RapidMiner Server is shipped with a pre-configured queue named DEFAULT, but it is possible for the user to create additional queues.
Note: All Job Agents initialized during the installation of RapidMiner server will automatically connect to the DEFAULT queue.
To view this list of queues go to Queues located on the left side of the page:
This will redirect you to the Queues page where each configured queue is listed along with it's status information:
When expanding a queue the details of this queue become visible
The table below explains the status information associated with a Queue
| Field | Description |
|---|---|
| Running jobs | the number of processes that are currently being executed |
| Pending jobs | the number of processes awaiting execution |
| Permitted groups | a list of all the groups that are allowed to submit processes to the queue for execution |
| Owners | a list of all the owners of this queue |
| Job Agents | the number of Job Agents connected to the queue |
Adding a Queue
To configure a new queue, navigate to Queues. Click on Add Queue button located at the top right side of the queues page:
The Add Queue dialog will appear, where you can enter the name of the queue.
After you have entered the queue's name and set the fitting permissions, click the Create button to complete the process. The new queue should be immediately visible in the Queues page.
Removing a Queue
To remove an existing queue, navigate to Queues. This will bring you to the Queues page.
Click on the delete button next to the queue name you want to delete.
Note: The DEFAULT queue cannot be deleted.
Managing a Queue's permissions
A queue can be configured to allow process execution to specific groups. To change these permissions navigate to Queues. This will bring you to the Queues interface.
Click the Edit link next to the Purge and Delete property of the queue.
This will bring up the Edit Queue dialog:
To permit a group to execute processes from the queue, select the group name from the left hand side group list and click > (Add) arrow. This action will move the selected group(s) to the group list on the right hand side.
To restrict a group from executing processes from the queue, select the group from the group list on the right hand side and click < (Remove) arrow. This action will move the selected group(s) to the group list on the left hand side.
When done click Save.
Connecting a Job Agent to an existing Queue
To install and connect to a Job Agent an existing queue, start the Job Agent and set the environment variable JOBAGENT_QUEUE_JOB_REQUEST to the queue.
Purging a Queue's pending executions
The queue supports to purge all pending executions, for example when a large amount of jobs have been accidentally submitted by a schedule.
The purge operation is destructive. There’s no way to get already purged jobs back into queue besides re-submitting them.
When pending executions exist in the queue, click on the Purge button and confirm the action. Depending on the number of pending executions, the operation might take several minutes to hours. Once purging is complete, the purge progress indicator will disappear from the responding queue action and all pending executes have been marked as stopped.
