TaskQueueScheduler

Content Type: Module
Categories: Scheduling

Overview

 The TaskQueueScheduler module delivers flexible, dynamic scheduling of microflows on available taskqueues while avoiding hardcoded (in Mendix Studio) ScheduledEvents and TaskQueue assignments. Via same UI it also allows running and tracking microflows in batch (on Batch taskqueue)

Documentation

Depends on Mendix 9 or higher as that introduced TaskQueues

Removed dependency on MxModelReflection module now using metamodel to get microflows without input parameter (to run on schedule queues)

Removed dependency on Validate module while all validations are still found in Validations folder

This TaskQueueScheduler model is developed to replace old MicroflowScheduler and ProcessQueue modules from AppStore

in a Mendix 9 compliant way, because both became deprecated with Mendix 9.

 

Microflow scheduling functionality:

Module delivers flexible, dynamic scheduling of microflows so we can avoid most hardcoded (in Mendix Studio) ScheduledEvents.

The Schedule handler (scheduled every minute) now puts microflows on the specified Schedule TaskQueue when specified runtime

is reached.

 

ProcessQueue functionality:

When a microflow is started on a Taskqueue via Schedule the Schedule_Overview page shows Queued/Running, Finished and

Cancelled tabs to track running microflows. Also it shows a Configuration tab where you can dynamically set/update the

TaskQueue a microflow is to run on. It is not required to preconfigure a microflow on a TaskQueue upfront (like in ProcessQueue).

A new (yet unknown) microflow started via Schedule will be added and run on default Batch TaskQueue automatically.

Via Configuration tab you can also directly run (non-scheduled) microflows without input parameters.

 

On QueuedAction:

As we run microflows on TaskQueue directly there is no longer a need to create a QueuedAction to do that.

QueuedAction are only used to run a microflow on batch chunks.

In Running tab you see how many batch chuncks are still on the queue.

The microflow stays running till all its batch chunks (QueueActions) are done.

On Finished tab you'll find the timing for each batch chunk of the microflow.

 

Please check _SAMPLES folder on how to do this.

  • All Schedules: batches based on BatchSize and CountSchedule_Overview page Configuration tab has a button to run this sample.
  • All Schedules: batch per objectSchedule_Overview page Configuration tab has a button to run this sample.

 

QueuedActions are deleted on completion as we have no need for them after that.

 

To conclude the TaskQueueScheduler module now delivers flexible, dynamic scheduling of microflows on available taskqueues

while avoiding hardcoded (in Mendix Studio) ScheduledEvents and TaskQueue assignments.

 

How to configure:

Download TaskQueueScheduler module from Appstore into your application.

Add Schedule_Overview page to your app's navigation.

Pages have Atlas_Default layout you probably like to change that to your app's layout.

Open Schedule_Overview-Confuguration and hit to initialize TaskQueue objects.

You'll also need to go there when you add/change Mendix TaskQueues in your application

 

YML adjustments so not all ScheduledEvents from Studio keep on running after switching to taskqueue scheduler:

# * * * Scheduled Events * * *

# ALL, NONE or comma delimited .

# no spaces in between SCHEDULED_EVENTS:

TaskQueueScheduler.ScE_SchedulerQueues

 

Updating to newer version:

In most cases upgrade should work smoothly but be extra careful when REPLACING existing module as then you could LOOSE your carefully scheduled microflows.

Suggested update procedure:

1. Export your current Schedule to an Excel file

2. Now replace TaskQueueScheduler module with newer version

3. Import Excel file to restore all Schedules

 

How to use:

By default there are 3 Schedule, 1 Batch and 1 Batch3 TaskQueue defined, but you can add more when desired.

Additional TaskQueue(s) can be created in any module, just remember to also update theTaskQueue objects to the new situation

via Schedule-Configuration button.

For each TaskQueue object you can set AllowScheduling or not (default)

Via Schedule-Configuration you can run microflows without input parameters on the fly.

Via microflow you can start a schedule with QueuedAction object input which allows for running big updates in small batches

or just specifying an other input object.

Starting microflows on a queue via StudioPro will work as always, but then the microflow will not appear in Queued/Running tab

and you do not have the option to change the TaskQueue dynamically

Schedule ACT_CleanProcessedQueueTask regularly to prevent unwanted growth of System.ProcessedQueueTask

 

Concurrency on a single thread taskqueue:

With multiple instances indeed microflows are picked up by every instance so there can be parallel execution (on different instance)

even with single thread taskqueues.

To avoid parallel execution with multiple instances you should start your microflow via ACT_RunMicroflowOnQueue_ByName_Once

as then Schedule will prevent same microflow being put on taskqueue again

Alternative: Define a taskqueue as ClusterWide 1 thread.

Mendix arranges tasks put on such queue to be execute after each other (even in multicluster environments)

 

Contineous deploy issue:

ASU_ microflow calls another microflow on a taskqueue to avoid longrunning ASU processing.

During deploy the CANDIDATE CF app runs the ASU microflow, but taskqueues run on all instances

so Mendix can process the queued microflow on an old (but still running) instance

of the app.

Problem: the old instance can have an outdated version of the queued microflow still.

Solution: Call ACT_PauseSchedule as first action in the main ASU_ microflow

Like this the queued microflows are only initiated when old instances are stopped.

 

Creating scheduled microflows automatically:

Call SUB_ScheduleMicroflow to create a scheduled microflow.

When microflow is already scheduled it will do nothing.

This particularly convenient to automatically check if all necessary microflows in your project are scheduled

Releases

Version: 1.2.5
Framework Version: 9.18.4
Release Notes: Fixed SCE_SchedulerQueues becomming inactive on error Added GetObjectById Java action. Use CommunityCommons.getGUID to pass objects to the Scheduler batch process (QueuedAction). Use GetObjectById to get the involved objects when processing starts from queue. Like this you can avoid putting your microflows on a task queue directly (hardcoded in Studio). Using Scheduler batch processes means you can track your batches in Scheduler Running tab
Version: 1.2.4
Framework Version: 9.18.4
Release Notes: Removed use of MxModelReflection, so now you can use new microflows instantly (as no MxModelReflection sync is needed anymore) Also provided _SAMPLE AllSchedules batch Please use this when there is a need to split the processing into a number of batches
Version: 1.2.2
Framework Version: 9.12.4
Release Notes: Fix for shifting scheduled times (after crash). Added SUB_ScheduleMicroflow to create a scheduled microflow automatically to assure relevant microflows are scheduled Schedule.LastDuration now measured in milliseconds always
Version: 1.2.1
Framework Version: 9.12.4
Release Notes: Fix for cleaning running QueuedAction microflows
Version: 1.2.0
Framework Version: 9.12.4
Release Notes: Added option to briefly pause the schedule to solve ASU_ issue with our continuous deploy procedure