So what’s coming up with the new throttle core we’ve been working on during the last few weeks? Here’s a quick overview of the new design and the features that has been made available to the dependent applications.
Cluster transparency
Cluster synchronization of run-time throttle data has been made fully transparent to the caller applications. For callers, it is as simple as calling a single method by passing basic parameters such as callerID and callerConfig references that returns back the throttle decision. There is no cluster related information the caller has to deal with anymore.
ThrottleContext throttleContext = throttle.getThrottleContext(contextID);
boolean isAllowed = throttleContext.canAccess(callerID, callerConfig);
Persistence of run-time throttle data
A throttle policy with a long spanned throttle window can now benefit from the in-built persistence engine so that the last known caller contexts are restored during the re-initialization process. In other words, you may reboot the whole cluster but the throttle core is now capable of restoring the last persisted set of caller context so that the cluster can service the new requests from where it stopped.
Cleanup
Abandoned caller contexts are cleaned up at the end of a configurable tolerance window using callers last access times.
Context groupings made easy
Throttle core can now be extended for various context grouping needs. For example, say you want to group callers based on their subnet masks or perhaps the reverse looked up domain names. To achieve this, you can easily extend the CallerConfiguration and CallerContext classes and override the caller identification methods to satisfy the grouping criteria you wish to use, and that’s really it! There is absolutely no need to modify the throttle engine!
Minimum impact to the main thread
Throttle handler has been enhanced to ensure that persistence and cluster replication only makes the least required involvement with the main request thread.
... and the Design
We are currently doing further integration tests with the APIM 1.6 and expect to append this as a carbon core component with the next releases. We are also planning to further enhance the throttle mediator based on this implementation bringing in all key considerations that have been addressed by this new design.
Cluster transparency
Cluster synchronization of run-time throttle data has been made fully transparent to the caller applications. For callers, it is as simple as calling a single method by passing basic parameters such as callerID and callerConfig references that returns back the throttle decision. There is no cluster related information the caller has to deal with anymore.
ThrottleContext throttleContext = throttle.getThrottleContext(contextID);
boolean isAllowed = throttleContext.canAccess(callerID, callerConfig);
Persistence of run-time throttle data
A throttle policy with a long spanned throttle window can now benefit from the in-built persistence engine so that the last known caller contexts are restored during the re-initialization process. In other words, you may reboot the whole cluster but the throttle core is now capable of restoring the last persisted set of caller context so that the cluster can service the new requests from where it stopped.
Cleanup
Abandoned caller contexts are cleaned up at the end of a configurable tolerance window using callers last access times.
Context groupings made easy
Throttle core can now be extended for various context grouping needs. For example, say you want to group callers based on their subnet masks or perhaps the reverse looked up domain names. To achieve this, you can easily extend the CallerConfiguration and CallerContext classes and override the caller identification methods to satisfy the grouping criteria you wish to use, and that’s really it! There is absolutely no need to modify the throttle engine!
Minimum impact to the main thread
Throttle handler has been enhanced to ensure that persistence and cluster replication only makes the least required involvement with the main request thread.
... and the Design
- The new throttle core encapsulates two maps. The first map keeps the caller configuration objects that is usually created during the initialization phase. These configuration objects are created using a set of predefined rules such as the throttling policy definition (i.e. WSO2 API Manager does exactly that through the tiers.xml).
- The second map contains the caller context objects that gets dynamically created based on the caller attributes such as application or resource identifiers. This is what we use to track run-time throttle data for the engine to decide on either granting or denying access to a resource.
- The purpose of the Replicator Task is to absorb the cluster replication complexities from the throttle engine so that the engine will no longer need to undertake the task of notifying context changes to the neighboring cluster members. The replicator task is also responsible of synchronizing the global counters between cluster members within a configurable time interval.
- PersisterTask too runs on a separate thread for periodically persisting the active caller contexts on to a configurable datastore. We are planning to incorporate the cluster manager’s coordinator selection into this so that only the cluster elected coordinator will dynamically pick the task of persisting and reloading the caller context list.
We are currently doing further integration tests with the APIM 1.6 and expect to append this as a carbon core component with the next releases. We are also planning to further enhance the throttle mediator based on this implementation bringing in all key considerations that have been addressed by this new design.
