How you can manage throttle out errors in WSO2 API Manager

You might face the above-mentioned error if you have an application in Dialog Ideabiz platform.Let’s start from the beginning.

Once you create an application on Ideabiz, you need to subscribe to the APIs you require. We use WSO2 API manager to manage the APIs. When admin approves your API subscriptions, he/she will assign a tier value for each API(5tps, 10 tps etc.). Also, they will assign a tier value for the application as well.

Let’s assume you wanted to create an application by using SMS and Payment API. The team has assigned 5tps for each API and 10tps for your application. You use SMS API only to send SMS to the users and payment API to charge the users on daily basis.

When considering each API, (as per above example) you should manage  5 transactions per second in the SMS API. Which means you can have 5 API calls per second. Let’s assume before you send the SMS, you want to charge Rs 1 from the user. Therefore you need to call payment API as well. Since the Payment API got allocated 5tps, you can manage 5 API calls per second for Payment API as well. Calling those APIs only 5 times per second would mean, you are hitting the endpoint successfully. Simply you can have 5 hits per second for each API and no more. If there were more than 5 requests within that second those additional requests will get throttled out (will  not hit the end point)

Now we have to consider, application tier as well. To have a smooth communication between each component of your application, you need to work within the application tier constraint as well. Since you have been allocated 10tps as the application tier, there is no point in allocating more than 10tps for APIs. API tier will always be set less than or equal to App tier.

Which means, altogether you can have 10 API calls per second. In above scenario, to complete this in an effective manner, you can initiate 5 Transactions from each API per second. So all together there are 10 transactions per second.

Now let’s move on to the main topic - “Throttle out”.

If you got above error in your application, one of the root causes could be, that you didn’t consider the above application/API tier allocated.

There are two types of throttle out errors.(https://docs.wso2.com/display/AM1100/Error+Handling)

900803: Application level throttle out error - Message is throttled out because application level is exceeded

900800: Message throttled out - The maximum number of requests that can be made to the API within a designated time period is reached and the API is throttled for the user.

If we consider 900800 error code, below is the error response we are getting.

<amt:fault xmlns:amt="http://wso2.org/apimanager/throttling">  
 <amt:code>900800</amt:code>  

 <amt:description>You have exceeded your quota</amt:description>  
<amt:message>MessageThrottled Out</amt:message>  
 </amt:fault>  

 <amt:code>900800</amt:code>  
<amt:fault xmlns:amt="http://wso2.org/apimanager/throttling">  
 <amt:message>Message Throttled Out</amt:message>  

 <amt:description>You have exceeded your quota</amt:description></amt:fault>  

Here’s the challenge. How we can avoid these throttle out errors.

Simply you can control the limit of the API calls per second inside the code. In different languages, it can be explained in different ways. But ultimately, you need to have a break in between these API calls. Below I have mentioned few of those methods in different programming languages.

In PHP:

<?php  
 $var=50;  
 $tps=5;  
 // current time  
 echo date('h:i:s') . "\n";  
 $time_start = microtime(true);  
      for($x = 0; $x <= $var; $x++){  
      $time_end = microtime(true);  
      //Get the duration  
      $duration=$time_end-$time_start;  
      //check whether duration time is greater than tps or not  
           if($duration>$tps){       
           echo "sleep";  
           // sleep for 1 second  
           sleep(1);  
           // wake up !  
           }  
      }  
 echo date('h:i:s') . "\n";  
 ?>  

In JAVA:

package com.tutorialspoint;  
  import java.lang.*;  
  public class ThreadDemo implements Runnable {  
   Thread t;  
   public void run() {  
    for (int i = 10; i < 13; i++) {  
      System.out.println(Thread.currentThread().getName() + " " + i);  
      try {  
       // thread to sleep for 1000 milliseconds  
       Thread.sleep(1000);  
      } catch (Exception e) {  
       System.out.println(e);  
      }  
    }  
   }  
   public static void main(String[] args) throws Exception {  
    Thread t = new Thread(new ThreadDemo());  
    // this will call run() function  
    t.start();  
     Thread t2 = new Thread(new ThreadDemo());  
    // this will call run() function  
    t2.start();  
   }  
 }  

Result:

Thread-0  10

Thread-1  10

Thread-0  11

Thread-1  11

Thread-0  12

Thread-1  12

Source: https://www.tutorialspoint.com/java/lang/thread_sleep_millis.htm

Above I have attempted to give a simple solution to one of the most critical problems that developers would face when creating an application on ideabiz platform.

All constructive comments are warmly welcome! :)