The Easy Way to Increase Your API Performance | anecdotes

Our platform, anecdotes.ai, is an Angular Single Page Application (SPA), which has several main components. When you log in into the platform, Angular triggers 3 API requests to the anecdotes backend, each representing a component. While waiting for a response, the UI shows a loading page so it’s clear why our goal, as backend developers, is to reduce this time as much as we can.

As you can see, we have:

• Framework endpoint: ~6 seconds
• Control endpoint: ~4 seconds
• Service endpoint: ~3 seconds

Summing  it up, I’ll use the words of Chrenobyl’s Dyatlov:

‍

‍

One evening, Tal Borenstein and I started to bounce around some ideas on how to reduce the loading time.

Here I should describe the stack that our backend is built on:

• Programming language — Python
• Database — Snowflake
• Web stack — Gunicorn + Flask
• ORM — SQLAlchemy

Each of these parts could potentially cause performance problems.

To take the discussion to a more professional level, we decided to use a profiler so we would have some idea of what was going on. I’ll focus on one specific endpoint (/control) but the general idea is the same for the other two.

Let’s review the first profiler iteration:

At first sight, it didn’t tell us a lot, but there was one thing that caught our attention:

The function “get_specific_global_control” was invoked 3586 times, and it triggered another function (“get”) 1084112 times (!), taking 40% of the processing time!

We jumped to the code and reviewed the implementation:

‍

We could immediately see that although we used a cache so that we wouldn’t query the database for each control, each invocation of “get_specific_global_control” took o(N) (where N is the number of controls in our system) to find a specific control!

This could have been immediately reduced just by keeping the cache as a dictionary and not as a list.

It would make the get_specific_global_control run in O(1) instead of O(N).

Now that we made this change, let’s review our second iteration of profiling and see if it helped:

You can see that now, control_facade:control_related_controls takes only 4%(!) of the processing time. It had previously taken 58% of the processing time.

So what’s next?

The next thing that caught our eye was the “get_services_that_automate” function, which now, after the second iteration, took 40% of the time.

This function should have returned a list of services that automated a specific control. This is a new property that was added recently, and we didn’t expect it to cause such significant performance degradation. The root cause for that “get_services_that_automates” function taking so long is that it queried the database, instead of using already-queried data. We bypassed this by adding this property as a column to the already-queried object (control), so we now get this property “for free”.

So here’s our third iteration of profiling. After holding the controls cache as a dictionary instead of a list AND refactoring “get_services_that_automates” to use an already-queried object instead of querying the database, we managed to reduce around 50%~ of the time for our APIs.

Let’s see now how the platform behaves and compare the results:

Summary

Finally, we can review the “performance blitz” results in a simple table, showing a performance enhancement of 67%.

So there you have it; how we improved performance by A LOT in one day. It involved a bit of playing around and experimentation but that’s what we love doing here at anecdotes—seeing how already-decent processes can be made even better and optimizing what’s already good.

‍