Employee Spotlight

How to Push a Massive Architectural Change | anecdotes

Shahar Glazner
March 28, 2024
Learn with anecdotes how to introduce architectural changes

Here at anecdotes, over the course of the last two months, we worked on shifting from a simple, static, single-tenant identity architecture, to a more robust, dynamic, multi-tenant architecture. This change is now in production, allowing great flexibility to how anecdotes manages customers and their users.

This shift has super interesting technical aspects, but in this blog post, I would like to focus on the management aspects of the project. In a retrospective analysis, I’ve learned a lot about how to push such a massive architectural change, without hurting velocity and progress in our core product. These lessons are true independent of a specific project and can be used in every project you are working on.

Master your idea

Making an architectural shift is hard. Things will break, developers will argue, and the product team will prefer to invest the time in features. You, as the project leader, must be prepared and master the details.

Start with a simple PoC that shows the benefits your architecture brings. Discuss the change with your coworkers. Write a list of things that the new architecture will solve. Be the ambassador of your idea. Consider available alternatives.

Communicate the architectural changes from day one

In retrospect, perhaps the best thing I did was to gather a meeting, with product and R&D, where I presented the PoC to them. From that point, the momentum shifted, from a project fully pushed by me, to a project that was fully supported by product and R&D. This may sound trivial, but a big change can’t be done by any individual, no matter how strong he or she is.

Use your colleagues

Doing a PoC is one thing, but taking a significant architecture change to production is another. It's just too much work. You’ll need to cover deployment, testing, migrations, UI & UX, backend & frontend, and communicating the change to customers—and that's only a partial list.

So while I worked on deployment, testing, migrations, and backend, my super colleague Tal Borenstein took frontend (while working on his day-to-day work), and Barak Alkalay, our outstanding product manager helped with customer flows, communication, and project management. He was also the one who spoke with our great designer, Inuchka, who designed the sign-in page and email templates.

Apart from giving credit to this great bunch of colleagues, I want to emphasize how important it is to enlist help. No matter how strong an individual is, nothing beats the value of cooperation.

Don’t hurt velocity

At the end of the day, architectural change, as important and necessary as it is, is just one piece of the puzzle.

You’ll still need to deliver features, fix bugs, and ensure progress with other aspects of your product.

Therefore, invest time in planning how you’ll keep the change from hurting velocity. In my case, for example, I deployed a whole duplication of the platform. That way, I could break things and test changes without interrupting others. To eliminate diverging from the original platform, I merged the changes that other developers pushed and re-deployed. I also communicated to the teams where my changes were, so we could postpone specific changes or, if necessary, take them together.

Things will go wrong

No matter how much you invest in planning and mastering your idea, eventually, something won’t work. It's unavoidable. There are just too many variables. But what makes the difference is how you react to these situations. For this point to be better understood, I have a great real-life example from this specific project.

From the beginning, we knew that multifactor authentication for user and password authentication would be a problem for us. While in the old architecture this area was fully covered by Okta (our old IdP), Google Identity Provider (our new IdP) doesn’t cover this area very well. We tried to prepare for it as best as we could, including taking endless calls with Google technical support and investing a lot of time in finding a creative solution. But we still didn’t find a solution that satisfied us.

And then we asked ourselves: “Do we even need user and password authentication?”

We then did two things:

  1. Gathered statistics on password usage in our platform.
  2. Looked for companies that don’t use usernames and passwords in their sign-in mechanism.
Password usage in anecdotes platform

Snyk.io and Circle CI passwordless sign-in page (thanks to Guy Boyarski)

We shared our results with our CPO Roiamior, and quickly it became an easy decision. Suddenly, a problem that kept us busy for a long time became an opportunity. Deprecating user and password authentication removed a lot of complexity we even didn’t plan to remove and increased security in a way we didn’t think of. We now don't need to maintain password complexity enforcement, reset and change password flows, store passwords securely, and much more.

Invest time and effort in thinking about migrations

So your architecture change is now near complete. If it’s a big change, it will probably break things. But the last thing you want is to make the customers feel it. In our case, there were two main challenges:

  1. The platform sign-in page is now dynamic, and every customer has its own sign-in URL ({customer}.anecdotes.ai instead of platform.anecdotes.ai).
  2. Users with a user and password authentication should migrate to SSO or SAML authentication.

We invested a lot of time understanding how to move to the new architecture without hurting customers. Finally, we came up with a migration process that mitigates both concerns.

To mitigate the first concern, we created a “main” sign-in page, that redirects users to their tenant.

The main sign-in page, deployed to platform.anecdotes.ai

We mitigated the second concern by allowing users with passwords to log in using anecdotes’ old IdP as external IdP.

Users with passwords can still log in using “Sign in with Okta”

Summary

Architectural changes are an integral part of the lifecycle of a product; things that worked once can become obsolete, new features may not be able to be implemented using the current architecture, or things that worked with a small number of customers just don't scale.

It's a matter of time until you experience changes. When you need to push such changes, remember that the technical aspect is just one aspect. It may be the most important one, but the success or failure of this change will also depend on many other aspects, which must be taken into account from day one as well.

Doing so will increase your chances of success, make you a better project leader, and eventually, it will enable future changes.

Shahar Glazner
Passionate about technology, sports, economy, chess and system architecture in a cloud-native world. A long-distance runner and Lead Developer at anecdotes.