Here at anecdotes, over the course of the last two months, we worked on shifting from a simple, static, single-tenant identity architecture, to a more robust, dynamic, multi-tenant architecture. This change is now in production, allowing great flexibility to how anecdotes manages customers and their users.
This shift has super interesting technical aspects, but in this blog post, I would like to focus on the management aspects of the project. In a retrospective analysis, I’ve learned a lot about how to push such a massive architectural change, without hurting velocity and progress in our core product. These lessons are true independent of a specific project and can be used in every project you are working on.
Making an architectural shift is hard. Things will break, developers will argue, and the product team will prefer to invest the time in features. You, as the project leader, must be prepared and master the details.
Start with a simple PoC that shows the benefits your architecture brings. Discuss the change with your coworkers. Write a list of things that the new architecture will solve. Be the ambassador of your idea. Consider available alternatives.
In retrospect, perhaps the best thing I did was to gather a meeting, with product and R&D, where I presented the PoC to them. From that point, the momentum shifted, from a project fully pushed by me, to a project that was fully supported by product and R&D. This may sound trivial, but a big change can’t be done by any individual, no matter how strong he or she is.
Doing a PoC is one thing, but taking a significant architecture change to production is another. It's just too much work. You’ll need to cover deployment, testing, migrations, UI & UX, backend & frontend, and communicating the change to customers—and that's only a partial list.
So while I worked on deployment, testing, migrations, and backend, my super colleague Tal Borenstein took frontend (while working on his day-to-day work), and Barak Alkalay, our outstanding product manager helped with customer flows, communication, and project management. He was also the one who spoke with our great designer, Inuchka, who designed the sign-in page and email templates.
Apart from giving credit to this great bunch of colleagues, I want to emphasize how important it is to enlist help. No matter how strong an individual is, nothing beats the value of cooperation.
At the end of the day, architectural change, as important and necessary as it is, is just one piece of the puzzle.
You’ll still need to deliver features, fix bugs, and ensure progress with other aspects of your product.
Therefore, invest time in planning how you’ll keep the change from hurting velocity. In my case, for example, I deployed a whole duplication of the platform. That way, I could break things and test changes without interrupting others. To eliminate diverging from the original platform, I merged the changes that other developers pushed and re-deployed. I also communicated to the teams where my changes were, so we could postpone specific changes or, if necessary, take them together.
No matter how much you invest in planning and mastering your idea, eventually, something won’t work. It's unavoidable. There are just too many variables. But what makes the difference is how you react to these situations. For this point to be better understood, I have a great real-life example from this specific project.
From the beginning, we knew that multifactor authentication for user and password authentication would be a problem for us. While in the old architecture this area was fully covered by Okta (our old IdP), Google Identity Provider (our new IdP) doesn’t cover this area very well. We tried to prepare for it as best as we could, including taking endless calls with Google technical support and investing a lot of time in finding a creative solution. But we still didn’t find a solution that satisfied us.
And then we asked ourselves: “Do we even need user and password authentication?”
We then did two things:
We shared our results with our CPO Roiamior, and quickly it became an easy decision. Suddenly, a problem that kept us busy for a long time became an opportunity. Deprecating user and password authentication removed a lot of complexity we even didn’t plan to remove and increased security in a way we didn’t think of. We now don't need to maintain password complexity enforcement, reset and change password flows, store passwords securely, and much more.
So your architecture change is now near complete. If it’s a big change, it will probably break things. But the last thing you want is to make the customers feel it. In our case, there were two main challenges:
We invested a lot of time understanding how to move to the new architecture without hurting customers. Finally, we came up with a migration process that mitigates both concerns.
To mitigate the first concern, we created a “main” sign-in page, that redirects users to their tenant.
We mitigated the second concern by allowing users with passwords to log in using anecdotes’ old IdP as external IdP.
Architectural changes are an integral part of the lifecycle of a product; things that worked once can become obsolete, new features may not be able to be implemented using the current architecture, or things that worked with a small number of customers just don't scale.
It's a matter of time until you experience changes. When you need to push such changes, remember that the technical aspect is just one aspect. It may be the most important one, but the success or failure of this change will also depend on many other aspects, which must be taken into account from day one as well.
Doing so will increase your chances of success, make you a better project leader, and eventually, it will enable future changes.